Olive trees don't live for thousands of years by accident. They're resilient. The most important rule: make sure to plant your tree in soil that drains well.
Below is more detail for olive tree care, courtesy of OliveTreeGrowers.com:
PLANTING OUTSIDE OR INSIDE IN A POT (see below)
In green, you'll see the ideal outdoor (in ground) planting zones for Olive Trees:
Olive trees require a well-drained soil and a sunny position. Avoid sites where water stands during rainy periods or where ground water seeps into a hole two feet deep. Do not, however, confuse the olive for a desert plant. It needs regular watering to thrive. Insufficient water will cause your tree to suffer, and even die if left too dry for too long. Choose a site that receives at least six hours of direct sun per day. Full sun is ideal.
Plant your tree at the depth it has been growing in the pot. Do not amend the soil with organic material, moisture-retaining polymers, fertilizer or anything else. Simply plant in the native soil (provided it is well-drained) and backfill with the same.
If your tree requires staking, it will already have a stake in the pot. A very young tree may require a heavier stake as it grows. Once the trunk caliper reaches 1.25 inches or more in diameter (or perhaps less for shrub-form or short trees), it will no longer require a stake. Until then, use a stake large enough to hold the trunk upright. Put the new stake in the same hole the previous stake occupied and tie the tree to the stake with arborists' tape such as comes with your staked tree. Do not use wires, water hoses, cloth, cables, guying systems or other means of securing your tree. A good stout stake and the proper tape is all you need.
Olive trees do not need special olive tree fertilizer but results will be exponentially more satisfactory with a good nutrition regimen. If planting is done after mid-August but before March 1, do not fertilize at the time of planting; wait until spring. Otherwise, fertilize after planting and regularly throughout the growing season. How regularly will depend upon the type of fertilizer used. We prefer a premium slow-release fertilizer with essential minor elements (zinc, boron, calcium, etc.). Use a fertilizer that has a nitrogen analysis of at least ten percent.
A fast-release farm-type fertilizer such as 10-10-10 or 13-13-13 may be used according to label directions. Many such fertilizers contain some minor elements and are widely available. Be sure to water very well after application. These fertilizers are generally not for use on plants in pots.
Liquid fertilizers may be used (again, a formulation with minor elements is best) but it should be remembered that liquid feeding is a fleeting thing and must be repeated often.
Organically-derived fertilizers are available and a good thing, though often more expensive and rarely contain the percentage of nitrogen preferred by olive trees. Top dressing with organic material such as composted manure or kitchen compost can be done but the grower should consult the current literature. It can be difficult to achieve a good balance of nutritional elements by this method. Always avoid placing compost or any fertilizer next to the trunk of the tree.
Avoid heavy applications of fast release fertilizer that could damage plants and leach or run-off into groundwater. Always read and follow label instructions. Do not fertilize after mid-August or before mid- March unless you live in a very warm climate.
If your tree is planted in a lawn area, take care that lawn maintenance practices do not harm the tree. Do not allow "Weed and Feed" products to be used within 30 feet of your tree. These products are designed to feed lawn grasses and destroy other plants. Remember that the roots of any tree extend far beyond the drip-line of the branches. Also, do not allow weed-eater operators near your tree. Weed-eater operators kill thousands of trees every year by "girding," or removing the bark from the bases of trees.
Use only pine straw for mulch and keep it back several inches from the trunk; do not allow a build-up of decomposed mulch around the base of the tree trunk. If pine straw is not available, you can mulch with pine bark or gravel.
Once established, olive trees are among the most drought-resistant trees in the world, but porous soils such as Florida sand are very inefficient at retaining moisture; olive trees in sandy soils must be watered often. You will have to water sufficiently to get your tree established and thereafter as necessary during dry periods. Low volume spray irrigation can be used effectively, but drip irrigation is of little or no use in sandy soils.
Olive trees do not require pruning in order to produce fruit, at least not until they are around 50 years old. It is okay to prune olive trees to achieve a desired shape but remember that they fruit on branches that grew during the previous spring and summer; cutting off a lot of such growth will preclude or greatly reduce fruiting potential for the next season.
It may be helpful to prune the top, upwardly growing, branches back by a few inches to encourage lateral growth, thus facilitating the picking of fruit. It is also a good idea to cut out small interior branches that will ultimately clutter the tree's appearance and provide protection for any pests or diseases that may be lurking around your site.
PLANTING IN A POT
Olive trees can grow in a pot indoors or outdoors depending on the weather. Olive trees like sunlight, so full to partial sun is perfect. The pot needs to have ample drainage and ensure the holes are not blocked by adding gravel, rocks, Styrofoam or even a crushed can to the bottom of the pot and then cover with soil. Potted trees need to be watered more often than those in the ground. Allow the soil to dry somewhat before watering again and then soak the soil.
You can fertilize your olive tree twice a year with slow release fertilizer. Once in the spring and once in the mid-summer. Prune your potted tree by pinching off the tips and cutting off low branches. Take your olive tree outside when weather permits to get some fresh air.