An extra hammer in the capital means an earlier worker, which means more accumulated turns of working improved tiles for the entire game. It also means extra growth, as you are stagnant fewer turns in the beginning while producing said worker. The turn "lost" by moving is regained by the time you have produced this worker.
It also provides for a more production oriented capital.
I'll happily (hopefully?) collect the ivory with a future city.
Other things being equal, I would totally agree. Getting an early worker and extra early production is a definite plus not to be ignored. But my point is that things are not equal; the terrain to the E is not as good as the starting location. You trade 8 forests for 5 (maybe 6 if there is one in the fog to the SE, but that looks like desert to me). That translates into 90 extra hammers which would pretty much offset the advantage of getting the early settler in the short term.
Mid-term you are sacrificing the Ivory for Silk, which is more than just an early vs. late resource tradeoff. The ivory also adds production: with a camp that plains ivory tile will yield 1F3H1C which is like a grassland hill with a mine. That offsets one of the 2 extra hills the eastern position will add plus a little extra commerce and happiness (neither of which should be sneezed at on Deity level).
And don't be too sure that you can get that Ivory in a later city. Bear in mind that on Deity level, the AI starts with 2 settlers which means they have 12 cities to your 1. On a pangaea map, that can easily translate into a de facto One City Challenge for the early game. It happened to me in about 4 out of the 6 practice games I tried.
Long term, you are trading workable tiles for 1 desert hill and 2 extra unimprovable coastal tiles. Plus you have that hidden tile 1S2E that I already mentioned which looks to me like desert or at least plains.
Mind you, I don't think the position 2E of the start would be a disaster. If we started with a Scout instead of a Warrior, I would move him to the desert hill to see if there were anything there to improve the prospects. As it is, however, I don't see any great advantage to wasting the extra turn. I think I would rather settle in place and try to redeem the desert hill with my first settler, if nothing better presents itself.