It is important to understand the difference between (A) sending a worker (or work boat) to "improve" a tile (e.g., build a farm, mine, lumbermill, oil well, etc. or change the terrain by, e.g., chopping a forest or draining a marsh) and (B) assigning the population of your city (citizens) to "work" those tiles to obtain their yield (food, hammers, gold, beakers, culture, or faith).
You can "improve" any tiles inside your culture borders (whether inside the first 3 rings of tiles or in rings 4 or 5), but you can only assign a citizen to "work" a tile in the first 3 rings.
You can see this most clearly in the Citizen Management tab (top right of the City View screen; for reasons I don't understand, City View opens by default with Citizen Management closed). Citizen Management allows you to set your "focus" for your citizens in that city, and also shows you what specific tiles they are "working."
You will see that the city center tile is always "worked" (no citizen is assigned to that tile), and each other citizen is either assigned to work a specific tile (green shading on worked tiles) or is assigned to work a specialist slot in, e.g., a workshop, university, factory, bank, etc. (see right hand side, below Citizen Management). You will quickly figure out that tiles in the 4th and 5th rings cannot have citizens assigned to them, and that you can't work more tiles than you have citizens (ignoring the city center tile).
Play around with setting different focuses to see how citizen assignments change. You can also manually assign citizens to work particular tiles (often referred to as "locking down" tiles, since the tile symbol turns into a little padlock).
The result is that you can improve tiles in rings 4 and 5 to obtain resources on those tiles (horses, oil, copper, wine, or whatever), both for purposes of happiness (luxuries) and building particular units and for purposes of trade, but you cannot "work" those tiles to obtain those tiles' yields.