That's just what the government wants you to think.
Seriously though, I swear I heard years ago that Jersey and NYS got in a tiff over where Liberty Island sits and it was determined that it was in Jersey. Does anyone else remember that or am I going crazy?
Edit: Figured it out. NY v NJ 523 US 767 (1998) references a compact where by NYS would own the island and NJ the surronding waters. The suit determined that additional parts of the island that were created via fill belong to NJ, not NYS.
So it turns out I was right; that Liberty Island does sit in NJ, even if the island itself belongs to NYS.