There are several things they might be doing. If you have a fresh water source and they don't, they might come and irrigate from your land to theirs. They will also build roads through your land to get to resources they don't have; they will build a colony once they reach the resource. If they possess a distant city, the shortest overland route may be through your territory. They will also trek through your lands to build a trade network with you or other civs. They can also be an excuse to aggression; if you order them to leave too many times, the civ will declare war on you which is kind of stupid if they were trying to connect to water, trade, or resources. Technically they don't operate as spies since they can't enter your cities unless you capture them.