People are taking it differently because you're trying to argue that moderates are the swing vote in matters of morality and should be respected as such, complete with agendas to convince the moderate to fight for someone else's rights. This by itself can be argued, but then you go on to say that you're willing to sacrifice that fight if it means a better budget or some other form of personal gain. You're making the explicit point that social progress is less important than money, which corrupts your initial argument that moderates are allies.
It corrupts your initial argument because it no longer puts a moderate in the position of ally, it puts a moderate in the position of convenience. You'll help people fight for their rights so long as it's convenient to your bottom line. This largely resonates with the right wing and not as much with the left. Then, it can't be too much of a surprise if you're treated more like you're on the right instead of the left. You made it clear that you are interested in left-aligned policies only so long as they don't impact you, and that you're willing to bargain with other people's lives the moment they do.
You marched for gay rights. Awesome. Now you're willing to usher in a party that loves conversion therapy and limiting the rights of queer people. Why? Because of the budget. You're willing to throw away your "marching for gay rights" because of a balance sheet. The people you supposedly fought for have become secondary, meaningless, in the face of the almighty dollar.
This is why "moderates" are labelled as fence-sitters. They go to the side that is most convenient, and switch the second the wind shifts. You are an ally for as long as being an ally benefits you. Once it no longer does, you couldn't care less about the fight. Why should those solidly on the left be enthralled with the pursuit of the moderate's support when the moderate's support is unreliable and wont to shift at a moment's notice?