It probably would be easier to part with meat, poultry, fish, etc. Emphasis on easier because I think the societal factor plays a decent role. People (in North America at least) are usually familiar with vegetarianism, and tend to know whether a dish contains meat or not (I consider meat to include poultry and fish). Fewer people are familiar with veganism, and even those that are would probably find it odd if you ate meat but no dairy/eggs/etc. There's also the factor that milk and eggs (especially) are ingredients in places where we don't always think about them. Some breads contain milk, others don't. Some sweets contain eggs, others don't. It's when you are buying pre-cooked items or eating out that this can present problems and inconveniences.
I'm not sure which I would miss more. I drink a lot of milk, but there's other beverages I enjoy, too. It's possible to make some pretty good vegan baked goods, so I think I could adjust to no eggs/dairy there, too, and I already prefer dairy-free sorbet to ice cream sometimes. I also am not one to eat eggs on their own, so I could see possibly going that route. On the other hand, I already eat plenty of vegetarian dishes, and vegetarianism isn't too restrictive when dairy/eggs are allowed. But for which would be easier, probably meat.
