The tradition has so much more to it than Marx even though he's foundational, obviously. He must be if you buy in to historical materialism and it is tautological why this is the case: no one can write a meaningful critique of capital until at least one generation of humans have come of age in what resembles a capitalist society.
However the basic contradictions between organizing a society and allowing for the accumulation of vast amounts of wealth and power go all the way back to ancient Greece. Aristotle himself noted that economic inequality was incompatible with democracy. Whenever some rightwing media pundit starts blathering on about western values or the western intellectual tradition I always end up wanting to throw up in my mouth a little because they ALL make it quite clear within minutes that they have no clue what they are talking about.
No one takes this stuff that religiously anywhere in the world at this point. No nation can, because the US and the Eurozone have a stranglehold on the global economy. Talking about it in that fashion totally misrepresents what's left of the socialist experiment which really does amount to handful of state-capitalist nation states that attempt to set floors on social welfare and aggressively pursue growth for themselves regardless of what the planners in Washington, London, and Brussels might want. The idea that this is voluntary is Orwellian doublespeak because the only alternative is crushing poverty. You will also live under constant threat of coup, military invasion, etc. Please note that, to a man, all capitalism apologists must either deny the crimes of empire or rationalize them.
The dialectical stuff is a picayune philosophical tangent and one does not need to understand it, or even necessarily buy into it, because it's of no practical importance. A common sense explanation of historical materialism and class warfare can be understood by almost anyone. Whether or not they will hear you out or not is another matter. Your enemies control the culture, education, all the media, the tech companies. Gramsci laid out why this was important.
To attempt to answer OP's question Marx's stateless society was seen as "the end of history" because all material conflict was hypothesized to have been removed from society. The basic premise of historical materialism is that all of human politics, and thus human history, is driven by material conflicts of interest and how they are resolved.
I agree that's not happening and it's why the state isn't going anywhere. Anarchism is a childish fantasy which is peddled to direct the time, attention, and energy people are willing to give to politics into avenues which can't accomplish anything. People remained passionately sincere about this ideology in large numbers through the late 19th and early 20th century, so it's impact can't just be hand waived away, but these conceptions have no chance of taking power anywhere in the world that matters at this point. The difference between now and then is that modern day anarchists use their ideology as a excuse to sit out of the struggle to seize the power of the state whereas their forebears did not. This difference probably can be attributed to Marx becoming taboo in the west.
However I do agree with the conception that human politics are the resolution of material conflicts of interest. All the other bullfeathers is just window dressing to hide this fact from people. Religion, idpol, culture war, in the modern day are all instruments used to heard us into political coalitions which do not represent our material interests.
The only real question left, as I see it, is whether or not you want a worker's state or a bourgeoisie state.
