Those sound more like standard stealth game problems than RPG issues. All stealth games (or games with stealth systems) from the first Thief to the latest Deus Ex encourage some degree of save scumming.
What makes Styx particularly bad ?
A lack of anti-frustration abilities for stealth games (see through walls, vision cones, short-term invisibilty) ?
Right, those were all specific things I experienced in the first ~30 minutes of that particular game, not every game. I was just using them to illustrate my earlier point about a game moving forward after the character or player fails to do something, or does it a little differently, a little inefficiently, or less than perfectly. You're right that the issue is common in stealth games, even though this game seems particularly bad in this regard (it's noticeably worse than the first
Styx game, which had the typical stealth-game problems).
Games that combine stealth with other things often end up solving the problem, perhaps by accident.
Styx is a "chute-style" game, where your path is prescribed but you have a little "wiggle room" to move around. An open-world exploration game, otoh, allows the player to reconnoiter an area, just by virtue of being open-world. The stealth mechanics in
Fallout 4 aren't any better than in
Styx, but it turns out to be a better stealth game. Of course, stealth only matters in
Fallout 4 if you play at the highest difficulty setting and use mods; in the vanilla game, it's so much easier to just get a suit of powered armor and an arsenal of weapons and destroy everything on sight (at this point, though, anyone playing
Fallout 4 unmodded is just being dense, and not only because of the stealth game). Someone earlier mentioned
Dishonored, either in this thread or the other games thread, which has the same issue; playing stealthily is so much harder and less efficient than simply killing everybody, and the consequences of bathing in everyone's blood aren't enough to discourage it. This isn't really "solving" the stealth-game problem, so much as shooting it in the head and declaring "problem solved."
I think the last, really good, stealth game I played was
Thief: The Dark Project. I'm not sure why the game design on stealth stalled after that. The next stealth game I remember playing was
Splinter Cell, which got into the "die, start over" style of play. I was so enamored with the setting and the graphics of that game, and some of the character's interaction with the environment, that I was happy to overlook it's glaring flaws ("I can climb along the outside of a moving train, but I can't step around this camera tripod without knocking it over? Seriously?"

). At the time, I chalked up some of the differences - good and bad - to the fact that
Splinter Cell was a console game. I guess it's possible that the demands placed on console games heavily influenced game design across all platforms in the last 15 years in ways that I'm not even aware of, and some things have been forgotten, like the welding techniques developed by shipyard workers during WWII.
The
Deus Ex games were pretty good, fwiw. I don't remember being aggravated by those, in this way. Some of the military sci-fi games, like
Mass Effect and
XCOM 2, have stealth mechanics that suit the role of stealth in those stories, which is all you need anyway. Stealth games raise the bar for themselves by making stealth more important.
Thief and
Styx go all the way to one end of the spectrum, making the player-character really poor at fighting.
Deus Ex and
Dishonored are somewhere in the middle, and the character is pretty deadly but a lot of things can be accomplished by finding a back window or a duct.
Mass Effect and
XCOM are at the other end, where stealth is just for the preliminary stages of a mission and you're mostly supposed to kill everything to progress.