And don't think I won't do it!
Meh, there's a big difference between altering a scenario file (at which point you might as well just create a new scenario) and altering a core game database file. The reason you don't edit the core database is that there's no fallback; if you screw it up and it breaks the game, you have no easy way to disable your changes and play the baseline game to see what went wrong. And it also screws up your ability to use any other mods, since you've now changed the baseline on which they operate. Scenarios, obviously, don't have these problems.
The Steam auto-updating would still be a problem, but like I said, at that point you might as well just create a custom Scenario. So copy NewWorldScenario into a NewWorldScenario2 directory, change the files in there, and see what happens. There, no Steam problems, since it'll only ever overwrite the original NewWorldScenario.
As to the specific Scenario question, I have no idea; I don't play Scenarios, and I don't buy DLC. (At least, I don't buy this sort of civ-specific DLC. If they were to sell, say, a SMAC expansion as a DLC then I'd be all over that.)
For the Privateer question, it's possible to add new units, so yes, you could add a Privateer-type unit. Unfortunately, the Hidden Nationality stubs don't seem to actually work, you can't easily add new 3D unit models, and it'd always put the new unit at the end of the build list, so it's not really easy to cleanly insert a new unit.