Are the MLF files only required to control loading order, or are they absolutely mandatory?
They only control load order (of directories), if load order does not matter (as there are no dependencies), you do not need to include them.
Generally speaking they never are needed for regular (i.e. BtS style) modular xml. The WoC modular xml at times spreads a definition across multiple files, e.g. the UUs, UBs and leaders of a civ are not defined in the civ xml (i.e. where the city list etc. is defined) to make those parts pluggable. As they (UU, UB, LH) all contain parts of the civ-definition, but cannot stand on their own (but need the main civ definition), you add a dependency that only loads them if the civ is already loaded and control load order so the civ is loaded first.
So, if you use WoC modular (which is far more advanced than BtS modular), you frequently would need MLF files, if you stick to BtS modular, you never need them.
Note: MLFs only influence the load order of directories, CIV has an inherent load order of files based on their type (e.g. *_CIV4CivilizationInfos.xml or _CIV4LeaderHeadInfos.xml). All files of one type are loaded ahead of all files of another type (which type is loaded first is defined by CIV). You cannot influence the CIV load order via MLFs, you can only tell the mod which dir to load first.
In other words, if I slip this into my mod code, will everyone who has already written a modular mod have to add something to what they have written, or are they safely able to keep using the old modular style, even though new and fancier toys do exist now to make life easier? (Mostly I want the WoC modular style so that readpass2 and readpass3 fields will be able to be used by modules).
BtS modular is a valid subset of WoC modular and does not need MLF. However there are some things you simply cannot do modular in BtS modular xml, e.g. techs always have to be in one large file and in the right order at that so all interdependencies are met.
In WoC xml you can do them modular (i.e. one file per tech), but then you would need MLF to control load order, just like order of appearance influenced load order for regular xml.
If you do mix and match BtS and WoC modular and your WoC xml in turn depends on stuff defined in BtS modular you still will have to make sure the BtS definition is loaded first, so the definitions are available when needed by the WoC xml (using MLFs to do so).
As long as everything is BtS modular only, MLFs are not needed at all however (with the restriction of some things simply not being possible in a modular way).