Of course, you should never edit the original files themselves. This should go without saying. But yes, editing the HandicapInfos table in XML can allow you to create intermediate difficulties.
Difficulty levels modify a ton of things at once: multipliers for city growth and research are obvious ones, and the AIs also get multipliers to unit upkeep costs, building upkeep costs, city population unhappiness, and so on, but the AIs even start with extra techs once you're above Prince. So yes, it'd be easy enough to tweak the values for one of the difficulties to make it slightly easier or harder, without having to do the same for every variable at once.
But be careful with this, because, like any other XML mod, it can conflict with other mods you might want to use. For instance, the base city unhappiness equation is 2 + 1/pop. But unfortunately, that "1" is stored as an integer. So, take my own mod, where the equation is 4 + 1.2/pop instead; the only way to get that 1.2 was to take the unhappiness scaling factors (which are stored as integer percentages) in the Handicaps tables and multiply them by 1.2. So the "60" for Chieftain becomes a 72, the 100 for Prince becomes a 120, and so on. Other mods that scale this population unhappiness use a similar method.
Now, a mod like mine CAN do this through SQL, which wouldn't conflict with your edits, but most don't. The migration from XML to SQL is an ongoing process for many modders.