You can do so with tiles.
Fair enough. Do you know whether a tile remains unassigned after the city governor is updated, like after a citizen birth? I'm not 100% sure but my guess is it doesn't, because I remember changing city-focus could fill those tiles again. I can implement it similarly for specialists, but as with tiles, the emptied slots could get re-filled after governor updates.
If that’s the case then manual specialist control should be the default or something. Having to tick an extra box in another part of the UI before I can unslot specialists is bad.
My first iteration was like that, left-click worked exactly as before and right-click locked/unlocked specialists (instead of opening civpedia). That was inconsistent with tile-locking though as there isn't a left/right click differentiation there. This also kept the old behavior of left-click automatically giving manual control, similarly inconsistent since you don't unassign all tiles when you click on one. I think that's worse than the current version as having to re-assign all specialists manually was pretty tedious. I can think of several workarounds:
a) Tile behavior: Left-clicking on a (governor) filled specialist slot removes the citizen, making it a laborer/unemployed. But this wouldn't be a solid "don't work this slot" command and the governor could re-fill it upon update. Left-click on an empty slot fills&locks it. I'm leaning towards this.
b) Left-clicking on a "locked" specialist removes the citizen and enables manual control without dismissing all other specialists. This would assure the unslotted specialist wouldn't get worked, but it turns manual control on so you'd have to update your specialists manually.
c) Left-click works exactly like before, specialists are locked with ctrl+click, or right click or whatever. Obviously I'm not a fan of this, it's inconsistent with tile-locking behavior and hides the feature deep.