We've tried several methods of deciding when a Justice can't rule on a case.
The
method, saying "back in the day, justices knew how to act".
The
method, of threatening a CC or never voting for the person again.
The
method, or just nagging at someone without making overt threats.
The
method, basically saying you can't decide and tossing a poll out there. (sorry Donsig, it's not meant to be a personally identifiable method, really) 
Then there's the tactic of acting
like "what conflict, I really don't understand."
Now, we can leave things status quo, or we can put a little common sense into the answer. How about this for ways we might decide Justice must step aside.
The

The

The

The


Then there's the tactic of acting

Now, we can leave things status quo, or we can put a little common sense into the answer. How about this for ways we might decide Justice must step aside.
- A Justice can decide to step aside. Not much chance with this one given there are so many people worried about being "bullied", but one can always hope.
- We could say that if enough people ask for a Justice to step away from a case, they must appoint a pro-tem.
- We could put a provision in the law for a poll on the matter. There would be two arguments about that poll though, should it be private and how long should it last.
- The other two members of the court might be able to require the 3rd one to appoint a pro-tem. Then you get into issues where more than one Justice is affected by a case.
- We could drop the whole idea anyway and decide there is no way to administer making the decision a pro-tem is needed, and trust our Justices to do the voluntary thing.