I wrote a utility function once to do that.

The idea was to linearise the RGB space into a 1D [0, 1] interval. For example doing (R + G*256 + B*256*256)/(256^3*256^2+256).

Then, everytime you need a new colour, you split an interval generated by previous samplings in 2. You pick the first colour as 0.0 in the new space, the second as 1.0, the third as 0.5, then 0.25, then 0.75 and so on. This kinda guarantees that if you generate few colours they have maximum 'separation' (though nor measured in terms of hue). As you generate more and more colours, they tend to get closer to one generated before, but always respecting a max-separation principle.

Once you have your [0, 1] value, you use the inverse function (triplet of functions actually) to go back to RGB.

In the basic implementation you get white and black as the two first colours. If you want something different, once you have generated your 'input' [0, 1] value, you can rotate it, say by a third, inside the same interval.

This works pretty well and it behaves deterministically (no unbounded number of retries).