Format providers are pluggable, but as far as I know, you can't set up a default format provider for a given type. Methods that take format strings by convention typically also have an overload that takes an IFormatProvider. You can implement two classes such as MyFormatProvider : IFormatProvider and MyCustomFormatter : ICustomFormatter.
When a format string such as "{0:dd}{0:or} {0:MMM} {0:yyyy}" is processed, your ICustomFormatter will be called 4 times, with "dd", "or", "MMM", and "yyyy" as the parameters so it can format each part separately. You would simply have to handle the "or" case and pass the rest onto the default format provider.
However, the caller will still need to pass in an instance of your custom format provider when formatting. For example:
string str = String.Format(MyFormatProvider.Default, "{0:dd}{0:or} {0:MMM} {0:yyyy}", date);
As far as I know, there's no escaping that requirement. I searched for a better way a while back when I had created a "relative" DateTime format string that displayed DateTimes like "1 hour ago", "3 days ago", etc. Couldn't figure out a way to avoid having to explicitly pass my custom IFormatProvider.