One observation is that your rule about distinct characters means that the maximum number of segments that any symbol in your set can occupy is equal to N_segments - N_symbols + 1. (And you can apply that recursively to the segments it doesn't occupy and the other symbols.)
The first corrolary of this is that, if you have a symbol of this maximum size, it must produce a legible symbol when you add any single extra segment.
The second corrolary of this is that, if you have a symbol of this maximum size, it also must produce a legible symbole when you add any combination of extra segments.
So, to beat your best of 3 for a 7-segment display, you can't use any symbols that occupy more than 4 segments. The possiblities there that I can see are (1 segment) - and ' in multiple places, (2 segments) =, r, 1, (3 segments) n, u, c in two places each, L, J sans hook, 7 sans hook, (4 segments) h, 4, [, ], o in two places, F, J and L with hook, 7 with hook, and t.
Several of the four-segment ones fit the first corrolary -- 4 (if you allow adding the lower segment to count as a Y), F, L with hook, and t. All but F fail the second corrolary when you make an upside-down A, and F fails because P and P-with-hook (add lower segment) are not distinct.
Thus, a set of four has to have a maximum symbol size of 3 segments. A third corrolary, then: If two symbols combine to make a symbol of maximum size for a set of one fewer symbols, then that combination must fit the conditions of the first and second corrolaries. That applies to all pairs of five segments, and rules out pairs of six segments.
Attacking from the other end, there's no way to combine the one- and two-segment sets to make a working set of three (- and ' can only make r, so that rules out r, and individually they don't work with = or 1; ' doesn't work with r or 1; and r and 1 don't work together). So, to get a set of four symbols, we have to include a couple of three-segment ones.
Any two of upper n and upper u or upper c make an upper o. Upper u and lower n make H and thence upside-down A and out. Upper u or upper c combine with 7, making q, from whence we get A and g. Upper u also combines with J sans hook, making y and thence upside-down again. Upper c and lower n make h-with-hook, and thence A and b-with-hook. Upper c combines with L, making E and thence b and P with hooks. Any two of lower n and lower u or lower c make o. Lower n with L makes b, or with J makes d, and either thence gets upside-down A. Lower c with J makes d, same problem. L and J make U, and again the upside-down A. J and 7 make ].
Thus:
upper lower
n u
n c
u c
u J
c n
c L
n u
n c
u c
J 7
There aren't any trios in there. 7 and L only show up in one row, remove those and J is only in one row, combining all lowers or all uppers makes o several ways, and there's only one upper-lower combination.
Thus, we'd need to add singles or pairs to these. Considering just which ones are valid individually, we have (for simplicity, not recording positions):
upper lower
n u ' 1
n c 1
u c 1
u J -
c n 1
c L 1
n u ' 1
n c r
u c 1 r
J 7
There's only one of these where there are two smaller symbols that go together -- the second-to-last. That's an upper r, and a right-side 1. u + c = o, u + r = G, u + 1 = J with hook, c + r = E, c + 1 = d, r + 1 = 7 with hook, u + c + r = 6, u + c + 1 = d, c + r + 1 = 8 -- and there it fails, since that's not distinct from the set of all of them.
Thus, I think that's an exhaustive proof that your set of seven-segment numbers is a maximal set.
Edit: The proof is only true if the initial set is complete. I left out ". It doesn't make triplets, doesn't work with all-lower or all-upper three-segment pairs, and doesn't work with J or 1, so it doesn't affect the result.
Also, things get slightly more interesting if - and _ are considered distinct, but not a lot; they don't trio with any of the other one- and two-segment symboles, and as a pair they don't combine with any upper or lower three-segment symbols, nor the J or 7. Likewise ' and , -- they again don't trio with the other one- and two-segment symbols, and as a pair don't combine with any of the three-segment pairs.
Edit 2: I also missed upper u or upper c and 7 make 9. Again, this doesn't go anywhere; even though u and c also go together, the trio makes 9 two ways. Neither of those go anywhere with smaller symbols either.
