I would personally go with the simple "keep sampling until you get a valid value" approach unless there's a chance that you'll have a very large white rectangle against a grey rectangle which isn't much bigger. To me, simpler is almost always better.
An alternative would be to work out how many possible pixels there will be, and generate a random number in that range. Then effectively number the pixels from top left to bottom right. Work out whether the given random sample is in the top section, the bottom section or the middle (which you can do by just seeing if it's less than the first pixel on the top line of the white rectangle, or less than the first pixel on the line below the white rectangle). Once you've worked that out, it's a simple matter of working out the row, then the pixel within the row. This isn't hugely hard, but it is pretty fiddly and easy to get wrong. Note that this is determining a single random pixel: as you're generating large squares, you should consider the range of valid pixels for the top left corner of the square, and find a sample in that range.