What are some programming techniques for converting SD images to HD images

Question

I'm taking programming class and instructor loves to work with images so most of our assignments involve manipulating raw RGB image data. One of our assignments is to implement a standard image converter that converts SD images to HD images and vice versa. I always take advantage of these types of assignments to go a little beyond what we were asked to do, so I added a basic anti-aliasing process that uses the average pixel color of the 3x3 surrounding pixels to improve the converted image. While it helps a bit, the resulting image still doesn't look good, which is ok because it's not expected to for the assignment. I've learned that converting an SD to HD images has shown to be much harder than down sampling to SD from HD just because SD to HD effectively involves increasing resolution when it is not there. Obviously, it is hard to create pixels from nothing, but I'd like enhance my anti-aliasing to something that provides better results when upscaling an image. Most of the techniques I find and read on the internet are far beyond my level of image processing and programming.

Can anybody suggest any better methods or processes to create good HD content from SD content that may be within my programming skill level? I know that's a difficult thing to gauge since you don't know me, but perhaps knowing that I can write c++ code to read in raw RGB data and upscale/downscale it with simple average-anti-aliasing will give you an idea.

Thanks in advance for all your help!

Answer 1

Unless I'm somehow mistaken, you're asking for resampling techniques. The first improvement from nearest neighbor is bilinear interpolation; the next step up from that is typically bicubic interpolation, which is significantly more complex. There are tons of resampling algorithms out there, but as you say, quality upscaling is hard in general, since you can't create new information.

Answer 2

From experimentation, usually what I've done in the past on NLE workstations, is to upsample/upscale, past the target resolution then down sample again. Then adding a sharpness filter.

Of course the bottom line is you can't create data that doesn't exist in the first place, but you can get a 'precieved' 'HD' look.

Answer 3

Interpolation never adds information. To restore information you must have a model for what caused the loss of information.

If information was lost due to blurring, you can try unsharp masking (a very old and easy technique used on videos). Google it.

Reverse diffusion can reduce apparent blurring due to undersampling (c.f. Partial Volume Reduction by Interpolation with Reverse Diffusion, Salvado et al. Int J Biomed Imaging. 2006). The introduction in this paper should answer many of your questions.

If information was lost due to a bad acquisition device, deconvolution is sometimes used. It is a very wide and active field of research. Deconvolution can give very good reconstruction if the right model is used, but it can also increase noise.

An entirely different problem is synthesizing new high resolution information when multiple low resolution images are available. These super-resolution algorithms use sub-pixel shifting to construct a model of what the true image looked like. The noise must be very accurately modeled or else the results will be bad. (c.f. "Super-resolution image reconstruction: a technical overview", Kang., IEEE Signal Processing Magazine, 20(3):21–36, May 2003.)

You'll have to get a good image processing book if you want to understand these algorithms. I suggest Image Processing Handbook, CRC Press, Third Edition, 1999, John C. Russ.