iPhone paint app (glPaint based). Blending with white background.

Question

Hi, I'm developing painting app. I've tried to do it with CoreGraphics/Quartz 2D and drawing curves algorithm is pretty slow. So we've decided to switch to the OpenGL ES. I've never had any OpenGL experience, so I found glPaint example from apple and started play with it.

I've changed erase method do make white background. How I stuck with brushes and blending. In the example Apple uses "white on black" texture for the brush (first on the pic below). But it didn't work for me (I played with different blending modes). So I've decided to use different brushes, but I didn't find the proper way. I found few questions on the stackoverflow, but all of them were unanswered. Here is a picture (from another question, thanks to Kevin Beimers).

So the question is how to implement stroke like "desired" in the picture. And how to blend 2 strokes closer to real life experience (blue over yellow = dark green).

Thanks.

There is current code (bit modified from glPaint) for the brush (from initWithFrame method:

// Make sure the image exists
if(brushImage) {
  // Allocate  memory needed for the bitmap context
  brushData = (GLubyte *) calloc(width * height * 4, sizeof(GLubyte));
  // Use  the bitmatp creation function provided by the Core Graphics framework. 
  brushContext = CGBitmapContextCreate(brushData, width, width, 8, width * 4, CGImageGetColorSpace(brushImage), kCGImageAlphaPremultipliedLast);
  // After you create the context, you can draw the  image to the context.
  CGContextDrawImage(brushContext, CGRectMake(0.0, 0.0, (CGFloat)width, (CGFloat)height), brushImage);
  // You don't need the context at this point, so you need to release it to avoid memory leaks.
  CGContextRelease(brushContext);
  // Use OpenGL ES to generate a name for the texture.
  glGenTextures(1, &brushTexture);
  // Bind the texture name. 
  glBindTexture(GL_TEXTURE_2D, brushTexture);
  // Set the texture parameters to use a minifying filter and a linear filer (weighted average)
  glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
  // Specify a 2D texture image, providing the a pointer to the image data in memory
  glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, brushData);
  // Release  the image data; it's no longer needed
        free(brushData);
  // Make the current material colour track the current color
  glEnable( GL_COLOR_MATERIAL );
  // Enable use of the texture
  glEnable(GL_TEXTURE_2D);
  // Set a blending function to use
  glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
  // Enable blending
  glEnable(GL_BLEND);
  // Multiply the texture colour by the material colour.
  glTexEnvf( GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE );

}

//Set up OpenGL states
glMatrixMode(GL_PROJECTION);
CGRect frame = self.bounds;
glOrthof(0, frame.size.width, 0, frame.size.height, -1, 1);
glViewport(0, 0, frame.size.width, frame.size.height);
glMatrixMode(GL_MODELVIEW);

glDisable(GL_DITHER);
glEnable(GL_TEXTURE_2D);
glEnableClientState(GL_VERTEX_ARRAY);
  glEnable(GL_BLEND);




// Alpha blend each "dab" of paint onto background
glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );


//glBlendFunc(GL_SRC_COLOR, GL_ONE);
glEnable(GL_POINT_SPRITE_OES);
glTexEnvf(GL_POINT_SPRITE_OES, GL_COORD_REPLACE_OES, GL_TRUE);

self.brushScale = 3;
self.brushStep = 3;
self.brushOpacity = (1.0 / 1.5);

glPointSize(width / brushScale);

//Make sure to start with a cleared buffer
needsErase = YES;
[self erase];

Answer 1

Let’s start by defining the type of blending you’re looking for. It sounds like you want your buffer to start out white and have your color mixing obey a subtractive color model. The easiest way to do that is to define the result of mixing C_brush over C_dst as:

C = C_brush × C_dst

Notice that using this equation, the result of mixing yellow (1, 1, 0) and cyan (0, 1, 1) is green (0, 1, 0), which is what you’d expect.

Having a brush that fades at the edges complicates things slightly. Let’s say you now have a brush opacity value A_brush—where A_brush is 1, you want your brush color to blend at full strength, and where A_brush is 0, you want the original color to remain. Now what you're looking for is:

C = (C_brush × C_dst) × A_brush + C_dst × (1 - A_brush)

Since blending in OpenGL ES results computes C = C_src × S + C_dst × D, we can get exactly what we want if we make the following substitutions:

C_src = C_brush × A_brush

A_src = A_brush

S = C_dst

D = (1 - A_brush)

Now let’s look at what it takes to set this up in OpenGL ES. There are 4 steps here:

1. Change the background color to white.

2. Change the brush texture to an alpha texture.
   By default, GLPaint creates its brush texture as an RGBA texture with the brush shape drawn in the RGB channels, which is somewhat unintuitive. For reasons you’ll see later, it’s useful to have the brush shape in the alpha channel instead. The best way to do this is by drawing the brush shape in grayscale with CG and creating the texture as GL_ALPHA instead:

   CGColorSpaceRef brushColorSpace = CGColorSpaceCreateDeviceGray();
   brushData = (GLubyte *) calloc(width * height, sizeof(GLubyte));
   brushContext = CGBitmapContextCreate(brushData, width, width, 8, width, brushColorSpace, kCGImageAlphaNone);
   CGColorSpaceRelease(brushColorSpace);
   

   ---

   glTexImage2D(GL_TEXTURE_2D, 0, GL_ALPHA, width, height, 0, GL_ALPHA, GL_UNSIGNED_BYTE, brushData);        
   

3. Set up C_src, A_src, S and D.
   After switching to an alpha texture, assuming that your brush color is still being specified via glColor4f, you’ll find that the default OpenGL ES texture environment will give you this:

   C_src = C_brush

   A_src = A_brush

   In order to obtain the extra multiplication by A_brush for C_src, you’ll need to set up a custom combiner function in the texture environment as follows (you can do this in the initialization function for PaintingView):

   glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE);
   glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_RGB, GL_SRC_ALPHA);
   

   Changing GL_TEXTURE_ENV_MODE to GL_COMBINE gives you C_brush × 0 (to see why this is the case, read section 3.7.12 in the OpenGL ES 1.1 specification). Changing GL_OPERAND0_RGB to GL_SRC_ALPHA changes the second term in the multiplication to what we want.

   To set up S and D, all you need to do is change the blending factors (this can be done where the blending factors were set up before):

   glBlendFunc(GL_DST_COLOR, GL_ONE_MINUS_SRC_ALPHA); 
   

4. Ensure that any modifications to A_brush outside of the brush texture are reflected across other channels.
   The above modifications to the texture environment only take into account the part of the brush opacity that come from the brush texture. If you modify the brush opacity in the alpha channel elsewhere (i.e. by scaling it, as in AppController), you must make sure that you make the same modifications to the other three channels:

   glColor4f(components[0] * kBrushOpacity, components[1] * kBrushOpacity, components[2] * kBrushOpacity, kBrushOpacity);
   

Note that the downsides to implementing your brushes with a subtractive color model are that colors can only get darker, and repeatedly drawing the same color over itself can eventually result in a color shift if it’s not one of the primary subtractive colors (cyan, magenta, or yellow). If, after implementing this, you find that the color shifts are unacceptable, try changing the brush texture to an alpha texture as in step 2 and changing the blend factors as follows:

glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

This will give you simple painting of your brush color over white, but no actual mixing of colors (the brush colors will eventually overwrite the background).