Character customization is a great asset for games today, especially for multiplayer online games. Players love customizing their characters and give them a unique look and tailor things to their liking. We recently explored a few methods for real time texture painting for our kids’ game, DrawFish. The game allows kids to paint a fish to their liking and play with it as it swims and jumps in the pond.

Texture Painting Approaches

Let’s see how we can paint textures at run time in Unity. The most straightforward approach is to read raw pixel values from the texture and modify them at runtime.

Updating Raw Colors

This approach uses GetPixels to read the color values in the texture, modifies it as per user input and finally applies it back using SetPixels.

public void Circle (Texture2D tex, int cx, int cy, int r, Color col)
{
  int x, y, px, nx, py, ny, d;
  Color[] tempArray = tex.GetPixels ();
  int length = tempArray.Length;
  for (x = 0; x <= r; x++) {
    d = (int)Mathf.Ceil (Mathf.Sqrt (r * r - x * x));
    for (y = 0; y <= d; y++) {
      px = cx + x;
      nx = cx - x;
      py = cy + y;
      ny = cy - y;

      if (py * tex.width + px >= 0 && py * tex.width + px < length) {
        tempArray [py * tex.width + px] = col;
      }
      if (py * tex.width + nx >= 0 && py * tex.width + nx < length) {
        tempArray [py * tex.width + nx] = col;  
      }
      if (ny * tex.width + px >= 0 && ny * tex.width + px < length) {
        tempArray [ny * tex.width + px] = col;  
      }
      if (ny * tex.width + nx >= 0 && ny * tex.width + nx < length) {
        tempArray [ny * tex.width + nx] = col;  
      }
    }
  }    
  tex.SetPixels(tempArray);
  tex.Apply ();
}

The main overhead in this approach comes from the Texture2D.Apply call. It actually applies all previous SetPixel and SetPixels changes. What Apply does behind the scenes is that it uploads the texture from the CPU to the GPU which is going to be very slow if you want to run at 60 FPS or even 30 FPS.

Using Render Texture

Render Textures are special types of Textures that are created and updated at runtime by using a Camera to render into them.

Render Texture Setup

To simulate real time painting using Render Texture, we will set up a Quad containing the original texture to be painted with an orthographic Camera pointing right at it and rendering into a Render Texture. This Render Texture is then used as a Texture on the fish.

Render Texture Setup

The quad in the background should have an unlit texture to prevent lightning artefacts on the texture. The original texture rendered on the fish will have lightning enabled, though.

Collider Setup

The next part is to map the clicks on the fish to a point on the Texture. For this, we will add a Mesh Collider to the fish and use Ray Casting to map the 3D world coordinates to the 2D UV coordinates for use on the texture.

RaycastHit hit;
Vector3 cursorPos = touch; // For mouse clicks, use new Vector3 (Input.mousePosition.x, Input.mousePosition.y, 0.0f);
Ray cursorRay = sceneCamera.ScreenPointToRay (cursorPos);
if (Physics.Raycast (cursorRay, out hit, 200, layerMask)) {
  MeshCollider meshCollider = hit.collider as MeshCollider;
  if (meshCollider == null || meshCollider.sharedMesh == null)
  	return false;
  Vector2 pixelUV = new Vector2 (hit.textureCoord.x, hit.textureCoord.y);
  uvWorldPosition.x = pixelUV.x - canvasCam.orthographicSize;// To center the UV on X
  uvWorldPosition.y = pixelUV.y - canvasCam.orthographicSize;// To center the UV on Y
  uvWorldPosition.z = 0.0f;

  return true;
} else {
  return false;
}

Dynamic Painting

Now that we have the position where we want to paint on the texture, all we need is to put stuff in front of our quad so it appears on the render texture. We can instantiate anything we want to control what kind of paint we would be doing. Create a Sprite with the following image:

Brush Texture

To paint different colors, we can just change the Color property on the sprite object. This also allows us to take advantage of static batching in unity to reduce draw calls to batch the sprites together.

void InstantiateBrush (Vector3 uvWorldPosition)
{
  // Get a new brush from the pool
  GameObject brushObj = brushPooler.GetPooledObject ();
  if (brushObj == null) {
  	// No brush found for reuse. Save texture
  	StartCoroutine (SaveTexture ());
  	return;
  }

  Color color = eraseMode ? eraseColor : brushColor;
  brushObj.GetComponent<SpriteRenderer> ().color = color; // Set the brush color
  brushObj.transform.parent = brushContainer.transform; // Add the brush to our container to be wiped later
  brushObj.transform.localPosition = uvWorldPosition; // The position of the brush (in the UVMap)
  brushObj.transform.localScale = Vector3.one * brushSize;// The size of the brush
  brushObj.SetActive (true);
}

Merging Brushes Into Texture

We can see the fish being painted in real time as soon as we instantiate the brushes in the scene. But we cannot keep on instantiating objects in the scene indefinitely or we will soon run out of memory. After we reach a certain point where a lot of brushes have been instantiated, we need to combine all of them into the texture being rendered on the quad and remove them from the scene. The painting can then resume by generating more brushes in scene until we reach the threshold again.

public IEnumerator SaveTexture ()
{
  yield return new WaitForEndOfFrame ();

  RenderTexture.active = canvasTexture;
  Texture2D tex = new Texture2D (canvasTexture.width, canvasTexture.height, TextureFormat.RGB24, false);
  tex.ReadPixels (new Rect (0, 0, canvasTexture.width, canvasTexture.height), 0, 0);
  tex.Apply ();
  RenderTexture.active = null;
  baseMaterial.mainTexture = tex; // Put the painted texture as the base

  // Clear brushes
  foreach (Transform child in brushContainer.transform) {
    child.gameObject.SetActive(false);
  }
}

Here is an example of how the painting works on the render texture and how it looks on the fish.

Painted Fish Render Texture

Painted Fish