Object Programming: Advanced Rendering Using Shader Objects

Applying Lookup Tables Using Shaders

The IMAGE_1D property on the IDLgrImage object lets you load color lookup table (LUT) values into a texture map and pass the LUT to a shader program. LUTs are useful for a number of tasks including:

The example code differs slightly from that presented here for the sake of clarity. Whereas the working example includes code needed to support user interface interaction, the following sections leave out such modifications to highlight the shader program components.

Basic LUT Shader Object Class

The shader_lut_doc object class inherits from IDLgrShader and contains the Filter method, just like the Basic RGB Shader Object Class. See that section for the base code or the example for the complete code. The one difference is this example uses the shader object VERTEX_PROGRAM_FILENAME and FRAGMENT_PROGRAM_FILENAME properties, which reference external shader program files for the vertex and fragment shader components.

Uniform Variable for LUT Example

In this example, a uniform variable named lut contains the values of the 256-element array of color table values. This can either be a custom LUT such as an enhanced greyscale color table, or one of the predefined IDL LUTs.

The following code creates a greyscale LUT defined by a curve rather than a linear ramp, making the dark areas darker and the light areas lighter. Notice that the 256-entry LUT is loaded into a one-dimensional image (an IDLgrImage object with IMAGE_1D property set). This IDLgrImage is automatically converted into a texture map for use by the shader. SetUniformVariable is called with the name of the uniform variable and the value (the image object) so the shader can access the texture map containing the LUT.

; Create enhanced grayscale LUT and store in 1-D IDLgrImage.  
x = 2*!PI/256 * FINDGEN(256) ;; 0 to 2 pi  
lut = BYTE(BINDGEN(256) - sin(x)*30) ;; Create 256 entry  
  
oLUT = OBJ_NEW('IDLgrImage', lut, /IMAGE_1D)  
  
; Store LUT in uniform variable named lut.  
self->SetUniformVariable, 'lut', oLUT  

The LUT is loaded into a texture map instead of a uniform variable array because it is more efficient to load and index the LUT when it is in a texture. In addition, under certain circumstances you can use bilinear filtering to interpolate between values in the LUT if it is in a texture map.

A side effect of using a texture map is it is limited by the maximum texture size (MAX_TEXTURE_DIMENSIONS in IDLgrWindow::GetDeviceInfo). On most hardware today this is 4096 by 4096 pixels, so if your LUT is larger than this you will need to work around this limitation (using a 2-D texture map is one possible solution). Also, as texture maps must be a power of 2 in size (128, 256, 512, 1024, etc.), ensure the size of your LUT is a power of 2 to keep it from being scaled to the next higher power of 2.

To display palletized images or to add color to greyscale images, simply load an RGB LUT into the 1D IDLgrImage rather than a greyscale LUT. The shader code remains exactly the same. (The shader_lut_doc__define.pro program lets you apply either the enhanced greyscale or one of IDL's pre-defined colortables.)

Hardware Shader Program for LUT Shader

This example reads the shader source from text files. The vertex shader (LUTShaderVert.txt located in examples/doc/shaders) contains the following code:

void main (void)   
{  
    gl_TexCoord[0] = gl_MultiTexCoord0;  
    gl_Position = ftransform();  
}  

This basic vertex program passes along the texture coordinate and then applies a transform to the vertex to correctly position it on the screen. The gl_TexCoord[0] is a varying variable that transmits data from the vertex program to the fragment shader program.

The fragment shader (LUTShaderFrag.txt located in examples/doc/shaders) contains the following code:

uniform sampler2D _IDL_ImageTexture;  
uniform sampler1D lut;  
  
void main(void)   
{  
    float i = texture2D(_IDL_ImageTexture, gl_TexCoord[0].xy).r;  
    gl_FragColor = texture1D(lut, i);  
}  

The fragment shader is where the lookup happens. The uniform variable, lut, which was defined in the IDL application using SetUniformVariable, contains the lookup table in a 1-D texture (of GLSL type sampler1D). As previously explained, the LUT is loaded into a texture map for efficiency.

The _IDL_ImageTexture variable is a reserved uniform variable that provides access to the 2-D base image (of GLSL type sampler2D). When a shader object is associated with an IDLgrImage object, and the uniform variable is not defined using SetUniformVariable in the IDL application, the base image object (a texture mapped onto a rectangle) is stored in a reserved uniform variable named _IDL_ImageTexture. The base image is the IDLgrImage to which the shader is attached. If it is attached to more than one image, the base image is the one currently being shaded. Non-base images are those passed to the shader program using SetUniformVariable.

Since more than one texture is used in the rendering of the image (the _IDL_ImageTexture base image texture and the lut texture), this is referred to as multi-texturing.

The GLSL texture2D procedure call reads the texel at the current texture coordinate. This procedure typically returns a floating point, four-element vector (containing red, green, blue and alpha values). But with a greyscale image, the red, green, and blue values are the same, so the appending .r keeps only the red channel and assigns it to the float i.

The GLSL texture1D procedure takes two parameters, the lut and i (the texture coordinate that instructs it which texel to sample). This value normally ranges from 0.0 to 1.0 (0.0 being the first texel, 1.0 the last). Since the value read from the image into i also normally ranges between 0.0 and 1.0, it is possible to use it directly as a texture coordinate to do the lookup.

When performing a lookup on the CPU, you directly access the LUT array using the pixel value as the index. A pixel value of 0 corresponds to the first entry in the LUT and a pixel value of 255 corresponds to the last entry.

However, in a shader program the texture coordinate lookup is possible because before a pixel reaches the fragment shader it is converted to floating point by OpenGL. In the case of an 8-bit greyscale image, the range is 0.0 to 1.0. That means a pixel with value 0 becomes 0.0 and 255 becomes 1.0. When doing the coordinate texture lookup on the GPU, the texture1D procedure does the lookup by using the converted pixel values where pixel value of 0 corresponds to the first LUT entry and a pixel value of 1.0 (converted from 255) corresponds to the last entry.

Assign LUT Shader Program to Shader Object

You need to supply the program code to the shader object so that it is available to the graphics card when it is needed. To accomplish this, you can use shader object properties VERTEX_PROGRAM_FILE and FRAGMENT_PROGRAM_FILE to associate external shader program components with the shader object.

Add the following code to the bottom of your Init function:

vertexFile=filepath('LUTShaderVert.txt', $  
   SUBDIRECTORY=['examples','doc', 'shaders'])  
fragmentFile=filepath('LUTShaderFrag.txt', $  
   SUBDIRECTORY=['examples','doc', 'shaders'])  
  
self->IDLgrShader::SetProperty, $  
   VERTEX_PROGRAM_FILENAME=vertexFile, $  
   FRAGMENT_PROGRAM_FILENAME=fragmentFile  

At this point, you can easily add image display code to your program and test your LUT shader. The result of applying one of IDL's pre-defined colortables appears in the following figure.

Figure 14-5: LUT Shader Example

Figure 14-5: LUT Shader Example

Software Fallback for the LUT Shader

The following code performs the LUT lookup. When there is not sufficient hardware support for shaders or when the FORCE_FILTER keyword is set on initialization, the colortables changes result from the following code instead of a shader program. You will likely find that performance slows significantly.

Function shader_lut_doc::Filter, Image  
  
; Allocate return array of same dimension and type.  
sz = SIZE(Image)  
newImage = FLTARR(sz[1:3], /NOZERO)  
  
; Get the LUT uniform variable.  
self->GetUniformVariable, 'lut', oLUT  
  
; Read the LUT data from the 1-D image.  
oLUT->GetProperty, DATA=lut  
FOR y=0, sz[3]-1 DO BEGIN  
    FOR x=0, sz[2]-1 DO BEGIN  
        ; Read from the image.  
        idr = Image[0,x,y]  
        ; Convert from 0.0-1.0 back to 0-255.  
        idr *= 255  
  
        ; Get the number of image channels.  
        szlut = SIZE(lut)  
        IF szlut[0] EQ 1 THEN BEGIN  
            ; Greyscale LUT, only 1 channel.  
            grey = lut[idr]  
            fgrey = FLOAT(grey) / 255.0  
            newImage[0,x,y] = fgrey  
            newImage[1,x,y] = fgrey  
            newImage[2,x,y] = fgrey  
            newImage[3,x,y] = 1.0  
        ENDIF ELSE BEGIN              
            ;; RGB LUT.  
            rgb = lut[*, idr]  
            frgb = FLOAT(rgb) / 255.0  
            newImage[0:2,x,y] = frgb  
            newImage[3,x,y] = 1.0              
        ENDELSE  
    ENDFOR  
ENDFOR  
RETURN, newImage  
END  

IDL always passes the image to the Filter method in RGBA floating-point pixel-interleaved format, so you don't have to worry about a lot of input data combinations. IDL also clamps the data this function returns to the [0.0, 1.0] range and scales it to the correct pixel range, usually [0, 255], for your display device.

At this point, you can test your work by writing a simple display program that loads your data into an IDLgrImage object, creates an instance of your shader_lut_doc object and attaches the LUT to your image object by setting the object reference of the shader in the SHADER property of IDLgrImage. You also need to set the FORCE_FILTER property on class initialization so that the filter fallback runs, even if you have shader hardware:

oLUTshader = OBJ_NEW('shader_lut_doc', /FORCE_FILTER)