2014-04-17

What do you mean by shaders? Learn how to create shaders with babylon.js

You may have noticed that we talked a lot about babylon.js during //Build 2014. If you do not, you can see the keynote for day 2 here and go directly to 2:24-2:28: https://channel9.msdn.com/Events/Build/2014/KEY02

Steven Guggenheimer and John Shewchuk demoed how the Oculus Rift support was added to Babylon.js. And one of the key thing for this demo was the work we did on a specific shader to simulate lenses as you can see in this picture:

I also presented a session with Frank Olivier and Ben Constable about graphics on IE and Babylon.js: https://channel9.msdn.com/Events/Build/2014/3-558

This leads me to one of the questions I often have about babylon.js: What do you mean by shaders???

So today I am going to try to explain you how shaders work.

You want to discuss about this article: reach me on Twitter: @deltakosh

Summary

The theory
Too hard? BABYLON.ShaderMaterial to the rescue
CYOS: Create Your Own Shader
Your shader?

The theory

Before starting experimenting, we must take a break and see how things work internally.

When dealing with hardware accelerated 3D, you must be aware of the fact that you will have to discuss with 2 CPU: the main CPU and the GPU. The GPU is a kind of extremely specialized CPU.

The GPU is a state machine that you set up using the CPU. For instance the CPU will configure the GPU to render lines instead of triangles. Or it will define that transparency is on and so on.

Once all the states are set, the CPU will define what to render (the geometry which is composed of a list of points (called the vertices and stored into an array called vertex buffer) and a list of indexes (the faces (or triangles) stored into an array called index buffer)).

The final step for the CPU is to define how to render the geometry and for this specific task, the CPU will define shaders for the GPU. Shaders are a piece of code that the GPU will execute for each of the vertices and pixels it has to render.

Some vocabulary: think of a vertex (vertices when there are several of them) as a “point” in a 3D environment as opposed to the point in a 2D environment.

There are two kinds of shaders: vertex shader and pixel (or fragment) shader.

Graphics pipeline

Before digging into shaders, let’s take a step back here. To render pixels the GPU will take the geometry defined by the CPU and will do the following:

Using the index buffer, three vertices are gathered to define a triangle: The index buffer contains a list of vertex indexes. This means that each entry in the index buffer is the number of a vertex in the vertex buffer. This is really useful to avoid duplicating vertices. For instance the following index buffer is a list of 2 faces: [1 2 3 1 3 4]. The first face contains vertex 1, vertex 2 and vertex 3. The second face contains vertex 1, vertex 3 and vertex 4. So there are 4 vertices in this geometry:

The vertex shader is applied on each vertex of the triangle. The primary goal of the vertex shader is to produce a pixel for each vertex (the projection on the 2D screen of the 3D vertex):

Using this 3 pixels (which define a 2d triangle on the screen), the GPU will interpolate all values attached to the pixel (at least its position) and the pixel shader will be applied on every pixel included into the 2d triangle in order to generate a color for every pixel:

This process is done for every face defined by the index buffer.

Obviously due to its parallel nature, the GPU is able to process this step for a lot of faces simultaneously and then achieve really good performance.

GLSL

We have just seen that to render triangles, the GPU needs two shaders: the vertex shader and the pixel shader. These shaders are written using a language called GLSL (Graphics Library Shader Language). It looks like C.

For Internet Explorer 11, we have developed a compiler to transform GLSL to HLSL (High Level Shader Language) which is the shader language of DirectX 11. This allows IE11 to ensure that the shader code is safe (You don’t want to reset your computer when using WebGL ):

Here is a sample of a common vertex shader:

precision highp float;

// Attributes
attribute vec3 position;
attribute vec2 uv;

// Uniforms
uniform mat4 worldViewProjection;

// Varying
varying vec2 vUV;

void main(void) {
    gl_Position = worldViewProjection * vec4(position, 1.0);

    vUV = uv;
}

Vertex shader structure

A vertex shader contains the following:

Attributes: An attribute defines a portion of a vertex. By default a vertex should at least contain a position (a vector3:x, y, z). But as a developer you can decide to add more information. For instance in the former shader, there is a vector2 named uv (Texture coordinates that allows to apply a 2D texture on an 3D object)
Uniforms: A uniform is a variable used by the shader and defined by the CPU. The only uniform we have here is a matrix used to project the position of the vertex (x, y, z) to the screen (x, y)
Varying: Varying variables are values created by the vertex shader and transmitted to the pixel shader. Here the vertex shader will transmit a vUV (a simple copy of uv) value to the pixel shader. This means that a pixel is defined here with a position and a texture coordinates. These values will be interpolated by the GPU and used by the pixel shader.
main: The function named main is the code executed by the GPU for each vertex and must at least produce a value for _gl_position_ (the position on the screen of the current vertex).

We can see in our sample that the vertex shader is pretty simple. It generates a system variable (starting with _gl__) named _gl_position_ to define the position of the associated pixel and it sets a varying variable called vUV.

The voodoo behind matrices

The thing in our shader is that we have a matrix named worldViewProjection and we use this matrix to project the vertex position to the gl_position variable. That is cool but how do we get the value of this matrix ? It is an uniform so we have to define it on the CPU side (using JavaScript).

This is one of the complex part of doing 3D. You must understand complex math (or you will have to use a 3D engine like babylon.js that we are going to see later).

The worldViewProjection matrix is the combination of 3 different matrices:

Using the resulting matrix allows us to be able to transform 3d vertices to 2d pixels while taking in account the point of view and everything related to the position/scale/rotation of the current object.

This is your responsibility as a 3D developer to create and keep this matrix up to date.

Back to the shaders

Once the vertex shader was executed on every vertex (3 times then) we have 3 pixels with a correct _gl_position_ and a vUV value. The GPU will then interpolate these values on every pixel contained into the triangle produced by these pixels

Then for each pixel, it will execute the pixel shader:

precision highp float;
varying vec2 vUV;
uniform sampler2D textureSampler;

void main(void) {
    gl_FragColor = texture2D(textureSampler, vUV);
}

Pixel (or fragment) shader structure

The structure of a pixel shader is similar to a vertex shader:

Varying: Varying variables are value created by the vertex shader and transmitted to the pixel shader. Here the pixel shader will receive a vUV value from the vertex shader
Uniforms: A uniform is a variable used by the shader and defined by the CPU. The only uniform we have here is a sampler which is a tool used to read texture colors
main: The function named main is the code executed by the GPU for each pixel and must at least produce a value for _gl_FragColor_ (The color of the current pixel).

This pixel shader is fairly simple: It reads the color from the texture using texture coordinates from the vertex shader (which in turn got it from the vertex).

To achieve this result, you will have to deal with a LOT of WebGL code. Indeed, WebGL is a really powerful but really low level API and you have to do everything by yourself from creating the buffers to defining vertex structures. You also have to do all the math and set all the states and handle texture loading and so on…

Too hard? BABYLON.ShaderMaterial to the rescue

I know what you are thinking: Shaders are really cool but I do not want to bother with WebGL internal plumbing or even with math.

And you are right! This is a perfectly legitim ask and that is exactly why I created Babylon.js.

Let me present you the code used by the previous rolling sphere demo. First of all you will need a simple webpage:

<!DOCTYPE html>
<html>
<head>
    <title>Babylon.js</title>
    <script src="Babylon.js"></script>

    <script type="application/vertexShader" id="vertexShaderCode">
        precision highp float;

        // Attributes
        attribute vec3 position;
        attribute vec2 uv;

        // Uniforms
        uniform mat4 worldViewProjection;

        // Normal
        varying vec2 vUV;

        void main(void) {
        gl_Position = worldViewProjection * vec4(position, 1.0);

        vUV = uv;
        }
    </script>

    <script type="application/fragmentShader" id="fragmentShaderCode">
        precision highp float;
        varying vec2 vUV;

        uniform sampler2D textureSampler;

        void main(void) {
        gl_FragColor = texture2D(textureSampler, vUV);
        }
    </script>

    <script src="index.js"></script>
    <style>
        html, body {
            width: 100%;
            height: 100%;
            padding: ;
            margin: ;
            overflow: hidden;
            margin: 0px;
            overflow: hidden;
        }

        #renderCanvas {
            width: 100%;
            height: 100%;
            touch-action: none;
            -ms-touch-action: none;
        }
    </style>
</head>
<body>
    <canvas id="renderCanvas"></canvas>
</body>
</html>

You can notice that the shaders are defined by

Eternalcoding

var life = new[] {"eat", "sleep", "code"}