Billboard Shader

capturevision

Hey you math codes and shader writers, I would like to make a billboarding shader that would work with any camera (standard, Peasy, ProScene, custom..). To my (very little) knowledge, a "generic" shader could be written as it is all about matrix transforms..

Do anyone have a good starting point or example (in P5 2.1.1 if possible) code that I could use to get started ?

Thank in advance :)

codeanticode

Hi, one alternative could be to use a custom point shader to render your sprite billboards, since Processing basically treats points as screen-facing polygons (i.e.: billboards). Take a look at section 10 in the PShader tutorial. A disadvantage with this approach is that you would need to add your own uniforms to handle lights, in case you need lighting.

capturevision

@codeanticode : Yes, andres, thank you once again..

But, when you say

A disadvantage with this approach is that you would need to add your own uniforms to handle lights, in case you need lighting.

I don't quite get it because, referring to our previous discussion, there are already buit-in uniforms that, once correctly named, are automatically passed from Processing to the shader pipeline.. These uniforms contain light information, don't they ?

Secondly, despite the fact that I have very little knowledge of GLSL, couldn't just a billboarding vertex shader be implemented ?

I have been looking at P5 1.5.1 GlGraphics "Billboards" example and tried to port it but with no result for the moment..

codeanticode

Actually, I was wrong, you can define lighting uniforms in your point or line shader, and as long as you set the lights using pointLight(), etc., then the values will be copied to the shader.

What is tricky about billboards is that you need to apply a different matrix transformation to the center of the billboard and to the corner points. In the GLGraphics shader, you have

vec4 pos = gl_ModelViewMatrix * gl_Vertex;
pos.xy += objCorn.xy;

gl_Position = gl_ProjectionMatrix * pos;

where ObjCorn is an additional attribute you need per vertex to specify the coordinates of the billboard corner with respect to the center. If you look at the point sprite example in the PShader tutorial, there is essentially the same calculation:

vec4 pos = modelview * vertex;
vec4 clip = projection * pos;  
gl_Position = clip + projection * vec4(offset, 0, 0);

but notice that points need an additional attribute, which is called offset. This attribute is calculated automatically by Processing depending on the type of point (square or round).

When you are rendering normal fill geometry, Processing doesn't create any additional attribute to pass per-vertex offsets to the vertex stage.

capturevision

Thanks,

I will investigate that.

capturevision

@codeanticode , still I can not use the point-based billboard shader for my project, as I would like to be able to rotate the billboard around the z-axis for some kind of effect.. But, it seems "point-based" shaders are not real geometry and thus cannot be rotated..

First question : is there some kind of trick that I could use to rotate a "point-based" shader billboard ?

Second question : If i try to go with quads based billboards, as you mentioned if your previous comment, the work is to add additional attribute to each vertex that "specify the coordinates of the billboard corner with respect to the center"..

Would you have any kind of code or pseudo-code for either of these methods, as I have currently no clue as where to start ;=

Thanks in advance

codeanticode

First answer: The point-based billboards can be rotated as any other shape in Processing, as point shaders receive the current modelview and projection matrices. However, you have to set lighting uniforms manually, since the renderer doesn't pass any lighting information to the point (and line) shaders. See the following code:

PShader pointShader;
PVector[] positions;
color[] colors;

PVector lightPos; 
PVector lightPosCamSpace;

float weight = 100;

void setup() {
  size(640, 360, P3D);

  lightPos = new PVector(0, 0, 300);
  lightPosCamSpace = new PVector();

  pointShader = loadShader("frag.glsl", "vert.glsl");

  strokeWeight(weight);
  strokeCap(SQUARE);
  stroke(255);

  positions = new PVector[100];
  colors = new color[100];
  for (int i = 0; i < positions.length; i++) {
    positions[i] = new PVector(random(-width/2, +width/2), random(-height/2, +height/2), random(-1000, 0));
    colors[i] = color(random(255), random(255), random(255)); 
  }   
}

void draw() {  
  background(0);

  translate(width/2, height/2);  

  pushMatrix();
  rotateX(frameCount * 0.05);
  rotateY(frameCount * 0.05);

  PMatrix modelview = getMatrix();
  modelview.mult(lightPos, lightPosCamSpace);
  pointShader.set("lightPos", lightPosCamSpace);

  popMatrix();

  rotateZ(frameCount * 0.01);

  shader(pointShader, POINTS);
  for (int i = 0; i < positions.length; i++) {
    stroke(colors[i]);
    point(positions[i].x, positions[i].y, positions[i].z);
  }
}

and the corresponding shaders:

vert.glsl

uniform mat4 projection;
uniform mat4 modelview;
uniform mat3 normalMatrix;

uniform vec3 lightPos;

attribute vec4 vertex;
attribute vec4 color;
attribute vec2 offset;

varying vec4 vertColor;

void main() {
  vec4 pos = modelview * vertex;
  vec4 clip = projection * pos;
  gl_Position = clip + projection * vec4(offset, 0, 0);

  vec3 normal = vec3(0, 0, 1);
  vec3 ecVertex = pos.xyz + vec3(offset, 0);  
  vec3 ecNormal = normalize(normalMatrix * normal);  

  vec3 direction = normalize(lightPos.xyz - ecVertex);     
  float intensity = max(0.0, dot(direction, ecNormal));
  vertColor = vec4(intensity, intensity, intensity, 1) * color;  
}

frag.glsl

varying vec4 vertColor;

void main() {  
  gl_FragColor = vertColor;
}

Second answer: it is not possible to add custom vertex attributes to shape geometry drawn, so you would need to handle the VBOs manually using OpenGL.

capturevision

@codeanticode,

thanks again andres but when I try and run the sketch (p5 2.1.1), I get the following exception :

java.lang.RuntimeException : Cannot compile vertex shader:0(10); undefined variable "modelview" , and the same "undefined variable" error for "projection" and "normalMatrix"..

Any idea ?

codeanticode

sorry, for some reason the code listings for the vertex and fragment shaders were not showing the lines at the top, now should be fixed.

capturevision

@codeanticode.. hm..no problem ;) Recopied the code and now :

java.lang.RuntimeException : Cannot compile fragment shader:0(10); undefined variable "vertColor

which is strange because this varying is declared in both shaders...

Can you actually run the sketch under P5 2.1.1 ? Any thoughts ?

Thanks in advance for your time.

codeanticode

yes, I can run it on 2.1.1

Do you have the line "varying vec4 vertColor;" in your frag.glsl file?

capturevision

@codeanticode, sorry, dunno what went wrong, it's working now.. :-)

I'm allmost where I wanna be, thanks to your advices.. Thanks !

Two things :

1) when I run the sketch, the light source seems to be rotating around the scene's center.. and the billboards are lit accordingly.. is it normal ? can this be avoided ?

2) One of my goals with this, is to render a planet with a halo. The planet is a sphere, and I'd like the halo to be drawn 'around' the planet. I can now do that whith the code you gracefully supplied..

But, I would be able to rotate the halo billboard around its own center, in order to make the halo appear to rotate around the sphere..

Any ideas ? Once again, thanks a million times for your involvement and time !

codeanticode

1) yes, simply remove the rotateZ call if you don't want the billboards to rotate around the screen's center.

2) Use pushMatrix/popMatrix to apply transformations only to the billboard containing the halo so the rotation doesn't affect the sphere.

capturevision

@codeanticode, @gotoloop, @poersch and all the glsl masters ;) :

Thanks but I have no clue as to how apply a texture to a billboarded point.. Furthermore, the fragment shader does not take any texturing into account..

Do you know how I could apply a texture to a billboarded point by moifying the above code ?

(sorry for double posting but I could not find how to delete a comment)..