Texture point-based billboard shader and strange uniform problem

capturevision

Hello, Following the code from this thread, and with the never failing suport of @codeanticode, I have the following sketch drawing points as billboard quads.

My goal is to texture these billboard quads with a semi transparent PNG, in order to render 'planets' (image attached below) :

First, here is my actual sketch :

PShader pointShader;
PVector [] positions;
color   [] colors;
PImage  billboardTexture;

PVector lightPos; 
PVector lightPosCamSpace;

float weight       = 100;    // billboard size.
int billboardCount = 1000;   // billboard count.


void setup() {
  size(1280, 720, P3D);

  // Init shaders and shader-related stuff..
  pointShader = loadShader("frag.glsl", "vert.glsl");

  lightPos = new PVector(0, 0, 300);
  lightPosCamSpace = new PVector();

  strokeWeight(weight);
  strokeCap(SQUARE);
  stroke(255);

 billboardTexture = loadImage("particle.png");

  // Generate billboards positions and colors.
  positions = new PVector[billboardCount];
  colors    = new color  [billboardCount];
  for (int i = 0; i < positions.length; i++) 
  {
    positions[i]  = new PVector(random(-width/2, +width/2), random(-height/2, +height/2), random(-1000, 0));
    colors[i]     = color(random(255), random(255), random(255)); 
  }

}


void draw() 
{  
  background(0);
  translate(width/2, height/2);  
  rotateZ(frameCount * 0.01);

  // Feed the shader.
  PMatrix modelview = getMatrix();
  modelview.mult(lightPos, lightPosCamSpace);
  pointShader.set("lightPos", lightPosCamSpace);

  //--> OFFENDING LINE : "the shader doesn't have a uniform called "billboardTex" !
  pointShader.set("billboardTex", billboardTexture); 


  // draw the point-based billboards..  
  shader(pointShader, POINTS);
  for (int i = 0; i < positions.length; i++) 
  {
    stroke(colors[i]);
    point(positions[i].x, positions[i].y, positions[i].z);  
  }
}

vertex shader :

uniform mat4 projection;
uniform mat4 modelview;
uniform mat3 normalMatrix;

uniform sampler2D billboardTex; 
uniform vec3 lightPos;

attribute vec4 vertex;
attribute vec4 color;
attribute vec2 offset;

varying vec4 vertColor;

void main() {
  vec4 pos = modelview * vertex;
  vec4 clip = projection * pos;
  gl_Position = clip + projection * vec4(offset, 0, 0);

  vec3 normal = vec3(0, 0, 1);
  vec3 ecVertex = pos.xyz + vec3(offset, 0);  
  vec3 ecNormal = normalize(normalMatrix * normal);  

  vec3 direction = normalize(lightPos.xyz - ecVertex);     
  float intensity = max(0.0, dot(direction, ecNormal));
  vertColor = vec4(intensity, intensity, intensity, 1) * color;  
}

fragment shader :

varying vec4 vertColor;

void main() {  
  gl_FragColor = vertColor;
  //gl_FragColor = texture2D(billboardTex, gl_PointCoord) * vertColor;
}

============

Question 1: Uniform is not found problem.

when I run the sketch, it runs but despite of the line pointShader.set("billboardTex", billboardTexture);, I get an error in red in the console stating that : "the shader doesn't have a uniform called "billboardTex". I don't understand that, as it is declared in the vertex shader..

Question 2 : How to modify the fragment shader to texture the billboard quad ?

As you can see, a commented line in the fragment shader says : //gl_FragColor = texture2D(billboardTex, gl_PointCoord) * vertColor;

I think, once question one is resolved, that this line should be correct in order to display the texture. Maybe I'm wrong. I'm a begginer in GLSL and need help on this one..

Thanks in advance, @codeanticode, @gotoloop, @poersch, @kosowski, and all the GLSL-fluent coders !

codeanticode

Hi, you should move the "uniform sampler2D billboardTex;" line to the fragment shader.

capturevision

Hi, Well.. I dit in my successive tries but I got the same message..

Here is now the code for the shaders (same error as before) :

Vertex Shader :

#version 140
uniform mat4 projection;
uniform mat4 modelview;
uniform mat3 normalMatrix;

uniform vec3 lightPos;

attribute vec4 vertex;
attribute vec4 color;
attribute vec2 offset;

varying vec4 vertColor;

void main() {
  vec4 pos      = modelview * vertex;
  vec4 clip     = projection * pos;
  gl_Position   = clip + projection * vec4(offset, 0, 0);

  vec3 normal   = vec3(0, 0, 1);
  vec3 ecVertex = pos.xyz + vec3(offset, 0);  
  vec3 ecNormal = normalize(normalMatrix * normal);  

  vec3 direction = normalize(lightPos.xyz - ecVertex);     
  float intensity = max(0.0, dot(direction, ecNormal));
  vertColor     = vec4(intensity, intensity, intensity, 1) * color;  
}

Fragment Shader :

#version 140

varying vec4 vertColor;
uniform sampler2D haloTexture;

void main() {  
  gl_FragColor = vertColor;
  //gl_FragColor = texture2D(texture, gl_PointCoord) * vertColor;
}

Also, once the 'missing uniform' problem solved, is the last line of the fragment shader correct in order to get the texture mixed with the color ?

codeanticode

if uniform is called "haloTexture", then you should have pointShader.set("haloTexture", billboardTexture) in the sketch code. And the fragment shader, should be something like:

varying vec4 vertColor;
varying vec2 texCoord;
uniform sampler2D haloTexture;

void main() {  
  gl_FragColor = vertColor;
  gl_FragColor = texture2D(haloTexture, texCoord) * vertColor;
}

However, one problem is that Processing doesn't send texture coordinate to the point shaders, so you need to calculate them manually. This is explained in the PShader tutorial (code listing 10.2, also on github here), the vertex shader in this case should calculate the textCoord varying along these lines:

uniform mat4 projection;
uniform mat4 modelview;
uniform mat3 normalMatrix;

uniform vec3 lightPos;

attribute vec4 vertex;
attribute vec4 color;
attribute vec2 offset;

varying vec4 vertColor;
varying vec2 texCoord;

void main() {
  vec4 pos      = modelview * vertex;
  vec4 clip     = projection * pos;
  gl_Position   = clip + projection * vec4(offset, 0, 0);

  texCoord = (vec2(0.5) + offset / length(offset));

  vec3 normal   = vec3(0, 0, 1);
  vec3 ecVertex = pos.xyz + vec3(offset, 0);  
  vec3 ecNormal = normalize(normalMatrix * normal);  

  vec3 direction = normalize(lightPos.xyz - ecVertex);     
  float intensity = max(0.0, dot(direction, ecNormal));
  vertColor     = vec4(intensity, intensity, intensity, 1) * color;  
}

Poersch

Quickly smashed together a working example using the custom point shader code referenced by @codeanticode:

PShader pointShader;
PVector[] positions;
color[] colors;
float weight = 100;
int billboardCount = 1000;


void setup() {

    size(1280, 720, P3D);

    String[] vertSource = new String[] {
        "#define PROCESSING_POINT_SHADER",
        "uniform mat4 projection;",
        "uniform mat4 modelview;",
        "uniform float weight;",
        "attribute vec4 vertex;",
        "attribute vec4 color;",
        "attribute vec2 offset;",
        "varying vec4 vertColor;",
        "varying vec2 texCoord;",
        "void main() {",
            "vec4 pos = modelview * vertex;",
            "vec4 clip = projection * pos;",
            "gl_Position = clip + projection * vec4(offset, 0, 0);",
            "texCoord = (vec2(0.5) + offset / weight);",
            "vertColor = color;",
        "}"
    };
    String[] fragSource = new String[] {
        "#ifdef GL_ES",
        "precision mediump float;",
        "precision mediump int;",
        "#endif",
        "uniform sampler2D sprite;",
        "varying vec4 vertColor;",
        "varying vec2 texCoord;",
        "void main() {",
            "gl_FragColor = texture2D(sprite, texCoord) * vertColor;",
        "}"
    };
    pointShader = new PShader(this, vertSource, fragSource);
    pointShader.set("weight", weight);
    pointShader.set("sprite", loadImage("https" + "://dl.dropboxusercontent.com/u/31893519/particle.png"));

    strokeWeight(weight);
    strokeCap(SQUARE);

    positions = new PVector[billboardCount];
    colors = new color[billboardCount];
    for(int i = 0; i < billboardCount; i++) {
        positions[i] = new PVector(random(-1000, 1000), random(-1000, 1000), random(-1000, 1000));
        colors[i] = (int)random(0xffffff) | 0xff000000; 
    }

}


void draw() {

    background(#000000);
    shader(pointShader, POINTS);
    blendMode(ADD);
    hint(DISABLE_DEPTH_TEST);

    translate(width / 2, height / 2);  
    rotate(frameCount * 0.01, 1.0, 0.75, 0.5);

    for(int i = 0; i < positions.length; i++) {
        stroke(colors[i]);
        point(positions[i].x, positions[i].y, positions[i].z);  
    }

}

Not entirely sure what your lighting code should achieve I dropped it.

capturevision

Hey, @codeanticode & @poersch : Thanks a lot to both of you for your patience and involvement ! Finally got it ! Cool. Though I'm a beginner in GLSL, I think I have noticed a strange behaviour (noticed several times).. Quite 'randomly' if I may say so, the code returned me a "the shader does'nt have a uniform called xxx' though I got shaders code in front of me.. It even happened radomly with other sketches.. Anyway, thanks to you both !

Poersch

No problem. The GLSL compiler is pretty smart (for compiler standards), it will drop not used variables and code (to some extend) to speed up the shader's execution. That means, in case you just define, but don't use a uniform or just use it in unreachable code blocks ( eg: if(false) { someVar += 0.1; } ), it will get dropped.

capturevision

Hey, good to know ;), thanks !

However, just a little question..

None of the billboards are 'occluded' by solid objects (i.e. you can allways see them, even behind an opaque object). ;(

I'm sure this has to do with depth/depthMask or blending, but I don't know how to implement it. I've tried method found on the first post of this page, and used gl.glDepthMask(false) before drawing billboards and gl.glDepthMask(true) afterwards but could'nt get it to work correctly..

My sketch is simple : draw a solid sphere in the center of the screen, calculate spherical coordinates for billboards, draw the sphere, and draw the billboards.

Code : (reduced as much as possible).

import javax.media.opengl.glu.*;
import javax.media.opengl.GL2;
import com.jogamp.opengl.util.gl2.GLUT;

PShader   pointShader;
PVector[] positions;
PVector[] positionsSphere;

color[]   colors;
int       billboardCount   = 200;
Sphere    mySphere;

GL2       gl;

void setup() 
{
    size(1280, 720, OPENGL);

    String[] vertSource = new String[] {
        "#define PROCESSING_POINT_SHADER",
        "uniform mat4 projection;",
        "uniform mat4 modelview;",
        "uniform float weight;",
        "attribute vec4 vertex;",
        "attribute vec4 color;",
        "attribute vec2 offset;",
        "varying vec4 vertColor;",
        "varying vec2 texCoord;",
        "void main() {",
            "vec4 pos = modelview * vertex;",
            "vec4 clip = projection * pos;",
            "gl_Position = clip + projection * vec4(offset, 0, 0);",
            "texCoord = (vec2(0.5) + offset / weight);",
            "vertColor = color;",
        "}"
    };
    String[] fragSource = new String[] {
        "#ifdef GL_ES",
        "precision mediump float;",
        "precision mediump int;",
        "#endif",
        "uniform sampler2D sprite;",
        "varying vec4 vertColor;",
        "varying vec2 texCoord;",
        "void main() {",
            "gl_FragColor = texture2D(sprite, texCoord) * vertColor;",
        "}"
    };

    mySphere = new Sphere(width / 2, height / 2, 0);
    for(int i = 0; i < billboardCount; i++)
    {
      mySphere.addSphereItem();
    } 

    pointShader = new PShader(this, vertSource, fragSource);
    pointShader.set("sprite", loadImage("https" + "://dl.dropboxusercontent.com/u/31893519/particle.png"));
    strokeCap(SQUARE);
}

void drawCenterSphere()
{  
    pushStyle();    
    sphereDetail(64);
    fill(255);
    stroke(0,20);
    strokeWeight(0.5f);    
    sphere(250);  
    popStyle();
}

void draw() 
{
    background(200);        
    lights();

    translate(mySphere.xPos, mySphere.yPos ,mySphere.zPos);
    rotate(frameCount * 0.01f, 1.0, 0.75, 0.5);

    drawCenterSphere();
    // render the billboards.    
    for (SphereItem item : mySphere.items)
    {      
      item.render();
    }
}

class Sphere {

  float xPos = 0;                   //X Position of the Sphere
  float yPos = 0;                   //Y Position of the Sphere
  float zPos = 0;                   //Z Position of the Sphere

  float radius = 250;                  //Radius of the Sphere
  ArrayList<SphereItem> items = new ArrayList();  //List of all of the items contained in the Sphere

  public Sphere(float posX, float posY, float posZ)
  {
    xPos = posX;
    yPos = posY;
    zPos = posZ;  
  }

  public void addSphereItem() 
  {
    SphereItem si = new SphereItem(this, random(1.3f ), random(TWO_PI * 2)); 
    items.add(items.size(), si);
  };

  public void render() 
  { 
    for (SphereItem item : items) 
    {      
      item.render();
    }       
  }
}



public class SphereItem 
{

  Sphere parentSphere;

  //Spherical Coordinates
  float radius; 
  float theta;
  float phi;

  //Speed properties
  float thetaSpeed   = 0;
  float phiSpeed     = 0;

  //Size
  float itemSize     = 5;

  int itemColor;

  public SphereItem(Sphere parent, float theta, float phi) 
  {
    parentSphere  = parent;

    this.theta   = theta;
    this.phi     = phi;

    itemSize     = random(5f);
    thetaSpeed   = random(-0.01, 0.01);
    phiSpeed     = random(-0.01, 0.01);
    itemColor    = color(random(255),random(255),random(255));   
  }

  public void update() 
  {
        theta  += thetaSpeed;
        phi    += phiSpeed;
  }

  public void render() 
  {
    float r = parentSphere.radius ;

    //Convert spherical coordinates into Cartesian coordinates
    float x = cos(theta) * sin(phi) * r;
    float y = sin(theta) * sin(phi) * r;
    float z = cos(phi) * r;

    update();
    renderAsBillboards (x, y, z); 

  }

  public void renderAsBillboards(float x, float y, float z)
  {
    float size = itemSize *20; 

    shader(pointShader, POINTS);    
    pointShader.set("weight", size);

    blendMode(BLEND);
    hint(DISABLE_DEPTH_TEST);    

    stroke(itemColor);
    strokeWeight(size);  

    point(x,y,z);

    hint(ENABLE_DEPTH_TEST);
    resetShader();

    blendMode(BLEND);

  }

}

The billboard rendering code is located in the SphereItem class at line 173..

Hm..Forgot to tell that the question is :

How to 'normally' occlude billboards so that they don't appear when drawn behind an opaque object ?

@poersch, @codeanticode, an idea ?

Thanks in advance.