Processing 1.0 - Processing Discourse - OpenGL Offscreen buffer for glow effect

moka

OpenGL Offscreen buffer for glow effect
Apr 2^nd, 2008, 8:20pm

HI,

In the past days I could not stop to think about how to get a real time glow/blur/shadow thing to work in Open GL using processing.

I read alot of interesting things about it like:
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=36

and came to the conclusion that the best way to do it would be to create a low res texture from the current frame and save it to an offscreen buffer, then blur it or blend it on top of the scene several times to get a blur/glow effect.

So my question is if processing supports any kind of offscreen buffer in direct opengl since I could not get any of the code to work from the link above.

I thought about using a PGraphics buffer but that would propably be really slow and I could forget about the real time idea.

Did anybody of you maybe try something like that allready? Or can I forget about that idea in processing?

taifunbrowser

Re: OpenGL Offscreen buffer for glow effect
Reply #1 - Apr 3^rd, 2008, 6:45am

You probably want codeanticode's PTexture libraries. Of course, if you know anything of opengl, you may be able to reproduce the entire library yourself.

But it may put you in the right direction, as it can "filter" video-card-buffered textures with a .xml shader format, which I'm pretty sure can be used for glow (I'm interested in what people do with this, actually)

Link: http://codeanticode.wordpress.com/2008/03/31/gltexture-library-for-processing/

moka Junior Member Offline Posts: 66	Re: OpenGL Offscreen buffer for glow effect Reply #2 - Apr 3^rd, 2008, 7:34am Thanks, I will give it a look!

moka Junior Member Offline Posts: 66	Re: OpenGL Offscreen buffer for glow effect Reply #3 - Apr 4^th, 2008, 7:28pm The library seems to be really nice. I realised a realtime blur using the pixels array to generate a texture and afterwards a shader.

taifunbrowser Junior Member Offline Posts: 81	Re: OpenGL Offscreen buffer for glow effect Reply #4 - Apr 6^th, 2008, 5:39am : heh : if you could post any code to this effect, I'd be really grateful. (trying to get similar cool-looking stuff for a game I'm hacking together) edit: especially the .xml files that the shaders read in: did you find a good tutorial that explains them or...?

ac

Re: OpenGL Offscreen buffer for glow effect
Reply #5 - Apr 9^th, 2008, 4:30am

Hi,

Here is a sample of an xml file:

<filter name="pulsating emboss">
<description>Emboss with pulsating grid</description>
<vertex>PulseGrid.glsl</vertex>
<fragment>DynEmboss.glsl</fragment>
<grid>
<resolution nx="10" ny="10"></resolution>
</grid>
</filter>

The three important tags are <vertex>, <fragment> and <grid>
In <vertex> and <fragment> you just specify the filename of the vertex and fragment glsl shaders. If no vertex shader is needed, just don't add a <vertex> tag.

The <grid> tag is used to define a rectangular grid the input texture is applied onto. The advantage of having a grid instead of just a rectangle is that you can modify the vertices of the grid inside the vertex shader, allowing for interesting distortions of the texture. However, if there is no need for such type of distortions, you can just skip the <grid> tag.

With <resolution nx="10" ny="10"></resolution> you just specify the number of points on the grid, along the x and y directions. In this case, the grid has 10x10 points.

So, if you don't need to change the vertex stage nor a grid, the xml file would reduce to something like this:

<filter name="gaussian blur">
<description>3x3 Gaussian blur filter</description>
<fragment>Blur.glsl</fragment>
</filter>

Now, the glsl shaders themselves have to follow certain conventions in order to be used inside a filter. These conventions are basically the way you are supposed to name the uniform parameters inside the shaders, for the filter to be able of recognizing those uniforms and link them to the parameters that can be passed from processing.

The naming conventions are the following:

1) the name of the source/input texture units should be "src_tex_unit" + n, where n = 0, 1, 2, etc. I.e., the first texture unit should be named src_tex_unit0, the second src_tex_unit1, and so on.

2) The offset of the source textures is identified by "src_tex_offset" + n, with n = 0, 1, 2, etc. The offset is, in other words, the inverse of the width and height of the texture, and represents the step size to move from one texel to the next.

At least one src_tex_unit0 uniform should be defined in the fragment shader, since a filter needs at least one source texture to operate on.

Then you can define the following optional uniforms to pass general parameters into the shader:

3) timing_data (of type vec2): the first component of this vector is the frameCount and the second is millis() at the time the shader program is executed. This allows to pass timing information into the shader.

4) par_flt1, par_flt2 and par_flt3 (of type float): these float uniforms can be used to pass float numbers into the shader.

5) par_mat2 (of type mat2), to pass a 2x2 matrix into the shader.

6) par_mat3 (of type mat3), to pass a 3x3 matrix into the shader.

7) par_mat4 (of type mat4), to pass a 4x4 matrix into the shader.

These uniforms don't need to be defined inside the shader, and also the shader can have other uniforms with different names. However, the advantage of having the uniforms listed here is that they are handled automatically inside the filter class. In order to set the value of these uniforms from processing, you use the GLTextureFilterParams object, inside which you have the parFlt1, parFlt2, parFlt3 float variables and parMat2, parMat3 and parMat4 float arrays. So, if the glsl shader that defines the filter has the float uniform par_flt1, you can set its value with the following code:

GLTextureFilterParams params = new GLTextureFilterParams();
params.parFlt1 = map(mouseX, 20, 640, 1, 30);
tex0.filter(pixelate, tex1, params);

I hope this helps. Please let me know if you have further questions.

moka

Re: OpenGL Offscreen buffer for glow effect
Reply #6 - Apr 22^nd, 2008, 10:45pm

I am currently still trying to work on the performance of my glow thing. I was wondering what would be the best way to modify the strength of the blur. I wrote a horizontal and vertical blur shader and call them through a for loop to get different strengths of my blur. But that really makes it go slow. Is there a better way?

Another question. Can I change the color, alpha values of a texture I want to render to the screen? that would give me the possibility to get a fake blur without shaders.

And do you know if you can use OpenGL Frame Buffer Object in Processing to make the offscreen rendering? I think it should work but I could not really get it to work completely right now.

ac

Re: OpenGL Offscreen buffer for glow effect
Reply #7 - Apr 22^nd, 2008, 11:56pm

I haven't implemented a glow/bloom effect myself yet, but it should work at reasonable framerates on a moderately new card (nvidia 6x00 or ati 1x00). But there are many things that could go wrong and affect the performance of the algorithm... I cannot really tell without looking at the code.

In any case, this guy made a nice implementation of an HDR algorith, where he uses also uses glow effect:

http://www.smetz.fr/?page_id=83

Check out the links at the bottom of the page.

Regarding the color of a texture and the alpha, you can call the glColor4f function before drawing the texture, and that would affect how the texture is drawn.

The FBO extension seem to working ok. At least in the GLTexture code I have no problem is using multiple FBOs.

moka

Re: OpenGL Offscreen buffer for glow effect
Reply #8 - Apr 23^rd, 2008, 12:13am

Okay, I will check that FBO stuff out.
Anyways I tried to call glColor4f before rendering the texture through tex0.renderTexture(0,0, width, height); but nothing seems to change.

To come back to the shader. Lets talk about your Gaussian Blur shader. What would you do if you want to blur the texture even more? Just call it once more? I have the same issue with other blur shaders. I cant really change the amount of blur values in my shader so I just have to call it over and over again to get a different result. Here is a sample shader for horizontal blur:

uniform sampler2D tex1;
float blur=0.0 ;
vec2 TexelKernelh[13];
float BlurWeights[13];
void main()
{
vec4 baseColor = vec4(0.0, 0.0, 0.0, 0.0);

float d = blur * 100.0 -300.0;
int i= 0;

TexelKernelh[0] = vec2( 0, -6 );
TexelKernelh[1] = vec2( 0, -5 );
TexelKernelh[2] = vec2( 0, -4 );
TexelKernelh[3] = vec2( 0, -3 );
TexelKernelh[4] = vec2( 0, -2 );
TexelKernelh[5] = vec2( 0, -1 );
TexelKernelh[6] = vec2( 0, 0 );
TexelKernelh[7] = vec2( 0, 1 );
TexelKernelh[8] = vec2( 0, 2 );
TexelKernelh[9] = vec2( 0, 3 );
TexelKernelh[10] = vec2( 0, 4 );
TexelKernelh[11] = vec2( 0, 5 );
TexelKernelh[12] = vec2( 0, 6 );

BlurWeights[0] = 0.002216;
BlurWeights[1] = 0.008764;
BlurWeights[2] = 0.026995;
BlurWeights[3] = 0.064759;
BlurWeights[4] = 0.120985;
BlurWeights[5] = 0.176033;
BlurWeights[6] = 0.199471;
BlurWeights[7] = 0.176033;
BlurWeights[8] = 0.120985;
BlurWeights[9] = 0.064759;
BlurWeights[10] = 0.026995;
BlurWeights[11] = 0.008764;
BlurWeights[12] = 0.002216;

for (int i = 0; i < 13; i++)

{

baseColor += texture2D( tex1, gl_TexCoord[0].xy + TexelKernelh[i].yx /d) * BlurWeights[i];

}

gl_FragColor = baseColor;
}

ac

Re: OpenGL Offscreen buffer for glow effect
Reply #9 - Apr 23^rd, 2008, 12:31am

To increase the amount of blur you need to increase the size of the kernel, which is equivalent to apply a small kernel many times:

http://en.wikipedia.org/wiki/Gaussian_blur

You can construct your own gaussian kernel by appplying the 2D formula using the desired sigma. Of course, the larger the kernel, the more expensive the calculation is, since you have to read more pixels around the center.

moka Junior Member Offline Posts: 66	Re: OpenGL Offscreen buffer for glow effect Reply #10 - Apr 23^rd, 2008, 12:40am okay, thanks!

moka Junior Member Offline Posts: 66	Re: OpenGL Offscreen buffer for glow effect Reply #11 - Apr 23^rd, 2008, 1:12am Could you post a small example of how to implement FBOs into processing? I read alot about it but could not really port it to processing on my own. I hope you dont mind :>

ac

Re: OpenGL Offscreen buffer for glow effect
Reply #12 - Apr 23^rd, 2008, 9:56am

Here you have an example of FBO usage to write to texture tex:

Code:


import processing.opengl.*;
import javax.media.opengl.*;

PGraphicsOpenGL pgl;
GL gl;

int[] tex = { 0 };
int[] fbo = { 0 };

void setup()
{
  size(640, 480, OPENGL);

  pgl = (PGraphicsOpenGL) g;
  gl = pgl.gl;

  // Create texture.
  gl.glGenTextures(1, tex, 0);
  gl.glBindTexture(GL.GL_TEXTURE_2D, tex[0]);

  gl.glTexParameteri(GL.GL_TEXTURE_2D, 
			   GL.GL_TEXTURE_MIN_FILTER,  
			   GL.GL_NEAREST);
  gl.glTexParameteri(GL.GL_TEXTURE_2D, 
			   GL.GL_TEXTURE_MAG_FILTER, 
			   GL.GL_NEAREST);
  gl.glTexParameteri(GL.GL_TEXTURE_2D, 
			   GL.GL_TEXTURE_WRAP_S, GL.GL_CLAMP);
  gl.glTexParameteri(GL.GL_TEXTURE_2D, 
			   GL.GL_TEXTURE_WRAP_T, GL.GL_CLAMP);
  gl.glTexImage2D(GL.GL_TEXTURE_2D, 0, 
			GL.GL_RGBA, 320, 240, 0, GL.GL_BGRA, 
			GL.GL_UNSIGNED_BYTE, null);

  // Create FBO and attach texture to it.
  gl.glGenFramebuffersEXT(1, fbo, 0);
  gl.glBindFramebufferEXT(GL.GL_FRAMEBUFFER_EXT, fbo[0]);
  gl.glFramebufferTexture2DEXT(GL.GL_FRAMEBUFFER_EXT,
					 GL.GL_COLOR_ATTACHMENT0_EXT,  
					 GL.GL_TEXTURE_2D, tex[0], 0);
  int stat = gl.glCheckFramebufferStatusEXT(GL.GL_FRAMEBUFFER_EXT);
 if (stat != GL.GL_FRAMEBUFFER_COMPLETE_EXT)
    System.out.println("FBO error");
  gl.glGenFramebuffersEXT(1, fbo, 0);
}

void draw()
{
  gl = pgl.beginGL();

  // Bind FBO, now everything is rendered into the texture.
  gl.glBindFramebufferEXT(GL.GL_FRAMEBUFFER_EXT, fbo[0]); 

  // Render stuff...

  // Unbind FBO.
  gl.glBindFramebufferEXT(GL.GL_FRAMEBUFFER_EXT, 0);

  pgl.endGL();
}

moka Junior Member Offline Posts: 66	Re: OpenGL Offscreen buffer for glow effect Reply #13 - Apr 23^rd, 2008, 12:56pm thank you so much!

moka

Re: OpenGL Offscreen buffer for glow effect
Reply #14 - Apr 23^rd, 2008, 7:32pm

If I render stuff into the FBO it tells me:
GL_ERROR at top endDraw(): 0506 UNKNOWN

if I render it after unbinding it, everything is fine.

import processing.opengl.*;
import javax.media.opengl.*;

PGraphicsOpenGL pgl;
GL gl;

int[] tex = {
0 };
int[] fbo = {
0 };

void setup()
{
size(640, 480, OPENGL);

pgl = (PGraphicsOpenGL) g;
gl = pgl.gl;

// Create texture.
gl.glGenTextures(1, tex, 0);
gl.glBindTexture(GL.GL_TEXTURE_2D, tex[0]);

gl.glTexParameteri(GL.GL_TEXTURE_2D,
GL.GL_TEXTURE_MIN_FILTER,
GL.GL_NEAREST);
gl.glTexParameteri(GL.GL_TEXTURE_2D,
GL.GL_TEXTURE_MAG_FILTER,
GL.GL_NEAREST);
gl.glTexParameteri(GL.GL_TEXTURE_2D,
GL.GL_TEXTURE_WRAP_S, GL.GL_CLAMP);
gl.glTexParameteri(GL.GL_TEXTURE_2D,
GL.GL_TEXTURE_WRAP_T, GL.GL_CLAMP);
gl.glTexImage2D(GL.GL_TEXTURE_2D, 0,
GL.GL_RGBA, 320, 240, 0, GL.GL_BGRA,
GL.GL_UNSIGNED_BYTE, null);

// Create FBO and attach texture to it.
gl.glGenFramebuffersEXT(1, fbo, 0);
gl.glBindFramebufferEXT(GL.GL_FRAMEBUFFER_EXT, fbo[0]);
gl.glFramebufferTexture2DEXT(GL.GL_FRAMEBUFFER_EXT,
GL.GL_COLOR_ATTACHMENT0_EXT,
GL.GL_TEXTURE_2D, tex[0], 0);
int stat = gl.glCheckFramebufferStatusEXT(GL.GL_FRAMEBUFFER_EXT);
if (stat != GL.GL_FRAMEBUFFER_COMPLETE_EXT)
System.out.println("FBO error");
gl.glGenFramebuffersEXT(1, fbo, 0);
}

void draw()
{
gl = pgl.beginGL();

// Bind FBO, now everything is rendered into the texture.
gl.glBindFramebufferEXT(GL.GL_FRAMEBUFFER_EXT, fbo[0]);

gl.glBegin(GL.GL_QUADS);
gl.glVertex2f(0,0);
gl.glVertex2f(0,200);
gl.glVertex2f(200,200);
gl.glVertex2f(200,0);
gl.glEnd();

// Unbind FBO.
gl.glBindFramebufferEXT(GL.GL_FRAMEBUFFER_EXT, 0);
pgl.endGL();
}