Implementing a gooey effect with a shader

solub · April 2018

I'm trying to replicate a web design trick known as "gooey effect" (see it live here). It's a technique applying SVG filters on moving ellipses in order to get a blob-like motion. The process is rather simple:

apply a gaussian blur
increase the contrast of the alpha channel only

The combination of the two creates a blob effect

The last step (increasing the alpha channel contrast) is usually done through a "color matrix filter".

A color matrix is composed of 5 columns (RGBA + offset) and 4 rows.

The values in the first four columns are multiplied with the source red, green, blue, and alpha values respectively. The fifth column value is added (offset).

In CSS, increasing the alpha channel contrast is as simple as calling a SVG filter and specifying the contrast value (here 18):

<feColorMatrix in="blur" mode="matrix" values="1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 18 -7" result="goo" />

In Processing though, it seems to be a bit more complicated. I believe (I may be wrong) the only way to apply a color matrix filter is to create one in a shader. After a few tries I came up with these (very basic) vertex and fragment shaders for color rendering:

colorvert.glsl

uniform mat4 transform;
attribute vec4 position;
attribute vec4 color;
varying vec4 vertColor;

uniform vec4 o=vec4(0, 0, 0, -9); 
uniform lowp mat4 colorMatrix = mat4(1.0, 0.0, 0.0, 0.0, 
                                     0.0, 1.0, 0.0, 0.0, 
                                     0.0, 0.0, 1.0, 0.0, 
                                     0.0, 0.0, 0.0, 60.0);


void main() {
  gl_Position = transform * position; 
  vertColor = (color * colorMatrix) + o  ;
}

colorfrag.glsl

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

varying vec4 vertColor;

void main() {
  gl_FragColor = vertColor;
}

PROBLEM:

The color matrix is partially working: changing the RGB values do affect the colors but changing the alpha values don't !

When trying to combine the shader with a Gaussian filter, the drawn ellipse stays blurry even after I set the alpha channel contrast to 60 (like in the codepen example):

PShader colmat;

void setup() {
  size(200, 200, P2D);
  colmat = loadShader("colorfrag.glsl", "colorvert.glsl");
}

void draw() {
  background(100);
  shader(colmat);

  noStroke();
  fill(255, 30, 30);
  ellipse(width/2, height/2, 40, 40);
  filter(BLUR,6);
}

The same thing happens when I implement the color matrix within @cansik 's Gaussian blur shader (from the PostFX library). I can see the colors changing but not the alpha contrast:

blurFrag.glsl

/ Adapted from:
// <a href="http://callumhay.blogspot.com/2010/09/gaussian-blur-shader-glsl.html" target="_blank" rel="nofollow">http://callumhay.blogspot.com/2010/09/gaussian-blur-shader-glsl.html</a>;

#ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

#define PROCESSING_TEXTURE_SHADER


uniform sampler2D texture;

uniform vec4 o=vec4(0, 0, 0, 0); 
uniform lowp mat4 colorMatrix = mat4(1, 0.0, 0.0, 0.0, 
                                     0.0, 1, 0.0, 0.0, 
                                     0.0, 0.0, 1, 0.0, 
                                     0, 0.0, 0.0, 60.0); //Alpha contrast set to 60


varying vec2 center;

// The inverse of the texture dimensions along X and Y
uniform vec2 texOffset;

varying vec4 vertColor;
varying vec4 vertTexCoord;

uniform int blurSize;       
uniform int horizontalPass; // 0 or 1 to indicate vertical or horizontal pass
uniform float sigma;        // The sigma value for the gaussian function: higher value means more blur
                            // A good value for 9x9 is around 3 to 5
                            // A good value for 7x7 is around 2.5 to 4
                            // A good value for 5x5 is around 2 to 3.5
                            // ... play around with this based on what you need <span class="Emoticon Emoticon1"><span>:)</span></span>

const float pi = 3.14159265;

void main() {  
  float numBlurPixelsPerSide = float(blurSize / 2); 

  vec2 blurMultiplyVec = 0 < horizontalPass ? vec2(1.0, 0.0) : vec2(0.0, 1.0);

  // Incremental Gaussian Coefficent Calculation (See GPU Gems 3 pp. 877 - 889)
  vec3 incrementalGaussian;
  incrementalGaussian.x = 1.0 / (sqrt(2.0 * pi) * sigma);
  incrementalGaussian.y = exp(-0.5 / (sigma * sigma));
  incrementalGaussian.z = incrementalGaussian.y * incrementalGaussian.y;

  vec4 avgValue = vec4(0.0, 0.0, 0.0, 0.0);
  float coefficientSum = 0.0;

  // Take the central sample first...
  avgValue += texture2D(texture, vertTexCoord.st) * incrementalGaussian.x;
  coefficientSum += incrementalGaussian.x;
  incrementalGaussian.xy *= incrementalGaussian.yz;

  // Go through the remaining 8 vertical samples (4 on each side of the center)
  for (float i = 1.0; i <= numBlurPixelsPerSide; i++) { 
    avgValue += texture2D(texture, vertTexCoord.st - i * texOffset * 
                          blurMultiplyVec) * incrementalGaussian.x;         
    avgValue += texture2D(texture, vertTexCoord.st + i * texOffset * 
                          blurMultiplyVec) * incrementalGaussian.x;         
    coefficientSum += 2.0 * incrementalGaussian.x;
    incrementalGaussian.xy *= incrementalGaussian.yz;
  }
  gl_FragColor = (avgValue / coefficientSum )  * colorMatrix;
}

Questions:

Why does the alpha channel contrast not work ? How can I "enable" it ?
Is it possible to enable it in the blurFrag shader posted above ?
Do you know a better way to implement that gooey effect ?

Any help would be much appreciated !

Thank you

jeremydouglass · April 2018

In Processing though, it seems to be a bit more complicated. I believe (I may be wrong) the only way to apply a color matrix filter is to create one in a shader.

If I am understanding right, you have blurred in a way that creates a variable alpha channel mask (rather than changing the rgb values), and now you want create a hard threshold on your alpha mask by multiplying the alpha() of each pixel by 18, then subtract 7. Is that right?

A shader is not the only way to manipulate color information -- you can use red(), green(), blue(), alpha(), or use bit fiddling and shifting of the ARGB color information

There is nothing wrong with a shader approach -- it can be really fast -- but perhaps try a simpler implementation first to get it working and tested before moving it into a shader?

Using a matrix is just one way of doing pixel manipulation -- and probably not even a particularly efficient way, as you are manipulating RGB by identity (x*1), so you are ignoring almost the entire matrix.

A * 18 - 7 is also not the most intuitive way of filtering your alpha channel, and it seems tuned. For a simple test, consider a two part rule:

if (A > threshold) {
  A = 255;
}
else {
  A = 0;
}

Okay, now loop through the pixels array with a for loop and set the alter each pixel alpha based on this rule. Something like this (not tested with an alpha image):

PImage blurred;

void setup() {
  size(600, 400);
  blurred = loadImage("blurred.png");
}

void draw() {
  background(0);
  blurred.loadPixels();
  for (int i=0; i < blurred.width * blurred.height; i++) {
    int a = (blurred.pixels[i] >> 24) & 0xFF;
    if ( a > 128) {
      blurred.pixels[i] = blurred.pixels[i] & 0xffffff | (0 << 24);
    } else {
      blurred.pixels[i] = blurred.pixels[i] & 0xffffff | (255 << 24);
    }
  }
  blurred.updatePixels();
  image(blurred, 0, 0);
}

solub · April 2018

Hi Jeremy, thank you for taking the time to answer my question.

...now you want create a hard threshold on your alpha mask by multiplying the alpha() of each pixel by 18, then subtract 7. Is that right?

It is right: multiplying by 19 to 60 (alpha contrast), then subtracting 9 (alpha offset).

A shader is not the only way to manipulate color information -- you can use red(), green(), blue(), alpha(), or use bit fiddling and shifting of the ARGB color information

I know, that's exactly what I tried previously (see our last discussion). Overall, I think this approach (loadPixels() / updatePixels()) is not appropriate when dealing with multiple moving objects, even with bit-shifting (good for pictures though).

Based on the example you provided, my sketch runs at 5 fps with 5 bouncing balls and for some reason they do not "melt" when colliding (no gooey effect).

With a shader, the same sketch runs at 60 fps with 700 bouncing balls and I get that metaball sort of effect. (Not exactly the one I'm looking for, obviously, but something quite close I came up with by fiddling with the color matrix).

test with 100 balls

I really believe the solution lies in a shader implementation. Let's hope someone can help me find a way to play with that alpha contrast channel in GLSL.

jeremydouglass · April 2018

OK! Possibly @nabr or @noahbuddy might have suggestions for alpha contrast in GLSL...?

noahbuddy · April 2018

To preserve transparency, with or without shaders, use an offscreen buffer (PGraphics). For example, saving a PNG image with transparent background.

I removed the contrast matrix from @cansik 's blur shader and instead put it into a separate filter.

In the blur shader, change the assignment to:

gl_FragColor = (avgValue / coefficientSum );

I think this is close...

pde:

PShader contrast, blurry;
PGraphics buf;

void setup() {
  size(200, 200, P2D);
  buf = createGraphics(width, height, P2D);

  contrast = loadShader("colfrag.glsl");
  blurry = loadShader("blurFrag.glsl");

  // Don't forget to set these
  blurry.set("sigma", 4.5);
  blurry.set("blurSize", 9);
}

void draw() {
  background(100);

  buf.beginDraw();
    // Reset transparency
    // Note, the color used here will affect your edges
    // even with zero for alpha
    buf.background(100, 0); // set to match main background

    buf.noStroke();
    buf.fill(255, 30, 30);
    buf.ellipse(width/2, height/2, 40, 40);
    buf.ellipse(mouseX, mouseY, 40, 40);

    blurry.set("horizontalPass", 1);
    buf.filter(blurry);
    blurry.set("horizontalPass", 0);
    buf.filter(blurry);
  buf.endDraw();

  shader(contrast);
  image(buf, 0,0, width,height);
}

colfrag:

#define PROCESSING_TEXTURE_SHADER

uniform sampler2D texture;
varying vec4 vertTexCoord;

uniform vec4 o = vec4(0, 0, 0, -7.0); 
uniform lowp mat4 colorMatrix = mat4(1.0, 0.0, 0.0, 0.0, 
                                     0.0, 1.0, 0.0, 0.0, 
                                     0.0, 0.0, 1.0, 0.0, 
                                     0.0, 0.0, 0.0, 18.0);

void main() {
  vec4 pix = texture2D(texture, vertTexCoord.st);

  vec4 color = (pix * colorMatrix) + o;
  gl_FragColor = color;
}

solub · April 2018

@noahbuddy This is fantastic ! The result is amazingly close to what I was hoping to achieve. Out of the various ways to code 2d metaballs I believe this is the most efficient one. I'm truely grateful for your help, you literally made my day.

@jeremydouglass Thank you !

BGADII · April 2018

Just a quick question, could someone explain what the purpose of colorfrag.glsl is and how this works? The variable declarations are really confusing. It would be really cool if someone could do a step by step tutorial explaining how this shader was made. This is a really basic but a great shading example and I think that if I fully understood what is going on here I could apply it to other things. Would make a great intro to shaders in processing because it looks like it could be pretty flexible in terms of rendering metaballs.

GoToLoop · April 2018

GLSL: https://en.Wikipedia.org/wiki/OpenGL_Shading_Language

BGADII · April 2018

Maybe I should have been more clear. My question is specific to this application, not GLSL in general. I've been looking at GLSL on shadertoy and the syntax is quite different especially at the top of the files when things are declared and initialized.

solub · April 2018

Hi @BGADII, I'm new to shaders as well so I can't go into much details:

#define PROCESSING_TEXTURE_SHADER

... is a "pre-processor" directive, it is used to explicitly set the shader type. I've read somewhere they were required up to version 2.1 but are no longer necessary in Processing 3.

uniform sampler2D texture;
varying vec4 vertTexCoord;

Unlike varying the uniform keyword indicates that texture is a constant variable (stays the same). Sampler2D is the variable type, indicating that it holds a 2D texture.

vertTexCoord holds the texture coordinates values.

uniform vec4 o = vec4(0, 0, 0, -7.0); 
uniform lowp mat4 colorMatrix = mat4(1.0, 0.0, 0.0, 0.0, 
                                     0.0, 1.0, 0.0, 0.0, 
                                     0.0, 0.0, 1.0, 0.0, 
                                     0.0, 0.0, 0.0, 18.0);

Instantiating the offset (vec4 o) and the colorMatrix (for further details read the first post)

vec4 pix = texture2D(texture, vertTexCoord.st);
vec4 color = (pix * colorMatrix) + o;

The original RGB pixels values are muliplied by 1 (no changes) and the alpha channel by 18 (increases the contrast). The offest is then added to the whole (negative alpha here to reduce the blur).

solub · April 2018

Personally I think the sweet spot lies somewhere:

between 8 and 11 for the alpha contrast

between -7 and -9 for the alpha offset

uniform vec4 o = vec4(0, 0, 0, -9.0); 
uniform lowp mat4 colorMatrix = mat4(1.0, 0.0, 0.0, 0.0, 
                                     0.0, 1.0, 0.0, 0.0, 
                                     0.0, 0.0, 1.0, 0.0, 
                                     1.0, 1.0, 1.0, 11.0);

bewteen 10 and 15 for "sigma"

bewteen 30 and 40 for "blurSize"

blurry.set("sigma", 14.5)
blurry.set("blurSize", 35)

Howdy, Stranger!

Categories

In this Discussion

Implementing a gooey effect with a shader

Best Answer

Answers