Performance of Filter()

I was thinking of better ways to make the previous frame's image disppear rather than starting each off with a blank by doing background(0); or something. I slapped a filter(BLUR); at the end to make each previous frame diminish smoothly but it literally brought my fps from 50-60 down to under 5... just that one line. Is that supposed to be right?

I'm using Processing 2.2.1 with P3D and fullscreen. What techniques do you use to do the above?

Tagged:

Answers

  • You should use a shader for the blur. By default it goes on the CPU. Look at examples > topics> shaders > sepBlur

    That should help a lot.

  • edited October 2015

    clankill3r is right, BLUR is just an int variable "pointing" to a CPU based filter. Use a hardware accelerated PShader instead.

    In case of gaussian blurring I'd recommend to use a two pass approach. This way you reduce the quadratic sample kernel (9 samples) to two linear sample kernels (2 * 3 samples) while getting the same effect:

    PShader blurH, blurV;
    
    void setup() {
        size(600, 600, P2D);
        initBlurH();
        initBlurV();
    }
    
    void draw() {
        rect(mouseX - 20, mouseY - 20, 40, 40);
        filter(blurH);
        filter(blurV);
    }
    
    void initBlurH() {
        String[] vertSource = {
            "uniform mat4 transform;",
    
            "attribute vec4 vertex;",
            "attribute vec2 texCoord;",
    
            "varying vec2 vertTexCoord;",
    
            "void main() {",
                "vertTexCoord = texCoord;",
                "gl_Position = transform * vertex;",
            "}"
        };
        String[] fragSource = {
            "uniform sampler2D texture;",
            "uniform vec2 texOffset;",
    
            "varying vec2 vertTexCoord;",
    
            "void main() {",
                "gl_FragColor  = 0.27901 * texture2D(texture, vec2(vertTexCoord.x - texOffset.x, vertTexCoord.y));",
                "gl_FragColor += 0.44198 * texture2D(texture, vertTexCoord);",
                "gl_FragColor += 0.27901 * texture2D(texture, vec2(vertTexCoord.x + texOffset.x, vertTexCoord.y));",
            "}"
        };
        blurH = new PShader(this, vertSource, fragSource);
    }
    
    void initBlurV() {
        String[] vertSource = {
            "uniform mat4 transform;",
    
            "attribute vec4 vertex;",
            "attribute vec2 texCoord;",
    
            "varying vec2 vertTexCoord;",
    
            "void main() {",
                "vertTexCoord = texCoord;",
                "gl_Position = transform * vertex;",
            "}"
        };
        String[] fragSource = {
            "uniform sampler2D texture;",
            "uniform vec2 texOffset;",
    
            "varying vec2 vertTexCoord;",
    
            "void main() {",
                "gl_FragColor  = 0.27901 * texture2D(texture, vec2(vertTexCoord.x, vertTexCoord.y - texOffset.y));",
                "gl_FragColor += 0.44198 * texture2D(texture, vertTexCoord);",
                "gl_FragColor += 0.27901 * texture2D(texture, vec2(vertTexCoord.x, vertTexCoord.y + texOffset.y));",
            "}"
        };
        blurV = new PShader(this, vertSource, fragSource);
    }
    

    In case of box blurring I'd stick to a single pass approach that samples between the actual pixels. This practically quadruples the actual sample count and almost creates the same visual effect as the gaussian blur:

    PShader blur;
    
    void setup() {
        size(600, 600, P2D);
        initBlur();
    }
    
    void draw() {
        rect(mouseX - 20, mouseY - 20, 40, 40);
        filter(blur);
    }
    
    void initBlur() {
        String[] vertSource = {
            "uniform mat4 transform;",
    
            "attribute vec4 vertex;",
            "attribute vec2 texCoord;",
    
            "varying vec2 vertTexCoord;",
    
            "void main() {",
                "vertTexCoord = vec2(texCoord.x, 1.0 - texCoord.y);",
                "gl_Position = transform * vertex;",
            "}"
        };
        String[] fragSource = {
            "uniform sampler2D texture;",
            "uniform vec2 texOffset;",
    
            "varying vec2 vertTexCoord;",
    
            "void main() {",
                "gl_FragColor  = texture2D(texture, vertTexCoord - 0.5 * texOffset);",
                "gl_FragColor += texture2D(texture, vertTexCoord - 0.5 * vec2(texOffset.x, -texOffset.y));",
                "gl_FragColor += texture2D(texture, vertTexCoord + 0.5 * vec2(texOffset.x, -texOffset.y));",
                "gl_FragColor += texture2D(texture, vertTexCoord + 0.5 * texOffset);",
                "gl_FragColor *= 0.25;",
            "}"
        };
        blur = new PShader(this, vertSource, fragSource);
    }
    

    While the gaussian blur is in general more "pleasing" to the eye, the optimized box blur is much faster and creates almost the same visual effect. I'd stick to the faster box blur.

    Edit: Changed the first line in the filter's main() method to gl_FragColor = something in case the graphics card driver isn't initializing the value correctly.

  • Great, I will try the box blur out :)

  • edited October 2015

    Thanks for your answers. But as someone who doesn't know much about filtering... How does one apply these examples to a sketch which which blurs the canvas? PImage theCanvas = get()

  • edited October 2015

    @clankill3r: Nice! Hope it works on your Mac. ;)

    @joySeeing: Just like in my code snippet, load the PShaders in setup() or some dedicated method called by setup(), render your scene in draw() and then call filter():

    // ...
    PShader myFilter; // Define some global var for the filter
    
    void setup() {
        // ...
        myFilter = new PShader(/* ... */); // Create/load the filter
    }
    
    void draw() {
        // ...
        filter(myFilter); // Use the filter on your main canvas
    }
    
  • @Poersch

    About the box blur. Why the 0.5? Why is it not 1. So you look at the next pixel. Now you look at half a pixel away. Is it so you can use the filter in such a little amount that it is hardly visible?

    Also changing the value to 4 for example and run the filter multiple times looks nice :)

    Screen Shot 2015-10-12 at 12.34.41 PM

  • @Poersch

    I did update it for my mac :) Using in and out and texture instead of texture2D in case your interested in writing my Mac compatible shaders :)

  • edited October 2015

    @clankill3r:

    In case of box blurring I'd stick to a single pass approach that samples between the actual pixels. This practically quadruples the actual sample count and almost creates the same visual effect as the gaussian blur

    The reason why I use "half offsets" is because it will force the graphics card to interpolate between 4 pixel values instead of using 1 (actually it always interpolates a little because of the low UV precision, so the performance is lost anyways). That means my second blur approach practically samples 16 pixels instead of just 4. Plus, using offsets equal to or greater than 1 will cause artifacts (at least if you use the filter in an accumulative manner) as you are not sampling your current pixel anymore. While this can look good and be exactly what you are looking for, it's not really a blur anymore.

    0.5 pixel offset:

    half-pixel-offset

    1.0 pixel offset:

    1-pixel-offset

    In my real life projects I always depend on at least GLSL version 1.30 (in/out/texture/etc.), but on the forums I always try to use GLSL version 1.10 because it is (by specification) supported by any OpenGL hardware - don't know why apple seems to ignore the official specs though.

  • Ah cool that is done really smart :)

    And if I learned one thing over the years, big companies != quality.

  • @clankill3r

    How do you re-write his examples for Mac exactly? That's why I was confused... was just getting video noise...

  • I only did the box blur.

    PShader box_blur;
    PImage img;
    
    void settings() {
      size(512, 512, P2D);
    }
    
    void setup() {
      box_blur = loadShader("box_blur.frag", "box_blur.vert");
      frameRate(999);
      img = loadImage("http://bellard.org/bpg/lena30.jpg");
    }
    
    
    void draw() {
      background(255);
    
      fill(255, 0, 0);
      ellipse(width/2, height/2, width, height);
    
      image(img, 0, 0);
    
      int n = (int) map(mouseX, 0, width, 0, 20);
      for (int i = 0; i < n; i++) {
        filter(box_blur);
      }
    
    
      surface.setTitle(""+frameRate);
    }
    

    vert:

    #version 410
    
    uniform mat4 transform;
    
    in vec4 vertex;
    in vec2 texCoord;
    
    out vec2 vertTexCoord;
    
    void main() {
        vertTexCoord = vec2(texCoord.x, 1.0 - texCoord.y);
        gl_Position = transform * vertex;
    }
    

    frag:

    #version 410
    
    uniform sampler2D texture;
    uniform vec2 texOffset;
    
    in vec2 vertTexCoord;
    
    out vec4 fragColor;
    
    void main() {
        fragColor += texture(texture, vertTexCoord - 0.5 * texOffset);
        fragColor += texture(texture, vertTexCoord - 0.5 * vec2(texOffset.x, -texOffset.y)); 
        fragColor += texture(texture, vertTexCoord + 0.5 * vec2(texOffset.x, -texOffset.y));
        fragColor += texture(texture, vertTexCoord + 0.5 * texOffset);
        fragColor *= 0.25;
    }
    
  • @clankill3r

    With a standard 2015 Macbook Pro with Intel Iris graphics card, I get Cannot compile vertex shader: ERROR: 0:1: '' : version '410' is not supported

    Thanks for your help.

  • Video noise..? Could you post your Code?

  • @Poersch

    Whenever I copy your code directly to my Macbook Pro Processing 2.2.1 I get this:

    Using @clankill3r 's Mac adjusted code gives me a different error.

  • Poersch had some code to find what GLSL version your drivers supported but I can't find it back. That might help. I have set it for 410 in my case since that's what I have and I'm not sure if the shaders I port work < 410.

  • Ok, well that's fine. Finally, one thing missing that blur.glsl doesn't do that BLUR does is make things eventually fade into nothingness (or the illusion of this anyawys). Any clever recommendations to make that happen?

  • Just to make clear: By fading into nothingness, you meant that each frame the previous blur result got blurred again right?

  • Probably. Such that (simple example), if you drew a picture of a square and did not draw it again on the next frame, it would become progressively harder to see the square outline as the blurring would continue.

  • edited October 2015

    @joySeeing: Looking at your screenshot it seems like your graphics card driver isn't initializing gl_FragColor with vec4(0.0). Updated my examples, please try them again.

  • @Poersch 4-o-clock?

    gl_FragCoord or did the update not get threw?

  • @clankill3r: Argh! These typos! :D (changed it)

  • @Poersch

    Thanks. And actually. I was playing around with the numbers and if I moved .5 (on second example) up to .99, everything begins to fade away quickly. Perfect!

    Thanks

  • Actually, I don't really know what I'm talking about. Not sure what I did to make it do this.. Hmm...

  • edited October 2015

    The filter blur is very slow. Do your own one direction at a time with a "window" of pixels. That way the the speed of 3 pixel blur is about the same as a 256 pixel blur. With the window of pixels, after its filled its just one on and one off and the sum in the middle remains constant. So instead of 256 operations on a 256 pixel blur, you're doing 3. The 256 pixel blur I do on a 1000 x 1000 image takes 34 ms.

  • @joySeeing: Yeah, that will modify the blur weighting but the image will never reach the average color (because of rounding errors) you'll need a bigger blur kernel for that. Try the 2 pass gaussian blur with a far biffer blur kernel.

    @shawnlau: Yeah, but a shader based blur is pretty fast. Shouldn't take more than a few thousand nanoseconds.

Sign In or Register to comment.