Animating with large datasets

Hi Everyone!

I have been working with P5 for a while now, but in my current project I'm working with large data sets for the first time. The FPS I'm getting when animating over 2000+ data points is not good, is there a way to improve upon this?

Thanks!

Answers

  • We'd have to see your code in order to determine if any optimisation is possible...

  • Share your code!

    Some types of optimizations include:

    • Drawing into buffer(s) (not redrawing the contents every frame)
    • Prerendering objects as sprites (to not redraw them every frame)
    • Grouping multiple objects into flocks (to not redraw them every frame)
    • Separating dynamic foreground from static background (so as not to redraw it every frame)
    • Drawing some material at lower framerates than others by skipping updates of those things on certain frame counts / intervals (...so as not to redraw every frame).

    Of course, if you are doing a lot of math for each point / object, there might be ways to optimize the math itself -- for example pre-calculating expensive shared terms and then sharing them....

  • Hi thanks for your answers already, sorry I forgot to add my code.. This is the content of my draw loop:

    p5.clear();
    
    this.data.forEach((dataPoint) => {
      if (dataPoint.userType === 1) {
        p5.fill(255, 204, 0);
        p5.ellipse(dataPoint.x, dataPoint.y, 10);
      } else {
        p5.fill(64, 224, 208, 75);
        p5.ellipse(dataPoint.x, dataPoint.y, 7);
      }
    });
    

    this.data has about 2000 items in it.

  • Well you could try pre-rendering the two different styles of ellipse into sprites. Not sure how much that improves things in p5js; but in theory it should help (as it should avoid a lot of repeated calculations). Maybe using WebGL might help?

    I'm suspicious that there may be more to your code than you reveal here :P

    Presumably there's some code affecting the dataPoint x and y coordinates each frame?

  • edited November 2017

    Well, I've made an optimized sketch based on what little code you've posted above. >-)

    It pre-creates p5.Color instances via color() to be used w/ fill(). *-:)
    https://p5js.org/reference/#/p5.Color

    And skips invoking fill() if previous p5.Color is already the same as current 1. \m/

    You can also view it running online @:
    https://CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001

    And here's the full sample code: B-)

    index.html:

    <script defer src=https://CDN.JSDelivr.net/npm/p5></script>
    <script defer src=sketch.js></script>
    

    sketch.js:

    /**
     * ColorDataPoints (v1.0)
     * GoToLoop (2017-Nov-28)
     *
     * Forum.Processing.org/two/discussion/25225/
     * animating-with-large-datasets#Item_5
     *
     * CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001
     */
    
    "use strict";
    
    class ColorDiam {
      constructor(c=0, d=1) {
        this.c = c, this.d = d;
      }
    }
    
    new p5(p => {
      const diams = Array(2);
    
      let data = [
        { x: 10,  y: 15,  userType: 1 },
        { x: 100, y: 150, userType: 2 },
        { x: 400, y: 60,  userType: 3 },
        { x: 450, y: 260, userType: 1 },
        { x: 200, y: 300, userType: 1 }
      ];
    
      p.setup = function () {
        this.createCanvas(500, 400);
        this.noLoop();
        this.colorMode(this.RGB);
        this.stroke(0).strokeWeight(1.5);
    
        diams[0] = new ColorDiam(this.color(255, 204, 0), 10);
        diams[1] = new ColorDiam(this.color(64, 224, 208, 75), 7);
      };
    
      p.draw = function () {
        this.clear();
        let colour, diam;
    
        for (const d of data) {
          diam = d.userType === 1 && diams[0] || diams[1];
          colour !== diam.c && this.fill(colour = diam.c);
          this.ellipse(d.x, d.y, diam.d);
        }
      }.bind(p);
    });
    
  • Thanks for your suggestion, GoToLoop! Sadly, when I increase the amount of items in the data array it gets just as slow :(

  • There is no way to optimize secret code that hasn't been shared. :(

  • edited November 2017
    • My sample sketch still got room to be optimized even further.
    • For example, we can sort() the array so all userType w/ value 1 comes before the others:
      https://Developer.Mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
    • This way, just 2 calls to fill() are necessary when they're iterated over within the loop. *-:)
    • Another 1 is to simply split the array into 2 arrays.
    • 1 whose elements got userType equal to 1 and the other not equal to 1. :ar!
    • Then wrap those 2 split arrays in another array. Thus we've got a 2D array outta it. :>
    • And that's the approach I've picked for the version 2. :\">
    • This time, an extra util function called splitByType() creates 2 arrays and push() the objects from the original array to either 1 or the other array.
    • At the end, it returns both split arrays wrapped up in another array. Here it comes: :)>-

    https://CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001

    index.html:

    <script defer src=https://CDN.JSDelivr.net/npm/p5></script>
    <script defer src=sketch.js></script>
    

    sketch.js:

    /**
     * ColorDataPoints (v2.0)
     * GoToLoop (2017-Nov-28)
     *
     * Forum.Processing.org/two/discussion/25225/
     * animating-with-large-datasets#Item_8
     *
     * CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001
     */
    
    "use strict";
    
    class ColorDiam {
      constructor(c=0, d=1) {
        this.c = c, this.d = d;
      }
    }
    
    function splitByType(arr) {
      const split1 = [], split2 = [];
    
      for (const obj of arr)
        obj.userType === 1 && split1.push(obj) || split2.push(obj);
    
      return [split1, split2];
    }
    
    new p5(p => {
      const diams = Array(2);
    
      let data2d, data = [
        { x: 10,  y: 15,  userType: 1 },
        { x: 100, y: 150, userType: 2 },
        { x: 400, y: 60,  userType: 3 },
        { x: 450, y: 260, userType: 1 },
        { x: 200, y: 300, userType: 1 }
      ];
    
      p.setup = function () {
        this.createCanvas(500, 400);
        this.noLoop();
        this.colorMode(this.RGB);
        this.stroke(0).strokeWeight(1.5);
    
        diams[0] = new ColorDiam(this.color(255, 204, 0), 10);
        diams[1] = new ColorDiam(this.color(64, 224, 208, 75), 7);
    
        data2d = splitByType(data);
      };
    
      p.draw = function () {
        this.clear();
    
        for (let i = 0; i < diams.length; ++i) {
          const {c, d} = diams[i];
          this.fill(c);
          for (const {x, y} of data2d[i])  this.ellipse(x, y, d);
        }
      }.bind(p);
    });
    
Sign In or Register to comment.