Animating with large datasets

CPG

Hi Everyone!

I have been working with P5 for a while now, but in my current project I'm working with large data sets for the first time. The FPS I'm getting when animating over 2000+ data points is not good, is there a way to improve upon this?

Thanks!

blindfish

We'd have to see your code in order to determine if any optimisation is possible...

jeremydouglass

Share your code!

Some types of optimizations include:

• Drawing into buffer(s) (not redrawing the contents every frame)
• Prerendering objects as sprites (to not redraw them every frame)
• Grouping multiple objects into flocks (to not redraw them every frame)
• Separating dynamic foreground from static background (so as not to redraw it every frame)
• Drawing some material at lower framerates than others by skipping updates of those things on certain frame counts / intervals (...so as not to redraw every frame).

Of course, if you are doing a lot of math for each point / object, there might be ways to optimize the math itself -- for example pre-calculating expensive shared terms and then sharing them....

CPG

Hi thanks for your answers already, sorry I forgot to add my code.. This is the content of my draw loop:

p5.clear();

this.data.forEach((dataPoint) => {
  if (dataPoint.userType === 1) {
    p5.fill(255, 204, 0);
    p5.ellipse(dataPoint.x, dataPoint.y, 10);
  } else {
    p5.fill(64, 224, 208, 75);
    p5.ellipse(dataPoint.x, dataPoint.y, 7);
  }
});

this.data has about 2000 items in it.

blindfish

Well you could try pre-rendering the two different styles of ellipse into sprites. Not sure how much that improves things in p5js; but in theory it should help (as it should avoid a lot of repeated calculations). Maybe using WebGL might help?

I'm suspicious that there may be more to your code than you reveal here :P

Presumably there's some code affecting the dataPoint x and y coordinates each frame?

GoToLoop

Well, I've made an optimized sketch based on what little code you've posted above. >-)

It pre-creates p5.Color instances via color() to be used w/ fill(). *-:)
https://p5js.org/reference/#/p5.Color

And skips invoking fill() if previous p5.Color is already the same as current 1. \m/

You can also view it running online @:
https://CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001

And here's the full sample code: B-)

index.html:

<script defer src=https://CDN.JSDelivr.net/npm/p5></script>
<script defer src=sketch.js></script>

sketch.js:

/**
 * ColorDataPoints (v1.0)
 * GoToLoop (2017-Nov-28)
 *
 * Forum.Processing.org/two/discussion/25225/
 * animating-with-large-datasets#Item_5
 *
 * CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001
 */

"use strict";

class ColorDiam {
  constructor(c=0, d=1) {
    this.c = c, this.d = d;
  }
}

new p5(p => {
  const diams = Array(2);

  let data = [
    { x: 10,  y: 15,  userType: 1 },
    { x: 100, y: 150, userType: 2 },
    { x: 400, y: 60,  userType: 3 },
    { x: 450, y: 260, userType: 1 },
    { x: 200, y: 300, userType: 1 }
  ];

  p.setup = function () {
    this.createCanvas(500, 400);
    this.noLoop();
    this.colorMode(this.RGB);
    this.stroke(0).strokeWeight(1.5);

    diams[0] = new ColorDiam(this.color(255, 204, 0), 10);
    diams[1] = new ColorDiam(this.color(64, 224, 208, 75), 7);
  };

  p.draw = function () {
    this.clear();
    let colour, diam;

    for (const d of data) {
      diam = d.userType === 1 && diams[0] || diams[1];
      colour !== diam.c && this.fill(colour = diam.c);
      this.ellipse(d.x, d.y, diam.d);
    }
  }.bind(p);
});

CPG

Thanks for your suggestion, GoToLoop! Sadly, when I increase the amount of items in the data array it gets just as slow :(

jeremydouglass

There is no way to optimize secret code that hasn't been shared. :(

GoToLoop

• My sample sketch still got room to be optimized even further.
• For example, we can sort() the array so all userType w/ value 1 comes before the others:
  https://Developer.Mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
• This way, just 2 calls to fill() are necessary when they're iterated over within the loop. *-:)
• Another 1 is to simply split the array into 2 arrays.
• 1 whose elements got userType equal to 1 and the other not equal to 1. :ar!
• Then wrap those 2 split arrays in another array. Thus we've got a 2D array outta it. :>
• And that's the approach I've picked for the version 2. :\">
• This time, an extra util function called splitByType() creates 2 arrays and push() the objects from the original array to either 1 or the other array.
• At the end, it returns both split arrays wrapped up in another array. Here it comes: :)>-

https://CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001

index.html:

<script defer src=https://CDN.JSDelivr.net/npm/p5></script>
<script defer src=sketch.js></script>

sketch.js:

/**
 * ColorDataPoints (v2.0)
 * GoToLoop (2017-Nov-28)
 *
 * Forum.Processing.org/two/discussion/25225/
 * animating-with-large-datasets#Item_8
 *
 * CodePen.io/GoSubRoutine/pen/aVRJQW/right?editors=001
 */

"use strict";

class ColorDiam {
  constructor(c=0, d=1) {
    this.c = c, this.d = d;
  }
}

function splitByType(arr) {
  const split1 = [], split2 = [];

  for (const obj of arr)
    obj.userType === 1 && split1.push(obj) || split2.push(obj);

  return [split1, split2];
}

new p5(p => {
  const diams = Array(2);

  let data2d, data = [
    { x: 10,  y: 15,  userType: 1 },
    { x: 100, y: 150, userType: 2 },
    { x: 400, y: 60,  userType: 3 },
    { x: 450, y: 260, userType: 1 },
    { x: 200, y: 300, userType: 1 }
  ];

  p.setup = function () {
    this.createCanvas(500, 400);
    this.noLoop();
    this.colorMode(this.RGB);
    this.stroke(0).strokeWeight(1.5);

    diams[0] = new ColorDiam(this.color(255, 204, 0), 10);
    diams[1] = new ColorDiam(this.color(64, 224, 208, 75), 7);

    data2d = splitByType(data);
  };

  p.draw = function () {
    this.clear();

    for (let i = 0; i < diams.length; ++i) {
      const {c, d} = diams[i];
      this.fill(c);
      for (const {x, y} of data2d[i])  this.ellipse(x, y, d);
    }
  }.bind(p);
});