(How) to use color() in code for a Processing library?

jeremydouglass

I'm looking at wrapping some code in a library, and original sketch code uses color() -- and the color datatype.

My question: how do you approach creating, passing around, and manipulating color in Processing library code, given that color is really an int under-the-hood, and that color() is a method of PApplet. Is it possible (or desirable) to reference the color pseudo-datatype at all in a Processing library? (it throws an error)

How should color methods such as color be called via PApplet (papplet.color()), or in some other way, or not at all.

For the sake of discussion, here is a silly PDE sketch that passes around and manipulates color with some declaration, assignment, and arguments.

color randC;
Colorer randRed;
void setup(){
  size(200,200);
  randC = color(255);
  randRed = new Colorer(randC);
  frameRate(2);
}
void draw(){
  background(randRed.r());
}
class Colorer  {
  color c;
  Colorer (color c_){
    c = c_;
  }
  color r(){
    return color(random(255),green(c),blue(c));
  }
}

How would something like this become library code?

GoToLoop

Best way to learn is hitting CTRL+SHIFT+E and check out how all the ".pde" files become 1 ".java" file.
It's pretty easy to spot that all color "datatypes" are transpiled as int.
For library development, be careful w/ function color(). It's incompatible w/ multi-threading!
Also fill(), stroke() & background() invokes color() internally. :-SS

jeremydouglass

Thanks @GoToLoop. If I'm understanding right:

One should Aadapt PDE color code to library code by:

1. for color, replace the color type with int
2. for color(), pass in PApplet pa and call pa.color() as with normal top-level Processing methods
3. ... unless multithreading, in which case color() is unsafe.

Right?

public class Colorer  {
  PApplet pa;
  int c;
  Colorer (PApplet pa_, int c_){
    c = c_;
    pa = pa_;
  }
  int r(){
    return pa.color(random(255),green(c),blue(c));
  }
}

GoToLoop

That's it! Just take notice though that random() is also a non-static PApplet method! :-&
Same for green() & blue(), which are as unsafe as color():
return pa.color(pa.random(255), pa.green(c), pa.blue(c));
BtW, I'd prefer p instead of pa, given it's gonna be typed in a lot, and 1 letter helps indeed: #:-S
return p.color(p.random(255), p.green(c), p.blue(c));

Lord_of_the_Galaxy

I'm a little late over here, but I think it is fairly easy to write the code for color() function. Am I missing something here?
Could you also tell me why it is unsafe while multithreading?I never knew that. @GoToLoop My personal opinion is that pa would be better than p because using a single letter may lead to confusion if you need that letter for some other variable. pa would be a lot less likely to be required. Also, most IDEs and even many advanced text editors will give you easy to use predictions and that means you could just use a full name like pApplet or parent.

GoToLoop

My personal opinion is that pa would be better than p...

Using just p to refer to the sketch's instance is pretty much standard within p5.js' circle: :bz

1. https://GitHub.com/processing/p5.js/wiki/p5.js-overview#instantiation--namespace
2. https://GitHub.com/processing/p5.js/wiki/Instantiation-Cases

GoToLoop

Could you also tell me why it is unsafe while multithreading?

Not only unsafe, but a train wreck!!! :-SS
Take a look at what happens when we call color(). Be aware that everything color-related like fill(), stroke(), background(), etc. depend on it: L-)

https://GitHub.com/processing/processing/blob/master/core/src/processing/core/PGraphics.java#L7750

public final int color(float v1, float v2, float v3, float a) {
  colorCalc(v1, v2, v3, a);
  return calcColor;
}

Notice it invokes an internal method called colorCalc().
However, it returns an unknown field called calcColor instead! :-/
And to make matters worse, it's declared final. Therefore we can't change it! ~X(

Now let's take a look what that colorCalc() is all about: (:|

https://GitHub.com/processing/processing/blob/master/core/src/processing/core/PGraphics.java#L7571

protected void colorCalc(float x, float y, float z, float a) {
  if (x > colorModeX) x = colorModeX;
  if (y > colorModeY) y = colorModeY;
  if (z > colorModeZ) z = colorModeZ;
  if (a > colorModeA) a = colorModeA;

  if (x < 0) x = 0;
  if (y < 0) y = 0;
  if (z < 0) z = 0;
  if (a < 0) a = 0;

  switch (colorMode) {
  case RGB:
    if (colorModeScale) {
      calcR = x / colorModeX;
      calcG = y / colorModeY;
      calcB = z / colorModeZ;
      calcA = a / colorModeA;
    } else {
      calcR = x; calcG = y; calcB = z; calcA = a;
    }
    break;

  case HSB:
    x /= colorModeX; // h
    y /= colorModeY; // s
    z /= colorModeZ; // b

    calcA = colorModeScale ? (a/colorModeA) : a;

    if (y == 0) {  // saturation == 0
      calcR = calcG = calcB = z;

    } else {
      float which = (x - (int)x) * 6.0f;
      float f = which - (int)which;
      float p = z * (1.0f - y);
      float q = z * (1.0f - y * f);
      float t = z * (1.0f - (y * (1.0f - f)));

      switch ((int)which) {
      case 0: calcR = z; calcG = t; calcB = p; break;
      case 1: calcR = q; calcG = z; calcB = p; break;
      case 2: calcR = p; calcG = z; calcB = t; break;
      case 3: calcR = p; calcG = q; calcB = z; break;
      case 4: calcR = t; calcG = p; calcB = z; break;
      case 5: calcR = z; calcG = p; calcB = q; break;
      }
    }
    break;
  }
  calcRi = (int)(255*calcR); calcGi = (int)(255*calcG);
  calcBi = (int)(255*calcB); calcAi = (int)(255*calcA);
  calcColor = (calcAi << 24) | (calcRi << 16) | (calcGi << 8) | calcBi;
  calcAlpha = (calcAi != 255);
}

Pay attention there are lotsa assignments to unknown non-local variables almost everywhere: ~:>

---

switch (colorMode) {
case RGB:
  if (colorModeScale) {
    calcR = x / colorModeX;
    calcG = y / colorModeY;
    calcB = z / colorModeZ;
    calcA = a / colorModeA;
  } else {
    calcR = x; calcG = y; calcB = z; calcA = a;
  }
  break;

---

calcRi = (int)(255*calcR); calcGi = (int)(255*calcG);
calcBi = (int)(255*calcB); calcAi = (int)(255*calcA);
calcColor = (calcAi << 24) | (calcRi << 16) | (calcGi << 8) | calcBi;
calcAlpha = (calcAi != 255);

---

"But where do all those fields come from?" we'd inquire: :-??

https://GitHub.com/processing/processing/blob/master/core/src/processing/core/PGraphics.java#L550

// internal color for setting/calculating
protected float calcR, calcG, calcB, calcA;
protected int calcRi, calcGi, calcBi, calcAi;
protected int calcColor;
protected boolean calcAlpha;

As we can dreadfully witness, the way Processing calculates a color is more like some Frankenstein ragdoll! :-q

Just imagine 2 or more threads calling any color-related function at the same time.
All those partial calculations are assigned to myriads of fields.
The final result (generally calcColor or calcAlpha) is pretty sure to go bonks! :O)

jeremydouglass

@GoToLoop -- I don't know much about Java thread safety, but how difficult would refactoring for thread safety best practices be in this case? (folding colorCalc into color, internalizing that stack of private call variables, etc.). Could it be submitted as an issue and a GitHub pull request? Or is the problem that dozens of interlocked color functions all share these non-thread-safe methods and variables, so they all need to be updated at once?

GoToLoop

Could it be submitted as an issue and a GitHub pull request?

I doubt they'd bother about it! They consider threading too advanced for the project.
Even though they've recently "revealed" the existing of method thread().

... but how difficult would refactoring for thread safety best practices be in this case?

I guess I'd encapsulate all of those 10 "internal" fields as a nested static class.
Let's just call it ColorCalc. Now instead of the current void:
protected void colorCalc(float x, float y, float z, float a) { }
It'd return: protected ColorCalc colorCalc(float x, float y, float z, float a) { }

Now method color() would be something like this: *-:)

public final int color(float v1, float v2, float v3, float a) {
  return colorCalc(v1, v2, v3, a).calcColor;
}

Instead of:

public final int color(float v1, float v2, float v3, float a) {
  colorCalc(v1, v2, v3, a);
  return calcColor;
}

Problem solved! color() and other related methods are now thread-safe! \m/

jeremydouglass

Well, I can see why refactoring all of Processing to make it thread-safe wouldn't be a project priority if most Processing users will never use threads. AWT isn't thread-safe either, right?

But that doesn't mean a patch would be rejected if it makes the code more robust without introducing unnecessary complexity or maintenance overhead. Why not improve color?

I will do a fork and submit a pull request with your proposed changes if you are willing to do a code review on them.

GoToLoop

But that doesn't mean a patch would be rejected if...

It's just it's almost impossible mission to convince the project's owners to allow any modification to internal stuff! But you can take the risk of course. :-j

Processing users will never use threads.

That's not my experience as an old answerer in this forum!
Almost every month there are questions about multi-window w/ runSketch()!
And every library which accepts xxxEvent() style callback functions relies on thread.
selectFolder() & selectInput() idem! :-\"

AWT isn't thread-safe either, right?

Dunno the answer. However, renderer JAVA2D, which is AWT-based, is the only 1 we can instantiate more than once w/ runSketch()! $-)

... if you are willing to do a code review on them.

Well, not well-versed on color calculation, but I'll try to review it for sure. ~O)

quark

Going back the original question I never ever use the colour type. In fact it isn't a data type at all it is simply an alias for int and Processing's preprocessor replaces colour with int. This is why it throws an error in Eclipse.

I have created 9 libraries and 2 tools for Processing and most of those have used the color() method without any concern about it being thread safe. In fact the vast majority of Java classes are not thread safe. Instead Java provides other classes and mechanisms for the software developers to make their programs thread safe.

PS2 changed the select??? methods from synchronous to asynchronous operation to prevent blocking in the main thread. Although multi threading a simple semaphore technique can be used to prevent any problems.

If you are creating multiple windows with runSketch there is no issue with using color because you have multiple PApplet objects each with their own color method running synchronously in their own thread. (it works fine in G4P)

GoToLoop

... and most of those have used the color() method without any concern about it being thread safe.

AFAIK, most of your libraries aren't threaded either.
Rather they synchronously hook to Processing's callbacks via registerMethod(). :>
Problem arises when the library itself is threaded. 8-X

If you are creating multiple windows with runSketch() there is no issue with using color() because you have multiple PApplet objects each with their own color() method running synchronously in their own Thread.

That's correct! As long the programmer knows how to call color() and other related methods under the desired target PApplet instance.

quark

... and most of those have used the color() method without any concern about it being thread safe.

You are right in that most of my libraries aren't threaded so this statement needs clarification. All of my libraries have been designed to run in the main event (animation) thread, therore I took no special precautions should the user decide to run the library in a separate thread. But then as I said the majority of the Java API is not thread safe.

To make a library thread safe would
1) Significantly increase the development effort
2) Significantly increase the maintenance cycle
3) Significantly reduce the library's run-time efficiency.

Over the last 8.5 years thousands of people have used my libraries and AFAIK not one of them have used them in a separate thread so why increase my workload and reduce the software's efficiency for some non-existent user.

The same argument applies to Processing and to Java. If the user wants separate threads then the solution is to use Java's concurrent access techniques. In fact my QScript library uses separate threads to evaluate time consuming algorithms/ expressions and avoids concurrent access issues by using Java's java.utils.concurrent API :)

Although Processing using multiple threads internally (generally for event handling) there are 2 areas which have generally caused issues for the user
- the select??? methods that I mentioned earlier and
- the thread() method "...a quick and dirty way to implement a simple thread"

A third area is runSketch() which can be used to create multiple windows. AFAIK this is not intended for general use, I suspect that the only reason the method is public is to simplify access to parts of the core code in other packages. It is also complicated to use and many queries have been from people who use it to create multiple windows but then need greater control over them.. Libraries such as G4P and controlP5 have taken the hard work away from the user and provided a simpler, feature rich access to multiple windows.

Personally I believe the any benefits of making the Processing API thread safe is outweighed by the effort needed to do it.

Lord_of_the_Galaxy

It is very funny indeed how a single question attracts so much attention when so many others lie untouched...
@GoToLoop Thanks anyways, I had no idea it was so complex. Also, I'm working on a project that would use multithreading, and I probably would have used the color function in it. You put a lot of effort into answering questions, and I sure appreciate it.

GoToLoop

Personally I believe the any benefits of making the Processing API thread safe...

@quark, I haven't mentioned a single word about turning the whole Processing's API thread-safe! =;
Actually I'd be very much against it for I'm a performance fanatic! =P~
And as common knowledge tells us, thread-safety may slow down the code a bit! :-O

It's just reasonable to expect that if we asynchronously mutate the sketch's main PGraphics canvas, by calling rect(), ellipse(), etc. from outside the "Animation Thread", we can corrupt it and even crash the whole app! :-SS

However, we're talking about color() here. And AFAIK, color() doesn't mutate the sketch.
So why is it dangerous to call it outside its "Animation Thread"? /:)

The answer is: extremely bad design!!! :O)
Its results are spread among 10 fields inside the PGraphics class, turning it über stateful! b-(

Simplest fix I've found is to create some class to encapsulate those 10 fields.
So the color calculation methods can instantiate it, store their calculations there, and then return that object as the final result.

This way, we can have any number of concurrent threads calling color() w/o being afraid of getting a wrong color result from it. O:-)
B/c each call to color() is an individual instance of those 10 internal fields. \m/

Lord_of_the_Galaxy

Woah, woah, woah!
You're telling me that the color() function only requires the colorMode variables that are specific to that PApplet. Then why is the design so funny? Why not just create the variables each time through the function? Wouldn't that improve thread safety? Or does just reading variables also cause trouble in multithreading?

GoToLoop

Why not just create the variables each time through the function?
Wouldn't that improve thread safety?

Not improve but completely fix it! Problem is that internal function colorCalc() isn't 1 result, but spawns 10, which it then stores on those mentioned 10 fields. :(|)

And functions stroke(), fill(), background(), clear(), tint(), ambient(), specular(), emissive() and color(), all rely on colorCalc() and those 10 fields. @-)

Let's take a look at fill() as an example:

https://GitHub.com/processing/processing/blob/master/core/src/processing/core/PGraphics.java#L6702

public void fill(float v1, float v2, float v3, float alpha) {
  colorCalc(v1, v2, v3, alpha);
  fillFromCalc();
}

protected void fillFromCalc() {
  fill = true;
  fillR = calcR;
  fillG = calcG;
  fillB = calcB;
  fillA = calcA;
  fillRi = calcRi;
  fillGi = calcGi;
  fillBi = calcBi;
  fillAi = calcAi;
  fillColor = calcColor;
  fillAlpha = calcAlpha;
}

If we modify colorCalc() to return a new instance of a class w/ 10 fields instead of storing them directly in its own PGraphics 10 fields, we make each call to it thread-safe! :ar!

public void fill(float v1, float v2, float v3, float alpha) {
  fillFromCalc( colorCalc(v1, v2, v3, alpha) );
}

protected void fillFromCalc(final ColorCalc cc) {
  fill = true;
  fillR = cc.calcR;
  fillG = cc.calcG;
  fillB = cc.calcB;
  fillA = cc.calcA;
  fillRi = cc.calcRi;
  fillGi = cc.calcGi;
  fillBi = cc.calcBi;
  fillAi = cc.calcAi;
  fillColor = cc.calcColor;
  fillAlpha = cc.calcAlpha;
}

Notice though that this solution would fix method color() only!
The other related color-functions would still be unsafe for concurrent calls! :-SS
B/c they store their state on even more fields within PGraphics! ~:>

jeremydouglass

I wonder what the performance implications of this would be. e.g. if color is often used while looping through every pixel in a pixel array and if now you are creating 250,000 colorCalc objects per draw loop when looping through pixels on a 500x500 pixel sketch.

GoToLoop

I wonder what the performance implications of this would be.

• As we can observe by its size, internal method colorCalc() is already reasonably slow.
• If we already know the color value in its hexadecimal form, we should prefer it rather than invoke color() for it.
• My propose would consist of simply adding 1 local instantiation of class ColorCalc per call.
• Then instead of assigning to those PGraphics's 10 fields, we'd transfer that to the local ColorCalc object.
• Of course, b/c we're now instantiating a class for each colorCalc() call, Java's GC is gonna have to get rid of that on some later appropriate time.
• And given that class ColorCalc doesn't even have any constructor, its task is to merely encapsulate 10 fields, it shouldn't be slow to instantiate it, I expect so: :\">

static public class ColorCalc {
  public float calcR, calcG, calcB, calcA;
  public int calcRi, calcGi, calcBi, calcAi;
  public int calcColor;
  public boolean calcAlpha;
}

Lord_of_the_Galaxy

The whole process seems rather slow. Is it handled by the CPU or the GPU? As far as I know, the GPU might be able to speed such a process up as the individual pixels are not dependent on each other and can be calculated in parallel.