Using the Picking library in Processing 2.0.

cullam

Using the Picking library in Processing 2.0.

in Contributed Library Questions • 1 year ago

So, for a project I'm working on, the Picking library seems absolutely perfect. However, it appears that in the current version of Processing, there have been some changes to the renderer that cause this library not to work. When I import the library, using the Add Library tool, and then call picker = new Picker(this);, I get a NoClassDefFoundError, on PGraphics3D. To the best of my understanding, PGrapics3D is no longer used, as everything has been switched to OpenGL. Is that correct?

Has anyone found a workaround for this, or better yet, an actual correction? I'd love to be able to use this library. Here is the link to the source code for it.
http://code.google.com/p/processing-picking-library/source/browse/#svn/trunk/Picking/src/picking?state=closed
Thanks!

Replies(13)

fraknoi

Re: Using the Picking library in Processing 2.0.

10 months ago

Hi Cullam,

I have the same issue, and a strong intrest in getting it resolved, too (like quite a few others, I beleive).

I have no solution, can only share my thoughts of non-specialist:

1- First I tried to figure if it is a fundamental issue (meaning that we should start looking for an alternative to Picking) , or "just" a configuration issue. I agree with your diagnostic, but a forum comment (about another issue) stated " PGraphics3D no longer exists in the processing.core package in Processing 2.0, but now it is in processing.opengl "

So, it makes me think that PGraphics3D is still usable, and therefore Picking might work if properly configurated.

2- A direction I tried, was to make sure that my openGL library was the correct one (not the old 1.5). Processing official doc states that In Processing 2.0, a new version of the OpenGL library replaces the one found in 1.x releases ... it is now part of the core (no need to use Import Library → OpenGL)

I tried to locate this new OpenGL library in my 2.0 config, but I can't - and since apparently it can't be downloaded separately, I assume that indeed it is imbedded in 2.0, and thereforefool-proof.

... And that's it ! I am stuck !

Anyone ?

cullam

Re: Using the Picking library in Processing 2.0.

10 months ago

I did some looking into re-implementing picking, But I didn't have enough base knowledge to go at it effectively, and couldn't really afford the time into learning enough about it. I've rolled back to Processing 1.5.1, where everything works fine. If the possibility to use picking, or a similar functioning library, comes up in 2.0, I'll re-upgrade.

chicoatomo

Re: Using the Picking library in Processing 2.0.

8 months ago

Hello

I've modified the library picking (Nicolas Clavaud) to work with processing version 2.0.b7.

You can define class Buffer and Picker, directly on your processing editor, or netbeans, or eclipse

Buffer Class

Copy code

public class Buffer extends PGraphics3D
{
protected int current_color = 0;
public Buffer(PApplet parent)
{
setParent(parent);
setSize(parent.width, parent.height);
}
@Override
public boolean displayable() { return true; }
public void callCheckSettings() { super.checkSettings(); }
@Override
public void lights() {}
@Override
public void smooth() {}
@Override
public void fill(int arg) {}
@Override
public void fill(float arg) {}
@Override
public void fill(float arg, float arg_1) {}
@Override
public void fill(int arg, float arg_1) {}
@Override
public void fill(float arg, float arg_1, float arg_2) {}
@Override
public void fill(float arg, float arg_1, float arg_2, float arg_3) {}
@Override
public void stroke(int arg) {}
@Override
public void stroke(float arg) {}
@Override
public void stroke(float arg, float arg_1) {}
@Override
public void stroke(int arg, float arg_1) {}
@Override
public void stroke(float arg, float arg_1, float arg_2) {}
@Override
public void stroke(float arg, float arg_1, float arg_2, float arg_3) {}
@Override
public void textureMode(int arg) {}
@Override
public void texture(PImage arg) {}
@Override
public void vertex(float x, float y, float z, float u, float v) { super.vertex(x, y, z); }
@Override
public void image(PImage arg, float arg_1, float arg_2) {}
@Override
public void image(PImage arg, float arg_1, float arg_2, float arg_3, float arg_4) {}
@Override
public void image(PImage arg, float arg_1, float arg_2, float arg_3, float arg_4, int arg_5, int arg_6, int arg_7, int arg_8) {}
@Override
protected void imageImpl(PImage image, float x1, float y1, float x2, float y2, int u1, int v1, int u2, int v2) {}
public void setCurrentId(int i)
{
// ID 0 to 16777214 => COLOR -16777215 to -1 (white)
// -16777216 is black
current_color = i - 16777215;
super.fill(current_color);
}
public int getId(int x, int y)
{
super.loadPixels();
// COLOR -16777216 (black) to -1 => ID -1 (no object) to 16777214
int id = pixels[y*width+x] + 16777215;
return id;
}
}

Picker Class

Copy code

public class Picker
{
protected PApplet parent;
public Buffer buffer;
public Picker(PApplet parent)
{
this.parent = parent;
buffer = new Buffer(parent);
//parent.registerMethod("pre",this);
//parent.registerMethod("draw",this);
}
public void begin()
{
if(parent.recorder == null)
{
parent.recorder = buffer;
buffer.beginDraw();
}
}
public void end()
{
buffer.endDraw();
parent.recorder = null;
}
public void start(int i)
{
if (i < 0 || i > 16777214)
{
PApplet.println("[Picking error] start(): ID out of range");
return;
}
buffer.setCurrentId(i);
}
public void stop()
{
parent.recorder = null;
}
public void resume()
{
parent.recorder = buffer;
}
public int get(int x, int y)
{
return buffer.getId(x, y);
}
public PGraphics getBuffer()
{
return buffer;
}
}

Example

Copy code

Picker picker;
float a = 0.0;
void setup()
{
size(200,200, P3D);
picker = new Picker(this);
}
void draw()
{
background(0,128,0);
a += 0.01;
picker.begin();
picker.start(0);
pushMatrix();
translate(80, 75);
rotateX(a);
rotateY(a);
fill(200,40,60);
box(50);
popMatrix();
picker.start(1);
pushMatrix();
translate(140, 75);
rotateX(a);
rotateY(a);
fill(60,40,200);
box(20);
popMatrix();
picker.end();
}
void mousePressed()
{
println(picker.get(mouseX, mouseY));
}

I hope that helps somewhat

Massimo

Re: Re: Using the Picking library in Processing 2.0.

8 months ago

This solution seems working fine! Thank you.

I'll try to include it into my Augmented Reality experiment and I'll let you know what about .obj files and other features I need there

Thank you again!

Ciao,

Massimo

EDIT: It did not work as expected read forward!

chicoatomo

Re: Using the Picking library in Processing 2.0.

8 months ago

Important Note:

I don't know exactly why, but for some reason background can't be correctly overloaded, so is very important to call picker.begin() after the background() instruction

Massimo

Re: Using the Picking library in Processing 2.0. get(x,y)

8 months ago

I checked and we may have an issue:

if the sketch size is bigger than the 200x200 size in your example I experience a small fps slow down. The problem is more evident with bigger size.

Do you have the same issue?

I found the problem is still the loadPixels() function that take too long. I think we have to find a solution based on andres snippet...

Massimo

Re: Using the Picking library in Processing 2.0.

8 months ago

I modified the Buffer class in order to avoid this frame rate problem:

Buffer.pde on gist

As you can see from the code above I mainly avoided loadPixels() function in favor of a clipped pixel array area of 1 pixel at mouse position...

Something like this:

pgl.readPixels(x, height - y, 1, 1, PGL.RGBA, PGL.UNSIGNED_BYTE, pixelColorBuffer);

What do you think?

Massimo

chicoatomo

Re: Using the Picking library in Processing 2.0.

8 months ago

Hi massimo

You're right.

With my class Buffer we need to load the entire frame in each loop of the rendering function.

However with your Buffer class we only load one pixel.

Your class besides being faster is more robust.

Thank you very much for your contribution.

I have tested it and works perfectly.

I have also another problem, perhaps we can solve.

If we render a PShape object, the picking function does not work properly.

I understand that the PShape class needs to be overloaded like the PGraphics3D class with the Buffer object.

greetings

Massimo

Re: Re: Using the Picking library in Processing 2.0.

8 months ago

There is a comment inside PGraphics that is pretty interesting:

shape.draw(this); // needs to handle recorder too

I'm investigating but for the moment I did find no more clues

chicoatomo

Re: Using the Picking library in Processing 2.0.

8 months ago

Hi all

I`ve found a important error in my solution.

the background is not managed properly in the recorder object

I'll try to find a solution

Sorry

aufsess

Re: Using the Picking library in Processing 2.0.

4 months ago

In terms of the performance issue, I was able to get the picker to run much faster (and grab the entire frame) by using GLGraphics (in 1.51) and the GLTexture class. While still slow, the .getTexture() and .updateTexture() commands didn't bog down my sketch nearly as much. Do you think it would be possible to get this same performance out of Processing 2.0?

I wonder if you could just create an offscreen buffer and get the performance boost by running it on your GPU rather than using a PImage?

EDIT: Well, I implemented my Picker code in 2.0 and it's a bit slower without the GLGraphics GLTexture(). I also tested the readPixels() command and it basically freezes if you want the whole buffer (which I do). I don't know how I would parse a larger grab this way (or whether it would be faster).

To make my picker work I use the following basic idea:

Create a PShape that is my mesh colored the appropriate "picker colors",

When I want to pick it, I draw it noSmooth() on an offscreen buffer called pickerB (just a regular PGraphics3D object). Then execute the following before I start grabbing pixels:

PImage tex = (PImage) pickerB;

Then I use the following to grab the pixels (it's much slower if I use get() or PImage, so this PImage is just a throwaway... ) Strangely this doesn't work with pickerB.loadPixels().

int id = ((PImage)pickerB).pixels[pixelNumber] + 16777215; // 16777215

I can explain the other parts if you like. No classes.

antiplast..

Re: Using the Picking library in Processing 2.0.

1 month ago

Hey guys,

Sorry, didn't have time lately to upgrade the lib for Processing 2.0.

I have finally managed to find some time to make adjustements, so the library now compiles for Processing 2.0 and I could run the examples, though there is this background issue you have noticed. Also, there may be this performance problem you're talking about.

FIY, the project is now hosted on GitHub, so you can fork and request changes.

http://n.clavaud.free.fr/processing/library/picking/

https://github.com/nclavaud/picking

antiplast..

Re: Using the Picking library in Processing 2.0.

1 month ago

Background-related issue is fixed, and Picking library is back in Processing's Library Manager.

Top Reply