Displaying millions of points

Programming Questions

duffmcscr..

Displaying millions of points

in Programming Questions • 2 years ago

Hello all,

I am working on a project that involves mapping gps coordinates of world cities. I have borrowed most of the code I am using from Jimthree:

http://forum.processing.org/topic/getting-started-with-glgraphics

In the above sketch, he is able to display a few hundred thousand points while maintaining 30frames/sec. My sketch began locking up after a few thousand points. I would like to use xml to import the data. Ideally, I would like to display more than a million points (at 30fr/sec). Is this possible using the method I am currently using (i.e. importing the points through xml)? Can this code be made more efficient?

Here is a link to the xml files I am using:

Cities of the world (~750 points): http://www.mapofthemap.com/maps/world.xml

Points in Italy (~33000 points): http://www.mapofthemap.com/maps/italy.xml

Here is the sketch so far:

import processing.opengl.*;
import javax.media.opengl.*;
import javax.media.opengl.glu.*;
import peasy.*;
PeasyCam cam;
GL gl;
GLU glu;
XMLElement xml;
//Declare the Arrays for storing co-ords.
float[] LAT = new float[1];
float[] LON = new float[1];
float[] coords_x = new float[1];
float[] coords_y = new float[1];
float[] coords_z = new float[1];
int r = 600;
void setup(){
size(1000, 600, OPENGL);
cam = new PeasyCam(this, 0, 0, 0, 1200);
cam.setWheelScale(50);
cam.setResetOnDoubleClick(false);
cam.lookAt(width/2, height/2, 1200);
cam.setMinimumDistance(800);
cam.setMaximumDistance(1600);
cam.rotateX(3*(PI/2));
cam.rotateY(PI/2);
float fov = PI/3.0;
float cameraZ = (height/2.0) / tan(fov/2.0);
perspective(fov, float(width)/float(height),cameraZ/200.0, cameraZ*1000.0);
//SetUp the GL thingys
gl=((PGraphicsOpenGL)g).gl;
glu=((PGraphicsOpenGL)g).glu;
//Load the point data into the arrays
load_data();
}
void draw () {
frame.setTitle(str((frameRate))+"fps");
background(50);
//More GL stuff I don't really understand yet.
PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;
//All GL drawing commands are wrapped in a beginGL()/endGL() loop
GL gl = pgl.beginGL();
//Move everything we draw into the middle of the screen (or could just move the camera)
gl.glTranslatef(width/2, height/2, 1200);
//Draw the points
draw_points();
pgl.endGL();
}
void load_data(){
xml = new XMLElement(this, "world.xml");
int index = xml.getChildCount();
//expand arrays to fit.
coords_x = expand (coords_x,index);
coords_y = expand (coords_y,index);
coords_z = expand (coords_z,index);
LAT = expand (LAT,index);
LON = expand (LON,index);
for (int i=0; i < index; i++){
XMLElement kid = xml.getChild(i);
String cityName = kid.getStringAttribute("name");
LAT[i] = kid.getFloatAttribute("LAT");
LON[i] = kid.getFloatAttribute("LON");
coords_x[i] = -r * cos(LAT[i]*PI/180) * cos(LON[i]*PI/180);
coords_y[i] = r * cos(LAT[i]*PI/180) * sin(LON[i]*PI/180);
coords_z[i] = r * sin(LAT[i]*PI/180);
//println("City name = "+cityName);
//println("Lat = "+LAT[i]);
//println("Lon = "+LON[i]);
//println("x = "+coords_x[i]);
//println("y = "+coords_y[i]);
//println("z = "+coords_z[i]);
}
}
void draw_points(){
gl.glPointSize(1);
gl.glBegin(gl.GL_LINES);
gl.glColor4f(1,0,0,.25);
gl.glVertex3f(0, 0, 0);
gl.glVertex3f(6000, 0, 0);
gl.glEnd( );
gl.glBegin(gl.GL_LINES);
gl.glColor4f(0,1,0,.25);
gl.glVertex3f(0, 0, 0);
gl.glVertex3f(0, 6000, 0);
gl.glEnd( );
gl.glBegin(gl.GL_LINES);
gl.glColor4f(0,0,1,.25);
gl.glVertex3f(0, 0, 0);
gl.glVertex3f(0, 0, 6000);
gl.glEnd( );
gl.glBegin(gl.GL_POINTS);
gl.glColor4f(1,1,1,1);
xml = new XMLElement(this, "world.xml");
int index = xml.getChildCount();
for (int i=0; i<index; i++){
gl.glVertex3f(coords_x[i], coords_y[i], coords_z[i]);
}
gl.glEnd();
}

Replies(6)

PhiLho

Re: Displaying millions of points

2 years ago

Looks like you load and parse world.xml on each frame... No wonder it is slow...

jeff_g

Re: Displaying millions of points

2 years ago

As Phillho said.. only do this once in your setup(), don't put it in your draw_points().

xml = new XMLElement(this, "world.xml");
int index = xml.getChildCount();

Also, there is no need to do this in your draw().

//More GL stuff I don't really understand yet.
PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;
//All GL drawing commands are wrapped in a beginGL()/endGL() loop
GL gl = pgl.beginGL();

Declare "PGraphicsOpenGL pgl;" and "GL gl;" globally (you already did gl);
Put this in your setup();

pgl = (PGraphicsOpenGL) g;
gl = pgl.gl;

Then put this in your draw()

pgl.beginGL();
//... OpenGL stuff...
pgl.engGL();

If you really want to make it fast, look at turning your draw_points() into a display list.

duffmcscr..

Re: Displaying millions of points

2 years ago

@phi.lho

You were right, I was loading and parsing on the same frame.

@jeff_g

Thank you for your help. With the changes you suggested the sketch is running very fast. Also, I will look into turning draw_points() into a display list (not sure what that is yet, but I will learn).

duffmcscr..

Re: Re: Displaying millions of points

2 years ago

@jeff_g

Thanks for the suggestion of 'display lists'. The frame rate has improved dramatically.

hondaman9..

Re: Displaying millions of points

1 month ago

Would you mind sharing your solution using display lists? I have a similar problem displaying large wireframe models and would like to try that technique.

Thanks in advance

amnon.owed

Re: Displaying millions of points

1 month ago

Is it really necessary to resurrect an old thread? That was two years ago. Now Processing 2.0 is out and has the convenient retained PShape that handles all this for you in the background. See: https://twitter.com/AmnonOwed/status/373931200122867713

Top Reply