We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
Index › Programming Questions & Help › Other Libraries › storing multiple data feeds using simpleML
‹ Previous Topic | Next Topic ›
Page Index Toggle Pages: 1
storing multiple data feeds using simpleML (Read 1506 times)
chiliberto
storing multiple data feeds using simpleML
Oct 3rd, 2007, 9:02pm  
Hi,

I'm using simpleML to get data from a page displaying XML.  The XML is coming out fine.  I'm able to make simple requests.  But when I try to make multiple requests and store the data of each request I get errors.  

I get a compile error "Null pointer exception" and this line gets highlighted in the code below:

Code:

for (j=0; j < headlines.length; j++) {


I think it's telling me that the headlines array is empty.  How can that be? A request for data was made on the lines just above it.  It should be full of data...

Code:


import simpleML.*;

// A Request object, from the library
XMLRequest xmlRequest;


String[] headlines; //Array to hold data from the request
String[][] headlinesArr; //Array to hold multiple headlines arrays


String[] dates = {"2007-09-24", "2007-09-23"};

void setup() {
  size(600,400);
  background(255);  
  
  //make xml requests and store data in a multidim array for later use
  for (i=0; i < dates.length; i++) {
    String url = "http://localhost:8888/sphider/data_xml.php?indexdate="+ dates[i];  
    
    xmlRequest = new XMLRequest(this, url);
    xmlRequest.makeRequest();
    
    //store the retrieved values in a multidim array by date
    for (j=0; j < headlines.length; j++) {
	  headlinesArr[i][j] = headlines[j];    
    }//end for j  
  }//end for i    
}

void draw() {
// some code that presents the text from the array
}

// When the request for the XML is complete
void netEvent(XMLRequest ml) {   
  // Retrieve an array of all XML elements inside "<entry*>" tags
  headlines = ml.getElementArray("entry");
}


I'd appreciate some help with this.  I'm new to Processing.  I've been using PHP, MySQL, JavaScript for years.  My code makes sense to me, so I'm puzzled as to why it doesn't work.  Am I not using the right part of my brain???

Thanks,
Chili'
Back to top
 
 
chiliberto
Re: storing multiple data feeds using simpleML
Reply #1 - Oct 3rd, 2007, 9:26pm  
Ack!  I just had a thought.  Maybe my XML feed is too slow!  It's not delivering data fast enough...

I need to find a new way to make multiple XML requests...  Any thoughts?

-GG
Back to top
 
 
extrapixel
Re: storing multiple data feeds using simpleML
Reply #2 - Oct 4th, 2007, 9:19am  
hi,

Your program does not wait for the netEvent() to be called after makeRequest(). It just coninues. That's why your headline array is still empty right after the request.

The netEvent() will be called as soon as the xml file is loaded completely, no matter in what state your main program is. So your program and the XMLRequest-object run in different threads. This is useful... for example if you want to load a big file and show a loading animation.

lets try it with one request. Pseudocode:


setup():
 makeRequest()
 xmlLoaded = false

draw():
 if xmlLoaded:
   some code that presents the text from the array
 else:
   draw waiting animation

netEvent():
 xmlLoaded = true
 fill up headlines array

So, for the multiple requests, I guess you either have to have an xmlRequest object for every request. Or you wait for one request to be completed before using its xmlRequest object to make a new request.

something like this

netEvent():
 nrLoaded++
 add this headline to your multidim array
 if nrLoaded == dates.length:
   xmlLoaded = true
 else:
   url = "http://..."+ dates[nrLoaded]  
   xmlRequest = new XMLRequest(this, url)
   xmlRequest.makeRequest()
   
Back to top
 
http://www.extrapixel.ch
WWW  
chiliberto
Re: storing multiple data feeds using simpleML
Reply #3 - Oct 5th, 2007, 9:15am  
Wow!  Thanks a ton, extrapixel.  You really shifted my thinking about using XML.  I didn't realize that another thread could be running while the rest of this stuff was going.  It makes sense now.

I rewrote my code so that it makes the XML requests and then a "loading..." message displays until the arrays are ready to be used.

However, I'm still having problems.  I'm try to put the data retrieved in each of the XML requests into a multidimensional array, but it doesn't want to do it.  Can you tell me, does this look right to you?

Code:

  //store the retrieved values in a new array
  for (i=0; i < headlines.length; i++) {
    headlinesArr[j][i] = headlines[i];
  }


I also tried this but it didn't work either:

Code:
 headlinesArr[j] = ml.getElementArray("entry");



Here's the full code (minus some extraneous text display, just to keep it short)... Is there something else that's not right in here that's causing this?

Code:

import simpleML.*;

// A Request object, from the library
XMLRequest xmlRequest;

int startTime;     // for the timer to make request every N seconds
String[] headlines; //Array to hold date from the request
String[][] headlinesArr; //Array to hold multiple headlines arrays

// variable to check if xml request was successful
boolean xmlLoaded = false; 

String[] dates = {
  "2007-09-23",
  "2007-09-24" 
};

int x, y = 60;
int i, j = 0, k = 0;

void setup() {
  size(600,400);
  background(255);  

  startTime = millis();

  // Load the font
  colorMode(RGB, 255);
  hint(ENABLE_NATIVE_FONTS);
  font = createFont("Helvetica", 18); 

  for(i = 0; i < dates.length; i++) {
    //make a new xml request
    String url = "http://localhost:8888/sphider/data_xml.php?indexdate="+ dates[i];  
    xmlRequest = new XMLRequest(this, url);
    xmlRequest.makeRequest();
  }

  frameRate(30);
}

void draw() {

  if(xmlLoaded){
    
    //go through each array of text
    for (i = 0; i < headlinesArr.length; i++) {

	// Every 5 seconds, draw a new line of text
	int now = millis();
	if (now - startTime > 5000 && k < headlinesArr[i].length) {

	  /*
	  do a bunch of stuff to display text...
	  */
		   
	  println(headlinesArr[i][k]);

	  y += 12;
	  k++;
	  startTime = now;
	}

    }

  } 
  else {
    // the xml hasn't finished loading into all the arrays yet, so show a loading message
    background(255);
    textFont(font, 12);
    textAlign(CENTER);
    fill(150);
    text("loading...", width/2, y); 
    //for debugging
    println("loading...");
  }    

}

// When the request for the XML is made
void netEvent(XMLRequest ml) {
  // Retrieve an array of all XML elements inside "<entry*>" tags
  headlines = ml.getElementArray("entry");

  //store the retrieved values in a new array
  for (i=0; i < headlines.length; i++) {
    headlinesArr[j][i] = headlines[i];
  }

  //update the new array counter  
  j++;

  //check if all the new arrays have been filled
  if(j == dates.length) {
    xmlLoaded = true;
    println("xml loaded"); 
  } 
}



urrrg???
Back to top
 
 
extrapixel
Re: storing multiple data feeds using simpleML
Reply #4 - Oct 5th, 2007, 8:45pm  
hi,

so, whats not working? probably you won't get into the "if (xmlLoaded)" part, because your date array has a length of 2, but you're only making one reqest (so xmlLoaded is never set to true). try the code you posted, but shorten the date array to one.
or - like in my pseudocode - start the other requests in the netEvent method, right after updating the counter. Basically just repeat this part:

String url = "http://localhost:8888/sphider/data_xml.php?indexdate="+ dates[i];
xmlRequest = new XMLRequest(this, url);
xmlRequest.makeRequest();

just change "i" to the counter, "j".

cheers,
patrick
Back to top
 
http://www.extrapixel.ch
WWW  
chiliberto
Re: storing multiple data feeds using simpleML
Reply #5 - Oct 9th, 2007, 11:21pm  
Patrick, thanks for you help with this.  I modified my code just as you said.  Makes sense to me...

However, I am still having problems with this.  It seems to be the way I'm loading variables into my multidimensional array.

Here's my updated netEvent() code:

Code:
 // When the request for the XML is made
void netEvent(XMLRequest ml) {
  nrLoaded++;

  // Retrieve an array of all XML elements inside "<entry*>" tags
  headlines = ml.getElementArray("entry");
  
  //for debugging
  println("start loading the multidimensional array");
  //store the retrieved values in a new array
  for (i = 0; i < headlines.length; i++) { 
    headlinesArr[nrLoaded][i] = headlines[i];
  }
  //for debugging
  println("done loading the multidimensional array");

  //check if all the new arrays have been filled
  if(nrLoaded == dates.length) {
    xmlLoaded = true;
  } 
  else {
    //make a new xml request
    String url = "http://localhost:8888/sphider/data_xml.php?indexdate="+ dates[nrLoaded];  
    xmlRequest = new XMLRequest(this, url);
    xmlRequest.makeRequest();
  }  //end if

}


I put in a line of text to tell me when it was done loading the multidimensional array.  It never gets to that line!  

I get this error message instead:

Oopsies.
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

When I comment out the lines to fill up the multidim array, it works fine as well.

Is there something I don't know about multidim arrays in Processing?  

Thanks again.
Back to top
 
 
extrapixel
Re: storing multiple data feeds using simpleML
Reply #6 - Oct 10th, 2007, 1:40am  
hi,

ok, tried it myself with the method I suggested and got the same error you get. Maybe this is one of these weird thread things that are mysterious to me ;)

so, here is another solution, which works for me. instead of starting the new request within the netEvent method as suggested, i just set the xmlLoaded variable to true. then, in the draw loop, i create the new request if xmlLoaded is true.

also note how the handling of the counter variables (i, j) changed.

EDIT: code was too long to post.
http://www.extrapixel.ch/simpleML/simpleML2.pde
Back to top
 
http://www.extrapixel.ch
WWW  
chiliberto
Re: storing multiple data feeds using simpleML
Reply #7 - Oct 10th, 2007, 2:33am  
That worked!  Thanks a ton...  I'm so much happier now!  

It looks like my biggest problem was not initializing the multidimensional array.
Back to top
 
 
extrapixel
Re: storing multiple data feeds using simpleML
Reply #8 - Oct 10th, 2007, 2:36am  
you're welcome. looking forward to see the final version Wink
Back to top
 
http://www.extrapixel.ch
WWW  
Page Index Toggle Pages: 1
‹ Previous Topic | Next Topic ›