We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpPrograms › Retrieving XML weather data
Page Index Toggle Pages: 1
Retrieving XML weather data (Read 2399 times)
Retrieving XML weather data
Aug 14th, 2009, 9:04am
 
Greetings everyone -  I'm fairly new to Processing (and programming in general). I'm writing a program that will take an XML feed of weather data, extract the current weather conditions, and return a color based upon the current weather.

I'm having trouble, however, getting the XML feed to update - once the program starts running, it seems to only get data from the XML feed once. When the weather feed updates every hour, the text doesn't update along with it. Ideally, the program will update itself as new information becomes available.

My (admittedly very preliminary) code is below- the if loop has been simplified so that it looks less messy. Is there something that I'm missing re: the XML feed?

thanks for your help!

Code:

import processing.serial.*;

int n;
XMLElement xml;
String cs;
color c;
Serial port;

void setup() {

 String feed = (xml weather feed URL) //actual URL removed
 
 PFont font;

 size(500,300);      //screen size, etc.
 background(0);

 font = loadFont("Arial-Black-48.vlw");
 fill(255);
 textFont(font, 13);

 String arduinoPort = Serial.list()[2];      
 port = new Serial(this, arduinoPort, 19200);
 
 xml = new XMLElement(this,feed);
 
}

void draw() {
 
   background(0);

   XMLElement lastreading = xml.getChild("observation_time");
   XMLElement weather = xml.getChild("weather");

   String reading = lastreading.getContent(); //last reading time
   String currweather = weather.getContent(); //current weather
   
   text(reading, 10, 215);
   text("Current Weather: "+ currweather, 10, 240);

   StringTokenizer st = new StringTokenizer(currweather, "\"<>,.()[] "); //tokenize each element in incoming string

   while (st.hasMoreTokens()){              

     String currentweather =st.nextToken().toLowerCase();            

       if (currentweather.indexOf("cloudy") >=0) {    
       c = color(153,51,0);
       }
     
       else if (currentweather.indexOf("sunny") >= 0) {
       c=color(153,51,17);
       }

     cs = "#"+hex(c,6);
     port.write(cs);
   }

delay(60000);
 
}

catch (Exception ex) {                                
 ex.printStackTrace();
 System.out.println("ERROR: "+ex.getMessage());

}

Re: Retrieving XML weather data
Reply #1 - Aug 14th, 2009, 9:45am
 
The data isn't updating, because you're only loading it once. Once it's been loaded, any changes on the remote end won't be seen, since the entire file is stored locally in memory. You need to move "String feed = (xml weather feed URL)" to the start of draw().
Re: Retrieving XML weather data
Reply #2 - Aug 14th, 2009, 10:28am
 
Quote:
The data isn't updating, because you're only loading it once. Once it's been loaded, any changes on the remote end won't be seen, since the entire file is stored locally in memory. You need to move "String feed = (xml weather feed URL)" to the start of draw().


Though bear in mind that IIRC by default draw() runs at 60fps and I doubt your XML feed refreshes that often, so that's perhaps a little inefficient.  A simple way to reduce the rate at which it reads is to use a counter variable:

Code:
int counter;

void setup() {
 size(20,20);
 counter = 0;  
}

void draw() {
 if(counter > 60) {
   // do stuff once per second based on 60fps.
   // ( note this shouldn't be relied on for accurate timing
   // since actual fps can be dependent on other factors)
   println("foo");
   // reset the counter
   counter = 0;
 }
 // increment the counter each frame
 counter ++;

}


If you want to be more accurate you could use millis(), or maybe a datetime function but that's probably overkill...
Re: Retrieving XML weather data
Reply #3 - Aug 17th, 2009, 11:26pm
 
Awesome, thanks! both replies helped out quite a lot. I was using a millis()-based timer before, but I agree that a simple counter is probably more appropriate.
Re: Retrieving XML weather data
Reply #4 - Aug 22nd, 2009, 9:57pm
 
If you knew that the feed was updating at a specific time, preferably every hour for this example, you could test like this:

Code:
if (minute() == 0) {
 // Re-fetch the feed
}


Or if it's multiple times per hour just expand it...

Code:
if (minute() == 0 || minute() == 15 || minute() == 30 || minute() = 45) {
 // Re-fetch the feed
}


But I'm new to Processing and I could be doing things inefficiently, but this works for me on my project. Best of luck.
Re: Retrieving XML weather data
Reply #5 - Dec 16th, 2009, 12:45am
 
Hi linwell,

i m very interested in using your code. Right now  I m working on a project that could need weather data.
Can you tell me where you getting the weatherdata? so I can try to run your code?

Best regards

Fabian
Re: Retrieving XML weather data
Reply #6 - Dec 16th, 2009, 5:09am
 
This page: Weather Widget links to two weather API (Yahoo! and Google ones).
There are probably others.
Re: Retrieving XML weather data
Reply #7 - Dec 16th, 2009, 9:18am
 
A couple of good habits to get into:
  • As others have said, don't pull from the data source more often than the data changes. No need to update more than once an hour. Updating a weather feed more than once every 20 minutes doesn't seem useful.
  • Randomize your lookup times across each instance of your app. Say for example you arbitrarily choose to pull the weather feed at 17 minutes past the hour. If your app becomes super popular and 10K people watch it simultaneously then the server is going to get hammered by 10K simultaneous requests at 17 past the hour, and none for the next 60 minutes. The solution is to add a random offset into the look up time for each app instance. So one app may look pull at 16:47 another at 23:13, etc. This way you spread the load on the server.
  • Exponential back-off. If there is an error reading from the server (say it was overloaded because 10K requests arrived at the same time), do not immediately retry. Instead wait 15 seconds and then try again. If that fails, wait 30 seconds, then 60, then 120, then 240, etc.


These are all good rules of thumb that keep server operators happy. Perhaps we should add this kind of throttling into Processing?
Page Index Toggle Pages: 1