Looking for tutorials on *real-time* data visualisation.

Neowso

I'm searching the web for tutorials on real time data visualisation: using API's, JSON or XML. Does anyone have any tips on good tutorials? Video preferred as I seem to understand better when I see something being done in front of me. I've read up on the data section of the processing website.

The 2x tutorials on Lynda.com seem really good but seem to deal with non realtime CSV files.

Thanks in advance :)

TfGuy44

Write your sketch using static sample data. If what you draw depends on the values in the data, then to get a real-time data visualization going, all you need to do is constantly fetch the latest data from your source.

Neowso

Thanks TfGuy44. It's the fetching from the source that I'm struggling with.

TfGuy44

Well, put your data-getting into it's own function to keep your code simple.

Then you probably want a timer so you aren't trying to get the same data every time you redraw a frame.

Even with those two requirements, your code should already have this format:

int time_to_fetch;
int time_between_fetches = 5000; // Five seconds.
Data data;

void setup(){
  size(...);
  get_data(); // Get initial data.
  // ... The rest of setup.
}

void draw(){
  draw_data();
  if( millis() > time_to_fetch ){
    get_data();
  }
}

void get_data(){
  time_to_fetch = millis() + time_between_fetches;
  // Do some action to get the data here.
  // This probably involves calling your API, which depends a lot on your API.
  data = ???;
}

void draw_data(){
  // Draw things based on the data here.
}

Without knowing the API you're trying to get data from, helping you beyond this point is hard. Most APIs come with documentation that describes how to get data from them... I'd go see their examples.

Neowso

I'm using Schiffman's data as a guide: https://processing.org/tutorials/data/

There is a live weather XML example, but when I run the code it shows an error and won't work.

I'd like to learn how to retrieve real time travel info, sort of like in this link: https://data.gov.ie/dataset/real-time-passenger-information-rtpi-for-dublin-bus-bus-eireann-luas-and-irish-rail/resource/c821fac3-ed2b-4901-b9ae-0dd1c6a748bf?inner_span=True

Neowso

Just to clarify what that error was. It was a null pointer exception:

TfGuy44

Have you tried visiting the URL in question? It's a null pointer because the URL gives you a 404 error => PAGE NOT FOUND!!

You should try finding a site that can serve you up some data...!

Neowso

Yeah, I tried.

Currently working on retrieving info from this link. Failed already with json, but it seems to indicate that xml can be retrieved too. Fingers crossed.

https://data.gov.ie/dataset/real-time-passenger-information-rtpi-for-dublin-bus-bus-eireann-luas-and-irish-rail/resource/c821fac3-ed2b-4901-b9ae-0dd1c6a748bf?inner_span=True

kfrajer

Check their documentation: https://data.gov.ie/dataset/real-time-passenger-information-rtpi-for-dublin-bus-bus-eireann-luas-and-irish-rail/resource/4b9f2c4f-6bf5-4958-a43a-f12dab04cf61?inner_span=True

An example code below. Notice you can copy and past the url request directly into a browser to see what data you are getting from the request. For more info, check examples and the reference:

https://processing.org/reference/loadJSONObject_.html

Related to your most recent post, please don't post pictures of code. Instead, copy and paste the code here directly in the forum and format that code properly as described in the sticky post in the forum.

Kf

//===========================================================================
// GLOBAL VARIABLES:
JSONObject json;


//===========================================================================
// PROCESSING DEFAULT FUNCTIONS:


//errorcode  "1"
//errormessage  "No Results"
//numberofresults  0
//stopid  "7602"
//timestamp  "02/04/2018 03:43:46"
//results  []


void setup() {
  size(400,600);

  json = loadJSONObject("https://" + "data.smartdublin.ie/cgi-bin/rtpi/realtimebusinformation?stopid=7602&format=json");

  int errorCode = json.getInt("errorcode");
  String timeStamp = json.getString("timestamp");
  String stopid = json.getString("stopid");

  println(errorCode + ", " + timeStamp + ", " + stopid);
}

void draw(){

}

Neowso

Thank you, kfrajer. I am re-reading that Schiffman webpage on data for the third time and some of it is finally sinking in, moreso about XML. This method of parsing JSON from a URL is very helpful. I appreciate you taking the time to post that. You are a gent :)