XML issues updating a Twitter variable sketch made in Processing 1.5.1

aep1324

Hello everyone,

I've tried endlessly to resolve this myself, but keep on running into walls... I feel like this is something the Processing community could do in their sleep!

I originally made a Processing sketch in 1.5.1 (back in 2011) that used the "Programming from A to Z" function by Daniel Shiffman to query Twitter's XML feed and record the mentions of the words "want" and "need". It would then display the results from each word on a running line graph, similar to an electrocardiogram.

I had made this into a web app using processing-1.1.0.js at the time, but it no longer works when viewed online as Java has become antiquated.

The online version is available here: http://goo.gl/ZlCQe

And the ZIP of the sketch is here: http://goo.gl/BgcRX

Now that I'm dusting this off and opening it with 3.1.1, I keep getting an error:

The class "XMLElement" does not exist

This thread answered my question, but when I changed all mentions of "XMLElement" to "XML", I got the new error:

The constructor "XML(PApplet,String)" does not exist

Since this happens in the RetrieveData.pde, code which I didn't write, I don't know where to start in updating the code. I'm also really dusty on Programming since 2011. I've tried launching this in Processing 1.5.1, but it no longer opens on OS X 10.11.

Once I had gotten the sketch running again, I was hoping to re-export it in 3.1.1 using the latest Processing.js (or up-to-date web export tool) to display it online.

I'm not exactly sure where to start, but any help is much appreciated!

-A

koogs

reference here:

https://processing.org/reference/XML.html

and a tutorial here:

https://processing.org/tutorials/data/

koogs

we'll need to see the code for anything more specific

aep1324

koogs,

Thanks for the response. I've already gone though the the XML reference and wasn't able to find a solution. I'll take a look at the tutorial you've mentioned. Here is the code from RetrieveData.pde that is not working. (The folder with both necessary pde files is in my first post.)

/** 
** Daniel Shiffman
** Programming from A to Z
** Spring 2006
** www.shiffman.net
** daniel.shiffman @ nyu.edu
*/

// A Simple Thread example
// Reads information from an XML feed every so often
// occuring to the "wait" variable

import java.util.ArrayList;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
/** import a2z.*;                  // Lookie, our code is a Processing library!*/

public class RetrieveData extends Thread {

  private boolean running;          // Is the thread running?  Yes or no?
  private int wait;                 // How many milliseconds should we wait in between executions?
  private ArrayList want_tweets;
  private ArrayList need_tweets;  
  private long want_since_id;
  private long need_since_id;
  private int want_count;
  private int need_count;
  private boolean available;        // Is new news available?

  /** Twitter url stuff */
  String search_url = "http://search.twitter.com/search.atom";
  String search_parameter = "?q=";
  String since_id_parameter = "&since_id=";

  // Constructor, create the thread
  // It is not running by default
  public RetrieveData (int w, int total) {
    wait = w;
    running = false;
    available = false;
    want_tweets = new ArrayList();
    need_tweets = new ArrayList();
    want_since_id = 0;
    need_since_id = 0;
    want_count = 0;
    need_count = 0;
  }

  // Overriding "start()"
  public void start ()
  {
    // Set running equal to true
    running = true;
    // Print messages
    System.out.println("Starting thread (will execute every " + (wait/1000) + " seconds.)"); 
    // Do wneedver start does in Thread, don't forget this!
    super.start();
  }

  // We must implement run, this gets triggered by start()
  public void run ()
  {
    while (running) {
      System.out.println("reloading. . ."); 
      check();
      // Ok, let's wait for however long we should wait
      try {
        sleep((long)(wait));
      } 
      catch (Exception e) {
      }
    }
  }

  // Our method that quits the thread
  public void quit()
  {
    System.out.println("Quitting."); 
    running = false;  // Setting running to false ends the loop in run()
    // We used to need to call super.stop()
    // We don't any more since it is deprecated, see: http://java.sun.com/j2se/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html
    // super.stop();
    // Instead, we use interrupt, in case the thread is waiting. . .
    super.interrupt();
  }

  public boolean available() {
    return available;
  }

  public synchronized ArrayList getwantTweets() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    return want_tweets;
  }

  public synchronized ArrayList getNeedTweets() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    return need_tweets;
  }

  public synchronized long getWantSinceId() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    return want_since_id;
  }

  public synchronized long getNeedSinceId() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    return need_since_id;
  }

  public synchronized int getWantCount() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    return want_count;
  }

  public synchronized int getNeedCount() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    return need_count;
  }

  public synchronized String getLatestWantTweet() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    String to_return = "";
    if (want_tweets.size() > 0) {
      to_return = (String)want_tweets.get(want_tweets.size() - 1);
    }
    return to_return;
  }

  public synchronized String getLatestNeedTweet() {
    // We should put a while (!available) loop here
    // but since we are explicitly only calling this function if available is true, we're ok
    available = false;
    notifyAll(); // let's notify everyone that available has changed
    String to_return = "";
    if (need_tweets.size() > 0) {
      to_return = (String)need_tweets.get(need_tweets.size() - 1);
    }
    return to_return;
  }

  private String constructUrl(String search_string, long since_id, int page) {
    String url = search_url;

    if (search_string != null) {
      url += search_parameter + search_string;
    }
    if (since_id != 0) {
      url += since_id_parameter + since_id;
    }
    if (since_id != 0) {
      url += "&page=" + page;
    }

    return url + "&rpp=500";
  }

  private synchronized void check() {
    want_tweets = new ArrayList();
    need_tweets = new ArrayList();

    String want_url = constructUrl("want", want_since_id, 1);
    println("\n\n\n\ntwitter-api-visualization::method() want_url " + want_url);
    String need_url = constructUrl("need", need_since_id, 1);
    println("\n\n\n\ntwitter-api-visualization::method() need_url " + need_url);
    try {

      XMLElement want_xml = new XMLElement(new PApplet(),want_url);
      XMLElement need_xml = new XMLElement(new PApplet(),need_url);

      int num_want_children = want_xml.getChildCount();

      int num_need_children = need_xml.getChildCount();

      /** show errors */
      println("want_errors: " + want_xml.getStringAttribute("error"));
      println("need_errors: " + want_xml.getStringAttribute("error"));

      for (int pages = 2; pages<8; pages++) {

        /** retrieve tweets */
        for (int i = 0; i<num_want_children; i++) {
          XMLElement child = want_xml.getChild(i);

          if (child.getChildCount() > 0) {
            /** println("child has child " + child);            */

            // get id and update since_id
            long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
            println("id " + id);
            if (id > want_since_id) want_since_id = id;

            // get content
            println("title " + child.getChild("title").getContent() + "\n");
            want_tweets.add(child.getChild("title").getContent());
            want_count++;
          }
        }

        for (int i = 0; i<num_need_children; i++) {
          XMLElement child = need_xml.getChild(i);

          if (child.getChildCount() > 0) {
            /** println("child has child " + child);            */

            // get id and update since_id
            long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
            println("id " + id);
            if (id > need_since_id) need_since_id = id;

            // get content
            println("content " + child.getChild("title").getContent() + "\n");
            need_tweets.add(child.getChild("title").getContent());
            need_count++;
          }
        }

        want_url = constructUrl("want", want_since_id, pages);
        println("\n\n\n\ntwitter-api-visualization::method() want_url " + want_url);
        need_url = constructUrl("need", need_since_id, pages);
        println("\n\n\n\ntwitter-api-visualization::method() need_url " + need_url);
      }

      /** retrieve tweets */
      for (int i = 0; i<num_want_children; i++) {
        XMLElement child = want_xml.getChild(i);

        if (child.getChildCount() > 0) {
          /** println("child has child " + child);          */

          // get id and update since_id
          long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
          println("id " + id);
          if (id > want_since_id) want_since_id = id;

          // get content
          println("content " + child.getChild("title").getContent() + "\n");
          want_tweets.add(child.getChild("title").getContent());
          want_count++;
        }
      }

      for (int i = 0; i<num_need_children; i++) {
        XMLElement child = need_xml.getChild(i);

        if (child.getChildCount() > 0) {
          /** println("child has child " + child);          */

          // get id and update since_id
          long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
          println("id " + id);
          if (id > need_since_id) need_since_id = id;

          // get content
          println("content " + child.getChild("title").getContent() + "\n");
          need_tweets.add(child.getChild("title").getContent());
          need_count++;
        }
      }
    } 
    catch (Exception e) {
      println("RetrieveData::method() " + e + "\n");
      println("Probably retrieved all we could for today");
    }

    println("want count " + want_count);
    println("need count " + need_count);

    available = true;
    notifyAll();  // let's notify everyone that the headlines have been updated
  }
}

koogs

well for a start, the twitter url in the code (http://search.twitter.com/search.atom) gives me

"The Twitter REST API v1 is no longer active."

it's 10 years old...

koogs

yeah, that needs ditching completely, it's no use anymore.

twitter4j is probably the way to go. here's an example of search:

https://github.com/yusuke/twitter4j/blob/master/twitter4j-examples/src/main/java/twitter4j/examples/search/SearchTweets.java

but integrating it into processing is another thing (people have done it)

you'll also need an api account from the twitter website because the new api requires authentication.

aep1324

Wow. Ok, that definitely answers the question. I guess that API that was active 5 years ago isn't any longer.

Really appreciate the help!