XML issues updating a Twitter variable sketch made in Processing 1.5.1

edited May 2016 in Library Questions

Hello everyone,

I've tried endlessly to resolve this myself, but keep on running into walls... I feel like this is something the Processing community could do in their sleep!

I originally made a Processing sketch in 1.5.1 (back in 2011) that used the "Programming from A to Z" function by Daniel Shiffman to query Twitter's XML feed and record the mentions of the words "want" and "need". It would then display the results from each word on a running line graph, similar to an electrocardiogram.

I had made this into a web app using processing-1.1.0.js at the time, but it no longer works when viewed online as Java has become antiquated.

The online version is available here: http://goo.gl/ZlCQe

And the ZIP of the sketch is here: http://goo.gl/BgcRX

Now that I'm dusting this off and opening it with 3.1.1, I keep getting an error:

The class "XMLElement" does not exist

This thread answered my question, but when I changed all mentions of "XMLElement" to "XML", I got the new error:

The constructor "XML(PApplet,String)" does not exist

Since this happens in the RetrieveData.pde, code which I didn't write, I don't know where to start in updating the code. I'm also really dusty on Programming since 2011. I've tried launching this in Processing 1.5.1, but it no longer opens on OS X 10.11.

Once I had gotten the sketch running again, I was hoping to re-export it in 3.1.1 using the latest Processing.js (or up-to-date web export tool) to display it online.

I'm not exactly sure where to start, but any help is much appreciated!

-A

Answers

  • we'll need to see the code for anything more specific

  • edited May 2016

    koogs,

    Thanks for the response. I've already gone though the the XML reference and wasn't able to find a solution. I'll take a look at the tutorial you've mentioned. Here is the code from RetrieveData.pde that is not working. (The folder with both necessary pde files is in my first post.)

    /** 
    ** Daniel Shiffman
    ** Programming from A to Z
    ** Spring 2006
    ** www.shiffman.net
    ** daniel.shiffman @ nyu.edu
    */
    
    // A Simple Thread example
    // Reads information from an XML feed every so often
    // occuring to the "wait" variable
    
    import java.util.ArrayList;
    import org.w3c.dom.Element;
    import org.w3c.dom.Node;
    /** import a2z.*;                  // Lookie, our code is a Processing library!*/
    
    public class RetrieveData extends Thread {
    
      private boolean running;          // Is the thread running?  Yes or no?
      private int wait;                 // How many milliseconds should we wait in between executions?
      private ArrayList want_tweets;
      private ArrayList need_tweets;  
      private long want_since_id;
      private long need_since_id;
      private int want_count;
      private int need_count;
      private boolean available;        // Is new news available?
    
      /** Twitter url stuff */
      String search_url = "http://search.twitter.com/search.atom";
      String search_parameter = "?q=";
      String since_id_parameter = "&since_id=";
    
      // Constructor, create the thread
      // It is not running by default
      public RetrieveData (int w, int total) {
        wait = w;
        running = false;
        available = false;
        want_tweets = new ArrayList();
        need_tweets = new ArrayList();
        want_since_id = 0;
        need_since_id = 0;
        want_count = 0;
        need_count = 0;
      }
    
      // Overriding "start()"
      public void start ()
      {
        // Set running equal to true
        running = true;
        // Print messages
        System.out.println("Starting thread (will execute every " + (wait/1000) + " seconds.)"); 
        // Do wneedver start does in Thread, don't forget this!
        super.start();
      }
    
      // We must implement run, this gets triggered by start()
      public void run ()
      {
        while (running) {
          System.out.println("reloading. . ."); 
          check();
          // Ok, let's wait for however long we should wait
          try {
            sleep((long)(wait));
          } 
          catch (Exception e) {
          }
        }
      }
    
      // Our method that quits the thread
      public void quit()
      {
        System.out.println("Quitting."); 
        running = false;  // Setting running to false ends the loop in run()
        // We used to need to call super.stop()
        // We don't any more since it is deprecated, see: http://java.sun.com/j2se/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html
        // super.stop();
        // Instead, we use interrupt, in case the thread is waiting. . .
        super.interrupt();
      }
    
      public boolean available() {
        return available;
      }
    
      public synchronized ArrayList getwantTweets() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        return want_tweets;
      }
    
      public synchronized ArrayList getNeedTweets() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        return need_tweets;
      }
    
      public synchronized long getWantSinceId() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        return want_since_id;
      }
    
      public synchronized long getNeedSinceId() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        return need_since_id;
      }
    
      public synchronized int getWantCount() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        return want_count;
      }
    
      public synchronized int getNeedCount() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        return need_count;
      }
    
      public synchronized String getLatestWantTweet() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        String to_return = "";
        if (want_tweets.size() > 0) {
          to_return = (String)want_tweets.get(want_tweets.size() - 1);
        }
        return to_return;
      }
    
      public synchronized String getLatestNeedTweet() {
        // We should put a while (!available) loop here
        // but since we are explicitly only calling this function if available is true, we're ok
        available = false;
        notifyAll(); // let's notify everyone that available has changed
        String to_return = "";
        if (need_tweets.size() > 0) {
          to_return = (String)need_tweets.get(need_tweets.size() - 1);
        }
        return to_return;
      }
    
      private String constructUrl(String search_string, long since_id, int page) {
        String url = search_url;
    
        if (search_string != null) {
          url += search_parameter + search_string;
        }
        if (since_id != 0) {
          url += since_id_parameter + since_id;
        }
        if (since_id != 0) {
          url += "&page=" + page;
        }
    
        return url + "&rpp=500";
      }
    
      private synchronized void check() {
        want_tweets = new ArrayList();
        need_tweets = new ArrayList();
    
        String want_url = constructUrl("want", want_since_id, 1);
        println("\n\n\n\ntwitter-api-visualization::method() want_url " + want_url);
        String need_url = constructUrl("need", need_since_id, 1);
        println("\n\n\n\ntwitter-api-visualization::method() need_url " + need_url);
        try {
    
          XMLElement want_xml = new XMLElement(new PApplet(),want_url);
          XMLElement need_xml = new XMLElement(new PApplet(),need_url);
    
          int num_want_children = want_xml.getChildCount();
    
          int num_need_children = need_xml.getChildCount();
    
          /** show errors */
          println("want_errors: " + want_xml.getStringAttribute("error"));
          println("need_errors: " + want_xml.getStringAttribute("error"));
    
          for (int pages = 2; pages<8; pages++) {
    
            /** retrieve tweets */
            for (int i = 0; i<num_want_children; i++) {
              XMLElement child = want_xml.getChild(i);
    
              if (child.getChildCount() > 0) {
                /** println("child has child " + child);            */
    
                // get id and update since_id
                long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
                println("id " + id);
                if (id > want_since_id) want_since_id = id;
    
                // get content
                println("title " + child.getChild("title").getContent() + "\n");
                want_tweets.add(child.getChild("title").getContent());
                want_count++;
              }
            }
    
            for (int i = 0; i<num_need_children; i++) {
              XMLElement child = need_xml.getChild(i);
    
              if (child.getChildCount() > 0) {
                /** println("child has child " + child);            */
    
                // get id and update since_id
                long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
                println("id " + id);
                if (id > need_since_id) need_since_id = id;
    
                // get content
                println("content " + child.getChild("title").getContent() + "\n");
                need_tweets.add(child.getChild("title").getContent());
                need_count++;
              }
            }
    
            want_url = constructUrl("want", want_since_id, pages);
            println("\n\n\n\ntwitter-api-visualization::method() want_url " + want_url);
            need_url = constructUrl("need", need_since_id, pages);
            println("\n\n\n\ntwitter-api-visualization::method() need_url " + need_url);
          }
    
          /** retrieve tweets */
          for (int i = 0; i<num_want_children; i++) {
            XMLElement child = want_xml.getChild(i);
    
            if (child.getChildCount() > 0) {
              /** println("child has child " + child);          */
    
              // get id and update since_id
              long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
              println("id " + id);
              if (id > want_since_id) want_since_id = id;
    
              // get content
              println("content " + child.getChild("title").getContent() + "\n");
              want_tweets.add(child.getChild("title").getContent());
              want_count++;
            }
          }
    
          for (int i = 0; i<num_need_children; i++) {
            XMLElement child = need_xml.getChild(i);
    
            if (child.getChildCount() > 0) {
              /** println("child has child " + child);          */
    
              // get id and update since_id
              long id = Long.parseLong(split(child.getChild("id").getContent(), ":")[2]);
              println("id " + id);
              if (id > need_since_id) need_since_id = id;
    
              // get content
              println("content " + child.getChild("title").getContent() + "\n");
              need_tweets.add(child.getChild("title").getContent());
              need_count++;
            }
          }
        } 
        catch (Exception e) {
          println("RetrieveData::method() " + e + "\n");
          println("Probably retrieved all we could for today");
        }
    
        println("want count " + want_count);
        println("need count " + need_count);
    
        available = true;
        notifyAll();  // let's notify everyone that the headlines have been updated
      }
    }
    
  • well for a start, the twitter url in the code (http://search.twitter.com/search.atom) gives me

    "The Twitter REST API v1 is no longer active."

    it's 10 years old...

  • Answer ✓

    yeah, that needs ditching completely, it's no use anymore.

    twitter4j is probably the way to go. here's an example of search:

    https://github.com/yusuke/twitter4j/blob/master/twitter4j-examples/src/main/java/twitter4j/examples/search/SearchTweets.java

    but integrating it into processing is another thing (people have done it)

    you'll also need an api account from the twitter website because the new api requires authentication.

  • Wow. Ok, that definitely answers the question. I guess that API that was active 5 years ago isn't any longer.

    Really appreciate the help!

Sign In or Register to comment.