Twitter Stream API Implementation?

edited December 2015 in Library Questions

Hello,

I was just wondering if anyone has a working example of a processing sketch that uses the Twitter Stream API?

The few examples I can find are either out of date or not working.

Thanks!

Answers

  • I'd be curious about this too

  • _vk_vk
    edited December 2015

    I'll post something later. When I get sober :)

  • Hey why wait? Drunk coding could be interesting! But seriously, thanks in advance _vk.

  • _vk_vk
    edited December 2015

    Well... not if chars are spinning :P

    Here an example. There is no rate limit check though. It will only be a problem if you filter for very popular stuff. I could not find my code to deal with it, and got no time to re do it no. But is not really a big deal. ( you see drunk code could be better than hangover code... Rate limits are for the serach API, i guess...)

    import twitter4j.util.*;
    import twitter4j.*;
    import twitter4j.management.*;
    import twitter4j.api.*;
    import twitter4j.conf.*;
    import twitter4j.json.*;
    import twitter4j.auth.*;
    
    TwitterStream twitterStream;
    
    
    void setup() {     
      size(100, 100);    
      background(0); 
      openTwitterStream();
    }  
    
    
    void draw() {     
      background(0);
    }  
    
    
    
    
    // Stream it
    void openTwitterStream() {  
    
      // OAuth stuff
      ConfigurationBuilder cb = new ConfigurationBuilder();  
      cb.setOAuthConsumerKey("FILL");
      cb.setOAuthConsumerSecret("FILL");
      cb.setOAuthAccessToken("FILL");
      cb.setOAuthAccessTokenSecret("FILL"); 
    
      //the stream object
      TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();
    
      // filter is used to pass querys to the stream
      // see twitter4j's java doc
      FilterQuery filtered = new FilterQuery();
    
      // if you enter keywords here it will filter, otherwise it will sample
      String keywords[] = {
        "hangover"
      };
    
      //track is like "search"... well kind of
      // for a better explanation go 
      // dev.twitter.com/streaming/overview/request-parameters"
      filtered.track(keywords);
    
      // the StatusListener interface is where the magic happens
      // code there will be executed upon tweets arriving
      // so we want to attach one to our stream
      twitterStream.addListener(listener);
    
      if (keywords.length==0) {
        // sample() method internally creates a thread which manipulates TwitterStream 
        // and calls these adequate listener methods continuously.
        // ref //dev.twitter.com/streaming/reference/get/statuses/sample
        // "Returns a small random sample of all public statuses"
        twitterStream.sample();
      } else { 
        twitterStream.filter(filtered);
      }
      println("connected");
    } 
    
    
    // Implementing StatusListener interface
    // the methods are pretty much self explantory
    // they will be called according to the messages arrived
    // onStatus is probabbly what you are lookikng for...
    StatusListener listener = new StatusListener() {
    
      //@Override
      public void onStatus(Status status) {
        System.out.println("@" + status.getUser().getScreenName() + " - " + status.getText());
      }
    
      //@Override
      public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
        System.out.println("Got a status deletion notice id:" + statusDeletionNotice.getStatusId());
      }
    
      //@Override
      public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
        System.out.println("Got track limitation notice:" + numberOfLimitedStatuses);
      }
    
      //@Override
      public void onScrubGeo(long userId, long upToStatusId) {
        System.out.println("Got scrub_geo event userId:" + userId + " upToStatusId:" + upToStatusId);
      }
    
      //@Override
      public void onStallWarning(StallWarning warning) {
        System.out.println("Got stall warning:" + warning);
      }
    
      //@Override
      public void onException(Exception ex) {
        ex.printStackTrace();
      }
    };
    
  • Hi, so I have tried running your code but I get the message: "Cannot find a class or type named "TwitterStream".

    Thanks

  • _vk_vk
    edited December 2015

    Hello. the code is working here with processing 2.2.1 and tw4j 4.0.3.

    Probably some change in tw4j API... We gota check...

  • Ok, now tested with latest stabel version tw4j 4.0.4 (still processing 2.2.1). And its working fine. Also the java doc makes no mention to changes in TwitterStream interface. So maybe you have a problem in your import. Perhaps you have not imported twitter4j-stream-x.x.x?

    Or ir can be a processing 3 issue, but i think it's not likely.

  • I'm going to have to study this and try it tomorrow. Big thanks!

  • _vk, it is probably an issue with my import. I have started again by placing the folder "twitter4j-4.0.4" into my processing libraries folder, I'm not exactly sure which files from the folder to add to the sketch.

  • _vk_vk
    edited December 2015

    Installing a java library is weird. there is this article:

    https://github.com/processing/processing/wiki/How-to-Install-a-Contributed-Library

    Last stuff is about non processing libraries.

    The way I did, and is working, is weird and involves changing the names of the jars it selfs. As I get a complaint about - and . in the names. The final result is like this (those are inside mysketchbook/libraries):

    Screen Shot 2015-12-07 at 12.10.42 PM

    note the name of the folder is changed, all jars got rid os dashes and point and 'lib' folder is renamed to library.

    Alternatively you can just drop the folder in your sketch in PDE and it will work ONLY for that sketch, which is not optimal.

  • Thanks _vk, That has solved the previous issue. However, I am now getting the message:

    "The type new StatusListener(){} must implement the inherited abstract method StatusListener.onStatus(Status)"

    I am unsure what this is referring to.

    Thanks again.

  • edited December 2015

    "The type new StatusListener() {} must implement the inherited abstract method StatusListener.onStatus(Status)"

    abstract methods are merely some description about what are their parameters' types and their return type. In other words, they are empty methods! :|

    In order for a non-abstract class to inherit from a class or interface containing abstract methods,
    the former must implement each of them.

    It's also highly advisable to prefix those method implementations w/ @ Override.
    So the Java compiler can check whether the implementation matches the abstract method.

    Take a look at @_vk's example for a StatusListener's anonymous instantiation. *-:)

  • @GoToLoop do you understand this importing non processing libraries? Could you help understanding this weird need to change the jar files names?

  • My only experience was once importing a library w/ 1 ".jar" file only. :|

  • Ok thanks anyway.

  • Temboo makes it easier to interface with Twitter. You loose some control for ease of use. Depending on how much data you need to request, it's either free or it requires a fee: https://www.temboo.com/processing

  • _vk_vk
    edited December 2015

    Also if using p5.js there is a amazing tutorial by Daniel Shiffman using a js library, I think is tweet. Worth watching.

    Actually he shows a way you could even use a java code (twitter4j) and deploy it via node and command line.

  • @REAS you probably could say if the method of importing twitter4j described above is correct, couldn't you. :)

  • Looking at Temboo, would I be right in thinking that it uses the search API rathe than the streaming API and can therefore not be left running for a long period of time due to the limits? I am simply trying to find a way to receive tweets with a certain (unpopular) hashtag, even if it is just 1 tweet per minute rather than all tweets with in real time. I'm still not having any luck. Thanks again

  • _vk_vk
    edited December 2015

    I'm still not having any luck.

    Why? the code I posted above does exactly that. What is going wrong? Still the same problem? @GoToLoop have told you how to go about the

    "The type new StatusListener(){} must implement the inherited abstract method StatusListener.onStatus(Status)"

    Just implement all methods, even if you just use some of them...

  • Yes, i'm still having trouble with that part. I'm not exactly sure what is meant by 'implement all methods' or how to go about doing so.

    Thanks again

  • _vk_vk
    edited December 2015

    In this case you should have asked here ;)

    just use this (copied from the post above)

    StatusListener listener = new StatusListener() {
    
      //@Override
      public void onStatus(Status status) {
        System.out.println("@" + status.getUser().getScreenName() + " - " + status.getText());
      }
    
      //@Override
      public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
        System.out.println("Got a status deletion notice id:" + statusDeletionNotice.getStatusId());
      }
    
      //@Override
      public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
        System.out.println("Got track limitation notice:" + numberOfLimitedStatuses);
      }
    
      //@Override
      public void onScrubGeo(long userId, long upToStatusId) {
        System.out.println("Got scrub_geo event userId:" + userId + " upToStatusId:" + upToStatusId);
      }
    
      //@Override
      public void onStallWarning(StallWarning warning) {
        System.out.println("Got stall warning:" + warning);
      }
    
      //@Override
      public void onException(Exception ex) {
        ex.printStackTrace();
      }
    };
    

    Whatever you want to do with the status arrived you do inside the onStatus(Status status) method, referring to the arrived Status (a tweet) as status. For instance you want to display the text from that status you would say

    StatusListener listener = new StatusListener() {
    
     //do your stuff here..
    //@Override
          public void onStatus(Status status) {
            String tweet = status.getText();
            text(tweet, 10, 100);
          }
    
      // Keep those methods here... this is it...
      //@Override
      public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
        System.out.println("Got a status deletion notice id:" + statusDeletionNotice.getStatusId());
      }
    
      //@Override
      public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
        System.out.println("Got track limitation notice:" + numberOfLimitedStatuses);
      }
    
      //@Override
      public void onScrubGeo(long userId, long upToStatusId) {
        System.out.println("Got scrub_geo event userId:" + userId + " upToStatusId:" + upToStatusId);
      }
    
      //@Override
      public void onStallWarning(StallWarning warning) {
        System.out.println("Got stall warning:" + warning);
      }
    
      //@Override
      public void onException(Exception ex) {
        ex.printStackTrace();
      }
    };
    

    But keep all other methods. even if they are empty..

    Each one is called on each event, e.g. if someone delete an status (tweet) you should delete it from your database (if you have stored it). The code to do this would go inside onDeletionNotice(StatusDeletionNotice statusDeletionNotice) method. So it would be called whenever a deletion notice arrives. is that clear?

    Post your code if you need further assistance.

  • Hi _vk,

    I haven't changed anything in the code so here is what I am trying to run (minus OAuth bits) :

    import twitter4j.util.*;
    import twitter4j.*;
    import twitter4j.management.*;
    import twitter4j.api.*;
    import twitter4j.conf.*;
    import twitter4j.json.*;
    import twitter4j.auth.*;
    
    TwitterStream twitterStream;
    
    
    void setup() {     
      size(100, 100);    
      background(0); 
      openTwitterStream();
    }  
    
    
    void draw() {     
      background(0);
    }  
    
    
    // Stream it
    void openTwitterStream() {  
    
      // OAuth stuff
      ConfigurationBuilder cb = new ConfigurationBuilder();  
      cb.setOAuthConsumerKey("");
      cb.setOAuthConsumerSecret("");
      cb.setOAuthAccessToken("");
      cb.setOAuthAccessTokenSecret(""); 
    
      //the stream object
      TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();
    
      // filter is used to pass querys to the stream
      // see twitter4j's java doc
      FilterQuery filtered = new FilterQuery();
    
      // if you enter keywords here it will filter, otherwise it will sample
      String keywords[] = {
        "hangover"
      };
    
      //track is like "search"... well kind of
      // for a better explanation go 
      // dev.twitter.com/streaming/overview/request-parameters"
      filtered.track(keywords);
    
      // the StatusListener interface is where the magic happens
      // code there will be executed upon tweets arriving
      // so we want to attach one to our stream
      twitterStream.addListener(listener);
    
      if (keywords.length==0) {
        // sample() method internally creates a thread which manipulates TwitterStream 
        // and calls these adequate listener methods continuously.
        // ref //dev.twitter.com/streaming/reference/get/statuses/sample
        // "Returns a small random sample of all public statuses"
        twitterStream.sample();
      } else { 
        twitterStream.filter(filtered);
      }
      println("connected");
    } 
    
    
    // Implementing StatusListener interface
    // the methods are pretty much self explantory
    // they will be called according to the messages arrived
    // onStatus is probably what you are lookikng for...
    StatusListener listener = new StatusListener() {
    
     //do your stuff here..
    //@Override
          public void onStatus(Status status) {
            String tweet = status.getText();
            text(tweet, 10, 100);
          }
    
      // Keep those methods here... this is it...
      //@Override
      public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
        System.out.println("Got a status deletion notice id:" + statusDeletionNotice.getStatusId());
      }
    
      //@Override
      public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
        System.out.println("Got track limitation notice:" + numberOfLimitedStatuses);
      }
    
      //@Override
      public void onScrubGeo(long userId, long upToStatusId) {
        System.out.println("Got scrub_geo event userId:" + userId + " upToStatusId:" + upToStatusId);
      }
    
      //@Override
      public void onStallWarning(StallWarning warning) {
        System.out.println("Got stall warning:" + warning);
      }
    
      //@Override
      public void onException(Exception ex) {
        ex.printStackTrace();
      }
    };
    

    Thanks

  • I can't run it now. I can check it later. Is this working? Do you get any messages? If so, which? In which line?

  • No, it still returns the message "The type new StatusListener(){} must implement the inherited abstract method StatusListener.onStatus(Status)" highlighting the line: StatusListener listener = new StatusListener() {

    Thanks

  • _vk_vk
    edited December 2015

    Well that's weird. I copy the code, insert oAuth data, hit run, and... Everything goes fine, getting tweets about hangover

    Screen Shot 2015-12-09 at 8.03.34 PM

    A note. the way I did in line 79 in your code, will fail to print the tweet to the canvas, as the background call in draw() will erase it immediately and the text is write outside the canvas. You may change size in setup() and get rid of background() or, better, just keep the original println(tweet) while debugging.

    I really don't see where the problem is, as the code runs perfect here...

  • Could it be an issue with the files in the sketch folder?

  • _vk_vk
    edited December 2015 Answer ✓

    As I said, I don't know. The error does not look like is an import error, I guess, as it is complaining about the implementation of the interface.

    But you could try to just drop in a NEW sketch the extracted twitter4j jars (the files inside lib folder, but the readme) - without changing anything - then copy/paste the code an run.

    The above about importing, with all the changes I made, is NOT to be made when you use the sketch's code folder. It's meant to have the library in processing's libraries folder and just use import in the code.

    The reason I'm suggesting the new sketch is to reassure there is no other silly errors, like missing brackets or whatever. As I see in the same sketch you got a lot more libraries.

    Try this virgin sketch with droped code

  • edited December 2015

    Placing ".jar" files inside "code/" subfolder should be a last ditch resort!
    Strive to get your libraries from the PDE's "Contribution Manager"!

  • I would do that but when I try to add the library from within processing, it doesn't appear on the list of available libraries

  • Success! Creating a new sketch as you advised did the job. Thanks for you help!

  • _vk_vk
    edited December 2015

    Good to hear. #:-S

    Placing ".jar" files inside "code/" subfolder should be a last ditch resort!

    Well that was a last ditch resort... Just for debugging

    Strive to get your libraries from the PDE's "Contribution Manager"!

    This is a non Processing library. A java library.

  • This is a non Processing library. A java library.

    Oh, I didn't know. Even so, it's much preferable to force it to install along Processing's libraries so it's available for all sketches. :ar!

Sign In or Register to comment.