Twitter Streaming API

farm402

Does anybody have experience with the Twitter Streaming API?

I managed to use the Search API with twitter4j,but I can’t find any running examples or tutorials on how to use the Streaming API…

_vk

Here an working example. For this I have the library files in a folder named 'code' inside sketch folder. This was tested using twitter4j 3.0.5

The interface StatusListener is where you can do whatever you want with your tweets :) Kind of... is there that they arrive

    TwitterStream twitterStream;

    void setup() {     
      size(100, 100);    
      background(0); 
      openTwitterStream();
    }  


    void draw() {     
      background(0);
    }  



    // Stream it
    void openTwitterStream() {  

      ConfigurationBuilder cb = new ConfigurationBuilder();  
      cb.setOAuthConsumerKey("FILL_IN");
      cb.setOAuthConsumerSecret("FILL_IN");
      cb.setOAuthAccessToken("FILL_IN");
      cb.setOAuthAccessTokenSecret("FILL_IN"); 

      TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();

      FilterQuery filtered = new FilterQuery();

      // if you enter words it will use them to filter, otherwise it will sample
      String keywords[] = {
        "love"
      };

      filtered.track(keywords);

      twitterStream.addListener(listener);

      if (keywords.length==0) {
        // sample() method internally creates a thread which manipulates TwitterStream 
        twitterStream.sample(); // and calls these adequate listener methods continuously.
      }
      else { 
        twitterStream.filter(filtered);
      }
      println("connected");
    } 


    // Implementing StatusListener interface
    StatusListener listener = new StatusListener() {

      //@Override
      public void onStatus(Status status) {
        System.out.println("@" + status.getUser().getScreenName() + " - " + status.getText());
      }

      //@Override
      public void onDeletionNotice(StatusDeletionNotice statusDeletionNotice) {
        System.out.println("Got a status deletion notice id:" + statusDeletionNotice.getStatusId());
      }

      //@Override
      public void onTrackLimitationNotice(int numberOfLimitedStatuses) {
        System.out.println("Got track limitation notice:" + numberOfLimitedStatuses);
      }

      //@Override
      public void onScrubGeo(long userId, long upToStatusId) {
        System.out.println("Got scrub_geo event userId:" + userId + " upToStatusId:" + upToStatusId);
      }

      //@Override
      public void onStallWarning(StallWarning warning) {
        System.out.println("Got stall warning:" + warning);
      }

      //@Override
      public void onException(Exception ex) {
        ex.printStackTrace();
      }
    };

farm402

Thanks for your help!

What name must the library file(s) in the folder “code” have? Just “twitter4j-core-3.0.5.jar”? Unfortunately I’m not familiar with implementing libraries like this.

_vk

That's not recommended I think, as duplicates the library files, and makes updating a mess. That said if you just drop those .jar files in the IDE, it will create the folder and put copies of the files there. A better explanation, including how to avoid this is here:

http://wiki.processing.org/w/How_to_Install_a_Contributed_Library

Note this is a java library, not a Processing one. At the bottom of the page...

The sketch folder of the above sketch is like this:

Actually I putted all files there, but I think they are not all needed.

farm402

Yes, that works (also with twitter4j 4.0.2). Thank you, really appreciate your help!!

However it seems like I can’t apply the same search requests like with the REST API, is that right?

_vk

They are two different APIs... You may explore them at twitter dev site

https://dev.twitter.com/docs/api/streaming

and

https://dev.twitter.com/docs/api/1.1

In the code above, however, the word "love" is being searched... In stream api fashion...

You might be interested in this one:

https://dev.twitter.com/docs/streaming-apis/parameters#track

farm402

Great, thank you!

Did you ever use locations to filter the stream? I’m not supposed to put the coordinates in the same list as the keywords?

_vk

The JavaDoc is you friend.

Look at FilterQuery class.

And also the examples(Java examples...).

I've done this, using an old version. It's actually getting all locations, but only tweets with geoLocation info.

TwitterStream twitter = new TwitterStreamFactory(cb.build()).getInstance();
// those are defining a box for locations, upper left corner and lower right, change here...
double[][] bb= {{-180, -90}, {180, 90}};
FilterQuery filtro = new FilterQuery();
filtro.locations(bb);
twitter.filter(filtro);

edwinre

Sorry if my lack of experience is showing, I'm pretty good with PHP and have a solid understanding of PHP object orientation, but this streaming API is making me tear my hair out! I really appreciate any help!

Here's what I want to do: I'm creating a social media management app of sorts, and I want to stream all @mentions and search terms my users have selected into a MySQL database. I think I most likely need Site Streams to do this, as the app will have many users accessing many different Twitter accounts.

1) In the writing I have been able to find, it seems PHP is not the ideal language for this. I have used the Phirehose lib and have had some success, but it still leaves me with many questions...

2) How can I use the account's oauth token and secret to connect to their account and retrieve @mentions and whatever search terms they've defined?

3) How can I run this process all the time? I've tested phirehose from the browser and had some success with it printing tweets from the people I'm following (via basic auth), but how do I run it all the time? I'm comfortable with the command line, but not ultra-experienced. I've seen things about job queuing and stuff like that, but no idea what to do there.

Thanks so much for any help in advance!

EDIT: I wanted to add I'm not against not using php, but it's the serve side language I know so i would prefer to use it.

plan de marketing

_vk

Well those are questions I kind of ask myself... Sorry can't help much here. Have you noticed that Site Stream is beta, and that it needs an special authorization from twitter?

Site Stream don't uses track as the example above. Accordingly to dev site...

I think (but I don't really know!) PHP should be a better option to handle the connection and behind the hood stuff for a site like you described, I believe it should be handled server side. Processing or Processingjs could work together for the displaying/visualization part maybe.

Have you seen the libraries page in twitter dev site? There are a bunch of PHP libraries there, including some very complete frameworks, I haven't really tried, but almost : ). I think maybe one of them can suits your needs.

But I see you have been there (Phirehose), so maybe you should go back and try to solve your questions with that library...

Or maybe stick to regular API...

Although, in the examples page linked above there are examples in using oAuth to get users permission to access their accounts...

Also there are some examples here also:

https://github.com/yusuke/twitter4j/tree/master/twitter4j-examples/src/main/java/twitter4j/examples

If you make any progress, could you kindly post them here? I'd like to know more on that.

cheers.

farm402

I think the Search API suits me better anyway, but feels good to have it running. Thanks again, _vk!