[twitter4j] what is the full answer from a query using the REST API?

eduzal

hi guys

i'm using twitter4j to get tweets from a search. I wanna know what are the fields that the result provides. in almos all the examples around we see .getText() for the message .getUser().getScreenName() for the username .getCreatedAt() for the timestamp of the tweet

what else do we get from the search? it looks like it has a JSON Object syntax, is there any link where the full answer is mapped, so I could know what infos I can retrieve out of a simple rest Search?

tks

koogs

twitter4j javadoc suggests that the result of a Search is a QueryResult which contains a list of Status objects.

http://twitter4j.org/javadoc/twitter4j/api/SearchResource.html

http://twitter4j.org/javadoc/twitter4j/QueryResult.html

http://twitter4j.org/javadoc/twitter4j/Status.html

google is your friend

_vk

https://dev.twitter.com/overview/api/entities-in-twitter-objects

actually the whole documentation...

https://dev.twitter.com/overview/documentation

eduzal

what i'm looking for is at the status page, also at the User Interface http://twitter4j.org/javadoc/twitter4j/User.html

I wasn't finiding it. Not very good at reading javadocs.

tks

_vk

also

https://gist.github.com/hrp/900964

https://gist.github.com/gnip/764239

eduzal

yeah! thanks _vk. i'm kind of figuring out what and how we recive stuff from twitter.

_vk

You could just save a JSON from a very complete tweet that you post your self, I mean with all entities, like: @user, http:/someUrl, some pict/video/, etc.

Use JSONFactory (something like this... Don't remember exactly) in twitter4j.

_vk

this can save jsons from your timeline

import twitter4j.util.*;
import twitter4j.*;
import twitter4j.management.*;
import twitter4j.api.*;
import twitter4j.conf.*;
import twitter4j.json.*;
import twitter4j.auth.*;

import processing.opengl.*;
import java.io.*;
import java.util.List;

ArrayList <String> toWrite = new ArrayList();
Twitter twitter;


void setup() {
  size(600, 400);


  //Credentials

  ConfigurationBuilder cb = new ConfigurationBuilder();
  cb.setOAuthConsumerKey("FILL");
  cb.setOAuthConsumerSecret("FILL");
  cb.setOAuthAccessToken("FILL");
  cb.setOAuthAccessTokenSecret("FILL"); 
  cb.setDebugEnabled(true);
  cb.setJSONStoreEnabled(true);
  Twitter twitter = new TwitterFactory(cb.build()).getInstance();

 System.out.println("Saving public timeline.");
        try {
            new File("statuses").mkdir();
            List<Status> statuses = twitter.getHomeTimeline();
            for (Status status : statuses) {
                String rawJSON = DataObjectFactory.getRawJSON(status);
                println("@" + status.getUser().getScreenName() + " - " + status.getText());
                println("rawJASON= "+ rawJSON);
                String fileName = "statuses/" + status.getId() + ".json";
                toWrite.add(rawJSON + "\n"+" "+"\n");
                System.out.println(fileName + " - " + status.getText());
            }
            saveStrings("saved.txt", toWrite.toArray(new String[0]));
            System.out.print("\ndone.");
            System.exit(0);

        } catch (TwitterException te) {
            te.printStackTrace();
            System.out.println("Failed to get timeline: " + te.getMessage());
            System.exit(-1);
        }


}//eof setup

and this can read them back and test several twitter4j's methods on an Status

import twitter4j.Status;
import twitter4j.TwitterException;
import twitter4j.json.DataObjectFactory;
String[] rawData;
int i = 0;


void setup()
{
  ConfigurationBuilder cb = new ConfigurationBuilder();
  cb.setDebugEnabled(true);
  cb.setJSONStoreEnabled(true);
  rawData = loadStrings("saved.txt");
}

void draw()
{
  try {
    Status status = DataObjectFactory.createStatus(rawData[i]);
    println("contributors: " + status.getContributors());
    println("\ncreated at: " + status.getCreatedAt());
    println("\n geoLocation: " + status.getGeoLocation());
    Long l = status.getId();
    println("\n id : " + l.toString());
    println("\n In reply to User: " + status.getInReplyToScreenName());
    println("\n In reply to Status: " + status.getInReplyToStatusId());
    println("\n In reply to User ID: " + status.getInReplyToUserId());
    println("\n Place: " + status.getPlace());
    println("\n Retweet Count: " + status.getRetweetCount());
    println("\n Retweeted Status: " + status.getRetweetedStatus());
    println("\n Source: " + status.getSource());
    println("\n Text: " + status.getText());
    //println("\n User: " + status.getUser());
    println("\n Is favorited? " + status.isFavorited());
    println("\n Is retweeted? " + status.isRetweet());
    println("\n Is retweeted by me? " + status.isRetweetedByMe());
    println("\n Is truncated? " + status.isTruncated());

    User u = status.getUser();
    println("******USER*******");
    println("\n\n\n User created at: " + u.getCreatedAt());
    println("\n Description: " + u.getDescription());
    println("\n favourites count: " + u.getFavouritesCount());
    println("\n number of folowers: " + u.getFollowersCount());
    println("\n number of friends: " + u.getFriendsCount());
    println("\n User id: " + u.getId());
    println("\n language: " + u.getLang());
    println("\n number of lists: " + u.getListedCount());
    println("\n location: " + u.getLocation());
    println("\n name: " + u.getName());
    println("\n back color: " + u.getProfileBackgroundColor());
    println("\n back image url: " + u.getProfileBackgroundImageUrl());
    println("\n back image url https: " + u.getProfileBackgroundImageUrlHttps());  
    println("\n profile image url: " + u.getProfileImageURL());
    println("\n profile image url https: " + u.getProfileImageUrlHttps());
    println("\n profile link Color: " + u.getProfileLinkColor());
    println("\n side bar border color: " + u.getProfileSidebarBorderColor());
    println("\n side bar fill color: " + u.getProfileSidebarFillColor());
    println("\n text color: " + u.getProfileTextColor());
    println("\n screen Name: " + u.getScreenName());
    println("\n status nulo?: " + u.getStatus());
    println("\n status count: " + u.getStatusesCount());  
    println("\n TimeZone: " + u.getTimeZone());
    println("\n site: " + u.getURL());  
    println("\n utc offset: " + u.getUtcOffset());
    println("\n contributors enabled? " + u.isContributorsEnabled());
    println("\n have i required folow? " + u.isFollowRequestSent());
    println("\n geoenabled? " + u.isGeoEnabled());
    println("\n back tiled? " + u.isProfileBackgroundTiled());
    println("\n using back image? " + u.isProfileUseBackgroundImage());
    println("\n protected? " + u.isProtected());
    println("\n show inline media? " + u.isShowAllInlineMedia());
    println("\n translator?? " + u.isTranslator());
    println("\n verified?? " + u.isVerified());


    if (status.getHashtagEntities().length >  0)
    {
      println("\n hashtag: " + status.getHashtagEntities()[0] + "  l "  + status.getHashtagEntities().length );
      HashtagEntity[] ht = status.getHashtagEntities();
      for (int i = 0; i < ht.length; i++)
      {
        println("** " + ht[i].getStart());
        String tex = status.getText();
        println(tex.charAt(ht[i].getStart()));
      }
    } else println("\nnoHashtag");






    if (status.getMediaEntities() != null && status.getMediaEntities().length >  0)
      println("\n media: " + status.getMediaEntities()[0] + "  l "  + status.getMediaEntities().length); 
    else println("\nnoMediaEntities");



    if (status.getHashtagEntities() != null && status.getHashtagEntities().length >  0)
      println("\n hashtag: " + status.getHashtagEntities()[0] + "  l "  + status.getHashtagEntities().length); 
    else println("\nnoHashTagEntities");


    if (status.getUserMentionEntities().length >  0)
      println("\n mention: " + status.getUserMentionEntities()[0] + "  l " + status.getUserMentionEntities().length);
    else println("\nnoMentionEntities");



    if (status.getURLEntities().length >  0)
    {
      //println("\n url: " + status.getURLEntities()[0] + "  l " + status.getURLEntities().length );
      URLEntity[] urlE = status.getURLEntities();


      for (URLEntity url : urlE)
      {

        println("\n URL: " + url.getURL());
        println("\nDisplay URL: " + url.getDisplayURL());
        println("\nExpanded URL: " + url.getExpandedURL());
        println("\nstart: " + url.getStart());
        println("\nend: " + url.getEnd());
      }
    } else 
    {
      println("\nnoUrlEntities");
    }




    println("\n\n============== rawJSON =================\n\n"+status);
  } 
  catch (TwitterException te) {
    te.printStackTrace();
    System.out.println("Failed to get timeline: " + te.getMessage());
    System.exit(-1);
  }

  noLoop();
}

void keyPressed()
{
  if (key == 'n')
  {
    if (i < rawData.length -3)
    {
      i+=3;
      println("\n\n\n\n **** next ****");
      loop();
    }
  }

  if (key == 'p')
  {
    if (i > 0)
    {
      i-=3;
      println("\n\n\n\n **** prev ****");
      loop();
    }
  }
}

eduzal

that's a very comprehensive list of responses. my goal now is to geolocate the tweets, put them in a map. and also get more than 100 tweets for a determined search.

i'm using

try {
QueryResult result=twitter.search(query);
tweets=(ArrayList)result.getTweets();

for (int i=0; i<tweets.size (); i++) {
  Status s=(Status)tweets.get(i);
  String m=s.getText();
  String u=s.getUser().getScreenName();
  Date d=s.getCreatedAt();
  SimpleDateFormat sdf=new SimpleDateFormat("dd/MM/yyyy HH:mm:ss");
  String dt=sdf.format(d);
  String loc=s.getUser().getLocation();
  double lat=s.getGeoLocation().getLatitude();
  double lon=s.getGeoLocation().getLongitude();
  String id=str(i);
  String headShot=s.getUser().getOriginalProfileImageURL();
  println(headShot);
  PImage hShot=loadImage(headShot);

  float x=i%6;
  float y=random(60, height-200);
  float tW=textWidth(u)+10;

  tweet.add(new Tweet(x, y, tW, u, dt, m, id, loc, lat,lon, hShot));
}

}

I'm receiving a null Pointer for the doubles. maybe cause it's gettin a null result for location, lat and lon?

eduzal

i put latitude and longitude under a conditional to get rid of thaerror, but i get another one that's not a programming issue. users are not putting their coordinates on the tweet...

//tweet geolocalization
      String loc=s.getUser().getLocation();
      GeoLocation geo=s.getGeoLocation();
      double lat, lon;
      if (geo != null) {
        lat=geo.getLatitude();
        lon=geo.getLongitude();
      } else {
        lat=1;
        lon=1;
      }

_vk

more than 100 tweets for a determined search.

http://stackoverflow.com/questions/26429965/twitter4j-count-the-number-of-tweets-within-24-hours-return-an-integer/26441898#26441898

https://dev.twitter.com/rest/public/search

from dev site:

The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.

eduzal

twitter4j seems like is missing the geocoordinates. in the API https://dev.twitter.com/overview/api/tweets#obj-coordinates

they have estabilished the Coordinates field as a new coordinates system for the tweet instead of .getLatitude() and getLongitude() that are deprecated now.

_vk

You may like the google group for twitter4j

https://groups.google.com/forum/#!forum/twitter4j

and specifically

https://groups.google.com/forum/#!searchin/twitter4j/coordinates/twitter4j/9Y3SHWRrFxI/gCwIQiqcQJ4J

eduzal

thanks. i'm messing around with the stream API right now. having some trouble dealing with data, specifically with displaying the data. i'm using the same format i posted in a few posts ago.

_vk

Note that Yusuke at tw4j forum is the developer of the library.

I just tested, and status.status.getGeoLocation() still working normally.

eduzal

the geoLocation() is working, but a lot of times is null. My intention is to put tweets in a map, but without the coordinates it wont happen. People on the twitterdev boards are also commenting the drop in the amount of tweets with geolocalization since a change at the mobile UI.

_vk

My self, I don't enable GeoLocation. Someone has talked about, without much precision, use the city or at least country data, to "geo" locate tweets, as most people do use this field.

_vk

And for the map part, I'd recommend you have a look in Unfolding Maps. If you haven't already.

eduzal

I already used the Unfolding Maps in another project. It rocks.

emo

Hi, i have the same problem , most people don't enable GeoLocation. This is my working [example] . How can i reverse the getLocation() to get Lat and Lon to use this instead getGeoLocation()?