In the process of working on a project that involves loading data and images from a cookie protected server, and I thought I'd share some of my findings. We are working with a custom installation of Jive ( http://www.jivesoftware.com/ a corporate social networking system) on our clients server. 

I need to load activity streams and and other data feeds from their system. The catch is that they have overridden Jive's regular way of logging in users and have their own OneLogin system. So when you go to their Jive install, you are redirected to a OneLogin page that you enter a password, then redirected back to Jive which checks the cookies and lets you access information. 

Jive has a nice rss system for all the data, however for security they have turned that functionality off, so I am using the REST api to get json files of the data to be loaded. But, the REST api also requires a user to be logged in. This requires me to pass a cookie (JSESSIONID) into my requests. This works great.... about 1/4 of the time :/ so to get around this limitation, I ended up asking for the json file until I get it; if it fails, it tries again. This pounds their server with requests, which is undesirable, but ends up working pretty reliably. 

Another consideration is that I need to update the content as regularly as possible to feel like a live view of their activity. So all this loading happens on a thread that I can keep calling ever 5-10 seconds without disrupting other parts of the application. 

Part of these Activity posts in Jive is the user's avatar image, and sometimes images they have posted. Initially I thought 'no big deal, loading images is easy in processing,' but, the url's to the images that is loaded in the json file are not .jpg/.png/whatever, they are server requests, which the server responds with base64 strings. So I was getting these strings and using a byte array to translate them to PImages. Then it gets frustrating. Most of the images I was trying worked great, but then I came across a few that had malformed headers. they had both jpeg and exif data, which apparently causes errors in the BufferedImage IOreading I was using.  my workaround for this was to simply save the byte array to a local .jpg using saveBytes(), then load that saved image back in. For some reason this works great! :) 

Anyway just thought I would share my findings and some code snippets in case anyone else was facing similar issues. so here's my cookie-passing html loading code:

1. JSONObject loadJson(String query, Boolean verbose) {
2.  JSONObject json = null;
3.  JSONArray dataArray = null;
4. 
   
5.  try {
6.    setupClient();
7. 
   
8.    HttpGet getRequest = new HttpGet(query);
9.    getRequest.addHeader("accept", "application/json");
10. 
    
11.    HttpResponse response = httpClient.execute(getRequest);
12.    org.apache.http.params.SyncBasicHttpParams params = (org.apache.http.params.SyncBasicHttpParams) httpClient.getParams();
13. 
    
14.    if (response.getStatusLine().getStatusCode() != 200) {
15.      if (response.getStatusLine().getStatusCode() == 401) {
16.        return null;
17.      } else {
18.        throw new RuntimeException("Failed : HTTP error code : " + response.getStatusLine().getStatusCode());
19.      }
20.    }
21. 
    
22.    InputStreamReader isr = new InputStreamReader((response.getEntity().getContent()));
23.    BufferedReader br = new BufferedReader(isr);
24. 
    
25.    String output = "";
26.    String out;
27. 
    
28.    while ( (out = br.readLine ()) != null) {
29.      output += out;
30.    }
31.    int jsonStart = output.indexOf("{");
32.    output = output.substring(jsonStart, output.length());
33. 
    
34.    try {
35.      json = new JSONObject(output);
36.    } catch (JSONException e) { 
37.      println("JSONException in loadJiveJson JSONEx : "+e);
38.    }
39. 
    
40.  } catch (ClientProtocolException e) { 
41.    e.printStackTrace();
42.  } catch (IOException e) { 
43.    e.printStackTrace();
44.  }
45.   
46.  if (verbose) {
47.    println("json here: "+json);
48.  }
49. 
    
50.  return json;
51. }
54. void bakeCookies () {
55.  cookieJsess = cookieMonster("JSESSIONID", jsession, "example.server.com");
56. }
57. BasicClientCookie cookieMonster(String c, String v, String domain) {
58.  BasicClientCookie cookie = new BasicClientCookie(c, v);
59.  cookie.setVersion(1);
60.  cookie.setDomain(domain);
61.  cookie.setPath("/");
62.  
63.  return cookie;
64. }
65. 
    
66. void setupClient() {
67.             bakeCookies();
68.    //create the httpClient
69.    httpClient = new DefaultHttpClient();
70.    //add the cookie to the client
71.    httpClient.getCookieStore().addCookie(cookieJsess);
72. }
73. 
    

Which you call by loadJson("rest/string/here", true); with a rest string as the first param.

I leave my httpClient open at the end of this loading so I can continue using it in my future update calls and for my image requests, which are done like this:

2. PImage loadProtectedImg(String url) {
3.    PImage pimg = new PImage(); 
4.    String output = "";
5.    try {
6.      HttpGet getRequest = new HttpGet(url);
7.      HttpResponse response = httpClient.execute(getRequest);
8. 
   
9.      byte[] bytes = EntityUtils.toByteArray(response.getEntity());
10.      ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
11.      saveBytes("image.jpg", bytes);
12. 
    
13.      //BufferedImage jimg = ImageIO.read(bis);
14. 
    
15.      pimg = loadImage("image.jpg");
16.      //pimg=new PImage(jimg.getWidth(), jimg.getHeight(), PConstants.ARGB);
17.      //jimg.getRGB(0, 0, pimg.width, pimg.height, pimg.pixels, 0, pimg.width);
18.      //pimg.updatePixels();
19. 
    
20.      if (response.getStatusLine().getStatusCode() != 200) {
21.        throw new RuntimeException("Failed : HTTP error code : " + response.getStatusLine().getStatusCode());
22.      }
23. 
    
24.    } catch (ClientProtocolException e) { 
25.      e.printStackTrace();
26.    } catch (IOException e) { 
27.      e.printStackTrace();
28.    }
29.    return pimg;
30.  }
31. 
    

Which returns the pimage to the string that was passed. the commented bits show how I was previously converting the byte array to a PImage, but it seems like saving it locally then loading back seems to be more robust. 

Sorry if that got a bit rant-ish, been working around the limitations of their REST api and specific installation for a few days now. hopefully some of this helps someone. Or if you have any suggestions on any of the process I'd love to hear!

ak