We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpIntegration › getting data via http problem
Page Index Toggle Pages: 1
getting data via http problem (Read 2485 times)
getting data via http problem
Sep 19th, 2005, 5:05pm
 
hi i'm trying to intergrate processing with a web database, there's system set up that i can get the variable info, but it gets it the first time fine, but i get null everytime i try to read for more data (but that's obvious because it gets the buffer data)  so i set up only get data when there is data, and if i keep sending to the server commands to get the data, sometimes the data coming back varies in sizes when it shouldn't.  like the string i ask it to read sometimes grow or shrink signifigantly.  this is making it very hard to use that data in processing.  i'm having to split the string into an array to get the right data, but it's hard to know where the data is when the data changes in size and when it splits i can't seem to find the data at the right place in the array.  but that's besides the point, the most important part is when the data changes, like new info is added, it crashes the program.  this is the error

java.net.SocketException: Software caused connection abort: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)

i can't seem to get around this, happens everytime, i think the server is rejecting my http request when there's new buffer info because it thinks i'm spamming the server so it closes the connection, is there a way around? i need to get up to the second updated info from the server.

here's the rest of my code



import processing.net.*;
String [] b;
Client client;
Client iclient;
String newByte;

void setup()
{
 size(200, 200);
 noStroke();
 // Open a TCP socket to the host:
 client = new Client(this, "localserver ", 80);
 iclient = new Client(this, "localserver ", 80);
 // Print the IP address of the host:
 println(client.ip());

 // Send the HTTP GET request:
 client.write("GET /jay/master.cgi?past=1d HTTP/1.1\n");
 client.write("HOST: localserver \n\n");
 iclient.write("GET /jay/active_member.cgi HTTP/1.1\n");
 iclient.write("HOST: localserver \n\n");
 framerate(1);
}

void draw()
{
 background(0);
 // Print the results of the GET:
 if (client.available() > 0) {
   String inByte= client.readString();
   // print((char)inByte);
   //char data[] = char(inByte);
   String  c[] = split(inByte);
   //String h []= loadStrixngs(c);
   //String n = c
   // b = split(c, RETURN);

   // println(c.length);
     client.clear();
 }

 //if(iclient.available()>0){
   try{
     newByte = iclient.readString();
   }
   catch(Exception e){
     println("oops");
   }
   //
   //if(newByte != null){
   try{
     //b = split(newByte);
   }
   catch(Exception e){
   }
   //for(int i=0; i<20; i++){
    println(newByte);
   // }
   try{
     if(int(b[b.length-1]) != 0){
     
      // println(b[b.length-1]);
     }
   }
   catch(Exception e){
   }
   // }
   

//  }
 //iclient.clear();
}
void keyPressed(){
 try{
   iclient.write("GET active_member.cgi HTTP/1.1\n");
   iclient.write("HOST: localserver \n\n");
 }
 catch(Exception e){
   println("oops");
 }
}

Re: getting data via http problem
Reply #1 - Sep 19th, 2005, 5:11pm
 
ok nevermind, i solved it, thanks to charles harvey *

here's the issue:

send only the get command only if it sees new data (sorry if that seems obvious, i've been racking my head all day to figure that out!)

so this very important

if(iclient.available()>0){
   iclient.write("GET /jay/active_member.cgi HTTP/1.1\n");
   iclient.write("HOST: localserver \n\n");
   }
Re: getting data via http problem
Reply #2 - Sep 19th, 2005, 9:54pm
 
actually, now i'm getting inconsistent data, like the source doesn't change but the string i get keeps growing or shrinking when i try to get it, anyone have any idea?
Re: getting data via http problem
Reply #3 - Sep 20th, 2005, 5:14am
 
ok, now i kinda have it figured out, i want to constantly check the data on the server via http instead of loadString because loadString takes up system resources and isn't that fast.

so to prevent the server from closing the socket because of too many requests in short time, i create connection, write and close all in a loop.  the server transmits the right data, but my read function inside the loop never gets any data.

code:

 iclient = new Client(this, "mlist.imgsrc.co.jp", 80);
   iclient.write("GET /jay/active_member.cgi HTTP/1.0\n");
   iclient.write("HOST: mlist.imgsrc.co.jp\n\n");

     int i= iclient.read();
     print(i);

    iclient.stop();

anyone know why this could be?
Re: getting data via http problem
Reply #4 - Sep 20th, 2005, 5:39am
 
ok, after starring at the code with my friends who are server admins, i randomly found the solution.  if the socket is opened up in a function outside of loop, then it works... don't ask why.  i still can't figure it out.


import processing.net.*;
String [] b;
 Client iclient;
String j;
String newByte;
int timeStamp=5;
int counter;
void setup()
{
 size(200, 200);
 noStroke();

openConnection();
 


 
 framerate(1);
}

void draw()
{
if(timeStamp!=second()){
 timeStamp=second();
 counter++;
 
}
 
if(counter>=5){
openConnection();
counter=0;
   }
if(iclient.available()>0){
     String i= iclient.readString();
     print(i);
}

 try{
   b = split(newByte);
 }
 catch(Exception e){
 }

 try{


   String temp[] = split(b[b.length-1], ",");
   if(int(temp[0])!=0){

   }


 }
 catch(Exception e){
 }

}
void openConnection(){
iclient = new Client(this, "mlist.imgsrc.co.jp", 80);
   iclient.write("GET /jay/active_member.cgi HTTP/1.0\n");
   iclient.write("HOST: mlist.imgsrc.co.jp\n\n");
}

anyways.  hope my talking to myself helps others to never go through what i just went through!!

o.O;;

i'm gonna get some sleep, good night everybody!
Re: getting data via http problem
Reply #5 - May 11th, 2007, 4:17pm
 
For those that cant seem to get queries from *some* sites going, try finishing the HTTP query with \r\n and not just \n

A double line break is recognised as the end of a request. Some sites (last.fm stats) require windows-styled end of lines (which are \r\n)

ie

client.write("GET / HTTP/1.0\r\n")
client.write("\r\n\r\n");
Page Index Toggle Pages: 1