loadStrings() question

lobodelmar

Hi everyone, I have been trying to make a simple web app using the promising p5.js library, but I am stuck and don't know how to move forward. It's really a mystery to me why the following lines don't work. I am just trying to load a simple text file and I can't. It keeps saying that it's empty when obviously it is not. I am not doing anything different from the default example show on p5js' website: https://github.com/lmccart/p5.js/blob/master/examples/async/loadStrings_callback/sketch.js

In fact I also tried the default example, replacing the string to load with: http://quarx.asfa.gr/data.txt and nothing. My server shows that it is sending the string... I don't know what p5js is doing. Any idea? Code follows below.

var result;
var processedResult;

function setup() {
  createCanvas(windowWidth, windowHeight);
  var URL = "http://" + "quarx.asfa.gr/data.txt";
  result = loadStrings(URL, getHistoryState);
}

function draw() {
  background(0);
  ellipse(50, 50, 80, 80);
  textSize(50);
  text(processedResult, 300,300);
}

function getHistoryState()
{
  processedResult = result;
}

lobodelmar

For some reason it does not allow me in the URL on line 6 to just put http://quarx.asfa.gr/data.txt That's what it should say, so if you try the code make sure you replace the part of line 6 after the "=" sign with the above link

GoToLoop

Split "http://" from the rest and concatenate them via + operator:
"http://" + "quarx.asfa.gr/data.txt";

quark

Is the text file on the same server/ in the same domain as the web page? If it isn't then that will be the problem

GoToLoop

var result, processedResult;

result = loadStrings(URL, getHistoryState);

function getHistoryState() {
  processedResult = result;
}

I still dunno much about JS yet but... that callback above doesn't make any sense!!! (~~)
Much probably variable result = null after loadStrings(). You just make processedResult = null too!

After some search, I've found loadStrings()'s reference here: http://p5js.org/reference/#/p5/loadStrings

loadStrings(filename,[callback])

[callback] Function: function to be executed after loadStrings() completes, Array is passed in as 1st arg.

I suppose we gotta have a declared parameter in the callback? :-/

function getHistoryState(lines) {
  processedResult = lines;
}

Well, that's far as my knowledge goes for JS! X_X I prefer to code in "CoffeeScript Mode" instead! :P

lobodelmar

@Quark, it should not matter if it is on the same server/domain or not.

@GoToLoop, thank you for your long and detailed answer. Unfortunately, what you recommended did not fix the problem. As I mentioned I tried the

default example from the p5.js website replacing the link with my link pointing to data.txt but still nothing. It basically refuses to load a file that is not local. I am busting my head over this.

Any oter ideas?

lobodelmar

@Quark, you are right. I tried it again with a file on the same domain and it worked. But, what I don't understand is if I want to use data from another site (using let's say its API) how would I do it? Would I not use one of these functions? How would I play with the API of another service?

GoToLoop

Well, after revisiting the lil' knowledge I had about JS's Function's arguments{} below:
http://javascript.info/tutorial/arguments

I've come up w/ this latest attempt. Untested though:

// forum.processing.org/two/discussion/6942/loadstrings-question
// javascript.info/tutorial/arguments

var lines;

function setup() {
  createCanvas(300, 200);
  smooth(4);
  frameRate(1);

  loadStrings("http://" + "quarx.asfa.gr/data.txt", function() {
    lines = arguments[0];
  }
  );
}

function draw() {
  print(lines[0]);
}

You should also take a look at preload(): http://p5js.org/reference/#/p5/preload
That approach has been working for "JavaScript Mode" a long time already! :D

lmccart

the url needs to be a single string, like you would grab from your address bar (ex: "http://google.com/data.txt", your example above has extra "<a href=" etc.)

there are two examples of the two different ways of using loadStrings (preload or callback) here: http://p5js.org/reference/#p5/loadStrings.

lobodelmar

@lmccart & @GotoLoop: I started with the examples on the p5.js site first and I looked into the two ways of doing things.

Here is the default example using JSON from p5.js. URL2 is the link given in the example and URL1 is my link from my service (note: sorry for the links in 2 parts but the forum software keeps turning them into links rather than leaving them as code).

var result;
var URL1  = "http://" + "terataki.x64.me:9080/history";
var URL2  = "http://" + "api.openweathermap.org/data/2.5/weather?id=5128581&units=imperial";

function preload()
{
    result = loadJSON(URL1);
    print('In preload(), the result has not finished loading: ');
    print(result);
}

function setup()
{
    createCanvas(400,100);
    textSize(18);
    textAlign(CENTER);
    fill(0);
    noStroke();
    print('In setup(), here is the result: ');
    print(result);
    text(result);
}

It still does not work. Even though URL1 is obviously right and it serves a JSON object, when I run it through p5.js/Javascript it gives me an error on the loadJSON line saying "SyntaxError: missing ; before statement". I don't see much difference in the syntax to the object URL2 is serving.

Any ideas? That's were I started from originally and I switched to loadString() because I could not figure out what's wrong with the above code and/or JSON object I am serving. Any ideas?

Thanks :)

GoToLoop

The URL needs to be a single String,...

It is a single URL String after the + concatenation! :-B

There are 2 examples of the 2 different ways of using loadStrings()...

Dunno whether it's my browser, but I can't see any examples provided there! :-SS
It's just the function's description w/o any further help! [-(

P.S.: It is indeed my main browser's fault. Some config here is blocking the examples before the "Description" part!

quark

it should not matter if it is on the same server/domain or not.

I tried it again with a file on the same domain and it worked

A JS application that attempts to load resources from another e.g. embedding video from youtube is called a 'cross-origin' application. In some APIs this is not allowed because of security concerns.

I am no expert in this area, if fact I would describe my self as a novice but I understand there is a technology called CORS (Cross-Origin Resource Sharing) but I think you will have to do some Googling.

lobodelmar

See my earlier answer above. I resorted to using JSON, but it complains again (different error this time). Really perplexed! (thanks guys/gals for your time)

GoToLoop

I've run my code via p5.js site, editing 1 of the loadStrings()'s examples, and it seemed to work: >-)

//var URL = "http://" + "api.openweathermap.org/data/2.5/"
//+ "weather?id=5128581&units=imperial"

var URL = "http://www.corsproxy.com/quarx.asfa.gr/data.txt";

var lines, length;

function setup() {
  createCanvas(1000, 800);
  smooth(4);
  frameRate(1);

  loadStrings(URL, function() {
    lines = arguments[0][0];
    length = arguments.length;
  }
  );
}

function draw() {
  if (lines) {
    text(lines, 10, 30, width, height);
    text(length, 10, 10);
    noLoop();
  }

  else print(lines);
}

lobodelmar

@GoToLoop, thanks for your time. I tried your example. It works with that one. Why does it not work with: URL = "http://terataki.x64.me:9080/history/" I really don't understand. Any ideas? The link obviously works. tp

GoToLoop

Sorry! Me neither! :-&

lobodelmar

Thanks @GoToLoop. Any ideas @lmccart ?

lmccart

if I understand correctly, it works when you edit the example on the p5js.org site but not locally? are you running a local server to view your files, are you using the p5 editor, or are you just opening the html files in browser? one problem could be that you may need a local server running in order to load external files. this tutorial explains a little more about how to start one if you're not already. https://github.com/lmccart/p5.js/wiki/Local-server

GoToLoop

Both URLs are remote & reachable! But somehow "terataki.x64.me" fails while "api.openweathermap.org" succeeds! @-)

lmccart

ohh I see... seems like this could be a cors issue then. let me look into it some more this weekend.

lobodelmar

@lmccart: Exactly! Both the JSON and loadStrings() examples on the p5.js site irrespective of the method (preload vs. callback) do not function with any of my services/data. I tried it many times. Both are servers run by me and are otherwise accessible. Here are both examples I have setup just to test them with increasing simplicity as you go down the list:

URL_A (json): "http://terataki.x64.me:9080/history/"

URL_B (simple text feed): "http://terataki.x64.me:9080/simplerHistory/"

URL_C (actual text file): "http://quarx.asfa.gr/data.txt"

It it were a cors issue would it not also fail to access that weather API used in the json example on the p5.js page? Thanks for looking into this @lmccart and for all your support @GoToLoop. Much appreciated :).

lmccart

@lobodelmar: the problem is that this server isn't packaging the data with these headers:

Acccess-Control-Allow-Origin: *

Acccess-Control-Allow-Headers: *

so when it gets back to the browser it's blocked. a couple options: if you have control of the server, you could modify it to add these headers in the response.

alternatively, http://www.corsproxy.com provides a little workaround that intercepts the data as it's coming back and adds on the headers before it gets to your browser. so rather than putting in the string you have, you could put in something like: "http://www.corsproxy.com/http://quarx.asfa.gr/data.txt"

not sure how this holds up in a production setting / what sort of traffic it supports though.

GoToLoop

Edited my example to use proxy-fied URL "http://www.corsproxy.com/quarx.asfa.gr/data.txt" and it worked! <:-P

lobodelmar

@lmccart, just came back from a quick holiday. Thanks for looking into this.

I do have control of those servers and I'll add the headers. I didn't even know that these existed. I have to look into how to do it. I found a CORS addon for flask which I am using. Will report progress here :). Thanks again. You solved a big mistery for me and I got to learn about headers! ( @GoToLoop, thanks for staying on top of this too :).

lobodelmar

@lmccart and @GoToLoop , I managed to find an elegant way to solve it by using a flask extension called flask-cors. You can install it using pip and then you just add 2 lines of code in your program and it works :).

Here is some info: https://pypi.python.org/pypi/Flask-Cors/ and a bit more here if needed: http://flask-cors.readthedocs.org/en/latest/

remember to import it (not mentioned in the above links) from flask.ext.cors import CORS

hope it helps marinero