Timeout / Async Issue with LoadJSON callback

edhebert

Hi-

I'm trying to retrieve lunar data from the US Navy's Astronomical API.

I'm trying to retrieve the data asynchronously when a date is chosen and the "submit" button is pressed. I'm using a callback function to (currently) console.log the retrieved data object. http://codepen.io/edhebert/pen/gmQBOE

If I click the button 20 times, I'll get a data object back some of the time. Other times, I receive nothing. I'm not sure whether the loadJSON is timing out, even though I'm specifying a callback to handle this asynchronously?

Is there something I'm missing? Is there a way to manually set a longer timeout in P5's loadJSON? The reference docs are fairly skimpy for loadJSON.

limzykenneth

The JSON resource link I extracted from your code is as follows "http://api.usno.navy.mil/rstt/oneday?date=06/12/1983&loc=Cambridge, MA" let me know if I got it wrong but for me this link is not loading at all, not even with a browser or an API request app (postman), which means the server got the request but is not responding at all or in a timely manner. Have a check that the API is working and the URL is correct.

loadJSON() uses Fetch API in the background and I don't think the API has provided a mechanism for setting a longer timeout yet.

GoToLoop

I was only able to visit that site & grab its ".json" file using Tor Browser!!! @-)
At least yesterday. Today that site doesn't seem to exist anymore! ~:>

BtW @limzykenneth, in latest p5.js version 0.5.8, loadJSON()'s implementation w/ Fetch API is now failing to load files via "file://": :-&

Fetch API cannot load file:///. URL scheme must be "http" or "https" for CORS request.

That error message shows up under SlimJet (Chrome-based browser) w/ --allow-file-access-from-files active!

Maybe you should turn off CORS request when fetching files locally under "file" URL scheme. :-??

But surprisingly, Firefox-based browsers still don't care about it at all and simply work even under p5.js 0.58! 8-}

edhebert

Wow - how about that?! The site is down. Maybe they got defunded! :)) Anyway, thank you for taking a look at it for me.

jeremydouglass

@edhebert -- it could be a reorganization due to a website redesign? Good luck finding it if it is still out there.

Here is the current USNO homepage and the last wayback copy of the API page --

• http://www.usno.navy.mil/
• http://web.archive.org/web/20161109202517/http://aa.usno.navy.mil/data/docs/api.php

kfrajer

I tried different calls from the API and nothing... You can contact them or just wait few more days and see if the system comes back to live again. Still, the question remains... how to handle timeout or if it is possible to set up the timeout in the first place.

Kf

limzykenneth

@GoToLoop Sorry for late reply, I don't check the forums so often. There's a bug in loadJSON() (and httpGet as well) in 0.5.8 that has since been fixed, 0.5.9 should be working as expected. Also, I don't think loadJSON() is supposed to work with file://, it should need a local server in any case, it working in Firefox is unintended.

It would be great if anyone can have a try at the lastest build of the library before 0.5.9 is released to catch any more bugs before it came out.

GoToLoop

Also, I don't think loadJSON() is supposed to work with file:// ...

Well, loadImage() always did and still do under v0.5.8. And up till v0.5.7, loadJSON() did as well!

It working in Firefox is unintended.

Firefox has never blocked scripts from running, even when served as file://. \m/

limzykenneth

I'm quite sure it never did work or never was intended to. Some reference: https://github.com/processing/p5.js/wiki/Local-server https://github.com/processing/p5.js/issues/822

You are right in saying that Firefox is more lenient but that is limited to Firefox only and I'm actually surprised they haven't patched it out. Chrome and other webkit browsers has disabled access to file:// origin for security reasons years ago and I agree with that, you don't want a website unlimited access to your file system. This behaviour is built into the browsers itself and CORS policy is determined by the server, I don't think there's anyway you can bypass it client side (other than JSONP).

GoToLoop

Firefox is more lenient but that is limited to Firefox only and I'm actually surprised they haven't patched it out.

@limzykenneth, there's nothing to be patched out at all! :-@
That's a Mozilla's decision. All Firefox family browsers have never lost this local file:// feature! \m/

What amazes me is the fact that almost no 1 knows about it, even though that's an eternal feature from a very old browser. Much older than Chrome itself! 3:-O

Seems like some kinda censorship psycho-blockage that we collectively have decided to forget about!

Why isn't this "easy-as-apple-pies" Firefox feature even listed as 1st option in: :-$
https://GitHub.com/processing/p5.js/wiki/Local-server

GoToLoop

Chrome and other webkit browsers has disabled access to file:// origin for security reasons years ago...

I'm quite sure it never did work or never was intended to.

So I shoulda been hallucinating that my fav SlimJet browser (Chromium-derived) has been running all my sketches all these years w/o any external gimmicks such as setting up some local server. 8-}

It's indeed true that most other browsers have turned file// off for scripts.
However, that's only on the surface! :>

For Chromium family browsers, there's a secret command line shortcut which re-activates that feature again! You can read about it below: <:-P
http://Chrome-allow-file-access-from-file.com/

GoToLoop

Now that file:// subject is clarified, I've got a clue for the fix I guess.
Maybe after checking out for file://, crossOrigin should be set to 'Anonymous', similar to how loadImage() did it: *-:)

https://GitHub.com/processing/p5.js/blob/master/src/image/loading_displaying.js#L99-L101

if(path.indexOf('data:image/') !== 0) {
  img.crossOrigin = 'Anonymous';
}

Sorry I dunno much about the internals on how to pull that out.
What I know is that in v0.5.8, loadJSON() doesn't load local files, while loadImage() still works! :\">

limzykenneth

@GoToLoop For the wiki, I don't think listing firefox as a potential get around is a good idea, it actually can be very counter productive especially when someone is done with their project and published it without using proper URLs and have their sketch fail everywhere but firefox. Spinning up a web server can give you a level of certainty that your code will work when published online. Plus, I don't think XHR works without a local server, regardless of the origin of the requested file.

In most cases, it is actually possible to get around not using the file:// origin at all. As long as your local resources are in the same folder, which is always a good idea than to have things everywhere on your computer, you can request then via relative paths. Relative paths are shorter, easier to understand, have no cross origin issue whatsoever (it is literally the same origin) and I am quite sure it works on 0.5.8 (I tested only with relative paths when implementing this)

GoToLoop

... especially when someone is done with their project and published it without using proper URLs and have their sketch fail everywhere but firefox.

There are absolutely no source code modifications for whichever approach is picked to run the "index.html" at all!

Whether we choose to run it via some Firefox-based browser, or some Chromium-based 1 w/ active parameter "--allow-file-access-from-files", a local or remote hosting server, etc., the code is exactly the same! [-(

It's just that for a Chromium-based browser, p5.js v0.5.8 is throwing an exception for file:// protocol for loadJSON(), which had never happened before! :-<

All of the other loading functions, which aren't Fetch API-based, like loadImage() for example, continue to work as they always have. :-\"

As long as your local resources are in the same folder, ...
... you can request them via relative paths.

Obviously, all file resources are within the confines of the "index.html"'s root folder.
Otherwise, the loading would fail even when using the local server approach! 8-X

GoToLoop

... it is actually possible to get around not using the file:// origin at all.

It's not a matter where the files are located! They're in the same folder as "index.html" btW!

loadJSON() has simply stopped working in my SlimJet browser if I attempt to run "index.html" w/o some server in p5.js v0.5.8! ~X(

• For the experiment, install any Chromium-based browser if you don't have 1 yet.
• Add parameter "--allow-file-access-from-files" to its shortcut:
  http://Chrome-allow-file-access-from-file.com/
• Or alternatively, execute it from terminal passing that argument.

Now create this "index.html" below for p5.js v.0.5.8 version:

<meta charset=UTF-8>
<script async src=https://CDN.JSDelivr.net/p5.js/0.5.8/mainfile></script>
<script defer src=RiverNames.js></script>

Download files "RiverNames.js" & "rivers.json" from:

1. https://ThimbleProjects.org/gotoloop/91698/RiverNames.js
2. https://ThimbleProjects.org/gotoloop/91698/rivers.json

You can see it running online at this link below: O:-)
https://ThimbleProjects.org/gotoloop/91698/

Having your Chromium-based browser already opened, w/ active parameter "--allow-file-access-from-files", drag the "index.html" into it.

It's expected to fail under v0.5.8: "Fetch API cannot load file:/// ... URL scheme "file" is not supported."

Now rename v0.5.8 to v0.5.7 within "index.html" and refresh the program page in the browser.
It's expected to successfully loadJSON() "rivers.json" now! \m/

GoToLoop

For the wiki, I don't think listing Firefox as a potential get around is a good idea, ...

@limzykenneth, it's beyond my comprehension the why not, given it's the most convenient & easier approach ever! \m/

If Firefox is the default browser, merely double click "index.html" and we got the whole sketch running! :)>-

Or we can right-click and choose Firefox to run it.
Or even drag "index.html" into it if it's already opened. ;)

There's no comparison to the hassle of dealing w/ terminal's command line after installing Python or Node.js' NPM! #:-S

Once we've got "--allow-file-access-from-files" shortcut for a Chromium-based browser, it's even better! That's exactly what I use here! =P~

Seems like the very thought of spreading the word on how to run ".js" programs w/o terminal commands is sinful heresy! >:)

limzykenneth

@GoToLoop loadImage and loadJSON may look similar but they are in fact totally different. loadImage is just wrapping native canvas API and is akin to creating an img tag and setting the src attribute. loadJSON from implementation well before 0.5.8 has always been AJAX/XHR based. loadImage is in a way more akin to a JSONP request than it is to loadJSON in that it doesn't require any cross origin request shenanigans while XHR always did. So what works for loadImage doesn't translate to loadJSON.

---

I don't know for sure why it worked on SlimJet for you before and not now. For 0.5.7 and before, loadJSON is using a library called reqwest, it basically wraps a bunch of native XHR calls, while 0.5.8 uses fetch API which is just a new native way to make XHR calls. If it failed because of the switch to fetch API then I suspect that whatever exception the flag provide, it did not cover the new API (another flag?). If you have a look at the implementation of httpDo (which loadJSON calls internally), you can see that it is thinly wrapping the native fetch API for no other reason than because the fetch API is already extremely simple. If you can find something that we can change on fetch API call to make it work on your chromium with flags and not affect other browsers, it can be considered to be added into the library. I recommend just look at the Fetch API itself if you can, I think it is quite easy to get started with.

---

Recommending just using firefox is potentially ignoring all other browsers. Not just p5.js but all js projects and most other software projects are made to be run on a variety of environment. If we encourage users to just develop on firefox because it solves one particular problem, they missed out potential bugs their code might have on other browsers and we miss out on fixing those bugs because nobody is using other browsers and we don't know about them until somebody's published sketch messed up on their impotant show because someone else uses Chrome or Edge to access it. There is no recommendation of it because it is not recommended.

It's less about how to run js or what browser should be used etc. but rather making sketches that can be easily written by anyone and shared to everyone. Imposing the need to use firefox or chromium with custom flags defeat the purpose. Your sketch should ideally run on as many environments as possible or else we could have ignored numerous platform/browser specific bugs and just say "run it on firefox, it'll be fine".

Command line can be intimidating, at least at first, that's why there was the p5.js-editor project which spins up a server whenever you want to run a sketch and why there are lots of ongoing work on p5.js web editor to get it out of alpha.

---

I mean all these are my opinions and intepretations which in no way means they necessarily reflect p5.js's opinion. If you still feel strongly about this, do open an issue on github so more people can pitch in on this discussion.