code that loads a html file, finds characters on the page and returns their values.

Snapper4298

I am a member of codepen.io and on my profile page it shows Followers: X and Following: Y. I want to write some code that loads the profile page into javascript, then search through the file to locate these "X" and "Y" fields then returns both values so that my profile code shows the up-to-date values stored on the codepen website. Have a look at the profile page here to understand what I am trying to do. Currently I have to manually update the HTML content every time I follow someone or someone follows me: https://codepen.io/brettmitchell

This may be a big ask as I am new to both processing and P5* so any fix, help or suggestions would be very much appreciated. Thank you for taking the time to review.

jeremydouglass

I want to write some code that loads

I get that you want to load the values from https://codepen.io/brettmitchell . What and where is this code that does the loading going to be running? In a p5 sketch? And is that sketch going to be hosted on your personal website?

Or are you trying to scrape codepen from somewhere within codepen?

Snapper4298

Thanks Jeremy. The code is a sketch found here: https://codepen.io/brettmitchell/pen/MyWBmO Currently to code is written with HTML and CSS so I thought I could add a javascript program to search the page it's loaded from, find the values from Followers: and Following, then when the script gets those values, push them into the HTML file to overwrite the (currently manually updated) values. So yes I am trying to scrape codepen from within codepen to show the real time values that codepen shows in the Following: and Followers: fields. Thank you so much for taking the time to review this problem. Brett.

jeremydouglass

@Snapper4298 --

I'm not an expert, so hopefully somebody else can answer your question in detail.

As described this might be possible if you are trying to load other data from codepen.io -- it wouldn't violate the same origin policy.

• https://en.wikipedia.org/wiki/Same-origin_policy

Otherwise, some aspects of web browsers and JavaScript are designed to prevent doing what you are describing in that way because it is "cross-site scripting":

• https://en.wikipedia.org/wiki/Cross-site_scripting

Very beautiful demo, by the way.

Snapper4298

I suspect what I'm trying to do won't violate their terms, but I am certainly not an expert and did not know about the info you provided which is very helpful thank you. I just thought this would be a nice addition to the code so that any time someone visits, they get the up to date information based on the database provided by codepen. I'm wondering if there is another way to do this, maybe just pick up the characters from the page that's displayed, but again, it seems to lean towards the "you're not allowed to do that" rather than "you can" I'm still learning so I may figure an acceptable way around it but I think for now, it's going to have to be put in the pending tray. Thanks for your help Jeremy, I really appreciate your advice. And thanks for your really kind feedback ^_^

jeremydouglass

If you do attempt it, here is one example of a discussion to get started:

http://scraping.pro/web-scraping-with-javascript-load-html-page/

Note that this person was in the end not able to do it because they were trying to load things from two different sites. If you are trying to load a page from codepen from a different page on codepen, then it might work for you.

Snapper4298

Thanks again dude, your explanation and links are news to me, and I can see in the last link some reading that will give more food for thought. Appreciate all your help and advice Jeremy, kindest regards, Brett