[Proposal] - Development Environment: JavaScript Prototype

dhruvdutt

Development Environment: JavaScript Prototype

Dhruvdutt Jadhav
GitHub: dhruvdutt | Website: dhruvdutt.github.io | Email: dhruvdutt.jadhav@gmail.com

---

Introduction / Abstract

The Processing Development Environment (PDE) has more than 250k unique users and is currently built in Java using custom components that make use of Java's “Swing” library. This proposal aims a prototype replacement for the PDE itself with a JavaScript-based solution.

---

Background

The Processing IDE has 2 major components: - Code Editor: 'frontend' - Compiler/runner: 'backend' that converts processing code to pure Java code, compiles and runs the program.

The current PDE user interface is built in Swing and the code editor is a standard text view. When we click "run", the PDE converts Processing code to pure Java source code and then there's a 'runner' component that does the job of actually running the Java program. Everything except this part will need to be basically written from scratch.

---

Project Description

The prototype we're planning would be a native desktop app built using web technologies like JavaScript. It would have a Processing code editor and will allow the user to compile and run the code through the Java compiler or runner.

---

Approach

For the prototype, we can imagine a basic version of VSCode or Atom like editor which lets you code in Processing language.

We will use a cross-platform framework, integrate a code-editor and rewrite the web components. More details are given below in technical specifications.

For the parsing the code, we'll need to pass it through the runner/compiler which resides inside the monolith PDE repo. Processing does have a CLI (command line tool) that basically takes a sketch folder and runs it in a "headless" version of Processing (without launching the full PDE). We can leverage or tweak that to our use for the prototype.

The UI editor and components can stay JavaScript based and will communicate with the Processing CLI to run/pause/stop the sketch. We can use Node.js Child Process to accomplish this and use the stdout from CLI through the child process (inside Electron) and then pop it.

An alternative to using CLI is to actually translate the sketch code to pure Java, compile and run it as a standard Java program. Processing's core libraries are standard java libraries, so it's possible to run it as a pure Java program too. But it will be significantly more work than using the CLI directly. We would need to use something like node-java to connect with existing Java APIs and form a bridge but it might take a while in itself to figure out and stitch all the internal libraries together before doing any communication.

I think right now, we can focus on the exposing more and more features via CLI as per the requirements of the prototype. This way, we can keep developing the current PDE and the improving CLI together, and at a future stage when things are stable enough we can entirely ditch the Swing components.

We can keep the CLI as a bridge to only do the essential compilation and parsing of the Processing code and implement the rest of the features within the new prototype gradually.

Apart from this, the PDE also supports error reporting, code completion (intellisense), refactoring features too. These features currently use other 3rd party Java libraries. So porting these features over might be a bit tricky initially. These could be implemented using libraries like Tern.js but we can keep this at the back of our minds and cover this once we get the initial prototype ready.

Building the complete PDE would definitely take lots of effort. I'm seeing this proposal as building a strong bedrock foundation on which we can scale and iterate incrementally.

One of the goals to build this new prototype is moving away from Java's Swing components code and embrace JavaScript for frontend and UI components. JavaScript is best known for it's terse declarative, flexible and dynamic behaviour which makes it easy to pick and faster to iterate. It will also attract a more active open-source community and will lower the bar of entry for contributions.

---

Technical Specifications:

Base Framework

A base framework that allows building native desktop applications using web technologies like JavaScript, HTML, CSS.

We'll use Electron as our base framework as it's one of the most popular projects used for building native applications and is backed by GitHub. Tools like VSCode, Atom are built on Electron which has similar use case as us.

Code Editor

For taking Processing code as input from user, we'll need a code editor that runs on web technologies (or an in-browser code editor).

There are again a bunch of choices here but I've picked CodeMirror which is one of the oldest and most actively maintained code editors. It has support for tons of languages out-of-the-box, has an active community forum and has way more users and downloads (npm downloads) even after combining all of its alternatives. Live use cases similar to us are Adobe Brackets and Chrome DevTools (in case we plan to build some console utilities at a later stage). Also, our very own Processing family's p5.js web editor uses CodeMirror as well.

CodeMirror has out-of-the-box support for Java which we can start off with and it also features something called "mode system" that allowing supporting custom language syntax highlighting and markup which we can use for our Processing code inside the editor. There's also a 3rd party library called CodeMirror Grammar which transforms a JSON grammar object into a syntax-highlight parser for CodeMirror. We can also look into it.

In terms of code editors, there are alternatives like Monaco which is built by the VSCode team at Microsoft. It's the go-to editor if the target users would be working primarily for JavaScript inside the editor. Monaco doesn't support many languages out-of-the-box but they have very good docs about writing custom syntax and languages.

UI Components

"The current custom components make use of Java's “Swing” library, which is even more outdated now than when it was first released".

We'll need to rewrite the UI components from scratch which should be fairly straightforward and can be concretely planned. We'll try to use native things as much as we can, like the menu bar which would be native and look platform specific (similar to how VSCode, Atom does). The alternative is we build a custom menu bar with custom styles that would be inline and sticky in the header at the top of the IDE. (similar to Google Docs/Sheets menu bar)

Frontend Library

We'll also need a frontend library to connect and render all the UI interfaces and for managing application state. I've chosen React.js as it provides a simple yet flexible API for building user interfaces, follows components based architecture pattern, wildly popular, actively developed, backed by Facebook and has one of the biggest JavaScript communities. Redux: For application state management, Reselect for memoizing selectors if needed. React and Redux is also used in the p5.js web editor.

Design

We can take inspiration from the p5.js web editor design looks minimalistic and beginner-friendly. We can port these styles directly to keep things consistent and tweak a few things like Processing's dark blue colour theme and logo. Also, it is recommended to create a standalone design kit components library in future that would contain common components styled along with a way to inherit/extends them. This can be consumed by both p5.js web editor and the new PDE or any future web-based project and can serve as a design spec for all Processing web projects.

Testing

Testing is the roof of any app. As the app grows, testing will help us scale and iterate faster. Lacks of tests has been one of the issues with the current PDE. It's hard to maintain them with limited resources. For the new prototype, most of the internal methods and APIs will keep on changing/refactoring. It doesn't make sense to follow TDD from Day 0. We would try to write modular code in a way which facilitates easy testing at a later stage once the internal APIs and methods are in a stable state.

Unit testing would be good to start off with. There are testing frameworks specialized for each type of tests but Jest works for most cases. It is backed by Facebook and is wildly popular in React - JavaScript ecosystem. This is something which can be revisited at a later point in time.

Localization, Internationalization and Accessibility

Since the PDE would be targetting a wide audience, l10n, i18n and a11y can't be neglected and are one of the most crucial things to support. This is also something in backlog at the moment till we decide the priority of these in context of the current prototype.

App Distribution

Currently, the PDE is only distributed via the downloads page and GitHub releases page. The PDE prompts you with an update message if a new version is available.

With the help of Electron, we can distribute it to Mac App Store, Windows Store and also do the update automatically in the background (similar to how VSCode, Atom updates itself). There would be platform specific executables/msi/zips/tgz for sure along with this.

Although app distribution as a whole can be addressed at a much later stage in the project, we should keep in mind the possibilities which could be unlocked.

Language

For the programming language, we'll be using JavaScript/ECMAScript ES6/7 with Babel (transpiler/JS compiler) which will allow us to write modular code using latest syntax, semantics.

I'm also open to and have experience using statically typed languages like TypeScript which might help to get Java developers onboard but IMO we should stick with JavaScript.

Package Manager

For managing and installing all the packages and dependencies inside our project, we'll use npm which comes bundled with Node.js We can also use Yarn alongside npm.

Tooling

We'll also using some additional tools along with all above libraries:

• For module bundling, we'll go with webpack. It will help us using ECMAScript modules, tree shaking (dead-code elimination), uglification, minification, code-splitting, etc.

• Linting: ESLint and Prettier to maintain code consistency and formatting code.

• Continous Integration: Setup Travis CI with npm scripts for automatic testing, linting, etc.

• Other Internal tools: JSDocs, pre-commit hooks with Husky, lint-staged, etc.

---

Summary

• The whole technical stack comprises of most popular and battled-tested tools having similar live use-cases as ours. Personally, I'm acquainted and have worked with all of the technologies listed.

• The technologies are very similar to the current p5.js web editor. This will help improve developers and contributors experience switching between projects.

• We can expect to see a good number of outside JavaScript contributions from one of the biggest open-source communities.

• The major objectives we've covered/unlocked are:

  • Allow more people to contribute to PDE's further development.
  • Provide better support for building the UI by using JavaScript.
  • Have more visually consistent cross-platform results.
  • Potentially open up other ways to distribute the PDE.

---

Scope

For the initial prototype, we can imagine a basic version of the current PDE with a code editor and an option to run the sketch. The technical plan proposed forms a good foundation for a future scalable product. Most of the UI components would be covered in the prototype itself and once we form the communication bridge with the CLI properly, we can incrementally port and migrate the rest of the features.

Development Process (Timeline)

The timeline will be planned once this initial proposal is approved and we finalize the things to cover in the initial prototype.

Discussion

• What is the current state of the Processing CLI in terms of stability and how regularly is it maintained?

• Are there any features that currently aren't supported/exposed by the CLI which we want to accomplish in the prototype version? Currently, the --help command shows these supported options.

• Finalize the scope and things to cover in the prototype.

---

About me whoami

I'm Dhruvdutt Jadhav, a software engineer and a student from DAIICT, India. I'm currently in my final year of masters and have also completed React Nanodegree from Udacity.

I'm one of the co-authors and maintainers of ReactiveSearch - a React based web and native UI components library for Elasticsearch. I'm also one of the members and collaborators of webpack team and have done noteworthy contributions to webpack-cli. I've also done other minor contributions to yarn, yarn-website, create-react-native-app, react-router and many other open-source libraries. You can catch all my contributions on GitHub.

I've also contributed to Processing's p5.js-web-editor in submitting some patches and upgrading major packages.

Apart from this, I've built many web-based production apps and have primarily worked with JavaScript and React in my most closed source projects. You can learn more about them in my resume.

I'm very proficient working with JavaScript and web technologies in general. I've also worked with Java at an intermediate level and have built an open-source library to manage multiple filesystems as a part of my college project. Know more about it here.

I learnt about Processing a while ago and would love to be a part of its family for GSoC and beyond.

My details:

• GitHub: dhruvdutt
• Website: dhruvdutt.js.org
• Email: dhruvdutt.jadhav@gmail.com
• Timezone: UTC +5.30 (India). I'm a nocturnal person but open to work under any timezone.

---

Note: Big shout-out to Manindra Moharana for helping me throughout with the approach.

Thank you very much everyone for taking out time to read my proposal. I really appreciate your feedback and guidance about it.

himanshuc3

Look likes a through workflow for a project. A lot of features proposed, good for you @dhruvdutt.

gaocegege

It is cool! I'd prefer it if you could add some diagrams to illustrate the arch or your design.

shiffman

Thank you @dhruvdutt for this excellent and very thorough proposal! I'm very impressed with how much you have thought this through and the level of detail. I'm inclined to agree that using the Processing CLI is the way to go.

I wouldn't want this to get in the way of the initial prototyping of a JS-based Processing PDE, but I can imagine a future where there is one PDE that can run both Processing/Java and p5/JS. (While this is not on the table yet, a possible future plan for the p5.js web editor is to wrap it with Electron for offline use.)

dhruvdutt

@himanshuc3, Thanks!

@gaocegege, I'll try to prepare a diagram over the week. Thanks for your feedback.

@shiffman, That's good vision. We can incrementally keep on improving the CLI for core runner and migrate rest of the features to JS gradually and have a single platform for everything. It would have enormous benefits in terms of development, contributions, consistency, speed and technology.

neilcsmith_net

@dhruvdutt very interesting and well described read! Something tangentially related but causing some "interesting" discussion within the Apache NetBeans community is this Oracle white paper on the future (or lack of it) for JavaFX and AWT/Swing in the JDK.

Something else I was looking at recently that may be more directly relevant was on embedding a Vaadin web application inside Electron. Now, definitely not suggesting that, exactly, but is it not worth considering reusing the existing non-UI bits of PDE and perhaps a code editor that supports Language Server Protocol that could be fed from the existing parser or derived from one of the existing Java compiler based ones?

https://vaadin.com/blog/desktop-uis-will-stay-alive-thanks-to-web-technologies https://github.com/jreznot/electron-java-app

dhruvdutt

@neilcsmith_net Thank you for your feedback.

I'm aware of Vaadin components. In fact, I've dived into their code while researching and have tinkered with it. They do work with Electron but I think at the end, it's just fancy Java code.

I've also talked about Vaadin components specifically with Manindra earlier and we agreed not to use them but to build our own JavaScript-based UI components. They are very easy and straightforward to build and we can get full control of each and everything. In Vaadin, we would be limited in terms of customizations (because of Java API) and after all, they do have limited things in their free plan.

We would anyway need to use JavaScript for code editor (which we can't build using Vaadin) and also for other Electron stuff. Another point is we're porting p5.js-web-editor styles which are web-based. If we use Vaadin, it would be difficult to create a cross-language mapping between styles for Java-based components (Vaadin) and JavaScript-based components (p5.js-web-editor). This is unnecessary complexity we're creating for which a simple solution exists. @shiffman's plan to unite things in future also goes in favor of JavaScript-based components.

Finally, some of the major objectives of building a new JavaScript-based PDE are: - To allow more people to contribute to its further development (i.e. moving away from Java components) - Provide better support for building the UI (customizations) - Have more visually consistent cross-platform results (styling).

All these can be easily accomplished using JavaScript to build web components.

neilcsmith_net

@dhruvdutt - sorry, I obviously did not explain myself very well - I am definitely not suggesting you use Vaadin at all!

I'm saying that specific post and code suggests an interesting approach to using a JS frontend with Electron talking to a local Java backend handling integration with compiler, parser, (Java) file handling, etc. It's an alternative approach to just using the CLI mode integration. It would allow reuse of useful bits of the existing PDE in the background while retaining all the benefits of moving to the JS approach. And you'd get better editor parsing, completion, etc. via LSP.

dhruvdutt

@neilcsmith_net, gotcha! LSP supports only one web-based code-editor i.e. Monaco. There's also a related open issue about LSP integration in Monaco. If we change the editor to Monaco, it can affect some other things like parity with the p5.js-web-editor.

Also, I'm not sure how feasible it is to build a LSP inside the current Processing monolith. If there's a clear path to set up an LSP like what you've suggested, then this is definitely something we can think of. I think we'll have to vscode-languageserver-node to build a Processing LSP similar to vscode-javac.

In my opinion, this approach seems a bit speculative and needs significantly more research. If we had to build a LSP from scratch, it could be a project in itself.

My first thoughts were to see how far we can get by just using the CLI.

gaocegege

@dhruvdutt @neilcsmith_net

The idea about LSP is really awesome! LSP is implemented in many editors: VS Code, Eclipse Che and so on. If we implement a LS, then it could be used in all these editors without code change.

But I think it can not met all our needs since the protocol has limited types of text hooks:

• didOpen
• didChange
• willSave
• willSaveWaitUntil
• didSave
• didClose

We could use LSP to implement the formatter/linter for Processing. But we have to implement the logic about running PAPPlets by our own via Processing CLI.

I think LSP could help us reuse the code about formatting in Java. But if we do not include the feature in this GSoC idea then we could put it in roadmap.

I used to implemented a linter plugin in VS Code: https://github.com/coala/coala-vs-code and if you have any question about LSP I am glad to help you.

neilcsmith_net

@gaocegege @dhruvdutt

We could use LSP to implement the formatter/linter for Processing. But we have to implement the logic about running PAPPlets by our own via Processing CLI.

Who said anything about just running LSP in the Java process? With Electron you're already looking at a multi-process system, right? If you look at the parts of the Vaadin post that I'm talking about it's running a Java VM within Electron to provide backend services.

What I am suggesting is something similar to having a client / server (but all local) architecture for the PDE Java editing, where you have a continuously running Java process that handles LSP, debugger interaction, sketch launching, etc. and communicates with the JS frontend in a similar way to how the various parts of Electron already communicate with each other.

That way, it's potentially possible to reuse non-UI Java code from the existing PDE behind the scenes, while getting all the benefits of a JS frontend. I'm always a fan of evolution not revolution ;-) otherwise you just end up with a great looking but less useful tool while you try and catch up on features.

Just my 2c - but now would be the time to consider that approach as it will impact how you propose to build this early on.

neilcsmith_net

OK, linking some conversations together! ;-) https://github.com/processing/processing/issues/5423

fry

Thanks for this very thoughtful proposal. A couple quick notes, then the two big pieces are on the Design and Scope section. The quick things first:

• Base Framework: Agree that Electron is the way to go

• Code Editor: p5.js using CodeMirror (and your experience with that) sounds like the right thing

• UI/Frontend: React seems to be the thing

• Testing: sure, with an eye toward maintaining parity with the current PDE. Though it's also a lower priority than getting something working and may be out of scope for a summer-length project.

• L10N, I18N, A11Y: are all important, yes. You'll want to build on the work that's already been done for the PDE, as significant effort has gone into localizing the various messages in the PDE itself.

• App Distribution: we could do the app stores now, but do not, mostly because they require additional work and restrict features. They're also somewhat incompatible with the "sketchbook" model we have in place. Basically it'd be extra effort to support a more-closed system, and I'd still have to produce non-app store versions. Auto-updating would be nice too, but these are all laundry list features that would be outside what can happen in a GSoC project (relative to the other priorities).

• Language: sounds good… I go back and forth on TypeScript, and can be convinced either way.

• Package Manager: aren't you more-or-less stuck w/ NPM if you're using Electron anyway?

• Tooling: sure

And the longer points:

Design

We'll want to stick with the current 3.x design. The p5.js editor is a different project, and has a different audience and use case. We're looking for a direct port of the current state of the PDE that we can build on later. It's a mature application that's received a lot of work, has a lot of users, and a lot of resources and materials that describe the location and use of various features. Switching to the p5.js editor layout would be a step backwards: it's very early in its development and would remove too many things.

A general-purpose editor is also not a design goal—that's a distraction to getting something that works well for our use case. There are tons of editors out there, we don't need another generic one. And if anything, I think the p5.js editor and PDE should probably be diverging from one another—to be more specific to their use cases—rather than converging. (That last point is not a Foundation statement, it's me thinking out loud.)

Scope

The current CLI is very limited. It's really just the minimum to allow folks to run the app headless from their favorite editor. Wiring the CLI to a simple text editor that looks a bit like the current PDE as an electron app would be the first week or two of the project.

The CLI isn't really the way to go for anything further than that. Put another way, if this is just a frontend on the CLI, then why do it at all? We should just be sending people to more complete editor platforms and IDEs. The point of the PDE is how it ties the project together: making it easy for people to get started, to install and use libraries, to not have to think about the CLASSPATH, Java version, etc.

The real meat of making this project work is two things: first is feature parity/alignment with the PDE, and the second is how to make the debug/autocomplete/etc code work with the frontend.

The feature parity issue is that while the PDE seems like a "simple" editor, there's a great deal of things that it can do, and that users expect it to do. A lot of the complexity is hidden from users, but rebuilding it is an enormous task. I don't expect we'd have 100% parity with the PDE in a single GSoC project. (But of course I'd love to be wrong!)

For debug/autocomplete/etc, that's much trickier. You'd be working with Jakub and me to sort out how to best wire all that to a JS frontend, because it's both complicated and also needs some housecleaning along the way.

Again, thanks for the proposal. Hopefully we'll have a chance to work with you on the project this summer.

dhruvdutt

@fry Thank you very much for taking out time to review. Here are my notes:

• L10N, I18N, A11Y: I think if we can pluck the text mapping file from the current PDE, it should be straightforward to set up inside the prototype.

• Package Manager: Yup, NPM comes by default but there's an alternative called yarn. It's not a replacement. It's just something you can use alongside. NPM should work just fine anyways.

• App Distribution: We can produce non-app store versions to start off with and come back to auto-updating and app distribution later once we get the prototype in a shippable state.

• Design: Sure, we can keep 3.x design. I think we'll have to write web styles from scratch. It should be pretty straightforward to do. If we have any old mocks about this design, please share. It's not needed but good to have for keeping the UI pixel perfect.

• Scope: I agree. CLI was the first thing that came to my mind but I knew we couldn't just pull off everything through it.

As you stated, there are two major slices of this project:

• feature parity/alignment with the PDE
• debug/autocomplete/etc code communication with the frontend

The feature parity issue is that while the PDE seems like a "simple" editor, there's a great deal of things that it can do, and that users expect it to do.

It would very helpful if you could clearly state what things we should cover for the first part other than the CLI integration with the code editor, UI design, web components and rest of the listed technical specifications.

Once we seal the scope for the first part, I'll make a detailed timeline and we'll see how much we can cover for the second part which is both trickier and complicated.

Again, thanks for the proposal. Hopefully we'll have a chance to work with you on the project this summer.

I feel electrified. This can't get more exciting. I would love to serve Processing. Thank you very much.

Chrisir

https://forum.processing.org/two/discussion/26861/code-snippets#latest

shiffman

Hi @dhruvdutt, thank you so much for your excitement and enthusiasm! @fry and I had a chance to talk and we have decided that before a new JS-based editor can be prototyped we should first develop a Processing/Java language server. I have updated the Project List to reflect this idea:

https://github.com/processing/processing/wiki/Project-List#-development-environment-processing-language-server

Let us know if you have any questions!

dhruvdutt

Hi @fry, @shiffman, I'm a bit upset hearing this but I do truly believe that the latter part of this project is complicated and might need extensive support from community peers. It completely makes sense to develop an LSP first which will be the backbone of the prototype and can work with other code editors as well.

I think apart from that, there is a great deal of things PDE houses, I wish I could work for it during the GSoC period but I'm sure we would need to build this JavaScript-based PDE later at some time.

I would be most obliged to work dedicatedly for it at any point in future when we have the right plan and resources for it. Let's archive this plan till then.