DB Manager for Processing

vesolba

@jeremydouglass said

@vesolba -- do you mean that you are currently creating a DB Manager for Processing? If so, this thread should be moved to Share Your Work. If you want early feedback from this forum, please post a link (e.g. Github). When version 1.0 of your DB Manager is ready, update here!

Yes. I needed a database connection for another project that I want to develop with Processing language. I have a prototype skecth about how to start the server and the basics, but I thought that it would be a good add-in for the Processing framework (Similar to the one with Netbeans or Eclipse), even if at first it is very simple, only for Java DB databases. Java DB (or Derby) is included in the Java SDK since Java 6.0, so it is not necessary download any more and it can be included in Processing sketches as embedded in the program and as local or remote server. By installing it in the Processing framework, it could be acceded concurrently from all processes working in the same Java Virtual Machine (JVM) as the database. I think I have follow all the steps by using the processing-tool-template and now it compiles well in Eclipse with the Ant schema created by the tool template and I can show you the initial windows. I am uploading the project to Github: vesolba/DBManager-for-Processing. This is the first time that I make use of Git so if I do something wrong, please let me know (may be I have upload some garbage). To develop the code I am using Eclipse Oxigen and to design the windows it is installed the Windowbuilder plug-in.

Currently I am trying to fill the databases JTree via DriverManager. Any help or suggestion will be welcome. You can download the code from the github, if there is any problem or I have to give you any kind of access grant, let me know also.

Greetings.

cameyo

Please, do it :x

jeremydouglass

Very interesting! Than you for sharing -- I look forward to trying it out.

• https://github.com/vesolba/DBManager-for-Processing

vesolba

@jeremydouglass

Hello! I've upload the project files in its current state. I think that this time my upload has been done well and with the correct files and directories. Even if it is not working very well yet, I've found the way to do the necessary tricky matter and soon it will be available without errors. But it is not working well yet. If you want to take a look, you can make an idea of how it will complete.

Greetings.

https://github.com/vesolba/DBManager-for-Processing

vesolba

Hello again. Just to let you know that I continue working. More hard than effectively but with the hope to have soon a good beginning for this DBManager.

jeremydouglass

@vesolba -- thanks so much for the update!

Great to see that you are making progress at https://github.com/vesolba/DBManager-for-Processing Do you have any questions or need any support from the community?

vesolba

@jeremydouglass

Hello! By now, I'm solving the needs and doubts by googling the web, but may be I could need them, thanks. This could be a big project, but I want to keep it as simple as possible and finish asap an elemental prototype with only the basics to create little databases in such a way that the user do not have to deal with concepts such as schemas, connections, etc., only with "databases", "tables", "columns", "indexes" and few more at same time that he is being informed on where are his databases in the disk, what directory is being accessed by the server, and what ways has to distribute the data with his code. Nevertheless, if anyone is interested in make a contribution or suggestion, or use the code to develop their own manager, he will be welcome. After finishing this Manager, i want to develop a project with Processing that will be based in the ideas on genetic algorithms given by Daniel Shiffman in his very appreciated book "The nature of code", by making persistent the knowlegde learned. I want to explain this better when I begin that project. Greetings.

jeremydouglass

Good luck, @vesolba -- excited to see what you come up with!

vesolba

Hello, @jeremydouglass, thanks for your good wishes. I've recorded a video showing the actual state of DBManager. Please, excuse my slowness some times because a hands problem (neuro matter)

The code has been upload to https://github.com/vesolba/DBManager-for-Processing

May be I need that someone try the tool in other OS as I only work currently with Windows. Thanks.

jeremydouglass

Thank you, @vesolba. I think that a simple DB manager will be a very nice alternative to Tables for some Processing projects.

I am on OS X -- I can't test right now, but sometime soon I will give it a try.

Thank you for sharing that helpful demo video. Your demo shows many many features of this DB manager! One thing the source code doesn't include an example sketch showing how a Derby DB is then used in a Processing sketch. Will the tool also include an example sketch? This is what will probably get new users excited about using it.

vesolba

@jeremydouglass Yes, in the tab "Code Gen." I want to include the necessary stuff to help user to include the access to the database he wants to use.

Sorry for not to have highlighted that there is an example previous to DBManager in https://github.com/vesolba/DBManager-for-Processing/PruebaJavaDB.pde

below is the code, you can copy paste it in the Processing editor and save it with the name PruebaJavaDB.pde:

import java.sql.Statement;

import java.sql.Connection;

import java.sql.ResultSet;

import java.sql.DriverManager;

import org.apache.derby.drda.NetworkServerControl;



Connection con;

void setup() {

  try {

    // Tell derby where to put the database files

    NetworkServerControl server = new NetworkServerControl();

    server.start (null);

    System.setProperty("derby.system.home", sketchPath("test"));



    // tell java which database-driver to use

    // Class.forName("org.apache.derby.jdbc.EmbeddedDriver"); // DB integrada en la aplicación

    Class.forName("org.apache.derby.jdbc.ClientDriver");      // Aplicación cliente con DB en servidor.



    // create a database named test

    // con = DriverManager.getConnection("jdbc:derby:test;create=true"); // DB integrada en la aplicación.

    con = DriverManager.getConnection("jdbc:derby://localhost:1527/test;create=true"); // Aplicación cliente con DB en servidor.

    

    Statement stmt = con.createStatement();

    println("Inicio");

    try {

        // drop the foo table if it exists

        stmt.execute("drop table foo");

    } catch (Exception e ) {

    }

    // create a table named foo and insert two rows

    stmt.execute("create table foo ( id integer not null, name varchar(20))");

    stmt.executeUpdate("insert into foo values (0, 'fooclient')");

    stmt.executeUpdate("insert into foo values (1, 'barclient')");

  } 

  catch (Exception e ) {

    e.printStackTrace();

  }

  println("Fin");

  noLoop();

}



void draw() {

  

  println("Inicdraw");

  try {

    // select all rows from the foo table

    Statement stmt = con.createStatement();

    ResultSet rs = stmt.executeQuery("select * from foo");

    while( rs.next()) {

      // print the content of the "name" column on the console

      println(rs.getString("name"));

    }

  } 

  catch( Exception e ) {

    e.printStackTrace();

  }

  

  println("fin draw");

}

I have tested this sketch and it works, sure! ;)

vesolba

Nevertheless, I will include more examples in the examples directory. But I want to draw your attention, especially, in that Processing sketches need to work that the processing environment itself be present, and that environment uses the Java Virtual Machine that since its 6th version includes by default the SGDB Java DB. That is to say that the only thing that Processing lacks in order to have a complete system of databases is a manager that allows them to be created without having to use an external program. Please, think about this ... ;)

vesolba

@jeremydouglass @rahaman Now, the DBManager helps you to include BLOBS in your databases

As well as XML files

I think it will be soon finished as a first stage.

jeremydouglass

Very exciting!

I will be interested to see how BLOBs could be integrated into a sketch -- for example, frames being saved to the database rather than to disk, or loading a BLOB into a PImage and then displaying it with image()....?

vesolba

Next, I will try to develop an example with those parameters. Hope show it soon here. Thanks.

vesolba

@jeremydouglass @rahaman Hello. This is to show you an example of Processing sketch that saves images in a table blob column and then load them into PImages that shows in the screen.

This may be done in two ways (at least). One of them saves the image as raw data in a table column defined as type blob, and te other wraps first the image in a blob object that then save in the same kind of column. You can see the sketch code here<--. It should work. If not, please let me know. Next Im gona try to save frames in the database. Enjoy. Greetings.

jeremydouglass

Very cool example.

vesolba

Thanks Jeremy. However, maybe there are too many lines of code for something that could be done with less trouble by taking advantage of that Processing can act as a framework for Derby in a two tiered architecture.

vesolba

@jeremydouglass @rahaman Hello again from Spain!!! How are you? Here winter is coming and we are well by now. This is to let you know that a new example is waiting to you wacht it at https://github.com/vesolba/DBManager-for-Processing/tree/master/examples/SaveFrame2DB

This time you can see how to save frames in a DB and read them after. Even more, it is done without use an aux file, directly from the screen to the DB and vice versa. The screen you should see after its execution is below.

Please, take in account that the use of big volume of data may require the use of threads and adding the "autocommit(false)" and "commit" clauses to the SQL code. These examples focuses in a global vision on how to do it and I have done the test with not very big sizes of data. Hope you enjoy.

jeremydouglass

Wow!!

This is really impressive stuff, @vesolba.

Have you considered making this available on the Processing Tools page ( https://processing.org/reference/tools/ ) and having it available through to install through the Contributions Manager > Tools tab in PDE? Do you think it is ready? It seems like it is...?

More people who aren't active on the forums should know that this exists so that they can use it.

vesolba

@jeremydouglass I am proposing to Elie Zananiri @prisonerjohn making them available in the examples page. I think that they could be seen there soon.

vesolba

If I did all well, the last version can be downloaded from https://github.com/vesolba/DBM4Processing-Examples/releases/tag/2.0

Im trying to do it my best but I am a noob in git matter.

jeremydouglass

@vesolba -- just pinging you to let you know that we are migrating to a new Forum.

I would still love to see this great tool submitted to Processing Tools page ( https://processing.org/reference/tools/ ) and available to install through the Contributions Manager > Tools tab in PDE.