Trying to simulate Paint app in processing, question about drawing effect

BodegaB

Hi guys, im trying to accomplish something in my code and i dont even have an idea how to do it, so ill try to explain it best as i can: I wanna be able to draw in processing, basically what i wanna do is something like a paint app we all have on Win OS, when i move mouse around (and when it is pressed for example. but doesnt have to be) i want it to make a drawing effect on background, now i already accomplished that but, but what i also want is when mouse moves away too fast, from one point to another (if its a mistake by human hand), in that case i dont want it to draw anything on board, but only when i move it relatively slowly,and also when i move it fast like that from point A to point B i just want that circle (ellipse) to show at the point B,again, dont want it to leave trace in between those points To make drawing effect im using PGraphics, and it looks something like this:

  x = constrain(x, 0 + radius/2, width - radius/2);
  y = constrain(y, 0 + radius/2, height - radius/2);


  paint.beginDraw();
  paint.fill(R, G, B);
  paint.noStroke();
  paint.ellipse(x, y, radius, radius);
  paint.endDraw();

that ellipse is the pointer which is making drawing effect on some background, x and y are mouse coordinates

cameyo

Some infos: layers

BodegaB

Wow, there is way too much going on in there, it would me took me forever to figure it out, could you pls help with this specific problem i have, i dont know if i was clear enough with description,if not ill try to explain better :)

BodegaB

Actually ill be more specific, im not working with the mouse at all, my "pointer" is manipulated by values from two sensors via arduino, and than i move object in front of them to make that drawing efffect in processing, but when i want to start drawing from different point on my background i dont want that line to be drawn from previus point to the point where i put my object,basically what i want is possibility to draw without those lines being continuous, i wanna be able to start draw at some other part of the background without connecting those two :/

jeremydouglass

@BodegaB -- re:

when mouse moves away too fast, ... I dont want it to draw [a line], also when i move it fast ... I just want that circle (ellipse) to show at the point B

So if your distance between the previous mouse and the mouse is not too far, then draw a line, otherwise draw an ellipse?

int far=50;
boolean skip;
void setup() {
  size(400,400);
  frameRate(10);
}
void draw() {
  if (dist(pmouseX, pmouseY, mouseX, mouseY)<far) {    
    line(pmouseX, pmouseY, mouseX, mouseY);
  } else {
    // ellipse(pmouseX, pmouseY, 10, 10);
    ellipse(mouseX, mouseY, 10, 10);
  }
}

kfrajer

I was going to suggest what jeremydouglass wrote. However, I find that pmouse could not be ideal for you as pmouse only stores the mouse position from the previous frame. You can have better control if you store your own previous mouse coordinates and you process them with your own algorithm. This will give you more flexibility. You will need to implement this anyways as you are not using actual mouse information but data from your arduino.

If you store, let's say, the last 10 previous positions, then you can draw only when the difference between positions if below certain threshold. Or you could draw based on speed. If zero speed (or very slow speed - think data received from normal hand shaking), then don't draw. If high speed, then don't draw. If the speed is withing certain low range, then draw.

Kf

BodegaB

Hm interesting, thank you both guys, but how would you suggest i store those values from arduino, just simply putting them in some array or? Becouse there is a lot of data being sent from arduino, i get new values every 100us or so .. hmm

BodegaB

@jeremydouglass nono, it is always drawing a line, but with that circle get it? I'm not using line() function in my sketch but pgraphics object, when i move that circle it leaves a trace behind on background, much like that line(), now that i think about it maybe i should use line() for drawing instead of pgraphics, and what u did there is actually ok, when i move mouse too fast nothing is being drawn on the background and that's what i want, only problem is like @kfrajer said, i don't actually use mouse but values from Arduino, so i need a different method to store those previous values than compare them somehow so i can tell it when to draw and when not to, those values i get through serial communication and im storing them in int variable

kfrajer

Run the following sketch:

void setup(){
  size(400,600);
   fill(150);
   noStroke();
}
void draw(){
  //background(0);   <<-----UNCOMMENT this line to see the effect
  ellipse(mouseX,mouseY, 35,35);
}

As you see, you need to call background to clear the sketch every time draw runs. This is a common way to do it and avoid those circles you were describing before.

For storing points, yes... that is part of the good challenge. Totally doable in Processing. As a matter of fact, it has been done in several posts before. I will suggest you check in the reference for arrays or ArrayList. If you go to the reference, check the example section as well.

How much data to collect? You need to filter what you collect. For starters, you can collect most of it and see how your system handles it. You can always include filters in your data retrieval and storage process. providing an example of your approach will help in this endeavor.

Kf

jeremydouglass

Non-mouse input can still be screened with just two variables (inX -> pinX). @kfrajer -- you might be right that storing an array and sampling of several previous values and taking their max might be slightly better at suppressing jerking motions in the data, but just using previous state seems to work pretty okay.

To know what the best approach is we need to know a lot more about the nature of the input and the goals!

I won't get into the serial aspect, but this is a demo:

// forum.processing.org/two/discussion/22478/trying-to-simulate-paint-app-in-processing-question-about-drawing-effect
// SkippingDraw

float tooFar;
float inX, inY, pinX, pinY;
boolean skip, farstep;
RandomJerkWalker rjw;

void setup() {
  size(400, 400);
  frameRate(10);
  reset();
}
void reset() {
  background(192);
  // center input
  inX=pinX=width/2;
  inY=pinY=height/2;
  // set jerkiness limit
  tooFar=15;
  // create random walk input
  rjw = new RandomJerkWalker(inX,inY,5,40);
}

void draw() {
  // clear screen every 100 frames
  if (frameCount%100==0) {
    reset();
  }
  // update random walk input
  rjw.update();
  inX = rjw.vec.x;
  inY = rjw.vec.y;
  // detect whether to draw or ignore input
  skip = dist(pinX, pinY, inX, inY) > tooFar;
  if (!skip) {
    // draw
    stroke(0);
    fill(255);
    ellipse(inX, inY, 10, 10);
  } else {
    // show where drawing would be hidden
    stroke(0, 48);
    fill(255, 48);
    ellipse(inX, inY, 10, 10);
  }
  pinX = inX;
  pinY = inY;
}

class RandomJerkWalker {
  float step;
  float stepNear;
  float stepFar;
  PVector vec;
  RandomJerkWalker(float x, float y, float n, float f) {
    step = n;
    stepNear = n;
    stepFar = f;
    vec = new PVector(x, y);
  }
  PVector update() {
    if (random(1)<.05) step = stepFar;
    if (random(1)>.75) step = stepNear;
    // if neither, continue with previous step;
    vec.x = constrain(pinX+random(-step, step), 0, width);
    vec.y = constrain(pinY+random(-step, step), 0, height);
    return vec;
  }
}

GoToLoop

http://studio.ProcessingTogether.com/sp/pad/export/ro.91FXxn8BG2fPL/latest

BodegaB

@jeremydouglass Ok, i think we are getting somewhere, and here is my code and what i work with, now what i originally wanted is this, when these values (x and y) go beyond those coordinates in first If() and than they come back in between those values but on some different coordinates than last time, i dont want my pointer go from last spot to the new one leaving drawing effect, i only want that pointer to show up on those last coordinates, right now in my code there is nothing to solve that issue, so i will definitely try your solution after i study it a little bit, don't have time today(its late here), feel free to give more advice regarding the code:

      float targetX = pointerX;
      float dx = targetX - x;

      if (abs(dx) > 0.1){
       x += dx * easing;
     }

      float targetY = pointerY;
      float dy = targetY - y;

      if(abs(dy) > 0.1){
        y += dy * easing;
      }

      x = constrain(x, 0 + radius/2, width - radius/2);
      y = constrain(y, 0 + radius/2, height - radius/2);



      paint.beginDraw();
      paint.fill(R, G, B);
      paint.noStroke();
      paint.ellipse(x, y, radius, radius);
      paint.endDraw();

      image(paint, 0, 0);

there are some things in this code that i left out, but they are not relevant to this subject.

Thx again once more for helping, all of you

jeremydouglass

@GoToLoop -- Nice example! The key thing with that GridWalker example is that it needs to have sudden "jerks" in order to simulate the things that OP is trying to suppress. Point.SPD can't be static final -- it needs to be a variable, and it should occasionally switch to a higher speed and then switching back after some short period of time.

At least, that was my approach based on OP descriptions without having any idea what is coming in from the serial sensor.

kfrajer

Now that you share your code, I can see my previous post about background not to be the case at all. The problem is that you are drawing in the PGraphics. You don't really need it. Here I present a modified version of your code after making some minor modifications and suppressing some lines in the body of your data processing. In this code, I use an ArrayList to store the points and those points are redraw every time draw() gets executed. I am expecting that after few seconds, your drawing will get overloaded if you store all the points.

import processing.serial.*;

Serial arduino;

final int radius = 25;
final float easing = 0.05;
final color kBLACK=color(0, 0, 0);

float pointerX; 
float pointerY;
float x;
float y;
PImage background;  
ArrayList<PVector> pts;

void setup()
{

  arduino = new Serial(this, "COM3", 9600);
  size(920, 760);
  pointerX = 0;
  pointerY = 0;
  ellipseMode (RADIUS);
  background = loadImage("scooby.png");


  pts=new ArrayList<PVector>();
  fill(kBLACK);
  noStroke();
}

void draw()
{

  if ((arduino != null) && (arduino.available()>0))
  {
    String message = arduino.readStringUntil('\n');
    if (message != null)
    {

      String[] value = split(message, ' ');  

      if (value.length!=2)
        return;   //NOTHING to do as we didn't get two values

      int leftSensor = int(trim(value[0]));
      int rightSensor = int(trim(value[1]));


      //  REMOVE LINES converting left/right sensor data into x,y points

      x = constrain(x, 0 + radius/2, width - radius/2);
      y = constrain(y, 0 + radius/2, height - radius/2);

      pts.add(new PVector(x, y));


      image(background, 0, 0, width, height);

      for (PVector v : pts)
        ellipse(v.x, v.y, radius, radius);
    }
  }
}

A second option is not to use an array list but only to draw points as they are processed. This means you will draw your background image once in setup and drawn any new point as they arrive:

import processing.serial.*;

Serial arduino;

final int radius = 25;
final float easing = 0.05;
final color kBLACK=color(0, 0, 0);

float pointerX; 
float pointerY;
float x;
float y;
PImage background;  

void setup()
{

  arduino = new Serial(this, "COM3", 9600);
  size(920, 760);
  pointerX = 0;
  pointerY = 0;
  ellipseMode (RADIUS);
  background = loadImage("scooby.png");

  fill(kBLACK);
  noStroke();
  image(background, 0, 0, width, height);  //DRAW BACKGROUND IMAGE once
}

void draw()
{

  if ((arduino != null) && (arduino.available()>0))
  {
    String message = arduino.readStringUntil('\n');
    if (message != null)
    {

      String[] value = split(message, ' ');  

      if (value.length!=2)
        return;   //NOTHING to do as we didn't get two values

      int leftSensor = int(trim(value[0]));
      int rightSensor = int(trim(value[1]));


      //  REMOVE LINES converting left/right sensor data into x,y points

      x = constrain(x, 0 + radius/2, width - radius/2);
      y = constrain(y, 0 + radius/2, height - radius/2);

      PVector v=new PVector(x, y);
      //image(background, 0, 0, width, height);
      ellipse(v.x, v.y, radius, radius);
    }
  }
}

You should also check this post, GoToLoop's example: https://forum.processing.org/two/discussion/16618/processing-with-arduino-void-serialevent#Item_5 This uses the noLoop/redraw function in tandem to draw serial data as it arrives from the arduino.

Notice i didn't address the question of jitter as @jeremydouglass and @GoToLoop have done a fantastic job. If you still want to do a jitter filtering, then you will need to keep track of the x/y data pairs in an array to have some history of your points and apply filtering based on previous data.

I think removing the PGraphics object could be enough for your case.

Kf

BodegaB

Ok i tried @jeremydouglass solution with that simple boolean skip; and if(!skip) ,i and it works, kinda, this is what i get on my screen:

So i moved my object from point A to B (in front of my sensors) but it still kinda starts to draw before coordinates are equal to point B, now im guessing its because of that easing effect where x and y values are "eased" by that value, any ideas how to avoid that from happening, would using PVector change anything about that?

jeremydouglass

So i moved my object from point A to B (in front of my sensors) but it still kinda starts to draw before coordinates are equal to point B

I'm honestly confused now. This is what you said before:

when mouse moves away too fast, from one point to another (if its a mistake by human hand), in that case i dont want it to draw anything on board,

In your picture, did you move things from point A to B too fast? Did it still draw it, even if it was moving fast? Are the lines that you circles in red things that shouldn't have displayed because of how fast the object was moving, but they displayed anyway? Right now I can't replicate that it my testing of the code I shared with you.

If instead of trying to suppress fast drawing you are trying to stop drawing, then draw again only once something arrives at a pre-defined point (B), then that is an entirely different thing.

Please take your time and be really specific: What are you trying to do?

BodegaB

I understand why you are confused, its entirely my mistake as speed doesn't really matter :/ . What i'm trying to do, like it says in the title, is simulate a process of painting (imagine if its on paper),so i have this object which is my pencil, when i move it to the certain coordinates i want it to leave a mark, and than i move it around the "paper" and it still leaves a mark behind it,but (this is a problematic part) now i want to draw from a different point on paper, so i move pencil away from paper and put it on the desired spot again, naturally pencil wont leave a mark in between these two points, but that is the issue with my object and sensor detecting it, when i move object away from point A and put it again on B it will just connect these two dots on my screen in processing, so that's why i want to eliminate that drawing effect when distance between two points(coordinates) suddenly is "huge", your example solves that issue in theory, but like you see in a pic above, that unwanted drawing effect still appears partially,now is it because of that easing effect or because object is not always detected properly (sensors are not perfect after all) that i don't know.. :) I hope this clears it all

jeremydouglass

How specifically is whether the pen is "touching" or not indicated in your data stream? You have said not just through speed / distance. For example, with normal input, whether the pencil is touching the paper is indicated by mouse press.

"Easing" is generally something added to simple data to simulate complex real world interaction -- turning several points or a line into a curve. It sounds like you have the opposite problem -- your input stream is complex, and you want to simplify it. However, I don't know what you mean by "easing effect" because I don't know your data source. Is it 2D coordinates of an actual brush? Often that kind of thing would be camera capture, data not serial data.

BodegaB

Well my sensors just detects object in front of him, and sends data every 100ms through serial to processing,that data are two integers,one for each sensor (so yes, 2D).

As you can see in my code (line 46) i made a condition, if object is not in margins of my "paper" than those values simply are not used anymore to calculate coordinates, so my circle stays at last known position, and that is how "not touching" is indicated. When i move object back to the "paper" (conditions in that if() statement are met), then it continues to move around.

When i said "easing effect" this is what i meant: https://processing.org/examples/easing.html you can see i used it in my sketch above (line 56 -67)

BodegaB

anything guys?

kfrajer

that unwanted drawing effect still appears partially,now is it because of that easing effect or because object is not always detected properly (sensors are not perfect after all)

Is it the sensor or the algorithm. That is a very good question. There is a way to test it. Instead of writing an algorithm to try to detect jumps in your data to enable disabling drawing, try another approach. Use for example, while a key is pressed, do drawing. Or use a key to enable the drawing mode, and another (or the same) key to disable the drawing mode. If you do it this way, you will learn more about the nature of your input data. This is very important and crucial to be able to go to a next stage and design an algorithm.

Related to easing, correct me if I am wrong, but what you want is to remove the jittering in the data, similar to what GoToLoop did in his demonstrations early before. What you want to do is apply a sort of high frequency filtering where you remove(discriminate) fast short moves while drawing.

Kf

BodegaB

Thx for suggestion,thats good idea,but my idea for this project is that everything is manipulated by that object (distance of it from sensor), so i wouldnt want to include any other input except that data from sensor via arduino.

I dont know, i would bet its because of the easing, i was thinking, maybe i can delay easing effect if that boolean skip condition is met,OR for example if that huge "jump" between values happen than delay drawing effect for 2s or something. I actually tried the latter but cant get it working with PGraphics, will try with your example though.

If you have any more suggestions let me know :)

jeremydouglass

@BodegaB -- I see that I misunderstood you about easing -- you were referring to your own easing in the code.

Yes, you are right:

1. try turning your easing off entirely, see what happens
2. first apply suppression on the un-eased coordinates. THEN apply easing on the un-suppressed coordinates.

BodegaB

This is what heppened when i turned the easing off :/, but not sure how make that 2. happen, been trying to achieve that but failed for now... any example would be welcome :(, this is above me..

kfrajer

What do you mean with the encircled sections in red? That is something you don't want?

Here below I present some code where I simulate x-y data using the mouse pointer as my data source. Each mouse point has an a noise component that you can use during plotting and calculation.

The filter part is not based on velocity as I mentioned before. It is based on distance. Although this could be simplified using mouse and pmouse data variables provided by Processing, one needs to consider that with this approach one is not limited to using only the previous point by one could use a longer history of previous points. This is not implemented here but can be added in case of building a more robust algorithm.

Kf

//REFERENCES: https://forum.processing.org/two/discussion/22478/trying-to-simulate-paint-app-in-processing-question-about-drawing-effect#latest

//INSTRUCTIONS:
//         *-- CONCEPT: Mouse pointer simulates data coming from an arduino unit. Jitter can be display/hide
//         *--          in real time. Jitter simulates the noise from physical sensors. An attempt
//         *--          is shown here to draw only non-jitter data. moreover, if the mouse pointer
//         *--          is moved fast from point a to point b, it will start drawing on point B without
//         *--          leaving marks. This requires the code to be fine tuned for both, noise intensity
//         *--          and filtering threshold.
//         *--
//         *-- This program draws points within the sketch area at the mouse position.
//         *-- When a point is added, a parallel point is created corresponding to its intrinsic noise.
//         *-- intrinsic noise is dependent on the intensity defined by noiseInt
//         *-- Press key "n" to toggle showing (and using) intrinsic noise.
//         *-- 
//         *-- There is also a basic filtering algorithm based on a distance treshold. It is defined as  
//         *-- a PVector "disThresh" where x is lower and y is higher distance threshold. Any 
//         *-- distance outside this range in considered invalid and not drawn.
//         *-- You can toggle the filtering algorithm using the key "f"
//         *-- 
//         *-- Notice points are added only if previous vs current point are not the same. This ensures
//         *-- the ArrayList storing points to grow slower instead of storing all points.
//         *--


//===========================================================================
// IMPORTS:


//===========================================================================
// FINAL FIELDS:
final int r=3;          //Point size

//===========================================================================
// GLOBAL VARIABLES:
ArrayList<PVector> pts;     //Pts from mouse
ArrayList<PVector> pnoise;  //Intrinsic noise associated to each point
boolean filterFlag;         //drawing flag for filtered points
PVector disThresh;  //Min/max range for drawing to be enabled

boolean mNoise;    //Introduce noise to mouse position
int noiseInt;      //Noise intensty


//===========================================================================
// PROCESSING DEFAULT FUNCTIONS:

void settings() {
  size(400, 600);
}

void setup() {

  textAlign(CENTER, CENTER);
  rectMode(CENTER);
  ellipseMode(RADIUS);

  fill(255);
  strokeWeight(2);

  pts = new ArrayList<PVector>();
  pnoise = new ArrayList<PVector>(); 
  disThresh = new PVector(5, 10);
  filterFlag=false;
  mNoise=false;
  noiseInt=(int)disThresh.x*2;
}



void draw() {
  background(0);

  if (frameCount%100==0) updateTitle();

  updatePts();

  for ( int i=0; i<pts.size(); i++) {
    PVector p=pts.get(i).copy();

    if (mNoise==true)
      p.add(pnoise.get(i));

    boolean dFlag=true;
    if (filterFlag==true) {
      dFlag=filteredPoint(i, mNoise);
    }

    if (dFlag)
      ellipse(p.x, p.y, r, r);
  }
}

void keyReleased() {

  //SHOW NOISE
  if (key=='n')
    mNoise=!mNoise;  //toggle noise flag

  //SHOW FILTERED PTS
  if (key=='f')
    filterFlag=!filterFlag;

    updateTitle();
}

void mouseReleased() {
}



//===========================================================================
// OTHER FUNCTIONS:

void updatePts() {

  if (dist(mouseX, mouseY, pmouseX, pmouseY)==0) return;

  //Create new position and its intrinsic noise component
  PVector mou=new PVector(mouseX, mouseY);
  PVector mn=new PVector(random(-1, 1), random(-1, 1));
  mn.mult(noiseInt);

  pts.add(mou);
  pnoise.add(mn);
}

//CALL ONLY AFTER updatePts()
boolean filteredPoint(int idx, boolean useNoiseFlag) {

  //GET PREVIOUS POSITION (checks for non-empty first... Error checking)

  PVector mou;
  PVector mn;
  PVector mouPrv;
  PVector mnPrv;

  int pidx=idx-1;

  mou=pts.get(idx).copy();
  mn=pnoise.get(idx);

  //Check in case of first point
  if (pidx<0) {
    mouPrv=mou.copy();
    mnPrv=mn;
  } else {
    mouPrv=pts.get(pidx).copy();
    mnPrv=pnoise.get(pidx);
  }

  if (useNoiseFlag) {
    mou.add(mn);
    mouPrv.add(mnPrv);
  }

  float d= mou.dist(mouPrv);

  return (d>disThresh.x && d<disThresh.y);
}

void updateTitle() {
  surface.setTitle("Pts:"+pts.size()+" noise="+mNoise+" Filter="+filterFlag);
}

BodegaB

The red circles is approximation of where my pointer should be positioned, but they are still spread around (so it was still showing some effect i wanted to avoid,but it looks better than with easing on) and as you can see they are "touching" each other only by the edge, and that is weird to me, but less that is important since i definitely need easing in my sketch anyway.

I can see you've put much effort into this, i cant thank you enough, this really helps a lot. Will look it up today ,as soon as i get time(its late here now). ;)

BodegaB

Thing is with this with your sketch i cant use PVector because of the values i get from sensor, at moments it will literally draw straight lines (vectors) because of the big difference in values he gets from sensor, and how often he gets it, here with mouse and pmouse it looks drastically different, so i need to use that easing effect nevertheless which causes this problem im trying to avoid. Other thing is, i am using PImage class in my sketch to load an image as a background, over which i then paint, with pvector i dont get that effect of drawing over the image, so i definitely need to use PGraphics to do that.