Get location of touch events on touchscreen device, similar to mouse location

osullic

I developed a number of Processing sketches many years ago, and just recently decided to embed one of these on my website using processing.js. (It's just a simple sketch where a group of lines follows the mouse location.) I have it working fine when viewed on my laptop's browser, but it doesn't work when viewed on my phone. I did read that processing.js should work with any HTML5-compatible browser.

I had assumed that mouseX and mouseY would receive the location of touch events on my phone's touchscreen, but then I realised that this might be where my problem is. I'd appreciate if someone could point me in the right direction for a straightforward way to get the location of touch events, in the same way that it is straightforward to get the location of the mouse.

Just this morning I discovered the existence of p5.js, and a quick look at the reference there shows that it has Mouse events and Touch events. Would it be more straightforward for me to abandon processing.js and just try to get my sketch working online with p5.js instead?

Thanks for your help, and sorry for the "newbie" question. I did some searching, but couldn't find exactly the answer to my question.

KevinWorkman

The mouseX and mouseY variables should work fine in Processing.js.

Can you post a small example sketch that we can run that doesn't work in Processing.js like it should?

osullic

Let me re-clarify: The mouseX and mouseY variables do work fine for me in Processing.js - when there is a mouse. But I'm trying to get my sketch to work on touchscreen devices too - when there is no mouse per se, but rather touch input. I had assumed that in these cases, mouseX and mouseY would receive values from touch events, but since my sketch is not working on touchscreen devices (well, on the 2 that I tested it on at least), I assume now that touch events don't pass values to mouseX and mouseY. Maybe as a first step, somebody could confirm or contradict this?

If it's the case that mouseX and mouseY should be receiving values from touch events, I'll certainly post my code then to see what the problem is.

KevinWorkman

I am contradicting this. Touch events should change the mouseX and mouseY variables.

I recommend posting a small example that doesn't work for you. Maybe a small sketch that simply displays mouseX and mouseY on the screen? Bonus points if you can post a link to a website that we can see what happens on our own phones.

Here are a bunch of dumb little Processing.js sketches I wrote that work fine on my phone.

osullic

Thanks for the reply.

I thought that my sketch wasn't working at all on touch devices, but it seems it's just not working as I expected. I thought I could drag my finger around the screen and mouseX and mouseY would be constantly updated, but it seems they're only updated when I tap the screen, and dragging my finger around the screen has no impact on mouseX and mouseY. Perhaps I need to investigate mouseDragged()

I'm adding my sketch here:

FollowingLine myLine = new FollowingLine();

void setup() {
    size(640, 480);
}

void draw() {
    background(255, 255, 255);

    myLine.update();
    myLine.display();
}

class FollowingLine {
    // a class to represent a line that follows the mouse
    // 'trailLength' stores the number of points that make up the line's "trail"

    int trailLength = 64;
    int[] previousXPositions = new int[trailLength];
    int[] previousYPositions = new int[trailLength];

    void update() {
        // add latest mouse position to trail array, but first...
        // shift previous positions in trail array
        for (int i = 1; i < trailLength; i++) {
            previousXPositions[i-1] = previousXPositions[i];
            previousYPositions[i-1] = previousYPositions[i];
        }

        // Add latest position to end of array
        previousXPositions[trailLength-1] = mouseX;
        previousYPositions[trailLength-1] = mouseY;
    }

    void display() {
        // loop through the trail array and draw a line between all previous trail positions
        for (int i = 1; i < trailLength; i++) {
            stroke(0,0,0);
            line(previousXPositions[i-1], previousYPositions[i-1], previousXPositions[i], previousYPositions[i]);
        }
    }
}

KevinWorkman

Do you have a link to this I can visit on my phone? I'll see what mine does.

osullic

Here: http://www.colinosullivan.com/Temp/processingTest.html

Thanks

blindfish

It works fine on my Linux desktop but I can confirm the same issue on an Android tablet in both Opera and Chrome: no lines are drawn. It's possible to hook a mobile device up to a computer and view the console output; so you could try and debug that way. Instructions for your OS/bowser combo will be available via a search... Failing that you could just try and output mouseX/Y as text values to the screen as a starting point.

It's pure conjecture but you have no explicit mouse events in your sketch: you're just relying on the standard mouseX/Y variables. Perhaps try adding a mouseDragged function and get it to output something to see if that works?

Jeanine

This question does not appear to be answered. I also have Processing JS sketches that do not work using touch on an iPad (that is the only mobile device I have tried). Here: http://faculty.purchase.edu/jeanine.meyer/ProcessingJS/rotatingCube.html is a modest variation of an example given in the Processing 3D tutorials. I use 3 images instead of one. Here is another example: http://faculty.purchase.edu/jeanine.meyer/ProcessingJS/ImageTravel.html

Again, they work on a desktop, but not on an iPad.

One problem appears to be that touch causes scrolling. I can turn scrolling off (I use a meta viewport tag and also addEventListener settings) but the touch still doesn't trigger an event.

blindfish

Looks like processingJS may support touch events...

ch3

I am currently running into the same issues, so this thread is rather useful. The link blindfish posted is interesting and I will give it a go.

To my understanding, it's the mouseRelease() that triggers correctly and I assume that's because the mobile browser ignores mousePressed() and mouseDragged() as it uses this functionality for panning the website and selecting text.

I have a simple test online with the default mouse functionality where I print messages with each corresponding function/global variable to make it clearer to what is happening.

http://pub.ch3.gr/js/test_05/

boolean MyMousePressed = false;

void setup()
{
  size( 500, 500, JAVA2D );
  colorMode(RGB,1);
  background(0);
  textSize(30); 
}


void draw()
{
  fill(0, 0.05);
  rect(0,0,width,height);

  if( mousePressed )
  {
    fill(color(1,0,0));
    textAlign(CENTER, CENTER);
    text( "global mousePressed", mouseX, mouseY );
  }

  if( MyMousePressed )
  {
    fill(color(1,1,1));
    textAlign(CENTER, CENTER);
    text( "MyMousePressed", mouseX+20, mouseY+50 );
  }  

}



void mouseClicked()
{
  textAlign(RIGHT, TOP);
  fill(color(0,1,0));
  text( "mouseClicked()", mouseX, mouseY );
}

void mouseDragged()
{
  textAlign(LEFT, TOP);
  fill(color(1,1,0));
  text( "mouseDragged()", mouseX, mouseY );
}

void mouseMoved()
{
  textAlign(LEFT, BOTTOM);
  fill(color(0,0,1));
  text( "mouseMoved()", mouseX, mouseY );
}

void mousePressed()
{
  MyMousePressed = true; 
  textAlign(RIGHT, BOTTOM);
  fill(color(1,0,1));
  text( "mousePressed()", mouseX, mouseY );
}

void mouseReleased()
{
  MyMousePressed = false;

  textAlign(CENTER, TOP);
  fill(color(0,1,1));
  text( "mouseReleased()", mouseX, mouseY );
}

NoraFaithrainbow

The issue is it doesn't recognize a tap as a mouse click. I encountered this issue my self. You will have to make a desktop version and a mobile version in order for it to work.

jeremydouglass

@NoraFaithrainbow is it necessary to make two sketches, or can you make a single sketch that contains both mouse event and touch event listeners so that either one will work as appropriate?

I'm not a p5.js expert, I'm just looking at the available event functions in the p5.js reference.

Jeanine

I'm back again. The touch events suggested in the link given by blindfish don't work for me. The error is that touches cannot be resolved or is not a field. Is there some library to include when preparing sketches for ProcessingJS?

Jeanine

The newest sketch I'm working on is here: (http://faculty.purchase.edu/jeanine.meyer/ProcessingJS/rainbowhelixIPAD.html) I did put in the mouseReleased so it does do something when you tap and move and then tap again.

Jeanine

SUCCESS! I added JavaScript code to the html file created in the standard way for ProcessingJS. http://faculty.purchase.edu/jeanine.meyer/ProcessingJS/rainbowhelixIPAD.html

osullic

I ignored this issue for 2 years! But approached it again this weekend, and just wanted to post my experience.

As Processing.js doesn't seem to have full support for touch coordinates, I decided to abstract mouse/touch coordinates out of Processing and just use plain JavaScript to determine these coordinates, and store them as JavaScript variables. Processing has access to JavaScript variables - so instead of using e.g. mouseX in Processing, I just reference the JavaScript variable instead.

I wrote a small on-event handler to handle mousemove, touchstart and touchmove events. I detect the type of the Event, and in the case of touch events, I access the first Touch by accessing touches[0]. Then the clientX and clientY properties give me the coordinates of the mouse or touch event. Just store these values to JavaScript variables, and they can then be accessed from the draw() function in Processing.

Success.

In HTML/JavaScript:

<script>
var coordX = 0;
var coordY = 0;

window.onmousemove = coordHandler;
window.ontouchstart = coordHandler;
window.ontouchmove = coordHandler;

function coordHandler(event) {
    switch (event.type) {
        case 'mousemove':
            coordX = event.clientX;
            coordY = event.clientY;
            break;
        case 'touchstart':
        case 'touchmove':
            var firstTouch = event.touches[0];
            coordX = firstTouch.clientX;
            coordY = firstTouch.clientY;
            break;
    }
}
</script>

In Processing:

void draw() {
    background(255, 255, 255);
    stroke(255, 0, 0);
    line(0, 0, coordX, coordY);
}

PS. My canvas is the full width/height of the browser viewport. I'm not sure if some adjustment to the above code would be required to get correct coordinates if the canvas is not anchored at (0,0) of the browser viewport for example.