Keyboard events and non-selected windows

reepca

Hello, so I was making a bit of an experimental project involving timed clicks (kind of like a macro, but with precise timing and the like) using Java's Robot class, and it worked as intended except for one problem - the way I toggled the timed clicking on and off was by changing a variable in keyPressed() depending on what 'Key' was when keyPressed() was called. Although the Robot was plenty willing to click outside of the display window, the program would stop registering the keys being pressed as soon as the display window was no longer selected - a considerable problem given that the reason I made the sketch was basically to push buttons outside of the sketch window. Is there a way to cause my program to listen to all keyboard events, not just the ones directed at it? So that I can have another window selected and still have it be capable of responding to keyboard input (for example, screen-recording software can be stopped by pressing a hotkey, but you can still have another window selected and hearing your key presses).

Any suggestions?

This doesn't really deal with core or contributed libraries, it's not really a hardware thing, and this seems like the most general place to put it, but apologies if there is a better place this should be.

KevinWorkman

This might not be exactly easy, as Java was designed to be pretty self-contained. To really get the job done right, you'd need to write native code. Gross.

You might be able to hack it by calling frame.toFront() or frame.requestFocus(), but those aren't guarantees, so you might also want to add a FocusListener to the Frame.

reepca

I'm somewhat new to general Java, having learned mainly with Processing, and I'm wondering if you could give me a general run-down of how the event listening system works (or redirect me to an existing explanation), as I tried looking at the Java API and felt somewhat lost.

Thanks for your time!

KevinWorkman

Recommended reading: http://docs.oracle.com/javase/tutorial/uiswing/events/index.html

reepca

Thanks! I took a look at it, but it seems that key press events are only dispatched to the window that has keyboard focus (and I can't find any way to change that), so to truly get it to work like those fancy screen-recording programs and stuff do would probably indeed require writing native code :'(. Makes me sad.

Thankfully, though, I think you've shown me a neat work-around. If I can use a FocusListener to know when my PApplet loses focus, I can simply call frame.requestFocus() and it should get focus right away again? So that when I have my Robot click on another window, I can cause my PApplet to immediately regain focus so that I can press the button to turn the macro off whenever I want?

Should that work out? It sounds like it should :).

KevinWorkman

You can try that approach, but note that the requestFocus() method can't guarantee that the OS actually gave the window focus.

Read more info on requestFocus() and requestFocusInWindow() in the Java API for Component, which is a parent class of Frame: http://docs.oracle.com/javase/8/docs/api/java/awt/Component.html#requestFocus--

reepca

Alright, so I tried adding a FocusListener (well, actually just something that extended FocusAdapter) to frame, but the behavior is really... strange. For starters, focusLost() isn't called until the program ends (in the FocusListener's focusLost() method is where I try to call requestFocus()/requestFocusInWindow()). And the behavior of frame itself as far as focus is concerned is bizarre. Calling isFocusable() returns true, calling requestFocusInWindow() returns false, and calling hasFocus() always returns false (even when the display window title bar shading indicates it's focused and the display window has been clicked to make sure it has focus).

Furthermore, requestFocusInWindow(), requestFocus(), and toFront() all appear to never cause the frame to gain focus - even after frame.setFocusableWindowState(true) has been called. Then again, by the behavior of the FocusListener, it never loses (or gains) focus, and by the behavior of hasFocus() it never has focus to begin with.

Any idea why such strange behavior with the entire focus system is occurring? Looking back at it, requestFocusInWindow() would only grant focus if it was called by a component within a window that already has focus, right? So if I want the top-level component to gain focus, I would need to call requestFocus()?

I guess it's probably a silly question, but frame IS the top-level component, right?

GoToLoop

Dunno barely anything. So it's just some silly info:

• Since PApplet extends Applet, it means it's also a Panel, Container & Component from AWT.
• Also, PApplet.main() instantiates a JFrame. So it's Frame, Window, Container & Component from AWT.

http://docs.oracle.com/javase/8/docs/api/java/applet/Applet.html
http://docs.oracle.com/javase/8/docs/api/javax/swing/JFrame.html

reepca

So I think I figured some of this out. The reason that the focus system seemed to be behaving so strangely is because frame isn't the component that does the listening for keyboard events - the PApplet is. Which is what i'm writing the code in. Meaning that where I was putting frame.requestFocus(), I should have simply been putting requestFocus() (or this.requestFocus()... or whatever). Upon making those changes, things started making a heck of a lot more sense - and I can even make the application gain focus! Well, once anyway. Putting requestFocus() in draw causes it to regain focus after you click somewhere else, but unfortunately only for the first click. Which isn't really what I wanted, so it's a half-success.

Interestingly enough, putting requestFocus() in the FocusListener's focusLost() method causes it to regain focus... according to the console, anyway. Any time that I click outside of the window, focusLost() is called, which then calls requestFocus(), which then calls focusGained(), which prints to the console the focus state of the PApplet using hasFocus() and tells me that it has focus... but apparently the OS didn't get the memo or something, because it doesn't bring the display window to the front or, more importantly, cause it to gain keyboard focus and start being dispatched those events.

Any idea about why requestFocus() only seems to work the first time around, and not at all when called by the FocusListener?

KevinWorkman

The requestFocus() method does not bring the frame to the front. For that, there's a handy toFront() method.

However, even calling toFront() doesn't seem to work.

I've created an example program for you. Posting an MCVE like this makes it much easier for others to experiment with your code:

import java.awt.event.FocusListener;
import java.awt.event.FocusEvent;
import javax.swing.SwingUtilities;
import javax.swing.JFrame;

void setup() {

  frame.setAlwaysOnTop(true);
  addFocusListener(new FocusListener() {

    public void focusLost(FocusEvent e) {
      println("Focus lost.");


      SwingUtilities.invokeLater(new Runnable() {
        public void run() {

          int state = frame.getExtendedState();
          state &= ~JFrame.ICONIFIED;
          frame.setExtendedState(state);

          frame.setVisible(true);
          frame.toFront();
          frame.requestFocus();
          requestFocus();
        }
      }
      );
    }

    public void focusGained(FocusEvent e) {
      println("Focus gained.");
    }
  }
  );
}

void keyPressed() {
  println("key pressed: " + key);
}

void draw() {
  ellipse(50, 50, 10, 10);
}

I've tried a few different things, but none of them worked like I expected. Maybe somebody else can take a stab at it.

I believe what's going on is actually the fault of the OS. Like I said, the focus system is at the mercy of the OS, and apparently Windows has a "feature" where instead of coming to the front, a window's icon on the taskbar will flash. This leaves it up to the user whether a window should be brought to the front, but it complicates things for a programmer who wants to create a window that stays in the front.

More info and other hacks you can try here: http://stackoverflow.com/questions/309023/how-to-bring-a-window-to-the-front

GoToLoop

It worked for me in Lubuntu 13.04 64-bit! =:)
And I've transfered all that boilerplate into init() and placed it in another tab for organization sake: *-:)

P.S.: After 2nd or 3rd focus lost, both focusGained() & focusLost() starts to fire uncontrollably! @-)

"Request Key Focus.pde":

---

/**
 * Request Key Focus (v3.11)
 * mod KevinWorkman & GoToLoop (2014/Jun)
 *
 * forum.processing.org/two/discussion/5316/
 * keyboard-events-and-non-selected-windows
 *
 * stackoverflow.com/questions/309023/how-to-bring-a-window-to-the-front
 */

void setup() {
  size(300, 200, JAVA2D);
  noLoop();
  frameRate(20);
}

void draw() {
  background((color) random(#000000));
}

void keyPressed() {
  frame.setTitle("Key " + (key == CODED | key <= ' '
    ? "unicode: " + keyCode : "pressed: " + key));
}

---

"Focus.pde":

---

import java.awt.event.FocusListener;
import java.awt.event.FocusEvent;

import java.awt.EventQueue;
import java.awt.Frame;

@ Override void init() {
  super.init();

  frame.setAlwaysOnTop(true);

  final Runnable r = new Runnable() {
    @ Override public void run() {
      int state = frame.getExtendedState() & ~Frame.ICONIFIED;
      frame.setExtendedState(state);

      //frame.setVisible(true);  // disallow minimize.
      //frame.toFront();         // no need in Linux.

      frame.requestFocus();
      requestFocus();
    }
  };

  addFocusListener(new FocusListener() {
    @ Override public void focusGained(FocusEvent e) {
      redraw();
      //print("Focus regained!\t\t");
    }

    @ Override public void focusLost(FocusEvent e) {
      EventQueue.invokeLater(r);
      //print("Focus lost!\t\t");
    }
  }
  );
}

---

KevinWorkman

The problem is that this depends completely on the OS. I'm not surprised this works on Linux, as the problem here is Window's "flash the taskbar instead of popping to front" feature.

reepca

Sorry about not responding for a long while, frankly the entire discussion sort of slipped my mind - thankfully after I sort of got it to work. The same code (I think, I don't remember making any changes since) that I had tried on my Windows XP laptop, where focusLost() calls requestFocus(), upon trying it on my brother's Windows 7 computer, produced the desired effect - it would regain keyboard focus every time it lost it, even though occasionally it would not visually appear to be at the front (and not flashing in task bar, either). So for some reason it seems to have worked for my brother, who wanted to use it, but not for me, who wanted to make it work. -.-

An alternative I noticed when googling was a library called JNativeHook that someone on StackOverflow linked to, which I didn't end up giving a try due to the unexpected success on my brother's computer, but which could be useful for someone trying to make a somewhat less shabby and work-around way of doing it (I'm assuming it's basically the afore-mentioned dread "native code" written for you?).

Thank-you for your informative and helpful answers! Is there a way to close the topic?