Is there a way to send a combination of keystrokes to an application? In particular I would like to simulate the CMD+M (or OPT-M depending on the current keymap configuration) with Claws Mail.
I can’t figure out how solve the problem with Claws and other GTK apps that do not accept commands invoking the executable with an additional parameter (e.g. —compose to create a new message).
I was thinking of creating a dirty hack detecting the application or its window and sending it the keystrokes. I’ve played a bit with hey without much success.
Hopefully more like AutoIt 
I’ve created a simple test app whose purpose is to reroute keystrokes to another application. Here’s the MessageReceived:
void
MainWindow::MessageReceived(BMessage *msg)
{
switch (msg->what)
{
case B_KEY_DOWN:
case B_KEY_UP: {
msg->PrintToStream();
BMessenger messenger("application/x-vnd.claws-mail");
if (messenger.IsValid()) {
std::cout << "is valid" << std::endl;
// be_roster->ActivateApp(3076);
// messenger.SendMessage(msg);
BMessage message, reply;
BMessenger nm;
message.what = B_GET_PROPERTY;
message.AddSpecifier("Messenger");
message.AddSpecifier("View", "WaylandView");
message.AddSpecifier("Window", 0);
if (messenger.SendMessage(&message, &reply, 1000000, 1000000) == B_OK) {
reply.PrintToStream();
if (reply.FindMessenger("result", &nm) != B_OK) {
std::cout << "Couldn't find view" << std::endl;
}
nm.SendMessage(msg);
std::cout << nm.Team() << std::endl;
break;
} else {
std::cout << "Couldn't talk to the app" << std::endl;
}
}
break;
}
default:
{
BWindow::MessageReceived(msg);
break;
}
}
}
Everything seems correct but Claws does not respond to the _KYU message.
I’ve tried with Genio itself but it does not respond either. What am I doing wrong?
Hmm, you’re right, this is quite tricky.
I took your MessageReceived function and put it into Icon-O-Matic 
. I then made it target StyledEdit, since I can easily create a compiled version of it with debug symbols.
Using the Debugger, I’ve been able to deduce that some BView somewhere inside of StyledEdit does receive the _KYU message. It then performs the default action of checking whether the key pressed was a tab for keyboard navigation. Upon realizing that it isn’t a tab, it finishes. My current guess is that the wrong BView is getting the message. I’ll look into this more later.
With StyledEdit I think there is a problem with which view has the focus?
My understanding is that the handling of shortcuts (like menu shortcuts) should be performed at the BWindow level, every view which receives a keystroke then consumes it or reroutes the message up to the hierarchy.
Ok, I managed to route keypresses into StyledEdit. I just had to target the right window and view. Oh wait, now I realize that’s what you said in the comment above 
.
In your case, though, you’re targeting the only view of the only window. Turns out that Claws Mail must be focused e.g. through ActivateApp before it processes keystrokes.
BTW, I only managed to get typing text into a textbox working. Keyboard shortcuts aren’t working currently for me and I didn’t spend the time getting those to work. Maybe you can figure that out?
Interesting. That’s something I’ve been wondering about…
Thank you for spending some time in this!
I’ve already tried it, there is a line commented out in my code above that uses the team id. Not the best way possible but I just needed for testing.
BTW, it didn’t work either.
The thing that is weird and promising at the same time is this:
- Launch Claws
- Launch my tiny silly app
- Press CMD-M
- Claws is activated and brought up
- The shortcut has no effect on it (we already know that)
But… if I now manually hit CMD-M on the Claws window it doesn’t work either! I need to click inside the window for the shortcut to work.
I think this is the key to everything. What’s the difference between ActivateApp and clicking on its window? And why sending a key up message doesn’t to the window has no effect?
Hmm. That’s different than the behavior I observed. Pressing ALT-A in Icon-O-Matic focused Claws Mail but did not actually select all the text in the text box I had selected. Pressing ALT-A again selected everything, no need to manually click on the window.
There are only two differences that I can think of between what you’re doing and what I’m doing:
- I have
ActivateAppin a different location. I put it inside the innermost if statement where I have access tonm.Team() - I am typing into the onboarding window’s text boxes where you input your name and email address.
It might be useful to give you the code I have, but, I don’t have access to it right now. Probably not until Monday.
I observed that if I pressed a key like a, a key down and a key up message would be sent. If, however, I pressed ALT-A, only a key up message would be sent. My guess was that the view I chose in IOM was not receiving the key down messages.
The Key down message is generated when you hit the modifier, another key down when you hit the key and finally a key up when you release the key. When you release the modifier last no message is generated
I wonder if asking input_server to send inputs would worl.
Regardless: claws is a gtk linux app, and probably accepts dbus commands. In theory dbus runs on haiku, if you want to script you could try that. Otherwise it’s definetely not easy to get such apps to respond : )
How keystrokes are handled is very well documented in The Be Book - Special Topics - The Keyboard
There is nothing special about them and they can be replaced with normal BMessages. The special thing to take care of is where you send the message, rather than targetting a specific view, you have to let BWindow::DispatchMessage route it properly (to menu shortcuts, or to the view that currently has keyboard focus, and then up the view hierarchy if needed).
AFAICT I can’t instruct the input server to do so but it requires a combination of a filter and an input device. There’s a sample in the BeSampleCode package called InputRecorder, IIRC
I’ve tried but it seems that Claws needs to be built with dbus support or I’m doing something terribly wrong with dbus as I’m not a linux guy… if I may so so.
I’ve already tried that route, apparentely targeting the Window does nothing in my case
All right, I got around to it. Here’s the code I have:
case B_KEY_DOWN:
case B_KEY_UP: {
message->PrintToStream();
BMessenger messenger("application/x-vnd.claws-mail");
if (messenger.IsValid()) {
std::cout << "is valid" << std::endl;
BMessage probe, reply;
BMessenger nm;
probe.what = B_GET_PROPERTY;
probe.AddSpecifier("Messenger");
probe.AddSpecifier("View", "WaylandView");
probe.AddSpecifier("Window", 0);
if (messenger.SendMessage(&probe, &reply, 1000000, 1000000) == B_OK) {
reply.PrintToStream();
if (reply.FindMessenger("result", &nm) != B_OK)
std::cout << "Couldn't find view" << std::endl;
if (be_roster->ActivateApp(nm.Team()) != B_OK)
std::cout << "Couldn't activate app" << std::endl;
nm.SendMessage(message);
std::cout << nm.Team() << std::endl;
break;
} else {
std::cout << "Couldn't talk to the app" << std::endl;
}
}
break;
}
There aren’t many differences between my code and yours. I renamed message to probe because it was colliding with another variable in my code. I also added a call to ActivateApp which it seems you already tried. But, hopefully, somehow, this code does the trick of at least letting you type into text boxes 
it does not work for me! I have copied and pasted your code and still don’t keystrokes don’t get through… What kind of result did you get?
Ugh, it is as I feared 
.
For me, I type a letter such as ‘a’ into Icon-O-Matic, and it instantly switches to the Claws Mail on-boarding window and puts the ‘a’ into the “Your name” box which I had selected.
Now, how do we fix it… You can try replicating my setup. I put the code posted in my previous comment into src/apps/icon-o-matic/generic/gui/stateview/StateView.cpp inside of the StateView::MessageReceived function. Then compile Icon-O-Matic (jam -j$(nproc) -q Icon-O-Matic), open it, focus the canvas, and press a letter. Now that’s probably a lot of work!
Alternatively, I may do some experimenting on my part such as putting the code into the BWindow’s MessageReceived and seeing what happens then or stitching the code into a different app.
Regardless, interesting problem… The only real difference I can think of that may have some significance is I’m using a BView::MessageReceived and you’re using BWindow::MessageReceived, but, surely that can’t be it? 
Edit: Better yet, you could send me your app’s code so that I could tinker on that myself.
I think the difference is in Claws and how it is configured. If by onboarding windows you mean the configuration of the account then I don’t have it. Because Claws is already configured and up and running in my case.
I configured Claws Mail so now I’m at the home screen where you can view and compose messages. Typing into text boxes and using the arrow keys to navigate the folders works. CMD-M does not. Same as before. It doesn’t work to input into the compose window. But that’s no surprise since it’s not window 0.
I’ve finally figured out how to do it and it works!
Let me clarify that sending a keystroke to whatever app requires a bit of hacking and reverse engineering.
Sending a keystroke should be as simple as such, we already have set up almost everything correctly. However, some applications do not simply respond to a B_KEY_UP message but require a precise sequence of messages.
It’s the case with Claws.
To analyze what happens inside the application first of all I have built a utility called spybmessage.
The source code is under the src/bin directory and apparently it’s not shipped with a standard installation.
spybmessage accepts the path to an application that will be started by hijacking its looper and installing a filter for the application itself and each of its windows.
This will let you see all the messages going through the app.
The Claws instance started within spybmessage clearly has a different signature so I needed to make some changes to the code above to send the keystroke messages to the right Team.
In the second place I have added a BButton to the Window to trigger all the logic. This is necessary because Claws needs a number of messages sent in a row in the precise order to be able to process the keystroke correctly. I think this generally applies to other applications to some extent.
After some reverse engineering sessions I have figured out the correct sequence of messages:
// build B_MODIFIERS_CHANGED
BMessage mchm('_MCH');
// build B_UNMAPPED_KEY_DOWN
BMessage ukdm('_UKD');
// build B_KEY_DOWN
BMessage kdm('_KYD');
// build B_KEY_UP
BMessage kum('_KYU');
Claws is now able to open the compose message window reacting to the CMD-M keystroke.
The application should be activated before sending the messages aotherwise nothing happens. Moreover, BRoster::ActivateApp does not work if the application has no window on the screen.
Here comes a dirty hack:
// ActivateApp activates the app only if it has a window on screen
// let's try with a hack by calling a private API
// AS_MINIMIZE_TEAM = 5
// AS_BRING_TEAM_TO_FRONT = 6
BPrivate::AppServerLink link;
link.StartMessage(6);
link.Attach<team_id>(messenger.Team());
link.Flush();
if (be_roster->ActivateApp(messenger.Team()) != B_OK)
std::cout << "Couldn't activate app" << std::endl;
else
std::cout << "Destination team" << messenger.Team() << std::endl;
It brings the team’s windows to the front even if they are minimized.
I’m not particularly proud of it (oh well, I am very proud of the overall solution but, you got it… 
) but I don’t know if there’s any public API or a clean way to achieve this. Can anybody chime in?
Here’s the full hacky code:
void
MainWindow::MessageReceived(BMessage *msg)
{
switch (msg->what)
{
case '__go':
{
BMessenger messenger;
msg->PrintToStream();
BMessenger msn("application/x-vnd.claws-mail");
// BMessenger msn(nullptr, 6143);
if (msn.IsValid()) {
std::cout << "is valid" << std::endl;
BMessage probe, reply;
probe.what = B_GET_PROPERTY;
probe.AddSpecifier("Messenger");
probe.AddSpecifier("View", "WaylandView");
probe.AddSpecifier("Window", 0);
if (msn.SendMessage(&probe, &reply, 1000000, 1000000) == B_OK) {
reply.PrintToStream();
if (reply.FindMessenger("result", &messenger) != B_OK)
std::cout << "Couldn't find view" << std::endl;
// ActivateApp activates the app only if it has a window on screen
// let's try with a hack by calling a private API
// AS_MINIMIZE_TEAM = 5
// AS_BRING_TEAM_TO_FRONT = 6
BPrivate::AppServerLink link;
link.StartMessage(6);
link.Attach<team_id>(messenger.Team());
link.Flush();
if (be_roster->ActivateApp(messenger.Team()) != B_OK)
std::cout << "Couldn't activate app" << std::endl;
else
std::cout << "Destination team" << messenger.Team() << std::endl;
// build B_MODIFIERS_CHANGED
BMessage mchm('_MCH');
mchm.AddInt64("when", 9274901865); // when = int64(0xbf28d629 or 3207124521)
mchm.AddInt32("be:old_modifiers", 32); // key = int32(0x52 or 82)
mchm.AddInt32("modifiers", 1058); // modifiers = int32(0x402 or 1026)
mchm.AddUInt8("states", 0); // states = uint8(0x0 or 0 or '')
messenger.SendMessage(&mchm);
// build B_UNMAPPED_KEY_DOWN
BMessage ukdm('_UKD');
ukdm.AddInt64("when", 9274901865); // when = int64(0xbf28d629 or 3207124521)
ukdm.AddInt32("key", 102); // key = int32(0x52 or 82)
ukdm.AddInt32("modifiers", 1058); // modifiers = int32(0x402 or 1026)
ukdm.AddUInt8("states", 0); // states = uint8(0x0 or 0 or '')
messenger.SendMessage(&ukdm);
// build B_KEY_DOWN
BMessage kdm('_KYD');
kdm.AddInt64("when", 9274925974); // when = int64(0xbf28d629 or 3207124521)
kdm.AddInt32("key", 82); // key = int32(0x52 or 82)
kdm.AddInt32("modifiers", 1058); // modifiers = int32(0x402 or 1026)
kdm.AddUInt8("states", 0); // states = uint8(0x0 or 0 or '')
kdm.AddInt8("byte", 109); // byte = int8(0x6d or 109 or 'm')
kdm.AddString("bytes", "m"); // bytes = string("m", 2 bytes)
kdm.AddInt32("raw_char", 109); // raw_char = int32(0x6d or 109)
messenger.SendMessage(&kdm);
// build B_KEY_UP
BMessage kum('_KYU');
kum.AddInt64("when", 9275046493); // when = int64(0xbf28d629 or 3207124521)
kum.AddInt32("key", 82); // key = int32(0x52 or 82)
kum.AddInt32("modifiers", 1058); // modifiers = int32(0x402 or 1026)
kum.AddUInt8("states", 0); // states = uint8(0x0 or 0 or '')
kum.AddInt8("byte", 109); // byte = int8(0x6d or 109 or 'm')
kum.AddString("bytes", "m"); // bytes = string("m", 2 bytes)
kum.AddInt32("raw_char", 109); // raw_char = int32(0x6d or 109)
messenger.SendMessage(&kum);
break;
} else {
std::cout << "Couldn't talk to the app" << std::endl;
}
}
break;
}
default:
{
BWindow::MessageReceived(msg);
break;
}
}
}
4 Likes