Input stack improvements

Development OS
#1

This is a topic we already discussed on both mail list and gerrit, but forums seems to be more popular nowadays so lets talk about it.

Haiku input stack has some points that can be improved and there are some missing essential features (for 2020’s standards). Those are some that comes to my mind now:

  • There is no HID Consumer support, basically, media keys, but complex items as volume knobs also fall here.
  • There is no multitouch support. This shouldn’t be difficult to add at application level the way Joysticks work now: SDL, Kivy, Qt, etc. App_server gestures is out of this scope.
  • There is no way to know what device id/vendor an input device belongs to, even device names are generic: USB Mouse 1
  • Imho, we need an easier way for users to at least, blacklist usb/i2c devices. A way to load quirks would be cool too.
  • Is i2c/hid working? Code seems to be there but I’ve had no luck with it.
  • SPI/hid is already a thing, at least on Apple devices, we have nothing here.
  • Bluetooth/hid is not working also. I don’t know what is missing here.

I have submited some patches for some of this points, I have some wip code for others, but in general, all of who are involved here agree that some refactor is needed and patches have to wait once we have a better input architecture.

HID drivers are, imho, a bit outdated. They send rigid structures to user space. This works well for simple devices but fails to accomodate modern complex ones.

A proposal has been made to move HID code from kernel to user land, altough it isn’t a hard task, it would need to rewrite a lot of input server device add-on code. I am researching the possibility of having an hybrid approach so both input models can live together while migration is done, but I haven’t arrived to a solution yet.

So… feedback is welcome. I am not a guru here, I’ve just want my devices to work on haiku and started working on it the best I can.

16 Likes
#2

The class which all input_server device add-ons extend upon, BInputServiceDevice, along with their local counterpart, BInputDevice, provide a Start() and Stop() method which could be wired up to an Enable/Disable toggle button in the Input preflet. This could be an easier fix for now.

#3

I agree that HID parsing should be handled in input_server add-on instead of kernel. Kernel driver should send/receive raw input packets.

It will be mostly moving kernel add-on code (Keyboard/Mouse/TabletProtocolHandler) to input_server add-on. Some new hid input_server device add-on can be introduced. Existing device add-ons can be kept as is for PS/2 devices.

#4

Look at the accelerant drivers, you could in theory make a new accelerant, then pipe info as a hid driver

#5

To sum it up: Haiku is missing drivers. Honestly I don’t think that moving code around will help much. Bluetooth/hid is not working, it’s not supported on OpenBSD either I’d say.
Everything is possible, but it’s not a priority for R1 obviously. FWIW the open input bug and enhancement list is already available here: Custom Query – Haiku

#6

I think it will help a lot with i2c and Bluetooth HID support. For my hardware I just implemented hardcoded packet reader and get data with i2c /dev node.

#7

Polling should only be a fallback. Why not implement ACPI interrupt support code instead?

1 Like
#8

The code for HID parsing is in a static library, which is already used by the i2c and usb drivers. This is not a perfect architecture, but I don’t see how moving this code around will make things easier.

Currently joysticks don’t go through input_server at all. There is a specific protocol for joystick/gamepad devices with the BJoystick class directly accessing the device. This should probably be kept as is to lower the latency as much as possible (direct communication between the driver and games using the joystick).

We could still move HID parsing to userspace, but then we also have to make changes to BJoystick, and see how we are going to handle legacy gameports and possibly other non-HID inputs. And the driver still has to decide if it should publish a joystick device for BJoystick or an input device for input_server.

#9

I think the problem is not where HID parsing is done either user space or kernel. The problem I see with current model, we send very rigid event structures. Mouse events are something like that:

typedef struct {
    bigtime_t timestamp;
    int32 buttons;
    int32 xdelta;
    int32 ydelta;
    int32 xwheel;
    int32 ywheel;
} mouse_movement;

So, this is modeled around a prototype mice, sending xwheel even if the mice lacks that feature, and obviously not making room for all extra buttons gaming mouses have. Perhaps, I am being a bit dramatic here and we only need to grow that structure with something like int32 keys.

For reference, Linux uses a small structure like this:

struct input_event {
        struct timeval time;
        unsigned short type;
        unsigned short code;
        unsigned int value;
};

Then, a mouse movement would send something like type: EV_REL, code: REL_X and whatever value. So, using a generic structure devices can send all kind of events. If am not wrong, there is an ioctl to query device capabilities.

My biggest issue are consumer controls, there are hundreds of them so mapping on int32 it is not feasible. Even using an array like keyboard does is pointless. Things get worse because consumer controls can be embedded on mouse/keyboard reports.

About bluetooth hid… What is missing?

1 Like
#10

Testing kernel code is much harder and error prone. input_server add-ons can be easily reloaded and server can be restarted on add-on crash.

HID parser can be moved to shared library. Applications can use input_server to identify and enumerate joysticks, but handle HID packets directly.

#11

There is already space for 32 mouse buttons there. Do you need more? If so, maybe your mouse is in fact a keyboard? :stuck_out_tongue:

That is a more valid point. To me, moving the HID parsing to input_server is a solution to that, which avoids designing a new protocol (we just reuse HID all the way up in our input stack). If we keep the HID parsing on the drivers side, we then have to design our own input protocol, or adopt Linux’s one.

Removing HID parsing from the drivers also moves more code to userspace, which is generally a good thing. The possible downside I can see is the open question of how to handle joysticks/gamepads (in a way that doesn’t add more latency by having an extra userspace process involved, and that also doesn’t require to still do some HID parsing on the kernel side).

Pretty much all of the bluetooth stack. We can pair devices but I’m not sure they are published in the device manager and if it’s possible to write drivers for them. And I don’t even know how well the pairing part works.

Also, bluetooth is complex because it’s partly devices (pairing, etc) and partly network (there are MAC addresses and several protocol layers). I don’t know what’s the best way to fit this into our driver model, and also how much of it we want to run in kernelspace?

Yes, I did not say anything against that. But that doesn’t help a lot as you said with I2C or bluetooth, where you would otherwise be reusing already written and tested code (the usb_hid library). It would solve several other problems (more flexible handling of devices, less code in kernel, easier to handle different device types that are not mouse and keyboard, easy access to a lot of device info from userspace). It would also introduce some new ones (how do we handle joystick/gamepads if we don’t want them to go through input_server?). Overall it seems to be a good move.

#12

Already answered in previous post edit.

#13

This problem can be solved by moving HID parser to userland because it can produce BMessage directly from HID packets and avoid use of rigid structures.

This looks like reinventing HID protocol.

1 Like
#14

This looks like reinventing HID protocol.

Yeah kind of… but a lot simpler. In fact, I had in mind to just use HID usage codes instead of reinventing new ones.

#15

Ok, lets say, hid driver just exposes “raw” endpoints on user space, ie: /dev/input/usb/hid0. Then, each time you query BJoystick for device count, it has to open and parse all hid* endpoints looking for a valid joystick report.

On the other hand, once the joystick report is parsed I don’t see any reason for BJoystick to increase latency compared to a kernel based approach. I guess kernel process has a higher priority than user space one, but joysticks use to work at 125Hz polling rate. Even on Linux there are user space joystick drivers so, I guess we have nothing to fear.

#16

Yes, the scenario I would be worried about is if we do something like device → input_server → app using BJoystick.

We want something direct between the device and BJoystick as it is now.

The question is, how will input_server and BJoystick decide which device is handled by input_server or by BJoystick. Currently this is decided by the driver which publishes the devices in different directories. If we remove HID parsing from the driver completely, the driver has no idea anymore. Which means it probably ends up in the responsibilities of input_server. I guess there would be a BJoystickRoster or BInputDeviceRoster class, that would handle input devices in input_server. Then BJoystick would communicate with that to enumerate the joysticks, and then take control of one by opening its device.

#17

To me the issue seems to be that to much functionality is being placed in one driver structure. Why does joystick have to use same driver as keyboard ???

This is exactly why beos had the accelerant/add-on driver design. As long as the hid /usb stack can parse and assign a device handling structure, there’s no need to make a swiss army knife out of hid.

My hot take is tgat hid should parse to function of device, and then expose to a handler or end point etc

#18

Because both use the same HID protocol. In BeOS times joysticks were usually connected to dedicated game port with different protocol so separate driver was relevant.

#19

USB HID devices don’t say “hey Im a joystick” or “hey I’m a keyboard”. They all say “hey I’m an HID device”. Then you have to parse a descriptor which adds some more info, usually it looks something like “oh I have 48 buttons, 2 axis, and a pointing device” or something like that. Deciding if something is a joystick or a keyboard from that is not as easy as everyone initially thinks. The hardware is already designed as a swiss army knife (the spec probably really includes that, at least it includes ways to say if a device is a baseball bat, there are different descriptors for a complete set of golf clubs, and it’s also possible for devices to describe where each button is located and which finger should be used to reach it).

That’s why there is a single driver: until we parse all this data, we have no idea if the device is a keyboard, a joystick, or just the volume buttons on your USB speakers (probably also exposed as HID).

Only for graphics cards. For everything else the drivers were fully in the kernel.

#20

Not being familiar with hid, don’t the device’s use a description string ? If not wow, that’s a massive design flaw. Maybe drop support for everything that doesn’t use identity/function broadcasting.

Haiku shouldn’t be trying to support every piece of hardware, particularly if it’s poorly designed.

Is it possible to make a addon template parser that say a user could enter the hid information button functions, device name and then the stack would decode from a definition-text file thus giving everyone the ability to map devices that are off spec manually ???

Kind of like the hardware abstraction layer in Machinekit/LinuxCNC

Or a more universal type of the older bjoystick add on you were developing ??? But at the system level for all devices and extend input server to use those files