if someone gets webcams working, I’ll buy it
1 Like
Some thoughts I had on this days researching.
- I am not convinced at all about using HID for wire protocol, HID reports have no timestamp so I am not sure if we should have to piggyback a timestamp on each report is passed to user space.
- We already have a pretty decent amount of HID Usage constants, we talked about using them for all input devices (ie: ps2, acpi) but I find it lacks some definitions for common acpi events: power off, key brightness up/down, rfkill,…
- Perhaps we can extend HUT with a custom namespace for those acpi events
- Our HID driver is too big. There is a lot of repeated code. USB Device and I2C Device are 99% equal, just some lines reading from usb interrupts or i2c ones.
- Mouse, Keyboard, Tablet and Joystick drivers are 80% the same.
I am trying to simplify all of this. Moving device interpretation to user space would allow us to have a cleaner driver. We may have to keep joysticks and boot keyboards (the one used at KDL) as separate drivers.
6 Likes
With the current design, we don’t have to use it for all devices. input_server can have different add-ons for HID and non-HID devices if that’s desirable. It would be similar to what we have for graphic cards: each device type comes with a driver-accelerant pair.
Yes, I did the minimal effort to move the HID parsing code to a static library instead of having it copied in all drivers. But more code could be moved to the library, which ideally should be turned into a module or perhaps even a separate driver.
In the separate driver case, USB and I2C would just provide an HID bus (in the newstyle driver system) and HID devices there. The devices would have functions like read_hid_descriptor, read_hid_report, write_hid_report. Then the HID driver would find these devices, call these functions, and finally publish something in userspace. So that’s yet another possible design.
It really depends where we want things to happen. Right now there is one driver for USB, and another driver for I2C, and the HID parser is a static lib that is eventually duplicated into both. That’s of course quite bad.
So our options are:
{USB,I2C} driver → publish HID device node in device tree → HID parser in HID driver → publish input device in /dev/input/generic/ → standardized “events” with generic input_server add-on → input_server
Or:
{USB,I2C} driver → publish HID device in /dev/input/hid/ → HID events (wrapped as appropriate) through the read()/ioctl() functions → HID parser in input_server HID add-on → input_server
The first solution allows us to have only one input_server add-on for everything, but requires us to define our own “events” protocol in the driver interface. Which is annoying because it’s yet another thing to standardize, and it constrains all future drivers to use that same interface.
I think the second solution gives us a bit more flexibility, the only end where we need to standardize is “how do we inject events into input_server?” and that’s a C++ API which can easily be extended as needed, and where we can provide some backwards compatibility. The second case also allows us to move a lot of the code to userspace (all the HID parsing).
If we look at the same example for, say, an ACPI input driver:
In the first case:
ACPI driver → publish ACPI nodes in the device tree (that’s already done) → ACPI_button driver → publishes input device in /dev/input/generic → from there it’s all the same code as above
In the second case:
ACPI driver → publish ACPI nodes in the device tree (that’s already done) → ACPI_button driver → publishes input device in /dev/input/acpi/ → ACPI input server add-on that knows how to handle these devices → input_server
In either case, there is no more /dev/input/keyboard/ and /dev/input/mouse/ directory. But more annoyingly, the second approach, because it moves the HID parsing to userspace, would be unable to provide /dev/joystick support for BJoystick. We could make joystick events also go through input_server, but it goes a bit against the idea that the Game Kit would be a quite low-level thing giving as direct access as possible to the hardware for lower latencies. So that’s certainly a win for the first solution, and a good justification for keeping the HID parsing in the kernel.
That’s how is implemented on Linux. Their HID driver is pretty simple, just map HID events into their generic input transport (evdev) and let user space deal with that. No mice/keyboard differentiation.
I know I am too biased by Linux input stack design :\
Which is annoying because it’s yet another thing to standardize, and it constrains all future drivers to use that same interface.
Imho, we currently are too far from standardization and that have ended in a lot code duplication and rigid input flows. In both using HID or a generic wire protocol, we can live with new and legacy input addons.
Also, I can’t come with some example where a generic protocol doesn’t fit for a device. But again, if there are, they can still living as is. Wacom perhaps?
It seems like we can wrap PS2 devices with HID, perhaps, we can use HID for… HID devices :D, fake PS2 with HID and just create a simple protocol for acpi events.
Some day we will have to implement platform drivers for Lenovo, Asus, HP, Slimbook, Tuxedo,… I know it sucks, but they are mandatory for basic functionality. Also, it has no sense to have several custom acpi/wmi driver protocols, and would be cool to route some of this events to user space, not just feed app_server.
But more annoyingly, the second approach, because it moves the HID parsing to userspace, would be unable to provide /dev/joystick support for BJoystick
Yes, I am aware of Joystick case. In any scenario, we need HID parsing on kernel, at least to separate joystick and catch keyboard for KDL (Linux also catch a fail-safe mice not sure why). So, we can simplify HID on kernel but it will still be there. If we fake PS2 devices as HID we also need it on kernel side.
It really depends where we want things to happen. Right now there is one driver for USB, and another driver for I2C, and the HID parser is a static lib that is eventually duplicated into both. That’s of course quite bad.
Yeah I know that was a temporal workaround, I was talking about original code. Protocol handlers, for example, are all doing almost the same, they cherry-pick some HID events and put them into a rigid structure.
The fact that it’s done this way in Linux does not automatically make it a good or a bad design.
Maybe there is no need to aim for a perfect design in the first try and a huge commit where you rewrite everything. Make gradual changes. Try an approach and find out how far you can take it before running into problems. Submit patches going in one direction and see how it goes.
That is how the current code ended up where it is, it isn’t so much designed this way, but everyone trying to play with the constraints and the vision they had at the time: BeOS developers designing the Game Kit to be low latency, us trying to keep things compatible with BeOS, not anticipating that USB-HID would end up being used also by I2C and Bluetooth a few years later, and so on. So yes, of course the end result isn’t that great.
But rather than the One Big Refactor To Solve All Problems, let’s solve problems one at a time. HID needs to be more separate from the drivers. Due to BJoystick, moving it all the way to userspace seems not possible. So let’s make it a kernel module or a driver.
ACPI needs a different event structure. It can be made generic and maybe later the other drivers can be updated to also use it. Or maybe we can start by making the “generic” event structure and have the existing drivers use that, and later on that makes adding the ACPI driver easier. The events for mouse and touchpads was recently changed, and we can certainly change it again.
But I think trying to fix all the problems at once is going to result in endless discussions, people arguing about things no one care about like USB Golf Irons, and no code being written.
1 Like
I’ve never been a Linux fan, that’s why I am on BeOS/Haiku culture since 2000. But, on this approach I have to say they nailed it. Perhaps some ACPI events as keystrokes is going too far (ie: Lid switch or tablet orientation).
Maybe we should come with an ACPI protocol in order to fit events and sensor readings like temperature and fans speed.
not anticipating that USB-HID would end up being used also by I2C and Bluetooth a few years later, and so on. So yes, of course the end result isn’t that great.
I don’t pretend to be destructive or even be critic with anyone. As a 20 year experienced engineer I am aware that designs get old no matter how smart you try to design it.
Maybe there is no need to aim for a perfect design in the first try and a huge commit where you rewrite everything. Make gradual changes. Try an approach and find out how far you can take it before running into problems. Submit patches going in one direction and see how it goes.
I swear I am trying it, but sometimes keeping two designs working at the same time with overlapped functionality is harder than a full blown refactor. But I have it on mind.
But I think trying to fix all the problems at once is going to result in endless discussions, people arguing about things no one care about like USB Golf Irons, and no code being written.
True. I already have a branch with a wip driver. I would like to have something to show at the end of this summer.
I didn’t mean “incremental” in the “let’s have two completely independant stacks side by side” sense, but more in the “let’s refactor all drivers to do one change, and then refactor them again 2 weeks later to do another change”. Seems reasonable because there aren’t that many input drivers at the moment.
I know Linux isn’t the best source for everything under the sun but everything isn’t bad under it. My wife wanted to use her 3rd party nintendo switch controller with wine. I tried the already available drivers for switch. Didn’t work because they only work for joycons and the switch pro controller. Turned out didn’t need them anyway. I wrote a small program to read joystick input from the input stack and was able to read all inputs except for a couple of new buttons linux isnt aware of… That didn’t help make it work in wine either. Wine prefers XInput+ ready devices for less headache. So I pulled down the xboxdrv code and fixed it to recognize the signature from the controller and allow remapping. Remapped the controller and now it works like a charm as an xbox360 controller under wine.
You could easily do something like this to filter all input and hand off to the driver registered to that device using a signature like that. The old BJoystick driver could be cannibalized to push the inputs into this stack as a generic joystick device, to keep old PC gamepads working. Then BJoystick can be refactored on top of the new stack and improved a bit but tha would prob break binary compatability.
I haven’t looked at that portion of haikus code in the last few years, so I don’t know whats there now. SDL also takes the event filter route and it does work rather well for X unknown devices at a time.
5 Likes
Yeah, I agree 100%. Linux input stack is plain simple and handy. Events can be read from user space using a simple ioctl based API, there are handy cli tools. It allows also, to create fake input devices from user space and push events into it! Useful for libusb-user-space drivers. Input API on kernel is also quite simple.
They use evdev for mices, tablets, keyboards, gamepad/joysticks and ACPI/WMI devices. There are even thirdparty drivers doing a kernel->user space->kernel->user space path.
Joysticks, however, are used directly from /dev endpoint the way Haiku does, while other events are piped into X11/Wayland stack.
I’ve been working into something similar, but I still have a lot of work to do, and I lack some knowledge that ends on bad designs I have to constantly address.
I think that placing a lot things in kernel is bad idea. If something can be placed in userland (HID parser and event generator for example), it should be placed in userland.
4 Likes
Would there be a difference in input latency between kernelspace and userspace input handling? If there is, even a seemingly negligible one, then go for whatever is lowest/fastest.
It wouldn’t because Haiku input architecture involve input_server add-ons anyway and efficiency will not change if move HID parser from kernel to input_server add-on. Swapping addends doesn’t change the sum.
7 Likes
This dnesn’t look like a clean and simple design to me…
In Haiku if you want to inject input events from userspace, you do it with an input server add-on and don’t need to feed the events into the kernel.
And if you need to access some hardware from a driver, and only in that case, you can pair your add-on with a kernel-side driver.
2 Likes
Clean and simple from user-space point of view. On Haiku this makes no sense because we have input-server add-ons and they (after several architectural changes) do not. So, the only portable way they have to synthesize input events is creating a fake device, and that’s something only kernel can do.
But, as I said, from user and user-space developer point of view, I am quite happy. In fact, kernel side it is also easy to work with.
I was working on a refactor of the input stack. I’ve been busy last two months so I haven’t finished yet, but, I would like to talk about so you mates can provide me some feedback about the design.
- There is a HID module with no user-space interface. HID drivers (usb, i2c,…) register at it so the module keeps track of a unique /dev/input/hid* pattern. A simple HID parsing can be also factorized in this module, but I am not sure if this would be needed, perhaps, in order to grab debug keyboards but I have a feeling that this can be done without having to do a deep report descriptor parsing.
- HID drivers manage its own user space interface (read, write, control, …)
- HID drivers just send raw HID events down to user space land or report descriptors on demand.
- Input server add-on just grabs all /dev/input/hid*
My doubt here, do I need a single big input add-on for all keyboard, mouse and touch devices? In fact, this is another doubt I have… Is it possible for two processes on user space, to open a device? Because I had planned BJoystick could open hid device and parse them looking for a compatible device, but this would happen simultaneously with input server doing a similar parsing.
I have the idea of reducing kernel side complexity, and also, reducing code/binary duplicity, which right now we have some. But it looks like this ends with a dirty user space side :\
Suggestions are welcome 
4 Likes
Yes, that’s not a problem. Each user gets a file descriptor and the driver manages that on its side (usually named “cookie” in the drivers), it can store persistent data in the cookie if needed (typically, file offsets for things that implement a read/write API will be stored in the cookie).