How to scale the UI properly with High DPI screens?

Development
61

Reading a bit more about typography, it seems to make sense to me at this point that we should not be using different point sizes to scale text; we should instead be having DPI (or some other “scaling factor”) affect the pixel size chosen from the corresponding point size. Thus, you might set a point size of 12, but get a pixel size equivalent to 1.5 times that due to DPI, etc.

Practically speaking, this would mean we add a “pixel size” to fonts in addition to the current (point) size, and then use the pixel size when computing metrics generally, which will be formed based on the point size and then a DPI (or another “scale factor”, per-monitor etc.)

4 Likes
62

That’s technically correct but:

  • It assumes you can reliably know the DPI of the display (from my experience, EDID data is often wrong about it)
  • It assumes that people want to think in typographic points, which I think is not the case. Either you need to do things accurately with pixels, or you don’t really care and you just want an unlabelled slider going from “tiny” to “huge”. The case where it would matter is if you do desktop publishing or word processing or things like that, where it’s convenient to set the zoom level in your app to 100% and be sure that the thing shows in actual size, as it would be printed. But in user interface, you don’t care about this, you just want the text to be large enough that you can read it, and small enough that you can fit as much data as possible on screen.
  • I see no reason to have two separate things in the user interface in appearance settings, or to have two separate APIs for the DPI and font size and ask developers to somehow combine the two to figure out an appropriate size for their controls. The only thing it would allow is people using millimeters or inches to specify how their user interface looks. Do we need that?

Also, we already have a pixel size for fonts. That’s font->GetHeight which gives the ascent and descent in pixels. Currently the display DPI is not taken into account for this. We could add it to the computation, but for UI development it essentially means “never use the font size to do anything because its relation with pixels depends on the display DPI”.

I think it is simpler to say that our current font size is not in typographic points, but in a device-specific unit that depends on the DPI. Especially because even if you know the DPI, you can’t really assume what the user want. Maybe they are using a huge IMAX screen but viewing it from very far, and so only big text should be used. Or conversely, maybe they’re using a VR headset where the screen is small and has very high DPI, but then there are lenses that make it cover your whole field of vision, and tiny text that would not be readable at all if looking at the screen can and should be used.

These are extreme examples, of course, but even when switching between a 12" laptop and a 24" display of the same resolution (say 1080p), I would probably use different font size settings, and not in a linear way. I don’t want the text to use the same number of pixels on these two displays as it would be too small on the 12" one. But I also probably don’t want the text to be the same size in inches/centimeters, because I will probably make the text a bit smaller on the 12" display so I can still fit two windows on screen side by side.

That’s how I’m thinking about it, in terms of using the OS and writing apps for it:

  • Ease of use for users, with a single place to adjust the size. Not a combination of font size and DPI (or even more settings?) with ability to adjust both.
  • Ease of use for developers, with a single place to get the information about “how large should I draw things”?

If that makes people happy, you can add any level of indirections and conversions between these two. But since I want just one value on the input side, and just one value on the input side, I don’t see the point in not doing the simple thing and using the same value.

To me it doesn’t matter which unit is used. A float value ranging from 0.6 to 3? It’s fine. A percentage fron 60% to 300%? I can handle that. A “font size” in some arbitrary unit ranging from 8 to 48? Why not, that does the job too.

And since the last of these 3 options was already available in the Be API, it was the simplest to just go with that. There was less code to rework.

If we were to change our mind about this, it would look like this:

  • Hardcode the UI font to always be size 12 no matter what
  • Add a “scaling” slider to appearance prefernces instead
  • Change BFont to multiply that 12 by the scaling factor before handing it to the font rendering code
  • Make sure all the code never calls font->Size and instead uses font->GetHeight (which is more complicated to use as it doesn’t return a single value) or StringWidth or directly some new GetDisplayScaling API when it needs to compute some size that should scale with the user interface

After doing this we still have not solved the issue of things that really care about the DPI (as mentionned earlier, desktop publishing and the like), since what these want is ignore the user set scaling, and make sure they can render thing at the actual size (“as would be printed”), and so they have to get the info from the source (that would be using BScreen to get the DPI, and then work in pixels with multiplying everything, or we can be fancy and add an API that allows to work directly in millimeters/inches/points/… like CSS does)

1 Like
63

No. User can edit setting if automatically calculated value is wrong of if user want custom scaling.

I see a reason. For example it may have sense to set smaller font but keeping scale ratio for BeOS compatibility. Or adjust font size for specific fonts that have diffrent actual letter size for the same font size. User may just want to set larger or smaller font keeping UI size unchanged.

This is logical DPI setting, not physical. User can set it to convenient value. Imagine printed document with point metrics looked from different distances.

This is scaling coefficient / logical DPI only. Font size is completely unrelated to UI elements size. Font size is just one of font properties like typeface and style, program should just take default font or allow to configure font.

No, it requires more code to rework. Font size based scaling will require minimum size special handling while scaling ratio don’t require that. No special handlings at all. Only pixel alignment may be needed for pixel-perfect rendering, but it still needed for font size based approach.

All HiDPI handling can be done just by setting BView scale or affine transform.

I have never seen such feature on desktop publishing software. It is probably not needed.

CSS works exactly the same as proposed here. It have pixels, it is logical pixels and subject to DPI scale. It is multiplied by devicePixelRatio.

2 Likes
64

Erm… the discussion seems to be very advanced but why wouldn’t something like this be fine?

  • Have a setting to set the physical DPI of the screen (with the default value coming from EDID) - this should not be presented as a “scaling option” since it really isn’t, it’s just a property of the screen, perhaps there should even be a warning text emphasizing that.
  • Font calculations are done the way they sould be done - taking in account the DPI of the screen since “points” are a physical measure
  • Apps keep calculating their UI elements’ sizes according to font size - directly calculating pixels from points is conceptually wrong, so any code using font->Size like that is wrong, but it should be simple to have a font->PixelSize which would perform the correct scaling.
  • Applications that need the actual DPI of the screen would simply use that value to render content while still using “font->PixelSize” to calculate UI elements

This way if the user:

  1. Has a high DPI screen - the OS will get it from EDID (with the user being able to override it since it is not reliable) and a 12pt font will actually have a 12pt size, and the user interface will be scaled accordingly. That means that for the same font value interface elements would have approximately the same size as long as the DPI setting is correct.
  2. Has multiple screens with different DPI settings - All screens will look coherent, as a 10pt text will look more or less the same in all screens.
  3. Wants to make things bigger - he sets the font size, since that is what he actually wants - a (physically) bigger font and UI elements.

This also doesn’t go the hack-ish way Windows went of repurposing its DPI settings for scaling (which IMHO is wrong) and is coherent - the user sets the native resolution and correct DPI and uses only font size to scale the interface. Of course nothing prevents the user from using wrong DPI values, just like nothing prevents the user from using lower resolutions for the same purpose (and yes, I have seen computer-illiterate people do that), but again, nothing prevents a user from misconfiguring his PC.

6 Likes
65

“100%” scale and “Actual size” scale are very common DTP features, and in fact indispensable for me. Of course, being able to calibrate this is important; it is a very unfortunate frustration to be using word processors, PDF viewers, or other DTP software that is set to “100%” and yet the PPI (pixels-per-inch) of the display does not match the programs’ assumptions.

2 Likes
66

(>.>)… I admit I thoroughly abuse DPI settings on Android, because the default UI of almost every app is huge and doesn’t allow showing much information. I always lie to the OS by changing the value so that everything is drawn smaller and more information-dense.

On Windows, the inability to set less than 100% scale is frustrating, because more and more applications adopt huge, bloated, “touch friendly” interfaces that waste most of my screenspace.

In web browsers, I almost always set the default font sizes two points smaller, too (partly to combat websites which increasingly waste screenspace). If changing the font size also caused raster graphics to scale (becoming fuzzy), that would be sometimes be frustrating. I notice in Chrome and Firefox, if the whole page is scaled up or down, then video rendering becomes slower and poor hardware some frames may be lost.

…so, yes, I agree with everything you say.

67

Why would I want the entire UI to scale with the font size?

I probably have the exact same screen as @PulkoMandy, and therefor have the same need as him to scale down the UI to make room for more real estate. But I also have bad eyesight, so I need to use larger fonts. I need to scale the UI down and the fonts up.

When I connect an external HiDPI monitor, the UI becomes to small for my eyes, so I need to scale it up. I still need larger fonts, so I also need to scale those up, but a little bit more than the UI.

Font size should not affect the scaling of the UI elements. They must be independent.

3 Likes
68

I don’t think those can be 100% independent. If you scale your fonts up, any UI elements that have text inside need to be big enough to accommodate or else the text will be cropped, which means UI elements would have to be scaled differently among themselves, which might be complicated and error-prone.

In the scenario I proposed if you connect an external HiDPI monitor there should be no need to change scales between the two monitors, since text should look at about the same size (physically) with both, and the GUI would be proportional. The only thing missing would be a “compact” theme that has smaller elements, but still proportional to font size, and I think your use case would be covered.

3 Likes
69

Yes, that is how it should work since they have a natural relationship.

Let me also clear up what I mean by scaling up a font. That just means changing to a larger font. Fonts should always render at the correct scale.

70

What are examples of UI elements the sizing of which are not in some way directly tied to the font size?

2 Likes
71

I’m assuming he meant things like margins and padding spaces in controls, like buttons with bigger text but not growing in size. Are those measures controlled by the theme?

72

My experience from a non Haiku project. I professionally write 3D renderers, and a decade ago went with a scaling factor design. This works well in a lot of scenarios, especially with fonts, however images need to interpolate (blury), and when you need pixel perfect layout, the scaling factor adds additional complexity to developers code since now you have to fight it. I also used a percentage based layout design.

For the most recent Vulkan based renderer I’ve created, I completely ditched the global scaling factor design, and force developers to account for it manually. Percentage based layout is also gone. As a developer, I find that easier to deal with since I no longer fight the engine when I need pixel perfect layout. In essence, I’ve gone back to the old way of doing things since it was more predictable.

As a developer of Haiku apps, I try to respect the be_plain_font->GetFontHeight() setting, and adjust my View layout accordingly.

Just thought I’d share.

4 Likes
73

This sounds like a nice idea.

Can anybody with more knowledge pitch in? How hard would it be to implement? I’m guessing the technical aspect (getting/changing DPI) wouldn’t be hard, but modifying all the apps to use the new information would be an exercise in frustration?

74

Scroll bars is an example of an UI element that Haiku do resize with font size, but has no relationship with fonts.

A button with a text label is an example of an UI element that has a relationship with fonts and should resize when the font size is changed. A button without a text label has no relationship with font size and should not be resized when the font size is changed.

75

Okay, but why? It is definitely easier to just scale everything metrically based on one value, and that includes fonts. Is there a reason that we should permit misproportioned UI elements, like scroll bars?

Also, it’s not quite true that scrollbars are unrelated to the font size. In some control looks, the arrows in the scrollbars are drawn with characters from the font, or at least can be. Furthermore, Tracker and other applications put a status indicator with text in the same row as the scroll bar, so they must be the same height, and thus the vertical bar needs to be the same height so they line up. Ergo, scrollbars do actually have a relationship with font size.

3 Likes
76

I agree, except for the fonts. Me and others are saying that the scaling metric should not be driven by font size, it should be driven by a scaling factor. That factor can in turn be driven by any metric that creates the best outcome, like display dpi.

The current implementation is font size → scaling factor → ui scale
It should be scaling factor → ui scale

Fonts should never be scaled as long as the unit is points, since points are a fixed unit of measurement. If the user wants larger fonts, they should just select a larger point size.

I have not argued for that. Reread my comments. My argument is that Haiku should be able to scale the UI 1:1 independent on font size.

5 Likes
77

Say, on a small netbook screen. You’ll want to make the text as large as possible and the other UI elements as small as possible, even if that causes the UI to deviate from how it was originally designed. That’s what splitting the font size and scaling factor would allow.

1 Like
78

OK, but thus far your one example of something supposedly independent of font size (scroll bars) turns out to be quite related… do you have any other examples? (I cannot think of any in particular.)

This, to me, sounds like it should probably be in its own ControlLook, or rather it should be a setting to reduce the UI spacing, rather than having the “scale” and the “font size” be truly separated. That is, if all people really want is a way to have a larger font size with smaller spacing, then that can be made possible without decoupling the scale itself from the font size.

I do think that the idea of having a per-screen/DPI “scale factor” which just adjusts the font size, instead of having to edit the font sizes in Appearance settings, make sense, though. But that would still be one factor in the end, just another way it might be adjusted in a way transparent to the application.

2 Likes
79

I just wanted to add something I learned today about macOS and display scaling. I kind of knew this already but did not look into the fine details until someone at work mentioned performance issues. But the short version is macOS is pretty much totally broken for most 4K displays.

The long version is the way macOS scales is either normal (1x) or Retina (2x). Anything in between, like 1.5x (which is a great scaling for most 4K monitors, equivalent to 1440p or 2560×1440) actually results in things being rendered at twice that (5K or 5120x2880) and then scaled down using bitmap scaling. The result is performance issues, blurriness, slow response, and pixel-imperfect rendering.

So I absolutely do not think macOS should be considered any kind of standard in how to do this, unless you happen to have a screen which has the right DPI to be rendered at 2x, like most MacBook or iMac screens. This is also why almost all the Retina iMacs uses 5K screen, which can be rendered easily at 2x and scaled down to 1440p.

For better or worse I think Windows and ChromeOS are better models to look at for how to do this scaling in a better way. Though if you read some Microsoft documentation about High DPI they have had a lot of issues too. It is a hard problem, especially once you add multiple monitors with wildly different DPIs. Of course multiple monitors are not currently supported on Haiku but when support is properly added scaling needs to be considered. For example a single application could have windows on different screens so the scaling factors need to be at the window level, not the application level. I mention this because it was apparently a problem in Windows, haha. I suspect it will be less of an issue for Haiku.

Overall though I think we can talk about this all day, but really we need someone from the “scale factor” group to maybe try to prototype this and see how it looks. Right now the font size scaling is what is implemented so for better or worse the burden of proof lies with the other side to try things out and see how it looks, both in the sense of visually and code complexity. Maybe X512 can give it a try once the RISC-V port is in good shape. I’m interested in learning more about this too but I don’t think I will be able to try anything for a while.

7 Likes
80

As Pulkomandy mentioned, using the font size for scaling display elements was chosen primarily, as it was the way of the least resistance.

Of course, we also need to support scaling based on DPI. Just consider multi monitor, or just frequently changing monitors - it’s the only way to make this user friendly. However, I would argue that we actually need both.

1 Like