How to scale the UI properly with High DPI screens?

Development
41

What do people (@waddlesplash, @nephele) feel is wrong with the scaling factor approach? The way ChromeOS and Windows does it seems okay. What is so bad about replacing the current calculation using font size 12 with just a scaling factor? Maybe if there is more information there it would help the discussion be more productive.

I do think we should not do it how macOS does it, which seems hacky. From what I have heard there it is always scaled up 2x and then if the user wants less than 2x it is scaled down using a bitmap scaling which makes everything look terrible.

Obviously you can’t do that with the current system, which I guess is your point, though as nephele says you could be less combative in your wording.

But I also wonder: how would it work with a scale factor? Do you want a scale factor AND global font size settings? Isn’t that even more complicated? How do those interact? Does the scale factor only affect general sizing and then there is still a font size? Or does the scaling affect the font size unless you also change that? This sounds very confusing to me. I think there is either a scale factor or the current font size settings, and then each app could still have individual font size settings if you need those bigger. In that sense there really isn’t a huge difference between the approaches with the exception that a scale factor might be less hacky in the code than some calculations with a hard-coded font size 12.

2 Likes
42

Scale factor is applied on everything including fonts. Some application may sometimes ignore scale factor, for example image viewer to show pixel-perfect raster image.

1 Like
43

Correct.

I haven’t said anything “combative”. You must have mixed up what somebody else said.

Scaling scales everything. It acts like zoom. 2.0 scaling doubles the size, 0.5 halves it. See my comment about retina display below for more information.

If you want to scale, you scale. If you want to change the font, you change the font. If you want to do both, you do both.

It’s been answered above.

My belief is that the designers of the current implementation do not know the history behind typographical units, and how they relate to modern technology. If they did, they would understand that the current implementation will never work.

Fonts are usually sized in points, which has a set relation to other measurement units. Points has a fixed size when printed. 10 pt is approximately 3.5 mm.

Monitors have a pixel density, stated in dpi. By knowing the monitor size and its dpi, fonts (and other graphical elements) can be displayed at its correct size, meaning a 10 pt font will be approximately 3.5 mm on screen. This is very valuable for graphical designers and typographers since they can correlate the scale of what they see on their screen to how it will turn out in print.

Apples implementation and their use of “retina” displays have been used as an example of how UI scaling should work, which I agree on. On two displays with the physically same size, but different resolutions, the rendering should be identical. The only difference is that the display with higher resolution should be perceived as more “crisp” since it can render the graphics with higher pixel density.

If I would like to change the fonts it should be rendered at the correct size without affecting the UI. If I want to change the size of the UI, it should be possible to do without affecting the correct size rendering of the fonts.

5 Likes
44

Haiku should have this ability built in to the system. The system can be made aware of the displays pixel density, which can then be used to display everything at the correct scale.

The Be-way would be to have optional scaling per workspace.

1 Like
45

The main point is that they completely ignore what a pixel is. When your UI rendering goes through various levels of scaling, it just becomes impossible to know what you’re doing. And there are some cases (admitedly quite specific cases) where you really want pixel perfect rendering.

Anyway, some people here seem to think that a scale factor would just magically work without any work from developers, and that would be easier. I want to say two things about this:

  1. You can already have that if you want to write your code that way. All you need to do is use the SetScale function in your view constructor. From then on, all your drawing will be “magically” scaled to the selected scale.
  2. This is not what we actually do anywhere in Haiku. The main reason being that we decided to not scale most items below a certain size. For example at any font size smaller than 12, scrollbars will keep the same thickness so it’s still possible to reach them with your mouse. There are various other places like this, where margins are either hardcoded (intentionally) to a fixed number of pixels, or scale non-linearly (in steps, or with a min or max value, or various other cases). There are also places where our vector drawings are designed with a specific grid in mind, for example icons in the HVIF format usually have most of their control points snapped to a 64x64px grid which means they will look better at multiples or fractional multiples of that size: 16x16, 24x24, 32,32, 48x48, etc. And so we don’t blindly scale them to some random arbitrary factor, but pick the closest size from this set.

So, whatever you do, doing scaling “right”, or if you don’t like this word, doing it the way we decided to do it, takes some design work, it is not just about multiplying all coordinates by some value.

Then, there is the entirely separate debate of how you let the user configure the thing. We decided to adjust things automatically according to the system font size. I think this is simple to understand for users: there is a single setting that affects the whole user interface. We could rename it to “UI size” if that make it easier to understand. We could have an API that divides it by 12 is developers don’t want to make that division in their own code. Or we can move the multiplication somewhere else (but that is less easy to fit with the existing BeAPI conventions and usage). That is implementation details.

Another route is having separate settings for font size and UI scale. I think this is confusing for users, and don’t see the use for two separate settings here. Are there cases where you want all the UI to be huge, but use a tiny font and have gigantic borders around it? Are there cases where you want a huge font, but a tiny UI scale so that the font will not fit buttons and nothing will render properly? Why?

It is not clear which of these things all of you are arguing for or against.

My personal opinion is:

  • Having a single place where the user can decide how the OS looks is better. I don’t care if it’s called “font size” or “scaling” or whatever fancy name, and I don’t care what the unit is (points, pixels, % of an unspecified thing…) but it should be a single setting
  • We should set that setting depending on the display DPI when we know that. Unfortunately, the DDC/EDID info on it seems to be far from reliable on some displays, so it should still be configurable (also because for the same screen, people could have different preferences)
  • The chosen solution to use the size as a base and do the adjustments according to that in the code works well in most cases. There are still some issues to solve in the layout kit (but most things have been done) and in apps that don’t use the layout kit
  • Having a global scale that blindly multiplies everything is easier for developers, but it turns out it does not look that great, and it’s better to finetune things a bit (especially for the smaller sizes where some things must have a minimal size). Yes, of course, it is more code to write. But the end result will look nicer, at all sizes.
5 Likes
46

How does font-based scaling with a hard-coded value of 12 cause pixel perfect rendering?

47

Yeah, and this is a problem on HiDPI screens. The changes I have been making to remove such hardcoded margins and replace them with font-size-scaled values leads to lots of be_plain_font->Size()/12 calculations, as mentioned earlier, which I do agree is a bad idea, and that computation should be performed in a central location.

Thinking about this more and discussing it with nephele and PulkoMandy, I think what we should do is move more of this to, or use more from, be_control_look. Already there are places in there that compute various metrics, e.g. margins, based on the font size. We should just use those more broadly, or add more if needed, instead of doing be_plain_font->Size()/12. This will also have the added benefit that other ControlLooks can then adjust metrics as they want.

3 Likes
48

If you use the font size as a magic value like a scale factor, it does not. If you use the font size as a guideline to pick the right metrics, it makes it at least possible.

1 Like
49

Why not use a metric that do not require a “guideline”? Like pixel density and scaling? That will result in pixel perfect and scalable accurate rendering.

1 Like
50

The point is that you need the scaling to be a guideline. As I said, you can’t simply multiply everything by some value and be done with it. I gave some examples already: icons, things that have a minimal size so they can be easily usable with a mouse even at small font sizes, things that don’t need to scale up infinitely with larger font sizes (for example spacing between widgets doesn’t really need to). So a guideline is indeed what we want, and not something that attempts to do the scaling automatically at a low level.

Then, most of the scaling is actually already handled by BControlLook and there are only very few places where we actually do the “font size / 12” thing: Search

Here is a rundown of these:

  • deskbar: scales the leaf in the menu. You can see the math is more complicated than just font size / 12, with an high and a low bound. Also scales the “application menu” bitmap when you have the deskbar in compact mode in a corner, here again some special casing is done to ensure that the icon size is rounded to match with the pixel grid
  • StyledEdit status view: this code is being rewritten to compute the font size independently of the system font and depending only on the scrollbar height
  • Notification preferences: to compute the minimum and maximum allowed window size. I’m not sure why we even have a setting to configure the size of notification popups.
  • BAlert: to scale the icon. Clamping is used so the icon is never smaller than 32x32 even for smaller font sizes.
  • FirstBootPrompt: to compute the initial window size. This one specifically handle the case of 12px fonts because it is the size at which it will be displayed (it is shown only once, and before you had any chance to adjust settings). This code needs to be revisited when we implement auto-configuration of the size
  • HaikuDepot: to set some column width. This is done in a strange way, probably StringWidth() should be used to make sure the columns fit some given string instead. Still, it is natural to base the size of columns that will show text, on the font size.
  • Tracker: to determine the size of icons in list view mode. Clamping is used, the icon is never smaller than 16x16px. Also, rounding to align to the pixel grid is used.

That’s it. That’s the complete list of places where things are coded in terms of “font size / 12”. A grand total of 8 places in the whole of Haiku, some of which are legit, some that are a legacy of quickly adjusting existing code to scale properly without rewriting it all. And most of them with special cases (min or max values, rounding, …) so it’s definitely just blindly scaling everything.

1 Like
51

I think the other concern enumerated in this thread was that there are currently 4 font size settings and you need to change them all; this may become more tedious in the future when font size settings are per-screen. Clearly, we need to come up with a better interface for changing these settings; but I think the basics stand.

4 Likes
52

No. For example Oberon framework Blackbox use universal logical metric units for all drawing. If you want, you can get pixel size and shift in units and do pixel-perfect rendering or align graphics to pixels.

I have experience to use Blackbox on multi-monitor system with different scaling settings per monitor and it works great when moving a window between monitors dynamically changing window DPI. And this framework was designed in 199x.

I think that this is a hack. With separate font and changing settings you have no problems with setting 100% scaling and 9 font size.

This all can be handled by formula on scaled units so drawing will be perfectly pixel-aligned. And it will be easier than current approach.

You also ignored my argument that font size have no clear meaning and unreliable. Actual font size meaning depends on specific font design and may vary a lot causing bad layout.

53

Because a lot of things like window border still not have scaling and hard to use on hi-DPI monitors.

54

Could medium and high DPI themes be used? That way constant folding will get rid of most of the abstractions and the only real trick will be multiple monitors which may need different themes from each other.

55

Why don’t you use scaling variables that can be assigned values according to the desired outcome? If the variables are fed with the the same value, the outcome will be 1:1 scaling. This will be very useful on HiDPI displays. If partial scaling is desired, different values can be fed into the variables. This can be transparent to the user, and could be driven by any metric, including font size.

My use case is that I want to scale the entire UI 4x (HiDPI) and change the font to a slightly bigger size. That cannot be done using the current implementation.

56

But that is not what I want from a user perspective. I don’t want to be messing with two related but independant settings to try to get the UI to look right. I want to pick a font size (or ideally, I want a reasonable size to be picked by the OS automatically according to the display DPI and resolution) and I don’t want to mess with it.

It is still unclear to me if you are telling this scaling setting should be exposed to users at all, and if it should in addition to, or in replacement of the font size setting we have currently?

Because it makes no sense to me.

Let’s take the example of a BButton.

The way it is done is (simplifying a bit, I’m ignoring the case where the button has an icon):

  • Compute the height of the label according to the font metrics (ascent and descent in pixels, not the font size in points)
  • Compute the width of the label using StringWidth (again, this gives a result in pixels)
  • Add some insets (there are two, called frame inset and background inset, one is proportional to the font size, the other is a fixed number of pixels)
  • Add some margins around the text on the left and right (proportional to the font size, but clamped to a minimum number of pixels)
  • Add some margins on top and bottom (proportional to the font height)
  • Finally, force a minimum size of 5x5 pixels for buttons with no text, and 75x5 for buttons with text

So, the size of the button is primarily defined by the actual metrics of the label in it. I don’t see how this code can in any case result in “bad layout” (which I assume would be the string somehow not fitting in the button?)

Here is how it looks at various font sizes:

You can see that at the smaller font sizes, the minimum width prevents the button from getting too narrow. At larger font sizes, the button will be a bit higher and less wide, as the horizontal margins don’t scale as much. Personally I am more concerned with how it looks and works at smaller font sizes, which is what I use (because my old laptop has a quite low resolution display). And especially going in that direction, just being proportional wouldn’t work, you need some minimal sizes and extra margins so things don’t become too small and remain clickable, and often just one or two pixels make all the difference. Probably at larger font sizes this isn’t as much a constraint?

Can you explain how adding another scaling factor will help with this logic? The UI is dealing with text a lot, everywhere, all the time. So in the end, the computations have to take the font metrics into account, there is no way around that. I don’t see how a separate scaling factor will help here.

The code I looked at is:
https://git.haiku-os.org/haiku/tree/src/kits/interface/Button.cpp#n640
https://git.haiku-os.org/haiku/tree/src/kits/interface/HaikuControlLook.cpp

1 Like
57

User usually only change scaling factor, font name and size is a part of theme.

It should be exposed in screen preferences with default value based on DPI provided by monitor. It should be saved per-monitor by using monitor serial number (or some another information to distinguish monitors) as a key.

Font size should be left as is and it is unrelated to monitor. When displaying text font size is multiplied by scaling factor as all other sizes.

User usually never set scaling factor less than 100% because it is expected to be designed to work with 800x600 monitor. So no “too small controls” logic is needed. If user choose smaller font size it will not affect spacings between controls. Spacings are only affected by scaling factor.

3 Likes
58

If a custom theme is rated 4x and generates borders and sliders and icons with their coordinates shifted left 2 bits to make them chunky enough to be used on a UHD monitor but allows fonts to be modified from the new default setting of 48 pixels high, then how is that different than what we have now? (Points are 72 DPI standard increments from the printing industry and should never be confused with pixel increments.)

I think themes and scale factors could be safely merged if my suspicions are correct. If the scale factors are constant powers of 2, a 32 bit ARM can scale them at almost zero-cost using the shifter in its pipeline, or so I’ve read.

59

I use 1.5 scaling on tablet.

60

Still possible but would require a multiply by 3 and a shift right 1. It works but costs more.