What are the performance implications of using an immediate-mode GUI compared to a retained-mode GUI?
Asked Answered
U

1

194

I am currently working on a standard Windows desktop application (standard meaning no fancy stuff: just buttons, text, sliders, etc.), and have decided to write a GUI framework on my own, after looking into some GUI frameworks and being repelled by all of them. Since it’s a hobby project, I am also willing to experiment, and decided to make the GUI immediate-mode, not retained-mode, as I really like the way it simplifies the code. Here is the question, though:

What are the performance implications of using an immediate-mode GUI compared to a retained-mode GUI, when using it for a typical desktop application?

I always hear that an IMGUI performs worse, since it has to redraw every frame (or, if it somehow caches, it still has to do the logic every frame). But of how much more are we talking here? Am I burning twice the CPU time? More? If I hypothetically ran 20 IMGUI programs, would it max out the CPU (presuming I already optimized it)? I just want to know the ballpark and whether the tradeoffs are still viable in a non-game environment, where there is no need to redraw every frame.

There is also one more implication concerning latency that I don’t understand. In the chapter discussing IMGUI in a work-in-progress book by Johannes Norneby, it is explained as follows:

Frame shearing

One aspect of IMGUI to be aware of in the context of real-time applications (constantly rendering new frames many times per second) is that user interactions will always be in response to something that was drawn on a previous frame. This is because the user interface must be drawn at least once for the user to be aware that there are widgets there to be interacted with. Most of the time this doesn’t cause any problems if the frame rate is high enough, but it is something to be aware of.

How is this any different in a retained-mode GUI? Does it mean that I have one more frame of input lag over a retained-mode GUI?

Unfold answered 22/11, 2017 at 21:33 Comment(13)
I strongly recommend you don't implement your own GUI library, even as a hobby project. It's extremely hard to get right, and there are a lot of details, and it's easy to get wrong. Even if no GUI library exists that you like, it would be far better to implement your hobby GUI library as thin wrappers around an existing library (basically, use one of the existing libraries with wrappers to make the API more like what you want). GUI libraries are too big to be a fun hobby projectJaponica
Think about how detailed or deep you want your GUI library to be. For example are you writing wrappers around an OS api, or are you bypassing the OS and writing directly to the hardware? You should check out WxWidgets and Qt to see how big the project is going to be.Seldon
Well I will certainly stop when I am not having fun anymore, right now I do, so I will continue. We'll see for how long. That is not really relevant to the question though, my point is also important to me if I decide to use an existing gui framework: is imGUI viable in a non gaming context, and if so, what are the performance implications?Unfold
I see now that I maybe stressed the part of it beeing my own framework a bit too much. I am interested in the performance implications wether or not I will end up with somebody else's framework.Unfold
Immediate mode rendering has been around for 30+ years. It is intuitive, uses limited hardware resources well, leverages the idea of having widgets that carve out their own niche in the main window. Retained mode rendering is a very different ball-game, you "retain" rendering by programming the GPU. Big difference, the GPU can do a much better job of spitting pixels onto the screen than you could ever do by writing code that runs on the CPU. But you can't ignore the need to program the GPU, and there isn't just one of them. It is the job of a framework that hides the differences.Margravine
@HansPassant You seem to imply that an immediate mode GUI could only use the CPU for rendering, not the GPU. I don't see why that would be the case, the "immedate" in immediate mode GUI really only applies to the control flow and how you think about graphical elements, how you render them is a different game. If I am not mistaken rendering would be possible in both manners and not limited to the CPU.Unfold
Did you consider uising Qt or FLTK for your GUI Interface?Ailina
@BasileStarynkevitch I considered Qt, but it is the culmination of everything I dislike about current GUIs: It mandates that it owns the program, not you, it dictates control flow, it is GIAGANTIC (download was ~40GB at the time, when it should just be a .lib and a header), It has its own build system, it wants to be everything (this thing is closer to an operating system than a GUI) and the list goes on. If I "just want a gui" this thing is a nightmare.I remember stumbling about FLTK and avoiding it because the website looks like its from the 90s, but I haven't really looked into it.Unfold
You could also try the FOX toolkit or make your software some Web server using e.g. Wt or libonionAilina
@BasileStarynkevitch I downloaded two of the example applications done with FOX toolkit, and they look quite dated and out of place, so thats not an option for me. Wt would in many ways be worse than Qt. I don't see how "add a webserver" would possibly get me closer to "I just want a simple GUI". It does the opposite of what I want, it makes everything more annoying. libonion doesn't even seem to have anything to do with UI, thats just a webserver library. I want to point out that this is not a question about which GUI framework to use, so suggesting them is beside the point anyway.Unfold
@pulp_user: You don't define what a "simple GUI" means, and GUIs are operating system specific. They are different on Linux, on FreeBSD, on Windows, on MacOSX. Also, GUIs are complex. See osdev.org to get an intuition why.Ailina
@BasileStarynkevitch You have gone completely off topic. This is a question about performance implications of a certain api-style for a GUI. I didn't ask for advice on what gui framework to use, or why existing ones don't have the properties I want from a GUI. So please stop trying to find the right GUI framework for me. Thanks.Unfold
It's 2 am and I just gad an epiphany. OP you wrote on the hacker news thread that you thought it was stupid to have to write an application "in" <insert framework> rather than have it as just a dependency of the application. That's something that has stuck in my craw for years without me realising it, and thanks to that simple sentence I see things much more clearly!Jonathanjonathon
U
279

Since there seems to be some interest in this question still (judging by the views), I thought I might as well post an update.

I ended up implementing an immediate-mode GUI as my master’s thesis, and have some numbers in on the performance. The gist is:

It's fine - quality of implementation dominates, not systematic characteristics.

Compared to many other existing retained mode GUIs, my immediate mode implementation generally performs a lot faster. The theoretical performance differences between the paradigms are eclipsed by the fact that most GUIs are horribly unoptimized. Overall, imgui is a completely viable approach to creating a GUI that responds fast and does not drain the battery.

I created a Spotify clone with about 50% of the UI elements, and rendering a single frame was in the microsecond range. In fact, the application consistently used less than 400 μs for a single frame. With V-Sync enabled on a 60 Hz monitor, this equates to roughly 3% CPU load on a single core (400 μs per 16 ms) for a naive implementation. Furthermore, a decent chunk of these 400 μs were caused by constant factors that wouldn’t increase the load with more UI elements (e.g., receiving input or setting up GPU state that doesn’t scale with UI complexity).

The perfectionist in me still disliked the fact that a GUI that did nothing was consuming cycles, but the upsides were tremendous: When the GUI was being heavily interacted with, or when the window was being resized, it was still hitting 400 μs! This blows many existing retained-mode GUIs out of the water. Try resizing Spotify, Windows Explorer, Visual Studio, or basically any other desktop application, and see how it reacts, to understand what I mean. My guess would be that Spotify goes down to about 2 fps on my PC when resizing.

And changing the UI is basically free. If you display a hundred buttons in one frame, and then replace them all with textboxes in the next, there is still no difference to the performance. Retained-mode GUIs tend to struggle in such scenarios.

Three more thoughts:

  • Most of the time is spent on text rendering, the rest is close to irrelevant. If you want to heavily optimize, this is going to be the thing to focus on. But even with little optimization, it can be made decent.

  • I doubt that the vast difference in performance can be explained only—or even at all—by the difference between the retained and immediate modes. For example, Spotify uses a Web stack for UI; that’s bound to be slow.

    I guess, WPF, Win32, and the likes are just slow because they can get away with it, or because they were optimized for completely different hardware than the one we use nowadays. They also do more, of course, but not remotely enough to justify the difference.

  • I’m sure you could make a retained-mode GUI that is a lot faster than my immediate-mode GUI. There is just little incentive for it in general, which is a little sad, to be honest; I like efficient things.

Update

Since a couple of people asked, I’ve decided to release my thesis.

Please be aware that you will be looking at something that was not meant for public release and doesn’t pass my personal lowest quality expectations for a publicly released piece of software (which is why I hadn’t released it in the first place). So please only use it for educational purposes and not for building actual software with it. I won’t be supporting it, either.

The download includes the thesis itself (in German!), some pre-built executables for Windows, and the source code (in C++):

https://1drv.ms/u/s!AsdZfH5hzKtp9zy1YZtHgeSMApOp?e=FbxLUs

Have fun!

Unfold answered 20/1, 2020 at 17:41 Comment(25)
An answer three years in the making. Great work! First question: how did you go about comparing the two paradigms? Second question: do you have a link to implementation details of your project? Thank you!Siqueiros
I came across the paradigm by Casey Muratori's Handmade Hero, and the DearImgui Library. I decided to explore this topic in my Masters Thesis, after I "just wanted a GUI for my programm" and learned how ridiculously bad the landscape of GUI-Frameworks for end user programs is. It's all horrible. So I took the "it can't be that hard" approach. I haven't uploaded my implementation anywhere, mainly because it is of truly academic quality (pretty crappy) and I don't want to curse others with it. If you want to take a look at an ImGUI implementation, check out DearImgui or Nuklear on Github.Unfold
I just realized I didn't really answer your first question: First, I compared them from a theoretical standpoint: In ImGUI you describe the state you want the UI to have, in retained mode, you specify the transitions between the states. I wont go deep into it, but it boils down to: retained mode can have n² complexity, while immediate mode pretty much stays linear. I then built an application to compare performance to other retained-mode applications. This was not very scientific, so there is no real knowledge to be had, besides "it works and mostly better so" (for my usecase).Unfold
I'm glad you didn't take the advice of the comment that said you shouldn't write your own :) Well done.Refrangible
Hey @Unfold I love your reply and the fact you made a modern application with ImGUI, I am also working on a custom ImGui project in the handmade spirit. Which aims to show how that many of the problems people have with ImGui can easily be addressed or does not exist, and then explore what it would be like to make the UI for Discord or Spotify as a demo. Would love to learn more about your project, ask a ton of questions, see how you solved various problems, and maybe get some feedback on my approach if you have time and are willing :) perhaps over Discord?Erskine
@Erskine I'd be totally up for that, but I have no idea how to exchange contact info over stackoverflow that doesn't involve posting comments for everyone to see. Any suggestions?Unfold
@Unfold That's great, thanks! Couldn't find a way to message you directly either, apparently that is not a thing on SO hehe. I've added you, so you can delete the comment with your username if you like:)Erskine
@Unfold well done! Have you tried not rendering every frame but only when there is some user input or internal state change with visible effect? Perhaps in some cases if you could filter those reliably you could save even the 3% CPU most of the time.Nephrectomy
please share your thesis when it's publicly released :)Unnecessary
Hoping that Justin will weigh in.Ketty
Congrats on making it to Hacker News: news.ycombinator.com/item?id=25624044 Please absolutely do consider sharing your code, even if it's "academic" quality. Only good can come of it. Besides, this is exactly what the CRAPL licence is for :pSanderson
Fantastic work. I guess part of this is because you just can't get away with things being slow in an immediate mode GUI. Retained mode may hide all sorts of slow paths. It makes me wonder if every retained mode GUI should have an immediate mode to make sure it remains performant.Vulvitis
OMG This is the greatest answer I have read on SO in a long time!!! NICE work :)Antecedent
@TomasAndrle No, because it wouldn't have helped much in my usecase. Since you only know that the internal state hasn't changed AFTER the application rebuilt the entire UI, you have already spent 80% of the total CPU time at that point. It didn't feel useful, given time constraints, to immensly complicate the logic (and make some design decisions) to shave of 20% or so. TBH, this optimization maybe would have actually been a net loss, since the caching-logic might have taken more time than the total rendering. But thats hard to tell. I would have gotten more out of improving text rendering.Unfold
@TasosPapastylianou so thats how all of you ended up here :D I will consider releasing it now, but the CRAPL license is way too wordy. Maybe I will just invent my own license: "This is shit code, please don't use it." should do.Unfold
Release it as beerware then. Short and sweet :)Sanderson
@TasosPapastylianou I don't like beer, so thats also not the license for me :D I will find something!Unfold
@Unnecessary unfortunately, my thesis is in german, so it won't be useful to the vast majority of people. I'm thinking about translating the chapter with examples though, that way I could bundle it with the source code, and people can at least figure things out by example. I don't trust a machine translated pdf to be very readable.Unfold
@Unnecessary I added a download link to my thesis. I opted to not translate the section, but clean up the code-examples so they will hopefully be understandable for other people (and won't crash immediately). A machine translated version I looked at looked ... ok-ish, so I hope the thesis itself can also be of some use for you.Unfold
The thesis looks great! Thank you for releasing it. (Modern machine translation is indeed good enough for it to be quite readable auto-translated into English.)Jurisdiction
Did you use opengl to draw your widgets? From the bold text I can't really deduce which one drains more battery? Retained Mode or Immediate Mode? Also which one is more customizable and flexible according to you?Metre
@Metre I used DirectX11. If you optimize enough, I think retained mode still comes out ahead, it's just that many current frameworks have horrible performance and this higher battery usage when interacted with. I can't really say "this one is better" in this regard, just that both are viable approaches. And customizable and flexible is sort of up to the implementation details. You can make very customizable retained mode and uncostomizable immediate mode guide, and vice versa. This is really more a question about how you design the api.Unfold
Now why is retain mode GUI slow?Consumerism
@Consumerism As I said in the answer, it's not an inherent thing about retained mode GUIs. The quality of implementation dominates performance. Most GUIs are slow because devs don't care, and since most GUIs are retained mode, that means most retained mode GUIs are slow.Unfold
Love the article. About the lack of performance in retained mode GUIs, I suspect that the thinking is that Humans are so slow. If a GUI reacts in 10 ms instead of 1 ms, we just won't notice. So it isn't worth the effort of optimisation...until the user drag resizes a very busy window.Sandblind

© 2022 - 2024 — McMap. All rights reserved.