Abstract

The state of the art graphical user interface (GUI) works on a basis of widget lifetimes that are bound to objects or handles. This common aproach which is called "Retained Mode GUI" (or RMGUI) has a few drawbacks. Firstly, the application data has to manually be kept in sync with the data that is shown by the widget otherwise the UI would show old data. The code locality suffers. This is because widgets have to be called in diffent regions of the code to (1) create them (2) update them (3) query / callbacks (4) destroying them. This leads to code for one widget being spread across many lines or files. Because the widget lifetimes have to be managed by the user, for each UI transition, specific code has to be written to decide which of the widgets should be destroyed or created or which ones should be kept. Finally, the use of events / callbacks complicates the control flow, the user does not know in which order (or in which thread) the callbacks happen and can't make assuptions over which events were already handled. An alternative approach is called Immediate Mode GUI (or IMGUI) which differes from RMGUI in that no UI widget has a lifetime. On every UI update, the user informs the IMGUI system over what should be visible on the screen. As a result the data is never out of date. Since there are no object lifetimes, there is also no need to manage them and a higher code locality is reached. IMGUIs are typically implemented without the use of callbacks, so the main application does not give up the control flow to a GUI library. IMGUIs are already widely used in games and game tooling. In this work we want to explore the possibilities of a IMGUI system in the context of Mixed Reality. The user interface will no longer be two dimensional; it can be attached to objects in 3 dimensional space or even to parts of the user, like a "hand menu". We believe IMGUI helps mitigating the additional complexities of GUIS in the MR setting and offers the needed flexibility.

Results/Implementation/Project Description

Conclusion

[ PDF (optional) ] 

[ Slides Kickoff/Final (optional)]