A widget library for large scale displays and immersive environments

-

An example of widgets rendered to a cube texture
In CAVE2, most omegalib applications display a user interface using the 3D menu system. On menus, it is possible to create a variety of standard controls like buttons, sliders, sub-menus etc. Internally, the menu system makes use of the omegalib widget library, which is part of omegaToolkit. This library is designed to support rendering of 2D widgets on large scale, cluster-driven displays. Since we want to support stereo displays, the widget library supports 3D rendering: Widget containers can draw themselves to arbitrary surfaces in the scene. That's how the menu system 3D mode works, but you can do the same with your own widgets, using the Container3DSettings class. Container output planes can also be attached to scene nodes, using the Container3DSettings.node  property. For instance, you can attach a 3D mode container to the default camera, to implement a 3D heads-up display. Or you can attach it to a movable object in the scene, to create a floating 3D label that follows the object. The Text3D class can be used for simple text-only labels, but the widget library gives you a lot more flexibility with your label design, and the kinds of data you can put on it. If you need more control over the output surface, you can also grab the container output texture directly and use it as a 3D object texture. See https://github.com/uic-evl/omegalib/blob/master/examples/python/uitotexture.py for an example of this.

Of course the widget library can be used to draw classic 2D interfaces. The ohelloWidgets C++ demo shows how to use the widget toolkit for 2D drawing. It also shows how to create custom widgets using OpenGL.
Widgets, widgets everywhere!
In order to support very different devices like your laptop, OmegaDesk, touch walls and CAVE2, the widget library accepts a variety of input sources to control interaction. Like the rest of omegalib, input handling is based on omicron, making it possible to interact with widgets using mouse pointers, SAGE pointers, touch screens, gamepads, tracked wands etc. In particular, widgets support both pointer-based and gamepad-based interaction. An interface can be used with a mouse or navigated with a game controller d-pad. The interaction mode (and a bunch of other properties of the widget library) can be specified in the omegalib configuration file.

When using gamepad navigation, the widget library does most of the work for you. For instance, putting widgets in a vertical layout container, will automatically set up navigation so that pressing the up/down buttons on a gamepad will move widget selection up and down. The same happens for horizontal containers. For free-layouts, or for navigation between containers, you can explicitly `link` widgets together to tell omegalib how interface navigation should work. This is done using the Widgets setPrevHorizontalWidget, setPrevVerticalWidget, setNextHorizontalWidget  and setNextVerticalWidget methods. The menuColumns example shows how to use custom navigation to create a multi-column menu.

Comments

Post a Comment

Popular posts from this blog

Parallel Gaussian Elimination Using MPI

-
Image
In this project, we had to implement a parallel solver for linear equation systems, using a technique known as Gaussian elimination (GE). As with many other algorithms for solving linear equation systems, GE is performed on the matrix representation of the system, Ax = b, where A is the coefficient matrix and b is the vector of known values. GE works by applying a set of elementary row operations (swapping rows, multiplying a row by a non-zero number, adding a multiple of one row to another) in order to turn the coefficient matrix into upper triangular form. The algorithm has been implemented in C, using the Message Passing Interface (MPI) API. The parallel GE program measured execution times for its four main components: data distribution, Forward Elimination , Pivot row identification, Back substitution. Read the full report here
Read more

Parallel Beam Tracing and Visualization of 200 Million Sonar Points

-
Image
For the final project in the UIC Spring 2013 Parallel Processing Class, I worked with a classmate to optimize our current implementation of a sonar beam tracer used for  The NASA ENDURANCE Project . The beam tracer is used to process the raw data collected by the ENDURANCE AUV in Lake Bonney, Antarctica, and correct sound beam paths using information about the water chemistry. The data is clustered and noise filtered to generate a final 3D Point cloud that can be visualized in our CAVE2 system. The ENDURANCE Sonar data in CAVE2. Photo(c) Lance Long. A lot of corrections and adjustments need to be applied to this data before the final results are acceptable. Some of the corrections derive from the sound speed estimation through the water column, AUV navigation corrections, water level changes and noise filtering thresholds. All this parameters influence the final result, and ideally researchers can tweak them and see the result of their actions in real time. Given the size of
Read more

Flickering 2D transform elements in Chrome: a simple fix

-
This quick post is to point out a fix to an issue I had to deal with a few weeks ago and was caused by a Chrome update. Components with absolute positioning and a 2D CSS transform started flickering when placed inside a scrollable component. A brief search online revealed several people with similar problems, but the suggested fixes were all very different. I tested several of them (including obvious changes to the z order) before finding the one that actually did the trick. What eventually worked for me was adding an empty 3D transform to the same elements with 2D transforms: transform:translate3d(0,0,0); I guess the reason why this works is that it forces elements to be drawn via a different rendering / compositing path than standard 2D elements.
Read more