Device Proxies and Device Interfaces

When writing a VR Juggler application object, direct access to hardware devices is not allowed. This is a design choice that helps facilitate the portability of VR applications by preventing them from depending on specific input device makes and models. Instead, the applications are granted access to the device through a proxy. A proxy is nothing more than an intermediary who forwards information between two parties. In this case, the two parties are a VR Juggler application and an input device. The application makes requests on the input device through the proxy.

Proxies are acquired through the Gadgeteer Input Manager, but the process is not entirely straightforward. To assist with the use of device proxies, Gadgeteer provides what are called device interfaces. Device interfaces hide the details of proxy acquisition through the Input Manager, but the concept of device interfaces is something that often causes confusion for those new to VR Juggler application programming. Two object-oriented design patterns are combined by device interfaces: smart pointers and proxies. Within this section, we aim to explain Gadgeteer device proxies and device interfaces clearly and simply. We begin with high-level descriptions of both and then move on to their use in VR Juggler application objects.

High-Level Description of Device Proxies

As noted above, access to input device data is granted through device proxies allocated by the Gadgeteer Input Manager. For each type of input device (digital, analog, keyboard/mouse, etc.), there is a device proxy class. As a programmer of VR Juggler applications, knowledge of such proxies does not have to be terribly in-depth. The fact is, most VR Juggler application object programmers will probably never need to know more about the interface of a type-specific device proxy than the return type of its data request method (usually named getData()). We will see more about the data request method in the explanation of device interfaces below. Most of the perceived complexity in the type-specific proxy classes is only important to Gadgeteer's internal maintenance of the active proxies. The following is the complete list of proxy classes in Gadgeteer 1.0 (used by VR Juggler 2.0):

gadget::AnalogProxy

The proxy type for analog input devices (those of type gadget::Analog). It is defined in the header file gadget/Type/AnalogProxy.h. The return type of gadget::AnalogProxy::getData() is float.

gadget::CommandProxy

The proxy type for command-oriented input devices (those of type gadget::Command). It is defined in the header file gadget/Type/CommandProxy.h. The return type of gadget::CommandProxy::getData() is int.

gadget::DigitalProxy

The proxy type for digital (on/off) input devices (those of type gadget::Digital). It is defined in the header file gadget/Type/DigitalProxy.h. The return type of gadget::DigitalProxy::getData() is gadget::Digital::State, an enumerated type. This means that values returned by gadget::DigitalProxy::getData() can be treated as integers.

gadget::GloveProxy

The proxy type for glove input devices (those of type gadget::Glove). It is defined in the header file gadget/Type/GloveProxy.h. The return type of gadget::GloveProxy::getData() is gadget::GloveData.

gadget::KeyboardMouseProxy

The proxy type for keyboard/mouse input handlers (those of type gadget::KeyboardMouse). It is defined in the header file gadget/Type/KeyboardMouseProxy.h. There is no method gadget::KeyboardMouse::getData(). Rather, the method to use for querying input data is gadget::KeyboardMouseProxy::getEventQueue(), the return type of which is gadget::KeyboardMouse::EventQueue. We explain more about this below in the section called “Using gadget::KeyboardMouseInterface.

gadget::PositionProxy

The proxy type for position tracking input devices (those of type gadget::Position). It is defined in the header file gadget/Type/PositionProxy.h. The return type of gadget::PositionProxy::getData() is gmtl::Matrix44f. The method gadget::PositionProxy::getData() takes an optional float paramter that indicates the units to use for the returned transformation matrix. The value must be a conversion factor from meters to the desired units.

gadget::StringProxy

The proxy type for string (text- or word-driven) input devices (those of type gadget::String). It is defined in the header file gadget/Type/StringProxy.h. The return type of gadget::StringProxy::getData() is std::string.

In summary, the important thing to know is that a proxy is a pointer to a physical device. Application object programmers should normally use the higher level device interface as the mechanism to acquire a proxy and read data from the proxied device. The device interface encapsulates some type of proxy that in turn points to an input device. That device can be a wand, a keyboard, a light sensor, or a home-brewed device that reads some input and returns it to Gadgeteer in a meaningful way. That is a lot of indirection, but it makes the handling of physical devices by Gadgeteer incredibly powerful.

High-Level Description of Device Interfaces

Device interfaces are designed to act as wrappers around type-specific device properties. This is implemented through the (template) class gadget::DeviceInterface<T>. Applications could use the proxy classes directly, but as we have already noted, acquiring the desired proxies from the Gadgeteer Input Manager is not wholly straightforward. The type-specific instances of gadget::DeviceInterface<T> (such as gadget::PositionInterface, gadget::DigitalInterface, etc.) simplify acquisition of proxies. Thus, typical VR Juggler application objects will have one or more device interface member variables and no proxy member variables.

The class gadget::DeviceInterface<T> is a templated class based on the proxy type it wraps. The following is the complete list of available gadget::DeviceInterface<T> instantiations in Gadgeteer 1.0 (used by VR Juggler 2.0):

gadget::AnalogInterface

This is a typedef for the template instantiation gadget::DeviceInterface<gadget::AnalogProxy>. It wraps the proxy type gadget::AnalogProxy and is used for reading data from analog devices. Include the header file gadget/Type/AnalogInterface.h.

gadget::CommandInterface

This is a typedef for the template instantiation gadget::DeviceInterface<gadget::CommandProxy>. It wraps the proxy type gadget::CommandProxy and is used for reading data from command-driven devices. Include the header file gadget/Type/CommandInterface.h.

gadget::DigitalInterface

This is a typedef for the template instantiation gadget::DeviceInterface<gadget::DigitalProxy>. It wraps the proxy type gadget::DigitalProxy and is used for reading data from digital (on/off) devices. Include the header file gadget/Type/DigitalInterface.h.

gadget::GloveInterface

This is a typedef for the template instantiation gadget::DeviceInterface<gadget::GloveProxy>. It wraps the proxy type gadget::GloveProxy and is used for reading data from glove devices. Include the header file gadget/Type/GloveInterface.h.

gadget::KeyboardMouseInterface

This is a typedef for the template instantiation gadget::DeviceInterface<gadget::KeyboardMouseProxy>. It wraps the proxy type gadget::KeyboardMouseProxy and is used for reading data from analog devices. Include the header file gadget/Type/KeyboardMouseInterface.h.

gadget::PositionInterface

This is a typedef for the template instantiation gadget::DeviceInterface<gadget::PositionProxy>. It wraps the proxy type gadget::PositionProxy and is used for reading data from position tracking devices. Include the header file gadget/Type/PositionInterface.h.

gadget::StringInterface

This is a typedef for the template instantiation gadget::DeviceInterface<gadget::StringProxy>. It wraps the proxy type gadget::StringProxy and is used for reading data from string (text- or word-driven) devices. Include the header file gadget/Type/StringInterface.h.

The typedefs are provided to make the application object code more readable.

In the application object, a device interface member variable is used as a smart pointer to the proxy. In C++, a smart pointer is not usually an actual object pointer. Instead, the class acting as a smart pointer overloads the dereference operator -> so that a special action can be taken when the “pointer” is dereferenced. The dereference operator is just another operator like the addition and subtraction operators, and overloading the deference operator allows some extra work to be done behind the scenes. On the surface, the code looks exactly the same as a normal pointer dereference, and in most cases, people reading and writing the code can think of the smart pointer as a standard pointer. It may also be convenient to think of a smart pointer as a handle.

With that background, we can move on to explain how gadget::DeviceInterface<T> uses these concepts. In user code, there will be instances of types such as gadget::DigitalInterface, gadget::PositionInterface, gadget::KeyboardMouseInterface, and the like. Once they are properly initialized, device interface objects (whatever their types may be) will act as smart pointers to the actual Gadgeteer device proxy objects that they encapsulate.

At this point, it is perfectly reasonable to wonder why Gadgeteer uses a concept that requires all sorts of documentation and explanation. The extra effort is worth it because it allows Gadgeteer to hide the actual type of the device being used. There is no need to know that some specific VR system uses a wireless mouse connected to a PC reading bytes from a PS/2 port that represent button presses. All that matters is knowing which buttons are pressed at a given instant. The class gadget::DigialInterface gives exactly that information, and it quietly hides the messiness of dealing with the mouse, its driver, and its communication protocol.

Using Device Interfaces

Before using a device interface, some objects must be declared. Programmers must choose the type that is appropriate for the type of devices relevant to a given application. All device interface objects must be initialized in the application object's override of the method vrj::App::init() method. Because we are dealing with a templated type (gadget::DeviceInterface<T>), every type-specific instantiation has the same interface. Hence, all type-specific device interfaces are initialized using the gadget::DeviceInterface<T>::init() method. This method takes a single string argument naming the proxy to which the interface will connect. The name can be the name of a proxy or a proxy alias, both of which are defined in VR Juggler configuration files. Example names are “VJHead”, “Wand”, “VJButton0”, and “Accelerate Button. Using meaningful symbolic names makes them easier to remember, and it also contributes to hiding the details about the physical device. With this system, no one needs to care how transformation information from the user's head is generated. Gadgeteer cares, but there is no need for it to tell anyone else. All developers care about is the head transformation matrix. An example of initializing a gadget::PositionInterface that connects with the user head proxy is:

gadget::PositionInterface head;

head.init("VJHead");

Remember that this has to be done in an application object's init() method. The actual object used would be a member variable of the application class. Note that here, the normal syntax for calling the method of a C++ object is used rather than using the dereference operator. Until it is initialized, the device interface object cannot act as a smart pointer.

Once device interface objects are all initialized and ready to use, it is time to start using them as smart pointers. VR Juggler and Gadgeteer are already working hard in the background to update device proxies, and the application is free to access them. It is usually best to acquire data from the device proxy through the device interface in the preFrame() method, but this may not necessarily be true for all proxies. Continuing with our example of a gadget::PositionInterface to the user head proxy, the following code shows how to read the transformation matrix for the user's head (in feet):

gmtl::Matrix44f head_mat = head->getData();

Believe it or not, the code really is that easy. Simply use the overloaded dereference operator to get access to the position proxy object hidden in gadget::PositionInterface to read data from the proxy. We now move on to explain the use of type-specific device interfaces.

Using gadget::AnalogInterface

Analog devices return floating-point data. As noted above, the return type of gadget::AnalogProxy::getData() is float. Behind the scenes, analog devices in Gadgeteer scale their input so that application objects always receive it in the range 0.0 to 1.0 inclusive (also denoted [0.0,1.0] in mathematically oriented descriptions). Hence, application objects can always expect analog data to be in that range regardless of the specific type of analog device being used.

Using gadget::CommandInterface

Command-oriented devices were introduced in Gadgeteer 1.0 Beta 1. They are an evolving device type geared towards complex input that can be interpreted at an abstract level. In Gadgeteer 1.0, such input comes in the form of spoken phrases that are reinterpreted as commands identified by unique integer values. In future versions of Gadgeteer, this device type will be used for scalable gesture recognition. The return type of gadget::CommandProxy::getData() is int, and it is up to the person configuring the command-oriented device to set up the command-to-integer mappings.

Using gadget::DigitalInterface

Digital devices are those that have distinct on and off states. The method gadget::DigitalProxy::getData() returns the current state of such a device as a value of the enumerated type gadget::Digital::State. This type is defined to allow for easy on/off testing, but it also provides state toggling information. The possible values of gadget::Digital::State are OFF (integer value 0), ON (integer value 1), TOGGLE_ON (integer value 2), TOGGLE_OFF (integer value 3). In Example 3.1, “Using gadget::DigitalInterface in an Application Object”, we see some example uses of the information returned by gadget::DigitalProxy::getData().

Example 3.1. Using gadget::DigitalInterface in an Application Object

  1 void MyApp::preFrame()
    {
       if ( mButton0->getData() )
       {
  5       // Set state for when mButton0 is pressed, has been pressed
          // since the last frame, or has been released since the
          // last frame ...
       }
       else
 10    {
          // Set state for when mButton0 is not pressed ...
       }
    
       switch (mButton1->getData()
 15    {
          case gadget::Digital::OFF:
             // Set state for when mButton1 is not pressed ...
             break;
          case gadget::Digital::ON:
 20          // Set state for when mButton1 is pressed ...
             break;
          case gadget::Digital::TOGGLE_ON:
             // Set state for when mButton1 has been pressed since
             // the last frame ...
 25          break;
          case gadget::Digital::TOGGLE_OFF:
             // Set state for when mButton1 has been released since
             // the last frame ...
             break;
 30    }
    
       if ( mButton2->getData() == gadget::Digital::TOGGLE_ON )
       {
          // Set state when mButton2 goes "high" (is toggled on) ...
 35    }
       else if ( mButton2->getData() == gadget::Digital::TOGGLE_OFF )
       {
          // Set state when mButton2 goes "low" (is toggled off) ...
       }
 40 }

Using gadget::GloveInterface

Using gadget::KeyboardMouseInterface

Input read from a keyboard and a mouse is provided through gadget::KeyboardMouseProxy, instances of which are acquired through gadget::KeyboardMouseInterface. Unlike most other device proxy types, gadget::KeyboardMouseProxy does not have a getData() method. Rather, it has a method called getEventQueue() with return type gadget::KeyboardMouse::EventQueue that is the “event queue.” Keyboard and mouse input is handled as events, either key press events or mouse events. Key press events come from the keyboard and are for both the pressing and releasing of individual keys or keys with modifiers (CTRL, ALT, and SHIFT). Mouse events include both the motion of the mouse in the X & Y axes and the pressing and releasing of mouse buttons which may or may not be associated with a keyboard modifier.

The event queue contains all the key press and mouse events that occurred since the last frame. Each event is contained in an object of type gadget::EventPtr[1]. The type gadget::EventPtr is a reference-counted smart pointer for instances of gadget::Event, which is in turn a base class for gadget::KeyEvent and gadget::MouseEvent. Each of these has its own reference-counted smart pointer, namely gadget::KeyEventPtr and gadget::MouseEventPtr. This seems like a lot of types to understand, but it is simple enough to use by keeping in mind that there are only two types of events: key press events and mouse events. Furthermore, user code should only be interested in gadget::KeyEventPtr and gadget::MouseEventPtr. The specific event type is determined through the method gadget::Event::type().

At this point, observant readers will be wondering how to downcast instances of gadget::EventPtr to either gadget::KeyEventPtr or gadget::MouseEventPtr. All three of the reference-counted smart pointer types make use of Boost shared pointers (instantiations of the type boost::shared_ptr<T>), part of the Boost smart pointer library. Boost shared pointers have their own version of the built-in C++ operation dynamic_cast<T,U>() called boost::dynamic_pointer_cast<T,U>(). It works the same way as dynamic_cast<T,U>(), but it is designed specifically for Boost shared pointers. In Example 3.2, “Using gadget::KeyboardMouseInterface in an Application Object”, we see how to put all of this together in order to handle keyboard and mouse input.

Example 3.2. Using gadget::KeyboardMouseInterface in an Application Object

  1 #include <boost/shared_ptr.hpp>
    #include <gadget/Type/KeyboardMouseInterface.h>
    #include <gadget/Type/KeyboardMouse/KeyEvent.h>
    #include <gadget/Type/KeyboardMouse/MouseEvent.h>
  5 #include <vrj/Draw/OGL/GlApp.h>
    
    
    // This is here to shorten the use of the function in preFrame()
    // boost::dynamic_pointer_cast<T,U>() below.
 10 using namespace boost;
    
    class MyApp : public vrj::GlApp
    {
    public:
 15    MyApp() : vrj::GlApp()
       {
          /* Do nothing. */ ;
       }
    
 20    virtual ~MyApp()
       {
          /* Do nothing. */ ;
       }
    
 25    void init()
       {
          mKeyboard.init("VJKeyboard");
       }
    
 30    void preFrame()
       {
          gadget::KeyboardMouse::EventQueue evt_queue =
             mKeyboard->getEventQueue();
          gadget::KeyboardMouse::EventQueue::iterator i;
 35 
          // Loop over all the keyboard and mouse events that
          // occurred since the last frame.
          for ( i = evt_queue.begin(); i != evt_queue.end(); ++i )
          {
 40          const gadget::EventType type = (*i)->type();
    
             if ( type == gadget::KeyPressEvent ||
                  type == gadget::KeyReleaseEvent )
             {
 45             gadget::KeyEventPtr key_evt =
                   dynamic_pointer_cast<gadget::KeyEvent>(*i);
                // Handle the key press event ...
             }
             else if ( type == gadget::MouseButtonPressEvent ||
 50                    type == gadget::MouseButtonReleaseEvent ||
                       type == gadget::MouseMoveEvent )
             {
                gadget::MouseEventPtr mouse_evt =
                   dynamic_pointer_cast<gadget::MouseEvent>(*i);
 55             // Handle the mouse event ...
             }
          }
       }
    
 60    void draw()
       {
          // Draw something ...
       }
    
 65 private:
       gadget::KeyboardMouseInterface mKeyboard;
    };

Using gadget::PositionInterface

Position tracking devices return data to application objects as 4×4 transformation matrices. The return type of gadget::PositionProxy::getData() is gmtl::Matrix44f, which was introduced in the section called “The gmtl::Matrix44f Helper Class”. All tracking devices return a full transformation matrix even if the physical tracking hardware is only capable of returning translation or orientation data.

Tip

When querying a positional device for its data, it is critical to ask for the data in the units that the application expects. An easy way to do this is to pass the result of getDrawScaleFactor() to gadget::PositionProxy::getData(). By default, gadget::PositionProxy::getData() returns data in feet, and the implementation of vrj::App::getDrawScaleFactor() returns gadget::PositionUnitConversion::ConvertToFeet. This is for backwards compatibility with VR Juggler 1.0 behavior. See Example 3.3, “Requesting Positional Data in Application-Specific Units” for an example of this. Refer to the section called “vrj::App::getDrawScaleFactor() for more information about getDrawScaleFactor().

Example 3.3. Requesting Positional Data in Application-Specific Units

  1 #include <gmtl/Matrix.h>
    
    #include <gadget/Type/Position/PositionUnitConversion.h>
    #include <gadget/Type/PositionInterface.h>
  5 
    #include <vrj/Draw/OGL/GlApp.h>
    
    
    class MyApp : public vrj::GlApp
 10 {
    public:
       MyApp() : vrj::GlApp()
       {
          /* Do nothing. */ ;
 15    }
    
       virtual ~MyApp()
       {
          /* Do nothing. */ ;
 20    }
    
       void init()
       {
          mWand.init("VJWand");
 25    }
    
       // Use meters for the application units.
       float getDrawScaleFactor()
       {
 30       return gadget::PositionUnitConversion::ConvertToMeters;
       }
    
       void preFrame()
       {
 35       // Request the current wand transformation matrix in
          // application units (meters).
          const float units = getDrawScaleFactor();
          gmtl::Matrix44f wand_mat(mWand->getData(units));
          // Do something with the wand transformation ...
 40    }
    
       void draw()
       {
          // Draw something ...
 45    }
    
    private:
       gadget::PositionInterface mWand;
    };

Using gadget::StringInterface

String (text- or word-driven) devices were introduced in Gadgeteer 1.0 Beta 1. They are an evolving device type geared towards textual or spoken input. In Gadgeteer 1.0, such input comes in the form of spoken phrases that are returned to the application object as strings matching those in a pre-defined grammar. In future versions of Gadgeteer, this device type may be used for additional forms of high-level input. The return type of gadget::StringProxy::getData() is std::string, and it is up to the person configuring the string device to set up the recognized grammar.

Stupefied Proxies

The indirection provided by device proxies facilitates run-time reconfiguration of hardware devices. If a hardware device breaks down while an application is running, the device can be replaced without shutting down and restarting the application. To support this capability, proxies can become “stupefied,” which means that they are not connected to a device and cannot return new data.

In general, application programmers do not need to worry about stupefied proxies. Data will be returned by the proxy whether it is stupefied or not, but if the proxy is stupefied, it cannot return new data. Stupefied proxies can occur as a result of an error in the VR Juggler configuration or because the hardware device pointed at by the proxy failed to start up correctly.

To determine whether a proxy is stupefied, the method gadget::Proxy::isStupefied() can be used. The stupefied state cannot be changed programatically by user code, however. Only the Input Manager is capable of reconfiguring a proxy to point at a new valid device. Hence, changing the stupefication state of a proxy from an application object will have no effect and may cause the application to crash.

Warning

In all versions of VR Juggler prior to 2.0 Beta 3, the word “stupefied” was misspelled as “stupified.” The method gadget::Proxy::isStupified() is retained in Gadgeteer 1.0 for backwards compatibility, but it will be removed in Gadgeteer 1.2 (which will ship as part of VR Juggler 2.2).

The Gory Details

What is truly amazing about Gadgeteer device interfaces is, despite their seeming complexity, there is really nothing to them. Trying to trace through the source code is a little tricky, but conceptually, it is all about pointers. Keep in mind that all this documentation was written using nothing more than the Gadgeteer header files as a reference.

As mentioned, the class gadget::DeviceInterface<T> provides the method interface for all the type-specific instantiations, including the overloaded dereference operator. The base class of gadget::DeviceInterface<T>, gadget::BaseDeviceInterface, maintains the name of the proxy and the proxy reference itself, and it provides the all-important init() method.

The beauty of it all is that the proxy object being pointed to by the device interface can be changed without affecting the execution of the user application. In other words, the proxies can be changed at run time to point to different physical devices. All the while, the user code is still using the smart pointer interface and getting data of some sort. This flexibility is one of the most important features of Gadgeteer, and it is important to understand.



[1] Behind the scenes, gadget::KeyboardMouse::EventQueue is a typedef for std::vector<gadget::EventPtr>.