Using VRPN and DIVERSE to Integrate Haptic Devices with Networked Immersive Environments

Introduction:

This report describes a study on how to link DIVERSE (Device Independent Virtual Environment Re-configurable Scalable Expandable) with VRPN (Virtual Reality Peripheral Network) to provide a haptic force feedback capability to DIVERSE. DIVERSE is free(GNU LGPL & GPL) software which consists of API and utilities which aid in and develop virtual environments. DIVERSE is composed of DIVERSE graphics interface for Performer(dgiPf) and the DIVERSE ToolKit(DTK). dgiPf is the graphics base needed to visualize virtual environment on any Linux/Irix platform able to run Performer binaries. DTK is a powerful utility kit which links local memory to various devices local and remote, and allows different dynamic shared objects(DSOs) to be loaded and unloaded at runtime. DTK is not a center-of-universe paradigm, i.e. DTK will not limit other programs/services from accessing its resources and vice-versa simultaneously sharing the environment. VRPN is similar in concept to DTK by bringing together devices, but it does this in a different way compared to DTK. These VRPN devices are serial devices, i.e. joystick, wand, tracking devices, haptic devices, button boxes, etc... However the haptic part of VRPN, its phantom server is a resource hungry program, very much like a center-of-universe paradigm. This report will discuss further about the details of getting these two entities to work together.

DTK service manages and shares memory on the local machine and gives access to dynamic shared objects which are limited and specific in functionality to the DTK services. One such example is creating a shared memory segment of floats, then allowing client on the same and/or remote machine to connect to this shared memory segment service. This shared memory segment can be accessed(read or written to) by any client connected to it, multiple clients can modify it, the value resident in memory will be that of the last client to modify the memory segment. These floats can now be read by another machine and used in the same manner as a part of local memory on the remote machine. This communication occurs via unreliable datagram packets(UDP) sockets connection from once DTK server machine to another DTK server machine.

Unlike the DTK server, the VRPN server on a windows machine is device dependent and is a standalone service. Currently VRPN only supports the haptic device under the Windows NT Operating system and General Haptic Open Software Toolkit(GHOST) Libs 3.1. The reason for this standalone service is that the VRPN service is run in a continuous loop without any pauses built into it. So due to this reason, the VRPN phantom server is a standalone haptic server PC which does not allow any other application to access system resources without experiencing extensive delays.

After realizing that VRPN already had an application programming interface for the phantom haptic device, it was easier to choose to add communication between DTK and VRPN to allow DTK access to haptic function rather than rewriting DTK server source to duplicate VRPN's API. UPD sockets were chosen to be the mode of communication between the two servers since both servers use this transport inherently.

Development:

To facilitate this project of linking DTK and VRPN together, the proper hardware and software needed to be acquired, installed and functioning correctly. This included the following list of items:

Hardware:

• 2 PCs[Pentium II class cpu, 256 Megs of RAM, 20 Gig Hard Drive, PCI expansion slots available, AGP video card, CDROM, Network card]
• 1 Phantom Haptic device[Phantom motor unit, Stylus pen, Amplifier Box, PCI interface card]

Operating Systems, system libraries and compilers:

• MS Windows 2000 Professional, Red Hat Linux 7.0(kernel 2.4), Slackware Linux 7.1(kernel 2.2.18), MS Visual Studio 6.0, GHOST Libs ver 3.1, GCC ver 2.9.6, glibc 2.1, GNU make 3.79.1

Software:

• DTK release 1.2.9, VRPN ver 5.02, Fast Light Tool Kit(FLTK) ver 1.0.10{FLTK is used when displaying graphical dtk-sliders dynamic shared object(DSO)}

The above described hardware and software is assembled into two PCs. One houses the phantom haptic device and runs Windows 2000, Visual studio, and VRPN phantom server code. The second PC contains a flavor of Linux, DTK, FLTK, and VRPN client. The second PC may be substituted with a functioning Irix workstation if desired. Other optional hardware for this demo included the VTCave(running on an 8processor ONYX) and the VT I-Desk.

Source code for the VRPN/DIVERSE interface is available here.

The source for both DTK and VRPN server was not modified but client(DSO) source was modified to make the connection possible. This would allow the end programmer to spend much less time if any at all when debugging DTK and VRPN server source code. Concentration could be focused on client application or DSOs that connect to the servers. Off-the-shelf DTK and VRPN installations would be sufficient to start work. Two clients would need to be modified, one from DTK and other from VRPN examples database. The DTK client will behave as a transmitter and send the data. The VRPN client will behave as receiver and collect that data. The data being of type float.

UDP socket client is then inserted into the main loop of the DTK DSO that reads a specific set of shared memory segment float values, this socket client then sends these packets to the UDP server running on a specific ipaddress on the network. The UDP server on that IP address continuously waits for UDP packets to arrive from any client source on the network and then proceeds to next step of the loop which communicates those floats to the VRPN client. The VRPN client then updates the VRPN server on the phantom haptic PC and produces force feedback to the haptic device end effector.

The following is a detailed step by step procedure for modifying DTK, VRPN, and UDP server/client code to generate an example demo. This demo will use input from DTK sliders to modify the spring stiffness coefficient(k) of a sphere shell and the damping coefficient(d) of center filled with viscous liquid media.

To begin, check that DTK and VRPN are properly installed on the systems by following the below steps on the two above said systems.

Checking DTK installation on the Linux Box/Irix workstation:

Make sure installed DTK bin directory is in current shell path. Locate current dtk installation directory by running "dtk-server --about" without quotes in a shell.

1. Run the DTK server on this machine (run in a shell). Type:
   dtk-server --verbose
2. Create a shared memory segment served by the server(from step 1) called "foo" (run in a second shell). Type:
   dtk-getSharedMem 24 foo
3. Run the DTK read floats program which reads the floats stored in the memory object called "foo" (run in that same second shell). Type:
   dtk-readFloats foo -r
4. Run the DTK sliders program which write floats into the "foo" object (run in a third shell). Type:
   dtk-sliders foo

The above steps should result in a corresponding update of float values in the second shell as the graphical slider bar is moved around within its range. This indicates that the DTK and FLTK installation was successful. Type "Ctrl-c" simultaneously to exit out of above running processes when done.

Checking the VRPN installation on the Windows 2000 workstation:

Logon as an Administrator. Open a command prompt window by clicking start menu, then run item, then type cmd in field and hit enter.

1. Change directory to VRPN_HOME\vrpn\server_src\pc_win32\Debug\ and run the phan_server program(no additional parameters). Once server has started, hold the phantom haptic device's stylus in resting position and wait for 10 seconds or till you hear/feel it click.
2. Start another command prompt shell, change directory to VRPN_HOME\vrpn\client_src\pc_win32\, then run sphere_client 0.1 0.1 Phantom@phantom.sv.vt.edu, where Phantom is the name of the haptic device on the machine whose network name/address is phantom.sv.vt.edu.

After the above steps, a virtual sphere should be felt by the haptic device end-effector, and the shell should be solid and impenetrable. This indicates that VRPN-haptic device installation was successful. Type Ctrl-c to exit the demo when done.

Checking the VRPN installation on the Linux box/Irix workstation:

Make sure that the phantom PC has been installed and is currently running phan_server, then change directory to VRPN_HOME/vrpn/client_src/pc_linux/ then type sphere_client 0.1 0.1 Phantom@phantom.sv.vt.edu

The sphere can now be felt at the phantom haptic device and should be exactly same feedback(feeling) as when client ran from phantom PC itself. This had successfully demonstrated that the VRPN client can connect to the VRPN server over the network.

Now that the server software is installed and working properly on the server and client machines, the client DSOs can be modified to integrated DTK shared memory with VRPN message passing clients using UDP socket client and server code.

Below is a generic UDP client and server code, if compiled as is, the result will be a UDP server which sits listening for UDP socket connection on port 1500 and the UDP client which had to be run (with the float/data variables/messages as command line parameters). These parameters will be passed to the server and displayed out to the screen on server side.

Checking UDP server and client:

Compile and function properly follow the compile/run procedure listed below.

1. Create a directory, store the udpClient.c and udpServer.c files there.
2. Compile the files with the following usage;
   gcc -Wall -o udpServer udpServer.c
   gcc -Wall -o udpClient udpClient.c
3. Test to see if client and server connect to each other properly,
   Type from that path, ./udpServer
   In a separate shell or from another machine type ./udpClient A.B.C.D float1 float2 float3 etc... where "A.B.C.D" is the network ipaddress of the machine where udpServer is running.

The server shell should output the floats that reflect the corresponding floats sent by the client program.

Now the UDP client udpClient.C can be added into the mainloop of a DTK DSO dtk-readFloats.c. Read-Floats is used to read floats from shared memory segments and output those values to the display. In addition to displaying these values, they can now be formatted and sent as UDP packets. The modified sample dtk-readFloats source code looks similar to the source in the file dtk-readFloatsUDP.c. This file can be compiled by running the following make file make2. This make2, dtk-readFloatsUDP.c, and the common_arg_stuff.o files must be in the same location, preferably in dtk-1.2.9/clients/utils_sharedMem/ directory. After client file is compiled, the server file must be modified by adding the VRPN client code from sphere_client.c into udpServer.c's mainloop. That modified udpServer/VRPN client should be similar to sphere_clientUDP.c file. To compile this code, rename the following make file Makefile.sphere to Makefile, place it into same directory and run make. Once the udp client and server code have been merged into DTK and VRPN clients, the task is complete. Now a dtk-slider on a remote machine can control the spring damping and stiffness coefficient dynamically on the phantom haptic device thru VRPN.

Checking DTK-VRPN UDP link:

Follow the procedure described below to test the link between DTK and VRPN. The following procedure will demonstrate that changing the slider position will send UDP packets to the VRPN client which will update the corresponding spring stiffness and damping values and pass them onto the VRPN phantom haptic server, which will produce the corresponding force feedback to the end effector.

Windows 2000 Machine:

1. Run the phantom server PC, startup phan_server as Administrator.

Linux Box:

1. dtk-server --verbose
2. from vrpn/client_src/pc_linux/ directory type sphere_clientUDP 0.1 0.1 Phantom@phantom.sv.vt.edu
3. dtk-getSharedMem 24 foo
4. dtk-readFloatsUDP foo -r
5. dtk-sliders foo

Now the sliders will be able to control the force feedback device's spring damping and stiffness coefficients, The above modified code and the step by step procedure detailed the connection link between DTK DSO and VRPN client allowing the shared memory segment from DTK to control VRPN haptic material property and force feedback. The above demo showed only two vrpn_ForceDevice properties being modified. The following is a full list of vrpn_ForceDevice properties that can be modified via DTK/UDP float client/server.

List of force device properties that control the haptic device:

List is extracted from vrpn_ForceDevice.h, and its laymen's explanation.

SurfaceKspring

surface spring stiffness coefficient

SurfaceKdamping

surface spring damping coefficient

SurfaceFstatic

surface static friction

SurfaceFdynamic

surface dynamic friction

SurfaceKadhesionNormal

coefficient of stickiness preventing pulling away from surface plane

SurfaceKadhesionLateral

coefficient of stickiness preventing sliding along surface plane

SurfaceBuzzFreq

buzz speed

SurfaceBuzzAmp

buzz height

SurfaceTextureWavelength

wavelength of surface texture

SurfaceTextureAmplitude

amplitude{height} of surface texture

FF_Origin

force field origin

FF_Force

force field force

FF_Jacobian

force field transformation gradient

FF_Radius

force field decay radius

Plane

solid impenetrable flat plane

Summary and conclusions:

References:

DIVERSE

http://www.diverse.vt.edu

Molecular Docking

http://www.sv.vt.edu/future/cave/resprj/idock/idock.html

VRPN

http://www.cs.unc.edu/Research/vrpn/

UDP sockets programming

http://www.pont.net/socket/index.html

IDock

http://www.ncsa.uiuc.edu/Vis/Projects/Docker/

Acknowledgments: