Electrical and Computer  
Logitech 6D Mouse            Backloop Audition for Telerobotics 
Backloop Audition for Teleoperation
Bats Use Echolocation
The goal of the project is to develop an inexpensive, aural method of providing sensory information for the operation of a remote manipulator
Tactile feedback is expensive and difficult, and not always suited to the task.  By providing auditory clues, it will be possible to inform the operator of conditions at the remote location, including but not limited to proximity, pressure, position, orientation, torque, and perhaps vibratory sensation and electrical activity.
Using a standard computer mouse to decouple actions from results, and a MIDI instrument to generate familiar musical tones, preliminary tests indicated that it is possible to locate a cursor position by auditory clues alone. In the two-dimensional configuration, the computer screen shows an ordinary cursor (a cross-shaped icon) which moves in response to input from a standard computer mouse, just as it would in any common application such as word processing.  In the research program, however, the operator is presented with a rectangular area in the center of the screen as a target, and auditory information corresponding to the mouse position.  In one form of the demonstration program, for example, an Organ tone represents mouse movement from side to side, and the Strings voice indicates movement forward and back.  As the mouse moves, the pitch of the musical notes changes.  By comparing the pitch of the computer-calculated tones to a reference pitch available with a right mouse click, it is possible for the operator to move the cursor into the target area -- without looking at the screen.
Logitech 6D Mouse A Logitech flying mouse makes it possible for the user to operate in three dimensions, with six degrees of freedom, using audible information generated by the computer, based on input from multiple sensors.  By using a proximity sensor based on the NASA Capaciflector principle, for instance, with a device mounted on the robotic gripper, nearness to an object can be detected.  Grip force detectors, and light sensors also provide input.  In the current configuration, output is provided by a common sound board and by a MIDI synthesizer  
The results from this research can be used to develop less expensive remote manipulators for hostile environments, to improve the performance of human-guided, robotically-ganged machine tools, to provide target information to physicians who use robotic surgical assistants, to extend the abilities of devices used in space, and to produce inexpensive, portable devices for handling hazardous and explosive materials.  Aural representation could also be used to create reasonably priced instruments to sense the vibrations of blood vessels, the hollowness of ducts, the rigidity of bone, or the electrical environment created by nerve activity, as a guide for surgeons who use robotic aids to perform microsurgery.  Application of the principles to aircraft, especially fighter jets, can be used to reduce pilot information overload by moving some of the inputs which now require his attention, to the subliminal plane.
Whatever the application, the auditory scene created by the device does not require interpretation -- it is merely understood -- and the operator uses the information, not as a tool, but as another sense.
Logitech Flying Mouse The Logitech MOUSE is a hand-held input device similar to a standard mouse, but free to move up and down and to rotate, in addition to the usual horizontal movements, generating input suitable for controlling a free-moving robotic arm and hand  
Robotic Arm
The robotic manipulator ARM was assembled from a Robix RCS-6 Construction Set, in a modified Chemist configuration.  The Robix interface is at the left, the radio frequency transmitting video camera can be seen in the upper middle position, and a small test object is at the lower right of the image.  The orange and black strobe lights warn of possible injury while weighting the assembly to prevent tipping.   
The Robix HAND has been modified to carry pressure and light sensors.  It is seen here approaching the test object. Robotic Jaws
Please visit to see an interactive, animated, virtual reality presentation of the Robix Construction Set, in a similar configuration.  The simulation can be viewed using a Java plug-in, or go to to download a free VRML browser.
  1. The MIDI interface was chosen as the sound source so that a separate device could be responsible for voice generation, leaving the computer free to control the robotic arm and do signal conditioning.  The current configuration outputs to a Korg 2000 electric piano, and to a Yamaha PSR510 electronic keyboard.
  2.  The MS Windows software which has been written, works with Sound Blaster compatible boards as well as with external instruments.  The present configuration uses a Sound Blaster AWE 64.
Copyright ©1998 JB Tait  981021.1956