The National Science Foundation's (NSF) Tokyo Office periodically receives and disseminates reports on research developments in Japan that are related to the Foundation's mission. NSF-sponsored researchers currently working in Japan prepare many of these reports. These reports present information for use by NSF program managers and policy makers; they are not statements of NSF policy. .
Ms. Allison M. Okamura prepared the following report. Ms. Okamura is a Graduate Research Assistant in the Dextrous Manipulation Laboratory of the Center for Design Research at Stanford University; her advisor is Dr. Mark R. Cutkosky. From January - April, 1998, Ms. Okamura conducted research at the Robotics Division of the Mechanical Engineering Laboratory (MEL) located in Tsukuba Science City. Her host at MEL was Dr. Kazuo Tanie. Ms. Okamura's research visit was supported by a Dissertation Enhancement Grant awarded by the Japan-Korea Program of the Division of International Programs at NSF. Ms. Okamura can be reached via email at allisono@cdr.stanford.edu
Robotic hands with multiple fingers are being developed with the goal of creating autonomous systems. These hands can manipulate objects while emulating the dexterity of human hands. Such systems could manipulate in hazardous and remote environments, alleviating danger for humans. Towards this goal, my dissertation seeks to develop algorithms for haptic (via touch) exploration of unknown objects in order to detect and record global shape, fine surface features, and other object properties. One of the difficulties in accomplishing exploratory tasks with multi-fingered robotic hands is the lack of practical tactile sensing and detailed shape recognition schemes. At the Mechanical Engineering Lab in Tsukuba, Japan, I used robotic fingers and optical waveguide tactile sensors developed by Dr. Hitoshi Maekawa for surface shape and feature detection/identification. The goal of this Dissertation Enhancement Award was to experiment with using available sensor data combined with contextual information to determine object information not readily available from other sensing systems such as force and vision.
For a multi-fingered robotic hand to autonomously manipulate and "explore" an unknown object, an exploratory procedure was previously developed [1]. This prior work used two three-degree-of-freedom planar fingers and a passive palm. At MEL, the exploratory procedure algorithm was modified for three two-degree-of-freedom fingers and simulation was used to determine the feasibility of manipulating various object shapes, given the workspace limitations of the robotic fingers available. The difficulty of planning the exploration increases because, with only two degrees of freedom in the plane, the contact point must change during manipulation.
The exploratory procedure allows the robot fingers to alternately manipulate and explore the object. During a manipulation phase, two or three of the fingers are used to grasp the object and they use rolling contacts in order to rotate and translate the object to a desired position. During an exploratory phase, two fingers grasp the object securely while the third finger slides over surface to a new position. By sequencing through manipulation and exploration phases for each finger, the entire surface of the object can be explored. In this project, only two-dimensional exploration was implemented.
The optical waveguide sensors developed by Dr. Maekawa can report the centroid of contact on a hemispherical fingertip, as well as the intensity (area) of the contact [2]. The sensor uses a PSD (Photo-Sensitive Device) rather than a CCD chip. Although less information about the contact state can be obtained, the analog output of the PSD allows for practical, real-time control. In previous experiments, the sensor was used to record surfaces by touching in discrete locations, and to adjust two-fingered grasps to account for rolling between the fingertips and the object [3,4].
I extended the application of the sensor for surface shape and fine feature detection during sliding contact. Preliminary experiments to verify the feasibility of sensing during sliding contact were performed by dragging the sensor manually over a flat surface. Despite deformation due to friction on the outer rubber skin of the sensor, the recorded contact normal direction was clean and accurate enough to be used in a control law. Two different versions of the optical waveguide sensor were tested: 20mm and 16mm diameter sensors. The two 20mm sensors had very linear response but the sensing area was limited to about 30 degrees from the primary axis on the sensor. The three 16mm sensors had a nonlinear but monotonic response and the sensing area was over 70 degrees. The 16mm sensor was calibrated using two different fourth order polynomial fits for the positive and negative angles of contact, in a single plane through the pole of the sensor. The fit and stiffness of the rubber skin over the smaller sensors was also modified in order to make the sensor output less susceptible to pulling on the skin due to friction during sliding.
A hybrid force/velocity control law was implemented for sliding the finger over the surface of an unknown object. The force is controlled in the direction normal to the surface at the contact point and the velocity is controlled tangential to the surface. Surface normal is determined using the optical waveguide sensor and the geometry of the hemispherical fingertip. In addition, the magnitude and direction of the force is obtained from the joint torque sensors on the robot finger. The Cartesian position is calculated from potentiometers on the joints. Gains for force control were adjusted to provide enough force for the tactile sensor to detect the normal direction of contact, but not so much that friction impaired smooth sliding. Gains for the velocity control were chosen to provide slow, controlled sliding over both smooth object surfaces and ridge features.
During sliding, the contact location in space and the direction of the contact normal are recorded. This information is used in real time to accomplish the hybrid force/velocity control, as well as in post-processing to extract object shape and feature information. From the recorded surface normal, contact intensity, and position/velocity of the fingertip, it is possible to determine the location and size of surface features. The surface feature I chose for this study was a small ridge (0.5-2.0 mm diameter) that was created by stretching electrical wire over a flat surface.
By inspection, the presence of small surface features is not easy to see in a direct plot of the surface shape (x versus y contact location in world coordinates). However, it is easily detected by observing a plot of the surface normal direction over time. Even more compelling is a time derivative of the surface normal, which shows a distinct spike at the location of feature. The size and shape of this spike, along with the time of occurrence, can be used to identify the size of the feature and its location on the surface of the object. Also, a time derivative of the contact intensity reveals the feature location because the contact area changes when sliding over a feature. This set of information could be automatically processed to return the feature's approximate size and location. Such high- level identification would be necessary for robot fingers to explore an object with complete autonomy.
Using contextual information and output from the optical waveguide sensors, detailed haptic exploration of unknown objects can be accomplished. Both global object shape and fine surface features such as small ridges may be measured using the optical waveguide sensors. Future work will include using multiple sensors and sensor fusion to extract detailed information about object surface properties and features. It will also include the development of an object model to efficiently store information obtained during exploration in such a way that the object can be recreated in a virtual environment. In addition, a higher level of intelligence in exploration may be used to explore certain areas of the object in more detail, given intriguing but incomplete data about object features. Thus, a preliminary exploration could serve as a guide for later, more detailed exploration, perhaps incorporating different types of sensors. My thesis will combine previous studies in exploratory procedures, the work accomplished at Japan concerning the recording of object shape and features, and the areas of future work mentioned above.
I'd like to express my appreciation to NSF and Dr. Kazuo Tanie for providing me with the extraordinary opportunity to perform research at the Mechanical Engineering Laboratory. Thanks also to the many students and researchers in the Robotics Division whose hospitality made my stay in Japan very enjoyable.
1. A. Okamura, M. Turner, and M. Cutkosky, "Haptic Exploration of Objects with Rolling and Sliding Contacts," Proceedings of the 1997 IEEE International Conference on Robotics and Automation, Vol. 3, pp. 2485-2490.
2. H. Maekawa, K. Tanie, and K. Komoriya, "A Finger-Shaped Tactile Sensor Using an Optical Waveguide," Proceedings of the 1993 IEEE International Conference on Systems, Man and Cybernetics, pp. 403-408.
3. H. Maekawa, et al., "Development of a Finger-Shaped Tactile Sensor and it's Evaluation by Active Touch," Proceedings of the 1992 IEEE International Conference on Robotics and Automation, pp. 1327, 1334.
4. H. Maekawa, K. Tanie, and K. Komoriya, "Tactile Feedback for Multifingered Dynamic Grasping," IEEE Control Systems, February 1997, pp. 403-408.