Visual servoing

Uses your system voice; voice and rate in settings.

Snapshot

≈ 9 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

Visual servoing, also known as vision-based robot control and abbreviated VS, is a technique which uses feedback information extracted from a vision sensor (visual feedback ) to control the motion of a robot. One of the earliest papers that talks about visual servoing was from the SRI International Labs in 1979.

Visual servoing, also known as vision-based robot control and abbreviated VS, is a technique which uses feedback information extracted from a vision sensor (visual feedback ) to control the motion of a robot. One of the earliest papers that talks about visual servoing was from the SRI International Labs in 1979.

Visual servoing taxonomy

There are two fundamental configurations of the robot end-effector (hand) and the camera: Eye-in-hand, or end-point open-loop control, where the camera is attached to the moving hand and observing the relative position of the target. Eye-to-hand, or end-point closed-loop control, where the camera is fixed in the world and observing the target and the motion of the hand. Visual Servoing control techniques are broadly classified into the following types: Image-based (IBVS) Position/pose-based (PBVS) Hybrid approach IBVS was proposed by Weiss and Sanderson.

The control law is based on the error between current and desired features on the image plane, and does not involve any estimate of the pose of the target. The features may be the coordinates of visual features, lines or moments of regions. IBVS has difficulties with motions very large rotations, which has come to be called camera retreat. PBVS is a model-based technique (with a single camera).

This is because the pose of the object of interest is estimated with respect to the camera and then a command is issued to the robot controller, which in turn controls the robot. In this case the image features are extracted as well, but are additionally used to estimate 3D information (pose of the object in Cartesian space), hence it is servoing in 3D. Hybrid approaches use some combination of the 2D and 3D servoing. There have been a few different approaches to hybrid servoing 2-1/2-D Servoing Motion partition-based Partitioned DOF Based

Survey

The following description of the prior work is divided into 3 parts Survey of existing visual servoing methods. Various features used and their impacts on visual servoing. Error and stability analysis of visual servoing schemes. of the camera and the plane of the features are parallel., discusses an approach of velocity control using the Jacobian relationship s˙ = Jv˙ . In addition the author uses Kalman filtering, assuming that the extracted position of the target have inherent errors (sensor errors).

A model of the target velocity is developed and used as a feed-forward input in the control loop. Also, mentions the importance of looking into kinematic discrepancy, dynamic effects, repeatability, settling time oscillations and lag in response. Corke poses a set of very critical questions on visual servoing and tries to elaborate on their implications. The paper primarily focuses the dynamics of visual servoing. The author tries to address problems like lag and stability, while also talking about feed-forward paths in the control loop.

The paper also, tries to seek justification for trajectory generation, methodology of axis control and development of performance metrics. Chaumette in provides good insight into the two major problems with IBVS. One, servoing to a local minima and second, reaching a Jacobian singularity. The author show that image points alone do not make good features due to the occurrence of singularities.

The paper continues, by discussing the possible additional checks to prevent singularities namely, condition numbers of J_s and Jˆ+_s, to check the null space of ˆ J_s and J^T_s . One main point that the author highlights is the relation between local minima and unrealizable image feature motions. Over the years many hybrid techniques have been developed. These involve computing partial/complete pose from Epipolar Geometry using multiple views or multiple cameras. The values are obtained by direct estimation or through a learning or a statistical scheme.

While others have used a switching approach that changes between image-based and position-based on a Lyapnov function.

The early hybrid techniques that used a combination of image-based and pose-based (2D and 3D information) approaches for servoing required either a full or partial model of the object in order to extract the pose information and used a variety of techniques to extract the motion information from the image. used an affine motion model from the image motion in addition to a rough polyhedral CAD model to extract the object pose with respect to the camera to be able to servo onto the object (on the lines of PBVS).

2-1/2-D visual servoing developed by Malis et al. is a well known technique that breaks down the information required for servoing into an organized fashion which decouples rotations and translations. The papers assume that the desired pose is known a priori. The rotational information is obtained from partial pose estimation, a homography, (essentially 3D information) giving an axis of rotation and the angle (by computing the eigenvalues and eigenvectors of the homography).

The translational information is obtained from the image directly by tracking a set of feature points. The only conditions being that the feature points being tracked never leave the field of view and that a depth estimate be predetermined by some off-line technique. 2-1/2-D servoing has been shown to be more stable than the techniques that preceded it. Another interesting observation with this formulation is that the authors claim that the visual Jacobian will have no singularities during the motions.

The hybrid technique developed by Corke and Hutchinson, popularly called portioned approach partitions the visual (or image) Jacobian into motions (both rotations and translations) relating X and Y axes and motions related to the Z axis. outlines the technique, to break out columns of the visual Jacobian that correspond to the Z axis translation and rotation (namely, the third and sixth columns). The partitioned approach is shown to handle the Chaumette Conundrum discussed in.

This technique requires a good depth estimate in order to function properly. outlines a hybrid approach where the servoing task is split into two, namely main and secondary. The main task is keep the features of interest within the field of view. While the secondary task is to mark a fixation point and use it as a reference to bring the camera to the desired pose. The technique does need a depth estimate from an off-line procedure.

The paper discusses two examples for which depth estimates are obtained from robot odometry and by assuming that all features are on a plane. The secondary task is achieved by using the notion of parallax. The features that are tracked are chosen by an initialization performed on the first frame, which are typically points. carries out a discussion on two aspects of visual servoing, feature modeling and model-based tracking. Primary assumption made is that the 3D model of the object is available.

The authors highlights the notion that ideal features should be chosen such that the DOF of motion can be decoupled by linear relation. The authors also introduce an estimate of the target velocity into the interaction matrix to improve tracking performance. The results are compared to well known servoing techniques even when occlusions occur. m˙ij and the velocity screw (v) is given as m˙_ij = L_m_ij v. This technique avoids camera calibration by assuming that the objects are planar and using a depth estimate.

The technique works well in the planar case but tends to be complicated in the general case. The basic idea is based on the work in [4] Moment Invariants have been used in. The key idea being to find the feature vector that decouples all the DOF of motion. Some observations made were that centralized moments are invariant for 2D translations. A complicated polynomial form is developed for 2D rotations.

The technique follows teaching-by-showing, hence requiring the values of desired depth and area of object (assuming that the plane of camera and object are parallel, and the object is planar). Other parts of the feature vector are invariants R3, R4. The authors claim that occlusions can be handled. and build on the work described in. The major differ- ence being that the authors use a technique similar to, where the task is broken into two (in the case where the features are not parallel to the cam- era plane).

A virtual rotation is performed to bring the featured parallel to the camera plane. consolidates the work done by the authors on image moments.

Software

Matlab toolbox for visual servoing. Java-based visual servoing simulator. ViSP (ViSP states for "Visual Servoing Platform") is a modular software that allows fast development of visual servoing applications.

See also

Robotics Robot Computer Vision Machine Vision Robot control

References

External links

S. A. Hutchinson, G. D. Hager, and P. I. Corke. A tutorial on visual servo control. IEEE Trans. Robot. Automat., 12(5):651—670, Oct. 1996. F. Chaumette, S. Hutchinson. Visual Servo Control, Part I: Basic Approaches. IEEE Robotics and Automation Magazine, 13(4):82-90, December 2006. F. Chaumette, S. Hutchinson. Visual Servo Control, Part II: Advanced Approaches. IEEE Robotics and Automation Magazine, 14(1):109-118, March 2007. Notes from IROS 2004 tutorial on advanced visual servoing.

Springer Handbook of Robotics Chapter 24: Visual Servoing and Visual Tracking (François Chaumette, Seth Hutchinson) UW-Madison, Robotics and Intelligent Systems Lab INRIA Lagadic research group Johns Hopkins University, LIMBS Laboratory University of Siena, SIRSLab Vision & Robotics Group Tohoku University, Intelligent Control Systems Laboratory INRIA Arobas research group LASMEA, Rosace group UIUC, Beckman Institute

Text derived from https://en.wikipedia.org/w/index.php?oldid=1328012649 under CC BY-SA 4.0; modifications have been made.

Sources cited in this revision

6 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.