Vision Based Robot Manipulator Control in 3D Space Using Fuzzy System and Neural Modeling

Author

Yazd, Safaeieh, Yazd university, Computer Engineering Department

Abstract

Visual servoing system controls a robot by visual feedback so that robot moves from any arbitrary start position to the target positions. The coordinates of points in three dimensions is needed in 3D space. In this paper, a Kinect camera is used to collect RGB images as well as workspace’s depth matrix. The control law is obtained using Jacobian matrix. Since, the mathematical model of robot and workspace, is unknown, artificial neural networks is applied to approximate inverse of Jacobian matrix by gathering data. The approximated neural models are used in control law directly. For each degree of freedom of the robot manipulator, a two-layer feedforward neural network is considered. The distance between end-effector and target in 3D space, and the shoulder joint coordinates are inputs of each of the networks and outputs are the fraction of the related robot joint changes to the image features changes (the elements of inverse of Jacobian matrix). The proposed method has been implemented on an industrial robot manipulator. The experimental results show that the proposed control system can move the end-effector to different target positions in workspace with good accuracy and fewer steps in comparision with the previous method.

Highlights

 

Keywords

References

[1]     H. Wang, M. Jiang, W. Chen, and Y. Liu, “Visual servoing of robots with uncalibrated robot and camera parameters,” Mechatronics, vol. 22, no. 6, pp. 661–668, 2012.
[2]     S. Hutchinson and F. Chaumette, “Visual Servo Control,” IEEE Robotics and Automation Magazine, vol. 4, no. 1, pp. 82–90, 2006.
[3]     M. Mitic, M. Lazarevic, B. Babic, and Z. Miljkovic, “Neural network Reinforcement Learning for visual control of robot manipulators,” Expert Syst. with Appl., vol. 40, no. 5, pp. 1721–1736, 2013.
[4]     H. Wang, Y. Liu, and W. Chen, “Visual tracking of robots in uncalibrated environments,” Mechatronics, vol. 22, no. 4, pp. 390–397, 2012.
[5]     P. P. Kumar and L. Behera, “Visual servoing of redundant manipulator with Jacobian matrix estimation using self-organizing map,” Rob. Auton. Syst., vol. 58, no. 8, pp. 978–990, 2010.
[6]     D. Kragic, H. I. Christensen, and others, “Survey on visual servoing for manipulation,” Comput. Vis. Act. Percept. Lab. Fiskartorpsv, vol. 15, no. 1, 2002.
[7]     S. Hutchinson, G. D. Hager, and P. Corke, “A Tutorial on Visual Servo Control,” IEEE Trans. Robot. Autom., vol. 12, no. 6, pp. 651–670, 1996.
[8]     F. Janabi-Sharifi, L. Deng, and W. J. Wilson, “Comparison of Basic Visual Servoing Methods,” Mechatronics, IEEE/ASME Trans., vol. 16, no. 5, pp. 967–983, Oct. 2011.
[9]     G. Allibert and E. Courtial, “Predictive Control for Constrained Image-Based Visual Servoing,” IEEE Trans. Robot., vol. 26, no. 5, pp. 933–939, 2010.
[10]   M. Sadeghzadeh, D. Calvert, and H. A. Abdullah, “Self-Learning Visual Servoing of Robot Manipulator Using Explanation-Based Fuzzy Neural Networks and Q-Learning,” J. Intell. Robot. Syst., vol. 78, no. 1, pp. 83–104, 2015.
[11]   I. Siradjuddin, L. Behera, T. M. McGinnity, and S. Coleman, “Image Based Visual Servoing of a 7 DOF robot manipulator using a distributed fuzzy proportional controller,” in IEEE International Conference on Fuzzy Systems, pp. 1–8, 2010.
[12]   I. H. Suh and T. W. Kim, “Fuzzy membership function based neural networks with applications to the visual servoing of robot manipulators,” IEEE Trans. Fuzzy Syst., vol. 2, no. 3, pp. 203–220, 1994.
[13]   Y. Takahashi, M. Takeda, and M. Asada, “Continuous Valued Q-learning for Vision-Guided Behavior Acquisition,” in International Conference on Multisenso Fusion and Integration for Intelligent Systems, no. 1, pp. 255–260, 1999.
[14]   M. P. Deisenroth, C. E. Rasmussen, and D. Fox, “Learning to control a low-cost manipulator using data-efficient reinforcement learning,” in International Conference on Robotics, Science & Systems, pp. 57–64, 2011.
[15]   D. I. Kosmopoulos, “Robust Jacobian matrix estimation for image-based visual servoing,” Robotics and Computer Integrated Manufacturing, vol. 27, no. 1, pp. 82–87, 2011.
[16]   F. Chaumette and S. Hutchinson, “Visual Servo Control,” IEEE Robotics and Automation Magazine, vol. 14, no. 1, pp. 109–118, 2007.
[17]   J. Qian and J. Su, “Online estimation of image jacobian matrix by kalman-bucy filter for uncalibrated stereo vision feedback,” in IEEE International Conference on Robotics and Automation, vol. 1, no. 1, pp. 562–567, 2002.
[18]   K. Stanley, Q. M. J. Wu, A. Jerbi, and W. A. Gruver, “Neural network-based vision guided robotics,” in IEEE International Conference on Robotics and Automation, pp. 281–286, 1999.
[19]   J. Pei, S. X. Yang, and G. S. Mittal, “Vision Based Robot Control Using Position Specific Artificial Neural Network,” in 2010 International Conference on Computational Intelligence and Communication Networks (CICN), pp. 110–115, 2010.
[20]   C. C. Cheah, C. Liu, and J. J. E. Slotine, “Adaptive Jacobian vision based control for robots with uncertain depth information,” Automatica, vol. 46, no. 7, pp. 1228–1233, 2010.
[21]   F. Nadi, V. Derhami, and M. Rezaeian, “Vision Based Robot Manipulator Control with Neural Modeling of Jacobian Matrix.” , vol. 8, no. 2, pp. 11–20, 2014.
[22]   “LJ Group,” http://www.ljgroup.com. access: june 2014.
[23]   R. Gascon B and M. Barraza R, “Six DOF Stereoscopic Eye-in-Hand Visual Servo System BIBOT,” in Robotics Symposium and Latin American Robotics Symposium (SBR-LARS), 2012 Brazilian, pp. 284–289, 2012.
[24]   C. Teuliere and E. Marchand, “Direct 3D servoing using dense depth maps,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1741–1746, 2012.
[25]   G. J. Garcia and F. Torres, “Guidance of Robot Arms using Depth Data from RGB-D Camera,” in Proceedings of the 10th International Conference on Informatics in Control, Automation and Robotics, pp. 315–321, 2013.
[26]   J. Stowers, M. Hayes, and A. Bainbridge-Smith, “Altitude control of a quadrotor helicopter using depth map from Microsoft Kinect sensor,” in IEEE International Conference on Mechatronics (ICM), pp. 358–362, 2011.
[27]   I. Siradjuddin, L. Behera, T. M. McGinnity, and S. Coleman, “A position based visual tracking system for a 7 DOF robot manipulator using a Kinect camera,” in The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–7, 2012.
[28]   T. Leyvand, C. Meekhof, Y.-C. Wei, J. Sun, and B. Guo, “Kinect Identity: Technology and Experience,” Computer (Long. Beach. Calif)., vol. 44, no. 4, pp. 94–96, Apr. 2011.
[29]   L. X. Wang, A Course in Fuzzy Systems and Control. Prentice Hall PTR, 1997.
[30]   M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot modeling and control. John Wiley & Sons New York, 2006.
[31]   “Laboratory of Computational Intelligence and Robot,” https://pws.yazd.ac.ir/lcir/?page_id=659
Journal of Soft Computing and Information Technology
Volume 5, Issue 4 - Serial Number 4
October 2017
Pages 76-85

Files

Share

How to cite

Statistics