Intuitive Path Generation Platform for Multiple Robots applicable toNew Media Contents

Beomyeong Park1, Jimin Lee1, Sangyup Yi2, Eungui Lee3 and Jaeheung Park1

Abstract— Industrial robots have been recently used forapplications in various other fields. However industrial robotplatforms are difficult to be used for an operator to create adetailed path for the robot arm to move from a starting positionto a target position. Furthermore, since each platform can onlyoperate robots of the same company, multiple platforms areneeded when using robots of different companies at the sametime. The suggested platform allows the operator to create robotarm path intuitively and control different types of robots. Totest the proposed system, paths for multiple robots were setthrough the UI, and the path following performance of thisplatform was verified via simulation.

I. INTRODUCTION

Industrial robots have been used in manufacturing sincethe early 1960s; they have nowadays become commonplacein various industries such as the semiconductor industry,the electric device industry, and the automobile industry.In the industrial workplaces, robots must be operated inareas away from people to ensure safety. However, human-robot interaction has been studied extensively to create waysfor people and robots to work together safely in the sameworkspace[1][2][3]. Owing to the progress in human-robotinteraction, robot arms have also been used for variousapplications such as surgery, filming movies, hotel service,cooking, and amusement park rides[4][5][6].

Recently, one application of interest involves using robotarms to execute special effects in filming to create new digitalmedia content using combinations of media contents androbot technology. When robot arm is used with camera fornew media contents, the trajectory of the robot arm mustbe generated so that it can pass through various positions atvarious speeds. Previous research proposed a platform thatgenerates the path of an industrial robotic arm and guar-antees real-time control[7]. The proposed platform providedcommunication that can control KUKA robot in real timeand provided a UI that can generate robot arm trajectory. Amethod of generating the entire path of the robot arm bysetting an intermediate point has been proposed.

*This work was funded by the Seoul Metropolitan Govern-ment (No.Cl152050) and the National Research Foundation of Ko-rea (NRF) grant funded by the Korea government (MSIP) (No. NRF-2015R1A2A1A10055798)

1Beomyeong Park, Jimin Lee, and Jaeheung Park are with GraduateSchool of Convergence Science and Technology, Seoul National University,Suwon, Republic of Korea, Jaeheung Park is a corresponding author of thispaper. (on2lord, jmpechem, park73)@snu.ac.kr

2Sangyup Yi is Wonik Robotics, Seongnam-si, Gyunggi-do, Koreaseanyi@wonikrobotics.com

3Eungui Lee is with Sangwha Co., Seoul, Koreaeungiu@sangwha.com

However, the previously proposed platform was designedto control only one industrial robot platform. Therefore,multiple platforms were required to control multiple robotssimultaneously. This limitation reduces the work efficiencyof the operator in the situation that the operator uses mul-tiple platforms to control multiple robot. In addition, it isimportant to generate paths that can guarantee the positionalaccuracy and timing of multiple robot movements, especiallywhen the end effectors of the robot arms meet at a specificlocation at an exact time.

In order to solve these problems, this paper suggests theplatform that is designed to generate multiple trajectories andto operate multiple robots. This platform supports the KUKArobot arm KR-120 and KR-16 and the Hanwha Techwinrobot arm HCR-5; linear tracks can also be operated andused in conjunction with a robot arm. In order to generatesynchronized trajectories of multiple robots, a drag and dropfunction using a mouse was added to the UI to designatethe middle point of the robot arm path. In addition, the pathspeed of the robot arm can be controlled using key framesto set the elapsed time between each path point.

The remainder of this paper is structured as follows.Section II explains the structure of the proposed platformincluding UI, inverse kinematics(IK) engine, and Bridgesoftware(SW). In Section III, paths for multiple robots arecreated using the platform and simulated. Section IV con-cludes this paper.

II. PLATFORM STRUCTURE

Figure 1 shows the structure of the proposed platform.The platform includes a UI tool, the robot and the Bridgesoftware.

A. User Interface Tool

Unity3D, which was used to create the UI tool, is a3D game physics engine and an integrated developmentenvironment (IDE) developed by Unity Technologies as anopen source platform[8]. The UI tool was designed to providethe operator with a convenient way to visually constructpaths for the robot arms. In addition, time-based keyframesare displayed to allow the operator to create paths for therobot arm using a mouse without requiring any professionalprogramming skills. The generated path for the robot arm issimulated by using the robot model selected in the UI.

The second and fourth windows of UI part in Figure 1show that the operator creates a path for the robot arm usingthe drag and drop method. The green box added to the secondwindow is moved by the drag and drop method to create

Fig. 1: Platform structure: UI tool(1. Selecting robot interface 2. Monitoring robot and generating robot path, 3. Setting andchecking the robot state and 4. Key frame), Bridge Software and Robot.

(a) (b)

Fig. 2: The joint coordinate information of available robotswith proposed platform : (a) KUKA Robot (b) HCR-5 Robot

the intermediate point. The white dot on the fourth windowindicates when the robot arm passes the intermediate point.The speed for the robot to move between two points can beadjusted by dragging and dropping the point on the keyframeto the left or right.

B. Inverse kinematics engine

The IK engine can be performed for the KUKA robot arm,the Hanwha HCR-5 robot, and a linear track which can beused in combination with each model.

KR-16 and KR-120 robot have the same link structure,therefore, one IK analysis can be used for both robotarms[9][10][11]. The inverse kinematics can be divided intoposition IK and orientation IK based on the wrist coordinatewhere axes 4, 5, and 6 intersect. Through the position IK,we can obtain solutions for four robot poses. Through theorientation IK, two solutions for direction control can beobtained and eight inverse kinematic solutions are calculated.In order for the robot arm to move continuously, the solutionis selected so that the robot arm joints minimally move[12].

The structure of the HCR-5 robot shown in Fig. 3 isdifferent from the KUKA robot and similar to the UR-5of the Universal Robot. Therefore, the inverse kinematics

Fig. 3: Two cases of robot arm and linear track moving

analysis of the HCR-5 robot is analyzed in the same way asUR-5 and has eight inverse kinematics solutions[13].

The linear track is not used as a single unit, but is used inconjunction with robot arm. The combined robot becomes aredundancy system with 7 DOF. In this paper, we proposesa method to utilize the inverse kinematics which is alreadyanalyzed for the robot arm.

As shown in Fig 3, we propose a method to intuitivelycreate a path by the input distance of the linear track(α). Ifthe input distance of the linear track is zero(α = 0), onlythe robot arm moves, or if the input distance is equal tothe entire movement distance(α = l), only the linear trackmoves. In other cases (0 < α < l), the linear track willmove by the input distance, and the robot arm will movethe remaining distance from the total travel distance. Theadvantage of using this method is that the existing inversekinematics analysis for the KUKA robot and HCR-5 robotcan be used as it is.

C. Bridge Software

Bridge SW facilitates the transfer of data between the UItool and industrial robot controller, thereby enabling real time

Fig. 4: Data transmission and reception between UI tool androbot through the bridge SW

control. The proposed platform uses different communicationto connect with each robot controller used by KUKA robotand Hanhwa robot. Figure 4 illustrates how the Bride SWcommunicates with the UI tool and robots.

The Bridge SW acts as a hub for sending and receivingdata between the UI tool and the robot. The UI tool transmitsthe joint position for the robot that was derived from theinverse kinematics to the Bridge SW, then the bridge SWsends the joint position data to the robot. In the other way theBridge SW receives information about the state of the robot(ready, operation in progress, completion) from the robot andtransmits the state to the UI tool.

III. EXPERIMENT

Figure 5 shows a simulation in which two KUKA robotsand linear track follow paths created by the operator using theUI. One of the two KUKA robot moves in conjunction withthe linear track. The white dot in the key frame at the bottomof Figure 5 denotes the intermediate points that the threerobots pass through. The difference between the intermediatepoints of each robot indicate that each robot moves at adifferent speed. The path shown in Figure 5 was generatedby the operator. In Figure 5 (a), two robot arms begin to movefrom starting point. Figure 5 (b) and (c) show that two robotarms pass intermediate points and two robot arms meet at thefinal point of the paths in Figure 5 (d). While both roboticarms follow the paths, the linear track on the right movesalong the input path. Creating a path in the UI and simulatingthe robot’s movement can be seen in the video[14].

IV. CONCLUSION

In this paper, we developed a platform that can operatemultiple robots simultaneously and include a UI that allowsan operator to intuitively create robot arm paths. The pro-posed platform can be linked with KUKA robots KR-16,KR-120, linear track and Hanwha Robot HCR-5.

The UI of the platform displays a robot model andprovides the user with an intuitive way to generate thepaths of robot arms. The Bridge SW sends and receivesdata using the appropriate communication method for eachrobot. The information that the Bridge SW receives from therobot is transmitted to the UI so that it can be observed bythe operator. In this paper, the proposed platform was usedto operate two KUKA robots and a linear track, generateappropriate paths, and simulate the operation of the robots.

(a) (b)

(c) (d)

Fig. 5: Simulation in which three robots follow the generatedpaths by using the proposed platform

In our future study, we aim to conduct experimentalvalidations with actual robots by creating paths for multiplerobots to move and interact.

REFERENCES

[1] M. A. Goodrich and A. C. Schultz, “Human-robot interaction: a sur-vey,” Foundations and trends in human-computer interaction, vol. 1,no. 3, pp. 203–275, 2007.

[2] R. Alami, A. Clodic, V. Montreuil, E. A. Sisbot, and R. Chatila, “Taskplanning for human-robot interaction,” in Proceedings of the 2005joint conference on Smart objects and ambient intelligence: innovativecontext-aware services: usages and technologies. ACM, 2005, pp.81–85.

[3] J. Mainprice and D. Berenson, “Human-robot collaborative manipula-tion planning using early prediction of human motion,” in IntelligentRobots and Systems (IROS), 2013 IEEE/RSJ International Conferenceon. IEEE, 2013, pp. 299–306.

[4] P. Chua, T. Ilschner, and D. Caldwell, “Robotic manipulation offood products–a review,” Industrial Robot: An International Journal,vol. 30, no. 4, pp. 345–354, 2003.

[5] M. Schwartz and J. Prasad, “Robosculpt,” in Rob— Arch 2012.Springer, 2013, pp. 230–237.

[6] J. Jung, J. Kim, S. Kim, W. Y. Kwon, S. H. Na, K. S. Kim, G. J.Suh, B. W. Yoo, J. W. Choi, J. C. Lee et al., “Application ofrobot manipulator for cardiopulmonary resuscitation,” in InternationalSymposium on Experimental Robotics. Springer, 2016, pp. 266–274.

[7] B. Park, J. Lee, K. Park, S. Lee, E. Lee, and J. Park, “New mediacontent platform using 6- dof industrial robot and 3d game engine,”Journal of Korea Robotics Society, vol. 12, no. 3, pp. 306–312, 2017.

[8] U. Techonologies, “Unity3d,” http://unity3d.com/, [Online; accessed26-Jan-2018].

[9] F. Chapelle and P. Bidaud, “Closed form solutions for inverse kine-matics approximation of general 6r manipulators,” Mechanism andmachine theory, vol. 39, no. 3, pp. 323–338, 2004.

[10] M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot modeling andcontrol. Wiley New York, 2006, vol. 3.

[11] M. Dahari and J.-D. Tan, “Forward and inverse kinematics model forrobotic welding process using kr-16ks kuka robot,” in Modeling, Sim-ulation and Applied Optimization (ICMSAO), 2011 4th InternationalConference on. IEEE, 2011, pp. 1–6.

[12] B. Park and J. Park, “Selection method of inverse kinematic solutionfor continuous robot arm motion,” in The 13th Korea Robotics SocietyAnnual Conference, 2018, pp. 596–597.

[13] K. P. Hawkins, “Analytic inverse kinematics for the universal robotsur-5/ur-10 arms,” Georgia Institute of Technology, Tech. Rep., 2013.

[14] “Experiment video,” https://youtu.be/TaJ6PUG3KMI/, [Online; ac-cessed 31-Jan-2018].