Lean Intelligent Assembly Automation (LIAA)
2nd Review – Workplace Configuration
Rasmus Hasle Andersen
Danish Technological InstituteForskerparken 105230 OdenseDenmarkTel.: +45 7220 1864 / +45 7220 2000E-Mail: [email protected]
2LIAA 2nd Review Brussels, 2016-05-11
• Workplace Design:
– Define Bill of Materials
– Define Bill of Processes
– Define Bill of Resources
– Design Workplace Concept
– Design Workplace
– Simulate Workplace
– …
– Export Workplace Design and Documentation
• Workplace Construction (not part of LIAA)
– Purchase/manufacture components
– Integrate components
• Mechanical
• Electrical
• Pneumatic
• Hydraulic
• Workplace Configuration:
– Configure individual modules• Hardware and Software modules
– Configure dependencies between module
• e.g. Hand-eye calibration
– Finalize Skill Network
– Set Skill Parameter
– Configure configuration view
– …
– Start Runtime
• Workplace Runtime:
– Execute Skills
– Track Human
– Visualize Information
– Identify anomaly
– Recover and continue
– …
LIAA Workflow
Workplace DesignWorkplace
ConstructionWorkplace
ConfigurationWorkplaceRuntime
3LIAA 2nd Review Brussels, 2016-05-11
Workplace Configuration– Contributing WPs
LIAA Configuration Workflow
<WP5>„Symbolic Skill
Execution Engine“ by DTI
<WP7>„Situation Assessment
Module“ by DTI
<WP7>„Human Activity
Recognition“ by TECNALIA
<WP5>„Task-to-Skill Generator“
by IPA
<WP3>„Safety Zone
Configuration “ by XX
<WP6>„AR Enhanced
User Interaction” by EON
<WP5>„Generic Vision Framework “
By DTI
<WP5>„Skill Editor”
by DTI
<WP5>„Generic Vision Calibration “
by DTI & Tecnalia
4LIAA 2nd Review Brussels, 2016-05-11
• Module configuration
– Intra module, and
– Inter module
• Process Configuration
– Process instruction
• Validation of process “suggestion” from the Skill Generation Module
– Process adjustment (and final validation)
• Situation Assessment setup
• Initiate run-time
Workflow - Overview
Module Configuration
Inter & intra module
Process Instruction
Skill Generation
Module
Situation Assessment
Configuration
Inspection of process
Runtime initiation
Process Adjustment
5LIAA 2nd Review Brussels, 2016-05-11
Workflow – Mapping Steps to Pilot Case (1/2)
Mapping of Configuration Workflow Steps to Pilot Case
OP
EL
FIS
CH
ER
DD
E
SP
IN
EA
TELN
ET
Step 01: Physical setup and system start
Step 02: Safety Configuration
Step 03: Vision Setup (teach new object) O – X
Step 04: Human Activity Recognition Calibration X X – O X
Step 05: Augmented Reality Calibration – – O O
Step 06: CAD Based Skill configuration – – X –
Step 07: Hand-eye Calibration O – X
– : Not part of the Pilot Case
X : Necessary Step
O : Desired step
: Tested & Validated
6LIAA 2nd Review Brussels, 2016-05-11
Workflow – Mapping Steps to Pilot Case (2/2)
Mapping of Configuration Workflow Steps to Pilot Case
OP
EL
FIS
CH
ER
DD
E
SP
IN
EA
TELN
ET
Step 08: Process inspection and adjustment X X X X
Step 09: Process verification X X X X
Step 10: Situation Assessment setup O O O O
Step 11: Modular GUI Setup X X X
Step 12: Start runtime X X X
– : Not part of the Pilot Case
X : Necessary Step
O : Desired step
: Tested & Validated
All steps are relevant for more than two Pilot Case
7LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Ensure the physical setup is completed
– Ensure the required modules are available
• Prerequisite
– Design and construction completed
• Effect
– Modules are connected (HW) and installed for the framework to use (SW)
Step 01 – Physical Setup and System Start
10LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Instruct the user to do the safety configuration on the Robot Controller
– A PDF is presented to the user detailing how to do the configuration.
• Or AR enhanced presentation
– Safety controller does not have interfaces which are configurable externally
• Prerequisite
– The safety must be specified in the design phase.
• Effect
– The LIAA cell is configured to operate safely in the given environment
Step 02 – Safety Configuration - Beta
12LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Effective and intuitive teaching of new objects using CAD models.
• Prerequisite
– CAD models of desired object must be available
• Effect
– The LIAA system is able to detect and localize the new object type
– Enables users/workers to teach a system new objects
• Benefit/Added value of the LIAA Solution
– Increases the flexibility and reusability of the LIAA framework
– Thoroughly tested and widely used calibration routine automatically included and exploited in the framework
Step 03 – Vision Configuration (teach new object)
14LIAA 2nd Review Brussels, 2016-05-11
Step 04 – Human Activity Recognition (HAR) Calibration
Multi-Kinects calibration
Select used Kinects and calibration marker
ResultsTransformation between Kinects and Robot
Robot-Kinects calibration
Configure Robot and Moveitparameters
Specify dimension of calibration grid Define number of Iterations
16LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Calibrate the virtual and the real world
• Effect
– The real and virtual worlds are aligned
– The AR enhanced Human-Robot Interaction can be used
• Benefit/Added value of the LIAA Solution
– Enables the framework to provide intuitive operator instructions
– Enables process validation though simulation of the process but visualized in the physical world
Step 05 – Augmented Reality Calibration
18LIAA 2nd Review Brussels, 2016-05-11
• Configuration :
– Prepare CAD models
– Select information and define process using interface
• Execution
– CATIA
– ROS
• Results
– XML files for skill parametrization
Step 06 – Cad Based Skill Configuration
19LIAA 2nd Review Brussels, 2016-05-11
• This is a teaser for the upcoming presentation on the Telnet Pilot Case
Step 06 – Cad Based Skill Configuration (video)
20LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Extrinsic calibration between camera and robot
• Prerequisite
– A “robot” module must be connected and configured
– A camera/vision modules must be connected and configured
• Effect
– The robot can directly use the poses originating from the vision system
• Benefit/Added value of the LIAA Solution
– Enables fast reconfiguration of the robot to/from camera transformations
– Performs a fully automated hand-eye calibration using the LIAA skill framework
Step 7 (a) – Hand-eye calibration
22LIAA 2nd Review Brussels, 2016-05-11
Step 7 (b) – Tecnalia Calibration
Manual Configuration (ROS .launch
file):
- Robot
- Vision sensor
- Number of images/poses
- Initial pose of the robot
- Calibration panel properties
Execution:
Results:
- Intrinsics
- Extrinsics
25LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Inspect and verify the process provided by the Skill generation module
– Modify/Adjust if necessary
• Prerequisite
– A skill representing the process must be available
• Effect
– The skill is verified to operate as expected
• Benefit/Added value of the LIAA Solution
– The user can exploit the Skill Generation module to provide a good estimate of the process description.
– Adjustments can be made to tweak the final operation
Step 8 – Process inspection (and adjustment)
27LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Test the entire skill execution to verify the skill execution
• Prerequisite
– A Skill modelling the process must be available
• Effect
– The skill has been verified to satisfy the goal of the process (nominal flow)
– The process configuration is completed
Step 9 – Process Verification
29LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Configure the available sensors
– Record the nominal behaviour of the skill
– Connected sensors
• Prerequisite
– Skill instruction and parametrization must be finished
• Effect
– Nominal behaviour learned and ready for monitoring during execution
– The Process
• Benefit/Added value of the LIAA Solution
– No need for anomaly detection code in skills
– Added robustness
Step 10 – Situation Assessment
31LIAA 2nd Review Brussels, 2016-05-11
• Purpose
– Enables the user to select which “views” to be visible
• Prerequisite
– Individual modules must register “views”
• Benefit..
– Individual modules can provide unique views
– Enables per user and per process customization of the visual interface
Step 11 – Modular GUI Setup
33LIAA 2nd Review Brussels, 2016-05-11
• Runtime activation
• Select main skill
• Click run
Step 12 – Start runtime
34LIAA 2nd Review Brussels, 2016-05-11
• Single point of access for configuration of all modules
• Intuitive configuration of individual modules
• Modular and dynamic configuration
• Simple process instruction, testing and validation
• Intra- and intermodule configuration
• Initiate run-time
• Can work as a stand-alone configuration tool-chain
• Can benefit from the integration with the LIAA design phase
Summary
Module Configuration
Inter & intra module
Process InstructionSkill Generation
Module
Situation Assessment
Configuration
Inspection of process
Runtime initiation
Process Adjustment
36LIAA 2nd Review Brussels, 2016-05-11
• Summarize the next development steps till 02/2017
– Integration of more modules (Individual Partners)
– Improve the configuration framework (DTI)
– Improvement of existing modules (Individual Partners)
– Tighter coupling with the Design tool-chain (DTI and IPA)
– Integration of the LIAA Skill Library (UR)
– GUI Enhancements based on partner feedback (DTI)
– Enhancing the robot control through configuration of robot path planning module(s) (DTI)
– Integrate configuration of (all) modules in the (unified) configuration view(all LIAA partners)
– Runtime visualization
Outlook for M31-M42
37LIAA 2nd Review Brussels, 2016-05-11
Thank you for your
Attention!
Questions?
For more information visit us at www.project-leanautomation.eu
Project CoordinatorMartin NaumannTel: +49 711 970-1291Fax: +49 711 970-1008E-Mail: [email protected]
Project OfficeTel: +49 711 970-1844Fax: +49 711 970-1008E-Mail: [email protected]
Contact Information