The Technology Enhanced Learning Living Lab for Manufacturing Environments.
Fridolin Wild1), Peter Scott1), Jaakko Karjalainen2), Kaj Helin2), Erik Isaksson3), Ambjoern Naeve3), Maurizio Megliola4), Gianluigi Di Vito4)
1) The Open University, UK 2) VTT, Finland, 3) Royal Institute of Technology (KTH), Sweden, 4) Piksel, Italy
ARgh! - kinesthetic learning.Augmented World Expo, 27.-29.5., Santa Clara/CA
2
Agenda
• Context ‘Manufacturing’• Process Model (‘Mix, Match, Optimize’)• Activity Model• Workplace Model• xAPI Tracking • (Analytics with cRunch)
Context ‘Manufacturing’
Workforce in EU: 225.6 M total employmentManufacturing EU: > 28.4 M1) employees
> 12.6 %1)
of which in SMEs: 59.9 %2)
Source: Eurostat, NACE R2 (2009), LFSI EMP A (2009)
1) Based on incomplete data, actual numbers higher: e.g. France missing.2) Based on incomplete data.
3
4
Workplace != Workplace
(pictures: d7.1, study: d3.3)
5
Job Performance Aids
- Independent, self-regulated learners- As individuals or in collaboration- Engage in problem solving and devise
new applications (‘prototyping’)
- Extreme levels of memorization training and rehearsal (‘training to memory’)
- Prepared to perform in challenging circumstances
- Scaffold people in carrying out an unfamiliar task Adaptivity
Adaptability
consumingdo-torials
creating do-torials
BLUE COLLAR WORKER EXPERIENCE
Enquire
Mix
Match
Optimise
?
Traces
Need, Problem
ActivityXML incl.
Constraints
Report, Analytics
Suggestion, Recommendation
Classifiable? Known? Unknown?
• Navigational positioning in taxonomy
• Discovery support
• Selection of existing mixes with ranked search
• Authoring of new or modified mixes
• Personalised suggestions for improvement of mix / experience
tracking
Constraints:• e.g. returned tool 15• e.g. watched video A• e.g. 14/15 in PT session• e.g. 0 FOD problems• e.g. energy < daily limit
activities:• e.g. job cards• e.g. tasks• e.g. learning paths
• Queries• (Reasoning)
Learning by Experience
(Wild et al., to appear; Wild et al., 2013)
BLUE COLLAR WORKER EXPERIENCE
Enquire
Mix
Match
Optimise
?
Traces
Need, Problem
ActivityXML incl.
Constraints
Report, Analytics
Suggestion, Recommendation
Classifiable? Known? Unknown?
• Navigational positioning in taxonomy
• Discovery support
• Selection of existing mixes with ranked search
• Authoring of new or modified mixes
• Personalised suggestions for improvement of mix / experience
tracking
Constraints:• e.g. returned tool 15• e.g. watched video A• e.g. 14/15 in PT session• e.g. 0 FOD problems• e.g. energy < daily limit
activities:• e.g. job cards• e.g. tasks• e.g. learning paths
• Queries• (Reasoning)
Learning by Experience
(Wild et al., to appear; Wild et al., 2013)
Sourcing: help to find or devise next-generation training solutions
for blue collar workers in manufacturing
Analytics: allow comparative analysis on such measures as time-to-
competence, long-term recall, maximum productivity achievable, time
to spot and remedy mistakes, …
Transformate: help reduce or eliminate effort needed to upgrade training solutions
8
Scenarios: AR goes HTML5!
explicit tacit
Problem
Summer /Winter cycle
Functional fabrics
Set up of the weaving mill.Each topic may hav.)
Prevent potential defects before they happen.Failed step: waste of
resources.
SB: Improve manufacturing process with attribute changes
The situation: as-is
PROFESSIONAL QUALIFICATION(e.g. courses, titles)
MATERIALS (e.g. yarn 76/2 710)
MACHINE
to-be: do-torial (1)
Each topic may have several alternative learning contents attached (objects, actions, choices...)
DornierMTV38
Yarn76/2 710
Machine parts Materials
OCCUPATIONAL HEALTH AND SAFETY
3D MODEL
Schlagbaum
SET UP PARAMETERS
12
ARgh! Component Architecture
13
ARgh! UI Model
TELL ME project3 May 2023
‘Traditional’ AR layero markerso 3D objectso Tied to object
PAINT GUN USE
PROFESSIONAL QUALIFICATION(e.g. courses, titles)
FURNITURE MODEL, 3D MODEL
MATERIALS (e.g. Pinewood)
do-torial
SOLVENTS, CATALYSTS & VARNISHES
PAINTS
Each topic may have several alternative learning contents attached (objects, actions, choices...)
Tinting and Varnishing (all spraying benefit from same technology)
OCCUPATIONAL HEALTH AND SAFETY
LOCATION
15
Do-torial demo: Varnishing for tropical climate
Interoperability
is a property that emerges, when distinctive information systems (subsystems)cooperatively exchange data
in such a way that they facilitate the successful accomplishment of an overarching task.
17
The Activity Model
18
The Activity Model
“find the spray gun nozzle size 13”
Messaging in the real-time presence channel and tracking to xAPI
Chaining of actions; modeling of ‘reactions’: onEnter / onTrigger
Styling (cascading) of viewports and UI elements
Constraint modeling:specify validation conditions and model workflow branching
e.g. smart player;e.g. request widget
19
Serialized Activity Model<activity id="fodw109” name="do-torial: learn about …” start="SprayGun”> <action id="SprayGun” viewport="actions" type="findtool” location="SprayBooth"> <message type=“trace" channel=“lrs">user %u% searches for tool spraygun</message> <summons type="onClick" removeSelf="true"> <toggle viewport="actions" type="action">video15</toggle> <toggle viewport="objects" type="object">SprayGun</toggle> <message type=“exit" channel="rpc">user %u% interacted spraygun</message> </summons> <constraint channel="rpc">user %u% interacted spraygun</constraint> <instruction><![CDATA[<h1>Find the Spray Gun</h1><p>bla…</p>]]></instruction> </action>
<action id="video15" viewport="actions” type="widget” location="SprayBooth"> <widget type="smartplayer" id="15” /> <message type="invoke" channel="rpc">user %u% accesses video15</message> <summons type="onClick" removeSelf="true"> <toggle viewport="actions" type="action">microass1</toggle> <message type="exit" channel="rpc">user %u% watched video 15</message> </summons> <constraint channel="rpc">user %u% watched video15</constraint> <instruction><![CDATA[<h1>Well done!</h1><p>…!</p> ]]></instruction> </action></activity>
20
The Workplace Model
21
The Workplace Model
The ‘tangibles’:Specific persons, places, things
The ‘configurables’:devices (styling),apps+widgets
The ‘triggers’:Markers trigger Overlays; Overlays trigger human action
22
Unified Reference Space: Workplace<?xml version="1.0" encoding="utf-8"?><Workplace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><Resources> <Tangibles>
<Place Name="SprayBooth" Category="Place" MarkerID="007"> <YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” /> <Labels IsEnabled="false">
<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” /> </Labels> </Place>
<Thing Name="SprayGun" Category="Tool" MarkerID="011" Thumbnail=“pistol-blue-small.png"><YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” /><Labels IsEnabled="false">
<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” /></Labels>
</Thing>
<Person Name="Jake" Category="Person" MarkerID="012"><YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” /><Labels IsEnabled="false">
<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” /></Labels>
</Person> <Tangibles> </Resources></Workplace>
23
Tracking with the Experience API
24
Analytics
plot(net, usearrows = TRUE, usecurve = T)
Technology Enhanced Learning Living Lab for Manufacturing Environments
25