Upload
emadrid-network
View
761
Download
0
Tags:
Embed Size (px)
DESCRIPTION
2011 06 01 (uned) emadrid aelsaddik uottawa tangible objects for e learning
Citation preview
Tangible Objects for E-Learning
Abdulmotaleb El Saddik
FIEEE, FCAE, FEIC
At a glanceCreation of Advanced Multimedia LO
Protection of LO
Navigation of LOR by VR metaphors
Adaptation of LO according to user context/profile & network QoS
LO Delivery to user
Introduction
Haptics Derived from the Greek verb “haptesthai” Refers to sensing and manipulation through touch Comprises both “Tactile” and “Kinesthetic” senses
Applications Education and Training Medicine: Tele-surgery, rehabilitation, etc. Entertainment Arts and Design …
Multimedia and Authoring
Multimedia contents Graphic images 3D models Audio and video files And recently haptic stimuli
Multimedia authoring tools Integrate the disparate media elements into a
cohesive multimedia application
Multimedia Authoring Tools
A major challenges in multimedia authoring is to make authoring complex multimedia titles as easy as using a word processor or drawing program
We propose a multimedia authoring tool for hapto-visual applications to make the development process a non-programming experience
HAMLAT
Blender: Open source Full-fledged 3D graphical renderer Physics and game engine Adaptive user-interface
Plug-ins incorporated in Blender source code Haptic panel in Blender GUI Haptic rendering extension Implemented in C++ using OpenHaptics and OpenGL
HAMLAT Implementation (demo)
A Multimedia Handwriting Learning and Evaluation Tool
Learning sensorimotor skills is difficult with graphical or auditory feedback
Sense of touch will be a must for physical guidance
Examples of physical guidance learning: Handwriting for visually impaired people Medical procedures and rehabilitation training Painting/sculpting techniques
Multimedia Learning Tool
Distinguished features Language choice: Arabic, French, English, Japanese
and Spanish
Multimodal: haptic, visual, and audio information for
each letter
Learning mode choice: full or partial guidance, or test
mode.
Evaluation Mode: quantitative evaluation using dynamic
time warping
Multimedia Learning Tool
Learning Mode -1-
Language Repository Including symbols images,
pronunciation files, and the
haptic stimuli of each letter
(using XML-based schema)
<?xml version="1.0" encoding="UTF-8" ?> <Language Name="Japanese_Katakana" Flow="left" xmlns="urn:language">
<MetaData> <AuthorName> Steve </AuthorName > <ContactInfo> [email protected]
</ContactInfo> <Copywrite> LGPL </Copywrite> <CreationDate> 06-05-2007
</CreationDate> </MetaData> <HapticDevice>
<DeviceName> Phantom </DeviceName> <Workspace> 160X120X0 </DeviceName> <MaxForce> 3.0 </MaxForce> <DOF> 3 </DOF>
</HapticDevice> <Alphabets>
... <Item Name="ka" /> <Item Name="ki" /> <Item Name="ku" /> <Item Name="ke" /> <Item Name="ko" /> <Item Name="sa" /> ...
</Alphabets> </Language>
Learning Mode -2-
Haptic player: Three modes of
operation: No guidance
Partial guidance
Full guidance
Evaluation Mode -1-
Test GUI
Evaluation Mode -2-
Dynamic Time Warping Measure the similarity between
test and reference handwritings
Tolerate both global and local shifts in the time domain
Startup position of the test pattern is shifted to that of the reference pattern
Given two time series// xi is a point in the test trajectoryX = x0, x1, …, xN-1 // yi is a point in the ref. trajectoryY = y0, y1, …, yT-1
D(xi,yi) = sqrt((abs(xi)-abs(yi))^2 + (ord(xi)-ord(yi))^2)DTW[0,0] = D(x0,y0)
For i=1 to T-1 DTW[0,i] = D(x0,yi) + DTW[0,i-1]For i=1 to N-1 DTW[i,0] = D(xi,y0) + DTW[i-1,0]
For i=1 to N-1 For j=1 to T-1DTW[i,j]=D(xi,yj)+min{DTW[i,j-1],DTW[i-1,j],DTW[i-1,j-1]}
Return (DTW[N-1,T-1])
Performance Evaluation -1-
Experimental Setup The Phantom Omni Device Application is developed with .NET 2.0 Framework and
programmed using C#. Bipolar grading scheme (Pass/fail) Threshold distance is computed based on subjective
analysis Three threshold values were estimated:
Difficult Medium Easy
Performance Evaluation -2-
The DTW distance for the “Tha” Arabic character
Performance Evaluation -3-
The DTW distance for the “i” Japanese character
Performance Evaluation -4-(Demo)
The DTW distances for five Japanese characters
CharacterThreshold Distance (x103 mm)
Easy Medium Difficult
Sa 5.5 2.5 1
Shi 4 1 0.5
Chi 5 2.5 1
Tsu 4 1.5 1
Na 10 6 1
At a glanceCreation of Advanced Multimedia LO
Protection of LO
Navigation of LOR by VR metaphors
Adaptation of LO according to user context/profile & network QoS
LO Delivery to user
Haptic signal prediction, compression, and watermarking
Haptic data stream
Compressed bitstream
Haptic Compression
Encoder
Adaptive Golomb-
Rice Coding
Code Table
Run Counter
Run-length Coder
Bitstream Formatting
Entropy Coding
8-samples Blocks
along x, y and z
F-DCT
Uniform Scalar
Deadzone Quantization
Simple Differential Predictive
Coding
Haptic Data Acquisition and Pre-
processing
Watermark haptic signals to protect copyright and authenticate content.
Case Study: Identifying Human Pattern with Haptics
The Experiment
Methodology
Dynamic Time Warping :
+ Nelder-Mead non-linear minimization
Spectral analysis: Fast Fourier Transform
Unsupervised Method: K-Means
3
1c
N
1i
2p2c,i
1c,i tldMS
Graphic Representation
Trial Timestamp Position X Position Y … Force X
( N)
1 0 0.23344 0.56768 0.00456
1 … .. … …
1 0.0123090 0.37676 0.98976 0,03767 Data
Verifying such feasibility (demo)
Virtual Check
Virtual MobilePhone
Performance of Classifier
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
FAR error rate
Virtual Check Maze Virtual Phone
Ambient Intelligent Engine
Performance
At a glanceCreation of Advanced Multimedia LO
Protection of LO by Digital Watermarking
Navigation of LOR by VR metaphors
Adaptation of LO according to user context/profile & network QoS
LO Delivery to user
Navigation
Tangible Objects: Demo1
Haptics: Demo 2
Mobile Phone: Demo 3
At a glanceCreation of Advanced Multimedia LO
Protection of LO by Digital Watermarking
Navigation of LOR by VR metaphors
Adaptation of LO according to user context/profile & network QoS
LO Delivery to user
E-Learning courses
Toddler Interfaces
Toddler Interfaces Main idea
Whenever a child utters an object’s name or taps on an object, a set of appropriate multimedia representations is displayed (i.e. images, Multi-lingual texts, audio…)
Techniques investigated Voice-based technique (Speech detection) Magic Stick technique (RFID and Bluetooth technologies)
Evaluation Several games were developed: i.e., Book game, Alphabet Learning, and
Object Identification
Alphabet Learning Evaluation
12 children, ages 4-5, evaluated the system
We Observed the following:
Children’s discussions
Their Writings Their Questions Simplicity of
using the Magic Stick.
Demo 1
Demo 2
At a glanceCreation of Advanced Multimedia LO
Protection of LO by Digital Watermarking
Navigation of LOR by VR metaphors
Adaptation of LO according to user context/profile & network QoS
LO Delivery to user
Clustering and 3D Mapping
a) MST tree b) Graph construction
T
CT1
CT3
CT2
CT4
CT5
T
CT1 CT2
CT5CT4CT3Semantic keyword metrics MST tree
Yahoo, Google Web-services
P2PDirectory
Remote Repository
Keyword terms1. Query Parsing and Search
2. Analyze each of the result links
Xml results3. Determine the high ranked keywords
4. Commit search with first n high ranked keywords
PFs
5. Calculation of the relevance between the high ranked keywords
6. Grouped search result Xml results
Overall clustering process design
Advanced search options
Sharing LO paramers
Searching/Sharing Learning Objects
Group interaction menu. Comprising options viz:a) Search related b) Save allc) Share alld) Make parent
Group sign
Dynamic road for each group of search results
Dynamic navigation layout
Sample Interface: Dynamic Traffic Layout
Multimodal Interaction (demo)
Search Result Clustering:A) KeywordB) User tagC) Term frequency
Remote Repository
Web-Search: Yahoo/Google
P2P DirectoryList
[ title ]
(a)
(b)
(c)
(e)
(d)
Multi-modal Interactions
Visualization metaphors Information organization Information repository
[ title ]
At a glance
Creation of Advanced Multimedia LO
Protection of LO by Digital Watermarking
Navigation of LOR by VR metaphors
Adaptation of LO according to user context/profile & network QoS
LO Delivery to user
Last thoughts
Acknowledgment
October 4, 2005 متشکرم