View
3
Download
0
Category
Preview:
Citation preview
A Virtual Multimedia Office
Jan van der MeerEricsson Telecommunicatie BVRijen, the Netherlands
2
Contents> Virtual Multimedia Office (VMO) concept> Demo of VMO> Architectural Overview of VMO> Mobile client> Wonderland enhancements> Next steps> Summary and Q&A
3
Simplified Network ArchitectureSimplified Network Architecture
4
Virtual Multimedia Office concept> Business Communication Suite> Brainstorm + Project Wonderland> Features needed:
Ericsson branded office or building Link to other collaboration prototypes Physical – virtual interaction Mobile access via laptop and mobile phone
User Stories> Remote
Access to VMO via laptop
> User ‘on the move’Access to VMO with mobile phone
> User @homeAccess to VMO with TV
Wonderland <> Virtual Multimedia Office
7
Virtual Multimedia OfficeImplemented Features> Mobile client (2D)> Improved video player (Motion JPEG)> Interactive Whiteboard> Incoming calls> Alert & Client control by Digital Photo Frame> Shared application audio> Hyperlink Cell
8
Demo and or video (part 1)> Ericsson dress-up
9
Contents> Virtual Multimedia Office (VMO) concept> Demo of VMO> Overview of VMO> Mobile client> Wonderland enhancements> Next steps> Summary and Q&A
10
Overall VMO architecture
WonderlandServer
Wonderlandclient
Softphone
JVoicebridge
PC
WonderlandJ2ME client
Mobile phone
AsteriskVoice Service(GSM)
MobileProxy
Server
ServerMasterClient
Project Wonderland (0.4)> Built on Darkstar platform> Open source toolkit for
creating 3D virtual worlds Aimed at business and educational useSupports high-fidelity audio, application sharing, telephone, video ..
> Cells….
12
Overall VMO architecture
WonderlandServer
Wonderlandclient
Softphone
PC
WonderlandJ2ME client
Mobile phone
Voice Service(GSM)
MobileProxy
Server
ServerMasterClient
JVoicebridge
Asterisk
VNC
13
JVoiceBridge> Software only audio mixer written in Java> Handling Voice over IP (VOIP) with SIP> Mixing, voice chat and sound detection> Stereo audio > Volume and channel (left/right) control> Individual voice mix
14
JVoice Bridge architectureControl
Request Handler
Conference Conference
Conference
Call
Call
Call
Call
Call
Call
Call Call
15
Asterisk> Open source telephony engine & tool kit> Very flexible> Switch (PBX) capability> Gateway> Media server
> Supports SIP and ISDN protocols
16
Application
Asterisk architecture
Application
Dialplan
Configuration
Configuration
ChannelChannel
Channel
Asterisk
PBX core
17
Virtual Network Computing (VNC)> Copy PC screen to other PC (or Wonderland)
18
Contents> Virtual Multimedia Office (VMO) concept> Demo of VMO> Architectural Overview of VMO> Mobile client> Wonderland enhancements> Next steps> Summary and Q&A
19
Demo and or video (part 2)> mobile client
20
VMO/Wonderland 2 mobile - architecture
Wonderlandclient
Softphone
PC
Mobile phone
Voice Service(GSM)
WL 0.4
WL 0.4SIP
SIP
WonderlandServer
JVoicebridge
Asterisk
Server
ServerMasterClient
WonderlandJ2ME client
MobileProxy
UBF
21
Mobile Proxy> Acts as ‘normal’ clients to server> Starts session when mobile accesses proxy> Converts ‘relevant’ CellSetup messages> Communication mobile <> proxy is UBF based> Universal Binary Format (UBF)
Parse efficient for machines Java, C++, Erlang, … drivers
22
Mobile Client – rendering (1)> Requirements:
Objects must be scalable (zoom in/out)
Objects must be rotatable Speed Simple image file parsing Assets same location other
extension (2D assets)
23
Mobile Client – rendering (2)> What went wrong:
SVG for everything SVG avatars & objects, bitmap (tiles) for worldmap
> What worked: Drawing methods javax.microedition.lcdui.graphics Figures: Arc, Line, Filled arc, Filled triangle
24
Mobile Client - screenshots
25
Mobile Client - UVG{'compound' # #
{'filledarc' #{'color' "#00ff00"}& {{0 0} 100 0 360}}&
{'triangle' #{'color' "#000000"}& {{-130 -130} {-130 130} {130 130}}}&
{'triangle' #{'color' "#000000"}& {{-130 -130} {130 -130} {130 130}}}&
}$
26
UVG editor
27
Mobile Client - Operation> Simple provisioning (SMS: xxxxx M JohnDoe)> Configuration (user specific JAD)> SMS returned with download link> At connection, Proxy logs into server + phone nr.> JVoicebridge sets up voice call (via Asterisk)
> User uses left-right to rotate avatar, up/downtranslates to forward/backward
> When near ‘sensitive’ cells, outline appears and menu commands (to operate on cell)
28
Contents> Virtual Multimedia Office (VMO) concept> Demo of VMO> Architectural Overview of VMO> Mobile client> Wonderland enhancements> Next steps> Summary and Q&A
29
Demo and or video (part 3)> video player> interactive whiteboard> hyperlink cell
30
Wonderland Enhancements> Incoming calls> Interactive Whiteboard> VideoPlayer> Alert Cell> Hyperlink Cell> Voice from the shared applications (browser..)
31
Incoming Calls> Original Function
Calls towards ‘speaker phones’ are possible Issue: Calls to logged-in users required
> New/Changed Adapted IncomingCallHandler class How: - at incoming call, check user logged-in
- present call at HUD
32
Incoming Call - screenshots
33
Incoming calls - architecture
Asterisk
PBX
JVoicebridge
WonderlandServer
IncomingCallHandlervoicelib
SMC
34
Interactive Whiteboard> Requirement
Interworking with remote and local teams Create ‘remote’ presence
> Approach Interactive Whiteboard controlled by Windows PC Using VNC to copy PC screen into virtual world Camera to show real-whiteboard-users Laptop showing virtual participants Speakerphones (real/virtual) for sound
35
Interactive Whiteboard
36
Interactive Whiteboard setup
37
Interactive Whiteboard setup (2)
38
Interactive Whiteboard setup (3)
Mario
Real
Virtual
39
Video Player > Original Player
Plays via buffer => texture Issue: Java Media Framework based
& problems with VNC
> New/Changed M-JPEG video library Why: M-JPEG = stream of pictures
Camera support (e.g Axis 206)
40
VideoPlayer - architecture
JMFBuffer
MPEG
Original
JPEG decoding
M-JPEG
Adapted
41
Wonderland Enhancements> Incoming calls> Interactive Whiteboard> VideoPlayer> Alert Cell> Hyperlink Cell> Voice from the shared applications (browser..)
Trigger external node (Chumby)> Requirement
Unobtrusive user interface to VMO screen Issue: click on object => “real world” reaction
> New/Changed Generate HTTP-request when object clicked How: - created Alert Cell
- HTTP address in XML config
43
Trigger external node - screenshot
44
Chumby control> Requirement
Unobtrusive user interface to VMO screen Issue: control client without keyboard
> New/Changed Modified WL client to accept Chumby control Adapted sunspot interface in WL client
Chumby setup
TiltSensors
SUN-SpotInterface
VMO/WL client
HTTP-proxyWebServerWeb
Server
Chumby Laptop
VMO/WL server
Wonderlandclient
Tunnel
46
Demo and or video (part 4)> chumby
47
Hyperlink Cell> Requirement
Fill in private data in browser form Issue: click on object => start pc browser with URL
> New/Changed Start browser when object clicked How: - created Hyperlink Cell
- client side invokes application
48
hyperlink Cell
VMO/WL client
Laptop
VMO/WL serverApplication
Hyper-linkCell
Hyper-linkCellCell setup URL-info
49
Audio for external applications> Requirement
Audio (not only visual) for shared external apps Issue: pick up sound, mix into world
> New/Changed Introduced new audio treatment in JVoiceBridge How: - capability to handle PulseAudio added
50
Audio for external applications
Audio from movie
Is ‘played’ viaAudio cell
51
Audio for external applications
Wonderland Server
JVoicebridge
AudioCell
TreatmentManager
PulseAudio
RecordStreamparec
Treatment“pulseaudio”
• When starting the shared Application, route audio output to pulseaudio library
• Example: export PULSE_SINK=.. firefox
52
Recommendations> Correct .xml configuration files> Stop (old) java processes> Disable firewall on (company) laptop
> Original JVoicebridge (in WL 0.4 package) had problems.We updated to a svn version from Dec 8, 2008
53
Contents> Virtual Multimedia Office (VMO) concept> Demo of VMO> Architectural Overview of VMO> Mobile client> Wonderland enhancements> Next steps> Summary and Q&A
54
Next steps> Serious research (Future Workspaces)> Trials / test with current release> Move to Wonderland 0.5> Extensions to improve experience
& shared working
> NOTE: Wonderland BOF sessionThis room; 6:45pm
55
VMO Development team> Project Management Marc Verschueren /
Gert-Jan Rijckenberg> Asterisk & presence Mario Goorden> VMO configuration Florin van Slingerland> Wonderland extensions Erik Reitsma> Mobile client/proxy Erik Reitsma> VMO video Sin Yuk Yan / Mieke Verheijen
56
Links> Wonderland lg3d-wonderland.dev.java.net> Darkstar www.projectdarkstar.com> JVoice Bridge jvoicebridge.dev.java.net> Asterisk www.asterisk.org> PulseAudio www.pulseaudio.org> VNC www.realvnc.com> UBF www.sics.se/~joe/ubf/site/home.html> Chumby www.chumby.com> Future Workspaces www.futureworkspaces.nl
57
Jan van der Meerjan.van.der.meer(#)ericsson.com
Ericsson Telecommunicatie BVRijen, the Netherlands
Recommended