View
7.718
Download
0
Category
Preview:
Citation preview
Simple Blue Template
Funambol Server
Architecture
Stefano Nichele
Overview
Funambol server components
Synchronization overview
Data synchronization service architecture
Push overview
Push Connection service
InboxListener and PIMListener
Funambol Server Architecture
Core component of the Funambol Server. It provides synchronization and push enginesAllows push mail by checking email accountsAllows pim push by checking the databaseAllows Client TCP PushDataSynchronizationServerInbox ListenerPim ListenerCTP ServerWe will refer to those as services. The DS service is the primary service, without that the Funambol Server cannot run.And we are going to change some names....
DataSynchronizationservicePim ListenerservicePush ConnectionserviceInbox Listenerservice
Funambol Server Architecture
DataSynchronizationservicePim ListenerservicePush ConnectionserviceInbox ListenerserviceDBMailServer
Synchronization
Push
Synchronization
Process to make equal two sets of data (repository, database)
One-way: data is coming just from one source (i.e. backup, restore)
Two-way: data is coming from both sources
PDABackupRepo
PDAMylaptop
Synchronization
Two different modes:
Full-sync: the sources are synchronized by comparing the whole sets of data (OMA terminology: slow-sync)
Fast-sync: the sources are synchronized by comparing just the changes that occurred since the last sync.
SyncML
Open standard for data synchronization
XML based
Supported by major companies such as Nokia, Motorola....Funambol :-)
Since 2002 defined by Open Mobile Alliance (http://www.openmobilealliance.org)
SyncML an example
1.2 SyncML/1.2 1 3 IMEI:004400061769830 http://syncserver.com/servlets/SyncML ... 1 3 0 SyncHdr http://syncserver.com/servlets/SyncML IMEI:004400061769830 200
Data Synchronization service
It's the core and the only required service of the Funambol server (primary service)
It governs synchronization and push
Extensible and highly configurable
SyncML 1.1 and 1.2
Data Synchronization service
AdminManagerSynchr.EngineEmail Sync SourcePIM Sync SourceGoogle Sync SourceOfficerPipelineManagerNokia SyncletRazr SyncletPushEnginePush SenderWebServicesAPIPluginsManagerPlugin APlugin BFrameworkDevice InventorySMSServiceSMS Provider
Data Synchronization service
PipelineManagerNokia SyncletRazr SyncletIt allows additional processing and manipulation of the messages exchanged between the server and the client.
The kind of processing that is performed in the pipeline is a message level processing, such as the manipulation of the message elements.
Possible applications are:
Encoding/decoding of item content
Item filtering
Item ordering
Message decoration adding/removing elements on a custom basis
Data Synchronization service
The idea behind the message processing pipeline is to be able to modify both incoming and outgoing messages:
in the former case, we want to be able to manipulate the message before it goes into the sync engine
instead, in the latter, we want to be able to change the message returned by the sync engine before sending it to the client.
Input and output pipeline components are called Synclet.
PipelineManagerNokia SyncletRazr Synclet
Synchr.EngineEmail Sync SourcePIM Sync SourceGoogle Sync SourceOfficerSERVLET
Input pipeline
Output pipeline
Pipeline Manager
Rawdata
SyncML
SyncML
Data Synchronization service
How are the synclets to perform selected ?
Synclets for any device
Synclets just for some devices: based on the user-agent of the requests
PipelineManagerNokia SyncletRazr SyncletExample of synclet configuration file
user-agent NokiaE61 com/funambol/server/engine/pipeline/phones-support/bsh/NokiaE65in.bsh
Data Synchronization service
Main responsibilities:
authenticate and authorize the user
identify the sources and the destinations of data to be synchronized
identify what data needs to be updated/added/deleted
determine how updates must be applied
detect conflicts
resolve conflicts
Sync.EngineEmail SSPIM SSGoogle SSOfficer
Data Synchronization service
Deep look at the synchronization engine
Sync.EngineEmail SSPIM SSGoogle SSOfficerSyncSessionHandlerSync4jEngineSync4jStrategyEmail Sync SourcePIM Sync SourceOfficer
SyncMLMessages
It represents the handler for a SyncML session. It coordinates and handles messages as dictated by the protocol.It coordinates the sync processdelegating almost all the synclogic to Sync4jStrategyGiven two sources of data,it performs the real synchronization processIt checks if the credentials provided by the client are validand if the user is authorized.A SyncSource is the interfaceversus the data store
Data Synchronization service
How can you interact with the engine process and how can you synchronize your data repository ?
Officer: it's the component that authenticates and authorizes a user. It's important since implementing your own officer you are able to authenticate a user on your user database.
SyncSource: it's the class used to access to the server data repository. You can develop your own PIM syncsource but also your source for MP3 or for whatever you want synchronization.
For more details, see the Developer Guide.
Sync.EngineEmail SSPIM SSGoogle SSOfficer
Data Synchronization service
SMSServiceSMS ProviderThe Data Synchronization service provides the required infrastructure to allow you an easy integration with your SMS Gateway.
SMS Service
it's the entry point for the SMS infrastructure
no specific SMS gateway logic
SMS Provider
it's the real interface towards your preferred SMS gateway (i.e. SubitoSMS or Clickatell)
SMSServiceSMS ProviderSMS Infrastructure
SMSGateway
Usually this is anHTTP requestSMS MessageRecipientSender
Data Synchronization service
SMSServiceSMS ProviderWhat can I do in the SMS Service ?
decorate the recipient/sender with additional information (like carrier/country)
work as proxy towards more than one SMSProvider selecting the one to use based on the recipient
provide smart functionalities (for example using the delivery detail/status information, a sms can be discharged if the previous one is still pending)
Out of the box we provide a simple SMS Service that forwards requests to an SMS Provider that just logs the messages.
Data Synchronization service
Plug-ins: configurable objects that can be deployed in the DS service in order to extend its functionalities. For instance a simple plug-in could send an email to the system administrator to notify him for startup and shutdown events.
By now the plug-ins are notified just for startup and shutdown events.
As for the other configurable components, they can be deployed without stopping the DS service just copying the xml bean file under com/funambol/server/plugin.
PluginsManagerPlugin APlugin B
Data Synchronization service
It's the core of the administration API. It can be accessed locally or remotely using web services.
It exposes the administration API via web services.
It's the component that handles devices and their capabilities
AdminManagerWebServicesAPIDevice Inventory....the latest DS service components
Funambol Server Architecture
DataSynchronizationservicePim ListenerservicePush ConnectionserviceInbox ListenerserviceDBMailServer
Push
Push
Push sequence in summary:
1. the Funambol server detects that a device needs to sync
2. the Funambol server sends a push message to the device. This is pkg #0 in the OMA specs
3. the device starts a sync so that its data is updated with the one on the server
FunambolServer
DataRepositoryDataRepository
123
Push
How can the server send the push message to the device ?
SMS seems the easiest way but SMSs are not free of charge
The client can run a TCP server so the server can open a TCP connection and sends the push message over that
The client can open a connection versus the server and keep it always open, ready for push message from the server
Funambol Server supports out of the box connection-less and connection-oriented push but SMS Push is easily implementable.
SMSPush
STPPush
CTPPush
SMSPush
Connection-lessPush
Connection-orientedPush
Push
Who does what ?
DSserviceMailServer
InboxListenerservicePIMListenerservice
PIMDBMailServer
WSAPI
PushEngine
Push Connectionservice
The Inbox Listener at regular time checks on the mail server if there are users to pushIf there are devices to push, the IL and the PIM listener call the DS service admin APIThe Push Engine tries to send the push message using connection-less pushThe Push Connection service tries to push the device using an existingconnectionThe Push Engine asks to the Push Connectionservice to push the device usingconnection-oriented pushThe PIM Listener at regular time checks on the DB if there are devices to push112345
Data Synchronization service
PushEnginePush SenderMain responsibilities:
Handling push :-)
WSAdmin API
NotificationEngineNotificationBuilderNotificationSenderPush Engine
DB
Push request
The NotificationEngineuses a NotificationBuilderto create the Push messageThe NotificationEnginechecks in the DB if there are old push messagesThe sender is the component used to deliver a push message to a deviceIf the sender is not able to deliver a push message, it is stored in the database1234
Data Synchronization service
SMSServiceSMS ProviderBuilder and sender configurable per device
Two available builder implementations (for SyncML 1.1 and for SyncML 1.2)
Just one sender implementation that tries:
connection-less push
connection-oriented push (if the previous one fails)
PushEnginePush Sender
Connection-less push
Main concepts:
the client must be able to run a TCP server waiting for connections from the server
when a push message must be sent, the DS service opens a socket and writes the message
the DS service must know the client IP address
the IP address must be reachable from the DS service
Connection-less push
AddressListenerServlet
DB
IP Address
Pending Push
IP Address
If the server is able to send the pending push to the client, the IP address is stored in the DB.123Address notification: process used by the client to notify its IP address
The device must notify its address any time it changes ! So it must be able to detect when it changes.
The device sends its IP address to the serverThe server verifies if the client address is reachable trying to send the pending push messages
Connection-less push
Issues:
The client must be able to run a TCP server
The client must be able to detect when its IP address changes
The IP address must be reachable from the server
Connection-oriented push
Main concepts:
The client opens a connection on the server and keeps it open
The server uses that connection to deliver push messages
...really simple ;-)
Connection-oriented push
Push ConnectionService
1The client opens the connection and sends its credentialsDSserviceWSAPIPushEngine2
The Push Connection service calls the DS service WS API to authenticate the client
UDP channel
The Push Sender sends a multicast UDP message with the push message21The Push Connection service receives the UDP message and checks if the recipient is connected3If the device is connected, the push message is written on the already open (by the client)connection
Connecting
Pushing
Push Connection service
Main challenge:
It must be able to handle millions of connections
Features:
Java Standard Application
Based on javax.nio
Based on Apache Mina (network application framework)
Clusterable
InboxListener and PIMListener
Both do the same work, at regular time they must check the data repository
Push ListenerFrameworkInboxListenerTaskInboxListener
Push ListenerFrameworkPimListenerTaskPIMListener
Both based on the PushListener Framework...
Push Framework Listener
Framework ready to use to implement any listener. By now used for:
InboxListener
PIMListener
ExchangeListener
OXListener
Pretty similar to a scheduler with basic functionalities for push and for HA/scalability
Java Standard Application (no J2EE)
Based on Jdk 1.5 thread features
JMX interface
Pluggable architecture
Push Framework Listener
Hey, this is not push.....it's polling !
InboxListenerserviceMailServer
PollingmodeInboxListenerserviceMailServerNotifiablemode
UDP notification
Polling mode works with all mail servers
Notifiable mode can be used just in some cases
Push Framework Listener
DBRegistry Monitor
...
...
User 1
User 2
User 3
User 4
Thread AThread BThread CThread DTask executor
Tasksqueue
Plugin Manager
UDP ListenerAt startup it loads from the database the list of tasks to perform and publish it in the tasks queueThread Pool.Any free thread pickup from the queue a task and starts its execution.It can contain scheduled tasks or One Shot tasksScheduled tasks
One Shottasks
It allows notifiable mode
Push Framework Listener
How does it scale ?
InboxListenerservice AInboxListenerservice CInboxListenerservice BInstance 1 of 3
Instance 2 of 3
Instance 0 of 3
Cluster
Any instance know its index and how many instances are in the clusterIt executes the tasks with (id % 3) = 0
It executes the tasks with (id % 3) = 1
It executes the tasks with (id % 3) = 2
Tasks Repository
The cluster is obtained using JGroups
Push Framework Listener
HA High availability
InboxListenerservice AInboxListenerservice CInboxListenerservice BCluster
Tasks Repository
Instance 1 of 3
Instance 2 of 3
Instance 0 of 3
It executes the tasks with (id % 3) = 0
It executes the tasks with (id % 3) = 1
It executes the tasks with (id % 3) = 2
Instance 1 of 2
Instance 0 of 2
It executes the tasks with (id % 2) = 0
It executes the tasks with (id % 2) = 1
Conclusions
DSserviceMailServer
InboxListenerservicePIMListenerservice
PIMDBMailServer
WSAPI
PushEngine
Push Connectionservice
112345SyncEngine
6Monitoring
Monitoring
WS API
Connection-less push
Request for connection-oriented push
Connection-oriented push
Synchronization
References
Funambol Forge - https://www.forge.funambol.org/
Funambol Developer Guide
http://download.forge.objectweb.org/sync4j/funambol-v7-developers-guide.pdf
Funambol design documents
https://core.forge.funambol.org/wiki/FunambolDesignDocuments
Open Mobile Alliance - http://www.openmobilealliance.org
myFunambol - http://my.funambol.com
Funambol mailing lists - https://core.forge.funambol.org/ds/viewForums.do
The End
Click to edit the title text format
Click to edit the outline text format
Second Outline Level
Third Outline Level
Fourth Outline Level
Fifth Outline Level
Sixth Outline Level
Seventh Outline Level
Eighth Outline Level
Ninth Outline Level
Recommended