View
3.653
Download
4
Category
Tags:
Preview:
DESCRIPTION
My keynote at Eclipsecon Europe 2013. One of the attractive qualities of OSGi is its role in enabling technologies that adopt it to manage the cost of their own success. Anything that gains adoption - in technology or elsewhere - picks up baggage as a result and needs to figure out how to deal with current installations while expanding in new directions. The WebSphere platform has been around for almost as long as Java and knows a thing or two about baggage but still manages to travel to many places with just a carry-on allowance. We adopted OSGi internally 8 years ago and have gradually increased our exploitation with each passing release, most recently and deeply with the lightweight WAS Liberty Profile. It hasn't all been plain sailing and we learned from a number of mistakes made along the way. When WebSphere Application Server first adopted OSGi it had over 10 million lines of code in a modest number of huge JARs. The engineering effort to modularize that into a “sensible” number of OSGi bundles was fairly significant. We had a global development team spread across a dozen labs and nearly as many timezones, all learning OSGi principles at the same time. What could possibly go wrong? We did not, for example, initially adopt the services part of the OSGi architecture but it’s how we can now start/stop individual technologies of the Java EE Web Profile on the WAS Liberty profile, in a 50MB install with a 2-second startup, while still supporting a massive deploy base of applications on older levels of Java EE. One of the challenges OSGi continues to face is over when to be “front of office” and when to be “back”. As the industry accelerates towards cloud, OSGi is an internal part of IBM’s strategy for high-density virtualized Platform-as-a-Service through WebSphere Liberty. Today’s cloud provisioning strategies, for example the buildpacks used by Heroku and CloudFoundry, are designed to be technology-agnostic. As a programming model for the cloud, OSGi is in a position of strength with its heavily service-oriented architecture. But in the spirit of agnosticism, one of the next steps OSGi needs to take is simply greater availability of the core OSGi framework in some of the more popular cloud platforms. Once there are more OSGi services running in those environments then the value and simplicity of autowiring OSGi services as cloud services becomes more apparent. Expectations and vision has to be managed up and down any organization that invests in OSGi - from the executive leadership team responsible for the business's bottom line, through the distributed architecture/development teams building tomorrow's technology on top of today’s, to the marketing and sales organization who need to sell the result to both IT and line of business. The value proposition has to be tailored to the audience. This is the story of how WebSphere has had outstanding success with the former four-letter acronym that IBM Marketing still wants to expand.
Citation preview
Ian Robinson, IBM Distinguished Engineer
WebSphere Foundation Chief Architect
Travelling Light for the Long Haul
About Me
IBM Distinguished Engineer
WebSphere Foundation Chief Architect
Over 20 years experience in
transaction processing and distributed
enterprise computing
Product strategy & development and
enterprise Java standards
Travels a lot, based in IBM’s Hursley
lab in the UK (near Southampton).
Season ticket holder for 3rd most
successful club in English football:
Ian Robinson
Carrying Baggage Can Be Expensive C
om
ple
xity
Time
Traditional
system
OSGi
Time K
no
wle
dg
e A
bo
ut S
yste
m
Based on http://www.tensegrity.hellblazer.com http://adaptevolve.blogspot.com
Traditional
system
OSGi OSGi**
“Co
gn
itiv
e b
urd
en
”
But Losing Baggage Can Be Worse
“Baggage” is something your users wants you to keep. Forever.
– Baggage == Business Continuity
WebSphere’s software support statement guarantees “N-2” for
application compatibility and platform support.
Engineering challenge to deliver the new without breaking the
old. For a long time.
– Including the crazy experiments that got out.
If I had a pound…
We Needed to Travel Lighter for the Long Haul
WebSphere AppServer and OSGi were
both born in 1998.
By 2006 WebSphere was 10 million lines
of Java code and growing.
Global development team of 100s in many
development labs in different timezones
around the world.
Tens of thousands of large customer
deployments in long-term support.
Classic struggle to increase ratio of
innovation : support
– Variable “stickiness” of innovation but
uniform expectation of support.
“Something had to change” (Part One)
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
1.0 2.0 3.0 3.5 4.0 5.0 5.1 6.0 6.1 7.0 8.0 8.5.5
2013
8.5
Level Name Summary
1 AdHoc Nothing
2 Modules Formal identity, decoupled from artifact
3 Modularity Formal module contracts, decoupled from identity
4 Loose-Coupling Services, semantic versioning, decoupled from
implementation
5 Dynamism Life-cycle awareness and independence, decoupled
from time
6 Devolution Modularity-aware repositories, collaboration,
governance, decoupled from ownership
Graham Charters, OSGi Community Event 2011: Towards a Maturity Model
OSGi Maturity Model Summary
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
1.0 2.0 3.0 3.5 4.0 5.0 5.1 6.0 6.1 7.0 8.0 8.5.5
2013
8.5
Componentized Build
Components have identity and version
Components produce a jar impl depends on interfaces at build time Factory in interfaces uses Class.forName to load impl at runtime
interfaces client impl
Pre-OSGi Modules (Build View) Pre-dated Maven
Level 2
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
1.0 2.0 3.0 3.5 4.0 5.0 5.1 6.0 6.1 7.0 8.0 8.5.5
2013
8.5
A B
init/start stop/destroy Java Bean Components
– Implements a specific
interface
Init/start/stop/destroy phases
– Started in a specified order
Makes use of:
– Class.getResource()
– Class.forName()
Runtime couldn’t enforce build
modularity
Expensive to maintain and
extend
C C
Pre-OSGi Modules (Runtime View) Level 2
WebSphere Gets OSGi (2006) Internal re-engineering while simultaneously
adding external business capabilities
Best-practice approach: start with small number
of large bundles and iterate over time
– Runtime modularity enforced
– Service maintenance and testing better targeted
– Runtime footprint no longer monolithic
Challenge: Significant learning experience
across worldwide team
Jar A Jar B
Jar C Jar D
Jar A Jar B
Jar C Jar D
Approach 1
Content of Jars
A - D
Approach 2
Level 3
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
1.0 2.0 3.0 3.5 4.0 5.0 5.1 6.0 6.1 7.0 8.0 8.5.5
2013
8.5
What’s Good for the Goose…
We had rebuilt WAS V6.1 on
top of OSGi.
This must surely make it
easier for developers to
benefit from OSGi in their
own applications. Right?
OSGi Moves Up the Stack
webA.jar
WEB-INF/classes/servletA.class
WEB-INF/web.xml
META-INF/MANIFEST.MF
webA.jar
WEB-INF/classes/servletA.class
WEB-INF/web.xml
META-INF/MANIFEST.MF
webA.jar
WEB-INF/classes/servletA.class
WEB-INF/web.xml
META-INF/MANIFEST.MF
Bundle Repository
Import-Package
webA.jar
WEB-INF/classes/servletA.class
WEB-INF/web.xml
META-INF/MANIFEST.MF
webA.war
WEB-INF/classes/servletA.class
WEB-INF/web.xml
WEB-INF/lib/…
logging f/w jar
persistence f/w jar
MVC f/w jar
DI f/w jar
webA.war
WEB-INF/classes/servletA.class
WEB-INF/web.xml
WEB-INF/lib/…
logging f/w jar
persistence f/w jar
MVC f/w jar
DI f/w jar
webA.war
WEB-INF/classes/servletA.class
WEB-INF/web.xml
WEB-INF/lib/…
logging f/w jar
persistence f/w jar
MVC f/w jar
DI f/w jar
webA.war
WEB-INF/classes/servletA.class
WEB-INF/web.xml
WEB-INF/lib/…
logging f/w jar
persistence f/w jar
MVC f/w jar
DI f/w jar
logging f/w jar
persistence f/w jar
MVC f/w jar
DI f/w jar
• Manage multiple
versions of libraries
across an enterprise
• Isolate application
dependencies from the
server runtime
• Centralized location to
deliver critical fixes
• Flexibility to update to
new versions one app at
a time
OS
Gi B
un
dle
Re
po
sito
ry: In
teg
rate
d
with
WA
S A
dm
in
• We did a better job for external apps
than we did for internal components
• “Services” were a key part of the App
support
• A Grade:
• Cheaper to maintain, extend and test
• Need do Better:
• Insufficient dynamism - especially in fast-
changing development environment
• Components too tightly-coupled – didn’t
deliver on desired lightweight footprint.
Lessons We Learned
The Omelette Challenge
Recipe:
Create a lightweight profile of WebSphere
that starts in under 2 seconds
Make it completely dynamic for all
changes to configuration
Provide an unzip install <50 Meg in size
Don’t break any eggs.
Provide complete backward compatibility
Application Manager HTTP Transport Feature Manager
jpa servlet
jsp
jsf
jndi jdbc
sessionDatabase monitor
wab
blueprint
jaxrs
json
restConnector
osgi.jpa
appSecurity
ssl
localConnector beanvalidation
ejblite cdi managedBeans
oauth collectiveMember ldapRegistry webCache
wasJmsClient wasJmsServer
wasJmsSecurity jmsMdb
jaxws
clusterMember
concurrent
wmqJmsClient wsSecurity
mongodb jaxb
collectiveController zosSecurity zosTransaction
Runtime Services
&
Config Model
Java EE
Web Profile
Java EE
Full Profile
What we had What we wanted Multi-bundle feature
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
1.0 2.0 3.0 3.5 4.0 5.0 5.1 6.0 6.1 7.0 8.0 8.5.5
2013
8.5
Time Independence is Fundamental
Recognized our error in not exploiting OSGi services when we
originally adopted OSGi.
Services were the ONLY way we could achieve the size and
dynamism objective without massive and unnecessary re-invention
A significant consideration for component design.
– Required us to replace existing factory patterns and configuration
management.
– Modular implementation already suitable
A v1
B v2
C v1.1
D v1
Co
mp
lexity
Time
OSGi**
package import
service reference
X
A La Carte Alongside the Prix Fixe 2012: “Liberty Profile” of WebSphere supports arbitrary
combinations of OSGi and Java EE “features”.
– Remember the eggs: any app running on the Liberty
profile of WAS runs unchanged on the full profile of WAS.
Same runtime bundles (mostly) but loaded and configured by
new OSGi subsystem-aware kernel as independent feature
subsystems (new in OSGi R5)
Entirely self-contained metadata to describe bundle content,
services published, & configuration metatypes.
Bundle A
Bundle B
Bundle C
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
1.0 2.0 3.0 3.5 4.0 5.0 5.1 6.0 6.1 7.0 8.0 8.5.5
2013
8.5
Feature
Manifest
Config
Metatype.xml
“Feature”
We use features as
units of:
– Deployment
– Configuration
– Extensibility
Level 5
Keep the Engine Under the Bonnet/Hood/Kühlerhaube
felix scr1.1
equinox metatype 1.2.0
Config Admin R4.2
equinox framework 3.8.2 (OSGi R5.0)
Bundle A
Bundle B
Bundle C
Feature
Manifest
Config
Metatype.xml
jsp-2.2
Feature Manager
WebSphere Liberty Kernel
OSGi details: • are available to extenders
of the platform. • stay on the inside for
users of the platform
Application developers & operators only see this
<server description=“server1”>
<featureManager>
<feature>jsp-2.2</feature>
<feature>jdbc-4.0</feature>
</featureManager>
<application name="tradelite"
location="tradelite.war" />
</server>
1 2
3 4
server.xml configuration
The Love That Dare Not Speak Its Name Or Why We Stopped Bragging About OSGi
WAS v8.5.5 Liberty
Profile &
WAS Developer
Tools for Eclipse
(WDT)
Unzip install and deploy IM or unzip to install. New option to
deploy “server package” of app + config + required subset of server runtime for highest density deploy
Liberty Extensions Add custom features and
integrate 3rd party components via Liberty
extensions interface
Fidelity to full profile WAS Same reliable containers & QOS.
Develop on Liberty profile and deploy to Liberty or full-profile WAS
Integrated tools Powerful tools in WDT Eclipse
feature. Enhanced for v8.5.5 prog models, Maven integration, ++
Lightweight cluster Mgmt Liberty servers can join a
lightweight cluster for workload balancing and high availability
Dynamic Server Profile Not static like Web Profile; configured
by app at a fine-grained level
Start fast, run efficiently Starts in <3s; Mem footprint
<50MB; (TradeLite benchmark)
Dynamically Extensible Install new features from repository (local or remote) with no svr restart
“Developer First” Focus Simplified, shareable XML server config. New
integrated messaging server, DynaCache support, new prog. models, such as Web Services, JMS & EJB-Lite.
Small Download 50MB for Web Profile features
Web Profile Certified Create web apps for the Java
EE Web Profile standard.
Level 5
Runtime Package Management
OSGi standards here to help
– Subsystems, Resolver, Repository
Flexible runtime assembly creates
opportunity for flexible runtime delivery
– Only install what you need
Feature repositories for developers and
runtime provisioning
– Enterprise Subsystem Archives (.esa)
content e.g. Liberty Repository or
– Subsystem metadata that refer to externally
hosted bundles e.g. Apache Karaf
>featureManager install jsp.esa
Application
Manager
HTTP
Transport
Feature
Manager
jpa servlet
Level 6
1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
1.0 2.0 3.0 3.5 4.0 5.0 5.1 6.0 6.1 7.0 8.0 8.5.5
2013
8.5
Liberty Repository
Looking Ahead:
What Are the Challenges
for OSGi?
Cooking is simplified by recipes.
– OSGi needs to do this.
OSGi standards provide high-quality
modular specifications
Vendors choose which specifications to
incorporate into their solutions
BUT no separation between application
and middleware.
– And not enough recognition of the
difference
Customers want standard solutions
The OSGi Application Framework
proposal for rich Web applications may
help…
C
om
ple
xity
Time
Traditional
system
OSGi
OSGi**
The Cognitive Burden of Cookery
OSGi Application Framework - A spring-board to cloud
What it could be:
A profile of specifications available for application use.
– Apps can rely on vendor solutions including these
Re-using existing technology where applicable
Enabling first-class exploitation of OSGi
Focussing on 80:20 rule
– Leave room for innovation to encourage vendor adoption
Supporting flexible provisioning depending on application
need
Recognizing the difference between applications and
containers. Embrace container management to simplify
app burden
https://github.com/osgi/design/blob/master/rfps/rfp-0160-ApplicationFramework.pdf
PaaS Buildpacks and OSGi in the Cloud
> cf push project [--buildpack
https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack]
Application -application
-server.xml
-resources cf push app with buildpack
Execution Agent
Isolated Address Space
Application
Middleware stack
Execution Agent
Isolated Address Space
Application
Middleware stack
Execution Agent
Isolated Address Space
Application
Middleware stack
Execution Agent
Isolated Address Space
Application
Middleware stack
Execution Agent
Isolated Address Space
Application
Middleware stack
Execution Agent
Isolated Address Space
Application
Middleware stack
Execution Agent PaaS
PaaS is a good opportunity for OSGi
Application Framework in the cloud
Heroku, Cloud Foundry and other
PaaS’ have extension points for
application stacks: “buildpacks”
Provision and scale OSGi applications
using appropriate buildpack for OSGi
stack in the cloud.
Ideal for simplifying the provisioning of
flexible “just what’s needed” app
instances for high density in the cloud
OSGi dynamic services a natural fit for
cloud shared services
Isolated App Instance Isolated App Instance
Application
Middleware stack
http://www.ibmdw.net/wasdev/docs/deploying-an-osgi-app-to-liberty-in-the-cloud/
http://thoughtsoncloud.com/index.php/2013/10/possibilities-abound-with-osgi-running-on-cloud-foundry/
Higher Density Less Hardware Less Cost
Make it Small: Provision the smallest middleware stack needed
– For the Java stack, IBM is doing this using OSGi (WebSphere Liberty
buildpack) and IBM Java.
Execution Agent
Isolated App Instance Isolated App Instance
Application
Middleware stack
Feature 2 Feature 1
Feature 3
* http://www.ibm.com/developerworks/library/j-multitenant-java/
Make it Smaller: IBM Multitenant
JVM*
– Isolated tenants
– Per-tenant Statics
– Control: threads, memory, cpu
Application tenant 2
Application tenant 1
Are We Nearly There Yet?
Software Engineering view
of Line of Business
LoB view of
Software (over) Engineering
Carrying baggage is part of the business.
Strategies to reduce its cost are as important
now as they were 40 years ago.
Technologies like OSGi help.
They help best when you know how, when
and to whom to sell it to internally in your
organization.
Thank You @ian__robinson
http://wasdev.net
Recommended