DoDDoD Software Transformation Software Transformation
SOASOAThe Road Toward Net-centric OperationsThe Road Toward Net-centric Operations
Michael BehrensMichael BehrensR2AD, LLCR2AD, LLC
Chief Technology OfficerChief Technology OfficerSupporting DISA GCCS-J ISupporting DISA GCCS-J I33
April 2005April 2005
Rob VietmeyerRob VietmeyerNCES Chief EngineerNCES Chief EngineerDefense Information Systems AgencyDefense Information Systems AgencyApril 2005April 2005
R2AD®, LLC
2
Agenda
• DoD Software Transformation - DISA
– Network Centric Enterprise Services(NCES) Overview
– DISA’s Pilot Effort & Lessons Learned
• Standardizing the Grid - GGF
– Open Grid Services Architecture (OGSA)
• Deployment and Maintenance
-- Albert Einstein
“We can’t solve problems by using the same thinking we used“We can’t solve problems by using the same thinking we used
when we created them”when we created them”
3
IT Complexity
• More than 400 different softwareapplications support JTF commanders
• Duplicity coupled with lack ofinteroperability and integration
• Tightly coupled interfaces preventingmigration and modernization
• Lack of full visibility and total access toinformation and functions
4
The New Imperatives
• How to Improve, Design, Implement, and
Manage large, complex Net-centric (SOA)capabilities?
• How to Lower Risks inherent in designing anddeploying and operating large, complex Net-
centric capabilities?
Shape Evolution of Enterprise ITvice
Build Systems
5
Why SOA: Technical View
• Interoperability: Integration across the networks, heterogeneous technologies,organizational boundaries
– Exposing S/W functions as consumable services, easing application and data sharing
– Transportability of application functions across organizational boundaries
– Platform for evolution of enterprise policies, standards, and functions
• Agility: Ability to dynamically reconfigure processes to meet changing operationalrequirements
– Separation of responsibilities and dependencies at the service interface
– Componentization of reusable software
– Technical and business abstraction
• Visibility: Common understanding of requirements and capabilities among consumers,planners, and providers
– Consumer: What information and functions are available
– Providers: What information and functions are needed
– Operations: Ability to identify and respond to events
– Business planning: Ability to identify service utilization patterns, ROI
• Transformation: Closer alignment of IT with business and warfighting processes– Abstraction of technical complexity
– Faster deployment of new technologies and processes, building block approach
– Enabling new capabilities and processes
6
Lessons we’re learning
• SOA is not just about technology…
– Business
– People & Culture
– Organization
– Information Philosophy
• Challenges remain
– Maturing technology and evolving standards
– Performance and security
• Web services = easy but deceptive
• Enterprise SOA = challenging but necessary
– Potential for HUGE benefits ….
8
Architecting the Service Environment
• Services Environment– Service Producers
– Service Consumers
• Services Infrastructure– Utility and environmental
services to facilitate andgovern producer andconsumer interaction
• Computing Infrastructure– Hardware and software
platform for hosting serviceproviders and serviceconsumers
• Network Infrastructure– Ubiquitous connectivity
UA UA
SMTA
IMTA IMTA
ROOTDSA
Global
DSA
Regional
DSA
MFI BMTA
MLA
BMTA
MLA
Policies, procedures, middleware and services tosupport enterprise -- Discovery and access -- Security -- Management and evolution ofAvailable information, functions and capabilities
People, Processes, Information,
Applications
NCES Increment 1 FOCUS
10
Service Discovery
Service Consumer
Service Consumer
PublishEnterpriseServices
DiscoverEnterpriseServices
Service Provider
Invoke
Enterprise Service Registry
LocalServices
AdvertisedServices
LocalRegistry
NCES Hosted
DOD Root Registry
Standards Base: Universal Description, Discovery
and Integration (UDDI)
11
Service Discovery
• UDDI Maturing– Previous implementations
• Complex
• Focused on technical abstraction and localimplementation
– Future direction• Foundation for enterprise SOA management and
governance
• Support for business (as well as technical) functions
• Improved support for integrating services registrywith associated artifact repositories
– Challenges• Registry federation and synchronization
• Supporting potentially conflicting needs of runtimeenvironment, development environment andmanagement environment
12
Pilot Security Services Architecture
ApplicationApplication
IdentityStore
Service
Consumer
Au
then
tica
tio
n Service ProviderService Provider
WebService
WebService
NCES Security ServicesNCES Security Services
PolicyStore
CertificateValidation
Service
PrincipalAttribute
Service
PolicyDecision
Service
PolicyAdmin
Service
PolicyRetrieval
Service
PolicyEnforcement
Point
PolicyEnforcement
Point
DOD PKI &
Global Directory
Standards Base: WS-Security, SAML, XACML
13
Challenges & Opportunities
• Today, false sense of comfort in our existing securityapproaches and mechanisms?
• SOA technologies are enabling abstraction of securitychallenges
– Mitigate vulnerabilities
– Improve functionality and dependability
• But we need …
– Standard trust models that extend beyond point-to-point / per-hoptrust
– Message addressing and routing standards. There are couple ofthem out there but they
• Need to converge
• Need to incorporate security into their design
– Policy standards beyond authorization (e.g., XACML)
• QoS, privacy, governance
• Support for policy discovery
14
Service Management
Standards Base: Web Services Distributed Management (WSDM)
Service Provider
EnterpriseService Registry
LocalServices
AdvertisedServices
Monitor
Management
Agent
Alerts System
Enterprise Service Manager
NET OPS
Network MgmtSystem Mgmt
Service ConsumerInvoke
Service Level Mgr
15
Service ManagementIntegrating the Services Environment with Computing Infrastructure
• Monitoring and management of the runtime environment
• Evolving marketplace and standards
• Potential for redundant or conflicting capabilities
– Sub-optimization through optimization?
CommunicationsManagement
ComputingInfrastructure Management
Serv
ice
Serv
ice
Serv
ice
Serv
ice S
erv
ice
Serv
ice
Serv
ice
Serv
ice
Serv
ice S
erv
ice
Serv
ice
Serv
iceServ
ice
Serv
ice
Serv
ice
Applications/SystemsManagement
Service Management
16
Engage with developers andstakeholders across your enterprise
• A Web-based global collaborative
software development solution
• Strengthens relationships with internalstakeholders, business partners, andclients
Project Tools• Administration
– Add members– Create roles
• Communication– Announcements– Discussion Forums– Mail Lists
• Artifact Repositories– File Sharing– Document Library– Version Control– Source Code Management
• Bug and Issue Tracking
17
Beware of the realities
Web services SOA– Ease of development can lead to poor implementation
• Services that are too granular causing inefficient communications and
overhead
• Tightly coupled designs tied to underlying technologies and
implementations
• Services tightly coupled to legacy business processes
– Limited understanding within the developer community
• Rapidly evolving technology
• Evolving standards and commercial implementations
• Lack of implementation guidance and best practices
– Limited understanding at a business level
• Impact of SOA technologies on the business
• Ability to align service evolution with business needs
• Approaches to SOA management and governance
• Need for new approaches to IT architecture, funding, oversight
18
SOA requires governance
• Enterprise policies and standards
– Maintain interoperability
– Maintain and improve information assurance
– Manage enterprise interdependencies
• New roles and authorities required
– Who (and how) establishes service identities and designatesthem authoritative
– Support for different rules and mechanisms for differentportions of the service’s life-cycle (proposed, developmental,testing, operational, deprecated, retired)
– Management of business processes that span traditionalorganizational boundaries. Judiciary functions to supportSLA processes??
• Ability to align service development and implementationwith business requirements
• Balance bureaucracy with need to support market driveninnovation and evolution
19
This is going to be interesting ….
• Systems
• Software interoperability
• Duplication of functions
• Data element standardization
• Common operating environment
• COTS
• Host applications locally
• Heavy clients
•• Service providers and consumersService providers and consumers
•• Business process integrationBusiness process integration
•• Optimization and specializationOptimization and specialization
•• Standardized business productsStandardized business products
•• Integrated network environmentIntegrated network environment
•• Commercial service providersCommercial service providers
•• Provide service globallyProvide service globally
•• Application delivery over networkApplication delivery over network
Yesterday TomorrowTomorrow
Migration to a SOAMigration to a SOA
-- David Alberts and Richard Hayes
““With the coming of the Information Age, there is an opportunity toWith the coming of the Information Age, there is an opportunity to
provide widespread access to information related services andprovide widespread access to information related services and
capabilities only dreamed about in previous eras.capabilities only dreamed about in previous eras.””
21
Grid and WS: Convergence
Grid
Web
The definition of WS-Resource Framework (WSRF)means that Grid and Web communities can moveforward on a common base. There is a migrationpath for NCES to Grid Computing via web servicesstandards. A new paradigm is being born.
WSRF
Startedfar apartin apps& tech
OGSI
GT2
GT1
HTTP
WSDL,
WS-*
WSDL 2,
WSDM
Have beenconverging
TODAY
OGSA
Security
GT3GT4
22
Grid Operating EnvironmentDynamic Automated Virtual Global Grid
Grid Operating Environment (GOE)
A concept infrastructure framework
system supporting coordinated resource
sharing and problem solving in dynamic,
geographically dispersed secure virtual
organizations.
Open Grid Services
Architecture (OGSA):
an open source,
community supported set
of services and protocols
including resource access
and security.
R2AD
Dynamic Automated Virtual Global Grid
R2AD, LLC
23
Automated Grid Computing in a GridOperating Environment (GOE)
1. Grid Computing
– Virtualization
– High PerformanceComputing (HPC)
2. Automation
– Installation
– Fielding
– Management
GOE brings two concepts together
• Data Grid
• Computational Grid (agent support)
• Resource Virtualization
• Scalability
R2AD, LLC
24
Typical Grid Layers
Grid-Enabled C4ISR Applications
Grid Visualization Grid Security
Grid Workflow
Grid VMs, Cache
Manager
Grid Middle Ware Engine(GT4 + OGSA + NCES)
High Performance & Secure Grid Networking (GIG-BE)
P
ac
kag
in
g-
ACS
GridDevelopment
- Grid RPC
- Grid MPI- MDA Scheduler
EMS
RSCs DECCs COCOMs DISA
network
Ref: Adapted from NaReGI-PSE National Grid Initiative slides from GGF - ACS-WG, OGSA Standardization Process
R2AD
25
What is OGSA
• Open service-oriented architecture based onWeb services for addressing Grid scenarios
• Component-oriented architecture
– Interchangeable components
• Meta OS functionalities
– Distributed and heterogeneous environment
• A rendering of these functions, based on Webservice architecture and specifications
• A GGF’’’’s flagship architecture and the blueprintfor industry standard grid computing
26
Context Services
InfoServices
InfraServices
SecurityServices
Rsrc Mgmt Services
Execution Mgmt
Services
DataServices
Policy
Mgmt
VO
Mgmt
Access
Integration
Provisioning
Cataloging
Boundary
Traversal
Integrity
Authorization
Authentication
WSRF WSN WSDM
Event
Mgmt
Trouble-
shootingDiscovery
Job
Mgmt
Logging
Execution
Planning
Workflow
Mgmt
Workload
Mgmt
Provisioning
Application
Mgmt
DeploymentConfigurationReservation
Naming
Self MgmtServices
Heterogeneity
Mgmt
Service Level
Attainment
QoS
Mgmt
Optimization
Information Services
Infrastructure Services
SelfMgmtServices
SecurityServices
Resource Mgmt Services
Execution Mgmt Services
DataServices
Context Services
27
OGSA Design Philosophy
• Service Oriented Architecture– Interface Extension (WSDL 2.0 ‘extends’ attribute)
– Resources as First Class Entities
• Expressed as WSRF-Resource Properties
– Data type extensibility and introspection
– Dynamic service/resource creation, migration, & destruction
• Component Based– Elements of the Architecture are pluggable
• Customizable– Support for dynamic, domain specific content, ...
– Within the same standardized framework
• Distributed Specification and Standardization– Identify and/or develop open and accessible standard
specifications• Active current work in GGF, OASIS, W3C, and DMTF.
28
Architecture OverviewInfrastructure Services
SYSTEMMANAGEMENT
UTILITYCOMPUTING
GRIDCOMPUTING
Core Services WSDM
OGSA-EMSOGSA Self Mgmt
GRID Computing, Utility Computing and System Management are different views of the same important problem domain.
Discovery
DAIS
VO Management
Information
Distributed query processing
ASP
Data Centre
Use Cases &Applications
Multi MediaPersistent Archive
InfrastructureServices
WS-Addressing
Privacy
WS-Base Notification
CIM-XML/JSIM
WSRF-RAP WS-Security
Naming
GFD-C.16
GGF-UR
Data Model
HTTP(S)/SOAP
SAML/XACML
WSDL
WSRF-LT
Trust
Data Transport
WSRF-RP
X.509
29
Execution Management Service
• WS-Agreement• Job Submission Description Language (JSDL)• Configuration Description Language• Application Contents Service (ACS)
Provisioning
• Deployment
• Configuration
App. Contents
Service
Information
Services
Service
ContainerData
Container
Accounting
Services
Execution
Planning
Services
Candidate Set Generator
(Work -Resource mapping)
Job Manager
Reservation
A typicalconfiguration
of EMS
30
Resource Management
• OASIS WSRF, WSN• OASIS WSDM-TC
– Management of Web Service (MOWS)– Management using Web Service (MUWS)
• GGF CMM WG– Resource Management in OGSA
DataService
OGSA ServicesLevel
Domain-specific capabilities
OGSA Capabilities
SecurityService
Infrastructure Level
Resource Level
ExecutionMgmt
Resources
WSRF, WSDMWSRF, WSDM
31
Security Services
• Authorization, Roles, and Access Privileges– Locally (site) managed– SAML and XACML Basis, Proxy Certs– Credential mapping provided by implementations.
32
Data Service Architecture
• Components of Grid Data Service
– Engine (Grid Data Service Factories)
– Activities
• Replicate, Query, Transform, Deliver
– Data Resource Implementations
– Role Mappers
XSLTTransform
Web RowSet, xml
DeliveryToURL
Ref: Open Grid Services Architecture Data Access and Integration (OGSA-DAI)
33
Enabling Technologies
Requirements& Design
Develop & LAN Testing
System IntegrationWAN Testing
Operational
R2AD
34
Services Around the World
• Runtime environment
– Geographically dispersed grid containers
• Development Environment
– Also geographically dispersed
– Setup to including installing all locally required components
• Configuration Management, Version history/access
• Compilers
• Editors/IDEs (and plug-ins)
• Project Management, Design tools
• Test grid containers
• Production Environment
– Software Release
• Unit, Stress Testing
• Installation Description
• Documentation, Delivery
– Maintenance
R2AD
Virtualization
R2AD
35
Fielding Considerations
• Patches/Version
– Where is what installed
– Scheduled/Timed with world wide usage in mind
– Rollback support? Backwards Compatibility
– common vulnerabilities and exposures (CVE) - IAVA
• System Comparisons
– What is different between two systems which should bethe same? Support automated re-hosting of services
• System Updates and Migration
– Incorporate old-version support (via service brokers)
– Monitor process, metric gathering & status, alerting
• Content Distribution
– Large file distribution
– Data and Service edge caching
R2AD
36
Example workflow
Grid Container
Distributes to nodes
1. Develop application
and Store in repository
Grid nodes
3. Submit
Job to
Scheduler
2. Edit application,
make final build.
3. Retrieve (recursively) all components not local which
are needed, comparing, checking, security, etc
Entry Point
ACS
ESM, JSDL, CDDLM,
GRAM
R2AD
ARI AAF
37
Service Installation & Management
• ACS working group within GGF
– IBM’s Solution Installation, NAREGI-PSE Grid (Japan)
– Possible Integration with Sun’s N1?
• Application (Service) Repository
– Grid Provisioning Appliance
– Secure (Signed & Trusted applications)
– Used by provisioning systems - a trusted repository
• Configuration Description, Deployment and LifecycleManagement (CDDLM)
– Working Group within GGF
– Deploying complex, distributed services
– Dynamic service configuration and management
– Automatically deploy, manage and remove services
• WSDM (Web Services Distributed Management)
– Technical Committee within OASIS, standard Mar05
R2AD
38
Example, from draft ACS Specification
Register
AA File
AA
AADescriptor
Transfer
ACS (site 1)
AA Repository
EPR
ACS (site 2)
AA Repository
EPR
<acs:AAID> <acs:name>
http://example.org/application1 </acs:name> <acs:version> <acs:major>1</acs:mager> <acs:minor>0</acs:minor> <acs:revision>0</acs:revision> </acs:version></acs:AAID>
<wsa:EndpointReference> <wsa:Address>
http://site1.example.org/ACS <wsa:Address> <wsa:ReferenceProperties> <acs:AAID> ... </acs:AAID> </wsa:ReferenceProperties> </wsa:Address></wsa:EndpointReference>
<wsa:EndpointReference> <wsa:Address> http://site2.example.org/ACS <wsa:Address> <wsa:ReferenceProperties> <acs:Originator>
http://site1.example.org/ACS </acs:Originator> <acs:AAID> ... </acs:AAID> </wsa:ReferenceProperties> </wsa:Address></wsa:EndpointReference>
Terms: Application Archive (AA), Application Archive ID (AAID)
39
Possible Usage at DISA
submitsegment
submission parameter
program binaries,initial data,configuration data,deployment procedures,self-management policies,and so on...
Deployed Systems
register
develop
archive
Development & Test
Environment
test
AA File
AA File
Application
Developer
Business Activity
Manager (e.g.: DISA)
R2AD
Secure CM’d Repository(Accredited Software)
PullSegment
Global InformationGrid (GIG)
40
Automated Deployment Example
Processor LayerNetwork Layer
WAN
1. Deploy Zero Administration aRchive (AAF) into the Grid Manager
Global Grid Framework Manager ReplicaAAF
Network Layer Processor Layer
AAF
WAN
2. Deployed Zero Administration aRchive (AAF) into the Grid Fabric
Global Grid Framework Manager Replica
Processor LayerNetwork Layer
WAN
3. Configuration of the Network Fabric Grid
Subnet
AAF Global Grid Framework Manager Replica
Processor LayerNetwork Layer
Solaris Solaris Solaris
Solaris
Linux
WAN
4. Configuration of the Processor Layer with OS
Solaris Solaris
W2KS W2KS W2KS
AAF Global Grid Framework Manager Replica
Processor Layer
cluster
Network Layer
WWW J2EE
WAN
5. Configuration of the Processor Layer with OS
WWW J2EE
W2KS W2KS
Oracle
Oracle
MSQL
Load
Bal.
AAF Global Grid Framework Manager Replica
Processor Layer
cluster
Network Layer
WWW J2EE:80
:80
:80
SUN SUN
WAN
6. Activation of Routes
WWW J2EE
W2KS W2KS
Oracle
Oracle
MSQL
Load
Bal.
AAF Global Grid Framework Manager Replica
R2AD
41
References
• GGF Standards Body
– http://www.ggf.org/
– OGSA Working Group
• https://forge.gridforum.org/projects/ogsa-wg
– ACS Working Group
• https://forge.gridforum.org/projects/acs-wg
ACS Structure of Application Archive
R2AD
Acknowledgements
The content of this presentation includes information
created by companies or groups below
OGSA-DAI