Embed Size (px)
Citation preview
Understanding and Addressing Architectural Challenges of Cloud-
Based Systems
M. Ali Babar CREST – Centre for Research on Engineering Software Technologies University of Adelaide, Australia Keynote Talk @ BDCloud 2014, Sydney, Australia December, 5, 2014.
Background Brief
M. Ali Babar Professor of Software Engineering, University of
Adelaide, Australia – Nov. 2013 - PhD in CSE, University of New South Wales Work History:
Reader, Lancaster, UK, Feb. 2013 – Nov. 2013. IT University of Copenhagen, Denmark: Dec. 2009 … Lero, Ireland: 2007 – 2009 NICTA, Australia: 2003 - 2007 JRCASE, Macquarie University: 2001 – 2003 Various industrial roles in IT: Prior to 2001
Research Interests: Software Architecture, Service Orientation, Cloud Computing, and Software Development Paradigm http://malibabar.wordpress.com
Cloud Computing Research Threads
Decision Support Systems
Processes for Engineering Clouds
Architecting Cloud Systems & Services
Outline
• What is Software Architecture & Why is It
Important?
• Key Facets of Cloud Computing & Architecture
• Systematically Building Architectural Knowledge for Cloud-Based Systems
• Cases of Leveraging Architectural Knowledge
• Concluding Remarks
Why is Architecture Critical & What is It?
Some Scenarios for Architectural Support
legalexperts
domainexperts
specificationtools
Scenario 1processing and data only in region A
Scenario 2data only in region B
Scenario 3data only in region Bprocessing only in region A
Region B
Region A
Region C
• A Public Agency Wants to Use Clouds for Storing and Processing Highly Sensitive Data.
• An Engineering Company Intends to Use Clouds for its Highly Confidential Documents.
• Leveraging Cloud Bursting without Violating Legal Constraints and Agreements with Customers
Software Architecture
• “The software architecture of a system is the set of structures needed to reason about the system, which comprise software elements, relations among them, and properties of both” (Bass, L., et. al., 2013).
• “Fundamental concepts or properties of a system in its
environment embodied in its elements, relationships, and in the principles of its design and evolution” (ISO/IEC 42010).
• Its all about DECISIONS in certain context – bad, good and better ones.
Relating Architecture with Cloud Computing
“Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.” (A definition by the US National institute of standards and technology (NIST))
Key Facets of Cloud Computing & Architecture
Reproduced from Figure 1 of The Future of Cloud Computing: Opportunities for European Cloud Computing beyond 2010.
Getting Architecture Right is Hard!
“..The life of a software architect is a long (and sometimes painful) succession of sub-optimal decisions made partly in the dark…” (Philippe Kruchten)
Knowledge Centric Design & Evaluation
Documenting
Architecturedesign
Specifying ASRs
ArchitectureEvaluation
Stakeholders
Prioritized Quality Attribute
Scenarios
Requirements, constraints
Patterns and tactics
text
text
text
“Sketches” of candidate views,Determined by patterns
Chosen, combined views plus doc’n.
beyond views
Adapted from: Hofmeister, C., et al., A general model of software architecture design derived from five industrial approaches. Journal of Systems and Software 80(1): 106-126 (2007).
Systematically Exploring Relevant Literature
Empirism as a Research Approach
• Empiricism – Generally being regarded at the heart of the modern
scientific methods, that our theories should be based on our observations of the world rather than on intuition or faith
• Empirical Software Engineering (ESE) – Empirical research in SE is the scientific use of
quantitative and qualitative data to understand and improve software product and software development process (Vic Basili)
– Data is central element to address a research issues related to processes, technologies, or people
• ESE Helps in Understanding and Developing/Improving Processes, Technologies, People
Key Areas of Architectural Research Challenges
Research Challenges for Architectural Support
• Interoperability – Support for interoperability for multiple collaborative services.
• Privacy – Identifying trusted cloud services to process sensitive data. – Data placement strategies based on privacy requirements.
• Scalability – Automatic identification of traffic patterns for dynamic scalability.
• Adaptability – Support adaptability of service transmission environment
according to specific Quality of Service (QoS) requirements and provide communication space specific to a customer’s needs.
• Service Selection – Collaboration among cloud service providers (service discovery,
advertisement and composition). – Market-oriented resource and service provisioning.
• Challenges – Dozens of Different Engineering Tools Required – Some Commercial Tools (IBM SameTime and MS
Communicator) Available but Across Vendor Integration is Problematic & Tools are Expensive
– Just In Time (JIT) Composition and Use of Services – Misalignment between Tools, Processes and People
• Proposed solution – Cloud-Based Infrastructure for Providing TaaS to
Distributed Engineering Teams
• Cloud Service Federation – Cloud brokerage for federated clouds. – Increase capacity by delegating tasks to federated clouds. – Inter layer mappings of corresponding layers of reference cloud
mode among federated clouds. – Decentralize deployment infrastructure of by multiple providers. – Limited resource in a single cloud provider in stressed data
centers. – Avoid cloud vendor lock-in.
• Service Level Agreement (SLA) Compliance – SLA specific cloud services discovery. – Cloud system behavior anticipation according to specific QoS
requirements. – Decentralization of consistency and scalability management of the
services.
Research Challenges for Architectural Support
• Pervasive Embedded Networks on Cloud – Management of mash-up services on shared cloud resources. – Compliance with concrete semantic structures for information
presentation and communication. – Domain Specific Data visualization from various types of data
sources (wireless devices, web applications and medical images).
• Workflows management on cloud – Business processes cooperation for processing sensitive data. – Rationally fragmenting a workflow model. – Deploy workflow fragments on the underlying collaborative
architectural components.
Research Challenges for Architectural Support
Building and Leveraging Body of Knowledge
Design Knowledge Support Architectural Knowledge
ReasoningKnowledge
Share (C)
Architect (A)
Evaluate (F)
Learn (E)
GeneralKnowledge
Design Knowledge
Synthesize (G)
Context Knowledge
Integrate (B)
Distill (H)
Apply (I)
Producer
Consumer
Trace (D)
Trace (D)
Trace (D)
Evaluate (F)
Key
Knowledge Type
Actor Consuming activityProducing activity
Traceability created by producers and used by consumers
Search / Retrieve (J)
• Lack of Architectural Knowledge usually Results in – Severe System Design Problems. – Huge Technical Debt on Suboptimal
Design Decisions.
• Guidance and Tools – Types of Architectural Knowledge. – Manage & Share Knowledge. – Architectural Description for Reuse.
• We Developed – A Characterization Scheme of
Architectural Design Knowledge.
– An Infrastructure for Capturing and Sharing Architectural Knowledge.
Classifying Cloud Architecture Knowledge
Discovering & Cataloguing Architecture Styles
Architecture Design Knowledge Ecosystem
Private Ecosystem A
Private Ecosystem C
Private Ecosystem
B
Company
Employee
Public Ecosystem create
customized AK input form
share AK View AK
IDE Modeling Tool
collaboration
Modeling
AK Consume
Implementing
Integration integration
AK Extraction
AK Consume
Requirement CM/Issue Tracking
KBase
Knowledge-Driven Infrastrucutre Design
Infrastructure for Collaborative Engineering
• Context – Supporting Large Distributed Engineering Teams.
• Challenges – Dozens of Different Engineering Tools Required. – Some Commercial Tools (IBM SameTime and MS
Communicator) Available but Across Vendor Integration is Problematic & Tools are Expensive.
– Just In Time (JIT) Composition and Use of Services. – Misalignment between Tools, Processes and People.
• Proposed Solution – Cloud-Based Infrastructure for Providing TaaS to
Distributed Engineering Teams.
Tools as a Service (TaaS)
!
Some Commercial Tools
Tool Description AgileZen Collaborative Project Management LucidChart Tool for Creating Different Models
MeetingSphere Group Meeting and Decision Support System
Microsoft Live Meeting Web Conference Service
Microsoft Project Project Management Solution
Microsoft Team Foundation Server
Source Control, Data Collection/Reporting and Project Tracking
Pidoco Software to Design GUIs for Web and Mobile Apps and Make it Live to Share with other Users
IBM Rational Suite Suite of Tools for Different Phases of Software Development Life Cycle
Cloud9 IDE Cloud-Enabled Online IDE
Eclipse Orion Cloud-Enabled IDE that can be hosted on private/public clouds.
eXo Platform Collaboration Platform and IDE.
Key Requirements for Architectural Support
• Hosting & Provisioning/De-provisioning Heterogeneous Engineering Tools.
• Maintaining Security & Privacy. • Management of Repositories of Requirements,
Tools and Services. • Seamless Integration of Applications and tools. • Composition of Just-in-Time Tools Suites. • Alignment of Processes, Tools, and People. • Workspaces Supporting Tools Collaboration &
Artifacts Traceability for Virtual Teams.
Envisioned High Level Architectural Solution
• Tools Hosted in Public or Private Clouds
• Data (Content Elements) I n t e g r a t i o n t h r o u g h C o m m o n S e m a n t i c Model Using Ontologies
Core Elements of TaaS Space High-level Architecture Overview
Semantic Integration Among Tools • Explicitly Describing Common Concepts • Mapping Between Tools Specific & Common Concepts
ASR and Knowledge Management Tool Modeling Tool
End Users End Users
Building Semantically Integrated Data Model
End to End Integration • Probes and Plugins to Map
Data of Tools onto Aggregated Ontology Model.
• Generating RDF Graphs from Aggregated Ontology Model.
ASR and KM Concepts Modeling Concept
A Suite of Ontologies for TaaS
• TaaS Space Ontology (SpaT) – Establishing Relations among Activities, Tasks and Artifacts.
• Capability Ontology (CapT) – Representing Capabilities of Tools and Users Requirements; Enabling Matching.
• Change Ontology (ChaT) - Monitoring and Tracking Changes Made to Different Artifacts with Different Tools.
• Annotation Ontology (Annt) – Annotating Artifacts for Establishing Trace Links between Artifacts and analyzing Impact of Change on Artifacts by Actions Taken.
Architecture of Integration Systems • Subsystem for Annotation, Semantic Integration and
Collaboration Notifications based on Ontology Model
Architectural Knowledge for Migrating Clouds
Architecture-Based Migration Process
Source: Kazman R., Woods S. G., Carrière S. J.: “Requirements for Integrating Software Architecture and Reengineering Models: CORUM II”, in proc. of the Working Conference on Reverse Engineering (WCRE'98), pp. 154, IEEE, 1998
Migrating Tools to Cloud Infrastructure
• Migrating Software Metrics Collection and Analysis Tool – called Hakystat
• Supporting a Large Number of Organizations for Process and Product Metrics for Monitoring and Improvement
• Organizations Require Elastic Computing and Storage Resources
• SaaS on IaaS (Amazon) or SaaS on PaaS (Goolge)
Features & resources
Architecture of Hackystat
Provides visualization of different metrics through GUIs Generates reports for
external clients
Provides weekly, monthly and yearly
abstractions of metrics
Provides daily abstraction of data
Receives and stores data and provides daily abstractions
Quality Attributes & Architectural Decisions Quality
Attributes Architectural Decisions
Amazon EC2 & S3 Google App Engine Scalability Replication of system services to meet
performance requirments. No action required. Scalability is handled by platform.
Separation of database layer into a new service that utilizes platform specific persistency features.
Refactoring of persistency components to make it compatible with Google Datastore persistence.
Portability A wrapper layer is added to ensure platform independence. A separate database layer to provide seamless transfer of database layer.
Portability to other platforms is not possible.
Compatibility System features are exposed through origonal REST API. A wrapper layer is added to provide abstraction to services cluster and their deployment configuration.
System features are exposed through origonal REST API.
Reliability & Autonomous Scalability
Façade/Waper layer to provide abstraction. Amazon’s Elastic Load Balancer ensures autonomous scalability.
Ensured by platform.
Efficient & effective deployments
Amazon Elastic Load Balancer ensures auto scaling as well as efficient and cost effective deployment configuration.
Deployment of application components on cloud is managed by platform.
Architectural Views of Hackystat in Cloud
!Source: Chauhan, M. A., Ali Babar, M., Towards Process Support for Migrating Applications to Cloud Computing, Int’l Conference on Cloud Computing & Service Computing, 2012.
Architecture-Based Cloud Migration Process
Source: Ahmed, A., Ali Babar, M., A Framework for Architecture-Driven Migration of Legacy System to Cloud-Enabled Software, Companion Volume of WICSA, 2014.
Cloud Migration Process Support
What is Needed?
Concluding Remarks
• Software Architecture Plays a Vital Role in Design and Evolution of Cloud-Based Systems
• Rapid Adoption of Cloud Computing has Created Huge Gap in Software Architecture Design Knowledge that can Result in Technical Debts
• Dozens of Architectural Related Challenges in Designing & Evaluating Cloud-Based Systems
• Systematically Building and Leveraging Architectural Design Knowledge is Important for Developing on or Migrating to Clouds
Acknowledgements
• Slides are based on the work that is being carried out in my group in close collaboration with several colleagues, students, and industrial partners.
• Some research challenges and promising solutions have been developed for joint research proposals.
• TaaS Platform work is being driven by Aufeef Chauhan.
• Professor Michael Sheng advised on the development of Ontological solutions.
