View
223
Download
0
Category
Tags:
Preview:
Citation preview
Introduction to Grid Application On-Boarding
Nick Werstiukwerstiuk@platform.com
Presentation Objective and Contents
Objective Provide a ‘Grid’ software vendor perspective on the
challenges, approaches and techniques used with our customers to on-board applications to their Enterprise Grid environments
Presentation Contents Context to Frame the Discussion Enterprise Motivations – Why On-board? Enterprise Challenges Choices Bring Flexibility and Complexity Example Scenario Conclusions
Context to Frame the Discussion
Context This presentation is made in the context of Enterprise users
across multiple industries (Electronic Design, Industrial Manufacturing, Financial Services, etc) who use their Enterprise grid for multiple applications.
When I say Grid on-boarding… what I mean is… taking an application (either homegrown or commercial ISV) and enable it to run that on a distributed infrastructure within the 4 walls of the Enterprise
Typical Enterprise customer has a range of application types and patterns that bring different challenges.
Sequential vs. Parallel Short Running vs. Long Running Job Oriented vs. Service Oriented Home Grown vs. ISV
Enterprise User Motivations – Why Onboard?
Application Business Owner Driven Application Performance
Need results faster – leverage parallel execution
Demand growth beyond the boundaries of a single server
IT /Infrastructure Driven Economy of scale – more applications means more
resources, means (generally) opportunity for a more efficient infrastructure
Aggressive promotion of centralized/managed grid infrastructure for an Enterprise Utility/Shared service.
Enterprise Challenges with On-boarding to the Grid
Balancing between needs of the Application Owners, Architects, Developers and the IT Infrastructure Application and language specific middleware and toolkits vs.
single approach for internal ‘standards’ “Server and Cluster Huggers” Multiple applications on the same infrastructure introduces
additional management challenges Different Application Middleware – Common Resources Service Levels Resource Sharing and Policies
Minimize Application Development and Integration Maintenance Leverage new resource and processor types without major
application migration pain
Choices Bring Flexibility and Complexity
Application Integration Driven In many cases ‘on-boarding’ is a script integration between the ISV and the
grid infrastructure ‘Easiest’ to do with Job based applications Quick assessment of the market says there are probably 100+ commercial
ISV’s across the high performance computing market that have this type of deployment.
Maintaining the integrations becomes a challenge for the ecosystem (A standards opportunity)
Application Development Driven How best to decompose the problem?
Single Program Multiple Data (SPMD) Multi-Program Multiple Data (MPMD)
What’s the Best middleware for the application and programming model OR the corporate grid standard
Another Silo or an Enterprise Grid. How do I enable non Grid developers to easily transition their applications to
the grid
Choices Bring Flexibility and Complexity
Which one is the Best? MPI OpenMP Task-Oriented / Service Oriented HPC Message Oriented Middleware JavaSpaces Map-Reduce Spring Batch JSR-237 Work Manager for Application Servers Multi-core/GPU Programming Models Emerging Parallel Languages Complex Event Processing CEP
Sample Paradigms for Parallel Applications
Model Paradigm Development Deployment Run
Map-Reduce (MR)
Scatter & aggregate for data parallel app
Split data into files, write functions that map input files into intermediate output files, then aggregate into result files
Copy MR functions to a file system like
• Need a special distributed file system• Limited node sharing with other apps
MOM, JMS, AMQP
Message-oriented master and workers
A master and its workers send & receive messages via a message server or bus
Manually copy programs to nodes
• Nodes are dedicated to a single app run
Multi-core Multi-core shared memory multi-threading
Convert a serial app in the master thread into parallel tasks for worker threads, with memory & cache access optimization
Manually copy a program to multi-core boxes
• Multi-core box is dedicated to a single app run
HPC service-oriented middleware
Scatter & gather; generic task-oriented service pool
Parallel tasks are scattered to a distributed service pool, results are gathered by clients or other tasks.
Service programs are automatically deployed to grid nodes on-demand
• Nodes & service instances are pooled & shared with other apps
Example Scenario
“Split the Problem” Developers Develop without knowledge of the grid
infrastructure Seamless use of their IDE’s and development tools
IT Provides the Infrastructure without getting into the details of the application
Leverage application developer package – promote from test environment into production
Enable Wider Enterprise Developer Usage Open Access to broad range of developers Enable IT to promote grid on-boarding to their application
developer communities
Example Scenario
Symphony Developer Edition: A grid-ready application middleware to allow developers to code, tune, test applications before deployment onto a grid infrastructure Development & testing of the application is done outside of the grid Eclipse IDE Integration No further code changes required to deploy to the grid No IT involvement & effort is needed Supports service oriented and batch applications Increases developer productivity - hides grid from developers
Middleware delivers the highest application performance Support millisecond round trip latency Enables developers to target applications with a range of runtime
characteristics No restrictions access for developers at my.platform.com
Go ahead and sign in and try it
Other Considerations
Capacity Planning New Resources vs. Fitting into Existing Grid
Data and I/O Impacts Data Cache Database Filesystem
End User Experience Desktop Integration ISV Integration End User Portal
Conclusions
Customer Objectives Balance between driving more applications and dealing with
political/organizational issues
Challenges Maintaining the traditional well understood application
integrations as the infrastructure and ISV evolves Emerging application patterns and approaches
Recommended