Upload
animesh-singh
View
1.143
Download
5
Embed Size (px)
Citation preview
@AnimeshSingh
How to build an event-driven, polyglot serverless microservices framework on OpenStack Animesh Singh, STSM IBM Cloud Platform OpenStack Summit Barcelona, October 2016
Serverless - Functions as a Service
Ease of getting started Full stack Control
Functions PaaS Containers IaaS
@AnimeshSingh
Cloud Computing Evolution
Bare Metal
IaaS
Container Orchestrators PaaS
@AnimeshSingh
Problem: It’s expensive to scale microservices, even on a PaaS
Explosion in number of containers / processes:
1. Increase of infrastructure cost footprint
2. Increase of operational management cost and complexity
Region B Region A
Break-down into microservices
Make each micro service HA
Protect against regional outages
Monolithic application
@AnimeshSingh
Problem: the programming and cost model doesn’t help
• Continuous polling needed in the absence of an event driven programming model.
• Charged for resources, even when idle.
• Worries persist about capacity management.
Swift
Application
Container VM CF
2
Polling
1b
Request
1a
@AnimeshSingh
Bare Metal
IaaS
Container Orchestrators PaaS
Enter Serverless!
Serverless
Introducing OpenWhisk, a fabric and platform for the serverless, event driven programming model
@AnimeshSingh
OpenWhisk- A distributed compute service to execute application logic in response to events.
@AnimeshSingh
OpenWhisk: How does it work?
} 1
Event Providers
OpenWhisk
Cloudant
Git
Weather
…
…
Data event occurs, e.g. - Commit on a Git Repository - CRUD operation on Cloudant - ….
Trigger execution of associated OpenWhisk action
2
… JS Swift ! Docker …
@AnimeshSingh
OpenWhisk: How does it work?
OpenWhisk
JS Swift ! Docker …
Incoming HTTP request, e.g. HTTP GET mynewcoolapp.com/customers
1 2 Invoke associated OpenWhisk action „getCustomers“
Browser
Mobile App
Web App
Variety of languages
The OpenWhisk programming model
@AnimeshSingh
Programming model • Services define the events they emit as triggers, and developers
associate the actions to handle the events via rules • The developer only needs to care about implementing the desired
application logic - the system handles the rest
T A R
@AnimeshSingh
Trigger: A class of events that can happen T
Programming model
@AnimeshSingh
Actions: An event-handler, i.e. code that runs in response to an event
A
Programming model
@AnimeshSingh
Programming model Actions: Multi-runtime support, e.g. JavaScript A
functionmain(msg){return{message:'Hello,'+msg.name+'from'+msg.place};
};
@AnimeshSingh
Actions: Multi-runtime support, e.g. Swift A funcmain(params:[String:Any])->[String:Any]{
varreply=[String:Any]()ifletname=params[“name”]as?String{ print(“Hello\(name)”) reply[“msg”]=“Goodbye\(name)”}returnreply
}
Programming model
@AnimeshSingh
Actions: In any other language by packaging with Docker A Programming model
@AnimeshSingh
Rules: An association of a trigger and an action R
R := T A
Programming model
@AnimeshSingh
Actions: Can be chained to create sequences to increase flexibility and foster reuse A
AA := A1 + A2 + A3
AB := A2 + A1 + A3
AC := A3 + A1 + A2
Programming model
@AnimeshSingh
Packages: A shared collection of triggers and actions P
A
A read
write T changes A translate A forecast
A post T topic
Open Source A myAction
T myFeed
Yours
T commit
Third Party
Programming model
OpenWhisk backend architecture
@AnimeshSingh
OpenWhisk Architecture
Controller
Kafka
Invoker Invoker … Invoker
Router (nginx)
NodeJs Java Swift
Consul Couch DB
22
Controller
… Invoker Invoker Invoker
1!
2!3,4!
5!
6!
7!
8!
OpenWhisk Topology
OpenWhisk on Openstack
@AnimeshSingh
Architecture: Distributed OpenWhisk on OpenStack
!
Bootstrapper Machine !!!!!!
@AnimeshSingh
OpenStack Configuration • Services: Tested on OpenStack Kilo and Mitaka releases. We expect at a minimum
following OpenStack Services available: Identity (Keystone), Compute (Nova), Image (Glance) and Network (Neutron).
• Capacity: For each component VM, we would recommend 4 vCPU, 8 GB RAM and 60 GB disk. So if you deploy just 1 Invoker VM, you can expect to have a minimum capacity of around 36 vCPUs, 64 GB of RAM and 480 GB local disk. For 25 concurrent invocations/sec, we would recommend around 10 Invoker VMs.
• Network: We would recommend creating a virtual router, network and subnet for OpenWhisk, ideally in its own tenant. Also except for the ‘Router’ none of the VMs provisioned would need an external IP, so once the deployment is complete we can assign a floating ip to the ‘Router’ from the external network.
@AnimeshSingh
External Shared Customer Network
OpenWhisk Tenant Private Neutron Network
Neutron OpenWhisk
Tenant Router
OpenWhisk VMs Other Tenant VMs
Other VMs
Other VMs
Other VMs
Tenant Private Network
Physical Router
OpenWhisk - VM Networking View
Other Tenant Router
@AnimeshSingh
Deployment Overview: • To kickoff things, spin a VM which we call ‘bootstrapper’, and it’s essentially using Gradle and
Ansible under the covers to do the build and deployment respectively. • Setup the bootstrapper using the instructions listed here
https://github.com/openwhisk/openwhisk/blob/master/tools/ubuntu-setup/README.md
• Once bootstrapper is ready jump to the following link to build and deploy OpenWhisk on OpenStack https://github.com/openwhisk/openwhisk/blob/master/ansible/README_DISTRIBUTED.md
• In addition to the default Ansible modules, we add Ansible Cloud Modules for OpenStack, and then based on a set of environment variables, we build and deploy OpenWhisk on your OpenStack. http://docs.ansible.com/ansible/list_of_cloud_modules.html
@AnimeshSingh
Deployment Parameters: • For instances and hosts file to be generated and managed by Ansible, set values for the following keys using
environment variables.
• OS_WSK_DB_VOLUME is optional. If not specified, local disk will be used instead of persistent disk for CouchDB.
export OS_FLAVOR=m1.medium
export OS_IMAGE=Ubuntu14.04-1Nic export OS_KEY_NAME=key_name
export OS_NET_NAME=network_name export OS_NET_ID=e489dcf2-4601-4809-a459-e3821a95d23a export OS_USERNAME=abcxyz
export OS_PASSWORD=******* export OS_PROJECT_NAME=OpenWhisk export OS_SECURITY_GROUPS=sec_group
export OS_WSK_DB_VOLUME=15
## Keystone v2 export OS_AUTH_URL=https://OpenStack_URL:5000/v2.0 export OS_TENANT_NAME="OpenWhisk"
export OS_TENANT_ID=a9e6a61ab914455cb4329592d5733325
## Keystone v3 export OS_AUTH_URL=https://OpenStack_URL:5000/v3 export OS_PROJECT_ID=a9e6a61ab914455cb4329592d5733325
export OS_USER_DOMAIN_NAME="domain"
@AnimeshSingh
• Use Whisk CLI or Node-RED to create Multi cloud Serveless Applications!
OpenWhisk deployed! Compose Serverless Applications
$ wsk property set --apihost openwhisk.ng.bluemix.net --auth 96294c7b-e6f4-4ccf --namespace “animeshsingh" $ wsk action invoke /whisk.system/utils/
echo -p message hello --blocking --result
{ "message": "hello" }
Summary
@AnimeshSingh
What you learned today
• We’re in the early days of an evolution that is empowering developers to write cloud native applications better, faster, and cheaper
• OpenWhisk provides an open source platform to enable cloud native, serverless, event driven applications
• Open source OpenWhisk can be run on OpenStack in a scalable manner
@AnimeshSingh
Experiment with OpenWhisk on Bluemix
bit.ly/ow-bm
@AnimeshSingh
Deploy OpenWhisk on OpenStack
http://ibm.co/2ecCyxD
@AnimeshSingh
Join us to build a cloud native platform for the future! OpenWhisk.org dwopen.slack.com #openwhisk bluemix.net
Thank You