View
7
Download
0
Category
Preview:
Citation preview
OpenStackCeilometer
Tong Li (IBM)
Brad Topol (IBM)
Agenda• What is Ceilometer?
• Brief history of Ceilometer
• Meters of Ceilometer
• Workflow
• Ceilometer Architect & Building Blocks
Collecting Process Pipeline
Transformer Publisher
Collector Data store drivers
Alarm Dispatcher
• Ceilometer ReSTful API
• Ceilometer Python Client
• Ceilometer in Devstack
• Inter-Component Security & High Availability
• Who are the users of OpenStack Ceilometer
• Ceilometer Road Map
• Q & A (Demo maybe)
Who are we?• Tong Li
– IBM senior software engineer
– Twitter @email4tong
– Email: litong01@us.ibm.com
• Brad Topol
– IBM distingushed engineer
– Twitter @bradtopol
– Email: btopol@us.ibm.com
What is Ceilometer?
“OpenStack measurement project”
Julien Danjou Ceilometer PTL
A system that keeps track of what is happening in
OpenStack clusters. It was originally created for billing but it has been evolved into an all purpose
measurement repository. It can be extended for purposes such as performance report,
troubleshooting, capacity analytics, smart
scheduling, optimization, ultimately BIG DATA etc.
Brief History of Ceilometer• Before year 2012 billing has been left out of OpenStack core so far
as it was not the primary problem and was non-trivial, yet almost every OpenStack deployment needs a way to track usage information
• Started in May 2012 by developers from Canonical, DreamHost, RedHat, Dell, Intel, AT&T, etc.
• Targeted to do metering, rating, billing
• Grizzly objective: The project aimed to become the infrastructure to collect measurements within OpenStack so that no two agents would need to be written to collect the same data.
• Havana objective: remains the same as Grizzly objective plus become an integrated OpenStack project.
• Current PTL: Julien Danjou
• 91 contributors, 7 core reviewers
Meters of Ceilometer (from Nova)
Meters of Ceilometer (from Neutron)
Additional measurements:
http://docs.openstack.org/developer/ceilometer/measurements.html
Workflow
• Collect interesting info from OpenStack components
(or any other components), create meters (or samples)
• Transform meters into other meters if needed
• Publish meters to any destination (including Ceilometer itself)
• Store received meters into data store
• Read data via Ceilometer REST API
TransformCollect Publish Store Read
Ceilometer Architecture
Collecting
Neutron SwiftGlanceCinder
Notification bus (AMQP) queue name: notification.info
Notification Handlers, Central Agents
APIAPI API API
Nova
API
Publishing pipeline Publishing
Polling
Ceilometercollector & agent
ReceiverReceiver
PublisherPublisher
Pipeline
TransformerMeterTransformer
Transformer
Publisher Receiver
Pipeline: a set of transformers mutating meters into something that publishers know how to send to external systems.
Configure pipelines @pipeline.yaml
New in Grizzly!
Transformer
MeterName=cpu.time
Value=5Timestamp=T
Transformer
MeterName=cpu.time
Value=20
Timestamp=T+1
MeterName=cpu.time
Value=80
Timestamp=T+2
MeterName=cpu.percentage
Value=9Timestamp=T+2
Transform meters into new meters!
New in Grizzly!
Publisher
MeterCeilometer
RPCPublisher
Other Publisher
CeilometerMessage queue
External system
AMQP
signed message
Billing, monitoring,
alarming, statistics,
capacity planning…
New in Grizzly!
Data Store
Message Queueceilometer.collector.metering
SQL Database
MongoDB Database
Meter
Dispatchers
Multiple backend choices
Default backend
HBase
IBM DB2
New inGrizzly
New inHavana
Alarm
Both alarm evaluator and alarm notifier are Ceilometer services
1. Alarm evaluator uses Ceilomter public APIs to pull meters to produce alarms according to alarm definition, then place the alarm message onto the alarm queue
2. Alarm notifier read the alarm queue then inform external systems (log, rest, test)
New inHavana
DispatchersNew in
Havana
Multiple dispatchers support allows Ceilometer to be extended easily with many different outlet for all meters coming into Ceilometer. Dispatchers can be easily developed and deployed to add more features. For example, sending email, text messages.
1. Dispatcher only needs to implement one method.
2. By default, database dispatcher is used even if the configuration file has no trace of any dispatchers.
3. To configure multiple dispatchers:[ceilometer.dispatcher]file = ceilometer.collector.dispatcher.file:FileDispatcherdatabase = ceilometer.collector.dispatcher.database:DatabaseDispatcher
Ceilometer ReSTful APIrequest
Database
Ceilometer APICeilometer API
Consumers
Billing, monitoring,
alarming, statistics, capacity planning…
HTTP
REST
w/ JSON &
XML
Storage abstraction
layer
Raw events:GET /v2/meters/vcpus
Statistics (sum, average, min, max…):GET /v2/meters/vcpus/statistics
Filter:GET /v2/meters/vcpus?q.field=project_id&q.value=foobar&q.op=eq
Group statistics by period:GET /v2/meters/vcpus/statistics?period=3600
New in Grizzly!
Ceilometer ReSTful APIresponse
Response can be in JSON or XML
Raw events:/v2/meters/storage.object.size
Statistics (sum, average, min, max…):/v2/meters/storage.api.request/statistics
Ceilometer Python Client
Ceilometer in Devstack
devstack localrc
• Enable the ceilometer metering services
enable_service ceilometer-acompute,ceilometer-acentral,ceilometer-collector
• Enable the ceilometer alarming services
enable_service ceilometer-alarm-evaluator,ceilometer-alarm-notifier
• Enable the ceilometer api services
enable_service ceilometer-api
Inter-Component Security
Ceilometer High Availability
Who uses Ceilometer?
https://etherpad.openstack.org/p/CeilometerFeeedback
Roadmap
Havana Icehouse
• Integrated Project ✓• Integration with Horizon• Publishing meters to other
systems ✓• Enhance SQL driver ✓• Alarming ✓• Integration with Heat• Deprecating APIv1 ✓• Completing APIv2 ✓• Move publishing part to
Oslo and other projects ✓• Tighter integration with
Nova ✓• Nova-scheduler integration
✓
• Incubated Project ✓• Integration
with Horizon ❌• Agents for other
componentso Swift ✓o Ceph? ❌o Nicira? ❌
• SQLAlchemy storage driver ✓
• Multi-Publisher ✓• API v2 ✓
o User accessible API
✓
o More aggregation ✓
o Multi-dimension ✓
Grizzly
• OpenStack Design Summit in Hong Kong
Questions?
http://launchpad.net/ceilometer
http://docs.openstack.org/developer/ceilometer
http://wiki.openstack.org/ceilometer
Freenode: #openstack-metering
Mailing List: openstack-dev [ceilometer]
Recommended