Analyzing Cloud Costs in a Standard IT Cost Model€¦ · In order to execute a tagging strategy, you will need to know details about the AWS resources that are in-scope: • AWS

Analyzing Cloud Costs in a Standard IT Cost Model

©2014 Apptio, Inc. All rights reserved. http://www.apptio.com


Executive SummaryEnterprise use of public cloud services has made managing IT costs more complicated than ever. CIOs and CFOs need to track, analyze and

manage not only traditional infrastructure and applications but also the company’s usage and cost of public cloud-based services such as

Amazon Web Services (AWS). Adding to the complexity of hybrid IT cost management, different parts of an organization require different

views of cloud cost and usage data. CFOs and Corporate Finance organizations typically look at costs from the corporate reporting perspective

of Cost Centers while CIOs, IT Finance, Infrastructure, and Application leaders need context and granularity to accelerate better decisions that

include cost by project, applications, services, and resource types such as compute, storage, and network.

AWS provides as many as 28,000 product offerings generating up to millions of billing line items each month, creating complexity managing

IT costs. To solve these challenges IT needs a standard solution that incorporates costs based on cloud usage with on-premises IT costs for an

integrated view of today’s hybrid IT business.

In Partnership with:

2©2014 2nd Watch, Inc. All rights reserved. www.2ndWatch.com


An Explosion of Cloud OfferingsAWS packaging and pricing is very flexible, and as a result, very complex. It includes 28,000+ possible services; the number and composition of

which are constantly changing. AWS organizes its service portfolio by categories, and within each category are several product groups (fig. 1.0).

Within each AWS product group are as many as 8,000 possible service offerings based on the options and configurations the

customer chooses. For example, within the EC2 (Elastic Cloud Compute) group in the Compute category, a customer will select an option from

each of following categories to configure their instance (fig. 2.0).

AWS Product Offering

Category Product Group

Compute

EC2 (Elastic Compute Cloud)

Elastic Load Balancing

Auto Scaling

Database

Amazon RDS

Dynamo DB

Amazon Redshift

Amazon ElastiCache

Application Services

Amazon AppStream

Amazon CloudSearch

Amazon Elastic Transcoder

Amazon FPS

Amazon SES

Amazon SNS

Amazon SQS

Amazon SWF

Analytics

Amazon EMR

Amazon Kinesis

AWS Data Pipeline

ApplicationsAmazon WorkSpaces

Amazon Zocalo

AWS Marketplace

3rd Party Business Software

3rd Party Developer Tools

3rd Party Infra Software

Generally, pricing is by instance and can include an up front charge (Reserved Instances only) combined with “per instance per hour” pricing

(typically under $1.00 and measured to a tenth of a cent), billed monthly, excluding tax. Both up front and usage charges are determined by the

options chosen above, as well as volume of usage and length of term (1 year or 3 years for Reserved Instances). In addition to instance-based

services there can be charges for data transfer in and out of AWS. An overview of Amazon’s pricing model can be found at http://aws.amazon.

com/ec2/pricing/ and billing information athttp://aws.amazon.com/ec2/faqs/.

Fig 1.0

Fig 2.0

EC2 Configuration Options

Operating Systems Linux (RHEL, SLES, etc), Windows…

RegionsUS East, US West (Oregon), US West (Northern California), EU, Asia Pacific…

Instance TypesOn-Demand, Reserved, General Purpose, Compute Optimized, GPU Instance, Memory Optimized, Storage Optimized…

Performance Level

# of “ECUs”, an Amazon-specific measure of integer processing performance (needed since underlying hardware can change)

Assigned Resources # vCPUs, # GB Memory, # GB Storage



Integrating Public Cloud into a Standard IT Cost ModelProduct configuration flexibility is great for users, but creates new challenges for IT management. When AWS generates monthly bills, each

resource yields dozens or hundreds of distinct billing line items. All of this fidelity is bundled under a single invoice that the finance department

pays each month, but the granularity and meaning is typically lost in cost reporting to IT and business leaders. There is no management view of

how much cloud spend is going to storage vs. compute vs. network, how much of the enterprise’s total compute spend is driven by public cloud

vs. traditional models, or how public cloud is impacting the costs of projects, applications, and services that the business consumes.

To understand cloud spend in context of the enterprise technology business, IT needs both a unified model for categorizing cloud and non-cloud

costs together, and automation to map millions of billing line items into the IT cost model each month.

To automate the mapping process, 2nd Watch and Apptio have worked together on a mapping table that specifies where each Amazon

product fits within a standard cost model at the “IT Sub-Tower” level (see illustration below). This mapping is now embedded in the Apptio Cost

Transparency application as part of an end-to-end solution for integrating AWS usage into the costing process, including direct integration with

AWS billing, cost categorization, modeling of total costs of cloud usage including internal labor, and self-service analytics on how cloud impacts

the broader IT cost model.

Apptio maintains this AWS product mapping table and regularly updates it when AWS makes changes. During each scheduled retrieval of

AWS billing records, the Apptio DataLink AWS connector automatically checks for updates to the Apptio mapping table, loading them into a

dedicated dataset within Apptio Cost Transparency. This alleviates the need to maintain or manually refresh these mappings.

Apptio TBM Unified Model™ (ATUM™) Cost Categories

Comprised of

Other

Other

Telecom

Telecom

Depreciation

Lease

Expense

Maintenance & Support

Facilities & Power

Consulting

Managed Service Provider

Cloud Service Provider

Outside Services

Depreciation

Lease

Expense


Software

Internal Labor

Internal Labor

Depreciation

Lease

Expense


Hardware

External Labor

External Labor

Workspace

Mobile Devices

Service Desk

Field Support

Cloud Desktop

End User

App Dev

App Support & Ops

LoBS oftware

Cloud Apps

Application

Ops Center

Project Mgmt.

Client Mgmt.

Cloud Ops

Delivery

IT Mgmt. & Strategic Planning

Enterprise Architecture

IT Finance

IT Vendor Mgmt.

IT Management

Security

Compliance

Disaster Recovery

Cloud DR

Security & Compliance

Central Print

Post Processing

Output

Circuits

Usage

Communication

LAN

WAN

Voice

Other Network

Cloud Network

Network

Database

Middleware

Mainframe Database

Mainframe Middleware

Cloud Platform

Windows

Linux

Unix

Mainframe

Converged Infrastructure

Cloud Windows

Cloud Linux

Tier 1

Tier 2

Tier 3

Tier 4

Cloud Storage

Cloud Archive

StorageCompute

Enterprise Data Center

Other Facilities

Data Center

IT Towers & Sub-Towers

Cost Pools & Sub-Pools

Using only data directly from AWS together with actuals from a company’s general ledger, Apptio Cost Transparency can categorize AWS costs into IT Sub-Tower categories such as Cloud Windows in Compute, or Cloud Archive in Storage. Going beyond the IT Tower level of the cost model up to Applications and Business Units requires additional data. (For more information on Apptio Cost Transparency cost model and solution, please visit http://www.apptio.com/applications/cost-transparency.)

Adding Business Context Through Billing & TaggingA key strategy for IT cost transparency is to help both IT and its business partners accelerate better decisions by putting cost into the context

where key decisions are made, including applications, projects, and business unit consumption. Apptio Cost Transparency does this by

correlating financial and IT data directly from management systems through an automated cost model. For example, Apptio can use a list of

which applications are supported by which servers and storage, from a spreadsheet or an application dependency mapping tool, as one of the

most accurate methods for determining the run costs of an application. However, many organizations have not yet adapted their management

processes and tools to maintain application relationship data for cloud infrastructure. For these organizations, there are two AWS mechanisms

which can be leveraged to provide needed business context: linked accounts and tagging.



Linked Accounts

AWS provides access to its products and services through individual accounts, each with its own credentials and the ability to purchase and

manage AWS services. Many enterprises have many AWS accounts at the team and departmental levels (whether they know it or not). These

can be spread throughout application development, projects, IT infrastructure, and business units. This decentralization fosters agility, but left

unmanaged, this fragmentation of unlinked accounts creates blind spots in cost and operational management, as well as increasing costs by

missing out on discounts for aggregate consumption volumes.

To gain the benefits of central visibility while preserving the benefits of decentralized agility, enterprises can link individual accounts

into one “master account” paid through an IT cost center. The linkage between individual accounts enables Apptio to pull in all usage and costs

for the enterprise, while using the owner of each individual account (e.g. an application team or business unit) as a way to analyze cost “by

customer” in self-service analytics. And, it can qualify enterprises for deeper volume discounts. Using a linked account paid by IT brings cloud

spend out of the shadows while still enabling local departmental, team, or business unit agility.

Apptio Cost Transparency provides analysis of public cloud costs broken out by product (per the service mapping), by customer (linked account owners), by provider (AWS, Microsoft AZURE™…) and more.

For workloads with infrastructures that do not need to share resources, individual AWS accounts can be used to group usage by workload or

project. This can be accomplished by establishing individual AWS accounts for each workload and mapping them directly to your enterprise

organizational structure. In the example below, AWS accounts are created for each project or workload then rolled up to provide consolidated

usage information by department and business unit.

Tagging

In many organizations there is a need to share key resources, such as databases, across multiple workloads. In these cases it is best to use AWS

tags to group usage and expenses. This method requires careful provisioning of resources and the creation of a schema to allocate shared

resources that cannot be tagged across the enterprise.



For organizations that lack a coordinated approach to tagging today, here is a five phase approach to get started:

Phase 1: Scope

Answer some basic questions to decide where to focus first:

• When and how do your business, application, project, and operations teams request and provision cloud services?

• Which of them have a tool, process, or policy for request and/or provisioning that tagging could snap into?

• If no such process exists, where is cloud usage big enough, or expected to grow enough, to focus on implementing a process?

• Which team would be easiest to move along quickly?

Phase 2: Tagging strategy

A tagging strategy will help ensure that operational tools, as well as cost analysis, will have the data needed to answer critical questions

accurately. Components of a Tagging Strategy include:

• An executive summary of the objectives, customer stakeholder groups, and SMEs. This summary will serve as the project’s north star and keep you on course in the later phases.

• “Key” and “value” definitions documented as customer’s standards.

• Key image map in Visio for ease of editing. This visual aid is especially helpful when AWS resources require multiple tags.

Phase 3: Determine which resources are in scope

In order to execute a tagging strategy, you will need to know details about the AWS resources that are in-scope:

• AWS resource type (EC2, RDS, Redshift, DynamoDB, S3 Bucket, etc.)

• AWS instance ID and corresponding account number

• Host name

• Availability zone / location

Tagging allows enterprises to assign its own metadata to each tag-

able resource. Tags do not have any semantic meaning to AWS

resources and are interpreted strictly as a string of characters. Tags

are made up of both a “key” and a “value”. AWS allows up to 10 keys

for each resource and each key can have unlimited values enabling

very detailed grouping of resources. Tagging should be set up based

on the needs of the organization and the AWS architecture design.

The image below illustrates how to establish a tagging scheme for a

2-Tier auto-scalable web application.

As the project moves from Web Sandbox to Web Staging to Web

Production, you can also use tags to track usage. When the application

is in the sandbox all resources are tagged with the key “Web Sandbox”

and the appropriate value (Environment, Owner, App, and/or IT

Tower). When the project moves to “Web Staging” you simply replace

the original key and values with the ones associated with the next step

in development.

Even basic tags can make it easier for Apptio Cost Transparency to

associate AWS resources to applications and services:

• How much of a particular application, or an entire application portfolio, is comprised of AWS services?

• How much of which projects are using cloud resources?

• Which cloud costs are for development vs. test vs. production?



Phase 4: Make regular adjustments for optimization

AWS releases several new products and features each month. In order to keep up with the growing demands of a new AWS deployment,

there will be ongoing adjustments and optimizations that can take several man hours each month.

• Maintenance of tags, standards, and best practices

• AWS resource tag automation

• AWS tag integration with cloud formation templates

• Onsite or virtual training on cost management

Limitations

As an enterprise expands it AWS usage, it will eventually reach limitations to a manual approach for managing individual accounts and tagging.

Managing numerous logins and passwords, as well as having to go through the AWS setup process for each account can be onerous. Also, AWS

tags are applied only to individual accounts.

When using AWS tags beyond application and project names, a tagging schema needs to be created and controlled. Mistakes in tagging can

cause a resource to be unassociated to applications and projects in the cost model. In addition, some AWS resources cannot be tagged, requiring

a different cost distribution scheme to be created and maintained to allocate those costs across the enterprise.

With limitations to account-based and tag-based grouping, the tracking methodology needs be aligned to the structure of the environment(s).

For environments where the resources are 100% dedicated to a specific workload, grouping by AWS account works best. Using AWS tagging is

ideal for environments where you need to share resources across multiple workloads.

Companies that want a more scalable approach for managing AWS accounts and tagging should consider solutions from third parties like 2nd

Watch. 2nd Watch 2W Insight enables grouping of tags across AWS accounts, and provides tools to track and analyze cloud costs by cost center,

business unit department etc. 2W Insight can be purchased by contacting us at www.2ndwatch.com.

About Apptio’s Cost Transparency Application

Apptio Cost Transparency is a SaaS application for managing the cost of IT by combining financial data from a company’s general ledger(s)

with IT asset and operations data from any management system. This results in the creation of an integrated understanding of what is driving

operating expenses and capital investments and how they are aligned for value. Cost information is made actionable with role-based views into

portfolios of projects, applications, and services; the cost and consumption of these portfolios by each business unit; the foundational costs

of vendors, labor, and infrastructure; and the quality of data from your management systems. Visit http://www.apptio.com/applications/cost-

transparency to learn more.

Apptio is the leading provider of cloud-based Technology Business Management (TBM) software that helps CIOs manage the business of IT. For more information, visit the Apptio website or the Apptio blog at www.apptio.com.

2nd Watch is an enterprise workload management provider that helps companies accelerate data center capacity growth through adoption of the public cloud. The company’s public cloud-native services and tools implement and automate critical workload management processes including migration, procurement, provisioning, operations, financial management, and governance. 2nd Watch has helped hundreds of customers increase agility and lower operation costs by shifting workloads into more than 75,000 instances in the public cloud. The venture-backed company is headquartered in Seattle, Washington. To learn more about 2nd Watch, visit www.2ndwatch.com or call 1-888-317-7920.

Documents

Analyzing Cloud Costs in a Standard IT Cost Model€¦ · In order to execute a tagging strategy, you will need to know details about the AWS resources that are in-scope: • AWS