Innovations Big Data

8/12/2019 Innovations Big Data

1/30


2/30I N N O V A T I O N S T H A T M A T T E R


3/30B I G D A T A

CONTENTST H E N EXT F R O NT I E R O FP U B L I C S E C T O R I N N O V A T I O N : B I G D A T A 3

C i t i e s a s C e n t e r s f o rI n n o v a t i o n : L o u i s v i l l e , K y 6

Building the Data-Driven Culture: LouieStat

Data Analysis Improves Emergency Health Services

T h e N ew N o rm a l :H ow D a t a D r i v e s Y o u r M i s s i o n F o rwar d 10

T h e U . S . A rmy s B i g D a t a P r o b l e m :A S e a o f D a t a b a s e s 12

The EMDS Solution: Rapid, Holistic, and Visual EMDS as an Evolving Process

A B i g D a t a I m p e r at i v e :H a v i n g a F l e x i b l e D a t a I n f r a s t r u c t u r E 16

MAK I N G B I G D A T A S T I C K A T Y O U R A G E N CY 18

Understanding Governments Big Data ChallengesOur 8 Best Practices for Big Data Analysis

H ow I n -M e m o ry C om p u t i n g i s C h a n g i n gH ow We T h i n k A b o u t B i g D a t a 24

G o v L o o p s B i g D a t a C h e a t S h e e t 25

A b o u t G o v L o o p 27

A c k n owl e d g em e n t s 27


4/303 I N N O V A T I O N S T H A T M A T T E R

THE NEXT FRONTIER OF PUBLIC SECTOR INNOVATION //

Although we are living in historic times,todays digital age holds many parallels to the

industrial revolution. During that era, there were

countless breakthroughs in manufacturing, cre-

ations of new products, workforce changes and

movements to create unions to protect workers

rights. These advancements were all shaped and

sculpted by the technological advancements of the

18th and 19th centuries.

We also witnessed the development of new gov-

ernment regulations and the emergence of new

markets, propelling america to the status of global

economic leader. Now we are in the midst of

another technological era, which is reshaping the

cultural and socioeconomic fabric of society. As we

sit on the cusp of remarkable innovations, we must

remember that this time, modern innovations are

powered by data.

Today, governments are using data to improve

our standard of living. This is certainly not only an

american phenomenon. Governments worldwide

have recognized that by leveraging data, they canimprove services to citizens and gain a competitive

advantage in a flattening world.

This report is part of govloops innovations that

matter series, in which we explore top innovations

that truly matter and how to make them stick in

your agency. Well explore how organizations are

using big data technologies and solutions to trans-

form their organizations. Weve talked with thought

leaders and experts in government and industry in

order to understand the best practices to leverage

big data as a mission-centric tool, driving innova-

tions for the public sector.

Govloop reports are designed to empower you and

give you the tools to excel with big data. Heres

what youll find in this report:

A local government spotlight showing howthe city of louisville, ky., Uses big data toimprove performance management.

A federal government case studyhighlighting the armys enterprisemanagement decision support program.

Industry insights on the current big datalandscape.

8 Strategies and best practices for smart big

data adoption and analysis.

Govloops big data cheat sheet.

Understanding big data will turn data into insights

and transform your agencys operations. Now is

the time to take charge of your data and learn the

best practices and methods to modernize your

agency. We understand that innovation is an intimi-

dating word. But were here to help. Lets start with

learning about the effect that big data analysis is

having across government.

B I G D A T A


5/30B I G D A T A

Big data analysis offers public-

sector institutions an unprec-

edented opportunity to transform

themselves through big data

analysis. But first they have to

understand and effectively in-

terpret big data. Many agencies

are struggling with that because

each has defined and invested

in big data analysis differently.

Across government, agencies

need software that can quickly

and securely unlock insights from

their information.

We can start by defining big

data as data that arrives in such

enormous quantities, at such a

fast rate and in such a diverse

array of mediums that it is impos-

sible to understand or leverage

through traditional methods.

Similarly, a big data problem

describes a situation in which

you are not able to use your data

to accomplish a goal or deliver

a service because one of those

characteristics (volume, velocity

or variety) prevents you from cap-

turing its value within a predeter-

mined window of opportunity.

This is just a starting point to

define big data. The real issue at

hand is understanding what dataanalysis can do for your agency.

For the public sector, the key is

to appreciate how big data has

the power to drive changes to or-

ganizational workflows, improve

productivity and efficiency, and

reduce costs and overhead.

A recent GovLoop survey of 256

public-sector employees ex-

plored the benefits of analytics

and big data in government. The

survey found that:

To fully leverage big data, leaders

face multiple challenges. Respon-

dents noted some of the chal-

lenges in our survey:

These challenges are explored

throughout this guide, along with

strategies and best practices to

avoid roadblocks to adopting big

data programs. Although chal-

lenges exist, many agencies have

moved to adopt big data pro-

grams. Our survey highlighted

four ways:

Additional benefits were noted,

such as improved morale and

communication between groups.

One respondent said big data

analytics shows the best return

on investment, benefits all stake-

holders and drives innovation.

The message is clear across gov-

ernment: In order to understand

the power of big data, agencies

need to invest the time to define,

learn and capitalize on informa-tion. Our report is the first step

to encourage the sharing of best

practices, use cases and informa-

tion sharing across government.

To start things off, we learned

some remarkable insights about

performance measurements from

the City of Louisville, KY.

Understanding Big Datas EffecT

on the Public Sector //

cited improved efficiency& productivity

said big data drives

improved decision-making

cited transparency &

accountability

said they use big data

to manage resources,budgets & controls costs

said big data helps control

waste, fraud & abuse

Performance

Analytics

Predictive

Analytics

Business

Intelligence

Financial

Analytics

Lack of organizational support

Unclear mission and goals

Lack of clarity on metrics

Data governance

Data locked in legacy systems


7/30B I G D A T A

C I T I E S A S C E N T E R S F O R

I N N O V A T I O N :

L O U I S V I L L E , K Y

Cities are already complex ecosystemsto manage. Today local government leadersare faced with dwindling budgets, deteriorating

infrastructure and the need to attract and retain

business for economic growth. We can confidently

say that local government administrators are chal-

lenged like never before. Yet, many governments

are moving quickly to transform their cities into

centers of innovation and reigniting service deliv-

ery by capitalizing on their most important com-

modity: data.

Faced with unprecedented challenges, cities no

longer have the option to do business as usual. In

Louisville, business is far from usual. Led by Mayor

Greg Fischer and the chief of the Office of Perfor-

mance Improvement (OPI), Theresa Reno-Weber,

Louisville has positioned itself to use data in trans-

formative ways. Through their vision and leader-

ship, Louisville has become rooted in data and

offers a framework for data innovation for other

municipalities to follow.

City government has so much data across so many

different data systems we are missing opportuni-

ties, Reno-Weber said. Were missing synergies

between some of our departments and missing

clues into information that can predict issues that

we will need to address.

That challenge has led the city to not only improve

data sharing across agencies, but to create open-

data strategies. In doing so, the city has created a

platform for civic innovation for citizens, business

and developers.

We are excited about the potential for both innova-

tion and economic growth as it relates to making

data more open and available to the public, Reno-

Weber said. Weve already seen some of that withour open-data push.

Louisville leaders have defined data as an impera-

tive for innovation and as the engine for economic

growth. As a pledge to the citys commitment,

Fischer signed into law an executive order mandat-

ing that data is open by default. The policy covers

many of the Open Data Policy guidelines from the

Sunlight Foundation, a nonprofit that is working

toward greater government openness and trans-

parency. When the executive order was announced,Louisville became one of the first municipalities to

codify the open by default provision.

In this section of our report, we highlight two in-

novations that matter from Louisville: highlighting

the LouieStat program and how they have city has

leveraged data to improve medical response times.



Building the

Data-Driven

Culture:

LouieStat //One of the premiere datainitiatives has been theLouieStat program, whichlaunched in 2012. Administered

by Reno-Weber, it was designed

to help citizens understand the

effectiveness and efficiency of

government programs by provid-

ing answers to three fundamen-

tal questions that every citizen

should know and have easy

access to:

What are the key servicesMetro Governmentperforms?

How well is Louisville MetroGovernment performing?

How can Louisville MetroGovernment performbetter?

The program brings Louisville city

leaders together to discuss con-

sistent metrics and benchmark

success within their department.

These insights take in various

data streams to guide success

and have built a culture of contin-uous improvement. There have

been a lot of opportunities that

we have identified to eliminate

waste and improve the tactical

way we deliver services for citi-

zens in the city, said Reno-Weber.

One example is the work that has

been done in regard to Louis-

villes emergency management

services. (See below.) Another

comes from the Public Works and

Assets Department. Their mission

is: To deliver superior customer

service, efficiently manage solid

waste and recycling, proactively

maintain and enhance city facili-

ties, assets, infrastructure and

Metro fleet, initiate and support

progressive environmental and

energy conservation programs

and champion innovative busi-ness practices.

On LouieStat, the data that is ag-

gregated is:

Dollars spent on overtime

Overtime hours paid

Hours lost due to workrelated illness and injury

OSHA recordable injuryrate

Employees with high sickleave consumption

Hours not worked

LouieStat also pulls data from

the Office of Management and

Budget, Louisville Zoo, HumanResources and Public Health and

Wellness. This resource provides

a snapshot of the citys perfor-

mance, and links together data

from various sources to improve

accountability and transparency.

OPI staff meets with department

leadership teams to regularly de-

fine and assess key performance

indicators. This focus allows

decision-makers to know what

results they are trying to achieve

and to measure success. Next,

OPI works to develop bench-

marks and spot areas of improve-

ment. Key metrics are routinely

evaluated to track if it has already

met its goal, approaching goal

or off goal. Additionally, every six

to eight weeks department leads

provide a report to the mayor that

highlights performance.

This initiative was inspired by the

work of Baltimores CitiStat and

Marylands StateStat. Reno-Weber

shared one specific example of

success from LouieStat, highlight-

ing work done to improve emer-

gency health services below.


9/30B I G D A T A

Data Analysis

Improves

Emergency

Health Services //Every municipality is facedwith the challenge ofimproving emergencyhealth services.Local governments invest

significant time understanding

how to build more efficient

emergency relief systems

without increasing budgets.

Reno-Webers team decided to

explore how to reduce the time

emergency medical services

(EMS) staff spent at the hospital

after dropping off a patient.

Any reduction in time would

increase efficiency and allow

staff to deliver emergency health

services more effectively.

One of the things we were look-ing at was: How do we get more

efficient use out of the ambulanc-

es that we already have?, said

Reno-Weber. With large expenses

tied to increasing ambulance

fleets and staffing, her team was

tasked with finding an innovative

solution to increase ambulance

pick-ups.

This was no easy task, sincethere are a series of protocols

that EMS staff must follow when

they arrive at the hospital. They

must fill out paperwork, clean

and disinfect the ambulance, and

prepare it for the next departure.

In a metro area as large as Louis-

ville, every second counts to get

EMS back on the road and as-

sisting citizens in need of health

services.

To discover new efficiencies, the

city decided to observe the drop-

off process at one of the busiest

hospitals in Louisville. The team

took notes and collected data,

breaking down each process.

We put someone in the emer-gency room tracking the time of

the crews and figuring out what

the different process steps were,

said Reno-Weber.

Within the first three months

of observing and analyzing the

process, EMS drop-off times de-

creased by 5 minutes. We said,

Theres definitely an opportunity

to improve, Reno-Weber said.

After analyzing the data, the

team set a new data standard

in place. The city now required

that ambulances must leave the

hospital within 30 minutes of

dropping off a patient. If that time

was to be extended, crews must

notify a supervisor.

Reno-Weber conducted the study

between September and Decem-

ber 2013. Today, crews are leav-

ing the hospital in 30 minutes or

less 90 percent of the time. That

is basically the equivalent of plac-

ing two additional ambulances on

the street, she said.

Compared to the same time

frame in 2012, the city of Louis-ville was able to deliver 18,000

more patients to the hospital.

This is an example of how a com-

mitment to understanding how

data can improve services within

a city government.

The Louisville case study shows

the power of understanding data

and turning data into insights.

Powered by data, Louisville hasimproved transparency, citizen

engagement and quality of the

services it provides.

You can follow

Theresa Reno-Weber at

@RenoWeber on Twitter.


10/309 I N N O V A T I O N S T H A T M A T T E RLearn more at www.cloudera.com


11/30B I G D A T A

The New Normal:

How Data Drives Your

Mission Forward //Today, government agencies are lookingat new ways to transform their agenciesthrough big data analysis.

As Joey Echeverria, Chief Architect, Cloudera

Government Solutions, said, Big data presents a

lot of opportunities to change the way government

agencies and organizations manage their business.

For government, big data is forcing leaders to think

about data analysis as a way to move their mission

forward, rather than as a by-product of their service

delivery efforts.

This is a new way of thinking for government, as

Echeverria noted, Government never really looked

at data as something as core to what it is trying to

accomplish, that is the real thing that big data is

changing. Its giving governments an opportunity

to see that they can integrate data and information

into their everyday processes. The integration of

data into process is essential for agencies, facilitat-

ing new insights, findings and better decision mak-

ing for organizations.

To help agencies integrate data and information,

Cloudera powers the enterprise data hub. An enter-

prise data hub is one place to store all data, for as

long as desired or required, in its original fidelity; in-

tegrated with existing infrastructure and tools; with

the flexibility to run a variety of enterprise work-

loads together with the robust security, governance,

data protection and management that organizations

require. The advantage of an enterprise data hub

is that you can bring together any kind of data and

access it in ways that makes sense for your agency.

It purposely tries not to be prescriptive about how

you are going to end up accessing your data. This

means you end up with the right techniques and

framework for your specific needs, without being

locked in to a pre-defined solution. Another key

benefit is the ability to centralize your data manage-

ment strategies.

Having data in a central location provides many

benefits, especially in terms of accessing informa-

tion and collaborating across your agency. Echever-

ria provides additional insights on the benefits, as

he said, Having one place to store all your data is

going be far more inexpensive rather than duplicat-

ing your data and storing over numerous tools. Andthats really where our platform and offerings differ

from whats been available in the past.

Not only does centrally storing data allow organiza-

tions to be fully integrated, but also information-

driven. By capturing data from web portals, web-

sites or from various ways government provides

services, agencies are creating a framework to

understand citizen needs. Capturing that data is just

the first step, the second step is using that informa-

tion to improve service delivery.

If you are providing services to your citizens

through portals or websites, you should be har-

vesting the logs and the interactions that those

websites generate. This information should flow

back into feedback on how you model and run your

agency, said Echeverria.

Like many technology adoption projects, the IT

deployment is only one stage. To have success

with big data, organizations need to think about

the changes to culture. Cloudera provides pro-fessional consulting services to help meet these

objectives.We work with our customers and clients

so that they learn our best practices, not just for

how to use technology, but how that technology

can be applied to their specific problem, said Ech-

everria.

Big data takes time and commitment. A best prac-

tice is dont go into a big data project with the idea

of revolutionizing your agency on day one, said

Echeverria. Government leaders must build consen-sus and support, and cast a vision of how big data

can impact their agency.


13/30B I G D A T A

As mentioned in the introduction, big dataproblems usually involve more than just

juggling petabytes of data.In its most essential form, a big data problem is the

inability to fully capture the value of data within a

predetermined window of opportunity. This is most

often due to the size, speed or format of the data

collected.

The U.S. Army had one such problem. Its current

inventory of information technology systems is

more than 3,600, many with unique access require-

ments. These systems are used to track 10,000 op-

erating units, which are comprised of more than 1.1million soldiers. They also track millions of pieces

of equipment -- from tanks to batteries -- and thou-

sands of training events. The Armys data infra-

structure is large, complex and dispersed among

stovepiped and legacy systems, and it operates

under various data owners and governance rules.

As a result, the process of gathering data to inform

decision-making about deployment and assess unit

readiness was cumbersome and time-consuming. It

also meant that leaders were basing their decisions

primarily on information within their particular spe-cialty area, often missing the rich layer of supple-

mental information available on other systems.

The response to this problem was the Armys

Enterprise Management Decision Support (EMDS)

system. This initiative is designed to serve as a de-

cision support tool for senior leaders. Its goal is to

provide a single access portal for culling pertinent

data from certain key databases.

What we do is reach across those systems, find

the most relevant information for our customersand bring that into one place, former EMDS Divi-

sion Chief Lt. Col. Bobby Saxon said. So it is one

username, one password, one location to go to, so

they can see critical information about personnel,

on readiness, on equipment, on installations and

on training.

In this way, EMDS served as both an aggregator

and curator of information.

T h e U . S . A r m y s

B i g D a t a P r o b l e m :A S e a o f D a t a b a s e s



Understanding unit readiness is paramount for the U.S. Armys operational success

The EMDS

Solution:

Rapid, Holistic, &

Visual//Now that the EMDS system per-

forms the front-end work of piec-

ing together data from disparate

database systems, the process

of leveraging the data is more

streamlined and timely. However,

speed was not the primary mo-

tivation for EMDS. Saxon noted

that making decisions under very

aggressive time-frames is simply

par for the course in the Army.

Instead, EMDS provides a crucial

additional layer of understanding

to this existing capability.

It takes not only the information

about the personnel community,

but it lays on top of that the infor-

mation for the equipment or the

training, he said. Then, we array

across the Armys Force Genera-

tion model [the Armys equivalentof a supply-chain management

system], so they see much more

of the picture.

In other words, decision-makers

have a much broader, compre-

hensive understanding of a given

situation, which allows them to

see the wider context in which

they are making their decisions.

In a sense, EMDS provides a

holistic view of the situation,

while also providing access to

the granular details in relevant

specialty areas when necessary.

Given these dual, somewhat op-

posing objectives, visualization

was an integral design compo-

nent. Prior to EMDS, information

came in a variety of forms, such

as spreadsheets, charts and

graphs. When you multiply this

output by the multitude of dif-ferent systems accessed, users

were left with the formidable task

of managing this unwieldy collec-

tion of information.

To contrast, EMDS has a number

of dashboards specifically de-

signed to illustrate the situation

in a friendlier, easy-to-understand

format. Perhaps more important-

ly, this format is now standard-

ized across the enterprise, which

provides users in different areas

access to the same information in

a common format.

Finally, EMDS provides users with


15/30B I G D A T A

the opportunity to discover new

or previously unknown pieces of

information, thereby adding an

additional layer of comprehen-

sion. The goal is to ensure that

users are able to focus on their

primary objectives, rather thanpiecing together ad hoc reports

by themselves.

EMDS as an

Evolving

Process //

When the EMDS program waslaunched in 2008, the Army

encountered a few challenges.

For one, the surrounding big data

community, both proprietary and

open source, was nowhere near

as advanced as it is today. More

important, however, were the

cultural barriers to sharing infor-

mation across the enterprise.

There was not a clear under-

standing from stakeholders about

how valuable all this information

in one place could be, Saxon

said. We still deal with some of

these challenges today. Informa-

tion is king, and if you are some-

one who has information that

other people may not have, it

may give you a bit of a leg up.

The Army is still working through

these cultural barriers, but the

progress thus far has been trans-

formative. Senior leaders are

continuing to push that all data is

the Armys data and that it should

be there for all of us to use to

make decisions, Saxon said.

EMDS also evolved from solely

providing a near-real-time snap-

shot of the present into a product

that provides historical and pro-

jected pictures in addition to hav-

ing the foundation for a planned

predictive analytical component.

The mind-set now is the belief

that we have access to nearly ev-

erything. Now people are starting

to think, Well where is the data?

How do I get my hands on it?

Previously the thinking was that

there was no way we even know

what that information is, Saxon

said.

Thus, EMDS has been a revolu-

tion in IT business process flows

but also in organizational think-

ing, which is the hallmark of a

successful big data solution.

Former EMDS Division Chief Lt. Col. Bobby Saxon


16/30
http://www.marklogic.com/what-is-marklogic/organizations-around-world-use-marklogic-mission-critical-applicationshttp://www.marklogic.com/what-is-marklogic/enterprise-nosqlhttp://www.marklogic.com/http://www.marklogic.com/


17/30B I G D A T A

A Big Data Imperative:

Having a Flexible Data

Infrastructure //Discussions about big data in governmentare generally dominated by the volume,velocity and variety of the data beingproduced by our agencies today.

Yet, what is often missing is a matching discussion

about the changing requirements for the agencys

data infrastructure. This infrastructure is just

as essential to improving the way services are

delivered to constituents as the data itself.

Kevin Shelly, VP of MarkLogic Public Sector, spoke

of the changing data landscape, The traditional

way to store operational data is in relational da-

tabase systems, which have been around for 30

years. While they are very good at what they do,

their specialty is transactional processing on struc-

tured data, and they havent really changed with

the times.

Today, government agencies understand the op-

portunity that big data presents. Many agencies are

looking to capitalize on the various data streams to

which they have access, whether it is social media,video, photos, documents, mobile, or other forms

of structured and unstructured data. It is this mix of

unstructured and structured data that presents a

challenge for organizations. How can we glean new

insights from this data?

Governments also recognize that it is no longer an

either-or proposition for structured and unstruc-

tured data. They need to capitalize on both forms of

data to drive innovation and meet mission need in

new ways. Shelly notes, Agencies need a databasethat can link structured and unstructured data so

you can leverage 100% of the data that is out there

to accomplish your mission or get closer to your

constituents.

To find value from structured and unstructured data,

agencies must invest in new generation databases.

For many agencies, this means looking at NoSQL

(Not Only SQL) databases. Relational databases

were developed primarily for use with structured

data, while NoSQL allows much more flexibility and

agility to collect, process and analyze a wide variety

of data sources. Also, unlike relational database

systems, No SQL databases allow you to ingest

the data without having to spend time and money

up-front developing a data model, hence you can

develop and deploy systems much quicker. The

reduced time to value is just one of the advantages

of the NoSQL approach.

I think there will be a very large shift towards

NoSQL databases, said Shelly. Ultimately what

MarkLogic can provide is a richer experience,

leading to improved mission capability and abil-

ity to provide enhanced services to constituents.

MarkLogic can operate on premise or in the Cloud,

ingest all data types, provide extensive search ca-

pabilities, and deliver the data, to various devices,

whether its a mobility platform, laptop or desktop

computer, said Shelly.

For many government agencies, now is the time to

start investing in big data, and thinking about how

data can drive a more efficient and effective agen-

cy. Data will continue to grow at a rapid pace, and

as Shelly observes, If the haystack keeps getting

bigger, it just takes more work to find the needle.

Therefore, now is the time for smart investments

such as an enterprise-grade NoSQL database withintegrated search and application services that can

handle all of the data, not just some of it.

Yet, navigating the big data frontier does not come

without challenges. Agency leaders are challenged

to clearly navigate the cultural changes to tech-

nology. As Shelly noted, Most of the challenges

are cultural; the MarkLogic technology is trusted,

proven, and innovative. Big data represents a cul-

tural change, where people need to look at things

differently and do things differently than their pre-decessors. This can mean everything from a new

business process, a change to workflows, and even

changing the way government delivers services.

Now is the time to invest, and re-imagine how

services are delivered and missions accomplished.

This process can start by assessing your data infra-

structure, and ensuring you have the right database

to support your agencys needs.


19/30B I G D A T A

Understanding

Governments Big

Data Challenges //Before we get to the best practices for big data

adoption, its important to set a foundation. Weve

identified four challenges from our research and

important trends to be aware of as you begin to

deploy big data challenges.

1. Large Capital Investments

The primary big data challenge for governments

is that most agencies have spent an enormousamount of capital on existing IT infrastructure ,

most of which was implemented long before big

data was even on the radar. This often means that

governments have the ability to capture large

quantities of data, but they dont have a feasible

way to process it in a timely manner. In other cases,

these systems were not built for the variety of data

sources currently available for capture.

M A K I N G B I G D A T A

S T I C K A T Y O U R

A G E N C Y

Former EMDS Division Chief Lt. Col. Bobby Saxon



2. Data Rests in Silos

Perhaps most crucially, many

systems exist in their own data

silos, without a clear method for

extraction and integration. This

challenge is coupled with the

fact that investing in big datasolutions can mean adopting

technologies that are potentially

expensive, untested, less secure

and developed by a third party

outside the government agency.

3. Finding the Needle in theHaystack

Governments collect so much

data -- and have such a variety of

sources and mediums to choose

from -- that it is difficult to pin-

point value. Even in the private

sector, it is common for corpora-

tions to find the majority of value

in a minor percentage of the

total data they collect. Therefore,

there is a real risk of investing in

an analytics apparatus that col-

lects and processes redundant

sources of information, especially

since existing systems operate

in isolation. At the same time,

requirements and use cases may

evolve, with capabilities unlock-

ing value previously inconceiv-

able without new advancements.

This presents a challenge for gov-

ernments to figure out the smart-

est and most efficient way to get

the most out of their investments.

4. No One-Size-Fits-AllSolution

There really is no one-size-fits-all

big data solution, just as the chal-

lenges and opportunities for data

use vary for each agency. This

presents potential adopters with

the challenge of finding the right

solution with the right attributes

while dealing with limited capital

and space to experiment.

Our 8 Best

Practices

foR Big Data

Analysis //Given these challenges, wehave identified eight bestpractices to help capturethe most value whileexpending the minimalamount of resources.

1. Executive LeadershipThe shift to big data from tradi-

tional data analytics may require

more of a revolution in thought

and organization than in the

technical solution itself. For one,

big data operations cannot ex-

ist within a silo. They must beintegrated across the enterprise,

with a common design layout

and operating procedures, as the

EMDS example illustrates. This

requires leadership and vision

that go beyond individual use

cases into a platform for future

integration at all entry points. Ad-

ditionally, there may be friction

among internal units that must

be addressed for proper insti-tutional cohesion. The Armys

senior leaders are still transform-

ing the culture to ensure all data

is shared across the enterprise,

which is vital to the success of

EMDS.

Finally, there will be costs -- fi-

nancial, time, labor -- so any

successful implementation will

require a leader to ask difficult

questions to ensure the invest-

ment is worth the cost and to see

the process through during peri-

ods of complication or difficulty.

2. Business beforeTreasure: Use Cases ComeBefore Technical Solutions

The key here is that the technical

solutions should be matched to

business requirements and not

the other way around. It is very

easy to become seduced by the

technology before clearly de-

fined use cases have been found.

This places the organization at

risk of wasting valuable time andresources implementing a broad

program with only marginal

value added. Therefore, it is vital

that data owners consult with

stakeholders, users and execu-

tive leaders to come up with very

specific business requirements or

problems.


21/30B I G D A T A

Technology is important, but the

people are critical, Saxon said.

It is also important that program

managers integrate their plans

into the larger strategic objec-

tives for the organization.

A best practice and lesson

learned from Louisville is that [big

data] needs to be tied to your

strategic vision or your strategic

goal or whatever it is you are

trying to accomplish in your area,

Reno-Weber said.

3. Know Thyself: DefineInitial Use Cases

Now that the emphasis is on

potential use cases, a clear iden-

tification of the way big data is

specifically going to address your

data problems must be estab-

lished. Think of the three Vs we

mentioned earlier in this guide:

volume, velocity and variety.

There are different solutions for

each of these problems, so the

best practice is to find your defi-

nition of what a big data solution

means to your agency.

Ours isnt as much a volume

problem as it is a difficult to

understand problem, Saxonsaid. It is the ability to answer

questions that were previously

beyond reach.

Therefore, the Army peeled away

at the broader, general defini-

tions of a big data solution and

crafted its own definition based

on its specific needs.

But how is this done? First,

clearly establish your realizationcriteria (such as reduced trans-

action times), business require-

ments (such as security require-

ments) and performance metrics.

It is better to start small and

specific, building on your initial

deployment than to try to capture

everything at once.

The questions you ask mat-

ter. There is a lot of data in city

government and in government

in general. You could run yourself

ragged trying to analyze all of

it and make it all good quality,

Reno-Weber said. The key is that

you should identify a clear proj-

ect or goal to solve, such as the

example of reducing ambulance

time at the hospital.

Although every agencys require-

ments are different, it is possible

to learn from their experiences to

help identify where big data solu-

tions may be applicable to your

organization.

4. Leverage ExistingResources & then Augment

In most cases, good big data so-

lutions are built on an already ro-

bust IT infrastructure. This allows

organizations to initially make

relatively narrow and precise

upgrades to address big data

problems without remaking their

entire technological enterprise.

It is helpful to think of this pro-cess like upgrading components

on your personal computer and

analyzing them for efficacy with

each phase. This is in contrast

to replacing your computer each

time a new innovation in process-

ing or storage is released.

Dont be limited by thinking

big data requires a huge infra-

structure and a lot of special-

ized tools, Saxon said. Start adatabase. Start accumulating

some of the data. You could even

use Microsoft Excel to start do-

ing data analytics to move down

that path. You will find nuggets

of information that will help you

sell your story and help you keep

moving forward.



Similarly, any new augmentation

should be able to do two things:

address the immediate data

problems and be easily scalable

to include future requirements for

later phases.

5. Integrate LegacySystems

The vast majority of existing

systems currently in operation in

government were not calibrated

to handle big data. As a result, it

may be difficult to quickly pro-

cess or extract pertinent data.

Also, these systems may not be

equipped to handle unstructured

data such as video feeds or socialmedia. This makes augmenta-

tions and upgrades a near ne-

cessity. However, integration of

older systems is essential to the

enterprise-wide analytical power

big data provides. Extra time and

consideration must be built into

program planning to allow for

proper field matching techniques,

data integration and processing

output times.

Reno-Weber said Louisville has a

number of systems for fire, health,

311, financial systems and human

resources, and her teams under-

stand the importance of sharing

and connecting these various

data streams.

You see the power of the data

when it is put in the hands of the

users and the operational teams,

she said. We ask, How can we

get our departments to be able

to access their own data and use

insights to best allocate resourc-

es? You cant do that if only one

or two people can get to informa-

tion.

6. Carefully Select YourVendors/Architecture

The explosion of advancements

in execution strategies and tech-

nological solutions makes this a

unique time in the relatively short

history of big data. In contrast to

just a few years ago, data owners

now have a wealth of options to

build out their big data infrastruc-

ture, from competing vendors to

a range of architectural options.

Therefore, it is vital that you care-fully select a solution -- and a

partner -- that best matches your

needs. In this case, it is best to

slow down to speed up. Taking


23/30B I G D A T A

your time at this critical juncture

will ensure future phases do not

require large-scale redesigns.

7. Focus on Governanceand Data Quality

Reno-Weber reminds us of theimportance of data quality and

governance strategies: Data is

not always going to be perfect,

but that should not stop you from

analyzing and working it, be-

cause the minute you start doing

that is the moment data quality

starts to get better.

She also notes the importance

of having consistency as to how

data is entered. The challenges

that I think a lot of us see is that

bad information in equals bad in-

formation out, she said. A lot of

the work that we have done over

the last two years has been to

get the quality of the input data

as good as possible, so what we

are analyzing on the back end is

valid as possible.

8. Have a Deep Bench:Train Employees to Serve inSupport Roles

Data is changing the roles and

responsibilities of the future

workforce. Having only a few

employees who know how to

derive insights from data will not

work. Instead, the capability must

extend across the agency.

You need to have a deeper

bench as it relates to people

who can access and analyze big

data in your organization, Reno-

Weber said. Agencies must place

an emphasis on training and

educating staff at all levels.

Leaping into the world of big

data, however you choose to

define it, is not a casual or easy

process. Following the bestpractices presented here will

help you take the right first steps

toward capturing value from your

data, which is vital for success.

Each step builds on the last, and

a careful plan and execution on

the front end can mean all the

difference on the other side.

Big data is changing the way

our government operates and istransforming communities. For

public-sector leaders, now is the

time to understand how to lever-

age data in new and transforma-

tive ways. Theres no denying

the power of data. Our report

has given you access to the case

studies, lessons learned and best

practices from big data leaders.

Now, its up to you to take this

information and bring your big

data knowledge into your agency.

GovLoop is always here to help.

Send us your questions, tell us

your case studies and lets workin partnership to transform the

public sector.

Louisvilles Chief of the Office of

Performance Improvement,

Theresa Reno-Weber


http://www.softwareaggov.com/


25/30B I G D A T A

How In-Memory Computing

is Changing How We Think

About Big Data //Government has been tasked tomodernize in a dynamic and quicklychanging landscape.

While most agencies have recognized the need

to leverage their data in transformative ways,

they still face challenges in building a sustainable

infrastructure for the future.

With shrinking budgets, agencies are trying to find

ways to modernize, but also be more cost effec-

tive. One critical way to accomplish this goal is for

more agencies to realize that there are technology

developments that will enable them to scale-up and

out while simultaneously leveraging their existing

infrastructure, said Michael Ho, VP for Software AG

Government Solutions.

One such IT development is the use of in-memory

computing. In-memory computing is a transforma-

tive technology that is already becoming a core

component of high performing applications within

government and with great results, said Ho.

In-memory computing is about achieving massive

scale within your existing infrastructure all while

supercharging the speed of your enterprise applica-

tions so you can tackle big data quickly. Fundamen-

tally, the challenge for any database environment

today is that with even the best built applications

hitting it, the amount of data that each application

user carries is growing exponentially. This is tax-

ing for your database because any time users do

anything, your app must do a round-trip to the data-

base, ultimately slowing it down and leaving usersstuck waiting. So what happens? Organizations

feel forced to continually buy more servers and

database licenses which are costly. In-memory com-

puting moves agency data sets to the applications

memory which means less work for your database

and more speed for your users. Ho said, What it

allows you to do is take any size of data, whether

its massive amounts or even smaller amounts, and

move that data closer to where the user ultimately

consumes it. This eliminates the need to wait for

the transaction gaps you get with normal data-

bases. What took hours or several minutes can take

microseconds with in-memory computing.

Ho believes that in-memory computing is the next

evolution of data storage and processing technol-

ogy. In-memory allows us to take all the systems

of records that you have in place, whether that be

your Hadoop cluster, data warehouse or existing

databases and applications, and speed them up to

help you meet the massive scale of data and de-

mand that you are going to receive from your end

users and enterprise, said Ho.

Modernizing does not need to be a scary proposi-

tion for agencies. Often, modernizing is as much

about learning how to leverage existing technology

as it is about investing in new technology. Ho noted

the benefits of in-memory computing to capitalize

on data. Thats where in-memory computing is re-

ally great, it allows you to complement your exist-

ing systems and blend the speed and scale to your

unique infrastructure, said Ho.

For public sector organizations, in-memory comput-

ing means that you do not have to start building

from the ground up. You dont need to re-design

your entire strategy from square one. Its abouttaking your existing investments and making them

more valuable by adding the proper pieces to

them, said Ho.

Ho believes that in-memory computing is changing

how organizations think about big data. Big data

as a notion is going to be more than just having

terabytes of data. Its going to be about helping

folks make the decisions they want to make, by

enabling them to see and analyze the data they

have, in the time window of relevance thats opti-mal. Thats where these technologies are focused,

said Ho.

To help organizations modernize, new technologies

will emerge that will help organizations to capital-

ize on their data. The future of big data is not about

size of data; its about speed and processing to

maximize value from information collected.



What is Big Data?Big data is characterized as data that arrives in

such enormous quantities, at such a fast rate and

in such a diverse array of mediums that it is impos-

sible to understand or leverage through traditional

methods. Similarly, a big data problem describes a

situation in which you are not able to use your data

to accomplish a goal or deliver a service because

one of those characteristics (volume, velocity,

and variety) prevents you from capturing its value

within a predetermined window of opportunity.

Why Does Big Data Matter?

Data is at the epicenter of the next technological

revolution. The organizations best poised to thrive

are those that treat their data as a valuable com-

modity a raw resource, similar to petroleum or a

precious metal. Big data solutions provide unique

opportunities to turn these raw resources into key

insights and ultimately transform the way your

agency does business.

G o v L o o p s

B i g D a t aC h e a t

S h e e tL o o k i n g t o g e t

s m a r t f a s t o n

b i g d a t a ? L o o k n o

f u r t h e r //


27/30


28/30


29/30


30/30

1 1 0 1 1 5 t h S t N W , S u i t e 9 0 0

W a s h i n g t o n , D C 2 0 0 0 5

P h o n e : ( 2 0 2 ) 4 0 7 - 7 4 2 1

F a x : ( 2 0 2 ) 4 0 7 - 7 5 0 1

Documents

Innovations Big Data