Best Practices in Siebel Crm Performance Management Monitor Measure and Manage the Enduser Experience

8/2/2019 Best Practices in Siebel Crm Performance Management Monitor Measure and Manage the Enduser Experience

1/16

Best Practices in Siebel CRMPerformance Management:

Monitor, Measure and Manage the End User Experience

e

xecutivewhitepaper


2/16

2Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience

Contents

Executive Summary

Why End User Performance Management is needed to optimize

Siebel CRM apps

ROI Isnt Real Until Its Realized

The ROI impact of issues with adoption and efficient and effective use

Traditional Performance Management Approaches Arent Enough

Monitoring system and application health is no longer enough

for service of business apps

Overview of Knoa Experience and Performance ManagerA review of the architecture and functionality of Knoa EPM

Knoa EPM Use Cases

Knoa EPM delivers results to the entire application team. This section

details use case examples compiled from Knoas customer base

You Can Manage What You Measure

The impact that real end user experience and performance metrics

can have on the application management team

Could You Benefit From Knoa EPM?

Four questions to help you evaluate the potential value of Knoa EPM

for your Siebel deployment

Knoa Software is

leading the performance

management industry

with end user experience

monitoring solutions

that reach beyond

infrastructure and

application performanceto the monitor, measure

and manage how end

users are utilizing the

application to optimize

business process

execution.

3

5

6

7

9

14

15


3/16


Executive Summary

CRM solutions, like Siebel, automate complex business processes where high

levels of application performance are crucial. Yet many frustrated execs find

that despite the small fortune they have spent on products to manage servers,

networks and applications, the business constituencies still complain about the

performance, usability and availability of the Siebel CRM application.

What is often overlooked is that application performance is just one of three key

drivers of return on investment (ROI) from your Siebel application. An efficient

infrastructure is critical. Its important to optimize the Siebel application for your

own unique business environment, user needs and usage patterns. But, its all

for naught if you do not build and sustain adoption and efficient and effective

use of the Siebel application by your end users.

How can you tell how if your Siebel solution performs for each person who

uses it? How can you identify which users have difficulty executing which

transactions? What does it mean when your end users say the system is slow?

How can you pinpoint the actual issues with the real data you need to take

corrective action? Knoa can give you the answers.

Enterprises with mission critical Siebel deployments do not have accurate

measurements of adoption, efficient use, and effective use of the application by

the end users. Without these measurements, companies are unable to identify and

resolve critical issues that impact the users ability to optimize business process

execution. The result is significantly reduced ROI, and significantly increased cost

of application support.

Knoa Software is leading the performance management industry with end

user experience management solutions that reach beyond infrastructure and

application performance to monitor, measure and manage how end users are

utilizing the application. Knoa customers not only know if core enterprise

applications are delivering an acceptable user experience, but also, if

application users are executing key processes effectively and efficiently

the key to achieving business value and ROI.

An end user performance

management package

from Knoa Software is

capturing the CIOs

attention for its ability to

boost worker productivity

and optimize application

performance. KnoasExperience and

Performance

Manage-ment suite lets

IT departments measure

response times, distinguish

between system and user

error and monitor work

flows all in real time

Brian Gillooly

Optimize IT


4/16


Knoa Experience and Performance Manager (EPM) monitors enterprise

applications from the perspective of the end user to capture the actual end user

experience. Knoa EPM captures transaction response times as well as ecosystemand application errors along with a time-synchronized workflow from all

users, at all locations, at all times, across the entire application. This information

is used for alerting and proactive remediation of end user issues (whether they

are system errors, user errors or performance problems). Knoa EPM also

analyzes and aggregates information to provide actionable insight into process

compliance, training requirements, adoption issues and a number of other

management dimensions.

Knoa EPM provides a complete picture of end user experience by focusing

on both the experience the end userreceivesfrom the application and the

performance the end userachievesworking with the application. You can think

of Knoa EPM as the third leg to the performance management stool

infrastructure, applications, and now, end user metrics. Think about it! The

infrastructure is monitored and measured and optimized; application execution

across that infrastructure is monitored and measured. In a recent study

conducted by Forrester Consulting an overwhelming 87 percent of IT executives

either strongly agreed (50%) or agreed (37%) that the best way to manage the

service of business critical applications is by measuring performance at the end

user level and rapidly resolving significant performance problems.

Clearly, theres a need for a change. In fact, the Forrester Consulting report

found that change is imminent. Sixty seven percent of the surveyed

organizations are either much more focused (27%) or more focused (40%)

on the end user experience than they were two years earlier.

Forrester concluded that best practices companies are intent on

collecting metrics on the end user experience.

For all the investment

in application

management solutions,

nearly two-thirds of

companies (66 percent)

rely upon help desk calls

as the main indicator

that end users areexperiencing response

time and/or availability

problems.

Forrester Consulting


5/16


ROI Isnt Real Until Its Realized

The initial investment in CRM applications is high. Software licensing costs can

range from $1,000 to $2,800 per end user. Implementation services are 2-3

times the software costs and there is investment in back-end infrastructure.

Ongoing costs are high, as well. Software maintenance averages 20%, annually.

Gartner reports that 60% of ongoing costs of a typical deployment are for

support staff. ROI is only realized from this investment if the end users adopt

the application and use it efficiently and effectively to improve the execution

of core business processes which is very difficult to achieve.

Consider this all too common scenario that afflicts many CRM deployments.

Months after deployment a high percentage of the functionality is not being

used. End users complain that the application is slow and doesnt work.

Some users find the software cumbersome to adopt into their routines andcreate their own non-compliant work-arounds. Employee turnover slowly

erodes the user base proficiency.

Flying Blind

Heres how one of Knoas customers described the problem that led them to

deploy Knoa EPM.

User adoption of our CRM application has been stalled by a number of factors

affecting field productivity. There are complaints about poor response time, remote

users cant access the system, and its not clear if there are system problems or

if they are just using the wrong procedure. And across the board there are

complaints that things take twice as long with the new system. In the absence of

comprehensive and representative data, many discussions regarding applicationperformance are based on reported user impressions, limited visual observations,

and occasional stopwatch timings. This has resulted in extrapolations about the

performance of the whole system that are inaccurate, and lead to decisions that

are not in the best interest of the business.

Enterprises with mission critical CRM deployments do not have accurate

measurements of adoption, efficient use, and effective use of the application by

the end users. Without these measurements, companies are unable to identify,

and resolve critical issues that impact the users ability to effectively execute.

The result is significantly reduced ROI, and significantly increased cost of

application support.

One Version of the Truth.

Knoa EPM gives the

enterprise real metrics for

adoption, efficient and

effective use of the

application. With end user

metrics, companies areable to identify and

resolve critical issues that

impact the users ability

to effectively execute, and

hamper ROI realization.


6/16


Traditional Approaches to Performance Management

Arent Enough AnymoreCRM applications like Siebel are complex. Its the flip-side of the flexibility they provide to meet a wide variety of customer

needs. Each Siebel customer has their unique business environment user needs, usage patterns, customizations, system

components, architectures and platform environments vary widely. In the face of this complexity, it is becoming increasingly

difficult for measurement and metrics taken at the infrastructure level (system, network or transaction management) to provide

a true perspective of the end user experience with the application.

In a recent study conducted by Forrester Consulting an overwhelming 87 percent of IT executives either strongly agreed

(50%) or agreed (37%) that the best way to manage the service of business critical applications is by measuring performance

at the end user level and rapidly resolving significant performance problems.

Knoa believes that the application is not performing from the end users

perspective whenever the end user is unable to effectively execute the keyprocesses required for their role. In our vision, application performance, from

the end user perspective includes:

Slow transactions

Cumbersome Navigation

System Errors

Non-intuitive user interface which causes excessive user errors

Knoa EPM goes beyond the response-time dimension to monitor and capture

metrics on the entire end user experience including system and user errors

as well as the user navigation and interaction with the application (workflow).

The monitoring capabilities of traditional performance management products provide a significant amount of insight

about application health (e.g. transactions that are failing; aggregate transaction response times, application availability).

Knoa EPM extends performance management to a whole new level by monitoring and managing the health of the end-user

population. Has the application been adopted? Are the end users using the application proficiently. Is the application enabling

the end users to efficiently and effectively? After all, adoption and efficient and effective use are the keys to ROI from the

application investment.

Snapshot of the End User Experiencewith online transactions(Source: USA Today, 2006)

Received Error Messages

Site Navigation Issues

Transaction Execution Issues

Login Issues

Content Issues

40%

37%

31%

31%

31%


7/16


Knoa Experience and Performance Manager

Knoa EPM gives the Siebel support team a comprehensive "fact-based"

perspective about the end user's experience with the application. Who's using it?

What's the response time? Who's getting errors? How many transactions are

being abandoned?

Knoa EPM is the only end user experience management solution that monitors

the entire end user experience from the perspective of the end user. Knoa EPM

monitors the Siebel application from the perspective of the end user to capture

the actual end user experience, as well as ecosystem and application errors,

and a time-synchronized workflow from all users, at all locations, at all times.

Knoa EPM uniquely captures a complete picture of the end user experience

and behavior: Actual User-Experienced Response Time for key system transactions,

such as navigating between application screens, save operations, execute

operations, etc.

Complete Quality of Experience, including system and application errors

and user created errors

Comprehensive perspective of application utilization which transactions

are used, in what sequence, for how long, etc.

Knoa EPM is based upon a passive monitoring engine that is deployed to

the client workstation along with a template specifically designed for the

target application. This client engine monitors end users interaction with the

application. Knoa EPM selectively captures the actual steps, and the sequence

of events in the users interaction with the application. In addition to this time-synchronized workflow, Knoa EPM captures a number of metrics around both

the applications performance and the end users performance:

All user errors

All system errors

Transaction response times

This information, called track data, is transferred to the Knoa EPM analysis

server which prepares a series of targeted analyses and reports. This

information is viewed through the Knoa EPM Console, which presents a series

of management dashboards and drill downs for analyzing and querying the

performance data.

The combination of

the user-base coverage

and application depth

provided by this

approach is tough

to beat. With this

information, companies

can manage theperformance achieved

by end users, as well

as the experience that

they receive.

Bernd Harzog

CEO, APM Experts


8/16


A Unique Architecture for End User Experience

ManagementKnoa EPM is based upon a unique architecture that was designed, from the

ground up for user-centric monitoring and experience management. Key

capabilities the architecture delivers are:

Automated Discovery

The out of box Knoa EPM solution automatically sees every Siebel transaction

and the steps that the user is taking before, during and after the transaction.

Knoa EPMs self-discovery capabilities eliminate, completely, the need for

customers to design a data collection strategy, define transactions to the system

or train the system to recognize new transactions. Automated Discovery means:

No up-front requirements definition or project scoping

Faster implementation using out-of-the-box templates lower overall cost of ownership

Adaptable technology to easily support the most aggressive release/update

schedules

Global Coverage

The out of box Knoa EPM solution gives you global coverage of the entire

Siebel application, across all instances and systems in the field. This global

coverage also gives you a comprehensive perspective on adoption and usage

across the entire user population.

Application usage

Transaction usage

Screen usage

Function usage

Active/idle time

User behavior

Non-Intrusive to Siebel Application

The Knoa EPM system does not insert tags or instrumentation into the Siebel

application, create any system events, or require any back-end processes

that impact application performance. The Knoa components do not require

integration with any components in the application landscape, whether front-

end or back-end.

Small Footprint/Low Impact

Knoa EPM uses a Seibel-specific template to selectively record the significantuser events that will produce actionable insight, without any discernable

system overhead or impact. It is this light operational footprint that enables

you to monitor and measure the user experience across the entire application

landscape.

The insight into actual

end user experience

gives us benefits across

the board, helping

target response time

issues, solve end user

problems faster, identify

usage and adoptionissues and even make

priority decisions on

ongoing application

investments.

Stuart Smith,

Director,

CRM Performance,

British Telecom


9/16


Knoa Use Case: Performance Management

Improve Siebel Application Performance

Excessive application response time is one of the most critical performance issues encountered by the users of todays complex

mission critical applications. Its also a huge support problem. With Knoa EPM you know the response time that was actually

experienced by the end-user for every transaction, for every user, every time.

A leading consumer packaged goods company had put millions of dollars over 3 years in a new Sales Force automation solution.

The number one complaint from users was long response times for key functions, such as creating and saving CRM records. The

QA/Performance team was only able to act on these performance issues once they escalated to critical levels.

The company implemented the Knoa EPM system to detect performance fluctuations for key functions in real-time. Using an

alert system that notified the IT team whenever response times in the field exceeded set tolerance thresholds, the team was able to

preempt the escalation of performance problems in many cases. In addition, by being able to isolate performance problems to

specific areas of the application, the team also reduced the time to resolve critical issues.

See the forest and the trees. Knoa EPM delivers results to the entire Siebel support team.

Here are some real use case examples submitted by Knoa customers:


10/16


Knoa Use Case:Application Utilization

Link investment and effort to business impact

With Knoa EPM you gain valuable, actionable insights into application utilization: Who is using the application and for how

long? Which areas and functions are most used? Which areas and functions are least used? What usage patterns are creating

strains on IT infrastructure? Knowing the usage profile for your Siebel application and your user base is the most fundamental

step towards application and user performance management.

The IT Support and Development team of a leading global financial services company received requests from their field users

to implement a costly and extensive functionality enhancement for their enterprise sales application. The IT team dedicated

considerable resources to implement this request and launch the new capability.

After the enhancement was implemented, using the Knoa application usage data, the IT Team discovered that few employees

actually used the new functionality. Based on this fact, and a detailed review of Business Operations based on Knoa data, they

made a decision to discontinue support. Eliminating the module from production saved the organization tens of thousands ofdollars in development time and resources.


11/16


Knoa Use Case: End User Proficiency

Drive efficient and effective use by managing errors

With Knoa EPM solution you can identify 100% of all user-experienced errors, whether they are caused by user actions or by

any of the application layers. Knoa EPM also gives you the context in which each error was encountered and the path that the

user took before encountering the error. This information is available in real-time for the end user support team.

By implementing Knoa EPM, the IT department of a leading computer services company realized that a large percentage of

the errors encountered by users were never reported to the Support Desk. In particular, a single error type, generated while

executing searches, accounted for 25% of all errors generated by the companys CRM application.

The productivity losses and infrastructure impact of repeated and failed queries executed by users amounted to thousands

of dollars each day.

Based on the Knoa data, the IT department promptly identified the issue, which was then addressed with a two-fold strategy:1) provide targeted user training in order to avoid the error, in the short run, and 2) re-engineer the search functionality of the

application to eliminate the error altogether, in the long run.


12/16


Knoa Use Case: End User Support

Accelerate Problem Resolution with end user metrics and workflow

Knoa EPM captures a complete record of end user behavior, before, during and after the transaction. This end user workflow data

transforms the end user support process.

A major electronics company was well into their deployment of a CRM solution. An inside sales reps in the Raleigh NC call center

was experiencing problems. The information being communicated to the support team from the help desk was very sketchy and

incomplete. In the past the CRM support team would spend hours on the phone with the inside rep, and often had to make a cost-

ly on-site support call to deal with issues like this. However, Knoa EPM enabled a dramatically different approach. Through the

management console, they reviewed this ISRs recorded experience with the application. First the team looked at the metrics and

found that the response time she was experiencing was fine well within the sub-3 second goal. Next they found that she was

generating four times more user errors than her peers. Investing further, by looking at a recorded record of her actual interaction

with the application, they found she was using an incorrect process to validate the customer information (a common process step

in many different transaction types). With this insight, her supervisor was able to instruct her in the proper customer validationprocess and the problems with the system were quickly resolved.


13/16


Knoa Use Case:Application Support/Engineering

Application Support/Engineering:

Key Performance Indicators (KPI) that report user errors on an aggregate level were useful in resolving another vexing issue

for that electronics provider. There were grumblings in the outside sales force that the opportunity management screen was

really difficult, slow and cumbersome.

When the KPI information for errors was reviewed, the application team noticed that virtually every end user was making the

same errors on these screens. The first instinct was to schedule an e-learning retraining session. However, a closer review of the

recorded work flow for a representative sample of users revealed that the errors were being caused when the sales reps were

trying to put data in extraneous fields that were on the screen but not required for the transaction to execute. The fix was

simple. No retraining was needed. The application team removed those fields from the screen and the errors were eliminated.

The end-to-end transaction time improved by 25% as well.


14/16


You Cant Manage What You Cant Measure.

Its rule #1 in the management handbook, isnt it? And the rule that is the driving

force behind the investment in CRM applications in the first place. But, without a

strategy to measure end user adoption, and without a strategy to gather insight

about end users experience and actual performance with the application, many

IT organizations are left with the reality that they cant manage and improve end

user adoption and build end user proficiency.

Knoa Experience and Performance Manager accurately determines how each

employee actually uses the application and provides input to comprehensive

reports that you can use to pinpoint training needs, adjust business processes,

and ensure that your end users are using the application to deliver all the value

you expect from the application. As an added benefit, Knoa also measures and

reports critical information about how your users experience the applicationsactual performance.

Application Support teams have real-time metrics to proactively address

response time issues problems

Help Desk personnel have immediate visibility user into actual user

interaction with the application, so they can quickly resolve issues with

minimal disruption to the end users

Training organizations can pinpoint which specific application areas (and

which specific users) require additional training

Application and Process Engineering can spot cumbersome work flows anderrors that are affecting end user satisfaction and efficiency

The CIO has comprehensive metrics from the entire application landscape

to support reporting to the business on SLAs and insight into application

usage and performance to make intelligent decisions about where to deploy

resources to improve the application performance and effectiveness

The business executives have actionable insight to drive the highest possible

levels of adoption, efficient and effective use of the application which is the

key to a business return on the investment

Questions Knoa Answers

for British Telecom

Is the application

performing effectively?

Are end users able

to effectively and

efficiently executecritical business

processes in other

words, has the training

program led to real

results?

Can we provide objective,

real-time data on the

user experience to allow

the help desk and key

users to resolve users

questions rapidly?

Can we maintain an

electronic audit trail that

tracks the completion of

key transactions?

What is the impact ofineffective procedures

on operations?


15/16


Can Knoa Experience And Performance

Management System Help You?Are you certain that all of your end users are actually using the functions

of the application required to execute their role?

Knoa EPM captures and reports real metrics about application usage. Who is

using which transactions? How long are they active? How long are they idle?

Knoa EPM gives you the ability to pinpoint and analyze user adoption issues

that are hindering the achievement of business results.

Do your end users ever use system performance as a key reason why

critical objectives are not met on time and on budget?

One of the benefits of Knoa EPM is that it gives you the ability to correlate

business outcomes with efficient, effective application usage. Knoa EPM system

captures and reports real metrics about the end users experience with theapplication, you will now know if the users were experiencing long transaction

response times or if there were a lot of system errors presented to the end user.

You will also know if the end users themselves were causing a lot of errors

by their own behavior. You will then know where to apply corrective action.

Perhaps it's a user training issue, or perhaps the errors are being caused by a

cumbersome or confusing user interface that can be corrected.

Do you suspect that there are a lot of problems and issues that are never

surfaced to the help desk?

Well, youre probably right. It is often reported in surveys of the service

management process that 70-80% of all end user problems are not reported

because the end users ignore the errors, abandon the transaction and/or find

workarounds. In fact, Forrester Consulting recently reported that an end user

typically experiences a problem 6 times before they call the help desk. Knoa

EPM gives you the ability to pinpoint and proactively remediate problems which

impact the efficiency or the effectiveness of the end user population using the

application to accomplish business goals. An EPM system will make sure you

have visibility into all errors for all users, in all locations, all the time.

When your end users do report problems to the help desk, do you find

they are wasting a lot of time trying to answer questions and document or

recreate the problems?

Knoa EPM captures the actual work flow that the user experienced before,

during and after the transaction. This information is immediately available to

help desk personnel so there is minimal business disruption from issues with theapplication. Employee productivity impact is kept to a minimum when there is

an issue (users spend less time on the phone with help desk and problems are

resolved faster).

Sixty seven percent of

surveyed organizations

are either much more

focused (27%) or more

focused (40%) on the end

user experience than they

were two years earlier.

Forrester Consulting2006


16/16


Expertise and Performance Management systems are becoming a priority for

the business executive and CIO who need to drive the highest possible levels of

adoption and efficient and effective use of core enterprise applications to drivea business return on investment. Only with the insight into end user metrics

that an EPM system can provide can you really know if your core enterprise

applications are delivering an acceptable user experience and if application

users have adopted the applications and are executing key processes effectively

and efficiently the key to achieving business value and ROI from the

application investment.

5 Union Square West4th FloorNew York, NY 10003

212-807-9608

www.knoa.com

Training

Support

Business

IT

About Knoa

Knoa Software is leading the

performance management

industry with end user experience

monitoring solutions that reach

beyond infrastructure and

application performance to the

monitor, measure and manage

how end users are utilizing theapplication to optimize business

process execution. Knoa

customers not only know if

core enterprise applications

are delivering an acceptable

user experience, but also, if

application users are executing

key processes effectively and

efficiently the key to achieving

business value and ROI. Knoa is

headquartered in New York, NY.For more information about

Knoa, please visit www.knoa.com.

Documents

Best Practices in Siebel Crm Performance Management Monitor Measure and Manage the Enduser Experience