Questions and Answers – Application Performance Monitoring Software RFP

Date: November 17 - 20

Date Question OPERS Response 1 11/17/15

New Relic

Section 8.1 lists the amount of servers you want to monitor. Can you provide the estimate of production and pre-production servers?

Production:

35 Servers

5 SQL Servers

1 DB2 Environment Pre-Production:

70 Servers

10 SQL Servers

3 DB2 Environments

2 11/17/15

New Relic

In section E. SELECTION CRITERIA: it mentions a live demonstration of the vendor's product and capabilities. New Relic is a SaaS offering and can easily be installed via the web in 5 minutes. Can I reach out to some of the developers at OPERS, whom we have had contact with in the past, and show them how they can deploy New Relic? Rather than use 'demo data' we can show exactly how we work in your environment.

The live demo (if necessary) would not occur until after the first round of scoring and the top vendors/products are identified. Vendors may not reach out to any of the developers at any time during the process. Vendors must communicate questions through Procurement Office and only during the specified Q&A timeframe.

3 11/17/15

New Relic

According to section G-2 of the RFP, Vendors may not attempt to communicate with OPERS concerning this RFP in any manner or at any time other than during the question and answer period." - Is that limited to the procurement team? Can I communicate with the developers outside of this window?

No – communication initiated by a vendor outside of the Q&A period with anyone at OPERS (including Procurement) would result in disqualification.

4 11/17/15

New Relic

Regarding submitting the proposal. Under section G it says we should submit 2 electronic copies to you and lists your physical address. Will emailing the 2 copies of the submission work?

Yes – email is fine.

5 11/18/15

New Relic

For EUM we have a product called New Relic Browser. This is priced based on the number of monthly page views. Can you ask your technical team for a rough estimate of either daily page views, monthly page views, or annual pageviews?

It seems as if End User Monitoring would need to be conducted using a smaller specific subset of the total number of end users. I’m also guessing these would be our employees, since it appears they would need to use a specific browser during the browsing to obtain the EUM metrics. I’m guessing maybe 10 External users and 10 Internal users?

6 11/20/15

New Relic

With our EUM / Browser data & tools, the data will all be in real time, collected from whatever browser the users are using. With our analytics everywhere approach your teams can then query and/or filter the data to provide your teams with front end app metrics (javascript errors, ajax, etc) as well as use metrics

OPERS averages approximately 6000 page views per day.

regardless of the type of browser. For that reason, we would need the rough estimate of total page views OPERS has per month in production. Or if easier average per day and we can do the math :)

7 11/18/15

New Relic

For the cover letter, the RFP says: 1.1 A statement that the Vendor has not submitted its proposal with the assumption that there will be an opportunity to negotiate any aspect of the proposal. Then in 8.3 it says: 8.3 State whether Vendor will negotiate its proposed fee if OPERS decides negotiation is appropriate as to any aspect of the proposals, including the fee, with the finalist(s). In no case, however, will the negotiated fee be higher than the fee submitted in the proposal. If we will negotiate the proposed fee as it states in 8.3, do we just state in the cover letter something to the affect of?: New Relic has submitted this proposal with the assumption that New Relic will not negotiate for a higher price or more favorable terms. However we are open to discussing a possible reduction in price given further details to support such a reduction.

Section 1.1 asks that Vendors submit their best pricing/solution the first time and should not assume OPERS will allow an opportunity to negotiate and aspect of the proposal. Section 8.3 asks Vendors to let us know if they are willing to negotiate IF OPERS decides negotiation is appropriate.

8 11/19/15

Dynatrace

How many applications is OPERS looking to implement? Are there target go live dates?

OPERS implemented a solution using Service Oriented Architecture. The solution consists of a User Interface (External and Internal); Service Tier and Data Access Tier. As such, OPERS intends to monitor each of the tiers in order to conduct root cause analysis. OPERS is expecting the vendor to provide estimated go live dates as part of their response.

9 11/19/15

Dynatrace

Is OPERS looking to have our staff maintain and manage the product? Or is the goal for us to train OPERS staff?

The goal would be to train OPERS staff.

10 11/19/15

Dynatrace

Is there a dedicated APM team? OPERS does not have a dedicated APM team at this time.

11 11/19/15

Dynatrace

The NPM requirements on appendix a are not covered by the details provided in the cost description(8.1), would there be an opportunity to scope the NPM infrastructure to determine what is needed to meet the requirements?

The focus of the RFP is to select an APM solution, however, if the solution can include an NPM component, we would ask the vendor to provide their best costing numbers based on their experience.

12 11/19/15

Dynatrace

The scope provided in section 8.1, is that inclusive of all the environments? The RFP defines the scope as both Production and two development environments. If 8.1 is only the production environment, we would need the scope for the 2 development environments.

The scope provided in section 8.1 is inclusive for all the environments.

13 11/19/15

Dynatrace

There is a requirement in regards to synthetic solution and multi-step scripts which we can fully support, but we would need to scope out how many scripts and how often OPERS wanted to execute them to provide a price.

Provide a unit cost for deploying a multi-step script. Include in your response costs for different frequencies (quarter hour, hourly, daily, weekly, etc.).

14 11/19/15

Dynatrace

What is the approximate number of annual visits to the applications that are targeted for instrumentation?

Based on the following definition: A visit would also be considered a session that is more than just a home page hit, it would be a several step transaction, ie logging in, checking account balance. OPERS averages approximately 6000 page views per day.

15 11/19/15

Dynatrace

Can alerts be set based on text-matching criteria – can you expand on the requirement? What is OPERS specifically looking for?

OPERS currently monitors logs and has alerts based on specific text found in the logs. Alerts are sent via email to key technical staff to investigate.

16 11/19/15

Dynatrace

Is there an existing solution in place right now? If yes, which one?

OPERS has leveraged several solutions in order to monitor applications. The goal of this RFP is simplify our monitoring solution by replacing as many of those pieces as possible with a single cohesive system.