Parallel Ops / Go-Live Checkpoint

Weekly Update – Friday MeetingJanuary 17, 2014

Agenda• IDT Debrief 9:00A – 9:20A

• Updated IDT Schedule and Conditions• IDT Report Review: IDT 1/7 and IDT 1/10

• Analysis and Announcements Updates 9:20A – 9:40A• Multi-Day Tags • Preliminary Scenario Results

• Look-Ahead 9:40A – 10:00A• Upcoming IDTs and Scenarios • Weekly Updates: Release Scorecard, Migration Report, Known Issues Log

• Settlements Update 10:00A – 10:15A• Readiness Update 10:15A – 10:45A

2

IDT Schedule and Conditions

3

IDT Date IDT Times Conditions Notes

1/21/14 Originally: 3:30P-8:30P2hr IDT begins at 5:30P

Now: 7:30A-4:30P7hr IDT begins at 8:30A

Regulation Pre-selection

Tieline Drop

SPP loses select tie-line measurements. SPP will simulate the loss of real-time data with significant tie-line data providers. MPs will not need to take any action; this is an internal process for our operators.

EDR Testing with specific MPs will occur during this IDT

Turnaround Ramp Rate set to 1

1/27/14 Originally: 4:30P-10:30P4hr IDT begins at 5:30P

Now: 7:30A-4:30P7hr IDT begins at 8:30A

Tieline CheckoutADIRegulation Pre-selectionTurn Around Ramp Rate to Regular Submission

2/25/14 8:30A-12:30P2hr IDT begins at 9:30A

Regulation Pre-selection

2/28/14 7:00P4hr IDT begins at 8:00P

Cutover for March 1 Go-Live

Ginny Watson

IDT 01/07 ReportIDT reports posted to spp.org here 7 calendar days after each IDT

spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test

IDT 01/10 ReportIDT reports posted to spp.org here 7 calendar days after each IDT

spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test

4Bryan Wood

Announcements

5

January Friday Meetings:Friday January 24: No Regular Friday Meeting

IDT 1/27 Prep Call at 3P

Friday January 31: Regular Parallel Ops / Go Live Checkpoint Meeting at 9A

SPP will retest Mitigation Scenarios (16.1 - DA, 16.2 - RUC, and 16.3 - RTBM) on Jan 28

•Any MP who wants to see results of Mitigation needs to fail the behavior test with at least one unit. Units will be picked up in the re-run.

•If you are interested in participating in Scenario 16.3 – RTBM Mitigation, please submit the Resource(s) you would like to see Mitigated via RMS (Parallel Ops Quick Pick) no later than Monday, Jan 27 at 12P CPT.

Ginny Watson

Multi-Day Tags for Day Ahead Market • An MP requested that SPP allow Monthly/Multi Day Tags for Day

Ahead Market Bids/Offers

• The Market Protocols only allow Day Ahead bids/offers up to 7 days prior to OD

• MISO has a Multi-day Process

• SPP will request an impact assessment for system changes necessary to allow Multi-Day Tags up to 7 days prior

• Depending upon cost and member approval this change would be a post Go-live enhancement

6Shari Brown

Shortage & Excess Capacity Scenarios

7Jarrett Friddle

SPP retested Shortage & Excess Capacity Scenarios (5.1, 5.2, 6.1, 6.2) in lower environments

•Defects described on the following slide caused Scenarios to fail

•Fixes are due in R1.9.3– Presuming patch correctly fixes reported issues on time, SPP will

retest Scenarios in lower environments by January 24– Detailed results will be presented to MPs.

8

• Summary – Scenarios in Parallel Ops:– 4 Scenarios, 2 Partial Scenarios, Dispatch Simulator Test successful:

Scenario 7 Must Offer Scenario 28a Scenario 15 Commitment Mismatch Part 2 Scenario 24 Scenario 13 DDR w/ Settlements Scenario 29 RSG Scenario 31 BDR w/ Settlements – success pending Settlements Scenario 36 GFA

Dispatch Simulator Test– Note: Dispatch Simulator Test in process again from Jan 15-17

• Summary – Retests Pending:– 2 entire Scenario retests pending

Scenario 11 scheduled for Jan 21; Non-Functional Testing Scenario under analysis

– 8 individual retest requests pending

Scenario Tracker as of Jan 8

Ginny Watson

Scenario 31 BDR w/ Settlements

9

Results• On the Markets side, this Scenario passed• Market Participant data was correct.• Resource behaved as designed in RTBM and EMS.• Dispatch instructions were observed via XML and Markets UI.

• Settlements • KIL #899, which will go into Production later this month, directly impacts the

charge types associated with BDRs and DRRs• Settlements is currently testing the resolution for KIL #899 and intend to have it

in Production for the Final Settlement

Jarrett Friddle

Non-Functional Testing

10Casey Cathey

Results page 1 of 2• Planned January 14th 10:00P – 12:00A• Extended 30 minutes to 00:30 so SPP could research RTBM solutions that were produced• Goal of Exercise was to test systems and identify areas that might need additional

attention in a failover event. Follow-up analysis and collaboration with vendors are underway to identify, test, and validate solutions. SPP’s planning team will recommend and communicate the next steps including timing and scope to MPs following analysis.

• Summary of Expected v. Actual Results • Other than exceptions noted, all systems functioned as expected.

Expected Result: EMS automatically fails over is fully functioning with RTGEN for CBA Actual Result: EMS failed over after manual intervention; once the

failover was completed, EMS functioned as expected

Expected Result: ICCP is fully functioning and sending setpointsActual Result: ICCP failed over correctly, functioned as expected

Expect Result: Marketplace Portal User Interface and API available for data entry and retrievalActual Result: Marketplace Portal and associated system UIs/APIs failed over correctly. Two external users experienced connectivity issues with the Markets UI. Users were able to reboot and connect. SPP is researching reported errors for additional information to provide to vendors as needed.

Non-Functional Testing

11Casey Cathey

Results page 1 of 2• Summary of Expected v. Actual Results • Other than exceptions noted, all systems functioned as expected.

Expect Result: Web Services (RTO_SS) availableActual Result: Remained available, as expected

Expected Result: Market System is fully functioning and solving RTBM for approx. 1 hrActual Result: RTBM missed intervals between 22:20, 22:25 and 22:30 while the failover was occurring. The first study solved at SPP’s backup facility at 22:35. SPP resources were onsite and continued to research the Market Systems during the exercise. Additional studies missed during the exercise were 23:10, 23:30, 23:35, and 23:50. SPP is working with the vendor to research and resolve this issue.

Expected Result: Email is available for sending and receiving communicationActual Result: Email was unavailable from the backup facility servers and was reverted to SPP’s primary facility while Markets trouble shooting occurred. Microsoft is on-site to assist in identifying solutions.

After the exercise was terminated, all systems and applications restored successfully to Chenal operations.

36.0 – GFA Carve Out SchedulesResults

•Preliminary Pass– MPs submitted BSS headers for all but one of the GFA Carve Outs

– No Import Interchange Transactions for GFA Carve Outs were submitted Only applies to 4 GFAs

MPs did not submit hourly BSS profiles for these import transactions

– SPP has enough data to test the GFA Carve Out process, and appropriate data to successfully test the settlements uplift distribution

– MPs should start seeing GFA Carve Out uplift charges on or around 1/22

12Jarrett Friddle

Look-Ahead: Jan 19 – Jan 31

13

Parallel Ops Calendar on spp.orgspp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents

• Settlements details on the Settlements Calendar marketplace.spp.org > Public > Settlements > CalendarsJan 19 – Jan 25 Schedule:Ramp Sharing Off Mon Jan 20: IDT Prep Call at 3PTues Jan 21: IDT Call 7:30A – 4:30P, 7hr IDT begins at 8:30A

Run Scenario #11 EDRWed Jan 22: IDT Debrief Call at 3P

CWGThurs Jan 23: CWGFri Jan 24: No regular P Ops Meeting. IDT Jan 27 Prep Call at 3P

Jan 26 – Jan 31 Schedule:Ramp Sharing OnMon Jan 27: IDT Call7:30A – 4:30P, 7hr IDT begins at 8:30ATues Jan 28: IDT Debrief at 3P and 16.1 / 16.2 ScenarioWed Jan 29: 16.3 Mitigation ScenarioFri Jan 31: Regular Friday Meeting at 9A, Parallel Ops Ends

Ginny Watson

Look Ahead Reminders• 1/21 IDT Call starts at 7:30 AM, 7 hour IDT begins at 8:30 AM

– Tieline Drop– Regulation Pre-Selection– EDR Scenario: Applicable MPs are in contact with SPP

This scenario will demonstrate the clearing and dispatch of an EDR resource for all qualified products.

• Please submit Spin/Supp as ‘Market’

• 1/27 IDT Call starts at 7:30 AM, 7 hour IDT begins at 8:30 AM– Tieline Checkout– Regulation Pre-Selection– ADI

• Please submit Spin/Supp as ‘Market’

14Bryan Wood / Jarrett Friddle

Migration and Maintenance Report

15

Migration and Maintenance Report posted to spp.orgspp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents

Wendy Reynolds

16

Release Scorecard posted to spp.org:spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents

Release Scorecard

Ginny Watson

Known Issues Log

17

• Please review the Market Trials Known Issues Log on spp.orgspp.org > Integrated Marketplace > Market Trials > Structured/Unstructured Test > Integrated Marketplace Market Trials Project Documentation Folder

Shari Brown

Market Trials Useful ResourcesIntegrated Marketplace Parallel Operations Project Documents spp.org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents

•Parallel Ops Presentations, Statistics, MP Meetings Folder•Parallel Ops Reference Documents Folder•Scenarios in Parallel Ops Folder > Scenarios Schedule•Parallel Ops Communication Plan•Parallel Ops MP Guide•Parallel Ops Calendar•Known Issues Log•Release Scorecard•Migration Report

Integrated Deployment Test (as part of Parallel Ops)spp.org > Integrated Marketplace > Market Trials > Integrated Deployment Test

•IDT MP Guide•Parallel Ops Calendar

Additional documents:•Settlements Calendar marketplace.spp.org > Public > Settlements > Calendars•Change Tracker spp.org > Integrated Marketplace > Marketplace Documents•Integrated Marketplace Structured/Unstructured Test Project Documents spp.org > Integrated Marketplace > Market Trials > Structured / Unstructured Test> Integrated Marketplace Structured / Unstructured Test Project Documents

18

SETTLEMENTS UPDATEBrent Wilcox

Brent Wilcox 19

Settlements Update - January • The December release was moved into production on Jan 8th as

scheduled. – Some days in early January were posted late due to the timing of the

migration of this release.

– A couple of defects were in the December release. A hot fix from Alstom is scheduled to be applied on 1/16.

• All January Operating Days will be processed using the December release. – Some Jan. initials were processed prior to the hot fix being installed

but all Jan. finals processed with the hot fix in place.

20Brent Wilcox

Settlements Update - February • Last statements (1/31 I) will be posted Feb 3rd.

– Please see highlighted section of February calendar for details

• Statements will remain on portal until noon on Feb 5th

• Settlement system cutover begins the afternoon of Feb 5th.

21Brent Wilcox

Settlements Update – February Cont’d • Market Participants can start to submit their Bilateral Settlement

schedules on Feb 22nd. – Thirty day window will not be followed in this instance.

– Any Bilateral Settlement schedules submitted prior to February 22nd will be deleted as a part of the system cutover

• Meter windows will open on OD+1 (3/2/14).

22Brent Wilcox

OFFER DATA ACKNOWLEDGEMENT

23

• Markets Test Data vs. Production Level Data– Market Offer data will NOT be purged from the Markets Systems prior to 3/1

Includes existing offer data submitted since June 2013 Market Trials start -- even MP offer data that was seeded by SPP for testing purposes

– Market Participant test offer data MAY become financially binding on 3/1 and beyond if preventative measures aren’t taken before 2/22

Update Market Offer Data

24Erin Jester

Update Market Offer Data• MP Required Action for Updating Market Offer Data1. Complete Acknowledgement Letter by 2/14

Market Trials Liaisons received RMS ticket on 1/17 requesting acknowledgement of responsibility to update Market offer data and understanding of risks associated with having non-production ready offer data in the Market Systems after 2/22

Acknowledgement to be completed via RMS no later than 2/14

Escalation will occur for MPs not completing acknowledgement by 2/14

– Update all Market Offer Data in the Markets System by 2/22 To avoid test data becoming financially binding, update offer and parameter

data for DA Resource Offers, RTBM Resource Offers and Mitigated Offers

Market offer data MUST be updated prior to 2/22 to ensure good, production-ready data for Go-Live

25Erin Jester

MP GO-LIVE VALIDATION CHECKLIST

26

• Go-Live Validation Checklist posted to Readiness workstream folder on 1/17

• Tool to identify applicable tests and perform internal, independent validations of systematic and functional readiness to cutover to Go-Live

• Tool should be used now and after the February Maintenance Window (2/22), prior to Go-Live (3/1)

• Go-Live Checklist is not to be returned to SPP. Only for MP use in confirming internal readiness.

• Any issues discovered while validating readiness should be addressed immediately via RMS

MP Go-Live Validation Checklist

27Erin Jester

• Populate the following table based on MP & Asset Types

• Based on your selections, applicable tests will be identified that you will want to verify internally prior to Go-Live.

MP Go-Live Validation Checklist

28Erin Jester

READINESS UPDATEErin Jester

29Erin Jester

General Readiness Updates

• Go-Live Approach (similar to TCR) will be posted the week of January 20 to the Readiness folder– Will include the MP Go-Live Validation Checklist

• iDashboard will be updated today (January 17) per the established schedule– Total Number of Readiness Metrics: 50

Number Complete (Blue): 36

Number on Track (Green): 7

Number at Risk (Yellow/Red): 6

Number Not Started (Gray): 1

30Erin Jester

Parallel Ops Exit Criteria Status• POX-01 (Scenario Completion) changed from yellow to green for the first

two weeks of January– Scenario completion will continue to be tracked

• There are three “at risk” Parallel Ops criteria:

31

ID Metric Name Status Variance

POX-20 OPS-15 CROW Outage Submission Yellow

12/28-1/11: 15/15 of resource-owning MPs who have reported outages have not reported them through SCADA and CROW within 30 minutes.

POX-21 OPS-17 ICCP Model Validation Yellow 1/41 of MPs ICCP models are not validated, but is expected

to be in service shortly.

POX-22 OPS-18 ICCP Control Status Tracking Yellow

12/15-1/4: 26% of MPs submitted control modes correctly 100% of the time.

SPP is adjusting how OPS-18 is reported, and metrics are expected to improve once these changes are made. See ICCP Control Status slides)

Erin Jester

OPS- 15: At Risk Readiness Metric

32

Metric ID Metric Title Entity Status

OPS-15 CROW Outage Submission MP Red• 4.23% of the total outages reported by resource-owning MPs

were reported within 30 minutes.

• 1.41% of the total outages reported by resource-owning MPs were reported between 30 minutes to 1 hour.

• 94.37% of the total outages reported by resource-owning MPs were reported beyond 1 hour.

Data as of January 11, 2014*

*Based on data from January 5-11, 2014

Erin Jester

CROW Topics for Discussion at CWG

• CROW Outage Submission Benefits– Avoid penalties

– Avoid commitments that are not physically possible

• CROW Outage Submission Impacts– Examples of what happens within the IM when incorrect

outages exist.

• CROW Outage Submission Penalties– Discuss the RUC Make-Whole-Payment Distribution Amount

penalties for DA and RUC

33Erin Jester

CROW Hot Topic Calls

• The intent is to minimize SPP’s potential for incorrect commitments.

• Help the members meet the Criteria requires.

(Entry of an outage to CROW within 30 minutes of the actual outage)

• CROW Hot Topic Net Conferences are scheduled for9:30 AM – 10:30 AM CST on 1/28/2014

3:00 PM – 4:00 PM CST on 1/30/2014

Register via the SPP Learning Center

34Erin Jester

CROW Recommendation

• Resource owning Market Participants should attempt to meet the OPS-15 metric prior to the end of Parallel Operations on 1/31. • Provides confidence that starting the Marketplace 3/1 with

outage communication coordination will not adversely impact economics and reliability due to outage mismatches in CROW versus Real-time situation.

• Resource-owning MPs may continue and are encouraged to practice meeting OPS-15 throughout the February Extended Maintenance Window

35Erin Jester

Update on Attachment AH

• Received all but 3, and a couple additional require updates:

36

RMS MP Status

6147 APX No response after initial contact

6173 Calpine Contact unavailable this week; will follow-up

6201 ETEC Expected this week

6227 Golden Spread Update required

6246 TEA Update required

Erin Jester