1 Market Data Transparency SCR 740 Business Training SCR 727 Phase II: Web Services ERCOT presents:

1Market Data Transparency

SCR 740 Business Training

SCR 727 Phase II: Web Services

ERCOT presents:


Course contents

• Overview: Accessing Data Requests

• Lessons 1-15: Using Data Requests

–The lessons will include an overview of each data request along with a business case.

• Error Codes


Course goals

• Execute ad-hoc data research

• Understand Point In Time Selectivity (PITS)

• Schedule an audit of your 727 database

• Interpret error codes


Digital certificates

• All Market Participants will be required to have a Digital Certificate to access the data requests.

• You will be prompted to select your Digital Certificate when you enter TML.

• You can use the same Digital Certificate for all TML services (for example MarketTrak).

• To request a Digital Certificate you must contact your USA.


Accessing Web Services

Access the Texas Market Link (TML) website and access the “Archive” tab.

https://tml.ercot.com/tibco/archive


Data Request - Schedule

The SCR 740 Web Services are located at the Data Request – Schedule link from the left hand menu.1. Under Schedule an

Extract, left-click Data Request - Schedule.


Data request index

You may click on the “?” icon to view a detailed description of each data request.


Help screen

At the top of each data request is a blue help menu icon.

Each help screen has a top portion which is specific to the active data request and a bottom portion with additional reference material.


Lesson 1

Verify ESIIDs I own


Business question

Example: I need to verify ownership of 5 ESIIDs as of July 16, 2006.

What ESIIDs do I own for all of time, as of a particular point in time?


Select Point in Time

Input the Point in Time information.

Required fields are always marked with “*”.

1. Select July 16, 2006 from the Point in Time date boxes or from the calendar icon.


Select output

Choose the type of data output under Output Results.

Choose the output file format from Output Format.

Use the Reset button to reset the screen settings at any time.


Upload file

Upload a CSV file of the ESIIDs in question.

The Upload File Contents button will validate the format of your upload file.

1. Use the Browse button to locate your upload file.

2. Select the type of data being uploaded from the Uploaded dropdown.

3. Click Upload File Contents.


ActiveX

Note: The web services all use ActiveX to upload files.


Text box

If you do not have ActiveX enabled, or choose not to use ActiveX, a text box will be provided so that you can paste the contents of the upload file into the web service.


Verify upload contents

You can now verify the contents of your upload file.

The type of data chosen for Uploaded will appear at the top of the file contents.


Audit ID

After clicking Submit Request, the Audit ID will be displayed.

Take note of the Audit ID. This is needed to identify your request in the status screen.


Transaction information

You can view the completed SOAP request and response messages under Transaction Information.


Data Request - Status

To view the status of your data request, go to the Data Request – Status screen.

1. Under Schedule an Extract, left-click Data Request - Status.


Status screen

The scheduled request will be located under the data request name on the status screen.


View request information

By clicking the Audit ID, you can view details of the data request submitted.


Status definitions

• INITIALIZED - The request has been received and is in queue for processing.

• RUNNING - The request is not processing. The payload file is currently being created.

• FAILED - The request fulfillment has failed. The payload file was not generated.

• AWAITING POSTING - Requested payload file has successfully been created. It is waiting to be made available for web download.

• POSTING FAILED - An error occurred while making the payload file available for web download.

• CANCELED - The request has been cancelled. It will not be processed.


ZIP icon

When the data request has finished, a ZIP file icon will appear.


ZIP file contents

Once the ZIP file has downloaded, you can access the output files.


Report Explorer

Once your entity’s first Data Request has posted, a new folder will be created in the Report Explorer.

This folder will be titled “Market Data Transparency”.

Please note that ERCOT recommends picking up files via the Data Request – Status link.


Input and parameter files

Every data request output ZIP file will contain the original input file and a file of the parameters issued for the request.

Input File

Parameter File


Additional output files

In the example, we selected “Both ESIID and ESIIDServiceHist Records” as our Output Results.

We received two additional files, one from the ESIID table and one from the ESIIDSERVICEHIST table.

ESIID File

ESIIDSERVICEHIST File


Additional comments

1. If you are not the owner of an ESIID in question, you will not receive any information for that ESIID.

2. The original upload file will be returned in the output ZIP file.

3. The turnaround for this data request will be less than 3 business days. This is a worse-case scenario. The best-case scenario is approximately is 10 minutes.


Quiz

What does the “Upload File Contents” button do? (Pick one answer.)

A.Check the file content for accuracy.

B.Check the file format for accuracy.

C.Both A and B.


Quiz: Answer

B. Checks the file format for accuracy.

The Web Service does not have the ability to dynamically check against the database whether you have uploaded accurate data. This would dramatically slow down the Web Service.


Lesson 2

Provide ESIIDs I own for a trade date


Business question

Example: I need a list of all owned ESIIDs as of July 16, 2006 for trade date July 12, 2006 that are ”NIDR” meter types and belong to the Springfield station.

What ESIIDs do I own for a particular trade date?



Input the Point in Time and Trade Date information.

1. Select July 16, 2006 from the Point in Time date boxes or from the calendar icon.

2. Select July 12, 2006 from the Trade Date boxes or from the calendar icon.


Select output

Choose the type of data output under Output Results.

Choose the output file format from Output Format.


Additional Criteria

Input the additional search criteria as available. 1. Type “SPRING” into

the STATIONCODE text box.

2. Type “NIDR” into the Meter Type text box.


Audit ID



Output files

• Output Results include:

– Input File

– Parameter File

– ESIID FileESIID File


Additional comments

1. An upload file of ESIIDs is not required for this service because both Trade Date and Point in Time are required arguments.

2. Profile Type, Weather Zone, Meter Type, Weather Sensitivity, and/or Time of Use are profile code components that may be used in lieu of the complete PROFILECODE.


Quiz

List some of the differences between this Web Service and “Verify ESIIDs I own”.

1. Trade Date is required.

2. An Upload file of ESIIDs is not required.

3. “Provide ESIIDs I own for a trade date” has more filtering options.


Lesson 3

Provide CMZONE assignments for a trade

date


Business question

Example: I need a list of all owned ESIIDs as of July 16, 2006 for trade date July 12, 2006, which are in the Houston 2006 (“H06”) congestion management zone.

What ESIIDs do I own for a particular trade date for a certain CMZONE assignment?


Input information

Select July 16, 2006 for Point in Time and July 12, 2006 for Trade Date.

Choose CSV for Output Format.

Type H06 for CMZONE.

Click Submit Request.


Audit ID



Output files


– Input File

– Parameter File

– CMZONE FileCMZONE File


Additional comments

1. There is only type of output provided in this service so no Output Type selection is required. The format is:

ESIID, UIDESIID, CMZONE, STATIONCODE, STATIONNAME, TRADEDATE


Lesson 4

Provide Non-IDR transaction data for ESIIDs

I own


Business question

Example: I need to verify Non-IDR transaction data loaded using 867_03 transaction ids for 5 ESIIDs as of July 16, 2006.

For provided Non-IDR transaction information, what data is loaded as of a particular point in time?


Input information

Select July 16, 2006 for Point in Time.

Choose List for Output Results.


Upload the “ESIIDfile.csv” file.

Click Submit Request.The “ESIIDfile.csv” upload file is in the format:

ESIID, TransactionID, StartTime, StopTime, MeterType.


Output files


– Input File

– Parameter File

– List of ESIIDs (same format as the input file)

List of ESIIDs


Additional comments

1. All Upload file fields are required and must be in UPPERCASE.

2. The date format for the upload file is YYYY-MM-DD3. The Upload file format for transactions is:

ESIID, TRANSACTIONID, STARTTIME, STOPTIME, METERTYPE


Quiz

Does this Data Request provide interval data as an option?


Quiz: Answer

No. This is a Non-Interval data transaction verification only.


Lesson 5

Provide IDR transaction data for ESIIDs I own


Business question

Example: I need to verify IDR transaction data using 867_03 transaction ids for 5 ESIIDs as of July 16, 2006.

For provided IDR transaction information, what data is loaded as of a particular point in time?


Input information


Choose LSCCH Records for Output Results.


Select ESIID, TransactionId, StartDateTime, StopDateTime for Uploaded.

Upload the “ESIIDs.csv” file.



Output files


– Input File

– Parameter File

– LSCCH File

LSCCH File


Additional comments

1. All Upload file fields are required and must be in UPPERCASE.

2. There are two Upload file formats for this service. The first is a list of UIDCHANNELCUT values and the second is a list of ESIID transaction records in this format: ESIID, TRANSACTIONID, STARTTIME, STOPTIME

3. The date format for STARTTIME & STOPTIME is in the standard XML form of YYYY-MM-DDTHH:MM:SS.


Lessons 6 - 11 preview

6: Provide Non-IDR usage data for a trade date

7: Provide ESIIDs where Non-IDR usage data is not loaded for a trade date

8: Provide IDR usage data for a trade date

9: Provide ESIIDs where IDR usage data is not loaded for a trade date

10: Provide Non-IDR usage data for ESIIDs I own11: Provide IDR usage data for ESIIDs I own


Lesson 6

Provide Non-IDR usage data for a trade date


Business question

Example: I need a list of all owned ESIIDs as of July 16, 2006 for trade date July 12, 2006, which have Non-IDR usage data loaded.

What ESIIDs have Non-IDR usage data loaded for a particular trade date?


Select output


Choose KH (kWh) for Meter Type.

Choose ESIID Usage Records for Output Results.




Audit ID



Output files


– Input File

– Parameter File

– ESIIDUSAGE FileESIIDUSAGE File


Lesson 7

Provide ESIIDs where Non-IDR usage data is not

loaded for a trade date


Business question

Example: I need a list of all owned ESIIDs as of July 16, 2006 for trade date July 12, 2006, which do not have Non-IDR usage data loaded.

What ESIIDs do not have Non-IDR usage data loaded for a particular trade date?


Select output






Audit ID



Output files

• Output Results includes:

– Input File

– Parameter File

– “Usage Not Loaded” File“Usage Not Loaded” File


Additional comments

1. Data request inputs are similar to the lesson 6 data request. This request provides a list of ESIIDs that do not have usage data loaded in ERCOT systems, therefore identifying missing usage.


Lesson 8

Provide IDR usage data for a trade date


Business question

Example: I need a list of all owned ESIIDs as of July 16, 2006 for trade date July 12, 2006, which have IDR usage data loaded.

What ESIIDs have IDR usage data loaded for a particular trade date?


Select output


Choose Load (CH4) for Channel.

Choose List for Output Results.




Audit ID



Output files


– Input File

– Parameter File

– List of ESIIDsList of ESIIDs


Lesson 9

Provide ESIIDs where IDR usage data is not loaded

for a trade date


Business question

Example: I need a list of all owned ESIIDs as of July 16, 2006 for trade date July 12, 2006, which do not have IDR usage data loaded.

What ESIIDs do not have IDR usage data loaded for a particular trade date?


Select output






Audit ID



Output files

• Output Results includes:

– Input File

– Parameter File

– “Did Not Load” File“Did Not Load” File


Lesson 10

Provide Non-IDR usage data for a point in time


Business question

Example: I need a list of all owned ESIIDs as of October 16, 2006, for trade dates July 1 – July 12, which have Non-IDR usage data loaded.

What Non-IDR usage data is loaded for a list of ESIIDs as of a particular point in time for a given range of trade dates?


Select output

Select October 16, 2006 for Point in Time, July 1, 2006 for From Trade Date and July 12, 2006 for To Trade Date .



Choose ESIIDs for Uploaded.









Output files


– Input File

– Parameter File

– ESIIDUSAGE FileESIIDUSAGE File


Lesson 11

Provide IDR usage data for a point in time


Business question

Example: I need a list of all owned ESIIDs as of July 16, 2006, for trade dates July 1 – July 12, which have IDR usage data loaded.

What IDR usage data is loaded for a list of ESIIDs as of a particular point in time for a given range of trade dates?


Select output

Select July 16, 2006 for Point in Time, July 1, 2006 for From Trade Date and July 12, 2006 for To Trade Date .


Choose LSCCH Records for Output Results.


Choose ESIIDs for Uploaded.









Output files


– Input File

– Parameter File

– LSCCH File

LSCCH File


Lesson 12

Verify records used in Data Aggregation


Business question

Verify Non-IDR or IDR data used in data aggregation for a trade date.

Example: I need to verify Non-IDR data used in data aggregation for a trade date July 12, 2006 for a point in time of July 20, 2006, with a LOSSCODE of “A”, a PROFILECODE of “RESHIWR_SOUTH_NIDR_NWS_NOTOU”, and at the Springfield station.




Input A for LOSSCODE.

Input RESHIWR_SOUTH_NIDR_NWS_NOTOU for Output Results.

Input SPRING for STATIONCODE.

Choose TDSP DUNS and input DUNS Number.

Select CSV for Output Format.



Output files


– Input File

– Parameter File

– ESIIDUSAGE File

– ESIIDSERVICEHIST File

–DEFAULT File – contains a list of ESIIDs that did not have data loaded for the trade date or historical usage.

ESIIDUSAGE File

ESIIDSERVICEHIST File


Additional comments

1. This is only available to the LSE. QSEs do not have access to this data request.

2. This data request provides all ESIID data (ESH and usage) used during the data aggregation settlements process.

3. This request can only be requested twice a day per DUNS Number.


Lesson 13

Provide all ESIID extract records


Business question

Example: I need to baseline my 727 database from the beginning of time.

This data request provides the following:• Record counts for each table in the SCR 727 ESIID Service

History & Usage extract.• Logical key data records used by Market Participants to

perform a gap analysis comparison between ERCOT’s baseline record set and a market participant SCR 727 ESIID Service History & Usage data set.

• Full data records needed to baseline a market participant SCR 727 ESIID Service History & Usage database.




Select Full Records for Output For MP Tables.


Click Submit Request.Full Records outputs can only be requested once per year per DUNS Number.


Additional comments

1. The Full Records choice for Output For MP Tables can only be requested once per year per DUNS number and the Logical Keys choice twice per year. The Record Count choice has no restrictions.

2. The output files for Full Records and Logical Keys choices will be delivered via mail on DVD within 10 days of the request. These files will not post to TML.

3. The Logical Keys choices for Output For MP Tables can be used as an Upload File in the next web service to get full records.


Lesson 14

Provide ESIID extract records for Logical Keys


Business question

Provide full data records for uploaded logical keys for a particular table.


Input information



Select ESIID for Table.

Upload the “LogicalKeys.csv” file.



Additional comments

1. The Upload File format is exactly the same format as the Logical Keys choice for Output For MP Tables in the prior web service.

2. This is the only web service where the Upload File can have more than 1,000 rows.


Lesson 15

Provide public extract data


Business question

Provide all public tables (dimensional tables) as of a particular point in time.

This includes: CMZONE, MRE, PGC, PROFILECLASS, REP, STATION, STATIONSERVICEHIST, TDSP


Input information





Additional comments

1. This data request has no restrictions. You may request public data tables as often as necessary.


Error Handling


Error codes

There are 4 error codes that users can receive when submitting data requests.

1. ERROR 1007 – Indicates one or many of the fields you have supplied data for are inaccurate.

2. ERROR 1121 – The user must click the “Reset” button and fill in the data request screen again as there was an error in the submission.

3. ERROR 1122 – The data request has timed out. Please try the service again or contact ERCOT for assistance.

4. ERROR 9999 – This is a general error code for technical problems. Please contact the ERCOT Help Desk and log a trouble ticket for further assistance.


Error examples

Some errors will return specific details, such as when an INVALID/INACCURATE/INCORRECT (not sure which is most appropriate) trade date is submitted.

Additional error information is available under the Transaction Information section for “Response”.


FINAL TEST


Test, question 1

What is the Date & Time format for all Data Requests?

YYYY-MM-DDTHH:MM:SS

For example: July 16, 2006 5:32pm would translate into “2006-07-16T17:32:00”. This is an industry standard XML date and time format.


Test, question 2

Can you use any of the Data Requests to baseline your 727 database?

Yes, “Provide all ESIID extract records”. You can do this once per year per DUNS number.


Test, question 3

What is the maximum number of rows allowed in an upload file?

1,000…except for “Provide ESIID extract records for Logical Keys”.


Test, question 4

Is Point in Time required for every Data Request? Why?

Point in Time is required for every web service. Data changes every day and Point in Time gives you the ability to say: “I want to see the data as it existed on a certain day…before any subsequent changes.”


Test, question 5

What is the absolute maximum turn around time for Data Requests?

Three days for web services which post to TML. Ten days for web services which require DVD delivery.


Test, question 6

Can I tell what I uploaded from the output zip file data?

Yes, the original upload file is always returned as part of the output zip file.


Questions?

Documents

1 Market Data Transparency SCR 740 Business Training SCR 727 Phase II: Web Services ERCOT presents: