EVLA Monitor & Control Software PDR E2E Interfaces: Observation Scheduling Complaints to: Boyd Waters John Benson, Barry Clark, Tim Cornwell, Rich Moeser,

EVLA Monitor & Control Software PDR

E2E Interfaces:

Observation SchedulingComplaints to: Boyd Waters

John Benson, Barry Clark, Tim Cornwell, Rich Moeser, Peggy Perley, Bruce Rowen, Kevin Ryan, Bill Sahr, Ken Sowinski, Boyd Waters

May 15, 2002 EVLA Monitor & Control Software PDR

Boyd Waters 2

What is “e2e”?

• End-to-End Data Management– Capture and Preserve data… – …in a computer-useable format…– THROUGHOUT THE PROJECT LIFECYCLE!

• Motivation:– Automate where possible– Make Radio Interferometry EASIERPOSSIBLE

Boyd Waters 3EVLA Monitor & Control Software PDR

May 15, 2002

e2e Architectural Diagrams

5. Observation Scripting Toolkit

Real-TimeScheduler

ControlScripts

10. Archive Toolkit

11. Pipeline Toolkit

3. Proposal Submission Toolkit

Submit Proposal

SubmitterIntentionalProposal

GenerateObserving

Scripts

TestEngineer

ObservingSystem

Monitor Data

Visibility Data

ArrayOperator

ImagePipeline

Visibility Data

Data

Bus

&

Storage

ObservationMonitor

SearchArchive

?Query

!Results

Monday, November 26, 2001 bwaters - page:1 of 1

End-to-End: More Detail

9. Real-Time

Observing

Toolkit

NormalizedProposal

Scenarios

project

ControlScripts

4. Proposal Management Toolkit

$

VISIOCORPORATIO

N

PrioritizedProposal

-or-

PrioritizeProposals

$

VISIOCORPORATIO

N

PrioritizedProposal

NormalizedProposal

NormalizeProposals

NormalizedProposal

IntentionalProposal

Scriptor

TAC

DataWrangler

7. Observation Evaluation Toolkit

EvaluateObservation

ObservationEval. ReportEvaluator

Researcher

ImagesVisibilitiesAncillary data reportsetc.

13. Calibration Source Toolkit

CalibrationData

CalibrationEditing

Monitor Data

Visibility Data

CalibrationScientist

6. Telescope Simulation Toolkit

SimulateTelescope

ProposalModeller

conditions

CalibrationData

Monitor Data

Visibility Data

ObservingScripts

ObservingConditions conditions

8. Observation Scheduling Toolkit

DynamicScheduler

conditions

Scenarios

project

Calibration Data is aspecific kind of“conditions” data.Calibration

Data

Scheduler

Visibility Data

CalibrationData

CalibrationData

Monitor Data

ImageControl Scripts -“as observed”

A “Scenario” is anordered list of Projects

The Real-TimeScheduler produces aQueue of ControlScripts (e.g. crd files)from a Scenario.

The Observing System providesfeedback to the Real-Time Schedulerby reporting the Control Scripts “asobserved”. The Observing Systemmay also raise events via MonitorData. TBD.


Boyd Waters 3


May 15, 2002


Real-TimeScheduler

ControlScripts

10. Archive Toolkit



Submit Proposal


GenerateObserving

Scripts

TestEngineer

ObservingSystem

Monitor Data

Visibility Data

ArrayOperator

ImagePipeline

Visibility Data

Data

Bus

&

Storage

ObservationMonitor

SearchArchive

?Query

!Results



9. Real-Time

Observing

Toolkit

NormalizedProposal

Scenarios

project

ControlScripts


$

VISIOCORPORATIO

N

PrioritizedProposal

-or-

PrioritizeProposals

$

VISIOCORPORATIO

N

PrioritizedProposal

NormalizedProposal

NormalizeProposals

NormalizedProposal

IntentionalProposal

Scriptor

TAC

DataWrangler


EvaluateObservation


Researcher



CalibrationData

CalibrationEditing

Monitor Data

Visibility Data



SimulateTelescope

ProposalModeller

conditions

CalibrationData

Monitor Data

Visibility Data

ObservingScripts



DynamicScheduler

conditions

Scenarios

project


Data

Scheduler

Visibility Data

CalibrationData

CalibrationData

Monitor Data






May 15, 2002


Real-TimeScheduler

ControlScripts

10. Archive Toolkit



Submit Proposal


GenerateObserving

Scripts

TestEngineer

ObservingSystem

Monitor Data

Visibility Data

ArrayOperator

ImagePipeline

Visibility Data

Data

Bus

&

Storage

ObservationMonitor

SearchArchive

?Query

!Results



9. Real-Time

Observing

Toolkit

NormalizedProposal

Scenarios

project

ControlScripts


$

VISIOCORPORATIO

N

PrioritizedProposal

-or-

PrioritizeProposals

$

VISIOCORPORATIO

N

PrioritizedProposal

NormalizedProposal

NormalizeProposals

NormalizedProposal

IntentionalProposal

Scriptor

TAC

DataWrangler


EvaluateObservation


Researcher



CalibrationData

CalibrationEditing

Monitor Data

Visibility Data



SimulateTelescope

ProposalModeller

conditions

CalibrationData

Monitor Data

Visibility Data

ObservingScripts



DynamicScheduler

conditions

Scenarios

project


Data

Scheduler

Visibility Data

CalibrationData

CalibrationData

Monitor Data






Boyd Waters 6

e2e/AIPS++ pipeline results


May 15, 2002

e2e:Calibrator Selection

• Prototyped Java interface

• Will do user testing very soon

• Expect to deploy soon

• Make catalog available in AIPS++ for processing

• Also make available as an NVO cone search – more later


Boyd Waters 8

e2e:Proposal Submission…

Example:

Gemini Proposal

Tool


Boyd Waters 9

e2e-EVLA Interaction Observation

SchedulingSoftware

Monitor &ControlSystem


Boyd Waters 10

e2e-EVLA II

Diagram Design:B. Clark

Scheduling Phases

Real-TimeScheduler

ControlScripts

TestEngineer

ObservingSystem

Monitor Data

Visibility Data

ArrayOperator

ObservationMonitor

9. RemoteObserving

Toolkit

Scenarios

ControlScripts


DynamicScheduler

conditions

Scenarios

project

Calibration Data is aspecific kind of“conditions” data.

Scheduler

CalibrationData

Control Scripts -“as observed”


GenerateObserving

Scripts

NormalizedProposal

project

$

VISIOCORPORATIO

N

PrioritizedProposal

-or-

Scriptor

Monitor and

Control


Boyd Waters 12

Dynamic Scheduling

Structure the Observation……so that we can…Respond to Events on a Short Time

ScaleChanging conditions (e.g. weather)Targets of Opportunity (e.g. GRBs)

…

Block-Based Scheduling


Boyd Waters 13

Observing Block

Preamble– The telescope setup for this

block

– Examples:

• Calibrators

• LO setup

“Post-amble”– Cleanup, book-keepingObserving BlockObserving Block

Pre

am

ble

~20(?) minutes nominal

“Post-a

mb

le”


Boyd Waters 14

Observing Block Constraints

• “Run this block until the calibrations converge”

• “I can’t run unless the previous block has run successfully”

• “I MUST run at 16:42:30 GST on 30 May 2002”– (fixed scheduling is dynamic scheduling with

time-domain constraints)


Boyd Waters 15

Block-Based Scheduler

Telescope sees ONE BLOCK AT A TIME:

Block Queue

Telescope

“ready for next block”

“here it is”

Observing BlockObserving Block

Pre

am

ble

“Post-a

mb

le”


Pre

am

ble

“Post-a

mb

le”


Pre

am

ble

“Post-a

mb

le”


Pre

am

ble

“Post-a

mb

le”

Implications:

•Simplifies the telescope state data

•Telescope reports block execution status back to the block queue

•All “observing logic” is maintained by the Block Queue


Boyd Waters 16

Block Execution


Pre

am

ble

“Post-a

mb

le”


Pre

am

ble

“Post-a

mb

le”Input Queue:

Execution:Observing Block

Observing Block

Pre

am

ble

“Post-a

mb

le”

3


Pre

am

ble

“Post-a

mb

le”

1


Pre

am

ble

“Post-a

mb

le”

1


Pre

am

ble

“Post-a

mb

le”

2


Pre

am

ble

“Post-a

mb

le”

3


Pre

am

ble

“Post-a

mb

le”

2


Pre

am

ble

“Post-a

mb

le”


Pre

am

ble

“Post-a

mb

le”

2


Pre

am

ble

“Post-a

mb

le”


Boyd Waters 17

Block Execution: Detail

Input Queue:

Execution:Observing Block

Observing Block

Pre

am

ble

“Post-a

mb

le”

3


Pre

am

ble

“Post-a

mb

le”

1


Pre

am

ble

“Post-a

mb

le”

1


Pre

am

ble

“Post-a

mb

le”

2


Pre

am

ble

“Post-a

mb

le”

3


Pre

am

ble

“Post-a

mb

le”

2

Archive:


Pre

am

ble

“Post-a

mb

le”

4


Pre

am

ble

“Post-a

mb

le”

5


Pre

am

ble

“Post-a

mb

le”


Pre

am

ble

“Post-a

mb

le”

MeasurementSet

MeasurementSet

MeasurementSet

MeasurementSet


Pre

am

ble

“Post-a

mb

le”

3


Pre

am

ble

“Post-a

mb

le”

1


Pre

am

ble

“Post-a

mb

le”

2


Pre

am

ble

“Post-a

mb

le”

3


Pre

am

ble

“Post-a

mb

le”

4


Pre

am

ble

“Post-a

mb

le”

ready ready ready ready

Pre

am

ble 2


Observing Block

Pre

am

ble

“Post-a

mb

le”


Pre

am

ble

“Post-a

mb

le”

2

ok ok ok ok

readyFailed!

?


Boyd Waters 18

Block Templates

• Debug blocks

• Template blocks

• “Default” blocks


Boyd Waters 19

Next Steps

• Gather and Codify Requirements– Observing Block constraints

• M&C Observing System Interaction– Formal Model

• Observing System Scripting– How instrument commands and observing

constraints are expressed

Documents

EVLA Monitor & Control Software PDR E2E Interfaces: Observation Scheduling Complaints to: Boyd Waters John Benson, Barry Clark, Tim Cornwell, Rich Moeser,