14
Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator Sanity via Automation The Systems Behind Flock 3p/Flock 2k Commissioning Nate Parsons, Alex Paul, Lawrence Leung ({nate,apaul,lawrence}@planet.com)

Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Dongtaijinai’er Salt Lake, China – May 3, 2016

Operator Sanity via AutomationThe Systems Behind Flock 3p/Flock 2k Commissioning

Nate Parsons, Alex Paul, Lawrence Leung ({nate,apaul,lawrence}@planet.com)

Page 2: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Jeddah, Saudi Arabia

Introduction

• Planet Mission 1: image the entire Earth’s landmass every day• Dove Launches, 2017

○ Flock 3p - 88○ Flock 2k - 48○ Flock 3m - 4

• Operations Philosophy: automate as much as possible, see“Commissioning the World’s Largest Satellite Constellation”†

• Until 2017, ADCS commissioning was manual

† https://digitalcommons.usu.edu/smallsat/2017/all2017/138/

Page 3: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Shallow Seas, Bahamas – February 4, 2016

CommissioningProblem: ADCS Commissioning

Page 4: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

CommissioningTimeline of events

Launch

Initial Checkout

Detumble, Panel

Deploy

Calibration Maneuver 1

Calibration Maneuver 2

... Nominal Operations

Calibration Maneuver N

Page 5: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Tylihul, Estuary, Ukraine

• Scheduling maneuvers○ After software updates○ Within orbital constraints○ Around contact plan

• Waiting for data• Handling bad data• Tracking progress• Dozens of heterogeneous satellites

CommissioningChallenges

Page 6: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

London Array Wind Farm, United Kingdom – April 17, 2016

Solution: The Sequencer

Page 7: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Sequencer: Goals

• Handle each maneuver• Execute deterministically• System state: easy to determine, understand, communicate• Allow manual intervention• Configurable• Utilize existing infrastructure

○ Automated contact scheduling○ Scheduling API○ Asset config service○ Batch processing system○ Alerts (Slack)

Page 8: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Sequencer: DesignCommissioning Events Sequencer State Machine

Schedule Maneuver

Uplink Schedule

Execute Maneuver

Downlink Telemetry

Process Telemetry

Uplink Calibration Parameters

Ground Space Init

Take

Check

Apply

Success

Cancel

Failure

Page 9: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

scheduleitems.py

Sequencer: Architecture

• Sequencer - command line interface• State Machine

○ Determined via asset config service○ State objects: when invoked, return action items

• Action Items○ Set config value (to set state)○ Skip schedule item○ Take schedule item○ Start processing job○ Apply configuration params

• Schedule Items○ Objects: know how to take and skip themselves

• Maneuver Modules○ Scheduling constraint logic○ Telemetry Validation/Processing○ Determine when to retry maneuver vs fail and alert

statemachine.py

actionitems.py

sequencer.py

maneuver interfaces

maneuver definitions

Page 10: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Central Pivot Irrigation, Saudi Arabia

● Manager○ For manual intervention○ Request/cancel/retry/apply/reprocess any sequence

● Kickoff Processes (runs every 2 hours)○ For each sequence,○ if it can advance,○ kicks off state processing

● Process State○ Invoke a single state per run, ○ execute the resulting action items

● Commissioning Report○ Gathers state of all sequences,○ stores in metrics system

Sequencer: UsageServices Deployed

Page 11: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Kashaga, Oil Field, Kazakhstan

Sequencer: Testing

1. Unit tests on state machine logic2. Manual tests of action items3. Regression tests on processing of past data

○ Pro: Exercised Sequencer/ADCS code interfaces○ Con: Require satellites with similar hardware

4. Piecewise with virtual satellites○ Pro: real mission control○ Con: no physics model

5. End-to-end retired satellites in space○ Pro: High fidelity○ Con: Hardware mismatch

6. End-to-end with production satellites in space○ Pro: Highest fidelity○ Con: Acceptable loss in production imagery

“All the testing”

Page 12: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Muir Woods & Mt. Tamalpais, California, USA – December 23, 2015

Results

Page 13: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Results

● All goals met● Satisfied operators● Stable architecture● Reduced ADCS commissioning time● Things to improve

○ Adapt for nominal operations○ Alert on “stalled” sequences○ Reduce scheduling latency

† https://digitalcommons.usu.edu/smallsat/2017/all2017/138/

Success!

Page 14: Operator Sanity via Automation - FSW Workshopflightsoftware.jhuapl.edu/files/2017/Day-1/11-Parsons... · 2017-12-05 · Dongtaijinai’er Salt Lake, China – May 3, 2016 Operator

Mercator Projection

Thank You Flight Software Workshop 2017Visit us at www.planet.com

Nate Parsons | Missions Software Team | [email protected]