Upload
esaesoc-darmstadt-germany
View
990
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Lossless packet compression: a technique to deal with bandwidth limitations
Citation preview
Lossless packet compression: a technique to deal with bandwidth limitations
David Evans
The problem with housekeeping..
A mix of different field lengths and data types
Different packets are then written into the same packet store
This mixed up data is hard to compress.
Initial steps..
ROSETTA
7zip14%
winzip27%
7zip9%
winzip16%
3) Group packet types before compressing
1) Recreate a real on-board packet store on the ground
2) Compress it with commercial products to determine level of information redundancy RICE
100%
A bit of lateral thinking..
....#...##...#..###.#.#.#.##............##.#..##....#.##.#..#..###.#.#######.......#...##...#..###.#.#.#.##...#........##.#..##....#.##.#..#..###.##......#.......#...##...#..###.#.#.#.##..#.........##.#..##....#.##.#..#..###.##.....##.......#...##...#..###.#.#.#.##.#..........##.#..##....#.##.#..#..###.##....#.#.......#...##...#..###.#.#.#.##.#.#........##.#..##....#.##.#..#..###.##....###.......#...##...#..###.#.#.#.##.##.........##.#..##....#.##.#..#..###.##...#..#.......#...##...#..###.#.#.#.##.###........##.#..##....#.##.#..#..###.##...#.##.......#...##...#..###.#.#.#.###...........##.#..##....#.##.#..#..###.##...##.#.......#...##...#..###.#.#.#.###..#........##.#..##....#.##.#..#..###.##...####.......#...##...#..###.#.#.#.###.#.........##.#..##....#.##.#..#..###.##..#...#.......#...##...#..###.#.#.#.###.##........##.#..##....#.##.#..#..###.##..#..##.......#...##...#..###.#.#.#.####..........##.#..##....#.##.#..#..###.##..#.#.#.......#...##...#..###.#.#.#.####.#........##.#..##....#.##.#..#..###.##..#.###.......#...##...#..###.#.#.#.#####.........##.#..##....#.##.#..#..###.##..##..#.......#...##...#..###.#.#.#.######........##.#..##....#.##.#..#..###.##..##.##...
traditional feed
transposed feed
7zip
winzip
9%
10%
....#...##...#..###.#.#.#.##............##.#..##....#.##.#..#..###.#.#######.......#...##...#..###.#.#.#.##...#........##.#..##....#.##.#..#..###.##......#.......#...##...#..###.#.#.#.##..#.........##.#..##....#.##.#..#..###.##.....##.......#...##...#..###.#.#.#.##.#..........##.#..##....#.##.#..#..###.##....#.#.......#...##...#..###.#.#.#.##.#.#........##.#..##....#.##.#..#..###.##....###.......#...##...#..###.#.#.#.##.##.........##.#..##....#.##.#..#..###.##...#..#.......#...##...#..###.#.#.#.##.###........##.#..##....#.##.#..#..###.##...#.##.......#...##...#..###.#.#.#.###...........##.#..##....#.##.#..#..###.##...##.#.......#...##...#..###.#.#.#.###..#........##.#..##....#.##.#..#..###.##...####.......#...##...#..###.#.#.#.###.#.........##.#..##....#.##.#..#..###.##..#...#.......#...##...#..###.#.#.#.###.##........##.#..##....#.##.#..#..###.##..#..##.......#...##...#..###.#.#.#.####..........##.#..##....#.##.#..#..###.##..#.#.#.......#...##...#..###.#.#.#.####.#........##.#..##....#.##.#..#..###.##..#.###.......#...##...#..###.#.#.#.#####.........##.#..##....#.##.#..#..###.##..##..#.......#...##...#..###.#.#.#.######........##.#..##....#.##.#..#..###.##..##.##...
traditional feed
transposed feed
Let’s try something really simple…
7zip
winzip
9%
10%
....#...##...#..###.#.#.#.##............##.#..##....#.##.#..#..###.#.#######.......#...##...#..###.#.#.#.##...#........##.#..##....#.##.#..#..###.##......#.......#...##...#..###.#.#.#.##..#.........##.#..##....#.##.#..#..###.##.....##.......#...##...#..###.#.#.#.##.#..........##.#..##....#.##.#..#..###.##....#.#.......#...##...#..###.#.#.#.##.#.#........##.#..##....#.##.#..#..###.##....###.......#...##...#..###.#.#.#.##.##.........##.#..##....#.##.#..#..###.##...#..#.......#...##...#..###.#.#.#.##.###........##.#..##....#.##.#..#..###.##...#.##.......#...##...#..###.#.#.#.###...........##.#..##....#.##.#..#..###.##...##.#.......#...##...#..###.#.#.#.###..#........##.#..##....#.##.#..#..###.##...####.......#...##...#..###.#.#.#.###.#.........##.#..##....#.##.#..#..###.##..#...#.......#...##...#..###.#.#.#.###.##........##.#..##....#.##.#..#..###.##..#..##.......#...##...#..###.#.#.#.####..........##.#..##....#.##.#..#..###.##..#.#.#.......#...##...#..###.#.#.#.####.#........##.#..##....#.##.#..#..###.##..#.###.......#...##...#..###.#.#.#.#####.........##.#..##....#.##.#..#..###.##..##..#.......#...##...#..###.#.#.#.######........##.#..##....#.##.#..#..###.##..##.##...
traditional feed
transposed feed
RLE14%
Our simple preprocessing and fax machine algorithm (less than a page of C++ in total) was compressing ROSETTA housekeeping almost as well as the best algorithm on the market
Does this work on all missions?
38.20Earth ObservationGoce28.00AstronomyHerschel25.93Technology DemoProba-118.23InterplanetaryVenus Express14.06InterplanetaryRosetta5.75Human SpaceflightColumbus
COMPRESSION (% of original)
MISSION TYPEMISSION NAME
Parameter and bit level compressibility
0
20
40
60
80
100
120
140
160
1 107 213 319 425 531 637 743 849 955 1061 1167 1273 1379 1485 1591 1697 1803 1909 2015 2121 2227
Figure 3. Compressibility (% of original size) of each bit column for a packet that compresses well [CDMU Herschel Periodic P1 HK Parameter Report]
Parameter and bit level compressibility
0
20
40
60
80
100
120
1 161 321 481 641 801 961 1121 1281 1441 1601 1761 1921 2081 2241 2401 2561 2721 2881 3041 3201 3361 3521 3681 3841 4001 4161
Figure 5. Compressibility (% of original size) of each bit column for a packet that compresses badly [Herschel SCM Mode TM]
Major influencing factor is AOCS design, mission environment is less important
Resistance from an unexpected quarter..
Impractical
Risky
Not worth it
How much data do we need to store before compressing it?
0
10
20
30
40
50
60
70
80
90
100
10 50 100 250 500 1000 5000
# Packets
% O
rigin
al s
ize
48
Will it run on real hardware?
The bit transposition and RLE algorithms were recoded to respect space coding standards and loaded to a real LEON2 processor in ESA-ESTEC.
The algorithm compressed a 42.4 kilobytes file (from VEX) to 6.55 kilobytes (i.e. 15% of the original) in 0.5 seconds.
Implemented in one day, worked first time.
DANGERTEST IN PROGRESS
Now let’s address the risk argument..
Impractical
Risky
Not worth it
Too Risky
The error correction on the space link works at frame level, i.e. either all the data in a frame is received or the full frame is lost.
Compression in itself does not increase the risk of information loss
ONLY IF The information in each frame can be interpreted independently of the others.
The simple beauty of RLE
“If two identical bytes are followed by a third, insert a one byte counter of how many more repeats follow”
“FF FF FF FF FF FF FF FF 1C” becomes “FF FF 06 1C”
Incredibly simple to compress and expand
Self contained – so no risk increase!
The effect of bit transposition
....#...##...#..###.#.#.#.##............##.#..##....#.##.#..#..###.#.#######.......#...##...#..###.#.#.#.##...#........##.#..##....#.##.#..#..###.##......#.......#...##...#..###.#.#.#.##..#.........##.#..##....#.##.#..#..###.##.....##.......#...##...#..###.#.#.#.##.#..........##.#..##....#.##.#..#..###.##....#.#.......#...##...#..###.#.#.#.##.#.#........##.#..##....#.##.#..#..###.##....###.......#...##...#..###.#.#.#.##.##.........##.#..##....#.##.#..#..###.##...#..#.......#...##...#..###.#.#.#.##.###........##.#..##....#.##.#..#..###.##...#.##.......#...##...#..###.#.#.#.###...........##.#..##....#.##.#..#..###.##...##.#.......#...##...#..###.#.#.#.###..#........##.#..##....#.##.#..#..###.##...####.......#...##...#..###.#.#.#.###.#.........##.#..##....#.##.#..#..###.##..#...#.......#...##...#..###.#.#.#.###.##........##.#..##....#.##.#..#..###.##..#..##.......#...##...#..###.#.#.#.####..........##.#..##....#.##.#..#..###.##..#.#.#.......#...##...#..###.#.#.#.####.#........##.#..##....#.##.#..#..###.##..#.###.......#...##...#..###.#.#.#.#####.........##.#..##....#.##.#..#..###.##..##..#.......#...##...#..###.#.#.#.######........##.#..##....#.##.#..#..###.##..##.##...
traditional feed
transposed feedInstead of losing rows, you now lose columns…..a small set of parameters over a longer time as opposed to all parameters over a short time.
Is this a problem? If so, we can already state our maximum exposure….
48 packets.
But what about if I can’t afford to lose anything?
Simply use the time and bandwidth released to send the frames twice or implement a closed loop..
Original Repeat
This compression technique does not increase mission risk and the resources it releases can be used to reduce risk far below that normally accepted.
Those missions that are concerned about the risk of information loss must compress their housekeeping telemetry. Especially in contingency cases.
One more…
Impractical
Risky
Not worth it
Time is money
The time saved by HKTM compression will reduce how long an operation takes and open up choices for when it can be performed.
This will cut reaction times, an important operational metric.
The time saved will enable operational feedback loops not possible before.
Operations costs are dominated by manpower.
Manpower costs are dominated by time.
Ease the pain of packet design
Preparation tasks can account for a significant proportion of the total operations cost. Part of that work is to carefully select parameters and their respective sampling rate in the packets.
Do we really need this parameter in this
mode?
What sample rate is needed?
A common solution is to design asynchronous events to flag significant changes. This sounds easy!
How does compressing telemetry in this way help?
It effectively removes all “compressible” parameters (vast majority) from the packet design trades.
No need to guess if these parameter histories might be needed one day
No need to design and test asynchronous events for these parameters
No need to predict spacecraft behavior - to decide on trigger metrics
One can simply select as many of these parameters to be in the housekeeping as
one wants
Improving observability
Can sample “compressible” parameters at high rates with a low impact on bandwidth usage.
Less need to guess or extrapolate when trying to reconstruct events
Less chance of missing important information due to low sampling rates(short abnormal behavior like spikes, transients, high frequency switching)
More chance of discovering correlations between parameter behavior and important events or trends
Richer, finer information on the ground to analyze
Is it compatible with the present infrastructure?
Packet Store
Compressed Packet Store
Packet extractionBit transpositionRLE encodingStore in special “frame length” packets
Mission Control Sys tem
Router
Transfer to mission control centre using normal PUS services, CCSDS frames, coding, modulation etc
Pre-processor
Recovered packets passed to miss ion control system for normal processing
Compressed packets routed to pre-processor for decompression
The future
It is fair to say that it is becoming a standard solution for future studies section at ESA/ESOC for Phase 0/A work.
Baseline for PROBA-3
Identified as an enabling technology for the Mars Atmospheric Sample Return Mission
ESA patent filed in the USA
Conclusion
The Bit-Transposition and RLE algorithm
• Compresses housekeeping data very effectively
• Is extremely simple
• demonstrated good performance on target hardware
• Requires little change to the present CCSDS/PUS based infrastructure
• Requires just 48 stored packets as input
• Does not increase the risk of information loss
It will
• Reduce ground station usage and increase mission return
• reduce ground reaction times
• enable more efficient use of operations manpower
• reduce engineering costs in the preparation phase
• increase spacecraft observability and telemetry flexibility
• decrease mission risk
Conclusion
Thank you
Backup: Comparison with RICE