Dr Laurence Sutton Calderdale and Huddersfield NHS Foundation Trust Halifax West Yorkshire

Image CompressionImage Compression

Dr Laurence SuttonDr Laurence SuttonCalderdale and HuddersfieldCalderdale and Huddersfield

NHS Foundation TrustNHS Foundation TrustHalifax West YorkshireHalifax West Yorkshire

Is Lossy Compression of Image Data Is Lossy Compression of Image Data detrimental to Image Interpretation?detrimental to Image Interpretation?

Will Lossy CompressionWill Lossy Compression

Become standardised Practice?Become standardised Practice?



Definition.Definition. Do we need to compress image data?Do we need to compress image data? Compression Methods.Compression Methods. Canada Health Infoway Project.Canada Health Infoway Project. Technical and Clinical Evaluation.Technical and Clinical Evaluation. Medico-Legal Aspects (Regulations).Medico-Legal Aspects (Regulations). Next Steps?Next Steps?


DefinitionDefinition

Image Compression is used to reduce the Image Compression is used to reduce the amount of Memory required to store the amount of Memory required to store the Image data – Image data – without degrading the (visual) without degrading the (visual) quality of the image to an unacceptable levelquality of the image to an unacceptable level




Dr. Laurence Sutton

Image Data Explosion …fuelled by clinical advancements

4 Slice

80 slices

8 Slice

160 slices

Single Slice

40 slices

16 Slice

640 slices

Number Of Slices (Millions)

4441100

1900 2000 27003600

6300

10300

18800

'85 '92 '97 '98 '99 '00 '01 '02 '03

• ~ 20 million CT slices produced by one imaging department

• 10 Tera Bytes of data in 1 year• Speed, Storage, and Interpretation a

HUGE challenge

VCT 64 now a reality Technology not

standing still


*(exact 1024 Byte)

ExponentialExponential In BytesIn Bytes or.......or.......

YottabyteYottabyte YBYB 10102424 Bytes Bytes 1 Quadrillion1 Quadrillion can‘t imagine!can‘t imagine!

ZettabyteZettabyte ZBZB 10102121 Bytes Bytes 1 Trilliard1 Trilliard 1000 x all ever spoken words1000 x all ever spoken words

ExabyteExabyte EBEB 10101818 Bytes Bytes 1 Trillion1 Trillion

PetabytePetabyte PBPB 10101515 Bytes Bytes 1 Billiard1 Billiard 500 Billion pages text500 Billion pages text

TerabyteTerabyte TBTB 10101212 Bytes Bytes 1 Billion1 Billion paper from 50,000 treespaper from 50,000 trees

GigabyteGigabyte GBGB 101099 Bytes Bytes 1 Milliard1 Milliard 400,000 written A4 pages400,000 written A4 pages

MegabyteMegabyte MBMB 101066 Bytes Bytes 1 Million1 Million

KilobyteKilobyte KBKB 101033 Bytes Bytes 1 thousand*1 thousand* ½ page A4 written text½ page A4 written text

ByteByte BB 8 bit8 bit 1 Byte1 Byte

bitbit bb

new vocabulary ? – 1 Googol = 10100

Open for new vocabulary?Open for new vocabulary?

Dr. Laurence Sutton

Long Term Storage2nd Copy

Online Storage

# o

f Re

trie

vals

Time (Years)

Digital

Film

10%

1 Year

Storage Management

Dr. Laurence Sutton

Storage TCOincreases

1991 1994 1997 2000 2003 2006

Co

st

Storage hardware –35%

Stora

ge m

anag

emen

t

+10

to 1

5% T

CO

Source: Associate Management Group

Management cost increases Data availability People Software Security Facilities Energy Others

TCO total cost of ownership





Image CompressionImage CompressionLossless or LossyLossless or Lossy

LosslessLossless Regenerated is guaranteed to be identical Regenerated is guaranteed to be identical

to the original imageto the original image Also known as “reversible compression”Also known as “reversible compression”

Lossy Lossy A form of image compression where the A form of image compression where the

regenerated image is regenerated image is notnot guaranteed to be guaranteed to be identical to the original imageidentical to the original image

Also known as “irreversible compression”Also known as “irreversible compression”



HistoricalHistorical

SCAR Paper 2000,Bradley J. EricksonSCAR Paper 2000,Bradley J. Erickson“ Increasing evidence suggests that some forms

of irreversible compression can be used with no measurable degradation in aesthetic or diagnostic value.”

Today however Primary Reading of Irreversibly compressed images is not an accepted standard of practice


Do we need to compress Image Do we need to compress Image Data?Data?

Compression is a form of information Compression is a form of information processing.processing.

The latter we do all the time and throw away The latter we do all the time and throw away data!data! Selected fluoroscopy ImagesSelected fluoroscopy Images Selected CT slices sent to PACSSelected CT slices sent to PACS Representative US ImagesRepresentative US Images System and Technique information loss (In Plane System and Technique information loss (In Plane

Loss)Loss)


In Plane LossIn Plane Loss

In Plane LossIn Plane Loss Not saving CT raw dataNot saving CT raw data Low dose techniques (quantum noise)Low dose techniques (quantum noise) Applying and saving only one CT reconstruction algorithmApplying and saving only one CT reconstruction algorithm Cleanup and interpolation in CR/DRCleanup and interpolation in CR/DR Any form of image processing (information theory!)Any form of image processing (information theory!) 8 or 10 bit display/windowing of 12-16 bit data8 or 10 bit display/windowing of 12-16 bit data Displaying any part of an image at less than 1:1 zoomDisplaying any part of an image at less than 1:1 zoom Non-linear, badly calibrated monitorsNon-linear, badly calibrated monitors Archiving on FilmArchiving on Film (lossy compression)(lossy compression)


Some types of Image CompressionSome types of Image Compression

Joint Photographic Experts Group Joint Photographic Experts Group JPEG JPEG (ISO/IEC). (OSI)(ISO/IEC). (OSI)

Graphic Interchange Format Graphic Interchange Format GIF GIF (LZW)(LZW) Potable Network Graphics Potable Network Graphics PNGPNG WWW. Image File FormatsWWW. Image File Formats GIF and PNG are losslessGIF and PNG are lossless JPEG can be losslessJPEG can be lossless


Some types of Image CompressionSome types of Image Compression GIF and PNG are losslessGIF and PNG are lossless LZW (PK-zip, built-in in RAID etc.)LZW (PK-zip, built-in in RAID etc.)

< 2:1 < 2:1 JPEG can be losslessJPEG can be lossless JPEG Lossless (Pulse Code Modulation JPEG Lossless (Pulse Code Modulation

compression)compression) Totally unrelated to normal “JPEG”Totally unrelated to normal “JPEG” 2 varieties defined in DICOM2 varieties defined in DICOM About 2.5:1 About 2.5:1

JPEG 2000 lossless modeJPEG 2000 lossless mode About 3:1About 3:1


JPEG2000 and JPEGJPEG2000 and JPEG

JPEG, (Lossy) Discrete Cosine Transform JPEG, (Lossy) Discrete Cosine Transform Technique resulting in blocky artefacts at Technique resulting in blocky artefacts at higher level of compression.higher level of compression.

JPEG2000, Wavelet based compression JPEG2000, Wavelet based compression technique which allows for higher technique which allows for higher compression ratios without overt artefacts.compression ratios without overt artefacts.

Part of JPEG2000 is now part of an ISO Part of JPEG2000 is now part of an ISO standardstandard

File ex, .jp2 or .j2cFile ex, .jp2 or .j2c


JPEG2000 AdvantagesJPEG2000 Advantages

Superior Compression Performance.Superior Compression Performance. Lossless and Lossy Compression.Lossless and Lossy Compression. Error Resilience.Error Resilience. Ideally suited to photographic type images.Ideally suited to photographic type images. Smoothing Artefacts at high compression Smoothing Artefacts at high compression

levelslevels Regenerated Images are less Noisy and Regenerated Images are less Noisy and

cleaner!cleaner! JPIP compression streamlining protocol.JPIP compression streamlining protocol.

JPEG Q=806:1 JPEG Q=50

11:1J2k 30:1

J2k 100:1

Explicit Lossy Processes


JPIPJPIP

Uses the minimum bandwidth required.Uses the minimum bandwidth required. Ability to only download the requested part of Ability to only download the requested part of

the image.the image. Quick view of a large image at low resolution.Quick view of a large image at low resolution. Ability to pull more data for higher resolution Ability to pull more data for higher resolution

images of smaller parts of the image.images of smaller parts of the image. Reduces computer processes at server and Reduces computer processes at server and

client ends.client ends.





Canada Infoway ProjectCanada Infoway Project

National EHRNational EHR Storage and Network Impact of 35M images Storage and Network Impact of 35M images

(1.5PB) annually.(1.5PB) annually. Cost of storage versus cost of ownership.Cost of storage versus cost of ownership. Lossy Compression could save Lossy Compression could save

CDN$100M/year.CDN$100M/year. Can Lossy (Irreversible) Compression be used Can Lossy (Irreversible) Compression be used

as a standard of practice?as a standard of practice? Commissioned series of projects to Commissioned series of projects to

evaluate……evaluate……



Research conducted so far and the evaluation of Research conducted so far and the evaluation of the effects on Image Quality…..2 Independent the effects on Image Quality…..2 Independent Reviews.Reviews.

Legal Implications of using Irreversible Legal Implications of using Irreversible compression…..2 Independent Reviews.compression…..2 Independent Reviews.

Current Regulations around the world…..1 Current Regulations around the world…..1 reviewreview

The outcome of the reviews is such that the CAR The outcome of the reviews is such that the CAR is motivated to adopt Irreversible Compression is motivated to adopt Irreversible Compression as a standard and to such an end …….as a standard and to such an end …….



Clinical Review in conjunction with Infoway Clinical Review in conjunction with Infoway to assess the most appropriate compression to assess the most appropriate compression ratios for a variety of different body parts ratios for a variety of different body parts for each modality.for each modality.

Canadian Association of Radiologist Canadian Association of Radiologist endorsement of the resulting Guidelines.endorsement of the resulting Guidelines.

Results SIIM (SCAR) June 2007 Rhode Island.Results SIIM (SCAR) June 2007 Rhode Island.


Outcomes.Outcomes. Review of research….Review of research….Both reviewers concluded that “based onscientific studies, irreversible compressionis a clinically acceptable option for thecompression of medical images.” Legal Review….

Presume no visual impact on the quality of the Image.

Main risk is Tort of Negligence. Reduce risk by universal adoption of the standard. Respectable minority of Radiologists


Outcomes.Outcomes. Legal Review, BUTLegal Review, BUT

Can not alter the Medical Record once it has Can not alter the Medical Record once it has been approved.been approved.

Requirement to compress the image prior to Requirement to compress the image prior to the Primary Read.the Primary Read.

BUT, that is ok isn't it?BUT, that is ok isn't it? In this country the report is still the legal In this country the report is still the legal

document.document.

Regulatory Review.Regulatory Review. Non of the regulatory bodies in Non of the regulatory bodies in

Canada,USA,UK,EU,Australia have prevented or Canada,USA,UK,EU,Australia have prevented or endorsed the use of imaging compressionendorsed the use of imaging compression


Technical and Clinical Review.Technical and Clinical Review. A Review and Comparison of Medical

Image Compression Algorithms

August 2004Kirk W. Finnis, PhD

Atamai Inc., 100 Perth Drive, Suite 500,

London, Ontario, Canada


Technical and Clinical Review.Technical and Clinical Review.Research Review

And Assessment of Compression performance.

Comparisons of different algorithms betweenMedical Images and Standard Image sets.

*Lossy Compression of Medical Images.


Technical and Clinical Review.Technical and Clinical Review. Effects of Lossy Compression on Diagnosis.Effects of Lossy Compression on Diagnosis. Usually centred on one modality and a Usually centred on one modality and a

Pathology.Pathology. DentalDental CT and MRCT and MR CXRCXR Digital AngiogramsDigital Angiograms MammogramsMammograms UltrasoundUltrasound





SloSlonene, Muka & Pilgram, , Muka & Pilgram, http://radiology.rsnajnls.orghttp://radiology.rsnajnls.org

CXRCXR

http://radiology.rsnajnls.org/


MammographyMammography

ECR 2007, German Study, Dr R Loose, NurembergECR 2007, German Study, Dr R Loose, Nuremberg 33MB down to 325KB using standard wavelet 33MB down to 325KB using standard wavelet

compression.compression. No statistically significant impact on quality or No statistically significant impact on quality or

visible differences in images for compression visible differences in images for compression rates up to 100:1rates up to 100:1

Significant Impact on storage costs and the Significant Impact on storage costs and the provision of Teleradiology servicesprovision of Teleradiology services


CAD CT Lung Nodule Volume CAD CT Lung Nodule Volume Assessment.Assessment.

Jane B Ko et al, Radiology 2005;237:83-88Jane B Ko et al, Radiology 2005;237:83-88 Low dose CT technique, ROILow dose CT technique, ROI No significant difference 10:1No significant difference 10:1 Difference observed at 20:1Difference observed at 20:1 Reduced volume measurements.Reduced volume measurements. Need to be aware of the issues in Computer Aided Need to be aware of the issues in Computer Aided

TechniquesTechniques


SummarySummaryMetrics used by researchers to assess compression algorithms included:• Quantitative measures - a measure of image quality– Mean Squared Error (MSE)– Peak Signal to Noise Ratio (PSNR)• Qualitative measures - a measure of diagnostic quality– Difference Image and Histogram– Receiver Operating Characteristics (ROC) analysis


Conclusions and CaveatsConclusions and Caveats Limits (Tolerance) Image CompressionLimits (Tolerance) Image Compression

MR/CT 10:1MR/CT 10:1 CR 20:1CR 20:1

JPEG2000 adopted as a DICOM StandardJPEG2000 adopted as a DICOM Standard Rapid Adoption by the Imaging Community.Rapid Adoption by the Imaging Community. Medico-Legal uncertainties exist.Medico-Legal uncertainties exist. Evidence supports the safe adoption of Evidence supports the safe adoption of

Lossy compression for storage and Lossy compression for storage and compression without compromising compression without compromising diagnostic quality.diagnostic quality.


Next Steps.Next Steps. ? Case for adoption of Lossy Compression in the ? Case for adoption of Lossy Compression in the

UKUK

Royal College Approval of process to validate the Royal College Approval of process to validate the case for Lossy Compression.case for Lossy Compression.

Adopt Lossy Compression as a safe standard for Adopt Lossy Compression as a safe standard for the purpose of diagnosing from medical images.the purpose of diagnosing from medical images.

Consider potential issues around computer aided Consider potential issues around computer aided measurements.measurements.

Documents

Dr Laurence Sutton Calderdale and Huddersfield NHS Foundation Trust Halifax West Yorkshire