26/07/2013

l 66TH MEETING OF THE ESRF l 30-31 May 2014 l Author 1

Amazon EC2, GPU computing, PyNX:Ptychography

Vincent Favre-Nicolin X-ray NanoProbe group, ESRF

AMAZON GPU COMPUTING

Gpu Accelerated computing instances: •  (old) G2: nVidia GRID K520 •  P2: nVidia K80 (early 2015) (12GB, 4 Tflops theor.)

•  P2.xlarge: 1 K80, 61 GB memory •  P2.8xlarge: 4 K80, 488 GB memory •  P2.16xlarge: 8 K80, 732 GB memory

•  Performance ? •  Usability for data analysis ?

26/07/2013

Page 2 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

PyNX: PTYCHOGRAPHY

•  On-going effort to provide tools for on/offline analysis for Coherent Imaging techniques

•  Focused on using GPU/OpenCL for faster computing •  Used at id01, id13@ESRF, running on dedicated GPU

machines (GPU: Titan X) •  2D Ptychography:

•  Coherent images taken at different positions on a sample •  100 to 1000 of images-moderatly fast data acquisition (1-100Hz) •  Dataset can be exported in CXI format (http://cxidb.org/cxi.html)

26/07/2013

Page 3 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

IPYTHON NOTEBOOK: PyNX.PTYCHO

•  Quick test : launch ipython notebook •  Machine:

•  debian 8 official •  Nvidia drivers, OpenCL, clFFT •  Scientific python packages + PyNX

•  Data (already transferred 87Mb) •  Go to browser •  Choose kernel for data analysis •  Tweak parameters •  Run analysis •  Change parameters as needed and restart…

LIVE DEMO

26/07/2013

Page 4 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

ssh -CX -L:8888:localhost:8888  admin@ec2.****.compute.amazonaws.com"ipython3 notebook --pylab=inline

IPYTHON NOTEBOOK: PyNX.PTYCHO

•  GPU Analysis works (including using multi-GPU) •  Some latency in initializing the GPUs ? (up to 20s) •  No issues otherwise •  Compared speed: GPU K80 (Amazon) Titan X (ESRF) Read data (cxi) 58 Mpixel/s 86 Mpixel/s 2D FFT (400x400, 32 stack) 116 Gflop/s 282 Gflop/s dt/cycle (AP, 1025 frames) 0.544s 0.223s

•  Notebook can easily be configured to automatically

be available when starting the machine

26/07/2013

Page 5 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN

AMAZON GPU COMPUTING: CONCLUSION

Pros: •  On-demand GPU availability •  Large computing power available •  Best for offline/post-experiment data analysis •  Extremely easy to provide AMIs with software for users when they

need it offline •  Avoid conflicts between different software by providing several AMIs •  Notebook analysis is very simple to use, flexible •  Remote GUI processing possible (ssh –X,..) •  Also great for tutorials (e.g. HERCULES) •  On-demand cost ($0.2-1/hour/GPU) Cons: •  GPUs a bit outdated (no Maxwell, no Pascal) -> performance /2

compared to Maxwell Titan X •  Notebook interface:

•  great only for linear data analysis process ? •  No point-and-click interactivity

•  Not for ‘big’ data experiments (>>1Tb compressed) ? TODO: •  Simplified user auth / data access

26/07/2013

Page 6 l PANDAAS working group l 12 December 2016 l Vincent FAVRE-NICOLIN