ScratchFS: A File System To Manage Scratch Disk Space For Grid Jobs Leandro Franco and Fabio Hernandez IN2P3/CNRS Computing Centre Lyon, France Features of ScratchFS • Implemented in user space (no kernel modifications needed). • Quota enforcement ( transparent to the users). • Life time management. • Real time usage details. • Security based on grid proxies instead of Unix users. Grid Security • All the security can be controlled independently of the user id that runs the job. • Different grid jobs running under the same user id will be recognized as different jobs belonging to different owners. • A mapping from grid certificates to local identifiers is no longer needed. Issues Since the actual operating systems were not designed to handle remote users in the way we conceive them in a grid world, we are obliged to use intermediate steps, like creating local accounts used to “convert” grid users to normal users. We are also forced to use different methods to monitor the space used by a job coming for the grid or to restrict the places where it can operate. Performance The performance tests showed us that although there is a price to pay for the functionality, it is not extremely high. Specially if it is compared to existing sandboxes or similar file systems. Bonnie++ I/O test 5G File 0 10000 20000 30000 40000 50000 60000 putc rew rite get_block Operations K B /S ec (Largeris better) Ext3 Fusexm p S cratch Kernel C om pilation Tim e (2.6.14) 0 5 10 15 20 25 30 35 Compilation Tim e in M inutes (sm alleris better) Ext3 Fusexm p Scratch http://grid.in2p3.fr/ scratchfs Userspace libfuse ScratchFS proxy verification ls /scratch/job/ VFS FUSE Kerne l Ext3 Job + Proxy End User Computing Element Batch Scheduler Worker Nodes Scratch FS Grid Middleware Computing Center

ScratchFS: A File System To Manage Scratch Disk Space For Grid Jobs

Download PPT Report

Upload
stacey-stuart
View
39
Download
2

Embed Size (px)

DESCRIPTION

Batch Scheduler. End User. Worker Nodes. Job + Proxy. Grid Middleware. Computing Element. ScratchFS. Computing Center. proxy verification. ls /scratch/job/. ScratchFS. libfuse. Userspace. Kernel. VFS. FUSE. Ext3. - PowerPoint PPT Presentation

Citation preview

ScratchFS: A File System To Manage Scratch Disk Space For Grid Jobs

Leandro Franco and Fabio HernandezIN2P3/CNRS Computing Centre

Lyon, France

Features of ScratchFS

• Implemented in user space (no kernel modifications

needed).

• Quota enforcement ( transparent to the users).

• Life time management.

• Real time usage details.

• Security based on grid proxies instead of Unix users.Grid Security

• All the security can be controlled independently of the user id that runs the job.

• Different grid jobs running under the same user id will be recognized as different jobs belonging to different owners.

• A mapping from grid certificates to local identifiers is no longer needed.

Issues

Since the actual operating systems were not designed to handle remote users in the way we conceive them in a grid world, we are obliged to use intermediate steps, like creating local accounts used to “convert” grid users to normal users. We are also forced to use different methods to monitor the space used by a job coming for the grid or to restrict the places where it can operate.

Performance

The performance tests showed us that although there is a price to pay for the functionality, it is not extremely high. Specially if it is compared to existing sandboxes or similar file systems.

Bonnie++ I/O test 5G File

10000

20000

30000

40000

50000

60000

putc rewrite get_block

Operations

/Se

c (

rge