1 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1Integration with the European DataGrid Infrastructure

Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus (CERN)

GRACE General Project MeetingBrussels, 20-21 September 2004

2 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Table of Contents

1. Introduction2. Setting the Scene3. Infrastructure4. Toolkit5. Content Sources6. Conclusions 7. Annexes8. Glossary9. References

• Describe risks, difficulties encountered and major decisions taken

• Describe the activity performed in collaboration with EDG/EGEE projects

3 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Setting the Scene

1. Grid Computing2. The EDG project3. The EGEE project4. WP6 Activity

1. Planning2. Risks identification and management 3. Major decisions taken4. Collaborations (EDG/EGEE)

9 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Table of Contents

1. Introduction2. Setting the Scene3. Infrastructure4. Toolkit5. Content Sources6. Conclusions 7. Annexes8. Glossary9. References

10 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Infrastructure

1. Integration with the Grid (GILDA)2. Sites testing (certification)3. Conclusions

A big effort was necessary for setting up the infrastructure mainly due to the transition from EDG to EGEE and different middleware releases. With the effective support of EGEE NA4 team the set up and integration of GRACE grid nodes was completed successfully.

11 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Table of Contents

1. Introduction2. Setting the Scene3. Infrastructure4. Toolkit5. Content Sources6. Conclusions 7. Annexes8. Glossary9. References

12 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Toolkit

1. Integration with the Grid (GRACE Grid jobs)2. Testing (integration tests and performance

tests)3. Conclusions

GRACE Grid components have been integrated with EGEE mw, “easy-deployment” solution has been adopted, integration tests performed by testval group, performance considerations

13 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Table of Contents

1. Introduction2. Setting the Scene3. Infrastructure4. Toolkit5. Content Sources6. Conclusions 7. Annexes8. Glossary9. References

14 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Content Sources

1. Definition of the CSs of interest for GRACE 2. CSs configuration (see list)3. CSs configuration tests (automatic scripts)4. Conclusions

Risk of not having enough CS for Computing and Engineering KDs (subscription mechanism not implemented), it has been decided to concentrate on General and Physics KDs

15 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

D6.1 Missing parts

1. Final performance tests (started last Friday) and considerations

16 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

GRACE Grid Jobs Testing in GILDA

• Tests in GILDA testbed (LCG 2.0/2.2)• Sandra Application: Includes Text

Normalizing and Categorizing• One job per search• Application submitted in the InputSandbox

17 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

GRACE Grid Jobs Performances (un-optimized)

0

2,5

5

7,5

10

12,5

15

17,5

20

22,5

25

0 500 1000 1500 2000 2500 3000

Input File Size in KB

Exe

cutio

n T

ime

in H

ours

(with

out G

rid o

verh

ead)

18 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

GRACE Grid Jobs Performance (optimized)

0

2,5

5

7,5

10

12,5

15

17,5

20

22,5

25

0 500 1000 1500 2000 2500 3000

Input File Size in KB

Exe

cutio

n T

ime

in H

ours

(with

out G

rid o

verh

ead)

19 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

GRACE Grid Jobs Performances (details)

0

1

2

3

4

5

6

7

8

9

10

0 500 1000 1500 2000 2500 3000

Input File Size in KB

Exe

cutio

n T

ime

in H

ours

(with

out G

rid o

verh

ead)

Text Normalizing

Categorizing

20 CERN – Roberta Faggian Marque, Jan Fiete Grosse-Oetringhaus

GRACE General Meeting, September 2004, Brussels

GRACE Grid Job Outlook

• Typical search result still needs O(hours) of processing• Possible Solution: Job splitting• Distribute Text Normalizing across many jobs• Merge Results for Categorization• Use of Job dependencies model (e.g. DAGMan)

• Tests in LCG 2.2 with optimized Sandra application ongoing (started on 17 Sep)

• First test in new EGEE middleware gLite successful