Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Maria Grazia Pia, INFN Genova 1
Publication patterns in HEP computing
M. G. Pia1, T. Basaglia2, Z. W. Bell3, P. V. Dressendorfer4
1INFN Genova, Genova, Italy 2CERN, Geneva, Switzerland 3ORNL, Oak Ridge, TN, USA 4IEEE, Piscataway, NJ, USA
CHEP 2012, NYC
Maria Grazia Pia, INFN Genova 2
Analysis topics ! General tools − Geant4 − ROOT
! HEP experiments − LEP
§ ALEPH, DELPHI, L3, OPAL
− BaBar − LHC
§ ALICE, ATLAS, CMS, LHCb, TOTEM
! Grid computing − LCG
! What they publish ! How much ! Where ! Citations ! Technology vs physics ! Software vs hardware ! Software/DAQ-trigger
Maria Grazia Pia, INFN Genova 3
Data sources ! Thomson-Reuters: ISI Web of Knowledge
− CERN subscription: since 1970, conference database not included − Search by keywords, collaboration name
! Journal web sites − IEEE TNS − NIM, Comp. Phys. Comm. (Elsevier) − JINST (IOP/SISSA) ➤ Full-text searches
! CERN databases − CERN Document System − Greybook
! Years: 1982-2011 (LEP), 1992-2011 (BaBar, LHC) − Reproducible sample
Maria Grazia Pia, INFN Genova 4
Data sample ! Contamination
− Non-pertinent entries in the data sample ! Omission
− Pertinent papers are not included in the data sample ➩ Cross-checks
− WoS/CDS, WoS/publishers’ web sites ! WoS inconsistencies and errors
− Total number of citations includes Conference database − Proceedings papers: false classifications and omissions ➩ Manually corrected whenever possible
! Automated analysis (whenever possible)
! Manual evaluation: abstracts and full-text papers − Some degree of subjectivity
Maria Grazia Pia, INFN Genova 5
S. Agostinelli et al. Geant4: a simulation toolkit NIM A, vol. 506, no. 3, pp. 250-303, 2003
J. Allison et al. Geant4 Developments and Applications IEEE Trans. Nucl. Sci., vol. 53, no. 1, pp. 270-278, 2006
2934 citations (14 May 2012)
2026 citations excluding proceedings
Most cited CERN publication in WoS (excluding Rev. Part. Properties)
574 citations (14 May 2012)
381 citations excluding proceedings
Many papers cite the NIM paper, but they omit citing the TNS one, even though both are indicated in http://cern.ch/geant4 Many papers that use Geant4 do not cite either reference
Citation analysis: until 2011 (reproducibility)
Maria Grazia Pia, INFN Genova 6
0
100
200
300
400
500
2003 2004 2005 2006 2007 2008 2009 2010 2011
Cita
tions
Year
Geant4 NIM Geant4 TNS
0 100 200 300 400
Radiat. Prot. Dosim. J. Korean Phys. Soc.
Radiat. Meas. Appl. Radiat. Isot.
J. Phys. G JHEP
Astrop. Phys. EPJC JINST NIM B
Phys. Lett. B Phys. Rev. C
Phys. Med. Biol. Med. Phys.
Phys. Rev. Lett. TNS
Phys. Rev. D NIM A
Citations
Geant4 NIM: Citing Journals
30% Physics
75% citations (plot)
0 50 100 150 200
ISOLDE ALICE
JET EFDA BES III N TOF
MiniBooNE LUNA
CDF HARP LHCb CMS
ATLAS BaBar
Citations
G4 NIM: Citing Collaborations
LHC HEP Other
16% citations (plot) 19% citations from collaborations
Born from LHC experimental requirements Multidisciplinary sources of citations
Maria Grazia Pia, INFN Genova 7
R. Brun and F. Rademakers ROOT - An object oriented data analysis framework NIM A, vol. 389, no. 1-2, pp. 81-86, 1997
I. Antcheva et al. ROOT - A C++ framework for petabyte data storage, statistical analysis and visualization Comp. Phys Comm., vol. 180, no. 12, pp. 2499-2512, 2009
540 citations (14 May 2012)
347 citations excluding proceedings
27 citations (14 May 2012)
20 citations excluding proceedings
AIHENP Workshop proceedings paper
Citation analysis: until 2011 (reproducibility)
Maria Grazia Pia, INFN Genova 8
0
10
20
30
40
50
60
1997
19
98
1999
20
00
2001
20
02
2003
20
04
2005
20
06
2007
20
08
2009
20
10
2011
Cita
tions
Year
ROOT Proc. ROOT CPC
0 20 40 60 80 100 120
NIM A TNS
Comp. Phys. Comm. Phys. Rev. C Phys. Rev. D
JINST Phys. Med. Biol.
EPJC Med. Phys.
JHEP NIM B
Lect. Notes Comp. Astropart. Phys.
Citations
ROOT Proc.: Citing Journals
0 1 2 3
AUGER BELLE
D0 GLAST
H1 HADES
JET-EFDA KIMS
PHOBOS R3B
RISING BABAR ALICE ATLAS
T2K N TOF
D0 CLAS
CDF CMS
Citations
75% citations
8% of all citations from collaborations Geant4 % ROOT %
Technology 30.3 49.6 Physics 29.9 18.2 BioMedical 13.9 6.0
Field of citing journals
Maria Grazia Pia, INFN Genova 9
HEP experiments
LEP • ALEPH • DELPHI • L3 • OPAL
BaBar LHC
• ALICE • ATLAS • CMS • LHCb • TOTEM
LEP: 1989 BaBar: 1999 LHC: 2008
Start of run
Collaboration members
Experiment
Mem
bers
0
500
1000
1500
2000
2500
3000
3500
ALEPH
DELPHI L3
OPALBabar
ALICEATLASCMSLHCb
TOTEM
Maria Grazia Pia, INFN Genova 10
Time distribution LEP: 1989 BaBar: 1999 LHC: 2008
Run start
Publication year Rescaled w.r.t. year of start run
1985 1990 1995 2000 2005 20100
50
100
150
200
Publication year
Year
Num
ber o
f pub
licat
ions
AllLEPBaBarLHC
−20 −10 0 10 200
50
100
150
Publications vs operation year
Year
Num
ber o
f pub
licat
ions
LEPBaBarLHC
Maria Grazia Pia, INFN Genova 11
Time distribution LEP: 1989 BaBar: 1999 LHC: 2008
Run start
Same as previous slide, rescaled by the number of experiment members
−20 −10 0 10 200.00
0.02
0.04
0.06
0.08
0.10
0.12Publications/member vs. operation year
Year
Num
ber
of public
ations
LEPBaBarLHC
1985 1990 1995 2000 2005 20100.00
0.02
0.04
0.06
0.08
0.10
0.12Publications/member vs. year
Year
Num
ber
of public
ations
LEPBaBarLHC
Maria Grazia Pia, INFN Genova 12
Publications Publications
Experiment
Num
ber o
f pub
licat
ions
0
100
200
300
400
500
600
ALEPH
DELPHI L3
OPAL
Babar
ALICEAT
LAS
CMSLH
Cb
TOTEM
generalphysicshardwareDAQ−triggersoftware
Technological publications
Experiment
Num
ber o
f pub
licat
ions
0.0
0.2
0.4
0.6
0.8
1.0
ALEPH
DELPHI L3
OPAL
Babar
ALICEAT
LAS
CMSLH
Cb
TOTEM
Share of hardware, software and DAQ-trigger
publications
Maria Grazia Pia, INFN Genova 13
Physics publications
LEP experiments completed their life-cycle LHC experiments: at an early stage of their physics production
●
●
●
●
●
●
●
●
●●0
100
200
300
400
500
Physics publications
Experiment
Num
ber o
f pub
licat
ions
ALEPH
DELPHI L3
OPAL
Babar
ALICE
ATLA
SCMS
LHCb
TOTEM
●●
●
●
●
● ● ● ●●
0.0
0.2
0.4
0.6
0.8
1.0
Physics publications/member
Experiment
Num
ber o
f pub
licat
ions
/mem
bers
ALEPH
DELPHI L3
OPAL
Babar
ALICE
ATLA
SCMS
LHCb
TOTEM
Maria Grazia Pia, INFN Genova 14
Technological publications
Roughly constant trends, once the number of publications is normalized to the number of collaborators
0
50
100
150
200
250
Technological publications
Experiment
Num
ber o
f pub
licat
ions
ALEPH
DELPHI L3
OPAL
Babar
ALICE
ATLA
SCMS
LHCb
TOTEM
● ●
● ●
● ●
●
●
●
●
● Software.DAQ−triggerHardware
Technological publications/member
Experiment
Num
ber o
f pub
licat
ions
/mem
bers0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
0.18
● ●
●●
●
●● ●
●●
● Software.DAQ−triggerHardware
ALEPH
DELPHI L3
OPAL
Babar
ALICE
ATLA
SCMS
LHCb
TOTEM
Maria Grazia Pia, INFN Genova 15
Software vs. hardware
Hardware publications: approximately 4 times more than software DAQ-trigger publications: approximately 1.3 times more than software
●
●
●
●
●
●
●
●
●
●
Hardware/software publications
Experiment
Ratio
0
2
4
6
8
10
ALEPH
DELPHI L3
OPAL
Babar
ALICE
ATLASCMSLHCb
TOTEM
●●
●
●
●
●
●
●
●
●0
2
4
6
8
10
DAQ−trigger/software publications
Experiment
Ratio
0
2
4
6
8
10
ALEPH
DELPHI L3
OPAL
Babar
ALICE
ATLASCMSLHCb
TOTEM
Maria Grazia Pia, INFN Genova 16
Journals hardware
DAQ-trigger
software
Journals
Journal
Num
ber
of public
ations
0
100
200
300
400
500
600
700
EPJC
JHEP
Nuc
l. Phy
s. B
Phys. L
ett.
B
Phys. R
ev. D
Phys. R
ev. L
ett.
Z. Phy
s. C
New
J.P
hys.
EPL
Astro
p.
Phys. R
ep.
CPC
JINST
NIM
A
NIM
BTN
S
EPJC
JHEP
Nucl. Phys. B
Phys. Lett. B
Phys. Rev. D
Phys. Rev. Lett.
Z. Phys. C
New J. Phys
EPL
Astrop. Phys.
Phys. Rep.
CPC
JINST
NIM A
NIM B
IEEE TNS
TNS
NIMA
JINST
Maria Grazia Pia, INFN Genova 17
Journals, LHC publications
Journal
Nu
mb
er
of
pu
blic
atio
ns
0
100
200
300
400
EPJC
JHEP
Nuc
l. Phy
s. B
Phys. L
ett.
B
Phys. R
ev. D
Phys. R
ev. L
ett.
Z. Phy
s. C
New
J.P
hys.
EPL
Astro
p.
Phys. R
ep.
CPC
JINST
NIM
A
NIM
BTN
S
EPJC
JHEP
Nucl. Phys. B
Phys. Lett. B
Phys. Rev. D
Phys. Rev. Lett.
Z. Phys. C
New J. Phys
EPL
Astrop. Phys.
Phys. Rep.
CPC
JINST
NIM A
NIM B
IEEE TNS
Journals: LEP and LHC
Still dominated by technological publications
LHC LEP
Dominated by physics publications
Journals, LEP publications
Journal
Nu
mb
er
of
pu
blic
atio
ns
0
100
200
300
400
500
600
700
EPJC
JHEP
Nuc
l. Phy
s. B
Phys. L
ett.
B
Phys. R
ev. D
Phys. R
ev. L
ett.
Z. Phy
s. C
New
J.P
hys.
EPL
Astro
p.
Phys. R
ep.
CPC
JINST
NIM
A
NIM
BTN
S
EPJC
JHEP
Nucl. Phys. B
Phys. Lett. B
Phys. Rev. D
Phys. Rev. Lett.
Z. Phys. C
New J. Phys
EPL
Astrop. Phys.
Phys. Rep.
CPC
JINST
NIM A
NIM B
IEEE TNS
Maria Grazia Pia, INFN Genova 18
Journals: pre- and post-2000
! IEEE TNS is the most popular journal for HEP technological publications in recent years
Journals, publications 1982−1999
Journal
Nu
mb
er
of
pu
blic
atio
ns
0
100
200
300
400
EPJC
JHEP
Nuc
l. Phy
s. B
Phys. L
ett.
B
Phys. R
ev. D
Phys. R
ev. L
ett.
Z. Phy
s. C
New
J.P
hys.
EPL
Astro
p.
Phys. R
ep.
CPC
JINST
NIM
A
NIM
BTN
S
EPJC
JHEP
Nucl. Phys. B
Phys. Lett. B
Phys. Rev. D
Phys. Rev. Lett.
Z. Phys. C
New J. Phys
EPL
Astrop. Phys.
Phys. Rep.
CPC
JINST
NIM A
NIM B
IEEE TNS
Journals, publications since 2000
Journal
Nu
mb
er
of
pu
blic
atio
ns
0
100
200
300
400
EPJC
JHEP
Nuc
l. Phy
s. B
Phys. L
ett.
B
Phys. R
ev. D
Phys. R
ev. L
ett.
Z. Phy
s. C
New
J.P
hys.
EPL
Astro
p.
Phys. R
ep.
CPC
JINST
NIM
A
NIM
BTN
S
EPJC
JHEP
Nucl. Phys. B
Phys. Lett. B
Phys. Rev. D
Phys. Rev. Lett.
Z. Phys. C
New J. Phys
EPL
Astrop. Phys.
Phys. Rep.
CPC
JINST
NIM A
NIM B
IEEE TNS
Maria Grazia Pia, INFN Genova 19
Citations The most cited papers are often the general reference papers about the detector published by each experiment
Citations of the most cited paper ALEPH: 340 DELPHI: 309 L3: 509 OPAL: 473 BaBar: 859 ALICE: 116 CMS: 129 LHCb: 101 TOTEM: 35 ATLAS: ATLAS pixel detector electronics and sensors: 185
Physics
CitationsN
um
be
r o
f p
ublic
atio
ns
0 20 40 60 80 1000
10
20
30
40
50
60
70
Hardware
Citations
Nu
mb
er
of
pu
blic
atio
ns
0 20 40 60 80 1000
20
40
60
80
100
120
140
DAQ−trigger
Citations
Nu
mb
er
of
pu
blic
atio
ns
0 20 40 60 80 1000
10
20
30
40
50
60
70
Software
CitationsN
um
be
r o
f p
ublic
atio
ns
0 20 40 60 80 1000
10
20
30
40
0 citations: 4% 0 citations: 17%
0 citations: 27% 0 citations: 25%
Maria Grazia Pia, INFN Genova 20
More references
more citations
References Physics papers cite
more references than technological
papers
Bibliographical entries in software papers are often
web sites
Physics
ReferencesN
um
be
r o
f p
ublic
atio
ns
0 20 40 60 80 1000
10
20
30
40
50
Hardware
References
Nu
mb
er
of
pu
blic
atio
ns
0 20 40 60 80 1000
10
20
30
40
50
60
DAQ−Trigger
References
Nu
mb
er
of
pu
blic
atio
ns
0 20 40 60 80 1000
5
10
15
20
Software
ReferencesN
um
be
r o
f p
ublic
atio
ns
0 20 40 60 80 1000
2
4
6
8
10
12
14
Maria Grazia Pia, INFN Genova 21
Pages ! The number of
pages of a paper depends on the format of the journal − 1 pageTNS ≈ 2.5 pagesJINST
! Different journal formats in the same category
! Evolutions of the format of some journals (e.g. NIM)
Physics
PagesN
umbe
r of p
ublic
atio
ns
0 10 20 30 40 500
50
100
150
200
Hardware
Pages
Num
ber o
f pub
licat
ions
0 10 20 30 40 500
20
40
60
80
100
DAQ−Trigger
Pages
Num
ber o
f pub
licat
ions
0 10 20 30 40 500
10
20
30
40
50
Software
PagesN
umbe
r of p
ublic
atio
ns
0 10 20 30 40 500
5
10
15
20
Maria Grazia Pia, INFN Genova 22
Sources of citations to physics papers
0 5 10 15 20 25
Nucl. Phys. A Phys. Atom. Nucl. Mod. Phys. Lett. A
Phys. Rep. NIM A
J. Phys. G Acta Phys. Pol. B
Int. J. Mod. Phys. A Z. Phys. C
JHEP Phys. Rev. Lett.
Nucl. Phys. B Proc. Suppl. Nucl. Phys. B
EPJC Phys. Lett. B Phys. Rev. D
Citations (%)
DELPHI ALEPH
0 5 10 15 20 25 30
Ann. Rev. Nucl. Part. Sci. J. Cosm. Astrop. Phys.
JINST New J. Phys.
Progr. Theor. Phys. Suppl. Int. J. Mod. Phys. A
J. Phys. G Nucl. Mod. Phys. Lett. A
Nucl. Phys. A Phys. Rev. C
Acta Phys. Pol. B Phys. Rev. Lett.
EPJC Phys. Lett. B
JHEP Phys. Rev. D
Citations (%)
CMS ATLAS
LHC LEP
Samples in plots account for >90% of citations
Citations to HEP physics papers mostly come from journals specialized in HEP and a few related fields (astroparticle and nuclear physics)
Maria Grazia Pia, INFN Genova 23
Sources of citations to technological papers
0 10 20 30 40 50 60
Int. J. Mod. Phys A
Comp.Phys. Comm.
Phys. Lett. B
Nucl. Phys. B Proc. Suppl.
JHEP
Phys. Rev. D
EPJC
JINST
TNS
NIM A
Citations (%)
CMS ATLAS
0 10 20 30 40
Rep. Prog. Phys. Rev. Mod. Phys.
Ann. Rev. Nucl. Part. Sci. Phys. Rep.
Int. J. Mod. Phys. A Acta Phys. Pol. B
JHEP Phys. Rev. D
Nucle. Phys. B Comp. Phys. Comm.
Z. Phys. C Nucl. Phys. B Proc. Suppl.
TNS Phys. Lett. B
EPJC NIM A
Citations (%)
DELPHI ALEPH
Citations from HEP physics and technology journals
LHC LEP
Maria Grazia Pia, INFN Genova 24
2008-2011 More refined analysis of technological papers published since start of LHC run
0
5
10
15
20
25
ATLAS CMS LHCb ALICE TOTEM LHC
Num
ber o
f pap
ers
TNS 2008-2011
Hardware Software DAQ-trigger
0
5
10
15
20
25
30
35
40
45
50
ATLAS CMS LHCb ALICE TOTEM LHC
Num
ber o
f pap
ers
NIM 2008-2011
Hardware Software
Maria Grazia Pia, INFN Genova 25
0
5
10
15
20
25
30
35
40
ATLAS CMS LHCb ALICE TOTEM LHC
Num
ber o
f sel
f-cita
tions
Hardware Software DAQ-trigger
0
5
10
15
20
25
30
35
40
ATLAS CMS LHCb ALICE TOTEM LHC
Num
ber o
f out
side
cita
tions
Hardware Software DAQ-trigger
TNS TNS
NIM A NIM A
Citations 2008-2011 Self-citations Outside citations
0
10
20
30
40
50
60
70
80
ATLAS CMS LHCb ALICE TOTEM LHC
Num
ber o
f sel
f-cita
tions
Hardware Software
0
10
20
30
40
50
60
70
80
ATLAS CMS LHCb ALICE TOTEM LHC
Num
ber o
f out
side
cita
tions
Hardware Software
Maria Grazia Pia, INFN Genova 26
LCG – LHC Computing Grid Sakamoto, H Data grid deployment for high energy physics in Japan CPC 2007 Shiers, J The Worldwide LHC Computing Grid (worldwide LCG) CPC 2007 Belov, S et al. LCG MCDB - a knowledgebase of Monte-Carlo simulated events CPC 2008 Yin, Fet al. Grid resource management policies for load-balancing and energy-saving
by vacation queuing theory CPC 2009
Malawski, M et al. Invocation of operations from script-based Grid applications Fut. Gen. Comp. Syst.
2010
Huedo, E et al. A modular meta-scheduling architecture for interfacing with pre-WS and WS Grid resource management services
Fut. Gen. Comp. Syst.
2007
Agarwal, A et al. GridX1: A Canadian computational grid Fut. Gen. Comp. Syst.
2007
Chytracek, R et al. POOL development status and production experience TNS 2005 Hatlo, M et al. Developments of mathematical software libraries for the LHC experiments TNS 2005 Pfeiffer, A et al. The LCG PI project: Using interfaces for physics data analysis TNS 2005 Munro, C et al. Measurement of the LCG2 and gLite File Catalogue's performance TNS 2006 Li, H Realistic Workload Modeling and Its Performance Impacts in Large-Scale
eScience Grids IEEE Trans. Par. Distr. Syst.
2010
Andreeva, J et al. High-Energy Physics on the Grid: the ATLAS and CMS Experience J. Grid Comp. 2008 Munoz, VM et al. A Decentralized Deployment Strategy and Performance Evaluation of
LCG File Catalog Service J. Grid Comp. 2011
Hou, S et al. PacCAF: a Grid Portal in Pacific Asia for the CDF Experiment J. Grid Comp. 2009 Kim, BK et al. A Composition of Monitoring Services for the LHC Computing Grid J. Grid Comp. 2009
WoS
Maria Grazia Pia, INFN Genova 27
LCG
2004 2006 2008 2010 20120
1
2
3
4
Publication years
Year
Nu
mb
er
of
pu
blic
atio
ns
Journals
Journal
Nu
mb
er
of
pu
blic
atio
ns
0
1
2
3
4
5
CPC
Fut. G
en.
TNS
Tran
s. P
ar.
J Grid
Com
p
0 5 10 15 20 25 30 350
1
2
3
4
Citations
Citations
Nu
mb
er
of
pu
blic
atio
ns
0 10 20 30 40 500
1
2
3
4
References
References
Nu
mb
er
of
pu
blic
atio
ns
Small sample of publications Hard to perform any statistical analysis
Maria Grazia Pia, INFN Genova 28
Conclusions ! Software is largely underrepresented in
HEP scholarly literature w.r.t. hardware ! Publication patterns appear similar in
the LEP and LHC era ! Citation patterns are different for
publications by HEP experiments and about general software tools
Publish! …and don’t forget to cite