arX

iv:0

909.

3981

v1 [

hep-

ex]

22

Sep

2009

1

We study the decay B− → DK∗− using a sample of 379×106 Υ(4S) → BB events collected with the BABARdetector at the PEP-II B-factory. We perform a “GLW” analysis where the D meson decays into either a CP -even(CP+) eigenstate (K+K−, π+π−), CP -odd (CP−) eigenstate (K0

Sπ0, K0Sφ, K0

Sω) or a non-CP state (K−π+). We

also analyze D meson decays into K+π− from a Cabibbo-favored D0

decay or doubly suppressed D0 decay (“ADS”analysis). We measure observables that are sensitive to the CKM angle γ: the partial-rate charge asymmetries ACP±,the ratios RCP± of the B-decay branching fractions in CP± and non-CP decay, the ratio RADS of the charge-averaged branching fractions, and the charge asymmetry AADS of the ADS decays: ACP+ = 0.09 ± 0.13 ± 0.06,ACP− = −0.23± 0.21± 0.07, RCP+ = 2.17± 0.35± 0.09, RCP− = 1.03± 0.27± 0.13, RADS = 0.066± 0.031± 0.010,and AADS = −0.34 ± 0.43 ± 0.16, where the first uncertainty is statistical and the second is systematic. Combiningall the measurements and using a frequentist approach yields the magnitude of the ratio between the Cabibbo-suppressed and favored amplitudes, rB = 0.31 with a one (two) sigma confidence level interval of [0.24, 0.38] ([0.17,0.43]). The value rB = 0 is excluded at the 3.3 sigma level. A similar analysis excludes values of γ in the intervals[0, 7]◦, [55, 111]◦, and [175, 180]◦ ([85, 99]◦) at the one (two) sigma confidence level.

arX

iv:0

909.

3981

v1 [

hep-

ex]

22

Sep

2009

BABAR-PUB-09/019SLAC-PUB-13793

Measurement of CP violation observables and parameters for the decays B±

→ DK∗±

B. Aubert, Y. Karyotakis, J. P. Lees, V. Poireau, E. Prencipe, X. Prudent, and V. TisserandLaboratoire d’Annecy-le-Vieux de Physique des Particules (LAPP),

Universite de Savoie, CNRS/IN2P3, F-74941 Annecy-Le-Vieux, France

J. Garra Tico and E. GraugesUniversitat de Barcelona, Facultat de Fisica, Departament ECM, E-08028 Barcelona, Spain

M. Martinelliab, A. Palanoab, and M. Pappagalloab

INFN Sezione di Baria; Dipartimento di Fisica, Universita di Barib, I-70126 Bari, Italy

G. Eigen, B. Stugu, and L. SunUniversity of Bergen, Institute of Physics, N-5007 Bergen, Norway

M. Battaglia, D. N. Brown, L. T. Kerth, Yu. G. Kolomensky, G. Lynch, I. L. Osipenkov, K. Tackmann, and T. TanabeLawrence Berkeley National Laboratory and University of California, Berkeley, California 94720, USA

C. M. Hawkes, N. Soni, and A. T. WatsonUniversity of Birmingham, Birmingham, B15 2TT, United Kingdom

H. Koch and T. SchroederRuhr Universitat Bochum, Institut fur Experimentalphysik 1, D-44780 Bochum, Germany

D. J. Asgeirsson, B. G. Fulsom, C. Hearty, T. S. Mattison, and J. A. McKennaUniversity of British Columbia, Vancouver, British Columbia, Canada V6T 1Z1

M. Barrett, A. Khan, and A. Randle-CondeBrunel University, Uxbridge, Middlesex UB8 3PH, United Kingdom

V. E. Blinov, A. D. Bukin,∗ A. R. Buzykaev, V. P. Druzhinin, V. B. Golubev,

A. P. Onuchin, S. I. Serednyakov, Yu. I. Skovpen, E. P. Solodov, and K. Yu. TodyshevBudker Institute of Nuclear Physics, Novosibirsk 630090, Russia

M. Bondioli, S. Curry, I. Eschrich, D. Kirkby, A. J. Lankford, P. Lund, M. Mandelkern, E. C. Martin, and D. P. StokerUniversity of California at Irvine, Irvine, California 92697, USA

H. Atmacan, J. W. Gary, F. Liu, O. Long, G. M. Vitug, and Z. YasinUniversity of California at Riverside, Riverside, California 92521, USA

V. SharmaUniversity of California at San Diego, La Jolla, California 92093, USA

C. Campagnari, T. M. Hong, D. Kovalskyi, M. A. Mazur, and J. D. RichmanUniversity of California at Santa Barbara, Santa Barbara, California 93106, USA

T. W. Beck, A. M. Eisner, C. A. Heusch, J. Kroseberg, W. S. Lockman,A. J. Martinez, T. Schalk, B. A. Schumm, A. Seiden, L. Wang, and L. O. Winstrom

University of California at Santa Cruz, Institute for Particle Physics, Santa Cruz, California 95064, USA

C. H. Cheng, D. A. Doll, B. Echenard, F. Fang, D. G. Hitlin,

I. Narsky, P. Ongmongkolkul, T. Piatenko, and F. C. PorterCalifornia Institute of Technology, Pasadena, California 91125, USA

R. Andreassen, G. Mancinelli, B. T. Meadows, K. Mishra, and M. D. Sokoloff

2

University of Cincinnati, Cincinnati, Ohio 45221, USA

P. C. Bloom, W. T. Ford, A. Gaz, J. F. Hirschauer, M. Nagel, U. Nauenberg, J. G. Smith, and S. R. WagnerUniversity of Colorado, Boulder, Colorado 80309, USA

R. Ayad,† W. H. Toki, and R. J. WilsonColorado State University, Fort Collins, Colorado 80523, USA

E. Feltresi, A. Hauke, H. Jasper, T. M. Karbach, J. Merkel, A. Petzold, B. Spaan, and K. WackerTechnische Universitat Dortmund, Fakultat Physik, D-44221 Dortmund, Germany

M. J. Kobel, R. Nogowski, K. R. Schubert, and R. SchwierzTechnische Universitat Dresden, Institut fur Kern- und Teilchenphysik, D-01062 Dresden, Germany

D. Bernard, E. Latour, and M. VerderiLaboratoire Leprince-Ringuet, CNRS/IN2P3, Ecole Polytechnique, F-91128 Palaiseau, France

P. J. Clark, S. Playfer, and J. E. WatsonUniversity of Edinburgh, Edinburgh EH9 3JZ, United Kingdom

M. Andreottiab, D. Bettonia, C. Bozzia, R. Calabreseab, A. Cecchiab, G. Cibinettoab, E. Fioravantiab,

P. Franchiniab, E. Luppiab, M. Muneratoab, M. Negriniab, A. Petrellaab, L. Piemontesea, and V. Santoroab

INFN Sezione di Ferraraa; Dipartimento di Fisica, Universita di Ferrarab, I-44100 Ferrara, Italy

R. Baldini-Ferroli, A. Calcaterra, R. de Sangro, G. Finocchiaro,

S. Pacetti, P. Patteri, I. M. Peruzzi,‡ M. Piccolo, M. Rama, and A. ZalloINFN Laboratori Nazionali di Frascati, I-00044 Frascati, Italy

R. Contriab, E. Guidoab, M. Lo Vetereab, M. R. Mongeab, S. Passaggioa, C. Patrignaniab, E. Robuttia, and S. Tosiab

INFN Sezione di Genovaa; Dipartimento di Fisica, Universita di Genovab, I-16146 Genova, Italy

K. S. Chaisanguanthum and M. MoriiHarvard University, Cambridge, Massachusetts 02138, USA

A. Adametz, J. Marks, S. Schenk, and U. UwerUniversitat Heidelberg, Physikalisches Institut, Philosophenweg 12, D-69120 Heidelberg, Germany

F. U. Bernlochner, V. Klose, H. M. Lacker, T. Lueck, and A. VolkHumboldt-Universitat zu Berlin, Institut fur Physik, Newtonstr. 15, D-12489 Berlin, Germany

D. J. Bard, P. D. Dauncey, and M. TibbettsImperial College London, London, SW7 2AZ, United Kingdom

P. K. Behera, M. J. Charles, and U. MallikUniversity of Iowa, Iowa City, Iowa 52242, USA

J. Cochran, H. B. Crawley, L. Dong, V. Eyges, W. T. Meyer, S. Prell, E. I. Rosenberg, and A. E. RubinIowa State University, Ames, Iowa 50011-3160, USA

Y. Y. Gao, A. V. Gritsan, and Z. J. GuoJohns Hopkins University, Baltimore, Maryland 21218, USA

N. Arnaud, J. Bequilleux, A. D’Orazio, M. Davier, D. Derkach, J. Firmino da Costa,G. Grosdidier, F. Le Diberder, V. Lepeltier, A. M. Lutz, B. Malaescu, S. Pruvot,

P. Roudeau, M. H. Schune, J. Serrano, V. Sordini,§ A. Stocchi, and G. WormserLaboratoire de l’Accelerateur Lineaire, IN2P3/CNRS et Universite Paris-Sud 11,

Centre Scientifique d’Orsay, B. P. 34, F-91898 Orsay Cedex, France

3

D. J. Lange and D. M. WrightLawrence Livermore National Laboratory, Livermore, California 94550, USA

I. Bingham, J. P. Burke, C. A. Chavez, J. R. Fry, E. Gabathuler,

R. Gamet, D. E. Hutchcroft, D. J. Payne, and C. TouramanisUniversity of Liverpool, Liverpool L69 7ZE, United Kingdom

A. J. Bevan, C. K. Clarke, F. Di Lodovico, R. Sacco, and M. SigamaniQueen Mary, University of London, London, E1 4NS, United Kingdom

G. Cowan, S. Paramesvaran, and A. C. WrenUniversity of London, Royal Holloway and Bedford New College, Egham, Surrey TW20 0EX, United Kingdom

D. N. Brown and C. L. DavisUniversity of Louisville, Louisville, Kentucky 40292, USA

A. G. Denig, M. Fritsch, W. Gradl, and A. HafnerJohannes Gutenberg-Universitat Mainz, Institut fur Kernphysik, D-55099 Mainz, Germany

K. E. Alwyn, D. Bailey, R. J. Barlow, G. Jackson, G. D. Lafferty, T. J. West, and J. I. YiUniversity of Manchester, Manchester M13 9PL, United Kingdom

J. Anderson, C. Chen, A. Jawahery, D. A. Roberts, G. Simi, and J. M. TuggleUniversity of Maryland, College Park, Maryland 20742, USA

C. Dallapiccola and E. SalvatiUniversity of Massachusetts, Amherst, Massachusetts 01003, USA

R. Cowan, D. Dujmic, P. H. Fisher, S. W. Henderson, G. Sciolla, M. Spitznagel, R. K. Yamamoto, and M. ZhaoMassachusetts Institute of Technology, Laboratory for Nuclear Science, Cambridge, Massachusetts 02139, USA

P. M. Patel, S. H. Robertson, and M. SchramMcGill University, Montreal, Quebec, Canada H3A 2T8

P. Biassoniab, A. Lazzaroab, V. Lombardoa, F. Palomboab, and S. Strackaab

INFN Sezione di Milanoa; Dipartimento di Fisica, Universita di Milanob, I-20133 Milano, Italy

L. Cremaldi, R. Godang,¶ R. Kroeger, P. Sonnek, D. J. Summers, and H. W. ZhaoUniversity of Mississippi, University, Mississippi 38677, USA

M. Simard and P. TarasUniversite de Montreal, Physique des Particules, Montreal, Quebec, Canada H3C 3J7

H. NicholsonMount Holyoke College, South Hadley, Massachusetts 01075, USA

G. De Nardoab, L. Listaa, D. Monorchioab, G. Onoratoab, and C. Sciaccaab

INFN Sezione di Napolia; Dipartimento di Scienze Fisiche,Universita di Napoli Federico IIb, I-80126 Napoli, Italy

G. Raven and H. L. SnoekNIKHEF, National Institute for Nuclear Physics and High Energy Physics, NL-1009 DB Amsterdam, The Netherlands

C. P. Jessop, K. J. Knoepfel, J. M. LoSecco, and W. F. WangUniversity of Notre Dame, Notre Dame, Indiana 46556, USA

G. Benelli, L. A. Corwin, K. Honscheid, H. Kagan, R. Kass, J. P. Morris, A. M. Rahimi, S. J. Sekula, and Q. K. WongOhio State University, Columbus, Ohio 43210, USA

4

N. L. Blount, J. Brau, R. Frey, O. Igonkina, J. A. Kolb, M. Lu,

R. Rahmat, N. B. Sinev, D. Strom, J. Strube, and E. TorrenceUniversity of Oregon, Eugene, Oregon 97403, USA

G. Castelliab, N. Gagliardiab, M. Margoniab, M. Morandina,

M. Posoccoa, M. Rotondoa, F. Simonettoab, R. Stroiliab, and C. Vociab

INFN Sezione di Padovaa; Dipartimento di Fisica, Universita di Padovab, I-35131 Padova, Italy

P. del Amo Sanchez, E. Ben-Haim, G. R. Bonneaud, H. Briand, J. Chauveau,

O. Hamon, Ph. Leruste, G. Marchiori, J. Ocariz, A. Perez, J. Prendki, and S. SittLaboratoire de Physique Nucleaire et de Hautes Energies,IN2P3/CNRS, Universite Pierre et Marie Curie-Paris6,Universite Denis Diderot-Paris7, F-75252 Paris, France

L. GladneyUniversity of Pennsylvania, Philadelphia, Pennsylvania 19104, USA

M. Biasiniab and E. Manoniab

INFN Sezione di Perugiaa; Dipartimento di Fisica, Universita di Perugiab, I-06100 Perugia, Italy

C. Angeliniab, G. Batignaniab, S. Bettariniab, G. Calderiniab,∗∗ M. Carpinelliab,†† A. Cervelliab, F. Fortiab,

M. A. Giorgiab, A. Lusianiac, M. Morgantiab, N. Neriab, E. Paoloniab, G. Rizzoab, and J. J. Walsha

INFN Sezione di Pisaa; Dipartimento di Fisica, Universita di Pisab; Scuola Normale Superiore di Pisac, I-56127 Pisa, Italy

D. Lopes Pegna, C. Lu, J. Olsen, A. J. S. Smith, and A. V. TelnovPrinceton University, Princeton, New Jersey 08544, USA

F. Anullia, E. Baracchiniab, G. Cavotoa, R. Facciniab, F. Ferrarottoa, F. Ferroniab, M. Gasperoab,

P. D. Jacksona, L. Li Gioia, M. A. Mazzonia, S. Morgantia, G. Pireddaa, F. Rengaab, and C. Voenaa

INFN Sezione di Romaa; Dipartimento di Fisica,Universita di Roma La Sapienzab, I-00185 Roma, Italy

M. Ebert, T. Hartmann, H. Schroder, and R. WaldiUniversitat Rostock, D-18051 Rostock, Germany

T. Adye, B. Franek, E. O. Olaiya, and F. F. WilsonRutherford Appleton Laboratory, Chilton, Didcot, Oxon, OX11 0QX, United Kingdom

S. Emery, L. Esteve, G. Hamel de Monchenault, W. Kozanecki, G. Vasseur, Ch. Yeche, and M. ZitoCEA, Irfu, SPP, Centre de Saclay, F-91191 Gif-sur-Yvette, France

M. T. Allen, D. Aston, R. Bartoldus, J. F. Benitez, R. Cenci, J. P. Coleman, M. R. Convery,

J. C. Dingfelder, J. Dorfan, G. P. Dubois-Felsmann, W. Dunwoodie, R. C. Field, M. Franco Sevilla,

A. M. Gabareen, M. T. Graham, P. Grenier, C. Hast, W. R. Innes, J. Kaminski, M. H. Kelsey, H. Kim,P. Kim, M. L. Kocian, D. W. G. S. Leith, S. Li, B. Lindquist, S. Luitz, V. Luth, H. L. Lynch,

D. B. MacFarlane, H. Marsiske, R. Messner,∗ D. R. Muller, H. Neal, S. Nelson, C. P. O’Grady, I. Ofte,

M. Perl, B. N. Ratcliff, A. Roodman, A. A. Salnikov, R. H. Schindler, J. Schwiening, A. Snyder, D. Su,

M. K. Sullivan, K. Suzuki, S. K. Swain, J. M. Thompson, J. Va’vra, A. P. Wagner, M. Weaver, C. A. West,

W. J. Wisniewski, M. Wittgen, D. H. Wright, H. W. Wulsin, A. K. Yarritu, C. C. Young, and V. ZieglerSLAC National Accelerator Laboratory, Stanford, California 94309 USA

X. R. Chen, H. Liu, W. Park, M. V. Purohit, R. M. White, and J. R. WilsonUniversity of South Carolina, Columbia, South Carolina 29208, USA

M. Bellis, P. R. Burchat, A. J. Edwards, and T. S. MiyashitaStanford University, Stanford, California 94305-4060, USA

5

S. Ahmed, M. S. Alam, J. A. Ernst, B. Pan, M. A. Saeed, and S. B. ZainState University of New York, Albany, New York 12222, USA

A. SofferTel Aviv University, School of Physics and Astronomy, Tel Aviv, 69978, Israel

S. M. Spanier and B. J. WogslandUniversity of Tennessee, Knoxville, Tennessee 37996, USA

R. Eckmann, J. L. Ritchie, A. M. Ruland, C. J. Schilling, R. F. Schwitters, and B. C. WrayUniversity of Texas at Austin, Austin, Texas 78712, USA

B. W. Drummond, J. M. Izen, and X. C. LouUniversity of Texas at Dallas, Richardson, Texas 75083, USA

F. Bianchiab, D. Gambaab, and M. Pelliccioniab

INFN Sezione di Torinoa; Dipartimento di Fisica Sperimentale, Universita di Torinob, I-10125 Torino, Italy

M. Bombenab, L. Bosisioab, C. Cartaroab, G. Della Riccaab, L. Lanceriab, and L. Vitaleab

INFN Sezione di Triestea; Dipartimento di Fisica, Universita di Triesteb, I-34127 Trieste, Italy

V. Azzolini, N. Lopez-March, F. Martinez-Vidal, D. A. Milanes, and A. OyangurenIFIC, Universitat de Valencia-CSIC, E-46071 Valencia, Spain

J. Albert, Sw. Banerjee, B. Bhuyan, H. H. F. Choi, K. Hamano, G. J. King,

R. Kowalewski, M. J. Lewczuk, I. M. Nugent, J. M. Roney, and R. J. SobieUniversity of Victoria, Victoria, British Columbia, Canada V8W 3P6

T. J. Gershon, P. F. Harrison, J. Ilic, T. E. Latham, G. B. Mohanty, and E. M. T. PuccioDepartment of Physics, University of Warwick, Coventry CV4 7AL, United Kingdom

H. R. Band, X. Chen, S. Dasu, K. T. Flood, Y. Pan, R. Prepost, C. O. Vuosalo, and S. L. WuUniversity of Wisconsin, Madison, Wisconsin 53706, USA

We study the decay B−→ DK∗− using a sample of 379×106 Υ (4S) → BB events collected

with the BABAR detector at the PEP-II B-factory. We perform a “GLW” analysis where the D

meson decays into either a CP -even (CP+) eigenstate (K+K−, π+π−), CP -odd (CP−) eigenstate(K0

Sπ0, K0Sφ, K0

Sω) or a non-CP state (K−π+). We also analyze D meson decays into K+π−

from a Cabibbo-favored D0

decay or doubly suppressed D0 decay (“ADS” analysis). We measureobservables that are sensitive to the CKM angle γ: the partial-rate charge asymmetries ACP±,the ratios RCP± of the B-decay branching fractions in CP± and non-CP decay, the ratio RADS

of the charge-averaged branching fractions, and the charge asymmetry AADS of the ADS decays:ACP+ = 0.09 ± 0.13 ± 0.06, ACP− = −0.23 ± 0.21 ± 0.07, RCP+ = 2.17 ± 0.35 ± 0.09, RCP− =1.03± 0.27± 0.13, RADS = 0.066± 0.031 ± 0.010, and AADS = −0.34± 0.43± 0.16, where the firstuncertainty is statistical and the second is systematic. Combining all the measurements and using afrequentist approach yields the magnitude of the ratio between the Cabibbo-suppressed and favoredamplitudes, rB = 0.31 with a one (two) sigma confidence level interval of [0.24, 0.38] ([0.17, 0.43]).The value rB = 0 is excluded at the 3.3 sigma level. A similar analysis excludes values of γ in theintervals [0, 7]◦, [55, 111]◦, and [175, 180]◦ ([85, 99]◦) at the one (two) sigma confidence level.

PACS numbers: 13.25.Hw, 14.40.Nd, 12.15.Hh, 11.30.Er

∗Deceased†Now at Temple University, Philadelphia, Pennsylvania 19122,USA‡Also with Universita di Perugia, Dipartimento di Fisica, Perugia,Italy

§Also with Universita di Roma La Sapienza, I-00185 Roma, Italy¶Now at University of South Alabama, Mobile, Alabama 36688,USA∗∗Also with Laboratoire de Physique Nucleaire et de Hautes Ener-

6

I. INTRODUCTION

The Standard Model accommodates CP violationthrough a single phase in the Cabibbo-Kobayashi-Maskawa (CKM) quark mixing matrix V [1]. The selfconsistency of this mechanism can be tested by over-constraining the associated unitarity triangle [2, 3] usingmany different measurements, mostly involving decaysof B mesons. In this paper we concentrate on the angleγ ≡ arg(−VudV

∗ub/VcdV

∗cb) by studying B meson decay

channels where b → cus and b → ucs tree amplitudes in-terfere. We use two techniques, one suggested by Gronauand London [4] and Gronau and Wyler [5] (GLW) and theother suggested by Atwood, Dunietz and Soni [6] (ADS)to study γ. Both techniques rely on final states that canbe reached from both D0 and D0 decays. As discussedin Ref. [7] the combination of the GLW and ADS observ-ables can be very useful in resolving certain ambiguitiesinherent in each of the techniques. In this paper we usethe decay B− → DK∗−(892) [8] to measure the GLWand ADS observables.

In the GLW analysis the D meson [9] from B− →DK∗− decays into either a CP -even (CP+) eigen-state (K+K−, π+π−) or a CP -odd (CP−) eigenstate(Ksπ

0, Ksφ, Ksω). The size of the interference betweenthe two competing amplitudes depends on the CKM an-gle γ as well as other parameters that are CP -conserving,discussed below. References [4, 5] define several observ-ables that depend on measurable quantities:

RCP± = 2Γ(B−

→ D0

CP±K∗−) + Γ(B+→ D0

CP±K∗+)

Γ(B− → D0Kπ

K∗−) + Γ(B+ → D0Kπ

K∗+),

ACP± =Γ(B− → D0

CP±K∗−) − Γ(B+ → D0

CP±K∗+)

Γ(B− → D0

CP±K∗−) + Γ(B+ → D0

CP±K∗+).

Here D0CP± refers to a neutral D meson decaying into

either a CP+ or CP− eigenstate.

RCP± and ACP± depend on the physical parametersas follows:

RCP± = 1 + r2B ± 2rB cos δB cos γ, (1)

ACP± = ±2rB sin δB sin γ/RCP±. (2)

Here rB is the magnitude of the ratio of the suppressedB− → D0K∗− and favored B− → D0K∗− decay ampli-tudes, respectively, and δB is the CP -conserving phasedifference between these amplitudes. In this analysis weneglect the effects of CP violation in D meson decays,and, as justified in Ref. [10], the very small effect of D0D0

mixing.

gies, IN2P3/CNRS, Universite Pierre et Marie Curie-Paris6, Uni-versite Denis Diderot-Paris7, F-75252 Paris, France††Also with Universita di Sassari, Sassari, Italy

RCP± is calculated using:

RCP± =NCP±

Nnon−CP

× ǫnon−CP

ǫCP±

(3)

where NCP± and Nnon−CP are the event yields for theCP and non-CP modes, respectively and ǫnon−CP andǫCP± are correction factors that depend on branchingfractions and reconstruction efficiencies. ACP± is calcu-lated using the event yields split by the charge of the Bmeson.

We define two additional quantities whose experimen-tal estimators are normally distributed even when thevalue of rB is comparable to its uncertainty:

x± = rB cos(δB ± γ) (4)

=RCP+(1 ∓ACP+) − RCP−(1 ∓ACP−)

4.

Since x± are also directly measured in Dalitz-plot analy-ses [11], the different results can be compared and com-bined with each other. We note that an additional setof quantities measured in Dalitz-plot analyses, y± =rB sin(δB ± γ), are not directly accessible through theGLW analysis.

In the ADS technique, B− → DK∗− decays to[K+π−]DK∗−, where [K+π−]D indicates that these par-ticles are neutral D meson (D0 or D0) decay products.This final state can be reached from B → D0K∗− andthe doubly-Cabibbo-suppressed decay D0 → K+π− orB− → D0K∗− followed by the Cabibbo-favored decayD0 → K+π−. In addition, the final state [K−π+]DK∗−

is used for normalization. We label the decays wherethe K and K∗ have the same (opposite) charge as “right(wrong) sign” where the labels reflect that one mode oc-curs much more often than the other.

In analogy with the GLW method we define two mea-surable quantities, RADS and AADS , as follows:

RADS =Γ(B− → [K+π−]DK∗−) + Γ(B+ → [K−π+]DK∗+)

Γ(B− → [K−π+]DK∗−) + Γ(B+ → [K+π−]DK∗+),

AADS =Γ(B− → [K+π−]DK∗−) − Γ(B+ → [K−π+]DK∗+)

Γ(B− → [K+π−]DK∗−) + Γ(B+ → [K−π+]DK∗+).

RADS and AADS are related to physically interestingquantities by:

RADS = r2D + r2

B + 2rDrB cos(δB + δD) cos γ, (5)

AADS = 2rDrB sin(δB + δD) sin γ/RADS . (6)

Here rD is the magnitude of the ratio of suppressedD0 → K+π− and favored D0 → K−π+ decay am-plitudes, respectively, while δD is the CP -conservingstrong phase difference between these two amplitudes.Both rD and δD have been measured and we use thevalues given in Ref. [12]: rD = 0.0578 ± 0.0008 andδD = 21.9+11.3

−12.4 degrees. Estimates for rB are in the range0.1 ≤ rB ≤ 0.3 [13, 14].

It has been pointed out in Ref. [13] that complicationsdue to possible variations in rB and/or δB as a resultof the finite width of a resonance such as the K∗ and its

7

overlap with other states can be taken into account usingan alternate formalism. However, in this paper we chooseto follow the procedures in Refs. [14, 15] and incorporatethe effects of the non-K∗ DKπ events and finite widthof the K∗ into the systematic uncertainties of our A andR measurements.

II. THE BABAR DETECTOR AND DATASET

The BABAR detector has been described in detail inRef. [16] and therefore will only be briefly discussed here.The trajectories of charged tracks are measured with afive-layer double-sided silicon vertex tracker (SVT) and a40-layer drift chamber (DCH). Both the SVT and DCHare located inside a 1.5 T magnetic field. Photons aredetected by means of a CsI(Tl) crystal calorimeter alsolocated inside the magnet. Charged particle identifica-tion is determined from information provided by a ring-imaging Cherenkov device (DIRC) in combination withionization measurements (dE/dx) from the tracking de-tectors. The BABAR detector’s response to various physicsprocesses as well as varying beam and environmental con-ditions is modeled with simulation software based on theGeant4 [17] toolkit. We use EvtGen [18] to model thekinematics of B mesons and Jetset [19] to model con-tinuum processes (e+e− → cc, uu, dd, ss). This analysisuses data collected at and near the Υ (4S) resonance withthe BABAR detector at the PEP-II storage ring. The dataset consists of 345 fb−1 collected at the peak of the Υ (4S)(379 ×106 BB pairs) and 35 fb−1 collected 40 MeV be-low the resonance peak (off-peak data).

This analysis is a combined update of the previousBABAR ADS [14] and GLW [15] studies of B− → DK∗−,which used 232 ×106 BB pairs. Other new features inthis analysis include the improvement in background sup-pression, the refinement of various candidate selectioncriteria, and an update of the estimation of systematicuncertainties. The major change is the choice of neuralnetworks in the GLW analysis over Fisher discriminants,which were used in the previous analysis. We verify theimprovements on both signal efficiency and continuumbackground rejection in the GLW decay channels withsimulated signal and continuum events. The increases insignal efficiency range from 3% to 14% for all channels ex-cept K0

Sφ, which has the same efficiency. For continuum

suppression, the neural networks perform 10% to 57%better across all channels except K+K−, which displaysthe same performance.

III. THE GLW ANALYSIS

We reconstruct B− → DK∗− candidates with thesubsequent decays K∗− → K0

Sπ−, K0

S→ π+π− and

with the D meson decaying into six decay final states:D0 → K−π+ (non-CP final state); K+K−, π+π− (CP+eigenstates); and K0

Sπ0, K0

Sφ, K0

Sω (CP− eigenstates).

We optimize our event selection criteria by maximizingthe figure of merit S/

√S + B, with S the number of sig-

nal events and B the number of background events, deter-mined for each channel using simulated signal and back-ground events. Kaon and pion candidates (except for thepions from K0

Sdecays) are selected using a likelihood-

based particle identification algorithm which relies ondE/dx information measured in the DCH and the SVT,and Cherenkov photons in the DIRC. The efficiencies ofthe selectors are typically above 85% for momenta below4 GeV while the kaon and pion misidentification rates areat the few percent level for particles in this momentumrange.

The K0S

candidates are formed from oppositely chargedtracks assumed to be pions with a reconstructed invari-ant mass within 13 MeV/c2 (four standard deviations)of the known K0

Smass [3], mK0

S. All K0

Scandidates are

refitted so that their invariant mass equals mK0

S

(mass

constraint). They are also constrained to emerge froma single vertex (vertex constraint). For those retainedto build a K∗− candidate, we further require that theirflight direction and length be consistent with a K0

Scom-

ing from the interaction point. The K0S

candidate flightpath and momentum vectors must make an acute an-gle and the flight length in the plane transverse to thebeam direction must exceed its uncertainty by three stan-dard deviations. K∗− candidates are formed from a K0

S

and a charged particle with a vertex constraint. We se-lect K∗− candidates that have an invariant mass within75 MeV/c2 of the known mean value for a K∗ [3]. Fi-nally, since the K∗− in B− → DK∗− is longitudinallypolarized, we require | cos θH | ≥ 0.35, where θH is theangle in the K∗− rest frame between the daughter pionmomentum and the parent B momentum. The helicitydistribution discriminates well between a B-meson decayand a false B-meson candidate from the continuum, sincethe former is distributed as cos2 θH and the latter has anapproximately flat distribution.

Some decay modes of the D meson contain a neutralpion. We combine pairs of photons to form π0 candi-dates with a total energy greater than 200 MeV and aninvariant mass between 115 and 150 MeV/c2. A massconstrained fit is applied to the selected π0 candidatemomenta. Composite particles (φ and ω) included inthe CP− modes are vertex-constrained. Candidate φ(ω) mesons are constructed from K+K− (π+π−π0) par-ticle combinations with an invariant mass required tobe within two standard deviations, corresponding to 12(20) MeV/c2, from the known peak values [3]. Two fur-ther requirements are made on the ω candidates. Themagnitude of the cosine of the helicity angle θH betweenthe D momentum in the rest frame of the ω and the nor-mal to the plane containing all three decay pions mustbe greater than 0.35 (this variable has a cos2 θH distri-bution for signal candidates and is approximately flat forbackground). The Dalitz angle [20] θD is defined as theangle between the momentum of one daughter pion inthe ω rest frame and the direction of one of the other

8

two pions in the rest frame of the two pions. For sig-nal candidates, the cosine of the Dalitz angle follows asin2 θD distribution, while it is approximately flat for thebackground. Therefore we require the cosine of the Dalitzangle of signal candidates to have a magnitude smallerthan 0.8.

All D candidates are mass constrained and, with theexception of the K0

Sπ0 final state, vertex constrained. We

select D candidates with an invariant mass differing fromthe known mass [3] by less than 12 MeV/c2 for all chan-nels except K0

Sπ0 (30 MeV/c2) and K0

Sω (20 MeV/c2).

These limits are about twice the corresponding RMSmass resolutions.

Suppression of backgrounds from continuum events isachieved by using event-shape and angular variables. TheB meson candidate is required to have | cos θT | ≤ 0.9,where θT is the angle between the thrust axis of the Bmeson and that of the rest of the event. The distributionof | cos θT | is uniform in BB events and strongly peakednear 1 for continuum events.

A neural network (NN) is used to further reduce thee+e− → qq (q = u, d, s, c) contribution to our data sam-ple. Seven variables are used in the NN with three beingthe angular moments L0, L1 and L2. These momentsare defined by Lj =

∑

i p∗i | cos θ∗i |j where the sum isover charged and neutral particles not associated withthe B-meson candidate. Here p∗i (θ∗i ) is the momen-tum (angle) of the ith particle with respect to the thrustof the candidate B meson in the center-of-mass (CM)frame. Additional details on the moments can be foundin Ref. [21]. The NN also uses the ratio R2 = H2/H0 ofFox-Wolfram moments [22], the cosine of the angle be-tween the B candidate momentum vector and the beamaxis (cos θB), cos θT (defined above), and the cosine ofthe angle between a D daughter momentum vector inthe D rest frame and the direction of the D in the Bmeson rest frame (cos θH(D)). The distributions of allthe above variables show distinct differences between sig-nal and continuum events and thus can be exploited bya NN to preferentially select BB events. Each decaymode has its own unique NN trained with signal andcontinuum Monte Carlo events. After training, the NNsare then fed with independent sets of signal and con-tinuum Monte Carlo events to produce NN outputs foreach decay mode. Finally, we verify that the NNs haveconsistent outputs for off-peak data (continuum data col-lected below the Υ (4S)) and qq Monte Carlo events. Theseparations between signal and continuum backgroundare shown in Fig. 1. We select candidates with neuralnetwork output above 0.65 (K+K−), 0.82 (π+π−), 0.91(K0

Sπ0), 0.56 (K0

Sφ), 0.80 (K0

Sω), and 0.73 (K−π+). Our

event selection is optimized to maximize the significanceof the signal yield, determined using simulated signal andbackground events.

We identify B candidates using two nearly indepen-dent kinematic variables: the beam-energy-substitutedmass mES =

√

(s/2 + p0 · pB)2/E20 − p2

B and the energydifference ∆E = E∗

B −√s/2, where E and p are energy

FIG. 1: Neural network (NN) outputs and results of the NNverifications of (a) K+K−, (b) π+π−, (c) K0

Sπ0, (d) K0Sφ, (e)

K0Sω, and (f) K−π+ subsamples of the GLW analysis. The

samples used to produce the output are shown as histograms.The signal (Monte Carlo simulation) is the shaded histogrampeaking near 1; the continuum (Monte Carlo simulation) isthe histogram peaking near 0. The off-peak data used tocheck the NN are overlaid as data points.

and momentum. The subscripts 0 and B refer to thee+e−-beam system and the B candidate, respectively; sis the square of the CM energy and the asterisk labelsthe CM frame. The mES distributions are all describedby a Gaussian function G centered at the B mass witha resolution (sigma) of 2.50, 2.55, and 2.51 MeV/c2 forthe CP+, CP− and non-CP mode, respectively. The∆E distributions are centered on zero for signal witha resolution of 11 to 13 MeV for all channels exceptK0

Sπ0 for which the resolution is asymmetric and is about

30 MeV. We define a signal region through the require-ment |∆E| < 50 (25) MeV for K0

Sπ0 (all other modes).

A potentially dangerous background for the B− →

9

D(π+π−)K∗−(K0Sπ−) channel is the decay mode B− →

D(K0Sπ+π−)π− which contains the same final-state par-

ticles as the signal but has a branching fraction 600times larger. We therefore explicitly veto any selectedB candidate containing a K0

Sπ+π− combination within

60 MeV/c2 of the D0 mass.The fraction of events with more than one acceptable

B candidate depends on the D decay mode and is alwaysless than 8%. To select the best B candidate in thoseevents where we find more than one acceptable candidate,we choose the one with the smallest χ2 formed from thedifferences of the measured and world average D0 andK∗− masses divided by the mass spread that includesthe resolution and, for the K∗−, the natural width:

χ2 = χ2M

D0+ χ2

MK∗−

(7)

=(MD0 − MPDG

D0 )2

σ2M

D0

+(MK∗− − MPDG

K∗− )2

σ2M

K∗−+ Γ2

K∗−/c4 .

Simulations show that negligible bias is introduced bythis choice and the correct candidate is picked at least86% of the time.

From the simulation of signal events, the total recon-struction efficiencies are: 12.8% and 12.3% for the CP+modes D → K+K− and π+π−; 5.6%, 8.9%, and 2.4% forthe CP− modes D → K0

Sπ0, K0

Sφ and K0

Sω; 12.8% for

the non-CP mode D0 → K−π+.To study BB backgrounds we look in sideband regions

in ∆E and mD. We define the ∆E sideband in the in-terval 60 ≤ ∆E ≤ 200 MeV for all modes. This region isused to determine the combinatorial background shapesin the signal and mD sideband. We choose not to usea lower sideband because of the D∗K∗ backgrounds inthat region. The sideband region in mD is slightly modedependent with a typical requirement that mD differsfrom the D0 mass by more than four and less than 10standard deviations. This region provides sensitivity tobackground sources which mimic signal both in ∆E andmES and originate from either charmed or charmless Bmeson decays that do not contain a true D meson. Asmany of the possible contributions to this backgroundare not well known, we measure its size by including themD sideband in the fit described below.

An unbinned extended maximum likelihood fit to themES distributions of selected B candidates in the range5.2 ≤ mES ≤ 5.3 GeV/c2 is used to determine signaland background yields. We use the signal yields to cal-culate the CP -violating quantities ACP and RCP . Weuse the same mean and width of the Gaussian functionG to describe the signal shape for all modes considered.The combinatorial background in the mES distribution ismodeled with the so-called “ARGUS” empirical thresh-old function A [23]. It is defined as:

A(mES) ∝ mES

√

1 − x2 exp−ξ(1−x2), (8)

where x = mES/Emax and Emax is the maximum mass forpair-produced B mesons given the collider beam energies

and is fixed in the fit at 5.291 GeV/c2. The ARGUSshape is governed by one parameter ξ that is left free inthe fit. We fit simultaneously mES distributions of ninesamples: the non-CP , CP+ and CP− samples for (i)the signal region, (ii) the mD sideband and (iii) the ∆Esideband. In addition the signal region is divided into twosamples according to the charge of the B candidate. Wefit three probability density functions (PDF) weighted bythe unknown event yields. For the ∆E sideband, we useA. For the mD sideband (sb) we use asb·A + bsb·G, whereG accounts for fake-D candidates. For the signal regionPDF, we use a ·A+ b · Gpeak + c · Gsignal, where b is scaledfrom bsb with the assumption that the number of fake Dbackground events present in the signal region is equal tothe number measured in the mD sideband scaled by theratio of the mD signal-window to sideband widths, andc is the number of B± → DK∗± signal events. The non-CP mode sample, with relatively high statistics, helpsconstrain the PDF shapes for the low statistics CP modedistributions. The ∆E sideband sample helps determinethe A background shape. In total, the fit determines 19event yields as well as the mean and width of the signalGaussian and the ARGUS parameter ξ.

Since the values of ξ obtained for each data sampleare consistent with each other, albeit with large statis-tical uncertainties, we have constrained ξ to have thesame value for all data samples in the fit. The simula-tion shows that the use of the same Gaussian parametersfor all signal modes introduces only negligible systematiccorrections. We assume that the fake D backgroundsfound in the mD sideband have the same final states asthe signal and we fit these contributions with the sameGaussian parameterization.

The fake D background is assumed to not violate CPand is therefore split equally between the B− and B+

sub-samples. This assumption is consistent with resultsfrom our simulations and is considered further when wediscuss the systematic uncertainties. The fit results areshown graphically in Fig. 2 and numerically in Table I.Table II records the number of events measured for eachindividual D decay mode.

TABLE I: Results from the fit. For each GLW D mode, wegive the number of measured signal events, the fake D contri-bution, ACP and RCP . The fake D contribution is calculatedby scaling the number of fake D events found in the mD

sideband region to the signal region. The uncertainties arestatistical only. We also show the number of measured signalevents split by the B charge for CP+ and CP− modes.

# Signal # Fake D ACP RCP

Non-CP 231 ± 17 5.0CP+ 68.6 ± 9.2 0.3 0.09 ± 0.13 2.17 ± 0.35(B+) 31.2 ± 6.2(B−) 37.4 ± 6.8

CP− 38.5 ± 7.0 0.0 −0.23 ± 0.21 1.03 ± 0.27(B+) 23.0 ± 4.8(B−) 15.5 ± 5.2

10

50

100

150

50

100

150 (a)

10

20

30

10

20

30 (b)

10

20

30

10

20

30 (c)

10

20

10

20 (d)

5.2 5.22 5.24 5.26 5.28 5.30

10

20

5.2 5.22 5.24 5.26 5.28 5.30

10

20 (e)

)2 (GeV/cESm

)2E

vent

s/(4

MeV

/c

FIG. 2: Distributions of mES in the signal region for (a) thenon-CP modes in B± decays, (b) the CP+ modes in B+,and (c) B− decays and (d) the CP− modes in B+ and (e)B− decays. The dashed curve indicates the total backgroundcontributions, which include the fake D backgrounds.

TABLE II: Number of signal events from the GLW fit forindividual D decay modes studied in this analysis. We alsoprovide the selection efficiencies (in %) and total correctionfactors (ǫc). The uncertainties are statistical only.

# Signal Selection Efficiency (%) ǫc (10−4)Non-CP

K−π+ 231 ± 17 12.76 ± 0.09 48.48 ± 0.96CP+

K+K− 41 ± 7 12.78 ± 0.05 4.90 ± 0.16π+π− 28 ± 6 12.34 ± 0.05 1.72 ± 0.11

CP−

K0Sπ0 21 ± 7 5.59 ± 0.03 4.10 ± 0.50

K0Sφ 8 ± 3 8.90 ± 0.04 1.30 ± 0.11

K0Sω 9 ± 4 2.35 ± 0.02 1.49 ± 0.30

Although most systematic uncertainties cancel forACP , a charge asymmetry inherent to the detector ordata processing may exist. We quote the results from thestudy carried out in Ref. [24], where we used B− → D0π−

(with D0 decays into CP or non-CP eigenstates) eventsfrom control samples of data and simulation to measurethe charge asymmetry. An average charge asymmetry ofAch = (−1.6 ± 0.6)% was measured. We add linearly

the central value and one-standard deviation in the mostconservative direction to assign a systematic uncertaintyof 0.022. The second substantial systematic effect is apossible CP asymmetry in the fake D background thatcannot be excluded due to CP violation in charmless Bdecays. If there is an asymmetry Afake D, then the sys-tematic uncertainty on ACP is Afake D × b/c, where b isthe contribution of the fake D background and c the sig-nal yield. Assuming conservatively that |Afake D| ≤ 0.5,we obtain systematic uncertainties of ±0.003 and ±0.040on ACP+ and ACP− respectively. Note that since we donot observe any fake D background in CP− modes, weuse the statistical uncertainty of the signal yield from thefit to estimate this systematic uncertainty.

Since RCP is a ratio of rates of processes with differentfinal states of the D, we must consider the uncertain-ties affecting the selection algorithms for the different Dchannels. This results in small corrections which accountfor the difference between the actual detector responseand the simulation model. The main effects stem fromthe approximate modeling of the tracking efficiency (acorrection of 0.4% per pion track coming from a K0

Sand

0.2% per kaon and pion track coming from other candi-dates), the K0

Sreconstruction efficiency for CP− modes

of the D0 (1.3% per K0S

in K0Sφ mode and 2.0% in K0

Sπ0

and K0Sω), the π0 reconstruction efficiency for the K0

Sπ0

and K0S[π+π−π0]ω channels (3%) and the efficiency and

misidentification probabilities from the particle identifi-cation (2% per track). The corrections are calculated bycomparing data and Monte Carlo using high-statisticsand high-purity samples. Charged kaon and pion sam-ples obtained from D-meson decays (D∗+ → D0π+) areused for particle identification corrections. For track-ing corrections, we use τ -pair events where one τ de-cays to a muon and two neutrinos and the other de-cays to ρ0hν where h is a K or a π. B0 → φK0

Sand

B0 → π+D−(D− → K0Sπ−) decays are used for K0

Scor-

rections, and π0 correction factors are calculated usingτ → ρν and τ → πν samples. The total correction fac-tors, which also include branching fractions and selec-tion efficiencies, used in the calculation of RCP± (Eq. 3)are given in Table II. RCP+ (RCP−) is calculated us-ing the sum of the individual CP+ (CP−) correctionfactors. Altogether, the systematic uncertainties due tothe corrections equal ±0.078 and ±0.100 for RCP+ andRCP−, respectively. The uncertainties on the measuredbranching fractions [3] and efficiencies for different D de-cay modes, are included in the calculation of the system-atic errors due to these corrections.

Another systematic correction applied to the CP−measurements arises from a possible CP+ backgroundin the K0

Sφ and K0

Sω channels. In this case, the observed

quantities AobsCP− and Robs

CP− are corrected:

ACP− = (1 + ǫ)AobsCP− − ǫACP+; RCP− =

RobsCP−

(1 + ǫ),

where ǫ is the ratio of CP+ background to CP− signal.An investigation of the D0 → K−K+K0

SDalitz plot [25]

11

indicates that the dominant background for D0 → K0Sφ

comes from the decay a0(980) → K+K−, at the levelof (25 ± 1)% of the size of the φK0

Ssignal. We have no

information for the ωK0S

channel and assume (30± 30)%of CP+ background contamination. The K0

Sπ0 mode has

no CP+ background. The value of ǫ for the combinationof CP− modes is (11± 7)%. The systematic uncertaintyassociated with this effect is ±0.02 and ±0.06 for ACP−

and RCP−, respectively.To account for the non resonant K0

Sπ− pairs in the K∗

mass range we study a model that incorporates S-waveand P -wave pairs in both the b → cus and b → ucs am-plitudes. The P -wave mass dependence is described by asingle relativistic Breit-Wigner while the S-wave compo-nent is assumed to be a complex constant. It is expectedthat higher order partial waves will not contribute signifi-cantly and therefore they are neglected in the model. Wealso assume that the same relative amount of S and P -wave is present in the b → cus and b → ucs amplitudes.The amount of S-wave present in the favored b → cusamplitude is determined directly from the data by fit-ting the angular distribution of the K0

Sπ system in the

K∗ mass region, accounting for interference [26]. Fromthis fit we determine that the number of non-K∗ KSπ−

events is (4 ± 1)% of the measured signal events. To es-timate the systematic uncertainties due to this source wevary all the strong phases between 0 and 2π and calculatethe maximum deviation between the S-wave model andthe expectation if there were no non-resonant contribu-tion for both ACP± [Eq. (2)] and RCP± [Eq. (1)]. Thisbackground induces systematic variations of ±0.051 forACP± and ±0.035 for RCP±.

The last systematic uncertainty is due to the assump-tion that the parameters of the Gaussian and ARGUSfunctions are the same throughout the signal region, ∆Eand mD sidebands. We estimate the uncertainties byvarying the width and mean of the Gaussian and ξ of theARGUS by their corresponding statistical uncertaintiesobtained from the fit. All the systematic uncertaintiesare listed in Table III. We add them in quadrature andquote the final results:

ACP+ = 0.09 ± 0.13(stat.) ± 0.06(syst.)

ACP− = −0.23 ± 0.21(stat.) ± 0.07(syst.)

RCP+ = 2.17 ± 0.35(stat.) ± 0.09(syst.)

RCP− = 1.03 ± 0.27(stat.) ± 0.13(syst.)

These results can also be expressed in terms of x±

defined in Eq. (4):

x+ = 0.21 ± 0.14(stat.)± 0.05(syst.),

x− = 0.40 ± 0.14(stat.)± 0.05(syst.),

where the CP+ pollution systematic effects are included.Including these effects increased x+ and x− by 0.035 ±0.024 and 0.023 ± 0.017, respectively.

TABLE III: Summary of systematic uncertainties for theGLW analysis.

Source δACP+ δACP− δRCP+ δRCP−

Detection asymmetry 0.022 0.022 - -Non-resonant K0

s π− bkg. 0.051 0.051 0.035 0.035Same-final-state bkg. - 0.019 - 0.061Asymmetry in fake D0 bkg. 0.003 0.040 - -Efficiency correction - - 0.078 0.100Same G and A shape 0.003 0.013 0.009 0.025Total systematic uncertainty 0.056 0.072 0.086 0.125

IV. THE ADS ANALYSIS

In the ADS analysis we only use D decays with acharged kaon and pion in the final state and K∗− de-cays to K0

Sπ− followed by K0

S→ π+π−. The ADS event

selection criteria and procedures are nearly identical tothose used for the GLW analysis. However due to thesmall value of rD the yield of interesting ADS events(i.e. B− → [K+π−]DK∗− and B+ → [K−π+]DK∗+)is expected to be smaller than for the GLW analysis.Therefore in order to reduce the background in the ADSanalysis the K0

S invariant mass window is narrowed to10 MeV/c2 and the K∗− invariant mass cut is reduced to55 MeV/c2. A neural network using the same variablesas in the GLW analysis is trained on ADS signal andcontinuum MC events and verified using off-peak contin-uum data. The separation between signal and continuumbackground is shown in Fig. 3. We select candidates,both right and wrong-sign, with neural network outputabove 0.85. All other K0

S , K∗−, and continuum sup-pression criteria are the same as those used in the GLWanalysis.

D → K−π+ and K+π− candidates are used in thisanalysis. Candidates that have an invariant mass within18 MeV/c2 (2.5 standard deviations) of the nominal D0

mass [3] are kept for further study. We require kaoncandidates to pass the same particle identification criteriaas imposed in the GLW analysis.

We identify B-meson candidates using the beam-energy-substituted mass mES and the energy difference∆E. For this analysis signal candidates must satisfy|∆E| ≤ 25 MeV. The efficiency to detect a B− →D0K∗− signal event where D0 → Kπ, after all cri-teria are imposed, is (9.6±0.1)% and is the same forD0 → K−π+ and D0 → K+π−. In 1.8% of the eventswe find more than one suitable candidate. In such caseswe choose the candidate with the smallest χ2 defined inEq. (7). Simulations show that negligible bias is intro-duced by this choice and the correct candidate is pickedabout 88% of the time.

We study various potential sources of background us-ing a combination of Monte Carlo simulation and dataevents. Two sources of background are identified inlarge samples of simulated BB events. One source is

12

NN Output0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

%/0

.02

0

1

2

3

4

5

6

7

8

9

NN Output0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

%/0

.02

0

1

2

3

4

5

6

7

8

9

FIG. 3: Neural network (NN) output and result of the NN ver-ification for the ADS analysis (see text). The samples usedto produce the output are shown as histograms. The signal(Monte Carlo simulation) is the shaded histogram peakingnear 1; the continuum (Monte Carlo simulation) is the his-togram peaking near 0. The off-peak data used to check theNN are overlaid as data points.

B− → D0K0Sπ− production where the K0

Sπ− is nonreso-

nant and has an invariant mass in the K∗− mass window.This background is discussed later. The second back-ground (peaking background) includes instances where afavored decay (e. g. B− → [K−π+]DK∗−) contributesto fake candidates for the suppressed decay (i.e. B+ →[K−π+]DK∗+). The most common way for this to occuris for a π+ from the rest of the event to be substitutedfor the π− in the K∗− candidate. Other sources of peak-ing background include double particle-identification fail-ure in signal events that results in D0 → K−π+ beingreconstructed as D0 → π−K+, or the kaon from theD0 being interchanged with the charged pion from theK∗. We quantify this background with the ratio of thesignal efficiency of wrong-sign decay to right-sign decaymultiplied by the right-sign yield from data. The to-tal size of this right-sign pollution is estimated to be2.4 ± 0.3 events. Another class of backgrounds is charm-less decays with the same final state as the signal (e.g.,B− → K∗−K+π−). Since the branching fractions formany of these charmless decays have not been measuredor are poorly measured, we use the D sideband to es-timate the contamination from this source. From a fitto the mES distribution using candidates in the D side-band we find 0.0 ± 1.1 events. We take the 1.1 events asthe contribution to the systematic uncertainty from thissource.

Signal yields are determined from an unbinned ex-tended maximum likelihood fit to the mES distribution

in the range 5.2 ≤ mES ≤ 5.3 GeV/c2. A Gaussian func-tion (G) is used to describe all signal shapes while thecombinatorial background is modeled with an ARGUSthreshold function (A) defined in Eq. (8). The mean andwidth of the Gaussian as well as the ξ parameter of theARGUS function are determined by the fit. For the like-lihood function we use a ·A+ b · G where a is the numberof background events and b the number of signal events.We correct b for the right-sign peaking background pre-viously discussed (2.4±0.3 events).

2

4

6

8

10

12

14

5.2 5.22 5.24 5.26 5.28 5.3

0

20

40

60

80

100

120

5.2 5.22 5.24 5.26 5.28 5.3

)2

(GeV/cESm

)2

Ev

en

ts/(

4 M

eV

/c

0

FIG. 4: Distributions of mES for the wrong-sign (top) andright-sign (bottom) decays. These decay categories are de-fined in the text. The dashed curve indicates the total back-ground contribution. It also includes the right-sign peak-ing background estimated from a Monte Carlo study for thewrong-sign (top) decays. The curves result from a simulta-neous fit to these distributions with identical PDFs for bothsamples.

In Fig. 4 we show the results of a simultaneous fit toB− → [K+π−]DK∗− and B− → [K−π+]DK∗− can-didates that satisfy all selection criteria. It is in thewrong-sign decays that the interference we study takesplace. Therefore in Fig. 5 we display the same fit sepa-rately for the wrong-sign decays of the B+ and the B−

mesons. The results of the maximum likelihood fit areRADS = 0.066±0.031,AADS = −0.34±0.43, and 172.9±14.5 B− → [K−π+]DK∗− right-sign events. Expressedin terms of the wrong-sign yield, the fit result is 11.5±5.3wrong-sign events (3.8 ± 3.4 B− → [K+π−]DK∗− and7.7 ± 4.2 B+ → [K−π+]DK∗+ events). The uncertain-ties are statistical only. The correlation between RADS

and AADS is insignificant.We summarize in Table IV the systematic uncertainties

relevant to this analysis. Since both RADS and AADS

are ratios of similar quantities, most potential sources ofsystematic uncertainties cancel.

13

2

4

6

8

10

2

4

6

8

10

5.2 5.22 5.24 5.26 5.28 5.3

0

2

4

6

8

0

5.2 5.22 5.24 5.26 5.28 5.3

2

4

6

8

)2

(GeV/cESm

)2

Ev

en

ts/(

4 M

eV

/c

FIG. 5: The wrong-sign sample shown in the top plot of Fig. 4split by charge. Upper plot shows the mES distribution of theB+

→ [K−π+]DK∗+ decays while lower plot presents thesame for the B−

→ [K+π−]DK∗− decays. The dashed curveindicates the total background contribution which includesthe right-sign peaking background.

For the estimation of the detection-efficiency asymme-try we use the previously mentioned results from thestudy carried out in Ref. [24]. We add linearly the cen-tral value and one-standard deviation in the most con-servative direction to assign a systematic uncertainty ofδAch = ±0.022 to the AADS measurement. To a goodapproximation the systematic uncertainty in RADS dueto this source is δRADS = RADS · AADS · δAch.

To estimate the systematic uncertainty on AADS andRADS due to the peaking background, we use the statis-tical uncertainty on this quantity, ±0.3 events. Withapproximately 12 B− → [K+π−]DK∗− events and173 B− → [K−π+]DK∗− events this source contributes±0.002 and ±0.024 to the systematic uncertainties onRADS and AADS , respectively. Similarly, the 1.1 eventsuncertainty on the same-final-state background leads tosystematic uncertainties of ±0.0061 and ±0.091 on RADS

and AADS respectively.As in Section III, we need to estimate the systematic

effect due to the non-resonant K0Sπ− pairs in the K∗

mass range. We follow the same procedure discussed inSection III. After adding in quadrature the individualsystematic uncertainty contributions, listed in Table IV,we find:

AADS = −0.34± 0.43(stat.) ± 0.16(syst.)

RADS = 0.066± 0.031(stat.) ± 0.010(syst.).

TABLE IV: Summary of ADS systematic uncertainties.

Source δRADS δAADS

Detection asymmetry ±0.0005 ±0.022Peaking bkg. ±0.0020 ±0.024Same-final-state bkg. ±0.0061 ±0.091Non resonant K0

s π− bkg. ±0.0073 ±0.126Total systematic uncertainty ±0.0097 ±0.159

V. COMBINED RESULTS

We use the GLW and ADS results and a frequentiststatistical approach [27] to extract information on rB

and γ. In this technique, a χ2 is calculated using thedifferences between the measured and theoretical valuesand the statistical and systematic errors of the six mea-sured quantities. The values of rD and δD are taken fromRef. [12], while we allow 0 ≤ rB ≤ 1, 0◦ ≤ γ ≤ 180◦, and0◦ ≤ δB ≤ 360◦. The minimum of the χ2 for the rB ,γ, and δB parameter space is calculated first (χ2

min). Wethen scan the range of rB and γ minimizing the χ2 (χ2

m)by varying δB. A confidence level for each value of rB andγ is calculated using ∆χ2 = χ2

m − χ2min and one degree

of freedom. We assume Gaussian measurement uncer-tainties and confirm this assumption using simulations.In Fig. 6 we show the 95% confidence level contours forrB versus γ as well as the 68% confidence level contoursfor the GLW and the combined GLW and ADS analysis(striped areas).

In order to find confidence levels for rB we use theabove procedure, minimizing χ2 with respect to γ andδB. The results of this calculation are shown in Fig. 7.The fit which uses both the ADS and GLW results hasits minimum χ2 at rB = 0.31 with a one sigma intervalof [0.24, 0.38] and a two sigma interval of [0.17, 0.43].The value rB = 0 is excluded at the 3.3 sigma level. Wefind similar results for rB using the modifications to thisfrequentist approach discussed in Ref. [28] and using theBayesian approach of Ref. [29].

Using the above procedure we also find confidence in-tervals for γ. The results of the scan in γ are shown inFig. 8. The combined GLW+ADS analysis excludes val-ues of γ in the regions [0, 7]◦, [62, 124]◦ and [175, 180]◦

at the one sigma level and [85, 99]◦ at the two sigmalevel. The use of the measurement of the strong phaseδD [12] helps to resolve the ambiguities on γ and there-fore explains the asymmetry in the confidence level plotshown in Fig. 8.

VI. SUMMARY

In summary, we present improved measurements ofyields from B− → DK∗− decays, where the neutralD-meson decays into final states of even and odd CP(GLW), and the K+π− final state (ADS). We express

14

ADS

GLW

ADS + GLW

(deg)γ0 20 40 60 80 100 120 140 160 180

Br

0.0

0.2

0.4

0.6

0.8

1.0excluded area has CL > 0.95

FIG. 6: 95% confidence level contours from a two dimen-sional scan of γ versus rB from the B−

→ DK∗− GLW andADS measurements. Also shown are the 68% confidence levelregions (striped areas) for the GLW and the fit which usesboth the GLW and ADS measurements. rD and δD are fromRef. [12].

the results as RCP , ACP , x±, RADS , and AADS . Thevalue rB = 0 is excluded at the 3.3 sigma level. Theseresults in combination with other GLW, ADS, and Dalitztype analyses improve our knowledge of rB and γ.

VII. ACKNOWLEDGMENTS

We are grateful for the extraordinary contributions ofour PEP-II colleagues in achieving the excellent luminos-ity and machine conditions that have made this work pos-sible. The success of this project also relies critically onthe expertise and dedication of the computing organiza-tions that support BABAR. The collaborating institutionswish to thank SLAC for its support and the kind hospi-tality extended to them. This work is supported by theUS Department of Energy and National Science Foun-dation, the Natural Sciences and Engineering ResearchCouncil (Canada), the Commissariat a l’Energie Atom-ique and Institut National de Physique Nucleaire et dePhysique des Particules (France), the Bundesministeriumfur Bildung und Forschung and Deutsche Forschungsge-

meinschaft (Germany), the Istituto Nazionale di FisicaNucleare (Italy), the Foundation for Fundamental Re-search on Matter (The Netherlands), the Research Coun-cil of Norway, the Ministry of Education and Science ofthe Russian Federation, Ministerio de Educacion y Cien-

Br0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1- c

onfid

ence

leve

l

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

σ1

σ2

GLW + ADSGLW onlyADS only

Br0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1- c

onfid

ence

leve

l

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

FIG. 7: Constraints on rB from the B−→ DK∗− GLW and

ADS measurements. The dashed (dotted) curve shows 1 mi-nus the confidence level to exclude the abscissa value as afunction of rB derived from the GLW (ADS) measurements.The GLW+ADS result (solid line and shaded area) is froma fit which uses both the GLW and ADS measurements aswell as rD and δD from [12]. The horizontal lines show theexclusion limits at the 1, and 2 standard deviation levels.

(deg)γ0 20 40 60 80 100 120 140 160 180

1- c

onfid

ence

leve

l

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

σ1

σ2

(deg)γ0 20 40 60 80 100 120 140 160 180

1- c

onfid

ence

leve

l

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

FIG. 8: Constraints on γ from a fit which uses both the B−→

DK∗− GLW and ADS measurements as well as rD and δD

from [12]. The horizontal lines show the exclusion limits atthe 1 and 2 standard deviation levels.

cia (Spain), and the Science and Technology FacilitiesCouncil (United Kingdom). Individuals have receivedsupport from the Marie-Curie IEF program (EuropeanUnion) and the A. P. Sloan Foundation.

[1] N. Cabibbo, Phys. Rev. Lett. 10, 531 (1963);M. Kobayashi and T. Maskawa, Prog. Theor. Phys. 49,

652 (1973).

15

[2] “CP Violation”, edited by C. Jarlskog, Advanced Serieson Directions in High Energy Physics, Volume 3, WorldScientific (1989).

[3] C. Amsler et al. (Particle Data Group), Phys. Lett. B667, 1 (2008).

[4] M. Gronau and D. London, Phys. Lett. B 253, 483(1991); M. Gronau, Phys. Rev. D, 58, 037301 (1998).

[5] M. Gronau and D. Wyler, Phys. Lett. B 265, 172 (1991).[6] D. Atwood, I. Dunietz, A. Soni, Phys. Rev. Lett. 78,

3257 (1997).[7] A. Soffer, Phys. Rev. D, 60, 054032 (1999).[8] Hereafter K∗− implies K∗(892)−. In this paper we use

only the K∗±→ K0

Sπ±, K0S → π+π− decay chain.

[9] By D we mean any linear combination of D0 and D0.[10] J. P. Silva and A. Soffer, Phys. Rev. D 61, 112001 (2000);

Y. Grossman, A. Soffer, and J. Zupan, Phys. Rev. D 72,031501 (2005).

[11] A. Giri, Y. Grossman, A. Soffer, and J. Zupan, Phys.Rev. D 68, 054018 (2003); BABAR Collaboration, B.Aubert et al., Phys. Rev. Lett. 95, 121802 (2005).

[12] E. Barberio et al., “Averages of b-hadron and c-hadronProperties at the End of 2007”, arXiv:0808.1297v3. TheBABAR convention for δD differs from the one in this ref-erence by π.

[13] M. Gronau, Phys. Lett. B 557, 198 (2003).[14] BABAR Collaboration, B. Aubert et al., Phys. Rev. D 72,

071104 (2005).[15] BABAR Collaboration, B. Aubert et al., Phys. Rev. D 72,

071103 (2005).[16] BABAR Collaboration, B. Aubert et al., Nucl. Instr.

Methods Phys. Res., Sect. A 479, 1 (2002).[17] Geant4 Collaboration, S. Agostinelli et al., Nucl. Instr.

Methods Phys. Res., Sect. A 506, 250 (2003).[18] D. Lange, Nucl. Instrum. Methods Phys. Res., Sect. A

462, 152 (2001).[19] T. Sjostrand, Comp. Phys. Commun. 82, 74 (1994).[20] BABAR Collaboration, B. Aubert et al., Phys. Rev. D 69,

032004 (2004).[21] BABAR Collaboration, B. Aubert et al., Phys. Rev. Lett.

89, 281802 (2002).[22] G.C. Fox and S. Wolfram, Phys. Rev. Lett.

41, 1501 (1978).[23] ARGUS Collaboration, H. Albrecht et al., Phys. Lett.

B 185, 218, (1987); ibid. 241, 278, (1990).[24] BABAR Collaboration, B. Aubert et al., Phys. Rev. D,

77, 111102 (2008).[25] BABAR Collaboration, B. Aubert et al., Phys. Rev. D,

72, 052008 (2005).[26] BABAR Collaboration, B. Aubert et al., Phys. Rev. D 73,

111104 (2006).[27] J. Charles et. al., Eur. Phys. J. C41,1 (2005).[28] BABAR Collaboration, B. Aubert et al., Phys. Rev. D 78,

034023 (2008).[29] M. Ciuchini et al., JHEP 507, 28 (2005).