The Organization of Online Outsourcing: Observational and FieldExperimental Studies

by

Elizabeth Lyons

A thesis submitted in conformity with the requirementsfor the degree of Doctor of Philosophy

Graduate Department of Rotman School of ManagementUniversity of Toronto

c⃝ Copyright 2014 by Elizabeth Lyons

Abstract

The Organization of Online Outsourcing: Observational and Field Experimental Studies

Elizabeth Lyons

Doctor of Philosophy

Graduate Department of Rotman School of Management

University of Toronto

2014

This thesis explores how digitized international labor markets affect hiring and the organization of production

using experimental and observational data from the world’s largest online contract labor market, oDesk.

Team-based production in knowledge work is becoming increasingly important for firm success, and with the

globalization of economic activities, teams of workers are increasingly diverse in their nationality and skill sets.

In the first chapter of this thesis I test whether these differences have meaningful implications for performance

by designing and conducting a natural field experiment to examine how national diversity affects the returns

to team work. I find that team work improves outcomes when workers are from the same country, and

worsens outcomes when workers are from different countries relative to groups independent workers. I also

find that these results are most pronounced for groups of workers with specialized skills sets. These findings

suggest that while technology is facilitating cross-border interactions with potentially large benefits such as

knowledge transfer, market growth, and access to higher paying jobs, participants in international markets

may benefit from investing in managing the costs associated with national diversity. Aside from the costs

national diversity introduces into team work, international labor markets may also introduce informational

costs for employers trying to hire from countries they know little about. Using observational data from

the same market, oDesk, the second chapter of my dissertation considers how employers from developed

countries and contract workers from developing countries overcome the information asymmetries associated

with remote work. This chapter finds that workers from less developed countries are disadvantaged relative

to those from developed countries in terms of their likelihood of being hired by employers from developed

countries, but that verifiable information benefits them relatively more. This suggests that reliable and easily

understood information can overcome some of the difficulties associated with hiring foreign workers. The

final chapter of my thesis considers some of the broader implications of the digitization of contract labor for

workers and firms by taking advantage of observational and survey data from oDesk.

ii

Dedication

For Zico who dreams my dreams for me. His love and confidence in me is my greatest inspiration.

For my Mom and Dad who gave me the freedom and encouragement to find my passion, and the supportand love to handle failure along the way.

iii

Acknowledgements

My doctoral committee chair and advisor, Ajay Agrawal has been instrumental in the completion of this

dissertation. I cannot express how thankful I am for the truly invaluable guidance he has provided me

throughout my studies. I would not be the scholar or the person that I am today had I not had the

advantage and privilege of his mentorship.

My doctoral committee members, Avi Goldfarb, Nico Lacetera, and Andras Tilcsik have been incredibly

generous with their time and ideas, and I am very grateful. I am also grateful for the support I received from

the faculty in the Strategy and Economics departments at the University of Toronto. I thank them for the

time they put in to the graduate courses and seminars, and for providing me with such valuable feedback

on my research.

I have been very fortunate in who my colleagues in the doctoral program have been. I am grateful

to Christian Catalini, Keyvan Vakili, Sandra Barbosu, and Jill Chown for their willingness to listen and

offer advice, and for all the great times we’ve shared. I am grateful to Octavio Martinez for being such a

reliable and good humored officemate and friend. I am grateful to Florenta Teodoridis and Laurina Zhang

in particular for their support over the past two years. I could not have hoped for better people to share

this experience with.

Many of the ideas presented in this dissertation were inspired by conversations with my uncle David

Malone. I thank him for sharing his experiences and thoughts with me. My sister Kathleen Lyons spent many

hours discussing my research designs and motivations with me. I am very grateful for her contributions and

support. I also thank David Wolever who provided excellent assistance in the development and evaluation

of the task given to my experiment participants.

I gratefully acknowledge financial support from the Centre for Innovation and Entrepreneurship at the

Rotman School of Management, the Martin Prosperity Institute, the Rotman School of Management doctoral

program, and the Social Sciences and Humanities Research Council of Canada grants #493140, #494787,

and #486944.

iv

Contents

Abstract ii

Dedication iii

Acknowledgements iv

Introduction 1

1 Team Production in International Labor Markets: Experimental Evidence from the

Field 3

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Research Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Conceptual Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3.1 Model Set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.3.2 Model Predictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.3.3 Model Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4.1 The Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.4.2 Contractor Pair Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

v

1.4.3 Hiring Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.4.4 Job Completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.5 Data and Estimation Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.5.1 Overview of the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.5.2 Estimation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.6.1 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.6.2 Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

1.8 Tables and Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1.9 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

1.9.1 Additional Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

1.9.2 Data Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2 Does Information Help or Hinder Job Applicants from Less Developed Countries in

Online Markets? 48

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.2 Empirical Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2.3 Data and Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.3.1 Dataset construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

2.3.2 Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.4 Empirical Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2.5.1 Main analyses: Likelihood of Being Hired for a Job . . . . . . . . . . . . . . . . . . . . 62

2.5.2 Robustness of Main Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

2.5.3 Employer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

vi

2.5.4 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2.6 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

2.7 Tables and Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.8 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

2.8.1 A Simple Theoretical Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

2.8.2 Additional Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3 Digitization and the Contract Labor Market: A Research Agenda 92

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

3.2 The economics of online contract labor markets . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.2.1 Work Process on oDesk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.2.2 Labor supply . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3.2.3 Demand for contract labor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

3.2.4 Market-making platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

3.3 Digitization and the distribution of work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.3.1 Geographic distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

3.3.2 Income distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

3.3.3 Boundaries of the firm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

3.4 Market design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

3.5 Social Welfare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.5.1 Matching made easier? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

3.5.2 Efficiency gains from production? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

3.7 Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Bibliography 125

vii

List of Tables

1.1 Variable Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

1.2 Pair Characteristics Summary Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

1.3 Pair Outcome Summary Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

1.4 Pair Characteristic Summary Statistics By Treatment . . . . . . . . . . . . . . . . . . . . . . 34

1.5 Effect of Team Work & National Diversity on Output and Productivity . . . . . . . . . . . . 35

1.6 Effect of Team Work & National Diversity on Individual Performance . . . . . . . . . . . . . 36

1.7 Effect of Team Work & National Diversity by Task Feature . . . . . . . . . . . . . . . . . . . 37

1.8 Effect of Team Work & National Diversity on Performance by Pair Skill Differences . . . . . 37

1.9 Effect of Team Work & National Diversity Coordination . . . . . . . . . . . . . . . . . . . . 38

1.10 Effect of Team Work & National Diversity on Effort . . . . . . . . . . . . . . . . . . . . . . . 38

1.11 Effect of National Diversity on Opinion of Teammate . . . . . . . . . . . . . . . . . . . . . . 39

1.12 Effect of Team Work & National Diversity on Performance, Pakistan Excluded . . . . . . . . 39

1.16 Effect of Team Work & National Diversity on Output, Ordered Logit . . . . . . . . . . . . . 42

2.1 Variable Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

2.2 Contractor Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

2.3 Job and Employer Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

2.4 LDC Status and oDesk Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

2.5 Differential Impact of Platform Specific Experience for LDC Contractors . . . . . . . . . . . . 78

viii

2.6 Differential Impact of Platform Specific Experience by Job Type . . . . . . . . . . . . . . . . 79

2.7 Differential Impact of Platform Specific Experience for LDC Contractors on Wages . . . . . . 80

2.8 Differential Impact of Platform Specific Experience by Employer Experience . . . . . . . . . . 81

2.9 Differential Impact of Platform-Specific Experience by Contract Type . . . . . . . . . . . . . 82

2.10 Descriptive Statistics Comparing Jobs Included and Dropped from Sample . . . . . . . . . . . 85

2.11 Contractor Descriptive Statistics for Sample Including Jobs with Multiple Hires . . . . . . . . 86

2.12 Contractor Descriptive Statistics across Employer Experience Levels . . . . . . . . . . . . . . 87

2.13 LDC Status and oDesk Experience Control Coefficients . . . . . . . . . . . . . . . . . . . . . 88

2.14 Differential Impact of Platform Specific Experience for LDC Contractors on Interviews & on

being Short Listed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

2.15 Differential Impact of Platform Specific Experience for LDC Contractors with Full Sample . . 89

2.16 Excluding Applicants Previously Hired by Employer & Employer Initiated applicants . . . . . 90

2.17 Robustness to Contractor oDesk Experience Measure . . . . . . . . . . . . . . . . . . . . . . . 91

ix

List of Figures

1.1 Experiment Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

1.2 Output by Treatment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

1.3 Productivity by Treatment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1.4 Javascript Job Posting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

1.5 PHP Job Posting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.1 Sample Distribution of Platform Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.2 Likelihood of being hired for a job for LDC applicants, overall and by level of platform experience 73

2.3 Estimated Likelihood of Being Hired by Platform Experience (in quintiles) and LDC Status . 74

3.1 Number of billing employers per quarter on oDesk, relative to total number of employers in

first quarter of 2009, by employer country income level . . . . . . . . . . . . . . . . . . . . . . 114

3.2 Quarterly wage bill on oDesk by employer country income level . . . . . . . . . . . . . . . . . 115

3.3 Number of working contractors per quarter on oDesk, relative to total number of contractors

in first quarter of 2009, by contractor country income level . . . . . . . . . . . . . . . . . . . . 116

3.4 Contractor quarterly earnings on oDesk by contractor country income level . . . . . . . . . . 117

3.5 Number of contractors per country on oDesk versus country population, on a log-log scale . . 118

3.6 Contractor mean hourly wage on oDesk by country, relative to that country’s estimated local

minimum wage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

3.7 Contractor mean hourly wages on oDesk, by country . . . . . . . . . . . . . . . . . . . . . . . 120

x

3.8 Average hourly wage on oDesk per quarter, by job category . . . . . . . . . . . . . . . . . . . 121

3.9 Quarterly wage bill per job category on oDesk (log scale) . . . . . . . . . . . . . . . . . . . . 122

3.10 Contractor job category concentration on oDesk by contractor country, over time . . . . . . . 123

3.11 Job category concentration on oDesk by contractor, over time . . . . . . . . . . . . . . . . . . 124

xi

Introduction

Information and communication technologies (ICT) are significantly changing the landscape of labor

markets. For instance, telecommunication is making working from home increasingly feasible (Bloom et al.,

2013), search engines expand the market for job postings and job applicants (Autor, 2001), and online labor

platforms facilitate short-term, low cost hiring. In this dissertation, I consider the impact ICT is having on

international labor markets by studying a setting in which contract workers located around the world are

being hired online and are completing their tasks remotely. In particular, in my first chapter, I investigate

whether groups of contractors in this environment are better off working separately or in teams, and how

this varies with national diversity. In my second chapter, I ask whether employers hiring foreign workers

online assess workers differently depending on whether they are from high income or lower income countries,

and whether verifiable information reduces this difference. My third chapter explores how these types of

labor markets may continue to impact firms and workers, and discusses areas for future research on these

topics.

Online contract labor markets connect employers in developed countries (DCs) with workers in less

developed countries (LDCs). For example, on oDesk, the largest online contract labor platform in the world

and the research setting I use throughout my dissertation, over 80% of contractors on oDesk live in low or

middle income countries, whereas over 90% of employers on oDesk live in high income countries (Agrawal

et al., 2013). Given this geographic distribution of labor supply and demand online, these markets offer the

possibility of reducing global income inequality by giving workers in lower income countries access to higher

paying jobs and employers access to lower cost work without many of the barriers present in traditional labor

markets. However, barriers to these exchanges may remain, and knowledge of what these barriers are and

how they can be overcome is critical for these markets to reach their full potential. All three chapters in

this dissertation consider obstacles in these markets that hinder exchanges between workers and employers

in different countries with the goal of developing a better understanding international market frictions.

As international online contract labor markets continue to grow in terms of both participation and

1

economic value, developing a better understanding of them is increasingly important. Moreover, these

platforms also provide close-to-ideal research settings for testing many questions that are important for

traditional labor markets and for digital markets more generally. Of particular relevance for the research

presented here, cross-country communication in online and traditional markets introduces both potential

costs and benefits (e.g., Lazear, 1999), and information asymmetries can result in uncertainty and inequality

in digital and offline markets (e.g., Akerlof, 1970; Cabral and Hortacsu, 2010). In addition, labor market

growth as a result of geographic expansion may introduce problems associated with market search (e.g.,

Stigler, 1962; Wilde, 1981), advantages associated with a larger supply of labor to select from (e.g., Petrongolo

and Pissarides, 2006), changes in worker incentives to invest in human capital, and changes in firm hiring

practices (Antras and Helpman, 2004; Grossman and Rossi-Hansberg, 2008).

Taken together, the chapters presented below both improve our understanding of international and

digital labor markets, and present important areas for further investigation on these topics. As labor markets

continue to span multiple countries and cultures, and as market digitization continues to expand, these topics

will become increasingly important for firms, workers, and the global distribution of income.

2

Chapter 1

Team Production in InternationalLabor Markets: ExperimentalEvidence from the Field

1.1 Introduction

The globalization of labor markets is an important phenomenon. The number of immigrants employed

in the U.S. grew from 11.7 million in 1994 to 19.3 million in 2003, and this pattern is not restricted to

the U.S. (International Labor Organization, 2012a).1 Also contributing to the international nature of labor

markets are multinational enterprises (MNEs).2 In addition to increasing labor market diversity, team-based

production in knowledge work is also becoming more important for firm success (Gardner, Gino, and Staats,

2012; Cummings and Haas, 2012).3 Given the extent of labor market globalization, hiring a multinational

labor force is becoming inevitable in many countries and industries. Combined with the growth in team-based

knowledge production, firm success will likely become increasingly dependent on the success of nationally

diverse teams of workers.

Information and communication technologies are also contributing to geographically dispersed intra-

1Brookings reports that the share of immigrants in the U.S. labor force grew from 4.9% in 1970 to 16.4% in 2010. Internationalmigration motivated by labor force opportunities is so common that, when combined, the population of these migrants is atleast equal to the population of Brazil, the fifth most populous country in the world (International Labor Organization, 2012b).

2MNE global investment increased by more than four times during the 1990s, and the size of total FDI stock as a percentageof world GDP in 2000 was 20%. Moreover, MNEs were responsible for at least 3.4% of the world’s employment by 2000 (Kim,2006).

3Cross-country partnerships in MNEs are ubiquitous (e.g., Hays, 1974; Manev and Stevenson, 2001), and the management ofcultural differences among workers from different countries remains a major challenge (e.g., Gong, 2003; Kirkman and Shapiro,2005; Makela, Andersson, and Seppala, 2012).

3

and inter-organizational collaboration (Agrawal and Goldfarb, 2008; Forman and van Zeebroeck, 2012). In

particular, the Internet is facilitating labor market globalization through international labor markets where

worker diversity may be even more pronounced than in localized labor markets. The international online

market for labor is already an economically important source of labor supply and demand, and its value is

positioned to grow substantially in the coming years (Agrawal et al., 2013). The Economist (2013) predicts

that combined, online labor markets will be worth $2 billion by 2014 and $5 billion by 2018. Although the

platforms that support these markets may reduce the cost of participation in international labor markets, they

may also impose communication costs that are not as prevalent offline. Importantly, online labor markets

replace face-to-face interactions with remote communication (Autor, 2001). Prior research suggests that

some of the coordination costs of virtual collaboration can be managed through changes in communication

technologies, tasks, team members, and the introduction of occasional face-to-face interactions (Hollingshead,

Mcgrath, and O’Connor, 1993; Farmer and Hyatt, 1994; Galegher and Kraut, 2003; Maznevski and Chudoba,

2000; Zakaria, Amelinckx, and Wilemon, 2004; Wilsona, Strausb, and McEvily, 2006), but in online labor

markets, these management tools may be more difficult to take advantage of. Given the prior research,

it is unclear whether firms would be better off avoiding assigning collaborative tasks altogether on these

platforms.4

In addition to the remote nature of work teams, employers select workers in the international online

market for labor from a pool of contractors located around the world. oDesk, the world’s largest online

market for contract labor, supplies employers with contractors from more than 120 countries, the vast ma-

jority of whom are from less developed countries (LDCs). Thus, teams made up of contractors from multiple

LDCs have the potential to be quite common in these markets. Because of the importance of communication

for success in work teams (e.g., Manz and Sims Jr., 1987; Hinds and Mortensen, 2005), national differences

in teams may be costly if they impede communication, for instance, by increasing conflict and decreasing

knowledge sharing and trust (Lazear, 1999; Stahl et al., 2009; Hjort, 2011; Hamilton, Nickerson, and Owan,

2012; Makela, Andersson, and Seppala, 2012). However, national differences also may have benefits, includ-

4More generally, it is not clear from the current literature whether teamwork in face-to-face settings is necessarily preferableto independent work (Hackman, 1998).

4

ing increases in creativity, problem-solving abilities, and innovativeness (Watson, Kumar, and Michaelsen,

1993; Guzzo and Dickson, 1996; Haas, 2010).5 Although the social sciences recognize cultural and ethnic

heterogeneity in teams as an important determinant of team performance, the direction and extent of the

effect of national diversity on team performance remains ambiguous (e.g., Ely and Thomas, 2001; Reagans,

Zuckerman, and McEvily, 2004; Hinds, Liu, and Lyon, 2011). Some of this ambiguity could be a result

of the empirical difficulties associated with studying the relationship between the organization of work and

performance-based outcomes.

This chapter aims to address the uncertainty about how virtual workers should be organized and the

causal impact of national diversity on team outcomes by considering how the returns to organizing remote

workers as teams are causally affected by national diversity. Accordingly, I designed and conducted a natural

field experiment on the online market for contract labor, oDesk, to test the role of national differences, holding

language constant,7 on work team success. Thus, I am able to identify the causal impact of organizing remote

workers as a team by comparing teams to sets of independent workers and therefore the differential impact

of national diversity on the value of virtual teamwork.

The experiment I ran, described in detail in Section 1.4, is a two-by-two design in which I randomly

assigned contractors hired to complete a web programming task that required both a Javascript and a PHP

programmer into groups of two. Groups were permitted to either work as a team or not and were either

from different countries or not. To reduce the potential for moral hazard problems within groups, I assigned

each group member to a specific portion of the task.8 Through random assignment of field participants

who were unaware of being studied and subject to conditions consistent with those that surround workers in

international online labor markets, this study addresses the difficulties associated with identifying the causal

impact of organizational forms without sacrificing a natural work setting (Croson, Anand, and Agarwal,

5Research in social sciences also has considered other dimensions of team diversity including experience and tenure (e.g.,Hambrick, Cho, and Chen, 1996; Huckman and Staats, 2009), gender (e.g. OReilly, Williams, and Barsade, 1997; Hoogendoorn,Oosterbeek, and van Praag, 2013) and expertise (e.g., Canella, Park, and Lee, 2008)6 Similar to the findings on cultural diversity,this research has found mixed effects of diversity.

7Given that language diversity may impact teamwork through different mechanisms than diversity due to cross-countrydifferences in social and other work-related values (e.g. Hofstede, 1983; Landes, 1999), I ensure a minimum level of Englishlanguage ability among participants in an attempt to reduce the effect of language diversity on my results.

8With this task assignment design, it is relatively easy to identify if a given group member cheats.

5

2007; Boudreau and Lakhani, 2011).

Section 1.3 provides a conceptual framework for thinking about national diversity. This framework de-

velops several propositions that predict what the effect of teamwork will be as a function of communication

difficulties and gains from complementarities. Specifically, the framework predicts that, without any differ-

ential benefits to complementarities across types of teams, (1) total group effort and output will be higher

with teams than independent work when communication requires a small proportion of effort and vice versa

when communication requires a large proportion of effort, and (2) the costs of even very difficult communi-

cation can be offset by sufficiently high benefits from complementarities. If communication is more difficult

in nationally diverse teams than in nationally homogeneous teams then, without differential benefits to com-

plementarities, the framework predicts that effort and performance will be higher in nationally homogeneous

teams than in nationally diverse teams and that effort and performance are more likely to be higher with

teamwork than with independent work when groups are nationally homogeneous. With differential benefits

to complementarities between worker effort, the difference between nationally homogeneous and nationally

diverse pairs becomes ambiguous.

The findings from the empirical analysis, reported in Section 1.6, show that, consistent with my concep-

tual framework, organizing workers into teams improves outcomes for contractors in nationally homogeneous

teams but worsens outcomes for contractors in nationally diverse teams. Specifically, relative to the sample

means, teamwork leads to a 30% increase in output and a more than 50% increase in productivity for na-

tionally homogeneous teams and a 33% decrease in output and a 75% decrease in productivity in nationally

diverse teams. Consistent with the idea that team members are more likely to rely on collaboration when

team members have different skill sets, teamwork has a more positive impact on outcomes for same-country

teams when team members have skill differences, but the opposite is true for cross-country teams. These

results are consistent with teamwork being costlier for diverse teams than homogeneous teams and also

suggest that overlapping skills may compensate for a lack of shared nationality. Further investigation of the

data suggests that, also consistent with my conceptual framework, nationally homogeneous teams are able

to effectively communicate whereas nationally diverse teams are not, and that teamwork benefits nationally

6

homogeneous pairs by increasing the returns to effort. In contrast, teamwork in diverse teams appears to

reduce the returns to effort.

By expanding our knowledge of multinational interactions in the labor force, this paper has implications

for research on diversity in the labor force as well as digital collaboration. This research also has practical

implications both for employers and workers participating in online labor markets as well as potentially for

those participating in more traditional labor markets, particularly as local labor markets become increasingly

diverse and as employees increasingly have the tools to work from home.9

1.2 Research Setting

I generate data for this analysis through a natural field experiment conducted using oDesk, an international

online market for contract labor, and the largest contract labor platform in the world in terms of earnings and

the fastest-growing.10 By offering more flexible conditions and competition than traditional labor markets as

well as the potential for more efficient matching and investments in human capital (Autor, 2001; Grossman

and Helpman, 2002; Horton, 2010, 2011), online labor markets such as oDesk may be able to absorb a

meaningful portion of jobs that are now completed offline and locally.11

oDesk operates as a mediator and matches employers with contract laborers who work on jobs remotely.

To hire on oDesk, employers create an account on the site and post jobs for which they are looking to hire.

Job postings include a brief description of the job, the type of contract being offered (fixed price or hourly

wage payment), and information about the employer, including the employer’s location and feedback from

prior hires on the site. Contractors interested in being hired for jobs can apply to them by submitting a bid

to indicate the amount they are willing to accept to work on the job and a cover letter describing why they

9In addition to being relevant for labor markets, the use and study of south-south partnerships is becoming common inNGO efforts. To the extent that development barriers are similar across LDCs, this is an important area of research. Forexample, the International Development Research Centre (IDRC) recently released a study outlining the factors that contributeto successful south-south partnerships in the field of genomics and health biotechnology (International Development ResearchCenter, 2009). Similarly, the United Nations Development Programme (UNDP) has initiatives to encourage China to engagein more south-south cooperation (United Nations Development Programme, 2011).

10The total amount spent on the site in the six years after the company was founded in 2004 was less than $25 million, inthe two-and-a-half years since, it has grown to $100 million.

11In addition to growing economically, oDesk has recently become an important setting for research. Papers that use datafrom oDesk include Agrawal, Lacetera, and Lyons (2013); Ghani, Kerr, and Stanton (2012); Pallais (2012); Stanton and Thomas(2012).

7

are suited for the job. Contractors also advertise their skills and abilities to employers through their profile

pages. Contractors set up their profile pages when they join the site. These pages can include information

on their education, prior work histories,12 and scores from tests administered by oDesk. Profile pages also

provide information on where contractors live.

A key feature of oDesk for the purposes of this research is the team room application. This service

allows employers to put hired contractors in teams so that they can work together online. In particular,

contractors working in the same team room can monitor what the others are doing and chat with each other

through instant messaging. In addition, the team room facilitates contractor monitoring through frequent

screen shots, memos, work diaries, and activity meters. Through the use of these applications, employers

are using work teams more and more frequently in these markets. During the past eight years on oDesk,

teams of two to three contractors have grown from 10 per month to more than 5,000 per month, and in

mid-2012, there were almost 20 teams of at least 64 on the site (Ipeirotis, 2012). Work teams can enhance

productivity in these markets, for instance by allowing workers to further specialize and share ideas (e.g.,

Banker et al., 1996; Hamilton, Nickerson, and Owan, 2003; Jones, 2011). As employers post more jobs and

more knowledge-intensive jobs online, the prevalence of work teams in the international online market for

contract labor will likely continue to grow.

Another important feature for this research is the option to interview. oDesk encourages employers to

interview contractors they are interested in hiring before making any offers. These interviews take place

through messages exchanged on the site.

1.3 Conceptual Framework

The intention of this framework is to give some formal structure to what the existing literature predicts

about the benefits and costs of teamwork and how this varies with team diversity. It is a simplified version

of some of the key findings in prior research that are relevant for the experiment reported in this paper. In

12Platform-specific work histories include a list of all jobs performed on oDesk with a brief description of each job, the hiringemployer, and feedback and a rating if the employer has chosen to provide them.

8

particular, the framework allows for performance benefits to teamwork and for differential communication

costs and benefits to complementarities for different types of teams. The production function reflects the

type of task developed for the experiment presented here but is intended to be general enough to be useful

in other settings as well.

1.3.1 Model Set-up

Production

A team is composed of two workers, i = 1, 2. Teams can be made up of two workers from the same country

or two workers from different countries. Contractors are assigned to complete a task within a finite amount

of time and can choose their own effort levels, ei. Production is a function of both workers’ efforts and

allows for complementarities between teammate efforts to reflect the idea that, for instance, they can share

knowledge. The production function is y = λe1 + λe2 + δ(λe1λe2), where λϵ[0, 1] is the fraction of effort

that goes towards production, 1 − λ is the fraction of effort that goes towards communication,13 and δ is

equal to 1 if a pair of workers works as a team and 0 if they work independently. When no effort goes into

communication or when workers work independently such that no communication occurs, λ is equal to 1.

Given the prior evidence that communication can be more difficult when team members are diverse, (Guzzo

and Dickson, 1996; Lazear, 1999; Stahl et al., 2009; Hegde and Tumlinson, 2011; Bengtsson and Hsu, 2013),

λ may be smaller for nationally diverse teams.

Contractor Utility

Each worker incurs a private cost of effort equal to e2i . The convexity of cost reflects the increasing disutility

of effort. Contractors receive a payment p for their work, where p is equal to a fraction π of total production.

This payment can also be interpreted as a reputational boost to reflect the incentives that contractors face

in the experiment.14 For simplicity, π equals to 1. This assumption should not change the predictions of

13Among other things, communication includes coordination.14On oDesk and online contract labor markets more generally, contractor reputations are important for success in the market

(Horton, Rand, and Zeckhauser, 2011; Pallais, 2012).

9

the model as long as utility is a monotonic function. Therefore, worker i’s utility is Ui = π(λei + λej +

δλeiλej)− e2i = λei + λej + δλeiλej − e2i .

1.3.2 Model Predictions

Independent Work

If working independently, worker i chooses effort to maximize Ui = ei + ej − e2i . Therefore, given the first

order condition 1− 2ei = 0, independent worker optimal effort is eIWi = 12 .

Team Work

If working in teams, worker i chooses effort that solves the first order condition λ+λ2ej−2ei = 0. Therefore,

ei =λ+λ2ej

2 , ej =λ+λ2ei

2 , and optimal effort for workers in teams is eTW = λ2−λ2 . Therefore:

Proposition 1 The effort of workers in teams is increasing in λ.

Proof.The derivative of eTW with respect to λ is 2+λ2−2λ(2−λ2)2 , which is positive.

The value of λ required for effort to be higher for workers in teams than independent workers is the λ that

solves λ2 + 2λ− 2 = 0, which is approximately 0.73. Therefore:

Proposition 2 For λ > 0.73, effort is higher with teamwork than with independent work. For λ < 0.73,

effort is lower with teamwork than with independent work. 15

Production is increasing in effort, thus, Propositions 1 and 2 predict that output will be higher on average

for teams made up of workers who do not need to invest a significant amount of effort into communication

than those made up of workers who do. In addition, output will be higher when work is done in teams than

when it is done independently if workers do not have to invest a large amount of effort into communication.16

15One question with this finding is: why would workers not simply ignore each other if communication is more difficult? Thedecision not to allow ignoring in this model reflects the nature of the team rooms used in this experiment. The set-up is suchthat workers in teams cannot work in separate “rooms”, which means that they are always able to see what their teammatesare doing even if both teammates have agreed not to verbally communicate with each other. More generally, when teams areworking in shared space, be it virtual or physical space, ignoring teammates altogether can be difficult and potentially costly ifit leads to negative sentiments within the team.

16For any λ > 0.69, output will be higher with teamwork than without.

10

However, if the the share of effort that goes towards communication is large enough, output will be lower

with teamwork than with independent work.

So far, by modeling output as a function of the amount of effort spent on production which is decreasing

in the amount of effort spent on communication, this framework considers the costs that prior literature

has found to be associated with national diversity in teams. Prior literature also has uncovered benefits to

national diversity in teams. The next subsection extends the model to take into account the possibility of

these benefits.

1.3.3 Model Extension

Now, suppose it is possible to gain more or less from teamwork so that production is now y = λe1 + λe2 +

δ(λe1λe2), where δ ≥ 1 with teamwork. With this specification, it is possible to allow for δ to be larger

for nationally diverse teams. This reflects the idea that, consistent with existing research, the gains from

complementarities may be larger for diverse teams, for instance because they have access to different types

of knowledge (Guzzo and Dickson, 1996; Lazear, 1999; Stahl et al., 2009; Watson, Kumar, and Michaelsen,

1993). In this case, worker i chooses effort to solve the first order condition λ+ δλ2ej − 2ei = 0. Therefore,

optimal effort with teamwork is eTW,ext = λ2−δλ2 . It is now possible that complementarities in teams

that require a large investment in communication can offset reductions in production in teams relative to

independent work.

Proposition 3 Communication that requires a high share of effort can be offset by high complementarities.

Proof.Suppose that λ = 12 , so that with δ = 1, teamwork would be less productive than independent

work. Then, a pair of workers with δ > 4 will be more productive with teamwork than without (eIW = 12 ,

eTW,ext = 12 when λ = 1

2 and δ = 4).

11

1.3.4 Discussion

This model predicts that teamwork is most beneficial when the majority of worker effort can be invested in

production rather than communication and when there are strong complementarities between teammates’

effort. Given prior findings that diversity in teams can increase costs of communication, it is likely that

communication requires more effort in nationally diverse teams than in nationally homogeneous teams. If

this is the case, this leads to the prediction that teamwork is more likely to be beneficial when teammates are

from the same country and that independent work is more likely to lead to higher output when teammates

are from different countries. Finally, given prior findings that diverse teams are better at problem solving

than homogeneous teams, it is possible that nationally diverse teams benefit more from complementarities

between teammate efforts. This leads to the prediction that for high enough complementarities, teamwork

can lead to higher output than independent work even when teammates are from different countries and face

difficult communication. In terms of optimal organizational design, these predictions suggest that teamwork

is preferred to independent work when workers are from the same country and communication is quite

easy and that independent work is preferred to teamwork when workers are from different countries and

communication is quite hard, unless the knowledge complementarities between workers can compensate for

the communication difficulties.17

1.4 Experimental Design

Identifying the causal impact of national diversity on the value of teamwork is problematic because teams

are selected and not randomly assigned. This endogenous team formation makes it difficult to disentangle

both the value of teamwork as compared to independent work and the differential impact of teamwork for

nationally diverse teams. For example, it is very likely that employers and workers choose which jobs to

complete as a team and that they choose who should work together in teams according to their performance

expectations. To address these concerns, I test the effects of national diversity on work team success by

17The predictions developed in this paper are largely consistent with those developed in Lazear (1999).

12

conducting a field experiment on oDesk.

To perform the study, I set up an employer account on the oDesk website and posted jobs to attract

contractor bids. I hired contractors who met the requirements for participation in the experiment and

randomly put them into groups of two to complete a programming task. I define teams in this study as

pairs of contractors working towards a common goal. Below I describe the task, team compositions, and the

hiring and work process used for the experiment.

1.4.1 The Task

To test my research question, I needed contractors to be assigned a task for which collaboration is not

unnatural. In addition, because I am specifically interested in testing the coordination and communication

costs associated with national diversity in teams, free-riding on the assigned task should have been difficult.

The task also needed to be consistent with the types of tasks posted on oDesk so that contractors did not

become suspicious about the purpose of the job.18 Moreover, an objective evaluation of the work contractors

did on the task was necessary. Finally, it must have been possible to complete the task remotely. Given

these constraints, I developed a a web development task that required both back-end coding (PHP) and

front-end coding (Javascript) to assign contractors hired for the experiment. I then asked hired contractors

to add a list of features in these languages to existing code. This is similar to a web development task for

which a group of workers are responsible for both the design and functionality of a web page.

To reinforce the collaboration aspect of this job and to reduce moral hazard in teams (Holmstrom,

1982), I hired one contractor in each pairing to complete the Javascript portion of the task and another to

complete the PHP portion. I assigned each pairing to add three features to the code, one that required only

Javascript programming, one that required only PHP programming, and one that required both. All hired

contractors received the same list of features to be added, and I instructed them to work on the features that

corresponded to the language for which they were hired. I display the instructions provided to contractors

18This was to minimize bias caused by contractors performing differently than they would in real labor market situations.For instance, if contractors believed they were being observed for the purposes of a research study, they may have tried harderthan they normally would have to complete the task or to behave in socially desirable ways (e.g., Harrison and List, 2004;Klebe Trevino, 1992; Levitt and List, 2007).

13

in 1.9.2. In order for the contractors in the teamwork condition to be able to work with each other, I gave

them a total of eight hours to complete the task; they had to work these hours within the same period as

the other contractors in their groups.19

1.4.2 Contractor Pair Composition

I randomly assigned each hired contractor to one of four types of groups of two, and all groups were made

up of one contractor hired to work on the Javascript code and one to work on the PHP code. As Figure 1.1

demonstrates, the experiment had two treatments: pairs were either from different countries or not and were

either permitted to work in teams or not. Pairs in the team treatment worked in the same team room as

each other, and those in the independent work treatment worked in separate team rooms and were unaware

of who their pair mate was so that team work was not possible. The purpose of this experimental design is

to compare the benefits of teamwork in homogeneous teams with the benefits of teamwork in cross-country

teams in order to identify the effect of multinational teamwork on performance.

To control for country-specific differences in contractor quality, I limited the number of countries from

which I hired. Based on information about the set of applicants my job postings attracted during the

pilot phase of my study, I included contractors from India, Pakistan, and Bangladesh in the experiment.

Contractors from these countries frequently applied to my job postings and bid an amount that was within

the hourly wage criteria. While these three countries are similar in many respects - until 1947, all three were

part of India - there are also important differences between them that may result in, for example, different

knowledge sets and communication styles.20

19Contractors had between midnight UTC one day until midnight EST the next. In the first round of this experiment’s pilot,I required contractors to work all their hours within the same eight hour period as the other contractor in their pairing toensure teams could work together in the teamwork treatment. Because of other commitments, this was very difficult for manycontractors, and many of them chose to end their contracts before working on the task. In addition, contractors did not seemto think it was a legitimate requirement, which may have resulted in some of them becoming suspicious of the purpose of thetask. In contrast, giving contractors about a day to finish the job is consistent with many of the jobs on oDesk and still gavecontractors the chance to work with their teammate if they wanted to. oDesk contracts automatically begin at midnight UTCon the start day specified in the contract, and because contractors were aware that my time zone was EST (this information wasposted on my employer profile), information gathered in the second round of the pilot suggested that they seemed to expect thedeadline to be midnight EST rather than midnight UTC. To minimize confusion, I decided to keep the deadline at midnightEST.

20The education system in each country is overseen by their respective governments, and the curricula in the countriesdiffer (National Curriculum & Textbook Board, Bangladesh, 2013; Ministry of Human Resource Development Government ofIndia, 2013; Islamabad, 2013), and Hofstede (2013)’s measures of labor market culture suggests noticeable differences across the

14

1.4.3 Hiring Process

Using an employer account on oDesk, I attracted contractor bids by posting two job types. One type of job

posting asked for bids from contractors able to code in PHP and the other for bids from contractors able

to code in Javascript. To minimize attrition among hired contractors, the job postings described the type

of work the task required, the date the work needed to be done on, and the number of hours contractors

had to complete the task. The job postings also specified that an hourly wage contract was being offered21

and the maximum bid that would be accepted (US$4.00).22 To avoid bias due to contractor selection into

job applications, the job postings did not mention that the work would be completed in teams nor did they

mention anything about country-specific requirements. I provide screenshots of the Javascript and PHP job

postings used for the experiment and further details about the postings in the data appendix of this paper.

The first applicant to each job posting who bid at most $4.00 and who was from one of the countries

included in my experiment received a list of interview questions. The purpose of the questions was to get

an idea of the level of Javascript and PHP knowledge each contractor had, their English language abilities,

and the countries they have lived in. The interview text sent to Javascript and PHP job interviewees is in

the appendix of this paper. I gave each interviewee two hours to reply to the interview questions before I

interviewed the next applicant who met the bid and country requirements was interviewed. I hired the first

interviewee to reply with the exception of interviewees who did not provide pertinent answers due to their

lack of English language ability.23

countries, particular in terms of their uncertainty avoidance, individualism, and power distance measures. Specifically, for thepower distance cultural dimension, Bangladesh has a score of 80, Pakistan 55, and India 77. In the individualism dimension,Bangladesh has a score of 20, Pakistan 14, and India 48. In uncertainty avoidance, Bangladesh scores 60, Pakistan 70, andIndia 40.

21This is to ensure contractors perform their work while logged into the team room. Hourly wage contracts on oDesk guaranteepayment to contractors for all hours worked as long as the hours are performed in the team room. In contrast, fixed pricecontracts do not guarantee payment so there is less incentive to perform work while logged into the team room.

22oDesk charges 10% of each transaction on the site so a $4.00 bid from a contractor’s perspective shows up as a $4.44 bidfrom an employer’s perspective (on a $4.44 bid, $4.00/hour would go to the contractor and $0.44 would go to oDesk).

23For instance, some interviewees replied with text unrelated to the questions and others provided answers to the questionsthat were not comprehensible.

15

1.4.4 Job Completion

Once hired, contractors were sent the appropriate instructions and the file with the code to be edited. There

were four versions of the job instructions to reflect whether the contractor was hired for the Javascript or

the PHP portion of the job and whether pairs were permitted to work as a team or not.24

In addition to describing the features to be added to the code, the instructions noted the timeline for

the task and the country that contractors’ co-worker lived in (i.e. the country of residence of the other

contractor in the pairing).25 The instructions asked contractors to note what they were working on while in

the team room and to include information on what they had completed when they turned in their work.

I asked contractors who had not turned in their work by the deadline to submit any work they completed.

Once contractors had turned in their work or after the deadline had passed, I asked them whether they would

be willing to answer a few questions about the job for a $0.50 bonus. The purpose of the questions was

to obtain a measure of how well contractors think they did on the job and their perceptions on teamwork.

Once contractors had completed the survey or indicated that they did not want to complete the survey, their

contracts ended. I provided feedback based on contractors’ output to all paid contractors.

After contracts ended and contractors had the opportunity to provide employer feedback, I sent them

a debriefing message to inform them that the job they had been hired for was for research purposes and to

describe the goal of the research.26

24I provide the four different versions of the instructions in 1.9.2.25I included contractors’ pair-mate’s country of residence in the task instructions because it is not possible to restrict access

to this information in the team work treatment, so I controlled for this knowledge across treatment groups to ensure that itwas not driving my findings.

26This message also explained that their identities would be protected and that if they did not feel comfortable being includedin the analysis, they would be dropped as an observation. None of the contractors in my sample asked to be removed from thestudy.

16

1.5 Data and Estimation Methodology

1.5.1 Overview of the Data

I collected data between January 2013 and June 2013. In total, I hired 324 contractors, who made up 162

pairs.27 There are 80 contractors and 40 pairs in the nationally homogeneous teamwork and independent work

treatment groups,28 and 82 contractors and 41 pairs in the nationally diverse teamwork and independent

work treatment groups.29 I based the number of observations on a sample size analysis done with data

collected during the pilot phase of this study, which predicted that I would need 40 observations per group

to obtain statistically significant differences between the groups at the 5% level with a power of 80%.

For each hired contractor, I collected information provided on their profile pages including their work

histories on and off oDesk, education, oDesk test scores, and advertised wages. I also recorded the number

of examples of Javascript and PHP work they provided in their interviews as well as the amounts they bid

on my jobs. For each job posting, I recorded the number of applicants and the amount of money I had spent

as an employer on the site up to that point.

After contractors completed their work or once the deadline had passed, I recorded information on the

number of features they added to the code, both with and without error, the amount of time they spent

working on the task, the difference between the number of features they reported having added and the

number they actually had added, which features they attempted to add, and the total amount they were

paid for the hours worked. I also recorded the total number of non-overlapping errors added by each team;

so, for instance, if both teammates added the same feature, this was only counted as once for the team level

measure of added features. In addition to these performance outcomes, I collected information on contractor

effort. Specifically, I recorded whether or not contractors attempted to implement each of the three features

27In 12 of the pairs, at least one of the contractors did not contact me again after receiving the instructions. I have codedthese contractors as not having completed any of the work. However, given that there might be reasons for disappearing otherthan their ability to do the work, I check whether the results are robust to excluding these pairs, and to assigning them thesample average performance (see 1.9.1).

28As discussed in Section 1.4, contractors assigned to pairs in the independent work treatment work in separate team roomsfrom each other. These contractors were not aware of who their pair-mates were, and were not able to communicate with them.

29The difference in the number of observations across groups is a result of some groups taking longer to meet the observationsrequirement than others. I did not want to restrict groups to fit into particular treatment groups.

17

in the task. Table 1.1 describes these outcome variables, the key independent variables and the contractor

and job descriptive variables.

I present summary statistics of hired contractor-pair characteristics in Table 1.2. I measure each variable

as the average between the two contractors in each pair. It is worth noting that hired contractors were, on

average, relatively inexperienced on oDesk given that about 60% of them had never received a rating on

the platform prior to being hired (though many of those without a rating had been hired once or twice

on the platform),30 that the average bid on the job was lower than the average wage contractors indicated

they would work for on their profiles, that more than 85% of hired contractors were male, and that the

average hired contractor had a Bachelor’s degree. I present summary statistics of pair outcomes in Table

1.3. Here it is worth noting that the average pair did not work the full 16 hours given (eight hours per

contractor). This could be because contractors are incentivized by both the financial returns to the job

and the potential reputational returns. In particular, contractors on oDesk receive feedback and a ratings

score once they have completed a job, and they may believe that working fewer hours will result in better

feedback. Alternatively, it could be because contractors on oDesk can only work on one hourly wage job

at a time, and they may have decided that their time was more valuable spent on another task. It is also

worth noting that the majority of pairs attempted to implement the Javascipt and PHP features but that

only 33% of pairs attempted the combined feature. This is consistent with this feature being more complex

because it requires both programming languages to be implemented.

To verify the randomness of the treatments across pairs, Table 1.4 presents hired contractor-pair char-

acteristics across the teamwork treatment (in Panel A) and across the national diversity treatment (in Panel

B). Comparisons across the four groups look similar, and I report them in Table 1.9.1. Given that I randomly

assigned pairs to treatment groups, characteristics should be statistically the same across the groups and,

with one exception, that is the case here. The exception is the number of oDesk tests taken across the

national diversity treatment. This measure is statistically different at the 10% level between the two groups.

30It is not very surprising that the contractors who participated in the experiment were inexperienced on the platform giventhe relatively low wage I offered for a job that requires a non-negligible amount of skill. Contractors with more experience onthe site are likely able to capture a higher wage for this type of task (Agrawal, Lacetera, and Lyons, 2013).

18

This appears to be because of an outlier in the nationally diverse group; the medians between the two groups

are identical.

Figures 1.2 and 1.3 compare pair outcomes across treatment groups. Figure 1.2 compares the number

of features implemented by pairs that worked independently and those that worked in teams and, within the

teamwork treatment, nationally diverse teams and non-nationally diverse teams. These averages show that

teamwork significantly improves outcomes when team members are from the same country and significantly

reduces output when team members are from different countries. These effects are economically significant

as well. In particular, teamwork improves output for nationally homogeneous pairs by 38% (p-value 0.04)

relative to nationally homogeneous independent contractor pairs and reduces output for nationally diverse

pairs by 30% (p-value 0.03) relative to nationally diverse independent contractor pairs.

Figure 1.3 compares the number of features implemented divided by the total number of hours worked

by pairs and shows the same pattern, that teamwork increases productivity when teammates are from the

same country and decreases it when they are from different countries. Again, these effects are economically

significant. Teamwork increases the productivity of nationally homogeneous pairs by almost 90% (p-value

0.04) relative to nationally homogeneous independent contractor pairs and decreases the productivity of

nationally diverse pairs by more than 50% (p-value 0.01) relative to nationally diverse independent contractor

pairs.

These comparisons are consistent with the propositions made in Section 1.3. Specifically, they suggest

that when a team is nationally homogeneous, communication is easy enough that teamwork outperforms

independent work and that communication when teammates are from different countries is sufficiently dif-

ficult that independent work leads to higher output and productivity. Moreover, the findings suggest that

complementarities are not high enough to compensate for the costs of diversity in this setting.

I collected the data from a randomized experiment; therefore the simple correlations reported here

can be interpreted as causal relationships. However, to further allay concerns about omitted variables and

to potentially increase precision, I estimate in the next section the effect of cross-country teamwork on

performance in a multivariate regression framework.

19

1.5.2 Estimation Strategy

The regression estimates derive from versions of the following linear model:

Yi = α+ β1TeamWorki + β2NationallyDiversei ∗ TeamWorki+

θXi + δCountryPairi + ψWeeki + ϵi (1.1)

where Yi is a measure of pair i’s success, measured both as joint output (the total number of features

implemented by the pair) and joint productivity (the number of features added divided by the total number

of hours worked). NationallyDiversei is an indicator for whether or not pair i is made up of contractors

from two different countries, TeamWorki in an indicator for whether or not pair i is working as a team, Xi

is a vector of controls for the average characteristics between contractors in pair i, CountryPairi is a fixed

effect for pair i’s pair of countries,31 andWeeki is a vector of binary indicators for the week during which pair

i worked. In this specification, β1 is the effect of teamwork on the performance of nationally homogeneous

pairs of contractors, and β1 + β2 is the effect of team work on the performance of nationally diverse pairs of

contractors. With the inclusion of country-pair fixed effects, I estimate β1 and β2 by comparing the average

performance of independent contractors within a given pair of countries with the average performance of

teams made up of contractors in the same pair of countries.

1.6 Results

1.6.1 Main Results

The results from estimating Equation 1, presented in Table 1.5, show findings that are consistent with those

presented in Figures 1.2 and 1.3. Columns 1-3 estimate the effects of teamwork and national diversity on

joint output, and Columns 4-6 estimate these effects on joint productivity. Columns 1 and 4 estimate the

31I include 3 contractor countries in the experiment, so there are six country pairings.

20

effects of teamwork and national diversity on performance without controls or fixed effects, Columns 2 and

5 add country pair and week fixed effects, and Columns 3 and 6 add controls. Including country pair and

week fixed effects, and controls does not effect the estimated coefficients on teamwork. I present the full set

of estimated control coefficients in 1.9.1. For the remainder of the paper, I focus exclusively on coefficients

estimated with the full set of fixed effects and controls.

The coefficient estimates in Column 3 show that teamwork is beneficial for nationally homogeneous

teams and harmful for nationally diverse teams. Specifically, teamwork increases output by 30% relative

to the independent contractor sample mean of 1.06 for nationally homogeneous teams and decreases it by

33% for nationally diverse teams. The coefficient estimates in Column 6 suggest even larger effects for

productivity. Specifically, relative to 0.14 (the mean productivity of independent contractors) teamwork

increases productivity by about 50% for nationally homogeneous teams and decreases productivity by about

75% for nationally diverse teams. For a clearer interpretation, I estimate the effects of teamwork national

diversity on joint output using an ordered logit regression. The results of this estimation are in 1.16. The

estimates show that team work among nationally homogeneous pairs significantly decreases the likelihood of

adding no features, and significantly increase the likelihood of adding 2 and 3 features. The reverse is true

for nationally diverse pairs of workers.

I also collected data on individual contractor performance. I measure individual performance indepen-

dently of teammate performance in the sense that, even when both contractors in a pair implement the

same feature, it still counts towards each contractor’s individual performance, whereas it only counts once

towards joint performance. I estimate Equation 1 with individual output and individual productivity and

present the results in Table 1.6. The results show that teamwork significantly improves individual output

for contractors in nationally homogeneous pairs but does not significantly impact their productivity. Team

work significantly reduces both output and productivity for contractors in nationally diverse pairs. Com-

bined with the results presented in Table 1.5, these findings show that teamwork does not meaningfully affect

performance by changing task feature duplication within pairs. Specifically, the size of the coefficients in

Table 1.6 relative to the sample mean performance are comparable to the size of the coefficients in Table 1.5,

21

suggesting that teamwork does not have a large effect on whether pair members complete the same feature

as each other or not.

Table 1.7 shows the coefficients from estimating Equation 1 with an indicator for whether or not each

feature included in the task was implemented. Columns 1 and 2 present the estimated coefficients of the

effect of teamwork for nationally homogeneous and nationally diverse pairs of contractors on the likelihood

of implementing the Javascript and PHP features, respectively. These results show that teamwork had more

of an effect on the implementation of the Javascript feature than the PHP feature for which teamwork had

no significant effect. Column 3 presents the estimated coefficient of the effect of teamwork for nationally

homogeneous and nationally diverse pairs of contractors on the likelihood of implementing the feature that

requires both Javascript and PHP. As with the implementation of the Javascript feature, teamwork signif-

icantly increases the likelihood that the combined feature will be implemented when pairs of contractors

live in the same country and significantly decreases the likelihood of implementation when contractors in a

pair are from different countries. Given that the features were to be added to a PHP-based script, these

results suggest that teamwork has a larger effect on performance when two types of knowledge are required

and therefore when coordination and communication are more important. The combined feature required

both PHP and Javascript to be implemented, and, while the writing of the Javascript feature only required

Javascript knowledge, contractor pairs may have benefited from knowledge of PHP when determining where

in the script to implement the feature.

The findings in Table 1.7 suggest that teamwork has a large impact when collaboration is more impor-

tant. It follows then that teamwork will be most important when contractors in a pair specialize in one of

the two skills required for the task, e.g., when one member knew Javascript but not PHP and the other

knew PHP but not Javascript. Testing whether this is the case helps clarify whether collaboration is driving

the main results. I do this test by estimating Equation 1 separately for pairs with skill differences and those

without any. I present the results of this estimation in Table 1.8. The estimates in Table 1.8 confirm that

the costs of teamwork for nationally diverse pairs are highest when contractors have different skill sets. The

coefficients on the interaction term in Columns 1 and 2 are significantly different at the 5% level and the

22

interaction term estimates in Columns 3 and 4 significantly different at the 12% level32.

1.6.2 Mechanisms

In this subsection, I consider two possible explanations for my findings. Consistent with the predictions

of the model presented in section 1.3, I explore the data to determine whether differences in effort due to

communication costs can explain the output and productivity differences. I also test for evidence of dislike

for national diversity as a mechanism for my results. I find evidence in support of the former explanation,

and no evidence in support of the latter explanation.

Communication

One possibility for differences in performance across the treatment groups is that the costs of communication

differ for nationally homogeneous and nationally diverse teams. I test this in two ways. First, I estimate

Equation 1 with the difference between what contractor pairs reported as having implemented and what

was actually implemented as the outcome variable. This measure is a proxy for how successfully contractors

communicated with their teammates by measuring whether contractors were aware of what their pair-mate

was able to complete. Second, I estimate Equation 1 with an indicator for whether or not contractors in

the same pair worked on implementing the same feature. This is a proxy for whether or not contractors

were able to divide up the work. I present the results of these estimations in Table 1.9. The results in

Column 1 suggest that nationally homogeneous teams were able to communicate effectively because the

error in what contractors reported as completed fell significantly relative to pairs of nationally homogeneous

contractors working independently of one another. The reverse seems to be true for nationally diverse teams.

In particular, contractors in nationally diverse teams appear less able to report what their pair-mate was able

to complete than independent pairs of contractors from different countries, though the coefficient estimate

is only significant at the 17% level. Column 2 does not suggest any significant differences across treatment

32The coefficients on the estimated effect of teamwork for nationally homogeneous pairs are higher with skill differences butnot significantly so (p-value=0.18 when output is the dependent variable, p-value=0.35 when productivity is the dependentvariable).

23

groups in whether contractors in a pair worked on implementing different features. However, there may be

benefits to working together on a feature. Taken together, these findings suggest communication is more

successful in nationally homogeneous teams than in nationally diverse teams.

Effort

I now test whether the findings could be due to changes in effort levels across the treatment groups. The

conceptual framework presented in Section 1.3 suggests that effort is less valuable when communication is

harder because a lower proportion of effort will be spent on production when more of it has to be spent on

communication. The framework also predicts that effort will be higher in teamwork than in independent

work when communication is easy enough. To test whether this prediction is consistent with contractor

behavior in the experiment, I estimate Equation 1 with indicators for whether or not a pair attempted to

implement each feature and whether a pair invested any effort at all in the task as measures of effort. I

present these estimates in Table 1.10.

The results show that effort levels did not change significantly for work on the feature that required

only Javascript knowledge or for the feature that required only PHP knowledge. However, effort levels do

appear to have differed across the treatment groups for the feature that required both PHP and Javascript

knowledge. This was the part of the task that likely required the most communication to be completed, and

therefore pairs that could communicate easily were more likely to benefit from teamwork when trying to

complete it and pairs that had a hard time communicating were less likely to benefit from teamwork when

trying to complete it. To make the latter point more concrete, when working independently, contractors

can invest all their effort into production. When working in teams, pairs with difficulty communicating may

realize that a large proportion of the effort exerted on the joint language feature (i.e., the third feature)

will go towards coordinating with their teammates rather than into production and, as a result, may not

be willing to invest any effort in trying to complete it if the gains from joint production do not outweigh

the costs. Consistent with this interpretation, effort on the joint language feature appears to increase for

nationally homogeneous pairs when they work in teams relative to when they work independently, and the

24

reverse appears to be true for nationally diverse teams. In addition to being statistically significant, the

results are economically significant. Teamwork in nationally homogeneous teams increases the likelihood

that a pair of contractors exerts effort on the third feature by more than 17% and decreases effort on the

third feature for nationally diverse teams by more than 26%.

One explanation for the drop in effort on the combined feature for nationally diverse teams is that these

pairs were less productive (as demonstrated in Table 1.5), and if pairs attempted the combined feature last,

nationally diverse teams were less likely to have time to work on this feature even if they intended to. The

coefficient estimates in Column 4 suggest that this productivity difference is not driving the fall in effort for

nationally diverse teams. The estimates suggest that nationally diverse teams of contractors are more likely

to invest no effort into any part of the task than nationally diverse independent pairs of contractors or than

nationally homogeneous teams or independent pairs. This finding holds when any of the country pairs are

dropped, so it is not being driven by a particular pair of countries.

Dislike for Team National Diversity

An alternative possible explanation for the detrimental impact of teamwork on cross-country pair success is

that contractors prefer the failure of a teammate from another country to their own success. Based on the

structure of the task and of contractor incentives on oDesk, the theoretical framework presented in Section

1.3 assumes that the incentives of the contractors in my study are aligned. However, the framework does

not take into account the possibility that contractors’ pay-offs change based on preferences for teammate

nationality unrelated to communication difficulties and complementarities. I investigate whether there is

any empirical evidence for this in the data by analyzing contractor survey responses.33 I asked contractors

in the teamwork treatment who agreed to answer the survey questions whether they thought they had done

more work than their teammates or vice versa (where 1 is “my teammate did almost all the work” and

7 is “I did almost all the work”) and whether they would be willing to work with their teammates from

this task or would prefer a different teammate for any follow-up tasks (where 1 is “I would prefer another

33The survey response rate is 83%.

25

teammate” and 7 is “I would prefer this teammate”). Contractors knew that the employer was going to

see these answers and therefore could have used them to damage their teammates’ reputations. Table 1.11

presents results from regressions of the cross-country team indicator on these survey questions.34 The results

give no indication that contractors in cross-country teams had a more negative view of their teammates or

that they were less likely to want to work with their teammates again.

These results are suggestive that contractors’ dislike for teammates based on their nationality is not

driving the results. However, the survey may not capture what contractors really thought and the response

rate is not 100%, so there may be some selection in who replied. Another way to test for this mechanism

is to restrict the sample to countries that are less likely to have negative feelings towards each other. Most

concerning in this regard are teammates from India and Pakistan, given the current dispute between these

countries.35 To test whether this dispute is driving the negative coefficient on the interaction between cross-

country teams and teamwork, I estimate Equation (1) excluding all pairs with at least one contractor from

Pakistan. I present the results from this estimation in Table 1.12. The results in Table 1.12 are consistent

with those in Table 1.5,36 and are not consistent with the negative effect of teamwork on nationally diverse

pair performance being driven by dislike for national diversity in teams.37

1.7 Conclusion

As technology continues to facilitate international markets, collaboration between market participants from

all over the world is becoming more common. As a result, multinational work teams are likely to become

increasingly necessary. Understanding the trade-offs associated with these teams has important implications

for employers and the economy more generally. This paper considers what the value of teamwork is and how

this varies with national diversity among contract workers and uses a field experiment in an international

online contract labor market to answer this question.

34I only gave these survey questions to contractors in the teamwork treatment, so the results are restricted to that sample.35Hjort (2011) shows that when team members are from conflicting ethnicities, productivity suffers.36The statistical significance of the coefficients in Table 1.12 is lower, but the sample size is also significantly lower, so that

is not surprising.37The results hold when I drop any one of the six country-pairs of contractors from the sample.

26

Findings from the experiment show that allowing teamwork improves outcomes for contractors in na-

tionally homogeneous pairs but worsens outcomes for contractors in nationally diverse teams. In particular,

nationally diverse pairs perform worse when working as teams than they do when working independently.

Consistent with the idea that team members are more likely to rely on collaboration when they have spe-

cialized skill sets, teamwork has a more negative impact on outcomes for nationally diverse pairs when team

members have specialized skills. Further investigation of the data suggests that nationally homogeneous

teams are able to effectively communicate whereas nationally diverse teams are not and that teamwork

benefits nationally homogeneous pairs by increasing the returns to effort. In contrast, teamwork in diverse

teams appears to reduce the returns to effort.

It is important to note some limitations of the analysis in this paper. First, I gave teams only eight hours

to work on their tasks, and team members had no prior interactions. Therefore, the experiment considers the

impact of national diversity on the value of teamwork during the period immediately after team members are

first introduced. For longer term labor contracts, as team members spend more time together, contractors

may learn how to deal with the communication difficulties that arise due to national differences. Second,

the experiment restricts analysis to three countries. The countries included are relatively similar compared

to many other country pairs so the results presented here may be a lower bound on the value of teamwork

for cross-country pairs. Future research is needed to fully address how country characteristics contribute

to collaboration difficulties between contractors in different countries. Third, the task assigned to workers

in this experiment has both routine and creative elements (e.g., Boudreau, Lacetera, and Lakhani, 2011),

which reflects many common real-world tasks; however, it may not require the type or extent of creativity

that prior research has shown diverse teams excel at (e.g., creative generation tasks). To the extent that

the assigned task is not a task that benefits from diversity, this study can be thought of as a test of the

costs associated with national diversity in teams.38 Finally, there is no hierarchy in the teams studied in this

analysis. In follow-up work, I plan to test whether assigning a team lead affects the role of national diversity

in teams by changing the type of communication that occurs within teams.

38However, if it is the case that diversity only benefits teams in cases of extreme task creativity, it is unlikely to benefit manyreal-world teams

27

This study contributes to the literature on diversity in teams by identifying the value of teamwork

separately for nationally diverse and nationally homogeneous pairs of contractors. Moreover, I identify the

effects of teamwork for nationally diverse and nationally homogeneous pairs of contractors on performance

when all contractors share a common language and test how skill specialization interacts with these effects.

In addition, I provide suggestive evidence that my findings are driven by changes in the returns to effort

and the costs of communication. This study also contributes to the literature on virtual teams by testing

these effects on pairs of contractors working online and remotely. Furthermore, this paper has implications

for research on immigrant labor market success. For instance, prior findings in this literature suggest that

employers are less willing to hire immigrants from LDCs than equally capable native workers (Leslie and

Lindley, 2003; Oreopoulos, 2011; Hunt, 2013). The results presented here suggest that some of this hesitancy

may be a result of the costs of assimilating different nationalities in the workplace.

The results presented in this paper also have important practical implications for managers, participants

in international markets, and policy makers. In particular, there are potentially large gains to cross-country

collaboration, including knowledge transfer, market growth, and access to higher paying jobs; however, the

findings suggest that managers hiring from an international pool of workers should be cognizant of the costs

of a diverse labor force and invest in managing these differences. This study suggests that one way for

managers to reduce the costs associated with national diversity is to ensure teammates have another form of

shared knowledge. In addition, the findings suggest that participants in international markets may benefit

from investing in culture-specific human capital. Education policy may play a role in smoothing the costs

of collaborating across countries by standardizing some of the content of curricula across countries and by

providing cultural training programs for international market participants.

28

1.8 Tables and Figures

Figure 1.1: Experiment Design

Notes: This matrix reports the four experimental conditions. The performance differences between the average in (1) and (2)and the average in (3) and (4) represent the overall effect of teamwork as compared to independent work. The performancedifferences between the average in (1) and (3) and the average in (2) and (4) represent the overall effect of having subjects ofthe same vs. different nationality in each pair. The difference between the performance of pairs in (1) and (2) measures theimpact of same vs. different nationality for subject working independently and the difference between (3) and (4) for subjectsworking in teams.

29

Figure 1.2: Output by Treatment

Notes: This chart compares mean output of pairs across treatment groups.

30

Figure 1.3: Productivity by Treatment

Notes: This chart compares mean productivity of pairs across treatment groups.

31

Table 1.1: Variable Definitions

Variable DescriptionDependent Variables:Joint Output Number of non-overlapping features added to code by the PairJoint Productivity Number of non-overlapping features added to code by the Pair

divided by number of hours workedIndividual Output Number of features added to code by the ContractorIndividual Productivity Number of features added to code by the Contractor

divided by number of hours workedImplement Combined Feature Equal to one if Pair successfully adds the feature that

requires both Javascript and PHP to the code, zero otherwiseBoth Attempt Same Feature Equal to one if both contractors in the pair attempt

to add the same feature to the code, zero otherwiseDifference between Reported and Completed Absolute difference between what contractors

reported as completed and what was actually completed by the PairAttempt Javascript Feature Equal to one if at least one Contractor in the Pair

attempts to add the feature that requires Javascript to the code, zero otherwiseAttempt PHP Feature Equal to one if at least one Contractor in the Pair

attempts to add the feature that requires PHP to the code, zero otherwiseAttempt Combined Feature Equal to one if at least one Contractor in the Pair

attempts to add the feature that requires both Javascript and PHP to the code,zero otherwise

No Attempt Equal to one if both Contractors in Pair do not attempt to add any featuresto the code, zero otherwise

Reported Doing More Work than Teammate Contractor response to survey question”Did you or your teammate do more work on this project?” Response range: 1(My teammate did almost all of the work) to 7 (I did almost all of the work)

Willingness to Work with Teammate Again Contractor response to survey question”6. Would you prefer working with your teammate from this task or someone elseon another project?” Response range: 1 (I would prefer working with someone else) to 7(I would prefer working with my teammate from this task)

Independent Variables:National Diversity Equal to one if Contractors in the Pair are from different countries,

zero otherwise.Team Work Equal to one if Contractors in the Pair are permitted to work as a team,

zero otherwiseSkill Differences Equal to one if at least one Contractor in the Pair has knowledge

of a relevant skill (i.e., Javascript or PHP) that their co-worker does not have,zero otherwise.

Outcome Descriptives:Total Hours Worked Total hours worked by Pair on jobAmount Paid for Hours Worked Total amount paid to Pair for jobContractor and Job Descriptives:Number of Job Applications Number of Contractors who applied to a job postingoDesk Rating Job size weighted feedback score out of five provided by prior oDesk EmployersNo oDesk Rating Equal to one if Contractor has no oDesk rating at time of hiring,

zero otherwiseoDesk Experience Number of contracts Contractors have been hired to complete on oDesk at time

of hiringProfile Picture Equal to one if Contractor has a profile picture, zero otherwiseEducation Measurement of highest level of education. Equal to one for College Diploma, two

for Bachelor’s Degree, three for Master’s Degree, four for Doctorate, zero otherwiseNumber of Non-oDesk Jobs Reported Number of non-oDesk jobs reported on profileAverage oDesk Test Score Average score out of five on oDesk tests takenNumber of oDesk Tests Number of tests taken on oDeskWage Bid Amount Contractor bid on job postingAdvertised Wage on profile Wage Contractors post on profile pages as amount they are

willing to work forFemale Equal to one if Contractor is female, zero otherwiseEmployment Agency Member Equal to one if Contractor belongs to an oDesk employment

agency, zero otherwiseNumber of portfolio Items Number of items in Contractor’s profile page portfolio

32

Table 1.2: Pair Characteristics Summary Statistics

Pair Averages Mean (Std. Dev.)Number of Job Posting Applications 8.218 (3.18)oDesk Rating Prior to Hire 4.577 (0.66)No Rating Prior to Hire 0.611 (0.37)Number of oDesk Contracts Prior to Hire 4.213 (7.466)Indicator for having a Profile Picture 0.843 (0.27)Level of Education 1.846 (0.688)Number of Offline Jobs Listed on Profile 1.191 (0.643)Average Score on oDesk Tests 3.428 (0.422)Number of oDesk Tests Taken 2.509 (2.283)Wage Bid on the Job 3.759 (0.607)Wage Posted on Profile 6.888 (3.543)Indicator for Female Contractor 0.145 (0.253)Indicator for Agency Membership 0.253 (0.312)Number of Items in Portfolio 3.87 (4.806)

N 162

Table 1.3: Pair Outcome Summary Statistics

Pair Outcomes Mean (Std. Dev.)Team Total Amount Paid 35.51 (20.408)Team Total Number of Hours Worked 9.35 (5.123)Number of Features Implemented 1.068 (0.773)Number of Features Implemented Divided by Hours Worked 0.131 (0.164)Difference between Actual and Reported Features Added 0.327 (0.555)Attempt Javascript Feature 0.920 (0.273)Attempt PHP Feature 0.821 (0.385)Attempt Combined Feature 0.327 (0.471)No Attempt 0.056 (0.230)Teammates Worked on the Same Feature 0.173 (0.379)

N 162

33

Table 1.4: Pair Characteristic Summary Statistics By Treatment

Panel A

Variable No Team Work Team Work p-value of difference

Number of Job Posting Applications 8.358 8.08 0.580(0.328) (0.378)

oDesk Rating Prior to Hire 4.527 4.62 0.470(0.11) (0.083)

No Rating Prior to Hire 0.636 0.586 0.397(0.042) (0.040)

Number of oDesk Contracts Prior to Hire 5.093 3.333 0.134(1.036) (0.542)

Indicator for having a Profile Picture 0.815 0.87 0.191(0.033) (0.026)

Level of Education 1.87 1.821 0.649(0.077) (0.076)

Number of Offline Jobs Listed on Profile 1.16 1.222 0.543(0.075) (0.067)

Average Score on oDesk Tests 3.416 3.44 0.736(0.044) (0.058)

Number of oDesk Tests Taken 2.531 2.488 0.905(0.239) (0.269)

Wage Bid on the Job 3.729 3.789 0.527(0.070) (0.065)

Wage Posted on Profile 7.422 6.355 0.326(0.329) (0.447)

Indicator for Female Contractor 0.13 0.16 0.440(0.029) (0.028)

Indicator for Agency Membership 0.228 0.278 0.315(0.035) (0.035)

Number of Items in Portfolio 3.92 3.821 0.896(0.562) (0.508)

Panel B

Variable No National Diversity National Diversity p-value of difference

Number of Job Posting Applications 8.15 8.287 0.786(0.318) (0.386)

oDesk Rating Prior to Hire 4.543 4.607 0.635(0.112) (0.080)

No Rating Prior to Hire 0.619 0.604 0.796(0.043) (0.04)

Number of oDesk Contracts Prior to Hire 3.625 4.787 0.324(0.713) (0.927)

Indicator for having a Profile Picture 0.863 0.823 0.356(0.029) (0.031)

Level of Education 1.875 1.817 0.594(0.079) (0.074)

Number of Offline Jobs Listed on Profile 1.2 1.183 0.866(0.076) (0.067)

Average Score on oDesk Tests 3.464 3.394 0.335(0.056) (0.045)

Number of oDesk Tests Taken 2.175 2.835 0.066*(0.207) (0.288)

Wage Bid on the Job 3.788 3.731 0.550(0.068) (0.067)

Wage Posted on Profile 6.611 7.159 0.326(0.329) (0.447)

Indicator for Female Contractor 0.131 0.159 0.495(0.029) (0.027)

Indicator for Agency Membership 0.231 0.274 0.38(0.036) (0.034)

Number of Items in Portfolio 4.269 3.482 0.299(0.588) (0.477)

Notes: Standard errors are in parentheses. * significant at 10%; ** significant at 5%; *** significant at 1%

34

Table 1.5: Effect of Team Work & National Diversity on Output and Productivity

(1) (2) (3) (4) (5) (6)VARIABLES Joint Output Joint Productivity

Team Work 0.350* 0.309* 0.300* 0.0773** 0.0681* 0.0698**(0.182) (0.175) (0.165) (0.0373) (0.0363) (0.0331)

National Diversity*Team Work -0.670*** -0.634** -0.630*** -0.171*** -0.176*** -0.168***(0.237) (0.250) (0.238) (0.0507) (0.0533) (0.0505)

National Diversity 0.125 0.0825**(0.177) (0.0358)

Constant 1*** 1.230*** 0.489 0.0939*** 0.131*** 0.176*(0.135) (0.224) (0.535) (0.0153) (0.0266) (0.0921)

Country Pair & Week Fixed Effects No Yes Yes No Yes YesControls No No Yes No No Yes

Observations 162 162 162 162 162 162Mean Dependant Variable,Independent Work 1.062 0.135R-squared 0.066 0.191 0.255 0.069 0.225 0.274

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Controls are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. Joint Output is the total number of features added by anobservation. Joint productivity is the total number of features added by an observation divided by the number of hours worked on the task by the pair.

* significant at 10%; ** significant at 5%; *** significant at 1%

35

Table 1.6: Effect of Team Work & National Diversity on Individual Performance

(1) (2)VARIABLES Individual Output Individual Productivity

National Diversity 0.046 0.025(0.102) (0.027)

Team Work 0.200* 0.036(0.107) (0.030)

National Diversity*Team Work -0.390*** -0.104***(0.143) (0.038)

Constant 0.303 0.084*(0.235) (0.049)

Observations 324 324Mean Dependant Variable,No National Diversity 0.675 0.121R-squared 0.176 0.122

Notes: An observation is an individual worker. Robust standard errors in parentheses. Country pair and week fixed effects included inall regressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. Individual Output is thetotal number of features added by an individual. Individual productivity is the total number of features added by an observationdivided by the number of hours worked on the task by the individual.

* significant at 10%; ** significant at 5%; *** significant at 1%

36

Table 1.7: Effect of Team Work & National Diversity by Task Feature

(1) (2) (3)VARIABLES Javascript PHP Combined

Feature Feature Feature

Team Work 0.263** -0.138 0.164***(0.109) (0.105) (0.059)

National Diversity*Team Work -0.325** -0.035 -0.277***(0.161) (0.150) (0.090)

Constant 0.444 -0.006 -0.096(0.402) (0.370) (0.209)

Observations 162 162 162Mean Dependent Variable,Independent Work 0.568 0.444 0.074R-squared 0.219 0.309 0.243

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects included.Controls included in regression are team averages for member education, platform experience, non-platform work experience, numberof platform tests, presence of a profile page, gender, wage bid, and agency membership. The dependent variable is equal to one if thefeature specified in the column headers was successfully implemented, and zero otherwise.

* significant at 10%; ** significant at 5%; *** significant at 1%

Table 1.8: Effect of Team Work & National Diversity on Performance by Pair SkillDifferences

(1) (2) (3) (4)Joint Output Joint Productivity

VARIABLES No Skill Skill Difference No Skill Skill DifferenceDifference Difference

Team Work 0.0670 0.436 0.0448 0.0923*(0.238) (0.262) (0.036) (0.053)

National Diversity*Team Work -0.242 -1.062*** -0.103* -0.228***(0.386) (0.328) (0.0569) (0.0848)

Constant 0.0867 1.768* 0.203* 0.156(0.785) (0.975) (0.114) (0.184)

Observations 75 87 75 87Mean Dependent Variable,Independent Work 1.083 1.044 0.132 0.137R-squared 0.534 0.437 0.566 0.310

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects included in allregressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. A pair has a skilldifference if the individuals in the pair have different knowledge of Javascript and/or PHP.

* significant at 10%; ** significant at 5%; *** significant at 1%

37

Table 1.9: Effect of Team Work & National Diversity Coordination

(1) (2)VARIABLES Difference between Reported Both Attempt Same

and Completed Feature

Team Work -0.298** -0.032(0.148) (0.103)

National Diversity*Team Work 0.262 0.063(0.190) (0.127)

Constant 0.544 0.034(0.413) (0.322)

Observations 162 162Mean Dependent Variable,Independent Work 0.395 0.160R-squared 0.172 0.186

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects included in allregressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership.

* significant at 10%; ** significant at 5%; *** significant at 1%

Table 1.10: Effect of Team Work & National Diversity on Effort

(1) (2) (3) (4)VARIABLES Javascript PHP Combined No

Feature Attempt Feature Attempt Feature Attempt Attempt

Team Work 0.037 -0.060 0.176* -0.033(0.059) (0.102) (0.099) (0.056)

National Diversity -0.071 0.065 -0.268* 0.134**Team Work (0.106) (0.147) (0.142) (0.074)Constant 0.586** 0.672 0.283 0.171

(0.255) (0.441) (0.353) (0.160)

Observations 162 162 162 162Mean Dependent Variable,Independent Work 0.914 0.778 0.222 0.037R-squared 0.148 0.121 0.272 0.186

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects included in allregressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. Independent variables aredummies that equal one if at least one pair member attempted to work on the feature specified in the column headers.

* significant at 10%; ** significant at 5%; *** significant at 1%

38

Table 1.11: Effect of National Diversity on Opinion of Teammate

(1) (2)VARIABLES Reported Doing More Willingness to Work

Work than Teammate With Teammate Again

National Diversity -0.067 0.598(0.324) (0.382)

Constant 4.851*** 4.899***(1.325) (1.569)

Observations 120 120Mean Dependent Variable,No National Diversity 5.598 4.133R-squared 0.182 0.375

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects included in allregressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. Independent variables arecollected through a survey.

* significant at 10%; ** significant at 5%; *** significant at 1%

Table 1.12: Effect of Team Work & National Diversity on Performance, Pakistan Excluded

(1) (2)VARIABLES Joint Output Joint Productivity

Team Work 0.307* 0.079**(0.178) (0.035)

National Diversity*Team Work -0.530 -0.176***(0.323) (0.061)

Constant -0.160 0.025(0.806) (0.155)

Observations 108 108Mean Dependent Variable,Independent Work 1.059 0.121R-squared 0.325 0.330

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects included in allregressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. Pairs with at least oneworker from Pakistan are excluded.

* significant at 10%; ** significant at 5%; *** significant at 1%

39

1.9 Appendix

1.9.1 Additional Tables

Table 1.13: Pair Characteristic Summary Statistics By Treatment

Variable No Team Work Team WorkNo National National p-value of No National National p-value ofDiversity Diversity difference Diversity Diversity difference

Number of Job Posting 7.9 8.805 0.170 8.400 7.768 0.407Applications (0.406) (0.509) (0.491) (0.575)oDesk Rating Prior to Hire 4.423 4.461 0.352 4.657 4.589 0.688

(0.198) (0.097) (0.110) (0.122)No Rating Prior to Hire 0.625 0.646 0.802 0.613 0.561 0.524

(0.061) (0.059) (0.061) (0.053)Number of oDesk Contracts 4.775 5.402 0.764 2.475 4.171 0.118Prior to Hire (1.284) (1.632) (0.588) (0.891)Indicator for having a 0.85 0.780 0.301 0.875 0.866 0.862Profile Picture (0.045) (0.050) (0.039) (0.866)Level of Education 1.925 1.817 0.487 1.825 1.817 0.959

(0.099) (0.118) (0.125) (0.090)Number of Offline Jobs 1.2 1.122 0.608 1.200 1.244 0.747Listed on Profile (0.129) (0.081) (0.082) (0.108)Average Score on oDesk Tests 3.497 3.339 0.073* 3.427 3.452 0.831

(0.064) (0.058) (0.095) (0.069)Number of oDesk Tests Taken 2.238 2.817 0.227 2.112 2.854 0.170

(0.296) (0.371) (0.294) (0.445)Wage Bid on the Job 3.734 3.724 0.943 3.842 3.738 0.423

(0.109) (0.091) (0.082) (0.100)Wage Posted on Profile 7.460 7.385 0.934 5.762 6.933 0.073*

(0.559) (0.698) (0.299) (0.565)Indicator for Female 0.100 0.159 0.313 0.163 0.159 0.943Contractor (0.041) (0.041) (0.042) (0.037)Indicator for Agency 0.200 0.256 0.414 0.263 0.293 0.670Membership (0.050) (0.047) (0.051) (0.049)Number of Items in Portfolio 4.950 2.915 0.070* 3.588 4.049 0.653

(1.032) (0.430) (0.558) (0.848)

Notes: Standard errors are in parentheses. * significant at 10%; ** significant at 5%; *** significant at 1%

40

Table 1.14: Effect of Team Work & National Diversity on Team Performance, Robustness toPairs with Contractors who Lost Contact

(1) (2) (3) (4)Pairs with Lost Contractors Average Performance Assigned Dropped from Sample

VARIABLES Joint Ouput Joint Producitivity Team Output Team Productivity

Team Work 0.271 0.062* 0.320* 0.070**(0.165) (0.033) (0.175) (0.034)

National Diversity*Team Work -0.576** -0.159*** -0.697*** -0.182***(0.233) (0.050) (0.259) (0.055)

Constant 0.491 0.161* 0.353 0.138(0.511) (0.092) (0.541) (0.097)

Observations 162 162 150 150R-squared 0.251 0.268 0.277 0.286

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects are included inall regressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. Joint output is the totalnumber of features added by an observation. Joint productivity is the total number of features added by an observation divided bythe number of hours worked on the task by the pair.

* significant at 10%; ** significant at 5%; *** significant at 1%

Table 1.15: Effect of Team Work & National Diversity on Team Performance, Full Set ofControls

(1) (2)VARIABLES Joint Output Joint Productivity

Team Work 0.300* 0.070**(0.165) (0.033)

National Diversity*Team Work -0.630*** -0.168***(0.238) (0.050)

Number of oDesk Contracts Prior to Hire -0.010 -0.002(0.012) (0.002)

Indicator for having a Profile Picture 0.169 -0.072(0.258) (0.051)

Level of Education -0.031 -0.002(0.098) (0.016)

Number of Offline Jobs Listed on Profile 0.020 0.022(0.125) (0.024)

Number of oDesk Tests Taken 0.001 0.012(0.034) (0.011)

Wage Bid on the Job 0.197* -0.003(0.106) (0.019)

Indicator for Agency Membership -0.545*** -0.072**(0.217) (0.035)

Indicator for Female Contractor -0.003 -0.031(0.293) (0.037)

Constant 0.489 0.176*(0.535) (0.092)

Observations 162 162R-squared 0.255 0.274

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects are included inall regressions. Joint Output is the total number of features added by an observation. Joint productivity is the total number offeatures added by an observation divided by the number of hours worked on the task by the pair.

* significant at 10%; ** significant at 5%; *** significant at 1%

41

Table 1.16: Effect of Team Work & National Diversity on Output, Ordered Logit

(1) (2) (3) (4) (5)Joint Output Predicted Values

VARIABLES Coefficient Estimates No Features One Feature Two Features ThreeAdded Added Added Features Added

Team Work 0.804* 0.088 0.513 0.357 0.041(0.460)

National Diversity* -1.741*** 0.356 0.540 0.097 0.008Team Work (0.670)

Independent Work 0.178 0.593 0.210 0.019

Observations 162Pseudo R-squared 0.1308Wald Chi Squared 50.89

Notes: An observation is a pair of workers. Robust standard errors in parentheses. Country pair and week fixed effects are included inall regressions. Controls included in all regressions are team averages for member education, platform experience, non-platform workexperience, number of platform tests, presence of a profile page, gender, wage bid, and agency membership. Joint Output is the totalnumber of features added by an observation.

* significant at 10%; ** significant at 5%; *** significant at 1%

1.9.2 Data Appendix

Supplementary Job Instructions Information

39

Below is the document sent to contractors hired for the job used in this experiment.

I would like some customizations made to DokuWiki, an open source PHP-based wiki engine. DokuWiki

uses plan text files so it does not need a database. The site is internal, not available to the public internet so I

cannot share the URL with you. For more information on DokuWiki, see http://en.wikipedia.org/wiki/DokuWiki

and https://www.dokuwiki.org/features.

The task is as follows: Please add as many of the below Javascript/PHP features in the attached

code as possible and submit the code with the added features as soon as you have added everything you

are able to. One feature isJavascript/PHP only and one uses both Javascript/PHP and PHP/Javascript

(you are to work on the Javascript/PHP part of this task). You will be paid for eight hours of work on

this project and all eight hours of work must be performed on Day of the week, Month Date. Another

contractor from country teammate is from has been hired to work on the PHP/Javascript features in this

code. You and this contractor will be working on this in the team room at the same time

39Italicized words indicate content that varies by the type of coding language the contractor was hired to complete. Boldedwords indicate content that varies by whether contractors are in the team work treatment or not.

42

so please communicate with each other to work through this task together/You will work

independently of this contractor. Unfortunately I am the hiring manager and I have little knowledge of

the technical aspects of the task. Therefore, I am not available to answer questions so just do the best you

can. Please send me your output and let me know which features you were able to add once the eight hours

is up. Please also update your memo to let me know what you are working on. Thanks again!

You can login with the username “admin” and the password “asdf”.

You’ll likely need to fix the permissions on the data/ directory so that the web server can write to them

(probably chwon -R www-data data/, where www-data is the user which your webserver runs as).

PHP Task

Login using either username or email address

Currently users must use their username to login. Allow them to use either their username or their

email address to login.

For example, the ”admin” user has the email address “admin@example.com” and the password “asdf”.

Allow the “admin” user to login either by entering the username “admin” and the password “asdf” OR by

entering the email address “admin@example.com” and the password “asdf”.

Javascript Task

Make a popup for the login dialog.

Currently clicking ”login” directs the user to a new page. Update this link so that, when clicked, a

popup containing the login form is shown (similar to, for example, http://www.meetup.com/).

Combined Javascript & PHP Task

Show when a page is being edited.

Make the “edit page” text red when the current page is being edited.

Use AJAX to poll the server every 15 seconds checking to see whether the current page is being edited.

43

If it is, change the color of the ”edit page” text to red. Change it back to the original blue when the page is

no longer being edited.

For example, if Alice and Bob both have the start page open, then Alice clicks “edit this page”, the

color of the “edit page” text in Bob’s web browser should change to red. Once Alice finished editing the

page (either by cancelling the edit or saving the new page) the color of the ”edit page” text should return

to the original blue.

Supplementary Job Posting Information

This section describes additional information about the job posting information beyond what was described

in section 1.4. Other than the title of the job and the job description, the information available to contractors

on job postings is standardized by oDesk. In particular, oDesk posts employer information on all job postings

to applicants can see how many contracts employers have hired for on the site, how much they have spent and

feedback from previous hires. In addition, all job postings have to specify the estimated time the contract

will last and the approximate number of hours per week the job will require. Both of these measures must

be selected by the employer from a list of pre-specified options. Screenshots of the Javascript and PHP job

postings used for the experiment are provided in the appendix of this paper.

An important requirement of oDesk job postings for the purposes of this experiment is that employers

must specify at the time of posting which team room contractors hired for the job will work in. Once

hired, contractors cannot be removed from this team room. Therefore, to ensure that contractors in the

team work treatment were only able to communicate about this task with their teammate and that those in

the independent work treatment were not able to communicate with any other contractors about the task,

one job posting for each participant was required. This institutional feature of the site made it necessary

to determine which jobs would allow team work among pairs of contractors and which would not before

contractors were able to apply for them. However, job postings do not indicate which team room the hired

contractor will work in, and the job postings do not differ by treatment group so applicants cannot have

known in advance of being hired whether they would be working with a teammate or not.

44

Below are screenshots of the Javascript job posting and the PHP job posting. The employer work history

and feedback is blocked out to protect the privacy of contractors on the site.

Figure 1.4: Javascript Job Posting

45

Figure 1.5: PHP Job Posting

46

Supplementary Interview Information

Below is the text provided to interviewees for the Javascript job.

Hello, thanks for applying to my job. I have four interview questions for you to answer. Please answer as

honestly as possible as we are looking to hire the person best suited to this job. Please answer all questions

in an oDesk message; Im not available to communicate over Skype.

1) If you have any Javascript experience, please give up to 5 examples of your experience.

2) PHP knowledge is not needed for this job but if you do have any PHP experience, please give up to 5

examples of your experience.

3) Please list all the countries you have lived and/or worked in.

4) In one paragraph, please describe why you think you are well suited for this job.

Also, please confirm whether you are able to work your hours on this job on (Date job is to be completed

on).

Below is the text provided to interviewees for the PHP job.

Hello, thanks for applying to my job. I have four interview questions for you to answer. Please answer as

honestly as possible as we are looking to hire the person best suited to this job. Please answer all questions

in an oDesk message; Im not available to communicate over Skype.

1) Javascript knowledge is not needed for this job but if you do have any Javascript experience, please give

up to 5 examples of your experience.

2) If you have any PHP experience, please give up to 5 examples of your experience.

3) Please list all the countries you have lived and/or worked in.

4) In one paragraph, please describe why you think you are well suited for this job.

Also, please confirm whether you are able to work your hours on this job on (Date job is to be completed

on).

47

Chapter 2

Does Information Help or Hinder JobApplicants from Less DevelopedCountries in Online Markets?

with Ajay Agrawal & Nicola Lacetera

2.1 Introduction

The growth of online markets for contract labor has been fast and steady. Workers in this market earned

about $700 million by 2009, and the market for online contract labor was estimated to be worth $1 billion

annually at the end of 2012. The quarterly wage bill on oDesk, the biggest online contract labor platform,

increased by approximately 900%, from $10,000,000 to almost $100,000,000 over the 2009-2012 period.1

North-South exchange dominates the pattern of trade in these markets; employers are predominantly from

high-income countries, whereas the majority of contractors are from lower-income countries. Current trends

in these markets show that this geographic distribution of labor supply and demand is likely to persist and,

in fact, become even more pronounced.

This globalization of traditionally local contract labor markets may increase the returns to outsourcing

1In addition, Elance went from having 125,000 jobs posted in the first quarter of 2011 to having almost 200,000 posted inthe first quarter of 2012, and contractor earnings on the site grew from about $70 million in 2009 to almost $150 million in 2011(Elance.com, 2012). Similarly, contractors registered on oDesk earned about $60 million in 2009, and this amount grew to over$220 million in 2011 (oDesk, 2012). See Agrawal et al. (2013); Financial Times (2012); Horton (2010) for detailed descriptionsof these markets.

48

and, more generally, lead to a more efficient organization of economic activity. This may be of particular

importance for small firms that have difficulty arranging traditional outsourcing agreements. Given their

growth and range of industries these markets are impacting, they have the potential to ultimately affect the

allocation of work as well the organization of firms, for example by affecting their optimal size.

The globalization and digitization of labor markets, however, is not without some major challenges for

companies. As shown also in other contexts where digitization is expanding, physical and cultural distance

is likely to affect online contract labor (Gaspar and Glaeser, 1998; Lyons, 2013). Studies on virtual workers

and virtual teams, for example, find that computer-mediated communication is problematic in a number of

ways (Maznevski and Chudoba, 2000); that establishing trust and organizational identity is more difficult

with online workers and teams (Pearce, 1993; Wilsona, Strausb, and McEvily, 2006); and that IT-mediated

communication might negatively affect knowledge transfer within organizations (Rosenkopf and Almeida,

2003). Scholars have also stressed how the reliance on online markets, for labor as well as for other goods

and services, introduces certain transaction costs due to the inability to meet face-to-face and solve particular

informational asymmetries, for example about motivation, effort, and certain qualities of prospective workers

(Autor, 2001; Malone, 1998).

These challenges may be of particular importance in the case of online contract labor markets, given

that contractors are increasingly from lower income countries, and hiring organizations are located in high

income countries. Differences in culture, language and attitudes may exacerbate the difficulties of organizing

the work of distant contractors and can present several types of informational asymmetries. Thus, although

hiring contractors located in developing countries may be financially cheaper, other cost considerations may

lead prospective employers to prefer contractors located in closer and more similar environments for whom

the asymmetries are perceived as less severe.

One way in which these transaction costs may be lowered is through the provision of verified information.

Although contract labor platforms do not verify all information, such as prior offline work experience, they

do provide verified information about activity that occurs on the site, such as work experience accumulated

49

on the platform.2

In this paper, we focus on the difference between contractors from less developed countries (LDCs)

applying for jobs posted by employers in developed countries (DCs) as opposed to those applying from

DCs and on the effect of verified information on the likelihood of recruiting DC versus LDC job applicants.

Specifically, we examine whether verified information about platform-specific work experience helps or hin-

ders LDC relative to DC applicants. The theory and evidence on the effect of information about credentials

on hiring decisions is ambiguous. On the one hand, this information might further penalize job applicants

at an initial disadvantage (LDC applicants in this case) because employers discount the information about

individuals in this group, thus giving a further lead to initially advantaged contractors. This is reported in a

number of studies, especially in the literature on labor market discrimination (Bertrand and Mullainathan,

2004; Carlsson and Rooth, 2007; Lahey, 2008). On the other hand, some studies show that information on

credentials may disproportionately benefit disadvantaged individuals because, at the margin, information

has a higher influence on the employers perception of the applicant, leading to a larger positive update in

beliefs (Figlio, 2005; Heckman, Lochner, and Todd, 2008; Lang and Manove, 2006; List, 2004; Tilcsik, 2011).3

We base our empirical analysis on 424,308 applications for 14,733 jobs posted on oDesk, the largest and

fastest growing platform for contract labor in the world at the time of writing. First we find that, all else equal,

applicants from LDCs are only about 60% as likely to be hired by employers from DCs relative to applicants

from DCs. This seems consistent with the idea that, despite the immediate financial savings, a number of

difficulties are identified by prospective employers when considering hiring from (geographically, socially and

culturally) distant locations. However, given the intent of the platform to aggregate and integrate labor

markets (Groysberg, Thomas, and Tydlaska, 2011), the magnitude of this hiring difference is striking. This

holds even after we control for most of the characteristics that employers observe (the ability to observe almost

everything that the employer observes is a particularly research-friendly feature of online labor markets) and

2Pallais (2012) shows that even small amounts of verified information, specifically previous work experience on a platform,can dramatically improve employment opportunities as well as wages for contractors.

3Altonji and Pierret (2001) offer a theoretical basis for this: they suggest that employers with little information aboutpotential hires may statistically discriminate on the basis of race but that the relationship between race and wages should fallas employers accumulate more information about worker productivity.

50

for job-level unobserved heterogeneity.

Second, the data indicate that there is an experience benefit; applicants with more platform-verified

work experience are more likely to be hired. Third, and most central to the objective of this research, we

find evidence of an LDC experience premium. Specifically, the benefit from verified experience information is

disproportionately higher for LDC relative to DC applicants. In other words, the LDC penalty is at least in

part due to a disproportionate difficulty in evaluating the quality of LDC applicants. The provision of verified

information about work experience is therefore particularly valuable for LDC applicants. In particular, back-

of-the-envelope calculations suggest that without the information provided by platform experience, employers

could be spending between 6% and 8% more on their oDesk wage bills by avoiding hiring from LDCs as a

result of uncertainty. In the Appendix, we propose a simple model to explain this finding, based on a form

of statistical discrimination due to different priors on the quality of applicants.

We also find that the LDC experience premium is not driven by a particular type of work (e.g., admin-

istrative, website development, writing) but rather is robust across most job categories. Furthermore, the

LDC experience premium applies to a variety of outcome measures in addition to our primary outcome (i.e.,

the probability of hiring a given applicant). In particular, the wage that individuals bid for a job increases

with experience for all contractors, but especially so for contractors in LDCs. Similarly, the likelihood of

being shortlisted and of being invited for an interview both increase with platform work experience, again

more so for LDC contractors. Our results also suggest that if an employer previously used the platform,

then she is more responsive to verified experience information when making hiring decisions, especially so

for LDC applicants. Thus, employer learning may be a factor in interpreting this information that concerns

prior experience.

Finally, other tools on the platform that reduce the risk of hiring a lower-quality contractor diminish the

value of verified experience information. In particular, hourly-wage contracts facilitate low-cost monitoring

through a feature on the platform, but fixed-fee contracts do not. We report that the LDC experience

premium is substantially higher for jobs that employ a fixed-fee contract. In other words, the monitoring

tool partially substitutes for the provision of verified experience information. Thus, there are multiple

51

channels through which online platforms can flatten the market for contract labor, and verified experience

information is only one of them. For example, Ghani, Kerr, and Stanton (2012) identify online ethnic hiring

by Indian-origin employers of contractors in India as another channel; Stanton and Thomas (2012) report

evidence that online employment agencies help contractors overcome the inexperience barrier.

Contract labor market platforms also encourage employers to rate the contractors they hire. These

ratings are observable by all market participants, and are intended to inform about a contractors reputation

on the site. There are several reasons why, in this paper, we focus specifically on experience rather than

ratings. First, as has been shown in prior research, online rating systems are susceptible to gaming (Brown

and Morgan, 2006). Second, there is very little variation in contractor overall rating scores which is consistent

with what has been found on other platforms such as Ebay (e.g. Cabral and Hortacsu, 2010). Finally,

experience plays a major role in rating scores given that ratings are accumulated with experience. Our focus

on an alternative measure of reputation is consistent with more recent research on information in online

markets (e.g. Elfenbein, Fisman, and McManus, 2012; Stanton and Thomas, 2012).

The findings on learning and monitoring also help to distinguish between an information interpretation

of our main result and a skills explanation whereby LDC candidates become disproportionately better with

experience. These explanations may be observationally equivalent as they could both lead to a dispropor-

tionate impact of experience on the likelihood of hiring an LDC applicant; however, our additional analyses

lend support to the former interpretation. For example, the stronger effect for experienced employers is

inconsistent with a skills explanation because inexperienced employers should equally value quality. In con-

trast, it is consistent with the information interpretation; employers learn from experience how to interpret

the information. Similarly, the findings on contract type are consistent with the information interpretation

but not with the skills explanations. If our main finding was the result of omitted variable bias (e.g., actual

applicant quality, instead of an asymmetry in beliefs about quality across the two groups), then we would

not expect to find a higher LDC experience premium for fixed price versus hourly contracts.

In addition to offering insights on how even limited amount of verified information might affect online

hiring patterns and, ultimately, the organization of work within and between firms, we also contribute to

52

a growing stream of literature on labor market globalization. Advances in ICT have contributed to growth

in offshoring both goods and services. Several papers note the potential productivity gains from service

offshoring (e.g. Antras and Helpman, 2004; Grossman and Rossi-Hansberg, 2008). A theory developed in

Antras, Garicano, and Rossi-Hansberg (2006) suggests that these productivity gains will be especially pro-

nounced with workers in LDCs. Although we do not test job performance outcomes for online contractors,

our findings have implications for potential informational barriers to performance gains from trade in ser-

vices. Our findings also have implications for research on labor market outcomes for migrants that considers

employer hiring practices in localized DC labor markets where immigrants from LDCs compete with immi-

grants from DCs and native workers and finds significantly lower success rates for LDC immigrants (Ferrer

and Riddell, 2008; Oreopoulos, 2011). Oreopoulos (2011) provides evidence that this may be because em-

ployers in DCs value work experience acquired in DCs more than similar experience accumulated in LDCs

and Dequiedt and Zenou (2013) provide theoretical evidence that employer statistically discriminate immi-

grants because of imperfect information. Our findings reinforce these interpretations and suggest that, even

in online labor markets, where technology has blurred the lines between developed on developing economies,

employers in DCs have difficulties assessing LDC worker quality. Our finding of the relative importance

of verifiable information for LDC workers might imply that immigrants from LDCs participating in more

traditional labor markets in DCs could benefit from carefully constructed and monitored skills certification

programs and that employers in DCs could also benefit from certification mechanisms that enhance their

ability to screen immigrant applicants.

We describe our research setting in Section 2.2, the data in Section 2.3, and our empirical design in

Section 2.4. Results are reported and interpreted in Section 2.5, and Section 2.6 concludes.

2.2 Empirical Setting

We conduct our study using data from oDesk, an online platform designed to facilitate employer-contractor

matches. The Silicon Valley-based company was founded in 2004 and experienced rapid growth every year

53

since. According to the company, since founding, the cumulative transaction value exceeds US$225 million,

the total number of jobs posted exceeds 1 million, and the total number of contractors that are part of the

oDesk network is approaching 1 million.4. In terms of the number and value of transactions per year, oDesk is

the largest company in its industry, which includes other rapidly growing online market makers for contract

labor such as Elance, Guru, and Freelancer. The Financial Times (2012) estimates that the combined

transaction volume across these platforms will be about $1 billion by the end of 2012. Overall, these online

platforms are similar to each other in terms of their purpose, structure, and business model, although there

are some differences in areas such as employer monitoring ability, secondary sources of platform revenue,

and the types of employer and contractor information provided.

Employers register on the platform and then post jobs on the site. Contractors register on the platform

and then bid for jobs. Bid information includes a proposed fee, cover letter (optional), and profile of

the contractor, which includes information such as education, work experience, and location. Employers

review bids and may short-list and interview promising bidders prior to making a decision and hiring a

contractor. The employer may decide against hiring any contractor and cancel their job without penalty.

Upon completing a job, the employer pays oDesk the pre-specified project fee and rates the performance of

the contractor. The contractor also rates the employer. oDesk pays the contractor and records the job in the

contractor’s job history. We utilize this latter piece of information as a measure of platform-specific work

experience.

Employers classify each job they post as being one of eight types: web development, writing & transla-

tion, administrative support, software development, business services, design & multimedia, customer service,

and networking & information systems. In addition, the employer provides a description of the job and the

skills required to complete it. Also, the employer specifies the nature of the contract: hourly or fixed fee.

oDesk adds other information to the posting including the employer’s location and their previous activity

on oDesk.

4In its first two years of business, oDesk experienced revenue growth of 4,573% (Deloitte, 2007) and continued to grow atmore than 1,000% per year (Chafkin, 2010) In 2007, total earnings on the site were less than $20 million. The same amountwas earned in September 2011 alone, with an average pay of $4,000 per job. In 2010, contractors earned almost $120 millionon the site.

54

Contractors advertise themselves by posting profiles that include information on their education, work

history (both on and off the platform), and country of residence. oDesk reports each contractor’s entire

oDesk work history, including the amount paid for each job, a description of each job and, for completed

jobs, employer feedback, on the contractor’s profile. In addition, oDesk offers contractors the option to

demonstrate their abilities by taking oDesk-administered tests, although posting the results is optional.

Although the majority of contractors work independently, some are associated with agencies that employ

staffing managers who handle job applications and take a percentage of the contractor fee.

oDesks’ business model is based primarily on transaction fees. Specifically, the platform does not charge

employers for posting jobs, but does charge employers 10% of the transaction value when a contractor is

paid at the end of a job. There are no additional fees charged to contractors.

2.3 Data and Descriptive Statistics

2.3.1 Dataset construction

We collect data on all job postings and applications on oDesk from the month of January, 2012. There

were 90,585 jobs posted during this period. Of these, 45,313 were filled (i.e. contractors were hired); only

one contractor was hired in 36,921 of the cases, whereas in the remaining 8,392, multiple hires were made

(with a range between 2 and 632). We focus on the cases where a single contractor was hired.5 Also, we

focus on postings for which at least one applicant was from an LDC, at least one was from a DC, and the

job was posted by a DC entity. Our final sample includes 14,617 job postings and 420,833 job-application

observations.6

5Direct experience on the platform and conversations with oDesk personnel revealed that jobs for which multiple peopleare hired may be posted for a number of different reasons. For example, employers may be running tests or trials in order tothen select one single contractor for a subsequent job. Although this is not common, oDesk and other platforms may also beused by researchers to run experiments, where typically multiple people are hired for one job. The motivations for posting andfilling these jobs (and possibly for applying for these jobs) are potentially very different than what we normally associate withemployer motivation for hiring in ways that would add noise to our data. For these reasons, we limit our sample to jobs forwhich only one applicant was hired.

6In the Appendix, we compare job characteristics between jobs that were dropped from and those included in our finalsample. In particular, we compare our final sample to: (1) the sample of jobs with multiple hires (and both DC and LDCapplicants); and (2) the sample of jobs with 1 hire and either only DC or LDC applicants. Although minor differences existbetween the three groups, they look similar along most characteristics, particularly when comparing the sample used in thispaper and the sample of jobs with only LDC or DC applicants, and the differences that do exist between our sample and the

55

Applicant success (being hired for a given job) is our main outcome variable. We code it as equal to 1

if a contractor is hired and 0 otherwise. We define LDC status using an indicator equal to 1 if a contractor

resides in an LDC and 0 otherwise, using the World Bank classification The World Bank Group (2011).

We operationalize platform-specific experience using the number of previous job contracts, with an indicator

that equals 1 if contractors have more than the sample median number of prior contracts (4) and 0 otherwise.

Figure 2 shows the distribution of online experience in our sample. The distribution is highly skewed, with

about 75% of applicants reporting less than 15 previous jobs, and a handful of individuals reporting 100

or more previous tasks completed. The distribution of offline job experience is even more skewed, with the

75th percentile being 2 jobs, and a few cases of 50 or more jobs (the max is 94). Thus, we use indicator

variables for job experience since these data are so skewed. However, in the Appendix, we report analyses

with alternative, more continuous (but still categorical) measures of online experience and our main findings

persist.

For each observation, we observe a wealth of information from all applicants’ profiles, corresponding

to almost everything that market participants observe. As further discussed in Section 4 below, this is a

particularly research-friendly feature of the data that allows us to control for almost all available information,

leaving very limited concerns for omitted-variable issues in the regression analysis. Specifically, we observe

contractors’ education, work history (both on and off oDesk), test scores, oDesk feedback rating, agency

membership, country of residence, oDesk advertised wage, wage bid for a given job, previous jobs held on

the platform, whether they have a profile picture, whether they were shortlisted and/or interviewed for the

job, and whether or not they were previously hired by the employer who posted the focal job. We also collect

summary information on the application letter; specifically, we have a measure of how original the content

of a letter is, relative to an automated form letter. Sending a form letter often reflects scarce interest in a

job. In our analyses below, we find that a higher share of original content does indeed correspond to higher

hiring probability. Finally, we have information on whether the application was initiated by the employer or

the contractor and on job and employer characteristics. Table 2.1 reports a description of all variables and

multiple hires seem largely due to more hires being made in the latter sample. Note also that our main regression results arerobust to including the sample of jobs with multiple hires (Table 2.15.

56

how we construct each.

2.3.2 Descriptive Statistics

Table 2.2 reports summary statistics on our sample of contractors (more specifically, contractors-applications),

and Table 2.3 reports statistics on our sample of jobs and employers. A large majority of the contractors in

the sample (364,921 or almost 87%) are from LDCs, and the average share of applications by LDC contrac-

tors for a given job is 77.7%. However, LDC contractors are only hired for 66.5% of the jobs.7 Of course, this

disproportionately low rate of hiring LDC contractors may be explained by differences in quality between

LDC and DC contractors or by differences in the types of jobs they apply for. We address these issues in the

regression analyses to follow. Foreshadowing that analysis, the descriptive evidence in the last two columns

of Table 2.2, where we report contractor characteristics, suggests that LDC and DC contractors are similar

on many dimensions.

Some differences, however, are worth noting: LDC contractors are slightly more educated than DC

contractors and they are also more than twice as likely to be members of employment agencies. Contractors

from DCs have higher test percentages on average than contractors from LDCs but, given that contractors

can delete scores, it is unclear whether this difference is because DC contractors do better on tests or because

they are more likely to delete bad test scores from their profiles. In addition, DC contractors have much

higher average advertised wages and wage bids than contractors from LDCs. LDC contractors are less than

half as likely as DC contractors to be invited to apply for a job by the employer and much less likely to have

been hired by the employer in the past. Finally, LDC contractors appear to write less original cover letters

than DC contractors. In summary, although there are some differences between the sample of DC and LDC

contractors, these differences do not appear to reflect clear differences in ability or quality.8

The raw data also suggest that experience on the platform, although similar on average between LDC

7A regression on whether a contractor from an LDC is hired for a job on the share of applicants from LDCs for that job,with one observation per job and the constant set at zero, estimates a slope of 0.89, significantly below 1.

8A Blinder-Oaxaca type of decomposition of the likelihood of being hired shows that of the 6.35 percentage points of differencein the likelihood of being hired for DC and LDC applicants (9.08%-2.71%), only about 6.4 percentage points can be attributedto the (observable) characteristics of the applicants.

57

and DC applicants, provides differential benefits in terms of likelihood of being hired. This likelihood is

positively correlated with work experience on oDesk for both LDC and DC contractors. However, in relative

terms, LDC contractors benefit more from oDesk experience. Specifically, DC contractors with experience

below or equal to the sample median of 4 previous jobs are about 4 times more likely to be hired than

LDC contractors in the same experience group (0.067 vs. 0.017), whereas the ratio declines to about 3:1

for more experience applicants (0.114 vs. 0.037); the hiring chances thus increase more than twofold for

LDC contractors with above-median experience, as opposed to a 60% increase for DC applicants. Therefore,

although a gap in hiring chances between LDC and DC workers remains, having more experience on the

platform appears disproportionately beneficial for LDC applicants.

We illustrate the large impact of oDesk experience for LDC applicants in Figure 3: here we report,

together with the average share of LDC applications per job, the share of LDC applicants that were hired

overall, the share of LDC applicants with low experience that were hired out of all those with low experience

that were hired, and the share of LDC applicants with high experience that were hired out of all those with

high experience that were hired. The figure shows a substantial difference in the likelihood that a hired

applicant is from an LDC, in the subgroups of high-experience and low-experience hired workers. The raw

data is therefore suggestive of DC employers’ reluctance to hire from LDCs (note also that the raw likelihood

of being hired for DC freelancers with no previous experience on oDesk is similar to that of an LDC with

online experience in the top quartile of the distribution); and the relative benefits to LDC contractors from

building a reputation by accumulating experience on the platform.

These results are consistent with the following interpretation, based on a form of statistical discrimi-

nation due to different priors on the quality of applicants. Previous job experience appears to represent a

positive signal that increases the likelihood of being hired, with the updating on quality being stronger for

applicants who are at an initial disadvantage in terms of employers having a lower prior about their ability

and/or higher uncertainty around this prior. This applies only for experience on the platform because it is

comparable among workers from different origins, and is also a signal of ability on the job because having

platform experience also implies having won a contract over competitors for a given job. In the Appendix,

58

we propose a simple formalization of this view. Interestingly, the descriptive statistics on wage bids display

a similar pattern: the increase in the natural log of wage bids for LDC applicants with and without above-

median experience is 0.32, as opposed to 0.24 for DC applicants. The fact that it might take a relatively

short period of time to accumulate this experience suggests that the effect is more likely due to the reliability

of this verified information, rather than due to the acquisition of skills from the experience. We address this

issue more formally in the empirical section. We now proceed to regression analysis to test the robustness

of these basic descriptive findings and their interpretation.

2.4 Empirical Strategy

To address our main research question - does verified information on work experience disproportionately

benefit LDC applicants? - we must consider a few identification concerns. First, employers may be less likely

to hire contractors from LDCs because they are of lower quality rather than because of their country of

residence. Similarly, employers may be more likely to hire contractors with high oDesk experience because

these contractors have other qualities that are valued by employers. However, unlike in labor markets where

employers and applicants are able to meet in person and learn more about each other in ways that are

unobservable to the researcher, the variables that we observe, and that we described in the previous section,

represent virtually all the information available to employers about applicants. Thus, controlling for these

variables in a regression framework considerably allays omitted-variables concerns.

What we do not observe are private interactions (offline or not mediated/recorded by oDesk) between

applicants and employers. Through these interactions, job posting entities may extract further information

on the quality and fit of applicants, potentially related to their origins as well as their experience level or other

observables. From direct experience with hiring on the site, personal interactions initiated by applicants are

minimal and do not provide much information about contractor quality beyond what is already provided in

profile pages. Moreover, some variables in our dataset could be more directly correlated with the likelihood

of informal interaction, before or during the job posting and hiring process. For example, in some cases, as

59

mentioned above, employers invite particular contractors to apply for jobs. Also, there are instances where

the pool of applicants includes some contractors who already worked for the same employer in the past. The

analyses reported below are robust to excluding jobs (and all applicants for those jobs) where any of these

two indicators is positive for at least one applicant (Table 2.16). Another source of information that we do

not observe and that is potentially related to the origin and other characteristics of the applicants is the

exact content of cover letters that applicants sent with their application. If, for example, applicants from

LDCs or with lower experience are worse at writing cover letters, then this might indicate lower quality.

Again, as mentioned above, we rely on a proxy for the content of the cover letter, as given by the share of

original content in the letter.

Second, we account for potential differences across job and employer characteristics by using a regression

model that conditions on job-employer characteristics. We model the effect of our covariates on the likelihood

of being hired through a conditional fixed-effect logit model (McFadden, 1974), where we group the data by

job posting (or employer-job posting), and the alternative set for each job posting includes the applicants to

that job. More specifically, we treat each application as a separate observation even though some contractors

apply for more than one job in our sample (Of the 420,833 job-application observations, we have 75,972 unique

contractor observations).9 This framework is appropriate in our setting for several reasons. First, employers

can only hire from the contractors who apply for their job and we require employer choice sets to reflect this

restriction. Second, it is likely that employers consider all their options when choosing whether or not to

hire a contractor so that each hiring decision is conditional on all other applicant characteristics. Third, this

model also explicitly assumes that each employer hires the applicant who maximizes her own utility. Fourth,

we calculate the likelihood of being hired in this model relative to each job (McFadden, 1974; Cameron and

Trivedi, 2009).

9We treat each application as a separate observation even though some contractors apply for more than one job in oursample. We could, in principle, run analyses with individual fixed effects. However, within individuals there is no variation inLDC status, and only for a handful of applicants does the online job experience, our other main variable of interest, move fromlow to high in the one month of data that we have. In addition, focusing only on those individuals with multiple applicationsand variation in the experience indicator would censor the sample as the employers would be modeled as choosing an applicantout of a subsample of all applicants for that job. Individual fixed effects would be a way to deal with remaining variation thatthe employers could observe and the researcher could not; however, given the types of interactions online as explained above,the detailed information we have on each applicant for a job, and the additional robustness tests we describe below, we believethat our empirical strategy addresses the possibility of biases from selection or omitted variables.

60

More formally, let Aj represent the set of k applicants for job j, let Yij be an indicator for whether

applicant i is hired. Each employer maximizes her utility according to the characteristics of alternatives:

Uij = α +Xiβ + ϵij , where Xi is a vector of applicant characteristics, is a vector of parameters, and ϵij is

the logit error term (type I extreme value). Therefore, the conditional probability that applicant i is hired

out of Aj applicants is:

P (Yi = 1|∑hϵAj

Yh) =eXiβ∑

hϵAjeXhβ

(2.1)

where is a vector of parameters to be estimated through maximum likelihood.1011 Our main regressors of

interest are an indicator for whether an applicant is from an LDC, and measures of previous job experience.

Third, besides the potential for omitted variables bias (at the individual applicant or job posting level),

our estimates may also suffer from selection bias. In particular, we may be concerned that more experienced

contractors are better at applying for jobs for which they are likelier to be hired. Because contractor ability to

apply for the right jobs should not vary with employer characteristics, provided that applicant characteristics

do not differ across these employer characteristics, we reject this interpretation of contractor learning below

by showing that online experience premiums vary with employer experience on oDesk whereas applicant

characteristics do not. We also provide analyses with alternative outcome variables as well as additional cuts

of the data to corroborate our main findings.

10Note that∑

hϵAjYh = 1 for each job, because there is only one hire per job in our case. The results are very similar

with alternative discrete choice specifications, such as alternative-specific conditional logit as well as mixed logit models withobservations grouped at the job-employer level. Alternative-specific conditional logit allow for a separate constant to beestimated for each alternative (the conditional logit model we use here is equivalent to an alternative-specific conditional logitwhere the constant terms are constrained to be the same). Mixed Logit (or random-coefficients) models allow for coefficientsto vary across groups, and also overcome a common limit of choice models given by the independence of irrelevant alternatives.The point estimates we obtained from mixed models are almost identical to the conditional logit estimates.

11A potential alternative specification would be to use a linear probability model with job-level fixed effects. This wouldmake the interpretation of the estimated coefficients more immediate. However, there would be some important limitations andconcerns. First, and related to the advantages of a conditional logit framework, a linear probability model would not reflectthe choice structure embedded in the hiring problem. In addition, in order for all applicant characteristics to be consideredin each individual hiring decision, we would have to make strong assumptions about how these characteristics enter into theemployer’s choice problem to be able to control for them. One alternative would be to control for all applicant characteristics ineach individual decision, but this would be very difficult, particularly with jobs that have many applicants. Finally, in a linearfixed-effect model there would be an inherent correlation in the error terms due to the fact that for each job posting one andonly one hire is possible (and observed). In any event, linear probability models convey very similar estimated marginal effects.The logit coefficient estimates, and in particular the coefficient on the main interaction term of interest, has an immediateinterpretation in terms of multiplicative effect.

61

2.5 Results

We structure our analysis in three steps, each based on estimating a specification of Equation (1). First,

we estimate the LDC penalty, the platform-specific experience benefit, and the LDC experience premium.

The penalty, the discount that the average employer applies to applicants from LDCs, is reflected in a

lower probability of hiring a contractor from an LDC compared to a DC after controlling for observable

characteristics such as education and off-platform experience. The platform-specific experience benefit is the

average increase in probability of hiring an applicant that has accumulated work experience on the platform

compared to one that has not. The LDC experience premium is the extent to which LDC applicants

experience a disproportionate benefit from accumulating platform-specific experience. We examine each of

these three relationships in the context of our main outcome measure (likelihood of being hired) and then

estimate them using two earlier-stage outcome measures (attaining an interview and being short-listed) as

well as with another key measure: wages.

Second, we show that the LDC experience premium is reasonably robust across job types (e.g., adminis-

tration, web development, and writing). We also report evidence that suggests interpreting the information

associated with platform-specific experience might require some investment in learning to use the platform;

we find that the LDC experience premium is larger for employers with more hiring experience on the platform.

Finally, we report findings suggesting that the LDC experience premium is greater under conditions that

are less conducive to monitoring (fixed-fee versus hourly contracts). In addition to providing further insight

on the characteristics of the LDC experience premium (concerning learning and monitoring), the latter two

results also provide further evidence that supports the causal interpretation we advance here concerning how

verified information about experience increases the probability of being hired.

2.5.1 Main analyses: Likelihood of Being Hired for a Job

We begin by estimating the LDC penalty (Table 2.4). The first specification is a pooled logit with standard

errors clustered at the job level, with no control variables. We then add controls and employer-job fixed

62

effects. The penalty is large and statistically significant in all specifications. In the uncontrolled logit

specification (column 1) the estimated LDC penalty is very large: the probability of an LDC applicant being

hired is less than a third than that of a DC applicant. When we add controls (column 2) the coefficient

estimate on the LDC indicator increases (odds ratio from 0.31 to 0.56), indicating that part of the LDC

penalty is explained by observable quality differences between LDC and DC applicants. A conditional fixed-

effect model (column 3) estimates a further reduced gap, indicating that there are probably also differences

in jobs and employers that affect the likelihood of a an employer (typically from a DC) hiring an applicant

from an LDC. These differences can include the quality of matching jobs with applicants, which may be

related to the origin of the applicant. Still, even after controlling for alternative-specific covariates and

employer-job heterogeneity, the odds that an LDC contractor is hired are estimated to be only slightly above

half that of a DC contractor. Interestingly, these values are similar to those found in studies of the likelihood

of hiring or call-backs for minorities (especially African Americans) compared to Caucasians (Bertrand and

Mullainathan, 2004; Pager, Western, and Bonikowski, 2009).

Next, we turn to our estimates of the platform-specific experience benefit. In the same three specifica-

tions as above (Table 2.4) we find that, on average, applicants benefit significantly, in terms of the probability

of being hired, from work experience on the platform (column 1). The estimated coefficient on the indicator

for platform-specific experience decreases when controls are added (column 2) but increases slightly when

job fixed effects are included (column 3). Overall, when we include controls and fixed effects, the estimate

indicates that having above-median experience on the platform increases the likelihood of being hired by 1.6

times. The estimates on the control variables, with the exception of offline work experience, are statistically

significant (Appendix Table 2.13), perhaps not surprisingly given the large sample size; however, most of

the estimates are small in magnitude. The estimates worth noting because of their size are the fraction

of originality in applicants’ cover letters, employer initiated applications, and the indicator for whether a

contractor was previously hired by an employer. These are all positive.

We then move to our primary phenomenon of interest the LDC experience premium. We do this by

interacting the LDC indicator with the oDesk experience indicator (Table 2.5). The estimated coefficients

63

suggest that LDC contractors benefit disproportionately from above-median platform experience compared

to DC contractors. The unconditional hiring ratio for low experience LDC contractors is 1.6% compared to

6.7% for low experience DC contractors. Therefore, the estimates (in odds ratios) imply that the predicted

probability of a DC contractor being hired increases due to experience by a factor of 1.233 (to 8.3%), whereas

for LDC applicants, the multiplicative factor is 1.233*1.413=1.742 (increasing the estimated likelihood to

2.8%).

In Appendix Table 2.17, we experiment with more fine-grained categorizations of the platform experience

variable, by splitting the sample by quintiles of experience level. The main findings persist. At the top

end of the distribution, 19 or more previous jobs, DC contractors are estimated 2.5 times as likely to be

hired as LDC contractors. However, for inexperienced applicants this ratio is much higher (4.2:1, see also

figure 3). Even still, inexperienced DC applicants are more likely to be hired than highly experienced LDC

applicants. Nonetheless, platform-specific experience significantly reduces the gap. In fact, it appears that

the median level of experience is somewhat of a turning point in terms of the increase in relative benefit from

platform-specific experience for LDC contractors. Note that the median level of platform-specific experience

is relatively low and given the average length of a contracted job can be accumulated in a short amount of

time.

The focus on relative changes in hiring is relevant for firm wage bills, contractor participation in the

market, and, more broadly, the potential impact these markets can have on global income distributions

and incentives for educational attainment. Given that contractors from LDCs charge much lower wages on

average than contractors from DCs, information that allows employers to more easily hire from LDCs has

meaningful implications for firm wage bills. In particular, employers in our sample have hired an average

of 16 contractors on oDesk. Using the odds ratios presented in Table 5, if hiring from the low experience

sample of contractors of these 16 contractors, about 13 would be from DCs and 3 would be from LDCs. Using

average LDC and DC contractor wage rates, each hour during which these 16 contractors are employed costs

employers $240.88. If hiring from the pool of contractors with above-median experience, employers hire 11

DC contractors and 5 LDC contractors which costs them $221.62/hour, a saving of $19.26/hour or 8% of the

64

firms wage bill. These calculations are only suggestive, and do not account for differences in wages across the

experience distribution. However, these differences are between contractors with more or less than four prior

jobs; a relatively small number given the average length of oDesk jobs. Moreover, as we show in subsequent

sections of the paper, the reduction in the inequality between hiring rates of contractors from DCs and LDCs

as contractors accumulate more experience appears to be a result of employer uncertainty associated with

hiring from LDCs. If this uncertainty were eliminated prior to experience accumulation so that the hiring

likelihood ratio between contractors from LDCs and DCs was the same among contractors with few prior

jobs as it is between those with above median experience, employers could save an economically significant

amount on their oDesk activities. In addition, reducing this uncertainty may increase the incentives for high

quality contractors from LDCs to participate in the market. If these contractors believe they will be passed

over for jobs when they first join the platform because of information problems rather than because of their

skill level, their time may be better spent in their local labor market where uncertainty about their abilities

is less pronounced. More generally, these markets have the potential to increase incentives for LDC labor to

invest in human capital that is valued by employers in DCs, thereby increasing the average quality of the

pool of labor participating in global labor markets. However, if contractors believe these investments will

not be appropriately rewarded by employers, they may be less likely to do so.

The findings also resonate with the evidence on discrimination in offline labor (and other) markets

that shows that the availability of more information disproportionately improves labor market prospects for

disadvantaged populations (Figlio, 2005; Heckman, Lochner, and Todd, 2008; Lang and Manove, 2006; List,

2004; Tilcsik, 2011). In our setting, it is remarkable how little experience is required to significantly increase

contractor success, especially for contractors who are at an initial disadvantage. This suggests an important

role for platform-mediated information in global online labor markets.

65

2.5.2 Robustness of Main Result

Job Types

To ensure that our main findings are not driven by one particular type of job, we perform a similar analysis

to that reported in Table 2.5, but separately for each job category. We restrict the analysis to job types

with at least 500 posted jobs and thus consider the following categories: administrative, web development,

writing, software development, and marketing (Table 2.6). Our results indicate that the LDC experience

premium persists for almost all job categories. One exception is software development. In this case, platform

specific experience benefits all contractors equally.

Wage bids

To the extent that information about experience disproportionately benefits contractors at an initial disad-

vantage, then wage bids should increase more with experience for LDC than DC contractors.12 We do find

evidence of this. In Table 2.7, we report the parameter estimates of this log-linear model:13

ln(wageij) = α+ β1LDCi + β2Experiencei + β3LDCi ∗ Experiencei + γ1Xij + ηj + ϵij (2.2)

where wageij is the wage bid by contractor i applying for job j. Because bids have a different meaning

for hourly and fixed contracts, we perform our analyses separately for these two types of contracts. The

other variables are as described in Model (1) above, and ηj represents fixed effects at the job-employer level

(standard errors are clustered at the same level as the fixed effects). An alternative specification that would

get closer to giving us causal estimates would include individual fixed effects, exploiting the fact that some

12In general we expect lower wages for LDCs due to the lower cost of living; however we do not expect contractor experienceto have a differential impact unless information about experience disproportionately affects prior beliefs. A Blinder-Oaxacadecomposition of the log of wage bids reveals that, of the about 0.70 difference in the natural logs of wage bids between DCand LDC employers, only 0.005 (for hourly jobs) and 0.015 (for fixed price jobs) is attributed to differences in (observable)individual characteristics.

13Comparisons of scale-corrected R-squared and sum of squared residuals (based on a normalized Box-Cox transformation,which is necessary to compare two models where the dependent variable in one of them is a nonlinear transformation of theother) show that the log-linear specification is a significantly better fit that a linear specification for wage indeed the wagelevel is highly skewed, making linear regressions less reliable. The R-squared from these corrected regressions about 2.5 timeshigher both for the hourly and fixed-price contract subsamples, and the chi-squared test for the better fit of the log specification[(N/2)*ln(higher SSR/lower SSR)] is highly significant.

66

individuals apply for multiple jobs over the period of interest. However, there is very little within-individual

variation in our main variables; the LDC indicator is invariant across observations for the same individual by

construction and the experience indicators do not vary because of the relatively short time span covered by

the data. Therefore, the evidence presented here should be taken as mostly descriptive, though informative.

The estimates show significantly lower wage offers (by about exp(-.502)-1=-39%, from column 3) for

inexperienced LDC workers as opposed to inexperienced DC workers bidding on hourly wage jobs; however,

the increase in wage offers by LDC contractors is about 65% higher than the increase for DC contractors

(15.6% increase vs. 9.3%). Similar results hold for fixed price bids, with stronger effects for these jobs. This

is consistent with the difference in employers’ ability to monitor contractors under the two contract types

and suggests that when monitoring is more costly, verifiable information about the applicant becomes even

more valuable. We also limit the sample to bids by the winning contractors. These bids can be taken as

better proxies for the equilibrium wage for that particular job, thus they should be more reactive to valuable

information. Estimates are similar to the full sample, especially for fixed price jobs.

Interviews and Short-listing

Two optional steps that an employer may take when considering an applicant for a job include interviewing

and short-listing. As we display in Tables 2.2 and 2.3, 11% of contractors are interviewed and those from

DCs are much more likely to be interviewed than those from LDCs; on average, about 3 interviews are

performed per job. Short listing is less common; only 3.8% of contractors in our sample were short listed and

DC applicants are more likely than LDC applicants to be short listed (5 out of 100 versus 3.6 out of 100).

Using an indicator for being interviewed or short listed as dependent variables in Model (1), we estimate the

LDC experience premium and report the estimated coefficients in Table 2.14. The main result concerning

the LDC experience premium persists with both upstream measures of success in the recruiting process.

67

2.5.3 Employer Learning

Do employers learn to interpret information about platform-specific experience, or is the value of this infor-

mation immediately obvious? The answer to this question is important for two reasons. First, it provides

insight into platform recruiting dynamics. Learning implies that employer recruiting patterns evolve with

experience, potentially shifting from hiring DC to LDC contractors. Second, evidence of employer learn-

ing reduces identification concerns. In particular, if the LDC experience premium were either the result

of contractors improving their quality due to the work experience or improving their ability to apply for

jobs (e.g., better cover letters or more effective pricing) due to experience, then we would not expect to

see variation across differing levels of employer experience. However, it is plausible that employers learn

to evaluate platform-based information, such as prior contractor experience on oDesk, as they themselves

gain experience on the site.14 Indeed, using data from a different online platform for contract labour (Free-

lancer.com), Mill (2011) finds that employers are more likely to hire contactors from a particular country if

they have had a positive experience doing so in the past.

We run the main analyses, as those in Table 2.5, but split the sample into two groups (Table 2.8):

1) employers with no experience hiring on oDesk as of January 2012, and 2) employers with prior hiring

experience. We also show the main result is robust to splitting the sample at the median level of employer

experience. The estimates suggest that, regardless of hiring experience, employers are less likely to hire LDC

contractors, but that employers with at least some previous experience appear to value online experience

more and, most notably, apply a greater LDC experience premium. We also show that the applicants who

apply for jobs posted by experienced versus inexperienced employers have virtually identical characteristics

(Appendix, Table 2.12). Thus, these results suggest that the premium is a result of employer preferences

and not contractor’ application decisions.

14A number of papers have used experience as a proxy for sophistication or ability to process certain signals in markets. Seefor example Altonji and Pierret (2001) and List (2003).

68

2.5.4 Monitoring

The platform provides employers with two types of contracts that they can use to engage contractors:

hourly or fixed fee. The contract type influences the ease of monitoring. Under hourly contracts, contractors

complete their work in a virtual team room where employers are able to monitor their output by way of screen

shots in 10 minute increments.15 The trade off for this level of monitoring is that employers are obligated

to pay contractors for their time regardless of the quality of work. In contrast, under fixed fee contracts

employers do not have easy access to monitor contractors as they work, but employers are able to withhold

payment if they deem that the output is of poor quality.16 In other words, employers are protected from poor

quality work through low cost monitoring in the case of hourly contracts and through optional payment in the

case of fixed fee contracts. Contractors are protected from employer reneging through guaranteed payment

in the case of hourly contracts and employer evaluations in the case of fixed fee contracts.17

To the extent the LDC experience premium is due to verified information rather than better quality, as

we posit, then it should be greater for jobs done under a fixed fee contract compared to those done under

an hourly contract since the employer is less dependent on this type of information in the latter case due to

their ability to monitor. We examine this by splitting the sample by contract type, and report our results in

Table 2.9. As expected, the LDC experience premium is significantly higher for jobs conducted under fixed

fee contracts.

This result is interesting for two reasons. First, it provides further insight into recruiting behavior

in the online platform context. Tools that lower the cost of monitoring may disproportionately benefit

disadvantaged populations; to some extent they may substitute for other tools that have a similar effect,

such as the verified information about prior experience on oDesk that is the central focus of this study.

Second, this result provides further evidence that is consistent with our causal interpretation. We would not

15oDesk takes screenshots of the work of contractors logged into team rooms every 10 minutes so that employers can observecontractors’ progress.

16Employers have the opportunities to not pay the fixed price if they are unsatisfied with the job. However, this happensvery rarely, most likely because of reputational concerns (not paying for a job might lead contractors to post bad reviews abouta given employer).

17Contractors can penalize employers for unfairly withholding payment in fixed fee jobs by giving them a poor rating,potentially deterring strong applicants from applying to subsequent jobs posted by that employer.

69

expect to see such a difference in the estimated LDC experience premium if it were driven by either better

quality applications or better quality applicants. Although still not conclusive, the evidence we report here

is broadly consistent with our interpretation that the LDC experience premium is due to LDC contractors

benefiting disproportionately from verified information concerning prior employment.

2.6 Discussion and Conclusion

We report evidence of an LDC experience premium: LDC contractors benefit more than their DC coun-

terparts, in percentage terms, from verified information indicating prior work experience on the platform.

The premium is present across multiple outcomes: likelihood of being hired, wage, likelihood of obtaining

an interview, and likelihood of being shortlisted; it is also present across most job types. We interpret these

findings as indicating that these platforms facilitate verifiable quality signals that help prospective employers

reduce information asymmetries associated with the quality of contractors, particularly those coming from

LDCs. Reducing these asymmetries could have important implications for increasing outsourcing efficiency.

This may be of particular importance for small firms that have difficulty arranging outsourcing agreements

in the absence of such easily available information. Rather than avoiding services (including high-skill) from

certain countries, firms may increase the value they derive from online labor by being better able to evaluate

the quality of foreign applicants.

With respect to the broader issue of employee reputation, our study contributes to the growing liter-

ature on international labor markets and the relationship between employee reputation and labor market

success (e.g., Banerjee and Duflo, 2000) and with literature that highlights the importance of reputation in

online markets (e.g., Dewan and Hsu, 2004; Jin and Kato, 2006; Resnick et al., 2006). Specifically, given the

importance of verified information about prior experience on oDesk for LDC contractors, such information

appears to act as a certification mechanism for contractor abilities. This mechanism is of particular impor-

tance when other signals of ability are difficult to interpret (perhaps due to a lack of familiarity with foreign

education institutions and employers).

70

Our results also suggest that, in the case of online markets for contract labor, this information benefits

disadvantaged groups relatively more. This is consistent with some prior literature (e.g., Heckman, Lochner,

and Todd, 2008) and inconsistent with other findings (e.g., Bertrand and Mullainathan, 2004). In addition,

the results presented in this paper are consistent with the evidence in Oreopoulos (2011), which shows that

skilled immigrants from LDCs are less likely to be hired in Canada than comparable DC workers. Oreopoulos’

findings suggest that this could be because employers have trouble inferring worker abilities from experience

accumulated in LDCs. Our paper is consistent with this interpretation and, further, implies that this

uncertainty over LDC contractors is present even in markets where the majority of the participants are from

LDCs. Moreover, our research suggests that this barrier to employment for contractors from LDCs may be

at least partially overcome through the use of third party verification (in this case provided automatically

by the platform).

This research also contributes to the growing stream of literature on skewness in online markets (e.g.,

Elberse and Oberholzer-Gee, 2006; Brynjolfsson, Hu, and Simester, 2011) and suggests that the information

provided by online markets for contract labor may reduce the skew of contractor outcomes under certain

conditions.

Finally, our results have implications for labor market policy, particularly as national borders decreas-

ingly restrict the flows of labour and services. Our estimates suggest that facilitating a reduction in employer

uncertainty over non-local experience (e.g., internationally recognized skills certification body) may reduce

transaction costs, encouraging more competition from LDC workers, and thus reduce DC wages.

71

2.7 Tables and Figures

Figure 2.1: Sample Distribution of Platform Experience

72

Figure 2.2: Likelihood of being hired for a job for LDC applicants, overall and by level ofplatform experience

Notes: The first column in the graph reports the average share of LDC applications per job. The second column displays theshare of LDC applicants that were hired overall; the third column reports the share of LDC applicants with low experiencethat were hired out of all those with low experience that were hired, and the fourth column reports the share of LDCapplicants with high experience that were hired out of all those with high experience that were hired.

73

Figure 2.3: Estimated Likelihood of Being Hired by Platform Experience (in quintiles) andLDC Status

Notes: This figure reports the estimated likelihood of being hired for DC and LDC applicants, as a function of theirexperience expressed in sample quantiles. The values for 0 previous jobs are taken from the summary statistics, and the valuesfor the other categories are estimated using the odd ratios in Table A8.

74

Table 2.1: Variable Names

75

Table 2.2: Contractor Descriptive Statistics

(1) (2) (3)Sample Full Sample DC Contractors LDC Contractors

Mean(SD) Mean(SD) Mean(SD)Median Median Median

Application Success (0/1) 0.035 0.087 0.027Contractor-LDC (0/1) 0.867Number of Prior oDesk Contracts 13.066 12.868 13.096

(25.612) (18.261) (25.181)4 3 4

High Platform Experience (0/1) 0.482 0.439 0.488Off Platform Work Experience (0/1) 0.321 0.390 0.310Education 1.196 0.916 1.237

(1.182) (1.174) (1.178)1 0 2

Current non-oDesk Employment Status (0/1) 0.517 0.559 0.510Average oDesk Test Score (0/1) 0.499 0.682 0.473Number of oDesk Tests (0/1) 0.409 0.393 0.412Wage Bid 8.242 16.871 7.147

(13.418) (17.764) (12.338)5.56 13.33 4.44

Fixed Price Bid 10.177 17.736 8.690(16.286) (15.872) (15.950)7.78 13.89 6.67

Profile Wage 6.490 14.756 5.226(19.797) (15.040) (20.130)

4 12 3Profile Picture (0/1) 0.838 0.811 0.842Agency Membership (0/1) 0.232 0.097 0.253Employer Initiated Application (0/1) 0.075 0.172 0.060oDesk Rating Score 3.177 3.124 3.185

(2.229) (2.309) (2.217)4.7 5 4.7

No rating Score (0/1) 0.695 0.662 0.700Previously Hired by Employer (0/1) 0.005 0.013 0.003Interviewed (0/1) 0.111 0.188 0.099Short Listed (0/1) 0.038 0.050 0.036Fraction of Cover Letter that is Original 0.301 0.479 0.276

(0.346) (0.371) (0.334)0.143 0.5 0.111

Number of Observations 420,833 55,912 364,921

Notes: This table reports summary statistics at the applicant-job level. For variables that are not binary, the standard deviation andthe median are reported in addition to the mean.

76

Table 2.3: Job and Employer Descriptive Statistics

Mean (SD)Median

Number of Prior Hires on oDesk 16.346(46.002)

4Job Type: (0/1)Administrative Services 0.100Business Services 0.030Customer Services 0.008Design & Multimedia 0Networks & Information Systems 0Sales & Marketing 0.089Software Development 0.069Web Development 0.280Writing & Translation 0.193Number of Interviews 3.214

(4.857)2

Fixed Price Contract (0/1) 0.508Job Budget 172.882

(947.152)50

Final Amount Paid 463.197(1979.873)

52.22Number of Applicants 29.005

(44.385)18

Number of Observations 14,617

Notes: This table reports characteristics at the employer-job level. For variables that are not binary, the standard deviation and themedian are reported in addition to the mean. Employers indicate how big the budget is for a job only if the job offers a fixed pricecontract. Only jobs that were completed during our period of observation have a final amount paid observation.

77

Table 2.4: LDC Status and oDesk Experience

(1) (2) (3)Model Logit Logit Conditional LogitOutcome Applicant success Applicant success Applicant successOverall Mean 0.035 0.035 0.035Mean for DC contractors 0.084 0.084 0.084

Raw est. Odds ratios Raw est. Odds ratio Raw est. Odds ratio

LDC -1.172*** 0.310 -0.580*** 0.560 -0.518*** 0.596(0.0217) (0.0253) (0.0264)

Platform Experience 0.751*** 2.119 0.450*** 1.569 0.464*** 1.590(0.0205) (0.0255) (0.0268)

Controls Yes YesJob FE YesObservations 356,480 356,480 356,480Chi test 3982 8095 5092Number of groups 12,508 12,508 12,508

Notes: The sample is restricted to jobs posted by employers from DCs and jobs for which one contractor was hired. Standard errorsclustered at the job level are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

Table 2.5: Differential Impact of Platform Specific Experience for LDC Contractors

Outcome Applicant successOverall Mean 0.035Mean for DC contractors with low oDesk experience 0.067

Raw est. Odds ratio

LDC -0.718*** 0.488(0.0373)

Platform Experience 0.209*** 1.233(0.0442)

LDC X Platform Experience 0.346*** 1.413(0.0475)

Controls YesJob FE YesObservations 356,480Chi test 5069Number of groups/cases 12,508

Notes: Results are from a conditional logit regression. The sample is restricted to jobs posted by employers from DCs and jobs forwhich one contractor was hired. Standard errors clustered at the job level are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

78

Table 2.6: Differential Impact of Platform Specific Experience by Job Type

(1) (2) (3) (4) (5)Sample Administrative Jobs Web Development Jobs Writing Jobs Software Development Jobs Marketing JobsOutcome Applicant success Applicant success Applicant success Applicant success Applicant successOverall Mean 0.014 0.042 0.063 0.056 0.025Mean for DC contractorswith low oDesk experience 0.031 0.081 0.093 0.078 0.083

Raw est. Odds ratios Raw est. Odds ratio Raw est. Odds ratio Raw est. Odds ratio Raw est. Odds ratio

LDC -1.087*** 0.337 -0.551*** 0.576 -0.895*** 0.409 -0.359*** 0.698 -0.812*** 0.444(0.116) (0.0777) (0.0829) (0.137) (0.139)

Platform Experience 0.341** 1.406 0.202** 1.223 0.127 1.136 0.363** 1.438 0.227 1.255(0.155) (0.0963) (0.0836) (0.175) (0.172)

LDC X Platform Experience 0.339** 1.403 0.297*** 1.345 0.478*** 1.613 -0.0566 0.945 0.426** 1.532(0.159) (0.102) (0.101) (0.186) (0.180)

Controls Yes Yes Yes Yes YesJob FE Yes Yes Yes Yes YesObservations 90,493 87,754 35,201 15,409 44,762Chi test 689.7 1575 1040 310.4 418.1Number of groups/cases 1,311 3,661 2,207 868 1,103

Notes: The sample is split by job type, limited to jobs with at least 500 postings in the final sample. The sample is restricted to jobs posted by employers from DCs and jobsfor which one contractor was hired. Standard errors clustered at the job level are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

79

Table 2.7: Differential Impact of Platform Specific Experience for LDC Contractors on Wages

(1) (2) (3) (4) (5) (6) (7) (8)Sample Hourly Wage Hourly Wage Hourly Wage Hourly Wage Jobs, Fixed Price Fixed Price Fixed Price Fixed Price Jobs,

Jobs Jobs Jobs Hired Contractors Jobs Jobs Jobs Hired ContractorsOutcome Log(Wage Bid) Log(Wage Bid) Log(Wage Bid) Log(Wage Bid) Log(Fixed Price Log(Fixed Price Log(Fixed Price Log(Fixed Price

Bid) Bid) Bid) Bid)Overall Mean 7.908 7.908 7.908 11.321 9.682 9.682 9.682 13.187Mean for DC contractorswith low oDesk experience 14.726 14.726 14.726 15.982 16.129 16.129 16.129 18.334

LDC -0.833*** -0.784*** -0.502*** -0.687*** -0.746*** -0.710*** -0.568*** -0.607***(0.013) (0.011) (0.0068) (0.035) (0.010) (0.010) (0.00759) (0.030)

Platform Experience 0.288*** 0.170*** 0.0888*** 0.196*** 0.195*** 0.142*** 0.114*** 0.124***(0.013) (0.013) (0.009) (0.040) (0.011) (0.011) (0.010) (0.032)

LDC X Platform Experience 0.0644*** 0.0660*** 0.0607*** 0.0508 0.133*** 0.123*** 0.0886*** 0.0686*(0.013) (0.012) (0.009) (0.043) (0.011) (0.011) (0.010) (0.036)

Controls Yes Yes Yes Yes Yes YesJob FE Yes Yes Yes YesObservations 221,943 221,943 221,943 6,182 134,537 134,537 134,537 6,326R-Squared 0.134 0.214 0.701 0.219 0.160 0.190 0.469 0.196Number of groups 6,182 6,182 6,182 6,182 6,326 6,326 6,326 6,326

Notes: This table reports estimates from linear regressions. Standard errors are clustered by jobs are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

80

Table 2.8: Differential Impact of Platform Specific Experience by Employer Experience

(1) (2) (3) (4)Sample No Prior Hires Prior Hires> 0 Below Median Prior Hires Above Median Prior Hires

(Prior Hires≤5) (Prior Hires>5)Outcome Applicant success Applicant success Applicant success Applicant successOverall Mean 0.043 0.033 0.038 0.032Mean for DC contractors with low oDesk experience 0.071 0.065 0.069 0.064

Raw est. Odds ratios Raw est. Odds ratios Raw est. Odds ratio Raw est. Odds ratio

LDC -0.656*** 0.519 -0.744*** 0.475 -0.692*** 0.500 -0.753*** 0.471(0.0680) (0.0446) (0.0499) (0.0562)

Platform Experience 0.312*** 1.366 0.165*** 1.179 0.230*** 1.259 0.184*** 1.202(0.0796) (0.0532) (0.0582) (0.0680)

LDC X Platform Experience 0.204** 1.227 0.406*** 1.501 0.297*** 1.346 0.409*** 1.505(0.0857) (0.0571) (0.0629) (0.0724)

Controls Yes Yes Yes YesJob FE Yes Yes Yes YesObservations 87,274 269,206 185,432 171,048Chi test 1354 3729 2616 2421Number of groups 3,713 8,795 7,060 5,448

Notes: The sample is split by employer experience level: we first distinguish employer with no previous experience from those with experience, and then between employersbelow and above the median level of previous experience as given by prior hires. The median number of prior hires in our sample is 5. Results are from conditional logitregressions. The sample is restricted to jobs posted by employers from DCs and jobs for which one contractor was hired. Standard errors clustered at the job level are reportedin parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.0181

Table 2.9: Differential Impact of Platform-Specific Experience by Contract Type

(1) (2)Sample Hourly Contracts Fixed Price ContractsOutcome Applicant success Applicant successOverall Mean 0.028 0.047Mean for DC contractors with low oDesk experience 0.051 0.084

Raw est. Odds ratios Raw est. Odds ratio

LDC -0.730*** 0.482 -0.713*** 0.490(0.0570) (0.0502)

Platform Experience 0.229*** 1.258 0.197*** 1.218(0.0686) (0.0585)

LDC X Platform Experience 0.293*** 1.340 0.412*** 1.510(0.0717) (0.0646)

Controls Yes YesJob FE Yes YesObservations 221,943 134,537Chi test 2843 2578Number of groups 6,182 6,326

Notes: The sample is split by contract type: hourly vs. fixed price. Results are from conditional logit regressions. The sample isrestricted to jobs posted by employers from DCs and jobs for which one contractor was hired. Standard errors clustered at the joblevel are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

82

2.8 Appendix

2.8.1 A Simple Theoretical Framework

Suppose that an applicant contract worker can be of two quality levels θ : θ and 0, where θ > 0. Assume

the quality is positive with probability β > 12 and 0 with the complementary probability 1 − β. Therefore,

the ex-ante expected quality of a worker is βθ. More precisely, the prior on β held by prospective employers

varies according to the origin of the applicants. It is higher for DC applicants than for LDC applicants.

Note also the variance of the ability level is a function of β. In particular, the highest variance is when

β = 12 . Given our restriction on the values of β, the quality of LDC applicants is expected to be lower and

also more uncertain. Assuming β ≥ 12 (rather than not constraining the value within any interval) can be

justified on a number of grounds. First, note in this binomial case the highest variance is reached when

β ≥ 12 . Therefore, if we believe that an issue with hiring LDCs is uncertainty, it makes sense to have the

LDCs closer to the level of β that is higher. Second, as also described below, one could think of β already

incorporating information about an applicant’s quality, and one can argue this leads to a first screening

where employers keep only contractors for whom they expect positive quality with a higher probability than

the toss of a coin. Suppose now there is a signal, y, which the employer may observe. She is more likely to

observe the signal if the applicant is of high quality. Formally:

Prob(y = 1θ = θ) = α1;

Prob(y = 1θ = 0) = α;

where 0 < α < α1 < 1. The difference between α and α1 can be considered to be the degree of informativeness

of the signal. If the two probabilities are the same, then the signal does not tell the employer anything more

about the quality of an applicant. Upon observing the signal, the employer updates her beliefs about the

quality of each applicant. In particular:

Prob(θ = θ|y = 1) = α1βα1β+α(1−β)

83

Note that, if α1 = α, then Prob(θ = θ|y = 1) = β, so there is no updating. Therefore, the change ∆ in the

expected quality of an applicant is given by:

∆ = α1βα1β+α(1−β) θ − βθ = (α1−α)(1−β)

α+β(α1−α) βθ

Now, the sign of δ∆δβ is opposite to the sign of the following expression:

(α1 − α)β2 + 2αβ − α

This expression is always positive for β ≥ 12 ; therefore ∆ is decreasing in β if β ≥ 1

2 . The increment

in expected quality upon observing the signal y is greater for applicants with ex-ante lower expected (and

more uncertain) quality.18 Note that the relative increase in expected quality can be expressed by ∆βθ

=

(α1−α)(1−β)α+β(α1−α) , which is, again, decreasing in β for all β.

18The differential impact of the signal can also be obtained from a framework where we assume an equal prior expected

quality, but different variability according to the origin of workers. For example, consider a case of three levels of quality: θ,

θ, and 0 where θ > θ > 0. For simplicity (and with some loss of generality), assume that the highest quality level is twice the

mid-level quality: θ = 2θ. The probability distribution over quality is such that:

θ =

0 with prob

(1−β)2

θ with probβ

θ with prob(1−β)

2

Assume β ≥ 13(so the distribution is single peaked). Note that E(θ) = θ regardless of the value of β, and that the variance if

this probability distribution is decreasing in β for β ≥ 13. Suppose that the quality signals is such that:

Prob(y = 1|θ = θ) = α

Prob(y = 1|θ = θ) = α

Prob(y = 1|θ = 0) = α0 = 0

where α > α. In this case, it can be shown that the change in the expected quality of an applicant is given by:

∆ =2α−2(α−α)βα−(α−2α)β

θ − θ

and this is decreasing in β. Equivalently, the change in expected value is higher when β is smaller, i.e. when the variance ishigher.

84

2.8.2 Additional Tables

Table 2.10: Descriptive Statistics Comparing Jobs Included and Dropped from Sample

(1) (2) (3) (4)Sample Our Sample Full Sample Multiple Hires LDC or DC Applicants Only

Mean (SD) Mean (SD) Mean (SD) Mean (SD)Median Median Median Median

Number of Prior Hires on oDesk 16.346 23.446 24.396 22.702(46.002) (87.713) (78.728) (154.346)

4 7 8 5Job Type: (0/1)Administrative Services 0.100 0.302 0.175 0.080Business Services 0.030 0.024 0.031 0.022Customer Services 0.008 0.020 0.016 0.005Design & Multimedia 0 0 0 0Networks & Information Systems 0 0 0 0Sales & Marketing 0.089 0.141 0.122 0.125Software Development 0.069 0.037 0.040 0.068Web Development 0.280 0.200 0.186 0.348Writing & Translation 0.193 0.115 0.257 0.144Number of Interviews 3.214 12.358 8.370 1.549

(4.857) (42.511) (19.639) (1.969)2 4 5 1

Fixed Price Contract (0/1) 0.508 0.345 0.040 0.525Job Budget 172.882 1296.6 5158.429 246.775

(947.152) (105813.1) (221791.9) (10220.15)50 45 30 40

Final Amount Paid 458.931 312.901 515.269 540.097(1917.097) (1428.755) (2906.188) (2544.201)53.675 23.865 20 55.55

Number of Applicants 29.005 138.685 50.735 5.200(44.385) (216.395) (89.479) (10.797)

18 59 26 2Number of Observations 14,617 672,677 5,051 18,337

Notes: Employers indicate how big the budget is for a job only if the job offers a fixed price contract. Only jobs that were completedduring our period of observation have a final amount paid observation.

85

Table 2.11: Contractor Descriptive Statistics for Sample Including Jobs with Multiple Hires

(1) (2) (3)Sample Full Sample DC Contractors LDC Contractors

Mean (SD) Mean (SD) Mean (SD)Median Median Median

Applicant Success (0/1) 0.049 0.097 0.041Contractor-LDC (0/1) 0.856Number of Prior oDesk Contracts 12.163 10.915 12.372

(24.467) (26.085) (24.179)4 2 4

High Platform Experience (0/1) 0.460 0.439 0.488Off Platform Work Experience (0/1) 0.320 0.390 0.310Education 1.168 0.837 1.222

(1.173) (1.145) (1.169)1 0 2

Current non-oDesk Employment Status (0/1) 0.502 0.516 0.500Average oDesk Test Score (0/1) 0.500 0.680 0.472Number of oDesk Tests (0/1) 0.406 0.370 0.412Wage Bid 7.457 15.206 6.468

(14.543) (17.537) (13.837)4.44 11.11 3.7

Fixed Price Bid 9.769 15.786 8.269(16.866) (17.537) (16.353)

7 11.11 5.56Profile Wage 6.125 13.389 4.910

(17.555) (16.780) (17.288)3.33 10 3

Profile Picture (0/1) 0.843 0.780 0.853Agency Membership (0/1) 0.219 0.088 0.241Employer Initiated Application (0/1) 0.106 0.294 0.075oDesk Rating Score 3.074 2.804 3.119

(2.259) (2.376) (2.236)4.6 4.5 4.6

No Rating Score (0/1) 0.673 0.597 0.686Previously Hired by Employer (0/1) 0.005 0.012 0.004Interviewed (0/1) 0.130 0.199 0.118Shortlisted (0/1) 0.036 0.042 0.035Fraction of Cover Letter that is Original 0.305 0.496 0.279

(0.349) (0.375) (0.337)0.143 0.5 0.111

Number of Observations 672,677 96,559 576,118

86

Table 2.12: Contractor Descriptive Statistics across Employer Experience Levels

(1) (2) (3) (4)Sample No Prior Hires Prior Hires>0 Below Median Above Median

Prior Hires Prior HiresMean (SD) Mean (SD) Mean (SD) Mean (SD)Median Median Median Median

Applicant Success (0/1) 0.043 0.033 0.038 0.032Contractor-LDC (0/1) 0.871 0.892 0.882 0.891Number of Prior oDesk Contracts 13.740 12.877 13.422 12.726

(26.540) (25.391) (25.147) (25.159)4 4 4 4

High Platform Experience (0/1) 0.494 0.483 0.490 0.481Off Platform Work Experience (0/1) 0.317 0.317 0.315 0.319Education 1.213 1.218 1.216 1.218

(1.192) (1.180) (1.188) (1.177)1 1 1 1

Current non-oDesk Employment Status (0/1) 0.522 0.520 0.521 0.519Average oDesk Test Score (0/1) 0.500 0.494 0.495 0.496Number of oDesk Tests (0/1) 0.409 0.418 0.411 0.421Wage Bid 8.676 7.680 8.191 7.613

(12.152) (11.745) (12.129) (11.538)5.56 5 5.56 5

Fixed Price Bid 10.225 9.480 9.943 9.379(13.296) (14.175) (12.692) (15.269)

8 7 7.78 6.67Profile Wage 6.397 5.926 6.150 5.924

(11.158) (10.828) (10.602) (11.237)4 3.15 4 3.15

Profile Picture (0/1) 0.828 0.845 0.835 0.848Agency Membership (0/1) 0.253 0.241 0.25 0.237Employer Initiated Application (0/1) 0.038 0.031 0.035 0.030oDesk Rating Score 3.236 3.226 3.235 3.221

(2.205) (2.208) (2.204) (2.210)4.8 4.8 4.8 4.774

No Rating Score (0/1) 0.709 0.707 0.709 0.705Previously Hired by Employer (0/1) 0.000 0.004 0.001 0.005Interviewed (0/1) 0.137 0.108 0.125 0.104Shortlisted (0/1) 0.060 0.035 0.052 0.029Fraction of Cover Letter that is Original 0.289 0.285 0.283 0.289

(0.337) (0.335) (0.335) (0.336)0.125 0.125 0.125 0.128

Number of Observations 87,274 269,206 185,432 171,048

87

Table 2.13: LDC Status and oDesk Experience Control Coefficients

(1) (2)Model Logit Conditional LogitOutcome Applicant success Applicant successOverall Mean 0.035 0.035Mean for DC contractors 0.084 0.084

Raw est. Odds ratios Raw est. Odds ratio

LDC -0.580*** 0.560 -0.518*** 0.596(0.0253) (0.0264)

Platform Experience 0.450*** 1.569 0.464*** 1.590(0.0255) (0.0268)

Off Platform Work Experience (0/1) -0.0119 0.988 0.0758*** 1.079(0.0220) (0.0234)

Fraction of Cover Letter that is Original 0.810*** 2.248 0.734*** 2.084(0.0281) (0.0302)

Profile Picture 0.214*** 1.238 0.284*** 1.328(0.0285) (0.0298)

oDesk Rating Score 0.128*** 1.137 0.145*** 1.156(0.0150) (0.0155)

No oDesk Rating Score -0.0856 0.918 -0.235*** 0.790(0.0777) (0.0797)

Log(Wage Bid) 0.323*** 1.382 -0.0350* 0.966(0.0151) (0.0194)

Average oDesk Test Score 0.225*** 1.252 0.262*** 1.299(0.0199) (0.0212)

Number of oDesk Test Scores 0.0672*** 1.069 0.0999*** 1.105(0.0204) (0.0217)

Agency Member -0.270*** 0.763 -0.240*** 0.787(0.0256) (0.0283)

Education -0.0595*** 0.942 -0.0462*** 0.955(0.00915) (0.00947)

Current Offline Employment Status -0.0325 0.968 -0.0648*** 0.937(0.0208) (0.0219)

Employer Initiated Application 1.203*** 3.331 1.709*** 5.521(0.0379) (0.0555)

Prior Hire 2.340*** 10.38 2.183*** 8.872(0.0742) (0.0911)

Constant -4.755*** 0.00861(0.0634)

Observations 356,480 356,480Chi test 8095 5092Number of groups/cases 12,508 12,508

Notes: The sample is restricted to jobs posted by employers from DCs and jobs for which one contractor was hired. Robust standarderrors are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

88

Table 2.14: Differential Impact of Platform Specific Experience for LDC Contractors onInterviews & on being Short Listed

(1) (2)Model Interviewed Short ListedOutcome Applicant success Applicant successOverall Mean 0.034 0.027Mean for DC contractors 0.063 0.05

Raw est. Odds ratios Raw est. Odds ratio

LDC -0.562*** 0.570 -0.499*** 0.607(0.0317) (0.0480)

Platform Experience 0.134*** 1.143 0.329*** 1.389(0.0396) (0.0550)

LDC X Platform Experience 0.305*** 1.357 0.205*** 1.228(0.0418) (0.0584)

Controls Yes YesJob FE Yes YesObservations 304,768 132,201Chi test 5006 61639Number of groups/cases 10,416 13,631

Notes: The specification is the same as in column 3 of Table 2.5, with two different outcome variables: being interviewed, and beingshort-listed.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

Table 2.15: Differential Impact of Platform Specific Experience for LDC Contractors withFull Sample

Outcome Applicant successOverall Mean 0.046Mean for DC contractors 0.067

Raw est. Odds ratios

LDC -0.672*** 0.511(0.0288)

Platform Experience 0.177*** 1.193(0.0342)

LDC X Platform Experience 0.290*** 1.337(0.0363)

Controls YesJob FE YesObservations 563,117Chi test 5150Number of groups/cases 17,099

Notes: standard errors are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

89

Table 2.16: Excluding Applicants Previously Hired by Employer & Employer Initiatedapplicants

(1) (2) (3)Sample No Prior Hires No Employer No Prior Hires or

Initiated Applications Employer Initiated ApplicationsOutcome Applicant success Applicant success Applicant successOverall Mean 0.036 0.033 0.034Mean for DC contractors 0.068 0.067 0.069

Raw est. Odds ratios Raw est. Odds ratio Raw est. Odds ratio

LDC -0.716*** 0.489 -0.796*** 0.451 -0.796*** 0.451(0.0385) (0.0440) (0.0450)

Platform Experience 0.229*** 1.257 0.257*** 1.293 0.254*** 1.289(0.0454) (0.0523) (0.0534)

LDC X Platform Experience 0.333*** 1.396 0.343*** 1.409 0.343*** 1.410(0.0488) (0.0563) (0.0576)

Controls Yes Yes YesJob FE Yes Yes YesObservations 319,312 245,500 224,503Chi test 3932 2732 2042Number of groups 11,454 8,134 7,619

Notes: Results are from conditional logit regressions. Sample is restricted to jobs posted by employers from DCs and jobs for whichone contractor was hired. Robust standard errors are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

90

Table 2.17: Robustness to Contractor oDesk Experience Measure

(1) (2) (3)Sample Full Sample Employers with 0 Employers with > 0

Prior Hires Prior HiresOutcome Applicant success Applicant success Applicant successOverall Mean 0.035 0.043 0.033Mean for DC contractors 0.097 0.112 0.092

Raw est. Odds ratios Raw est. Odds ratio Raw est. Odds ratio

LDC -0.834*** 0.434 -0.696*** 0.499 -0.894*** 0.409(0.0529) (0.0978) (0.0629)

Platform Experience in 0.319*** 1.376 0.436*** 1.547 0.267*** 1.3072nd Quintile (0.0736) (0.134) (0.0881)Platform Experience in 0.393*** 1.482 0.502*** 1.652 0.348*** 1.4163rd Quintile (0.0747) (0.138) (0.0887)Platform Experience in 0.532*** 1.702 0.753*** 2.123 0.435*** 1.5444th Quintile (0.0767) (0.140) (0.0919)Platform Experience in 0.680*** 1.974 0.860*** 2.363 0.603*** 1.8275th Quintile (0.0756) (0.137) (0.0906)LDC*Platform Experience in 0.0849 1.089 -0.108 0.898 0.167* 1.1822nd Quintile (0.0807) (0.148) (0.0963)LDC*Platform Experience in 0.319*** 1.376 0.213 1.237 0.365*** 1.4413rd Quintile (0.0730) (0.134) (0.0871)LDC*Platform Experience in 0.416*** 1.517 0.155 1.168 0.530*** 1.6994th Quintile (0.0729) (0.132) (0.0875)LDC*Platform Experience in 0.511*** 1.667 0.285** 1.330 0.609*** 1.8385th Quintile (0.0704) (0.127) (0.0846)Controls Yes Yes YesJob FE Yes Yes YesObservations 326,480 87,274 269,206Chi test 5343 1430 3932Number of groups 12,508 3,713 8,795

Notes: Results are from conditional logit regressions. Sample is restricted to jobs posted by employers from DCs and jobs for whichone contractor was hired. Robust standard errors are reported in parentheses.∗p < 0.10 ∗ ∗p < 0.05 ∗ ∗ ∗ p < 0.01

91

Chapter 3

Digitization and the Contract LaborMarket: A Research Agenda

with Ajay Agrawal, John Horton & Nicola Lacetera

3.1 Introduction

We motivate this chapter on the digitization of the market for contract labor with three observations. First,

this market is growing rapidly in terms of the number and variety of participants and transactions. Second,

in contrast to the highly localized exchange of services typical in the traditional offline market for contract

work, the online market is dominated by long distance North-South (as defined below) trade. Third, the

online platforms that facilitate trade in this market introduce seemingly small informational frictions that

have significant effects on outcomes. We describe each of these market features in turn.

The growth of online markets for contract labor has been fast and steady. According to Horton (2010),

workers in this market earned about $700 million by 2009, and the Financial Times (2012) estimated this

market to be worth $1 billion annually by the end of 2012. Additional details from oDesk, the largest

online marketplace for contract labor in terms of earnings, provide further insight into the growth of this

market. The number of employers billing on the site per quarter increased by over 800% between 2009 and

2013 (Figure 3.1), and the number of working contractors per quarter increased by approximately 1,000%

over the same period (Figure 3.3). In pecuniary terms, the quarterly wage bill on oDesk increased by

approximately 900%, from $10,000,000 to almost $100,000,000 over the same four-year period (Figure 3.2).

North-South exchange dominates the pattern of trade in these markets (e.g., relative to North-North,

92

South-South, and South-North). In other words, employers are predominantly from high-income countries1

and contractors are mainly from lower-income countries. We classify countries as “high-income” using the

2012 World Bank list of high-income countries. We classify the remaining countries as “lower-income.” In

Figure 1 we illustrate that not only are there more employers on oDesk from high- compared to lower-income

countries, but the number from high-income countries is also growing at a faster rate. Similarly, the wage bill

per quarter is significantly greater and growing faster for employers from high- versus lower-income countries

(Figure 3.2). While the contrast is not quite as extreme on the contractor side of the market (a significant

number of contractors are from high-income countries), there were approximately three times as many lower

versus high-income contractors in 2009 and that difference increased to five times by 2013 (Figure 3.3). This

does not simply reflect a growing volume of small jobs performed by contractors from lower-income countries.

In Figure 3.4 we illustrate that the wage bill reflects a similar pattern in terms of contractors from high-

versus lower-income countries.

A number of studies examine how seemingly small information frictions may significantly influence

matching outcomes in online markets for contract labor. Perhaps the most dramatic finding is the one

reported by Pallais (2012). In this study, Pallais conducts a field experiment where she “treats” 952 randomly

selected contractors by hiring them and then providing feedback on their performance. Then she compares

the subsequent employment performance of these treated contractors with a set of 2,815 other contractors

(“controls”) who applied for her posted jobs but whom she did not hire and therefore did not post information

on. She reports that, for those with no prior work experience on oDesk, the subsequent income of treated

contractors almost triples relative to the income of control contractors over the following two-month period.

She then takes a number of steps to provide further evidence that the observed increase in employment

performance is due to the information she posted to the platform about the contractor (i.e., rating and

feedback), rather than due to other explanations such as human capital accumulation by the contractor due

to the experience of doing the job. The reason this result is so dramatic is because the treatment is so small:

the job is only a 10-hour data entry task, the rating is only a single score out of five, and the feedback is

only a single sentence: “It was a pleasure working with [x].” In fact, for inexperienced workers, the marginal

effect of a more detailed comment that specifies data entry speed, accuracy, following of directions, and

timely task completion is not statistically distinct from zero. In other words, the trebling of income is caused

1We define high-income countries according to the World Bank classification available at http://data.worldbank.org/income-level/HIC.

93

by minimal information provided by the employer based on a remarkably small job. Although the observed

effect is based on low-wage data-entry specialists who propose wages of $3 per hour or less, the effect of such

a seemingly small amount of information is striking. It points to an important market friction present in this

online setting. The author draws a welfare implication from her finding: “Under plausible assumptions, the

experiment’s market-level benefits exceeded its cost, suggesting that some experimental workers had been

inefficiently unemployed.”

Similar information frictions are reported in other studies in this market setting. Stanton and Thomas

(2012) estimate the effect of information from intermediaries on contractor employment. They find that

inexperienced contractors affiliated with an intermediary have substantially higher job-finding probabilities

(almost double) and wages (15%) at the beginning of their careers on oDesk. Agrawal, Lacetera, and Lyons

(2013) examine the relative role of information about experience on oDesk for contractors from high- versus

low income countries. They find that information about platform-based work experience disproportionately

benefits contractors from low-income countries (approximately 40% premium). In a related study, Mill (2011)

finds that once an employer on Freelancer has a good experience with a contractor from a particular country,

then the employer is more likely to hire someone else from that country. Also related, Ghani, Kerr, and

Stanton (2012) report that members of the Indian diaspora hiring on oDesk are more likely to hire workers

in India than are other employers. Finally, Horton (2012) finds that recommendations increase the likelihood

of a hire in job categories with fewer qualified candidates. In each of these cases, seemingly small amounts

of information have significant effects on employment outcomes, suggesting that information frictions play

an important role in the matching process online.

With these three market features in mind - rapid growth, North-South trade, and sensitivity to information-

based frictions - we turn to analyzing the basic economics of online markets for contract labor in Section 3.2.

In doing so, we consider the characteristics of both the demand and supply sides, stressing the incentives

that lead employers as well as contractors to utilize this channel. The main trade-off that we consider is

between the reduction in search, communication, monitoring, and transportation costs on the one hand and

the potential for new sources of information-related frictions to arise on the other. We then describe the

role that online contract labor platforms play in facilitating matches between demand and supply and in

addressing some of these trade-offs. Again, we use evidence from oDesk to provide an in-depth illustration.

Drawing on these insights regarding the basic economic properties of online markets for contract labor,

94

we outline a research agenda predicated on three lines of inquiry. These include: 1) distributional effects, 2)

market design, and 3) welfare. We describe each in turn.

In Section 3.3, we ask: How will the digitization of this market influence the distribution of economic

activity? We consider distribution along three dimensions. First, we contemplate the distribution of work

across geographies. Will digitization shift the distribution of contract work towards lower-income coun-

tries? Second, we question the distribution of income within and across countries. Will digitization further

accentuate income inequality by amplifying superstar-type distributions whereby only a small fraction of

contractors capture a large fraction of rents (although some of these individuals may be in lower-income

countries)? Finally, we raise the question of outsourcing. Will digitization lead to a shift in the distribution

of work across firm boundaries, constricting the boundary of the firm due to a lowering cost of contracting

out discrete jobs? The answers to these lines of inquiry regarding distribution-related effects of digitization

will have important implications for understanding the effect of digitization on the overall organization of

work and thus implications for social welfare.

Next, in Section 3.4, we raise this question: How might market design features influence matching in

the digital setting? We describe above the impact of ratings and feedback, a market design feature common

across most platforms. In the digital setting, platforms can add or change market features at reasonably low

cost. However, the ease with which they can be added, deleted, or changed belies the influence they may

have on matching outcomes. While contracting platforms employ many interesting market design features,

we focus our discussion on five: 1) performance feedback (e.g., ratings), 2) machine-aided recommendations

(for employers and contractors), 3) the allocation of visibility, 4) pricing to reduce congestion, and 5) job

category specification. Although platforms in the online contract labor market do not have the match-setting

power that is typically analyzed in the market design literature (i.e., directly matching trading partners in

settings such as kidney exchanges and medical student-hospital matching), they do influence which matches

are ultimately formed and under what terms. Thus, as market design features evolve, so will the types of

matches they facilitate.

Finally, in Section 3.5, we ask: How will the digitization of this market affect social welfare? In particular,

we specify two channels through which digitization may generate efficiency gains: 1) better matching, and 2)

better production. With regards to matching, the shift from local to global search along with the utilization

of market design features enabled by the digitization of relevant information may lead to efficiency gains.

95

With regards to production, a reduction in coordination costs that enables more flexibility in terms of the

location and timing (asynchronous) of work as well as a finer division of labor due to the feasibility of

contracting out smaller jobs, which enables more specialization, may lead to efficiency gains. At the same

time, however, new frictions may lead to new forms of welfare losses.

We conclude by outlining three primary challenges to this research agenda. First, offline data for

this sector is costly to obtain but is required to estimate the causal effect of digitization on changes in

distributional properties (geography, income, firm boundaries) and welfare effects. Second, the economic

salience of particular market design features may be fleeting since the market is evolving quickly and subject

to rapid technological change. Finally, data ownership is concentrated among a few platforms that seem

interested in engaging with the research community but have interests that are not fully aligned. Despite

these challenges, this research agenda identifies opportunities to shed light on questions that are of first-order

importance from both a scholarly and economic relevancy perspective.

3.2 The economics of online contract labor markets

Like other digitized markets, the most salient features of online labor markets are the potential for a large

number of transactions and services to be provided by suppliers who may be geographically distant from

buyers. What are the implications for the demand and supply of services in this context? Who supplies labor

online? What entities search for online services, and what are the trade-offs they face? What institutions

contribute to clearing these markets? To address these questions, we begin by discussing how oDesk works.

This will frame the ensuing discussion on labor supply, labor demand, and market-making platforms.

3.2.1 Work Process on oDesk

To post jobs on oDesk, employers have to register on the site by giving their contact details and information

on their company, including name, owner, and location. Once registered, employers are free to post as many

jobs as they like. Job postings include a description of the task, the location of the employer, and the type of

contract being offered. oDesk supports two contract types—hourly wage and fixed price. Beyond the different

payment structures, the contracts have different implications for monitoring and duration specifications.

Specifically, when posting an hourly-wage job, employers have to specify the expected number of hours per

week and the number of weeks required to complete the job. They stipulate a limit on the number of hours

96

per week a contractor can work. When posting a fixed-price job, employers have to specify the budget and

deadline. Employers can make job postings public (so that any contractor can apply to them) or private (so

that only contractors they invite can apply to them).

To be hired on oDesk, workers similarly must register on the site by giving their contact details, name,

and location as well as by setting up a profile page. Profile pages are meant to advertise contractors to

potential employers and can include a description of skills, education, work experience outside of oDesk,

oDesk-administered test scores, certifications, whether or not they belong to an agency, and oDesk-specific

work histories and feedback scores. Once they have set up their profile pages, contractors can apply to jobs

by submitting cover letters and bids to job postings. A bid indicates the amount a contractor is willing to

be paid to work on a job.

Employers have the option to interview and negotiate over bids with applicants before hiring and to hire

as many contractors as they like. Once hired, contractors complete tasks remotely. Contractors submit their

work to employers online and are paid via oDesk. Employers have the option to give contractors bonuses

and can also reimburse expenses through oDesk.

After each job, employers give contractors a rating out of five based on six criteria: skills, quality,

availability, deadlines, communication, and cooperation. Each contractor also has an overall feedback score,

which is a job-size-weighted average of the individual scores. Contractors can provide their employers

feedback scores based on the same criteria; employers have a similarly constructed overall score. oDesk

provides this service in exchange for 10% of every transaction made on the site.

In addition to oDesk, Elance, Freelancer, and Guru are among the largest online contract labor markets.

Elance and Guru were both launched in 1999, followed by oDesk in 2005 and Freelancer in 2009. These sites

are similar in that they allow employers to find and hire short-term workers by registering on the platform

and posting jobs to attract applicants. Similarly, they all allow registered contract workers from around the

world to apply for jobs posted on the sites by bidding on them and to advertise themselves to employers

with profile pages. These platforms earn revenue by charging a percentage of each transaction or member

fees to workers and, in some cases, both. In addition to providing a (virtual) place for demand and supply

to meet and for the market to clear, these platforms have evolved over time toward addressing some of the

key challenges of labor markets in general and online labor markets in particular.

While the other major platforms in the industry share several features with oDesk, they differ on

97

certain dimensions. The primary variations lie in the services they provide participants. For instance, some

support contractor employment agencies while others do not, some offer guaranteed payment for hourly

wage contracts while others do not, and at least one of the major platforms does not have a virtual office

application. Perhaps the most significant difference concerns Freelancer, which supports both traditional

hiring and crowdsourcing. Given that crowdsourcing has different implications for matching and production,

findings from research based on traditional hiring may not generalize to crowdsourcing settings.

3.2.2 Labor supply

What are the incentives for individuals to supply labor online? One of the most important benefits to having

access to online contract labor markets, especially for individuals participating from lower-income countries

who are more constrained in terms of opportunities, is that these marketplaces dramatically increase the

pool of available jobs. In addition to increasing the number of opportunities, they also increase the likelihood

that contractors will find suitable matches for their skills and preferences.

Contractors also benefit from an increase in flexibility in this market setting. For the most part, these

transactions are contract-based: workers are not employees and therefore have more control over their

schedules and how they allocate time between the provision of these services and other activities (e.g.,

another job, family, leisure, etc. (The Economist, 2010)). In a survey of workers on oDesk, more than 80%

state that the flexibility and freedom associated with working on the site is a major benefit of online work.

Evidence also shows that the flexibility provided by telecommunication contributes to a significant increase

in female labor force participation (Dettling, 2012). Thus, these online marketplaces may induce women

previously out of the labor market to enter. Especially for contractors in the developing world, who make

up the vast majority of workers, easier access to job opportunities from entities in higher-income countries

might also imply higher earnings.

Some of the characteristics leading to benefits in participating in these markets may also be sources

of costs and risks for contractors. In particular, the contractual nature of these labor relations might lead

to more uncertainty about the duration and conditions of a work relationship. The dramatic increase in

participation in these markets and the typical profile of participants as relatively highly educated suggest

that on balance these markets represent viable and appealing opportunities for a large set of individuals.

98

3.2.3 Demand for contract labor

The online market for contract labor offers several benefits to employers relative to traditional offline markets.

It lowers the cost of search, communication, and transportation, which benefits trade in various services,

such as data entry, translation, and software development. This also enables access to a broader pool

of prospective workers with potentially more suitable skills and possibly at more competitive wage rates.

Although oDesk has a range of organization types and sizes that use the platform, the access to a large and

diverse pool of contract workers provided by these platforms is particularly unique for small, entrepreneurial

ventures. For instance, in a survey of employers using oDesk, more than half consider themselves start-ups.

However, the relative lack of face-to-face interactions might make it difficult for employers to extract

high-bandwidth information (Autor, 2001). Furthermore, the increased heterogeneity of applicants make

comparisons among them more challenging; for instance, comparing seemingly similar school degrees or job

experiences of applicants from different countries may be problematic, particularly for novice recruiters. In

addition to hidden-quality problems, an obvious issue for prospective employers is the difficulty in monitoring

and verifying effort from a distance and through an Internet-mediated transaction.

3.2.4 Market-making platforms

Consistent with other two-sided markets, intermediaries in online markets take actions to ensure the partici-

pation of both suppliers and buyers (Armstrong, 2006; Rysman, 2009). As mentioned above, a key challenge

in online contract labor transactions arises from the limited access to high-bandwidth information about

both applicants and employers (Autor, 2001). Online contract labor platforms are increasingly providing

features that attempt to solve these information problems.2 First, platforms provide a verification and

standardization device for some of this information; for example, although offline work experiences and edu-

cational attainments cannot be easily compared across individuals, especially if they come from very different

institutional and cultural contexts, employers can more easily compare work experience accumulated by con-

tractors on the platform (i.e., the number of jobs, duration, types, as well as performance as expressed by the

rating given by the employers and workers). This information is available in online contract labor markets

on contractor profiles, and platforms generally do not allow contractors to delete or block this information

from their profiles, thus reducing selectivity issues and increasing the reliability of these signals. Platforms

2Dellarocas (2006) provides a review of reputation systems designed to solve information problems in online markets.

99

also offer the possibility for applicants to perform standardized tests that offer some easy-to-assess quality

measures for prospective employers. Moreover, some platforms support contractor agencies or companies.

Contractors in an agency can cooperate to apply for and complete jobs on the site. Some evidence illustrates

that agencies help reduce information asymmetries (Stanton and Thomas, 2012).

In addition to providing quality information, online contract labor platforms also help solve challenges

relative to the observability and verifiability of effort, on both the worker’s and employer’s sides, through

various mechanisms. Direct monitoring is available on some platforms through virtual office applications3.

Contractors who perform their work while logged into these virtual offices are monitored through regular

screen shots and activity logs. To provide incentives for contractors to accept this degree of monitoring, some

platforms guarantee contractor payment for hourly wage work only if it is performed while logged into the

virtual office. Along with direct monitoring, workers’ ratings represent a potentially powerful reputational

mechanism for aligning their objectives with employer objectives.

Likewise and as in other online markets, moral hazard issues can arise on the part of employers (see,

for instance, Resnick and Zeckhauser (2002) and Cabral and Hortacsu (2010) for a discussion of moral

hazard in online markets). For example, employers could refuse to pay for work performed outside virtual

offices or to reimburse expenses. However, contract workers can rate their experience with an employer on

most platforms, thus reducing concerns about the risk of exploitative behavior and reneging on previous

agreements. Furthermore, both employers and contractors can file disputes if they feel they’ve been unjustly

charged or underpaid. Platforms act as mediators in these disputes and ultimately decide how they should

be resolved.

3.3 Digitization and the distribution of work

Keeping in mind the incentives and frictions facing employers and contractors that we described above,

we turn to contemplating how the digitization of this market may influence the distribution of work. We

consider and describe in turn distributional effects along three dimensions: geography, contractor income,

and firm boundaries.

3Evidence shows that strict monitoring is important for the success of working from home. Bloom et al. (2013) study aChinese travel agency that decided to try having some employees work from home. The study finds significant gains fromworking from home in terms of worker productivity and satisfaction. This may be partially a result of the firm’s carefulmonitoring of telecommuting workers. Dutcher and Jabs Saral (2012) highlight the difficulties that may arise if telecommutingworkers are not properly monitored by showing experimental evidence that non-telecommuting workers perceive that theirtelecommuting counterparts are shirkers.

100

3.3.1 Geographic distribution

The reduction in search, communication, and monitoring costs brought by the digitization of contract labor

markets raises the possibility of improving employer-contractor matching and thus enhancing gains from

trade. A consequence of this is a potential impact on the geographic distribution of work. Perhaps the most

immediate and dramatic gains are those based on cross-region wage variation. Indeed, the dramatic growth

in activity on oDesk seems to be primarily of this nature. Specifically, employers in high-income countries

hire contractors from low-income countries, even for small jobs that were previously infeasible offline due to

transaction costs. As reported in Figure 3.1, not only were there more than 10 times as many employers

from high compared to lower-income countries by late 2012, but the growth rate of employers from high-

income countries was much higher than that from lower-income countries. The gap was even greater when

expressed in terms of the wage bill rather than the number of employees (Figure 3.2). Conversely, by 2013,

approximately 4.5 times as many contractors were from lower- compared to high-income countries (Figure

3.3). The trends so far suggest that the spread will continue to increase over time since the number of

contractors from lower-income countries is growing at a faster rate. Figure 3.4 confirms this trend also exists

in terms of the total monthly wage bill, not just the number of contractors, despite the fact that, as one

might expect, wages are higher for contractors in more developed countries.

Although access to lower-cost labor is one reason for recruiting distant contractors, employers report

other reasons as well. In a survey of its users conducted by oDesk, 76% indicated that “remote is less expen-

sive” was a primary reason they were interested in using the platform. However, 46% selected “can get work

done faster remotely,” 31% selected “difficult to find local talent,” and 21% selected “no room/equipment.”

Thus, in addition to the reduced cost of accessing lower-wage workers, enhanced matching seems to benefit

from gains on multiple dimensions.

Countries vary in terms of their level of participation in online contract labor markets. For example,

on oDesk approximately 10 times as many contractors are from the Ukraine as from Spain, even though

the two countries are similar in size (populations in 2013: Ukraine 45m; Spain 47m; however, Spain’s

economy is approximately 10 times larger: 1.4 trillion USD compared to 0.165 trillion USD for the Ukraine).

We illustrate this in Figure 3.5, where we plot the number of contractors on oDesk per country against

population. Nations such as Mexico, Brazil, and China appear to be under-users (participation below what

their population would predict), whereas the Philippines, Bangladesh, and India appear to be over-users.

101

The variation in usage of this digital marketplace may simply reflect offline employment opportunities.

oDesk contractors from Bangladesh and the Philippines, for example, earn significantly more than local

minimum wages, perhaps partly explaining their disproportionate use of the platform. However, contractors

from China also earn significantly above the local minimum wage on average yet under-use the platform

relative to other nations. Furthermore, contractors from several countries, like Australia, earn only slightly

more than the local minimum wage, on average, and yet seem to be over-users. This variation reflects the

relative benefits and costs, including opportunity costs, faced by the labor force in each country. Factors such

as proficiency in English (the language used on the site), Internet access, and education levels all affect the

returns to engaging with a digitized labor market platform such as this. As these online markets grow, they

will provide researchers with useful data to better understand offline employment opportunities (particularly

where reliable government data is sparse) and the relative returns to different forms of education in a global

work environment. In addition, they will provide a setting for further analysis on the extent to which

geographic, language, cultural, and other forms of distance influence flows of trade in labor.

The different composition of online contract workers across countries may also explain the unexpectedly

high average wages received by contractors in certain countries, such as China, Poland, and Russia, as

reported in Figure 3.7. Contractors from these three countries in particular are primarily concentrated

in software development, information systems, and web development, which offer higher wages on average

than most other types of work on oDesk: by 2013, the average wage in software development ($16) was

approximately double that of writing and translation ($8) and more than triple that of administrative

support ($4) as well as customer support ($5) and sales and marketing ($5) (Figure 3.8). Furthermore, the

quarterly spend in software development and web development is significantly greater than in any other

category (Figure 3.9). We plot the concentration of total contractor wage bill by country over time in Figure

3.10. Russia and Ukraine stand out as especially concentrated in only a few sectors (software development in

particular)4. In contrast, contractors from the US and the Philippines do work across many categories. This

variation in the geographic distribution of work by category likely reflects language, education, and offline

work opportunities. That said, Figure 3.11 indicates that software is one of the least concentrated sectors

in terms of the distribution of total wages across countries.

4This is consistent with the geographic distribution of work on Kaggle, an online data science competition platform, wheresoftware programmers are disproportionately located in Eastern Europe.

102

3.3.2 Income distribution

The digitization of contract labor markets may affect the distribution of income across workers. However,

the direction of this effect is ambiguous. On the one hand, digitization could amplify income inequality

by way of the superstar effect (Rosen, 1981), whereby the shift to lower search costs enables employers to

identify and contract for the best workers (or workers supplying the best value) in a global rather than local

context such that the distribution of the total wage bill skews further towards a minority of contractors.

On the other hand, digitization could reduce inequality due to more information leading to less mainstream

skills in the “long tail” being more efficiently matched (Anderson, 2006).

Researchers report evidence of both types of effects resulting from digitization. For example, Tucker

and Zhang (2007) find that when consumers on a wedding vendor website are able to see the popularity

of a given vendor, sales concentrate around the more popular vendors. This suggests that online feedback

systems have the potential to increase skewness. Elberse and Oberholzer-Gee (2006) find similar support

for video sales. In other cases, the reverse is true. Zentner, Smith, and Kaya (2012) show that online video

rentals are less concentrated around blockbusters than physical rentals, Peltier and Moreau (2012) show that

online book sales in France are less concentrated around superstars than offline, and Brynjolffson, Hu, and

Simester (2011) find that Internet sales for women’s clothing are less concentrated than catalog sales. All of

these papers identify search cost differences as a core explanation for the results.

Superstar and long tail effects are not necessarily mutually exclusive, and both in fact may be at work in

the context of online markets for contract labor. This is because they are influenced by related but distinct

characteristics of the services traded in this market. Vertical differentiation (quality) drives the superstar

effect, whereas horizontal differentiation (variety) drives the long tail effect (Bar-Isaac, Caruana, and Cunat,

2012). Therefore, subject to demand constraints, they may coexist. The superstar effect will result in

increased income inequality as employers tend towards the highest quality (or best value) contractors based

on a global rather than local search. Thus, income will shift from contractors supplying the best value

locally to those supplying the best value globally. Increased demand will drive up the wages of the highest-

quality workers, mainly in cases where the spread is greatest between local and global wages (i.e., low-income

countries). The superstar effect may be exacerbated due to information asymmetries and features of the

market.

At the same time, horizontally differentiated contractors (e.g., those who specialize in less common

103

areas) whose offline wages are lower due to limited local demand for their expertise may particularly benefit

from digitization since the shift from local to global matching may disproportionately increase the demand

for their skills relative to the supply. For example, a software developer in Malaysia who learns to program

in a new cutting-edge language (e.g., django) may benefit from digitization since by connecting to the

global market that contractor will likely face a greater increase in demand for that skill than an increase in

competition for supplying that skill.

In summary, digitization may shift the income distribution in a manner that benefits contractors with

skills that are vertically differentiated (i.e., higher quality), horizontally differentiated (i.e., scarce), or lower

cost (due to fewer local offline opportunities) at the expense of those with skills that are neither differentiated

nor low cost (i.e., mediocre quality, common skills, in high- or middle-income countries). The net effect of

such a shift is ambiguous, both at the country level and the individual level. At the country level, although

the immediate effect of digitization may be to decrease income inequality as the total wage bill shifts from

high- to low-income countries due to expanded search for skills and lower wage rates in low-income countries,

the resulting increase in productivity of firms in high-income countries may further increase offline wages

there, offsetting the effect of offshoring. At the individual level, while digitization will favor the highly

skilled relative to the less-skilled, particularly in high-income countries the services provided by a contractor

have increasing marginal costs, unlike products with low marginal costs such as music, books, and software.

Therefore, enhanced matching and constrained supply may at least partially offset increased competition and

thus temper the extent to which digitization amplifies the skewness of income distribution at the individual

level.

Information asymmetries may also affect income distribution. The available evidence shows that even

small amounts of (employer- or platform-provided) information have a large effect on future employment

prospects (Pallais, 2012; Agrawal, Lacetera, and Lyons, 2013). On the one hand, this may increase the

skewness of income distribution because contractors who obtain a small lead early on, in terms of online

work experience with a positive public employer review, may experience subsequent gains and benefit from

increasing returns (at least in the short term). On the other hand, to the extent that online markets facilitate

low-cost trials for employers to test working with novice contractors and then publicize their quality, the

digitization of this market may decrease skew through the increased public revelation of contractor quality.

The fact that a small amount of verified work experience online is associated with a disproportionate increase

104

in winning subsequent jobs for contractors in low-income countries (Agrawal, Lacetera, and Lyons, 2013)

seems consistent with this latter view.

3.3.3 Boundaries of the firm

How will the digitization of this marketplace influence the boundary of the firm? Economic theory suggests

that because digitization lowers transaction costs (search, communication, and monitoring), the returns to

contracting in the market increase relative to performing these services in-house. For example, Grossman and

Rossi-Hansberg (2008) model the tension between the benefits (lower cost of labor) and costs (coordination

and monitoring) of offshoring to examine precisely the effects of a decline in the cost of offshoring, focusing

on the productivity effect of increased offshoring. Similarly, Antras and Helpman (2004) present a model

of North-South trade where final-goods firms choose whether to vertically integrate into the production of

intermediate goods or to outsource them. Their model offers an explanation for variation in firm boundary

decisions (in equilibrium, some firms outsource while others do not, and those that do vary in their out-

sourcing location choice) based on the variation in firms’ productivity levels. Although the authors do not

focus on the effect of falling transaction costs associated with outsourcing per se, the influence of this on

firm boundaries is a natural implication of their model.

Several studies report empirical evidence that digitization is associated with a contraction in the bound-

ary of the firm. For example, Abramovsky and Griffith (2006) report that more ICT-intensive firms purchase

a greater amount of services on the market (rather than vertically integrating) and are more likely to purchase

offshore, Brynjolffson et al. (1994) report that investment in IT is correlated with a subsequent decrease in

firm size, and Hitt (1999) shows that an increase in IT use is correlated with a decrease in vertical integration.

A recent survey conducted by oDesk of its users sheds further light on the relationship between digi-

tization and firm boundary decisions. Two of the survey questions offer insight on how employers perceive

the online platform relative to alternatives for performing contracted work. One of the survey questions

asks: “If there had not been an appropriate oDesk contractor available for this project, then what would

you most likely have done?” Of the 6,912 respondents, only 15% indicated they would have turned to a

local hire, whereas 22% replied they would have worked extra hours, 9% replied they would have delayed

or canceled the project, and 50% indicated that they would have used some other remote source. Although

there is room for alternative interpretations of these responses (for example, “other remote sources” could

105

refer to other online contract labor platforms such that the results under-represent the fraction who would

hire locally in the absence of any online platforms), one possible explanation is that the digitization of this

marketplace directly affected the boundary of the firm in only a minority (15%) of the cases. A second

oDesk survey question asks: “Thinking about the last time that you hired a contractor through oDesk,

what alternatives did you consider?” In this case, respondents were able to select more than one option.

Again, only 15% selected “hiring an employee,” whereas 58% selected “doing it myself.” Shifting from local

to distant contractors appears to be a more significant economic effect from the digitization of this market

than contraction in the boundary of the firm. Indeed, 40% of respondents indicated that a “local contractor”

was an alternative they considered when they last hired a contractor through oDesk.

It is important to note that the majority of oDesk users are small businesses (90% of 7,098 survey

respondents indicated that their business had 10 employees or less, with an overall average firm size of 2.6

employees). This raises the question of how the effect of digitizing this marketplace may vary across firm size.

For example, do small firms benefit disproportionately from digitization? We cannot draw this conclusion

simply from observing a high fraction of small-firm users. First, the 90% small-firm user population may just

reflect the distribution of firm sizes in the economy (interestingly, respondents reveal that 68% are part-time

businesses, 69% are home-based businesses, and the average firm age is 2.7 years). Second, the survey sample

distribution may not reflect the population distribution. Perhaps small firms are more likely to respond to

the survey. Still, one might conjecture that small firms are more likely to hire contract workers since large

firms are better able to aggregate tasks into full-time jobs and thus avoid the contracting and discontinuity

costs associated with task-based hiring.

3.4 Market design

Platforms in online contract labor markets do not have the match-setting power typical in other contexts

that the market design literature has considered (e.g. Roth and Peranson, 1999; Roth, 2002; Milgrom, 2011)

because, unlike kidney exchanges and medical student-hospital matching systems, they are not centralized.

However, an inability to set matches explicitly does not imply an inability to influence which matches are

ultimately formed and under what terms. The position of the platform vis-a-vis the marketplace is more like

that of a government that sets policies to encourage efficient market outcomes without dictating trades. The

platform decides how often and in what context participants are exposed to each other, what information

106

is collected by parties, and how this information is displayed. Platforms also set policies about what trades

are permissible, how entry is gained, what contracts and prices are allowed, and so on. The platform may

also make recommendations and set defaults. A few market-design decisions in this softer match-making

environment are worth considering to explore how these features affect matching.

First, platforms are in the position to provide standardized and verified information. For example,

because oDesk does not permit contractors to delete ratings or comments provided by employers after a job

is completed, this information is possibly distinct from what contractors might include in their resumes and

thus valuable to potential future employers. In the introduction above, we describe two studies that report

findings indicating that online work history information has a significant influence on subsequent matching

outcomes (Pallais, 2012; Agrawal, Lacetera, and Lyons, 2013). Furthermore, platforms can provide additional

tools for contractors and employers to reveal standardized and verified information about themselves. For

example, oDesk provides a series of standardized tests that contractors are able to take so that they may

post their scores in order to communicate their proficiency in a specific domain.

Given that wading through too much information is costly for a potential employer, does a simple

overall performance score convey an optimal amount of information? Would a more detailed scoring system

enhance matching? Pallais (2012) reports that detailed feedback had no effect on subsequent outcomes,

relative to simple feedback, for inexperienced contractors. However, for experienced contractors, the extra

detail did make a difference. Furthermore, the Pallais result may underestimate the effect of a more detailed

rating system since her feedback was conveyed via text rather than, for example, a simple ranking on five

dimensions. Given the apparently high sensitivity to ratings and feedback, further research into market

design features that address this particular type of information friction seems a fruitful direction for future

research.

Second, because contractors have many decisions to make (such as what jobs to apply for, what wage to

bid, what skills to learn), as do employers (who to hire, whether to use a fixed or variable fee contract, when

to offer a bonus and how much), the digitized nature of these platforms, just like in other online markets, will

likely lead to the development of algorithmic assistance with decision-making. These recommendation sys-

tems will augment human decision-making by, for example, reducing the search costs of market participants.

One potential problem with recommending applicants is crowd-out. Recommending one worker presumably

puts another worker at a disadvantage. However, Horton (2012) shows that the quantity and quality of

107

matches can be improved via algorithmic recommendations to employers about candidates to recruit for

their openings, without significant crowd-out effects.

Aside from the obvious recommendations about who to trade with and at what terms, the platform can

also make other kinds of recommendations. It can, for example, advise parties of best practices in how to

manage a working relationship, such as suggesting more communication, periodic raises, and performance

evaluations. One interesting challenge of recommender systems is the trade-off between learning and rec-

ommending; recommender systems rely on natural decision-making to explore the space of alternatives to

train models, but sufficiently good recommender systems that save their users substantial costs are likely to

displace natural decision-making. So, maintaining some natural decision-making will eventually be costly,

at least to some users.

Another area where algorithmic recommendations might particularly influence matching is in helping

individuals make good decisions about the accumulation of human capital, particularly around which skills

to learn. Traditionally, such decisions are made a small number of times by relatively uninformed individuals

who receive one-time feedback about their choices. In offline markets, decisions about human capital in-

vestments are difficult to observe. Online, these choices are more visible and measurable. On platforms like

oDesk, an enormous amount of information illustrates which combination of skills command higher wages in

any particular domain. This enables recommender systems to distill which skills are most valuable to learn

given a contractor’s existing capabilities; the system further learns by observing how contractors perform

via experimentation.

Third, how should contract labor sites allocate visibility? Which applicants should be listed at the top

versus the bottom on an employer’s screen? The large size and value of the search engine optimization (SEO)

market provides some indication of the importance of visibility. Should allocation preserve assortativity (e.g.,

contractors with higher feedback ratings or hours worked are given more visibility)? Should each worker be

given at least some visibility? If visibility is auctioned off, what would be the efficiency and distributional

properties of such an allocation? While this topic has received much attention from researchers in the

private sector at companies like Google and eBay concerning other markets, it remains an open question

in the context of the market for contract labor. Yet, this issue is important. Market design decisions

concerning the allocation of visibility will surely influence matching outcomes, which in turn will influence

both distribution and welfare effects. Moreover visibility relates to congestion, which we discuss next.

108

Fourth, platforms may need to control congestion due to the fact that posting (and applying for) a job

is almost costless. The low cost of applications may lead to an everyone-applies-to-everything equilibrium in

which each application also carries virtually no signal value. This was partly the motivation for introducing

the AEA signaling mechanism (Coles et al., 2010), in which job-market participants are given two (and

only two) signals to send to schools. The school’s knowledge of the scarcity of signals makes those signals

informative. Accordingly, platforms may consider job application quotas. However, as described above, this

strategy might penalize new entrants with low probabilities of being hired (Pallais, 2012). It also ignores

employer heterogeneity, with some employers preferring many applicants and others few. Another potentially

interesting approach is to allow the employer to decide the cost of applying. These are additional areas for

research that reflect the peculiarities of this market.

A fifth interesting market-design feature is the creation of submarkets and categories that are often

defined through some combination of geography and time to coordinate activities and thus create a sufficiently

thick market (e.g., the creation of industrial districts for specific sectors). The platform must attempt to

define at some level of detail the various services being supplied and then organize the market accordingly.

In the language of machine learning, there is both a clustering task (finding the meaningful groups of

jobs/contractors based on historical data) and a labeling task (being able to assign a new job to one of the

identified clusters based on that job’s attributes).

The five market features we describe above represent only a fraction of those that may be important

for influencing the matching of employers and contractors as well as the way in which work is managed and

produced online. The unique feature of this line of inquiry, relative to the one described above concerning

distribution and the one below concerning social welfare, is that this research can be performed without offline

data. That is because online features can be compared against each other with respect to the behaviour they

elicit from users. So-called ”A/B testing,” which refers to controlled randomized experiments that allow for

identifying causal relationships between variations in market design features and subsequent user behaviour,

has already become a standard industry practise for determining the relative performance of competing

market design features. That is likely the reason that the majority of research concerning online markets for

contract labor relate to this line of inquiry whereas there is very little so far on the other topics.

109

3.5 Social Welfare

Two immediate consequences of digitization in this market may have important welfare implications. First,

digitization may lead to better matching because the pool of prospective workers and employers increases

dramatically due to the decline in costs associated with distance. Second, digitization may lead to efficiency

gains from production due to lower coordination costs. We discuss both lines of inquiry below.

3.5.1 Matching made easier?

The ease of access to online contract labor markets, due to the development of platforms such as oDesk,

Freelancer, Elance, and Guru, has the potential to considerably increase the pool of both job seekers and

employers and to reduce search costs. Matching models, particularly as applied to labor markets, predict that

this will lead to efficiency gains due to lower search costs and a lower likelihood of mismatches (Petrongolo

and Pissarides, 2001; Wheeler, 2001).

However, opposite forces are also at play. While information technologies reduce the role of distance for

search and execution of work, they also lead to a more heterogeneous pool of both workers and employers.

In addition, the absence of personal interactions typical of offline and more localized labor markets precludes

access to soft or high-bandwidth information about both job seekers and prospective employers (Autor,

2001). This introduces uncertainty that in turn may lead to an overall reduction in the quality of workers

(Akerlof, 1970) and/or to search frictions (Stigler, 1962). These search frictions could be exacerbated if

quality is difficult to determine (Wilde, 1981), which is quite possible because of the diverse labor pool.

Although theories of search and matching specific to online labor markets have not yet been developed,

a growing body of evidence, described above, points to the presence of these informational problems and the

ways in which they are addressed in online contract labor platforms, (Horton, 2012; Pallais, 2012; Stanton and

Thomas, 2012). A common pattern to a number of these studies is to look at the presence of preferences for

certain geographic locations of workers as a way to alleviate uncertainty about workers’ quality (Mill, 2011;

Ghani, Kerr, and Stanton, 2012; Agrawal, Lacetera, and Lyons, 2013). An implication here is that online

contract labor platforms contribute to the alleviation of informational asymmetries by providing verifiable,

standardized information (such as previous experience on the same platform) for all workers, regardless of

their origin.

The broadening of the pool of workers and employers and, at least potentially, the increased likelihood

110

of good matches, is also likely to have implications for wages and income distribution. The fact that

in online contract labor markets the number of workers outweighs the number of employers in every job

category suggests that while many workers may be left unemployed, employers have a relatively good chance

of finding a worker who meets their criteria, with wages driven down (Petrongolo and Pissarides, 2006).

However, because worker backgrounds may vary more than in traditional labor markets, a relatively small

number of workers may meet the job requirements. As a result, wage offers could be higher than expected.

This suggests that in job categories with many qualified workers, the wages will be lower than in those with

few qualified workers relative to the number of job postings. As the market evolves, wage differences between

job types should begin to disappear.

3.5.2 Efficiency gains from production?

Digitization may lead to efficiency gains in production due to lower coordination costs that enhance, for

example, contractor flexibility, discretization of work into smaller jobs enabling more specialization, and

remote team work. For instance, Dettling (2012) reports that flexibility provided by IT contributes to an

increase in the female labor force participation. More broadly, digitization may enable efficiency gains in

production through lowering the cost of outsourcing.

Of course, outsourcing and offshoring predates the development of online contract labor markets. Of

particular relevance here are theories of service outsourcing and offshoring (e.g. Bhagwati, Panagariya, and

Srinivasan, 2004; Francois and Hoekman, 2010). Combined, these theories predict that the gains to service

outsourcing are potentially significant. However, they focus on relatively long arm’s-length contracts between

relatively large firms rather than on the short contracts between small organizations and individuals, typical

of online markets.

Outsourcing services to online contract labor markets is also likely to lead to geographically dispersed

production, even within narrowly defined tasks. For example, work teams may be composed of individuals

who are not necessarily co-located. Lazear (1999) argues that cultural diversity in work teams is costly and

should only occur when skill complementarities exist between teammates to offset these costs. It may be

harder to meet these conditions in very diverse online labor markets than it is in more traditional labor

markets. Two recent studies based on online labor markets focus on task completion and the effects of team

organization, communication structure, incentives, and motivation on performance. Lyons (2013) provides

111

field experimental evidence on how nationally diverse communication impacts online team production and

finds that nationally homogeneous teams benefit from working together but that diverse teams perform better

when members work independently of one another. Related to the topic of online labor market partnerships,

Horton (2011) uses survey data from the crowdsourcing site Mechanical Turk to show that workers believe

employers on the site are more fair and honest than offline employers.

3.6 Conclusion

We identify three broad lines of inquiry as central to the digitization research agenda. All three focus on the

effect of digitizing the market for contract labor. The first concerns welfare effects, the second distribution

effects, and the third user behavior effects. All three are set in the context of the market for contract labor

but have broader implications for digitization in other settings.

Access to data will pose a challenge to fully addressing these questions. In contrast to data from online

platforms that collect information on hiring (as well as pre- and post-hiring) transactions at a granular level

and at low cost, it is costly to obtain even a basic level of offline contracting data. Yet, to fully address the first

and third lines of inquiry outlined above, offline data is required to estimate the causal effect of digitization

on changes in distributional properties (geography, income, firm boundaries) and welfare. This is likely why

most of the first wave of studies concerning the digitization of this market focuses on market design-related

subjects (e.g., experience, agencies, ratings) since these questions only require observing within-platform

variation in user behavior and do not require linking these data to non-platform-participants.

While the second line of inquiry concerning market design, information frictions, and user behavior is

largely spared from the requirement to link with offline data, the greatest challenge to this research in the

short and medium term will likely be the rapid evolution of the industry. As illustrated above, the industry

is growing rapidly. In addition, complementary technologies, such as those associated with mobile and social,

are changing rapidly. As such, market design features that seem salient today may be less relevant relative

to other features in the future. For example, monitoring technologies such as work rooms with screen shots

were only recently introduced and are already standard practice across many platforms. Furthermore, they

are likely to be replaced soon with better technology such as streaming screen video. While the ultimate goal

of research of this type is obtaining a deeper understanding of human behavior rather than of a particular

market design feature, the economic salience of the feature is often important for generalizability and yet may

112

be fleeting due to the rapid pace of change in this setting. Still, insight into user response to informational

frictions is an important contribution.

Whereas the distribution and welfare lines of inquiry are most likely to be led by scholars and policy

makers, the market design-related research will almost surely include important contributions from industry

since this issue is of first-order importance for product development and competition. This has already been

the case with oDesk (Horton, 2010, 2012) as well as with other market design issues on platforms such as

Google (Varian, 2007; Choi and Varian, 2012), eBay (Blake, Nosko, and Tadelis, 2013), and Yahoo (Ghosh

and McAfee, 2011; Lewis and Reiley, 2011). Industry interest coupled with their access to high-quality data

may significantly accelerate progress on this research frontier. At the same time, the competitive implications

of market design insights may inhibit the dissemination of this type of research, and thus the overall impact of

industry interest in this subject on the rate and direction of progress on this part of the agenda is ambiguous.

Given the role that platforms play as the central collectors of data in these markets, they will influence the

direction of research on all three lines if inquiry through their decisions regarding providing researchers with

access to their data. Early signs are promising for the research community since many of the most prominent

platforms have established Chief Economists or similar types of research-friendly leadership positions and

encourage employees to participate in the academic community by publishing their research and participating

at conferences and other scholarly events.

Given the rapid growth rate of the online market for contract labor, this research agenda is economically

important. The welfare-related line of inquiry will help us better understand the potential private and social

benefits due to the digitization of this sector of the economy. The distribution-related line of inquiry will

shed light on how the benefits of digitization may be allocated across countries and individuals as well

as its impact on the structure of the firm. Finally, the market design-related line of inquiry will provide

further insight into the importance of particular information frictions and human behavior in the digital

world as we explore user reactions to platform features, many of which are common across sectors outside of

contract employment. Overall, these insights will be of great interest to scholars, policymakers, and industry

participants alike.

113

3.7 Figures

Figure 3.1: Number of billing employers per quarter on oDesk, relative to total number ofemployers in first quarter of 2009, by employer country income level

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

0

1

2

3

4

5

6

7

8

9

2009 2010 2011 2012Quarters

Num

ber

of d

istin

ct e

mpl

oyer

s bi

lling

that

qua

rter

,rel

ativ

e to

firs

t qua

rter

, 200

9

Employer country type

High Income Country

Non High Income Country

Notes: This figure uses data collected from oDesk to show the relative number of billing employers perquarter, by country income status. We use the 2012 World Bank list of high-income countries for ourcountry classification. The base quarter is the first quarter (i.e., January, February, and March) for 2009.Although the count looks like it is exactly 1 for HIC in 2009, it is slightly below—there were a small butnon-zero number of employers from lower-income countries during that quarter.

114

Figure 3.2: Quarterly wage bill on oDesk by employer country income level

●

●

● ●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

$0

$25,000,000

$50,000,000

$75,000,000

2009 2010 2011 2012Quarters

Wag

e bi

ll pe

r qu

arte

r

Employer country type

High Income Country

Non High Income Country

Notes: This figure uses data collected from oDesk to show the quarterly wage bill by the employer’scountry’s income status. We use the 2012 World Bank list of high-income countries for our countryclassification.

115

Figure 3.3: Number of working contractors per quarter on oDesk, relative to total number ofcontractors in first quarter of 2009, by contractor country income level

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

0

1

2

3

4

5

6

7

8

9

10

2009 2010 2011 2012Quarters

Num

ber

of d

istin

ct c

ontr

acto

rs b

illin

g th

at q

uart

er,r

elat

ive

to fi

rst q

uart

er, 2

0102

009

Contractor country type

High Income Country

Non High Income Country

Notes: This figure uses data collected from oDesk to show the relative number of working contractorsper quarter, by country income status. We use the 2012 World Bank list of high-income countries for ourcountry classification. The base quarter is the first quarter (i.e., January, February, and March) for 2009.

116

Figure 3.4: Contractor quarterly earnings on oDesk by contractor country income level

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

$0

$20,000,000

$40,000,000

$60,000,000

2009 2010 2011 2012Quarters

Wag

e bi

ll pe

r qu

arte

r

Contractor country type

High Income Country

Non High Income Country

Notes: This figure uses data collected from oDesk to show the quarterly wage bill, by the contractor’scountry’s income status. We use the 2012 World Bank list of high-income countries for our countryclassification.

117

Figure 3.5: Number of contractors per country on oDesk versus country population, on alog-log scale

Notes: This figure uses data collected from oDesk to show the count of contractors who have ever workedon oDesk by country, versus the 2012 World Bank estimate of that country’s population. Both axes arelog-log scale. We only include countries with 500 or more ever-active contractors.

118

Figure 3.6: Contractor mean hourly wage on oDesk by country, relative to that country’sestimated local minimum wage

US UK

Notes: This figure uses data collected from oDesk to compare the mean hourly wage (log scale). To estimatehourly wages, we restrict our attention to hourly contracts in the first half of 2013. Harmonized minimumwage data is difficult to acquire. As a proxy, we use the Wikipedia estimates, as of May 2013.

119

Figure 3.7: Contractor mean hourly wages on oDesk, by country

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Philippines

Bangladesh

Kenya

Sri Lanka

Pakistan

Egypt

Indonesia

India

Serbia

Bulgaria

Romania

Vietnam

Spain

Ukraine

Canada

United Kingdom

United States

Belarus

Russia

Australia

Poland

China

$0 $5 $10 $15 $20Mean wage ($/hour)

Con

trac

tor

coun

try

Notes: We estimate hourly wages using a sample of all hourly contracts in the first half of 2013. We excludeobservations of less than 10 cents and more than $100, as these observations are more likely to not be truehourly wages but rather individuals using the time tracking software provided by oDesk or approximatinga fixed price contract of some kind with a high hourly wage. For each wage estimate, we include a 95%confidence interval. Note that for high population countries like India and Philippines, these confidenceintervals are so narrow that they appear to be point estimates.

120

Figure 3.8: Average hourly wage on oDesk per quarter, by job category

● ●●●● ● ●● ●● ● ● ●● ●

● ●● ●●

●●

●●●● ●

● ●● ● ●

●●

●●

●

●●

● ●

●

●●

●

●● ●●● ●● ●● ●● ●● ●

●

●● ●● ● ●

● ●● ●

●●● ● ●

●●

●● ●●●●● ●●● ●● ●

●

● ●

● ●

●●

●

●●

●● ● ●●

●●● ●

●●

●● ●●●● ●●●

●

●●

●

●

●

●●●

●

●

●● ●

●

AdministrativeSupport

BusinessServices

CustomerService

Design &Multimedia

Networking &Information

Systems

Sales &Marketing

SoftwareDevelopment

WebDevelopment

Writing &Translation

$0

$5

$10

$15

$0

$5

$10

$15

$0

$5

$10

$15

2010 2011 2012 2013 2010 2011 2012 2013 2010 2011 2012 2013Quarters

Mea

n ho

urly

wag

e (U

SD

)

Notes: This figure uses oDesk data to show the mean hourly wages per quarter in each of the main oDeskcategories of work.

121

Figure 3.9: Quarterly wage bill per job category on oDesk (log scale)

Administrative Support Business Services Customer Service

Design & Multimedia Networking & Information Systems Sales & Marketing

Software Development Web Development Writing & Translation

$100,000

$1,000,000

$10,000,000

$100,000

$1,000,000

$10,000,000

$100,000

$1,000,000

$10,000,000

2009 2010 2011 2012 2009 2010 2011 2012 2009 2010 2011 2012Quarter

Qua

rter

ly s

pend

(U

SD

), lo

g sc

ale

Notes: This figure uses oDesk data to show the total quarterly wage bill by job category.

122

Figure 3.10: Contractor job category concentration on oDesk by contractor country, overtime

India

Philippines

Russia

Ukraine

United States0.05

0.10

0.15

0.20

2009 2010 2011 2012 2013 2014Quarter

Her

finda

hl in

dex

of c

ontr

acto

r ca

tego

ry c

once

ntra

tion

(by

wag

e bi

ll)

Notes: This figure uses oDesk data to compute a quarterly Herfindahl for a select number of oDeskcontractor countries. We compute the index by treating oDesk job categories as “firms” and contractorcountries as “industries.” To compute this measure, for each quarter, we estimate the share of dollarsearned by contractors from a particular country, in each category. We then report the sum of the square ofthese shares. The higher the index, the more concentrated workers from that country. For example, if anindex is near 1, it would mean that nearly all workers from that country work in a single category.

123

Figure 3.11: Job category concentration on oDesk by contractor, over time

Administrative Support

Business Services

Customer Service

Design & MultimediaNetworking & Information Systems

Sales & Marketing

Software Development

Web Development

Writing & Translation

0.025

0.050

0.075

0.100

0.125

2010 2011 2012 2013 2014Quarter

Her

finda

hl in

dex

(by

wag

e bi

ll)

Notes: This figure uses oDesk data to compute a quarterly Herfindahl for each job category, treating eachcountry as a “firm” and each category as an “industry.” To compute this measure, for each quarter, weestimate the share of dollars within a category earned by contractors from each country. We then reportthe sum of the square these shares. The higher the index, the more that particular category is dominatedby workers from a particular country. For example, if an index is near 1, it would mean that nearly all workin that category is completed by workers from a single country.

124

Bibliography

Abramovsky, Laura and Rachel Griffith. 2006. “Outsourcing and Offshoring of Business Services: HowImportant is ICT?” Journal of the European Economic Association 4 (2-3):594–601.

Agrawal, A. and A. Goldfarb. 2008. “Restructuring Research: Communication Costs and the Democratiza-tion of University Innovation.” American Economic Review 98.

Agrawal, A., J. Horton, N. Lacetera, and E. Lyons. 2013. “Digitization and the Contract Labor Market: AResearch Agenda.” Working Paper 19525, National Bureau of Economic Research.

Agrawal, A., N. Lacetera, and E. Lyons. 2013. “Does Information Help or Hinder Job Applicants from LessDeveloped Countries in Online Markets?” Working Paper 18720, National Bureau of Economic Research.

Akerlof, George A. 1970. “The Market for “Lemons”: Quality Uncertainty and the Market Mechanism.”Quarterly Journal of Economics 84 (3):488–500.

Altonji, Joseph G and Charles R Pierret. 2001. “Employer learning and statistical discrimination.” TheQuarterly Journal of Economics 116 (1):313–350.

Anderson, Chris. 2006. “The Long Tail: How Endless Choice Is Creating Unlimited Demand.” .

Antras, Pol, Luis Garicano, and Esteban Rossi-Hansberg. 2006. “Offshoring in a knowledge economy.” TheQuarterly Journal of Economics 121 (1):31–77.

Antras, Pol and Elhanan Helpman. 2004. “Global Sourcing.” Journal of Political Economy 112 (3):552–580.

Armstrong, Mark. 2006. “Competition in two-sided markets.” The RAND Journal of Economics 37 (3):668–691.

Autor, David H. 2001. “Wiring the Labor Market.” Journal of Economic Perspectives 15 (1):25–40.

Banerjee, Abhijit V and Esther Duflo. 2000. “Reputation effects and the limits of contracting: A study ofthe Indian software industry.” The Quarterly Journal of Economics 115 (3):989–1017.

Banker, R. D., J. M. Field, R. G. Schroeder, and K. K. Sinha. 1996. “Impact of Work Teams on ManufacturingPerformance: A Longitudinal Field Study.” Academy of Management Journal 39:867–890.

Bar-Isaac, Heski, Guillermo Caruana, and Vicente Cunat. 2012. “Information Gathering Externalities for aMulti-Attribute Good.” The Journal of Industrial Economics 60 (1):162–185.

Bengtsson, O. and D. H. Hsu. 2013. “Ethnic Matching in the U.S. Venture Capital Market.” ManagementDepartment Working Paper .

Bertrand, Marianne and Sendhil Mullainathan. 2004. “Are Emily and Brendan more employable than Latoyaand Tyrone? Evidence on racial discrimination in the labor market from a large randomized experiment.”American Economic Review 94 (4):991–1013.

125

Bhagwati, Jagdish, Arvind Panagariya, and T. N. Srinivasan. 2004. “The Muddles over Outsourcing.”Journal of Economic Perspectives 18 (4):93–114.

Blake, Thomas, Chris Nosko, and Steven Tadelis. 2013. “Consumer Heterogeneity and Paid Search Effec-tiveness: A Large Scale Field Experiment.”

Bloom, Nicholas, James Liang, John Roberts, and Zhichun Jenny Ying. 2013. “Does Working from HomeWork? Evidence from a Chinese Experiment.”

Boudreau, K. J., N. Lacetera, and K. Lakhani. 2011. “Incentives and Problem Uncertainty in InnovationContests: An Empirical Analysis.” Management Science 57:843–863.

Boudreau, K. J. and K. Lakhani. 2011. “’Fit’: Field Experimental Evidence on Sorting, Incentives andCreative Worker Performance.” Working Paper 11107, Harvard Business School.

Brown, Jennifer and John Morgan. 2006. “Reputation in online markets: some negative feedback.” Universityof California, Berkeley .

Brynjolffson, Eric, Yu Hu, and Duncan Simester. 2011. “Goodbye Pareto Principle, Hello Long Tail: TheEffect of Search Costs on the Concentration of Product Sales.” Management Science 57 (8):1373–1386.

Brynjolffson, Eric, Thomas W. Malone, Vijay Gurbaxani, and Ajit Kambi. 1994. “Does Information Tech-nology Lead to Smaller Firms?” Management Science 40 (12):1628–1644.

Brynjolfsson, Erik, Yu Hu, and Duncan Simester. 2011. “Goodbye pareto principle, hello long tail: Theeffect of search costs on the concentration of product sales.” Management Science 57 (8):1373–1386.

Cabral, Luis and Ali Hortacsu. 2010. “The dynamics of seller reputation: Evidence from ebay.” The Journalof Industrial Economics 58 (1):54–78.

Cameron, Adrian Colin and Pravin K Trivedi. 2009. Microeconometrics using stata, vol. 5. Stata PressCollege Station, TX.

Canella, A. A. Jr., J.-H. Park, and H.-U. Lee. 2008. “Top Management Team Functional BackgroundDiversity and Firm Performance: Examining The Roles of Team Member Colocation and EnvironmentalUncertainty.” Academy of Management Journal 51:768–784.

Carlsson, Magnus and Dan-Olof Rooth. 2007. “Evidence of ethnic discrimination in the Swedish labor marketusing experimental data.” Labour Economics 14 (4):716–729.

Chafkin, M. 2010. “The Case, and the Plan, for the Virtual Company.” URL http://inc.com.

Choi, Hyunyoung and Hal Varian. 2012. “Predicting the Present with Google Trends.” Economic Record88 (s1):2–9.

Coles, Peter, John Cawley, Phillip B. Levine, Muriel Niederle, Alvin E. Roth, and John J. Siegfried. 2010.“The Job Market for New Economists: A Market Design Perspective.” The Journal of Economic Perspec-tives 24 (4):187–206.

Croson, R., J. Anand, and R. Agarwal. 2007. “Using Experiments in Corporate Strategy Research.” EuropeanManagement Review 4:173–181.

Cummings, J. N. and M. R. Haas. 2012. “So many teams, so little time: Time allocation matters ingeographically dispersed teams.” Journal of Organizational Behavior 33 (3):316–341.

Dellarocas, Chrysanthos. 2006. “Reputation mechanisms.” Handbook on Economics and Information Systems:629–660.

126

Deloitte. 2007. “2007 Technology Fast 500: Annual list of the fastest growing companies in North America.”(7).

Dequiedt, Vianney and Yves Zenou. 2013. “International migration, imperfect information, and brain drain.”Journal of Development Economics 102:62–78.

Dettling, Lisa J. 2012. “Opting Back In: Home Internet Use and Female Labor Supply.”

Dewan, Sanjeev and Vernon Hsu. 2004. “Adverse selection in electronic markets: Evidence from onlinestamp auctions.” The Journal of Industrial Economics 52 (4):497–516.

Dutcher, E. Glenn and Krista Jabs Saral. 2012. “Does Team Telecommuting Affect Productivity? AnExperiment.”

Elance.com. 2012. URL http://www.elance.com.

Elberse, Anita and Felix Oberholzer-Gee. 2006. Superstars and underdogs: An examination of the long tailphenomenon in video sales. Division of Research, Harvard Business School.

Elfenbein, Daniel W, Ray Fisman, and Brian McManus. 2012. “Charity as a substitute for reputation:Evidence from an online marketplace.” The Review of Economic Studies 79 (4):1441–1468.

Ely, R. J. and D. A. Thomas. 2001. “Cultural Diversity at Work: The Effects of Diversity Perspectives onWork Group Processes and Outcomes.” Administrative Science Quarterly 46:229–273.

Farmer, S. M. and C. W. Hyatt. 1994. “Effects of Task Language Demands and Task Complexity onComputer-Mediated Work Groups.” Small Group Research 25:331–366.

Ferrer, Ana and W Craig Riddell. 2008. “Education, credentials, and immigrant earnings.” Canadian Journalof Economics/Revue canadienne d’economique 41 (1):186–216.

Figlio, David N. 2005. “Names, expectations and the black-white test score gap.” Tech. rep., NationalBureau of Economic Research.

Financial Times. 2012. “Virtual Working Takes off in EMs.” .

Forman, C. and N van Zeebroeck. 2012. “From Wires to Partners: How the Internet Has Fostered RDCollaborations Within Firms.” Management Science 58.

Francois, Joseph and Bernard Hoekman. 2010. “Services Trade and Policy.” Journal of Economic Literature48 (3):642–692.

Galegher, J. and R. E. Kraut. 2003. “Computer-Mediated Communication for Intellectual Teamwork: AnExperiment in Group Writing.” Information Systems Research 5:110–138.

Gardner, H. K., F. Gino, and B. R. Staats. 2012. “Dynamically integrating knowledge in teams: Transformingresources into performance.” Academy of Management Journal 55 (4):998–1022.

Gaspar, Jess and Edward L Glaeser. 1998. “Information technology and the future of cities.” Journal ofurban economics 43 (1):136–156.

Ghani, Ejaz, William R. Kerr, and Christopher T. Stanton. 2012. “Diasporas and Outsourcing: Evidencefrom oDesk and India.” Working Paper 18474, National Bureau of Economic Research.

Ghosh, Arpita and Preston McAfee. 2011. “Incentivizing High-Quality User-Generated Content.” Proceedingsof the 20th International Conference on World Wide Web :137–146.

Gong, Y. 2003. “Subsidiary Staffing in Multinational Enterprises: Agency, Resources and Performance.”Academy of Management Journal 46:307–338.

127

Grossman, G. M. and E Helpman. 2002. “Integration versus Outsourcing in Equilibrium.” The QuarterlyJournal of Economics 117:85–120.

Grossman, Gene M. and Esteban Rossi-Hansberg. 2008. “Trading Tasks: A Simple Theory of Offshoring.”American Economic Review 98 (5):1978–1997.

Groysberg, Boris, David Thomas, and Jennifer Tydlaska. 2011. “oDesk: Changing How the World Works.”Harvard Business School Organizational Behavior Unit Case (411-078).

Guzzo, R. A. and M. W. Dickson. 1996. “Teams in Organizations: Recent Research on Performance andEffectiveness.” Annual Review of Psychology 47:307–338.

Haas, M. R. 2010. “The double-edged swords of autonomy and external knowledge: Analyzing team effec-tiveness in a multinational organization.” Academy of Management Journal 53 (5):989–1008.

Hackman, J. R. 1998. “Why Teams Don’t Work.” :245–267.

Hambrick, D. C., T. S. Cho, and M-J. Chen. 1996. “The Influence of Top Management Team Heterogeneityon Firms’ Competitive Moves.” Administrative Science Quarterly 41:659–684.

Hamilton, B. H., J. A. Nickerson, and H Owan. 2003. “Team Incentives and Worker Heterogeneity: AnEmpirical Analysis of the Impact of Teams on Productivity and Participation.” Journal of PoliticalEconomy 111:465–497.

———. 2012. “Diversity and Productivity in Production Teams.” Advances in the Economic Analysis ofParticipatory and Labor-Managed Firms 13:99–138.

Harrison, D. A. and K. J. Klein. 2007. “Whats the Difference? Diversity Constructs as Separation, Variety,or Disparity in Organizations.” The Academy of Management Review 32:1199–1228.

Harrison, G. W. and J. A List. 2004. “Field Experiments.” Journal of Economic Literature 42:1009–1055.

Hays, R. D. 1974. “Insuring Success and Avoiding Failure.” Journal of International Business Studies5:25–37.

Heckman, James J, Lance J Lochner, and Petra E Todd. 2008. “Earnings functions and rates of return.”Tech. rep., National Bureau of Economic Research.

Hegde, D. and J. Tumlinson. 2011. “Ethnicity-Based Investment Selection and Performance: Theory andEvidence from US Venture Capital.” Working paper.

Hinds, P., L. Liu, and J. Lyon. 2011. “Putting the Global in Global Work: An Intercultural Lens on thePractice of Cross-National Collaboration.” The Academy of Management Annals 5:135–188.

Hinds, P. J. and M. Mortensen. 2005. “Understanding Conflict in Geographically Distributed Teams: TheModerating Effects of Shared Identity, Shared Context, and Spontaneous Communication.” OrganizationScience 16:290–307.

Hitt, Lorin M. 1999. “Information Technology and Firm Boundaries: Evidence from Panel Data.” Informa-tion Systems Research 10 (2):134–149.

Hjort, J. 2011. “Ethnic Divisions and Production in Firms.” Working paper.

Hofstede, G. 1983. “The Cultural Relativity of Organizational Practices and Theories.” Journal of Interna-tional Business Studies 14:75–89.

———. 2013. “The Hofstede Centre: National Culture.” Tech. rep. URL http://geert-hofstede.com/

countries.html.

128

Hollingshead, A. B., J. E. Mcgrath, and K. M. O’Connor. 1993. “Group Task Performance and Commu-nication Technology: A Longitudinal Study of Computer-Mediated Versus Face-to-Face Work Groups.”Small Group Research 24:307–333.

Holmstrom, Bengt. 1982. “Moral hazard in teams.” The Bell Journal of Economics :324–340.

Hoogendoorn, S., H. Oosterbeek, and M. van Praag. 2013. “The Impact of Gender Diversity on the Per-formance of Business Teams: Evidence from a Field Experiment.” Management Science Articles inAdvance:1–15.

Horton, J. J., D. G. Rand, and R. J. Zeckhauser. 2011. “The online laboratory: Conducting experiments ina real labor market.” Experimental Economics 14 (3):399–425.

Horton, John J. 2010. “Online Labor Markets.” Internet and Network Economics 6484:515–522.

———. 2011. “The condition of the Turking class: Are online employers fair and honest?” Economic Letters111 (1):10–12.

———. 2012. “Computer-Mediated Matchmaking: Facilitating Employer Search and Screening.”

Huckman, R. S. and B. R. Staats. 2009. “Fluid Tasks and Fluid Teams: The Impact of Diversity in Experienceand Team Familiarity on Team Performance.” Manufacturing Service Operations Management 13:310–328.

Hunt, J. 2013. “Are Immigrants the Best and Brightest U.S. Engineers?” Working Paper 18696, NationalBureau of Economic Research.

International Development Research Center. 2009. Science and Innovation News.

International Labor Organization. 2012a. International Labour Migration Statistics.

———. 2012b. Labor Migration.

Ipeirotis, Panos. 2012. “The Emergence of Teams in Online Work.” .

Islamabad, UNESCO. 2013. “Teacher Education in Pakistan.” http://unesco.org.pk/.

Jin, Ginger Zhe and Andrew Kato. 2006. “Price, quality, and reputation: Evidence from an online fieldexperiment.” The RAND Journal of Economics 37 (4):983–1005.

Jones, B. 2011. “The Knowledge Trap: Human Capital and Development Reconsidered.” Working paper.

Kim, K. B. 2006. “Direct Employment in Multinational Enterprises: Trends and Implications.” MULTIWorking Paper, International Labour Organization 101.

Kirkman, B. L. and D. L. Shapiro. 2005. “The Impact of Cultural Value Diversity on Multicultural TeamPerformance.” Volume Advances in International Management 18:33–67.

Klebe Trevino, L. 1992. “Experimental Approaches to Studying Ethical-Unethical Behavior in Organiza-tions.” Business Ethics Quarterly 2:121–136.

Lahey, Joanna N. 2008. “Age, Women, and Hiring An Experimental Study.” Journal of Human Resources43 (1):30–56.

Landes, D. S. 1999. The Wealth and Poverty of Nations: Why Some Are So Rich and Some So Poor.

Lang, Kevin and Michael Manove. 2006. “Education and labor-market discrimination.” Tech. rep., NationalBureau of Economic Research.

129

Lazear, Edward P. 1999. “Globalisation and the Market for Team-Mates.” The Economic Journal109 (454):15–40.

Leslie, D. and J. Lindley. 2003. “The Impact of Language Ability on Employment and Earnings of Britain’sEthnic Communities.” Economica 68:587–606.

Levitt, S. D. and J. A. List. 2007. “What do Laboratory Experiments Measuring Social Preferences RevealAbout the Real World?” Journal of Economic Perspectives 21:153–174.

Lewis, Randall A. and David Reiley. 2011. “Does Retail Advertising Work? Measuring the Effects ofAdvertising on Sales via a Controlled Experiment on Yahoo!”

List, John A. 2003. “Does market experience eliminate market anomalies?” The Quarterly Journal ofEconomics 118 (1):41–71.

———. 2004. “The nature and extent of discrimination in the marketplace: Evidence from the field.” TheQuarterly Journal of Economics 119 (1):49–89.

Lyons, Elizabeth. 2013. “Team Production in International Labor Markets: Experimental Evidence fromthe Field.”

Makela, K., U. Andersson, and T. Seppala. 2012. “Interpersonal Similarity and Knowledge Sharing withinMultinational Organizations.” International Business Review 21:439–451.

Malone, T. 1998. “Laubacher (1998), The dawn of the e-lance economy.” Harvard Business Review .

Manev, I. M. and W. B. Stevenson. 2001. “Nationality, Cultural Distance, and Expatriate Status: Effectson the Managerial Network in a Multinational Enterprise.” Journal of International Business Studies32:285–303.

Manz, C. C. and H. P. Sims Jr. 1987. “Leading Workers to Lead Themselves: The External Leadership ofSelf- Managing Work Teams.” Administrative Science Quarterly 32:106–129.

Maznevski, M. L. and K. M. Chudoba. 2000. “Bridging Space Over Time: Global Virtual Team Dynamicsand Effectiveness.” Organization Science 11:473–492.

McFadden, D. 1974. “Conditional logit analysis of qualitative choice behavior, Zarembka P., Frontiers inEconometrics.”

Milgrom, Paul. 2011. “Critical Issues in the Practice of Market Design.” Economic Inquiry 49 (2):311–320.

Mill, Roy. 2011. “Hiring and Learning in Online Global Labor Markets.” .

Ministry of Human Resource Development Government of India. 2013. “Department of School Education &Literacy.” http://mhrd.gov.in/schooleducation.

National Curriculum & Textbook Board, Bangladesh. 2013. “Curriculum.” http://www.nctb.gov.bd/.

oDesk. 2012. URL http://www.odesk.com.

OReilly, C. A. III, K. Williams, and S. Barsade. 1997. “Group Demography and Innovation: Does DiversityHelp?” Research in Management Groups and Teams 1:183–207.

Oreopoulos, P. 2011. “Why Do Skilled Immigrants Struggle in the Labor Market? A Field Experiment withThirteen Thousand Resumes.” American Economic Journal: Economic Policy 3:148–171.

Pager, Devah, Bruce Western, and Bart Bonikowski. 2009. “Discrimination in a Low-Wage Labor Market AField Experiment.” American Sociological Review 74 (5):777–799.

130

Pallais, Amanda. 2012. “Inefficient Hiring in Entry-Level Labor Markets.” Working paper.

Pearce, Jone L. 1993. “Toward an organizational behavior of contract laborers: Their psychological involve-ment and effects on employee co-workers.” Academy of Management Journal 36 (5):1082–1096.

Peltier, Stephanie and Francois Moreau. 2012. “Internet and the Long Tail versus superstar effectdebate:evidence from the French book market.” Applied Economics Letters 19 (8):711–715.

Petrongolo, Barbara and Christopher A. Pissarides. 2001. “Looking into the Black Box: A Survey of theMatching Function.” Journal of Economic Literature 39 (2):390–431.

———. 2006. “Scale Effects in Markets with Search.” The Economic Journal 116 (508):21–44.

Reagans, R., E. Zuckerman, and B. McEvily. 2004. “How to make the team: Social networks vs. demographyas criteria for designing effective teams.” Administrative Science Quarterly 49:101–133.

Resnick, Paul and Richard Zeckhauser. 2002. “Trust among strangers in Internet transactions: Empiricalanalysis of eBay’s reputation system.” Advances in Applied Microeconomics 11:127–157.

Resnick, Paul, Richard Zeckhauser, John Swanson, and Kate Lockwood. 2006. “The value of reputation oneBay: A controlled experiment.” Experimental Economics 9 (2):79–101.

Rosen, Sherwin. 1981. “The economics of superstars.” American Economic Review 71 (5):845–858.

Rosenkopf, Lori and Paul Almeida. 2003. “Overcoming local search through alliances and mobility.” Man-agement science 49 (6):751–766.

Roth, Alvin E. 2002. “The Economist as Engineer: Game Theory, Experimentation, and Computation asTools for Design Economics.” Econometrica 70 (4):1341–1378.

Roth, Alvin E. and Elliott Peranson. 1999. “The Redesign of the Matching Market for American Physicians:Some Engineering Aspects of Economic Design.” American Economic Review 89 (4):748–780.

Rysman, Marc. 2009. “The economics of two-sided markets.” The Journal of Economic Perspectives23 (3):125–143.

Stahl, G. K, M. L. Maznevski, A. Voigt, and K. Jonsen. 2009. “Unraveling the Effects of Cultural Diversity inTeams: A Meta-Analysis of Research on Multicultural Work Groups.” Journal of International BusinessStudies 41:690–709.

Stanton, Christopher T. and Catherine Thomas. 2012. “Landing the First Job: The Value of Intermediariesin Online Hiring.” Working paper.

Stigler, George J. 1962. “Information in the Labor Market.” Journal of Political Economy 70 (5):94–105.

The Economist. 2010. “Work in the Digital Age: A Clouded Future.” .

The World Bank Group. 2011. “Data: Countries and Economies 2011.” .

The Economist . 2013. Online Labour Exchanges: The Workforce in the Cloud.

Tilcsik, Andras. 2011. “Pride and Prejudice: Employment Discrimination against Openly Gay Men in theUnited States1.” American Journal of Sociology 117 (2):586–626.

Tucker, Catherine and Juanjuan Zhang. 2007. “Long tail or steep tail? A field investigation into how onlinepopularity information affects the distribution of customer choices.” .

United Nations Development Programme. 2011. “UNDP Associate Administrator Concludes China Tripwith a Promise to Strengthen South-South Cooperation.” .

131

Varian, Hal R. 2007. “Position Auctions.” International Journal of Industrial Organization 25 (6):1163–1178.

Watson, W. E., K. Kumar, and L. K. Michaelsen. 1993. “Cultural Diversity’s Impact on Interaction Processand Performance: Comparing Homogeneous and Diverse Task Groups.” Academy of Management Journal36:590–602.

Wheeler, Christopher H. 2001. “Search, Sorting, and Urban Agglomeration.” Journal of Labor Economics19 (4):879–899.

Wilde, Louis L. 1981. “Information Costs, Duration of Search, and Turnover: Theory and Applications.”Journal of Political Economy 89 (6):1122–1141.

Wilsona, J. M., S. G. Strausb, and B. McEvily. 2006. “All in due time: The development of trust incomputer-mediated and face-to-face teams.” Organizational Behavior and Human Decision Processes99:16–33.

Zakaria, N., A. Amelinckx, and D. Wilemon. 2004. “Working Together Apart? Building a Knowledge-SharingCulture for Global Virtual Teams.” Creativity and Innovation Management 13:15–29.

Zentner, Alejandro, Michael D Smith, and Cuneyd Kaya. 2012. “Bricks, clicks, blockbusters, and long tails:How video rental patterns change as consumers move online.” Tech. rep., Mimeo, Carnegie Mellon.

132