Jason Hartline

The Theory of Crowdsourcing

Jason Hartline
February 23, 2011

Open video window

Abstract: Crowdsourcing contests have been popularized by the Netflix challenge and websites like TopCoder and Taskcn.  What is crowdsourcing? 

Imagine you are designing a new web service, you have it all coded up, but the site looks bad because you haven't got any graphic design skills.  You could hire an artist to design your logo, or you could post the design task as a competition to crowdsourcing website Taskcn with a monetary reward of $100.  Contestants on Taskcn would then compete to produce the best logo.  You then select your favorite logo and award that contestant the $100 prize.  

In this talk, I discuss the theory of crowdsourcing contests.  First, I will show how to model crowdsourcing contests using auction theory.  Second, I will discuss how to solve for contestant strategies.  I.e., suppose you were entering such a programming contest on TopCoder, how much work should you do on your entry to optimize your gains from winning less the cost of doing the work?  Finally, I will discuss inefficiency from the fact that the effort of losing contestants is wasted (e.g., every contestant has to do work to design a logo, but you only value your favorite logo).  I will show that this wasted effort is at most half of the total amount of effort.  A consequence is that crowdsourcing is approximately as efficient a means of procurement as conventional methods (e.g., auctions or negotiations).

Bio: Prof. Hartline as been a member of the EECS faculty since 2008.  At northwestern he teaches undergraduate courses on algorithms and data structures, and graduate courses on mechanism design and approximation.  Before joining Northwestern he was a researcher at Microsoft Research in Silicon Valley where he worked on auction theory and its application to selling advertisements on Internet search engines.  His Ph.D. is in computer science from University of Washington, Seattle.