Governing through Prizes and Challenges

With Andrew Young

Late last year, confronted with the rapidly spreading Ebola crisis, medical and development experts turned to an unusual source for solutions. Following President Obama’s call for new and innovative ways to fight the disease, USAID and its partners launched “a grand challenge” that would effectively open source ideas. The challenge, launched online at http://www.ebolagrandchallenge.net/, allowed virtually anyone in the world (not just experts) to submit ideas and join the conversation. The most promising ideas will eventually receive funding and other forms of support.

The Fighting Ebola challenge represents but one example of an increasingly common trend in which governments — and other entities — seek to solve big problems using new technologies and innovative approaches. The most common approaches involve grand challenges and prize-induced contests, both of which offer a range of advantages for government agencies, especially in an era of tightening budgets.

In 2010, President Obama, in his Strategy for American Innovation, called on all US government agencies to increase their use of prizes and challenges to address the most pressing problems facing the United States. Subsequently the Office of Management and Budget (OMB) issued a policy framework to guide agencies in using prizes to mobilize “American ingenuity,” while The General Services Administration (GSA) launched Challenge.gov, a one-stop shop where entrepreneurs and citizen-solvers have participated in hundreds of these public-sector prize competitions.

According to former U.S. Chief Technology Officer Todd Park, perhaps the greatest advantage of these more open approaches to problem solving comes from widening the pool of potential problem solvers beyond the “usual suspects.” In addition, prizes and challenges allow governments to establish ambitious goals without having to predict which individual, team or approach is most likely to succeed (thus reducing the riskiness of funding decisions at the outset), and to stimulate private-sector investment that is potentially much greater in value than the prize amount itself.

Although the realized and potential benefits of further leveraging prizes and challenges are being recognized to a greater degree, further inquiry is needed to optimize the design and deployment of grand challenges in the interest of maximizing impact.

What’s in a Name? Prizes versus Challenges

Prizes and challenges are similar in many ways, most notably in the way they shift the locus of innovation from inside government agencies to more public channels. There are, however, important differences between the two approaches — differences that are worth understanding and considering for any government agency (or other entity) seeking to address the major problems of our time.

Prize-induced contests generally rely on monetary prizes to inspire private-sector engagement with societal problems. In contests, Park notes, “you draw in unusual suspects along with the more usual suspects.” Incentives are, of course, critical to drawing in these unusual suspects. Monetary prizes can help spur private-sector action by offering a lump sum to winners of contests. These lump sums typically represent a return on investment for the winners, and use the psychology of gamification to incentivize previously untapped innovators and encourage them to engage with difficult problems.

Grand Challenges are similar to prizes in the way they tap into the wisdom and experience of “unusual suspects.” Unlike prizes, however, grand challenges are less focused on financial incentives (though many include a significant monetary component) than on spurring collaboration and participation to solve large, seemingly intractable public problems. Grand Challenges have been used by the United States government since at least the 1980s. They have received renewed attention recently, however, in part because technical tools for collaboration and co-operation have become so much more sophisticated.

One of the central goals of Grand Challenges is to foster cross-sectoral collaboration — for example, among actors from the private, non-profit and governmental sectors. Using modern technologies, Grand Challenges can also include a crowdsourcing component. By bringing together insight and experience from a variety of actors with different backgrounds, Grand Challenges can inspire out-of-the-box thinking and innovative approaches to problem solving.

Current Grand Challenges offered by the U.S. government include NASA’s “Asteroid Grand Challenge,” which seeks to identify and address all asteroid threats to the human population; the “Brain Initiative,” a collaboration between the National Institute of Health, DARPA and the National Science Foundation, which seeks to uncover new prevention and treatment methods for brain disorders like Alzheimer’s, autism and schizophrenia; and the Department of Energy’s “SunShot Grand Challenge,” which seeks to eliminate the cost disparity between solar energy and coal by the end of the decade.

Mapping what works and why?

As part of its efforts to map what is known regarding the way governance is being transformed, the GovLab identified a diversity of important, impactful work concerning prizes and challenges. The ongoing work of Harvard Business School’s Karim Lakhani and George Washington University’s Zoe Szajnfarber are notable examples. The following initiatives and resources offer a brief overview of current state of play.

One of the most comprehensive studies of grand challenges in the public sector can be found in “Investing in Research and Innovation for Grand Challenges,” a study commissioned to help guide the European Union’s use of such programs. The European Commission’s “A Rationale for Action” paper, part of the Europe 2020 Flagship Initiative, also discusses the role of grand challenges in solving large public problems.

A number of studies exist assessing the efficacy and usefulness of prize-induced contests. McKinsey has produced a report (“And the winner is…”),, which is focused on prizes in the philanthropic sector, but the majority of its findings can be applied to other sectors. Other reports include Heidi Williams’ “Innovation Inducement Prizes: Connecting Research to Policy” and Kevin C. Desouza’s “Challenge.gov: Using Competitions and Awards to Spur Innovation.” Joel Wooten, assistant professor at the University of South Carolina, has also studied prize-induced contests, focusing particularly on feedback to participants and the visibility of submissions.

Perhaps the most in-depth discussion of contests occurred in June 2012 at the Collaborative Innovation Summit on the use of public sector prizes to spur innovation in the federal government, hosted by the White House Office of Science and Technology Policy, NASA Center of Excellence for Collaborative Innovation, Case Foundation and Joyce Foundation. Similar conferences and workshops exist in more specific areas, like the IEEE Life Sciences Grand Challenges Conference, and outside the government realm, like the World Open Innovation Conference.

Seeking Answers to Questions…

In order to better understand the true impact — and potential — of prizes and challenges, a systematic research program will be required. The MacArthur Foundation Research Network on Opening Governance, a collection of 12 interdisciplinary experts studying the field, is canvassing a list of key questions we need to seek answering toward understanding innovation in governance better along with a variety of other activities. Some of the questions and issues we need to start answering include:

What metrics can help measure the impact of prize and challenges? What design variables enable successful implementation? By definition, prizes and challenges are focused on achieving audacious goals. We need a way to capture their impact and to understand if they have achieved their goals. We also need to understand what aspects of design affect outcomes. For example, could prizes and challenges be more effective if they gave participants more guidance on achieving step-by-step milestones? Is it practical for agencies to develop a timeline of sorts to help guide the progress of participants and stakeholders?

What kinds of participation and engagement are most likely to lead to desired outcomes? Despite grand challenges’ efforts to engage a variety of stakeholders, we need to consider the possibility that actors often work individually, with minimal or no coordination. What steps can we take to increase collaboration? In addition, what is the desirable size of entities involved in challenges and prizes? Prize-induced contests are often associated with small startups. Are the entities engaging with grand challenges similarly small and/or newly created, or are more established businesses and NGOs also likely to be working on challenges (and how does this affect outcomes)?

What measurable and concrete impact have prizes and challenges already had? A number of prizes and challenges already exist, and potentially have much to teach us. For example, we can study existing innovations to better understand what effects grand challenges have had on workforce development, job skills and labor participation. In addition, existing programs might offer lessons about the relationship between the private sector and government. For instance, are grand challenges being used as a means of government procurement? Do businesses engage with prizes and challenges as a type of audition for government contract work?

Are prizes and challenges the best way to fund research? In the absence of concrete evidence and clear metrics, the belief that prizes and challenges are the best way to solve public problems remains speculative at best. It is important to test this assumption carefully. Would governments be better served devoting a small amount of money to traditional R&D instead of largely relying on the private sector to deliver results? This is particularly important to ask given that results are likely to be incremental and take time to manifest. Are prizes and challenges, therefore, the best use of public funds, and how can they be designed to maximize the use of such funds?

What incentives are most likely to draw in “unusual suspects” and deliver desirable outcomes? While the incentive structure for prizes is relatively clear, some grand challenges do not offer monetary rewards, and others offer enormous prize purses toward which participants must devote significant time and resources to have any hope of winning. Governments (and other entities) therefore need to consider whether there are enough incentives to draw in private actors, especially considering the scope of the challenges and the financial and time commitments required. In addition, we need to consider the role of altruism (and the attendant positive media coverage) as an incentive: is it sufficient to encourage private sector entities to participate? Finally, we need to ask whether grand challenges are sometimes too grand: does their sheer scope occasionally overwhelm potential participants and act as a barrier to participation?

What is the best way to evaluate submissions for challenges and prizes? Prizes and challenges are typically evaluated by relatively conventional judging panels. But are these the best means of evaluating the often unconventional and audacious proposals that are being sought? If in fact judging panels are an adequate means of evaluation, then who should sit on the judging panels? Individuals or entities that evaluate a proposal act as critical gatekeepers; it is essential to better understand and optimize their role.

Turning Answers into Actionable Insights

Finding answers to these questions will not only be important to increase understanding of what works and what doesn’t (and why) but also to develop a set of actionable insights and deliverables. In particular, based upon our research to date, we need to work toward:

• The development of a set of guidelines and toolkits to allow practitioners to innovate and experiment with prizes and challenges in an evidence-based way that simultaneously feeds into a broader understanding of the field; and
• Targeted training for decision-makers inside and outside of government who stand to benefit from the informed, strategic deployment of challenges and prizes.

Stefaan Verhulst is the co-founder and chief of research and development of The GovLab at NYU. Andrew Young is the associate research director of The GovLab. The GovLab seeks to improve people’s lives by changing how we govern. More at http://thegovlab.org/