Give Chance a Chance – a Lottery Decides Which Daring Research Ideas Receive Funding

When deciding on the eligibility of project applications, strict formal and technical specifications play a role, and the projects are also subjected to careful examination by experts. However, it is sometimes difficult for exceptionally creative research projects to survive this selection process because their full potential is not always perceived. In order to give such hidden pearls an equal chance, the Volkswagen Foundation is taking a new path. Part of the funding available will be decided on the basis of a lottery. What is the background to this experiment and what experiences has the Foundation made with it so far?

Science funding organizations place very different demands on their applicants. For example, some of them focus more on projects with an interdisciplinary focus, international cooperation or, as in the case of the Volkswagen Foundation, researchers who explore or want to explore research fields beyond the mainstream. In its “Experiment!” funding initiative, the Foundation invites applicants to submit proposals that pursue bold research ideas that fundamentally challenge established knowledge, that seek to establish unconventional hypotheses, methods, or technologies, or explore entirely new directions of research.

Although the funding amount of 120,000 euro and the funding period of up to 18 months may appear rather “small” compared to the Foundation’s other initiatives, the application requirements are comparatively low-threshold. On the one hand, a short outline application with a maximum of a thousand words is sufficient. On the other hand, the selection is done anonymously so that unknown early-career researchers have just as great an opportunity as established researchers – possible personal connections to the reviewers or knowledge about preparatory work done by the applicants have no influence on the selection itself.

These criteria leave a lot of scope for applicants. The unusual offer means that totally extraordinary or at first glance even outlandish ideas also have a chance. The result has been that the Foundation received several hundred proposals by each of the five deadlines so far – peaking at 704. Selecting 15 to 20 applications from such a large number, i.e. only around two to three percent which can eventually be funded, confronts the Foundation and the jury with an enormous challenge – not only in terms of time and personnel, but also in terms of content.

In addition, in the past, not all the topics submitted – which can come from all areas of the natural, engineering and life sciences – could be covered by the jury members. This is because the size of the jury naturally has to be limited, usually to a group of about ten people. Some niche subjects were therefore underrepresented and sometimes found no strong advocate among the reviewers. Some ideas appeared in similar form in several applications, and some applications were so controversial that it was hardly possible to reach a clear decision for or against. Although these are all known problems of the peer review process, alternative solutions to the dilemma have so far not been used to any significant extent by the Foundation or elsewhere by international funding organizations.

The Volkswagen Foundation first experimented with a lottery procedure for the “Experiment!” selection procedure in the autumn of 2017, putting the review process that has been practiced for decades to the test. Out of the 594 applications received, 119 were shortlisted. They corresponded formally and conceptually to the given criteria and were presented to the eight-member jury for discussion – whose expert knowledge remains indispensable for the quality assurance in the final selection.

At the meeting, the jury was able to agree on the direct approval of 17 applications, with one expert drawing his “wild card” – a kind of joker – and thus able to enforce a motion against the vote of the others. In addition, the jury members sorted out 34 applications that, despite the pre-selection, did not meet the required criteria. Then began the “new” part – all remaining 85 applications went into the lottery drum. A total of 17 numbers were drawn from the drum, analogous to the 17 previously selected by the jury. Among them were five duplications of outline proposals that had already been selected by the panel of experts – resulting in the bottom line of 29 approvals. The draw was decided completely impartially and free from unconscious bias, both in terms of research fields as well as factors such as age or gender. And since the number of possible allocations had been doubled in advance by the Foundation, the applications favored by the jury are being approved in the same measure as in previous years.

The Foundation therefore looks forward to more diversity among its funding allocations through the introduction of the semi-randomized procedure. Although it is also very satisfied with the projects selected in the previous rounds – some subsequently having raised large sums of funding to significantly expand the initial research – an increase in the range of subjects was nevertheless an expressed wish within the funding department. Moreover, the process serves to relieve the jury members, who are often torn in the decision-making process as to which application gets the last free place and which fails to make it by a wafer-thin margin.

However, the Foundation is not able to draw on any further experience or comparative values. Worldwide, the number of funding organizations that use a randomized or partially randomized selection process can currently be counted on the fingers of one hand. Only very few serious attempts, such as those undertaken by the Health Research Council of New Zealand, are known to date – although in this case, for example, only between three and eleven funding grants were determined by lottery each year. In order to be able to make robust statements on the basis of a sufficient amount of data, the Foundation will continue to try out the lottery procedure in the next three calls for proposals of “Experiment!”. In addition, with the help of accompanying research, it will exhaustively examine different aspects of the process: The selected and the completed projects will be compared with regard to their results as well as their quality, possible publications, and similar factors. A particular challenge here is the heterogeneity among the grants, i.e. their wide range of expertise, the promotion of young researchers as well as established professors, and the conception of individual and cooperation projects. The interest being shown by other institutions in this pioneering experience is already considerable.