Does randomly drawing the selection committee improve the quality of academic recruitment?

The project focuses on the effects of the last change in the rules of recruitment in Italian Universities. Starting from 2009 (DM27/03/09), the rules to appoint the recruiting committees were reformed switching from an election by members of the discipline to a random selection from the voting constituency. The rationale of the norm was to prevent  any ex-ante agreement on the would-be winner.

To check whether this reform made the difference, we plan to compare the final outcomes of the academic competition run after the reform to a comparable pool of competition run just before the reform. The comparison will be checked with respect to the quality of the winners as measured by their bibliometric indicator scores. It could be that the process is still not random and the people who participate to the competition are somehow correlated to the committees and for this reason it is crucial to see who are the candidates that do not take part into the competition although they had applied for it. In this setting the randomization might fail due to  self-selection of participants in the competition. Given the impossibility of collecting information before and after the reform from all Italian universities, we have selected 4 large universities (Milan, Padua, Rome and Naples) with a sufficiently large number and variety of local competitions in different fields. We restrict to competitions for full and associate professorships, neglecting those for assistant professorship because in most cases we do not have significant bibliometric measures of the applicants. We go to the final official verbal of each competition in order to obtain information on the pool of applicants; the selecting committee (distinguishing between the member appointed by the local schools and the others); existence of candidates that withdraw from the competition before its conclusion; the winners and the winner that is finally hired by the local school.