"Quality not quantity !" The German Research Foundation refuses bibliometrics as assessment criteria
le 2 mars 2010
The Deusche Forschung Gemeinschaft (German Research Foundation) in charge with research assessment and funding in Germany has published a Press Release which points out the dangers of bibliometrics. Wil other countries understand these dangers ?
Press Release No. 7 | 23. Februar 2010 “Quality not Quantity” – DFG Adopts Rules to Counter the Flood of Publications in Research
Funding Proposals and Final Reports to Include Fewer Publication Citations / Kleiner : “It’s the content that matters”
With the motto “Quality not quantity”, the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) has implemented new measures to counter the flood of publications in research. Germany’s central research funding organisation today presented new regulations that will take effect on 1 July of this year regarding the number of publications that can be cited in funding proposals and final reports. This measure seeks to assure that in future, researchers submitting proposals and reports to the DFG only include a limited number of particularly significant publications, rather than an arbitrary number, thereby reducing the importance placed on publication lists and numerical indices. At the same time, more emphasis should be given to the actual description of the research project. “By doing this, we want to demonstrate that content matters more to us when evaluating and funding research,” said DFG President Professor Matthias Kleiner.
The new regulations were recently decided upon by the DFG Senate at the suggestion of the Executive Committee and subsequently agreed by the Joint Committee. They apply to two central areas in funding proposals and final reports : the publication lists of applicants included on CVs, and references to publications that directly relate to the research project.
On their scientific curriculum vitas, applicants will be permitted to cite a maximum of five publications. “They should be the five that applicants consider to be the most important among their scientific work,” emphasised DFG President Kleiner. With regard to publications with direct relevance to the respective project, only two publications should be cited per funding year. A researcher applying for individual grant funding for three years may therefore cite up to six publications. Should more than one applicant be involved, up to three publications may be listed per year.
In all cases, reference should only be made to work that has either been published or where publication is imminent. In the case of the latter, papers should be submitted in manuscript form, together with confirmation of acceptance by the publisher. Manuscripts that have been submitted for publication but not yet accepted may no longer be listed.
Through restricting the number of publication citations, the main content of the proposal, describing the applicants’ research objectives and preliminary work, is once again highlighted. This main part should be comprehensible in itself and serve as the basis for the review.
With these regulations, the DFG wants to counteract the quantitative factors that have been increasing for years in terms of research publications. “Whether in performance-based funding allocations, postdoctoral qualifications, appointments, or reviewing funding proposals, increasing importance has been given to numerical indicators such as the H-index and the impact factor. The focus has not been on what research someone has done but rather how many papers have been published and where. This puts extreme pressure upon researchers to publish as much as possible and sometimes leads to cases of scientific misconduct in which incorrect statements are provided concerning the status of a publication. This is not in the interest of science,” stressed the DFG President.
The cases of misconduct did not play a decisive role in initiating the new regulations ; thought had already been given to this in the past and clearly had wider implications, said Kleiner, who also made reference to similar rules in other countries. For example, in the USA, proposals submitted to the National Science Foundation were only permitted to cite five publications as a reference to the proposed project with up to five further publications whereby the National Institutes of Health permitted a total of 15 publications.
The new regulations will change the work of many thousands of researchers, in the view of the DFG. The DFG currently receives over 20,000 proposals annually. “In preparing their proposals, researchers must be much more selective in the future and limit themselves,” said Kleiner. Decreasing the number of publication citations will also impact reviewers, enabling them to assess even more intensively the applicants’ scientific careers and research projects.
“All of this will benefit both research and our funding work, even if the new regulations will not be able to be implemented overnight and a certain amount of rethinking and convincing will be required until we have achieved our goals,” concluded Kleiner. The DFG will promote acceptance of the new regulations and insist upon their compliance. “Researchers who continue to cite 50 publications instead of five will have their proposals returned with a friendly but clear request for revision,” said the DFG President. “However, we hope that we will not have to revert to the ‘return to sender’ option too often.”