In the US House of Representatives, bill HR 3433 would add “grants transparency” to the review and awarding of federally funded grants and cooperative agreements. The bill would require publication of awarded grant proposals within 15 days of notice of award, agency actions to screen reviewers of grants for conflicts of interest, encourage merit-based review of awards, and the publication of the names and employers of reviewers of grants in the same grant program in the six months prior to each award.
There are problems (and more problems) with the way the present peer review system is managed by the federal government, and there is enough money in play–upwards of $60b–that it makes good sense to look for improvements and innovation.
One problem that has been argued is that while peer review of publication makes good sense, peer review of grants appears to benefit the established order–just *not* the thing that efforts at discovery and transformation would seek out. Terttu Luukkonen and Maria Nedeva write:
A conclusion of the above findings is that, despite the aims, there are definite limitations within peer review for promoting truly innovative research. These limitations arise from the very essence of peer review, namely, its ability to judge the value of proposed research against the current knowledge boundaries. The paper raises the question of whether public funders can through their accountability requirements fund truly innovative and radically new research or whether it is in the domain of private funders.
While in theory a peer review might encourage such exploration, in practice it appears that peer review tends to narrow the range of acceptable proposals, track grant work to support established research directions, and reward the directions favored by the majority of reviewers. Scoring systems that favor points accumulated or consensus would then be just the wrong thing–the more “likely” and “acceptable” the work, the less radical and challenging to the prevailing consensus the work will be. The result is a substantial reduction in the things studied, the methods used, and the challenges to the status quo of scientific work. Once string theory is established as the proper object of study, then everything is drawn toward string theory.
Thus, while review of award proposals is certainly called for, the way in which the review takes place is a legitimate subject for public debate and not something to be withdrawn into behind a curtain of technical expertise and scientific secrecy. As Paul Feyerabend once argued, perhaps so strongly for rhetorical effect, in science “anything goes.” An analysis of “proper methods” does not ensure that discoveries will be made, any more than by scoring how organized and well written a proposal is says anything much about the merits of the inquiry itself. While it may be advisable to disregard “kooks and cranks”–one has to be careful that one is not also throwing out radical science, science that challenges the status quo, science that works outside of institutional claims, and science that does not care about career academic credentials. Some of the most significant work in science began outside institutionalized science, the science that universities hire for, that national societies reward with membership. Federal grants programs would do well to ensure that the desire to review for merit does not end up gratifying only those with credentials and at the forefront of “mainstream” science. While mainstream efforts are worthy, they are hardly the exclusive domain of science, especially discovery science–transformative, paradigm-shifting, disruptive, career-threatening, equipment-challenging, investment-ruining science.
A second problem with the current peer review system is the abuse of the system itself. The recent revelations about climate science are illustrative. There, it is clear that leaders in the climate science establishment sought to prevent the participation of investigators that would challenge their methods, data, results, claims, and paradigm. These investigators have worked in close collaboration to ensure their work got funded and others’ work discredited and excluded. They served as publication reviewers for draft articles that challenged their own work, they forced journal editors to resign for publishing articles that they disagreed them, and refused to release their own primary data or software while advocating for the use of their publications as the basis for a substantial political agenda reaching to all segments of the economy. In short, their behavior was and is unethical, adverse to scientific debate, and much too self-interested with regard to competition for future funding and political influence. It is not clear how far into the peer review of government grants this behavior goes, but it is clear that no one will be able to get at it if the reviewers and their employers are not made public.
A number of organizations have written letters opposing HR 3433, among them a joint letter from AAU, APLU, and COGR–organizations that were on the losing side of the Stanford v Roche argument. These organizations in their letter, among other concerns, objected to both the publication of funded proposals and to the disclosure of peer reviewers. One would think that universities, in particular, would be in favor of publication and transparency, and would be concerned with regard to any kind of behavior that might threaten the public trust–and government confidence–in the review of proposals to be funded with public money. Why would the universities be so *against* publication and transparency in awards of public funding for basic scientific research, which is the speciality of university-based work?
Here is what the AAU, APLU, COGR letter has to say:
Section 7404 part (d) (2) (B) COPY OF PROPOSAL, APPLICATION, OR PLAN – Research proposals contain the ideas, procedures, and preliminary research findings of a scientist that make the case for the federal agency to fund a particular area of research that could result in significant breakthroughs. Posting full grant applications would make such ideas and preliminary results available to anyone,domestic and foreign, looking for a shortcut to further their own research or to steal intellectual property, thus undermining the hard work and intellectual capital the applicant and institution have already invested in the project. Moreover, under the newly reformed U.S. patent law (which allows for the first person or entity to file for a patent to receive it), making full grant proposals public will undermine the competitive position of the U.S. in one of the few arenas in which we still maintain a global competitive advantage. This section of the bill should be revised to require only the posting of abstracts.
It is quite breathtaking, as reasoning goes: a proposal may contain ideas that if published after the award could give someone else a “shortcut.” The apparent argument is that the “hard work and intellectual capital” of the proposers is not published, and that the work of science is not collaborative, but exists in order to win a string of federal awards, announcing as little as possible for the use of “just anyone.” This is just the wrong argument. Why would an institution make it? Could it be that universities fixated in trying to make money from faculty research are determined to prevent publication of ideas pending an effort to find a way to make people pay for the results? Is there not something of an organizational conflict of interest in this? It is as if not only do the institutions feel entitled to have all inventions that faculty make, but also to demand that the ideas brewing in their heads be sequestered in order to ensure that all inventions that faculty could make, or inspire, get made and inspired with a university ownership claim.
I understand: if a university were a company and its faculty employees who had given up their personal interests in favor of management direction and control, and if the money being used for the research came with no strings attached, then it would make perfect sense for management to go all protective about what its personnel were thinking and doing, limiting publication to the minimum necessary to document proper expenditure of the funds. But of course, none of these are true. A university is not a corporation, its faculty have not given up their independence, and the money coming in has strong public commitments associated with it, not the least of which are in Circular A-110, such as at 2 CFR 200.315 and 2 CFR 200.316.
The argument given by the AAU et al. amounts to a claim that universities should behave like, and be treated like corporations though they have few of the defining characteristics of corporations. The corporate characteristics, in fact, much as they might help to organization administration, would appear to be adverse to the broader imperative to direct federal funding to universities rather than to industry directly.
If universities merely adopt the methods of industry, then they are worse than industry when it comes to innovation. Universities lack the immediate connection to practical application, to product development plans, and the production and value chains that allow not only for action but also for meaningful review of design and direction. As corporate entities, there is next to no reason for the government to fund universities other than to circumvent WTO subsidy rules and to bring home regional pork for legislators.
The argument AAU et al. make about stealing intellectual property is difficult to understand. What could this intellectual property be? It can’t be an established form of intellectual property such as a patent, copyright, or trademark, since one can’t “steal” such things by reading a funded grant proposal. One would think that one has plenty of time from when a proposal is drafted to when it is funded to take care of business such as filing a patent application. What this argument really appears to be about is trying to treat scientific information as a trade secret for competitive advantage in securing more federal funding relative to other researchers.
If this is indeed the argument, then what is being called “stealing” is in fact what in other circumstances would be called “publishing” and “collaborating” and “scholarly exchange.” “Shortcuts” are in fact what publishing scientific information is all about. Preventing others from taking “shortcuts” is equivalent to “running up the overall costs of scientific progress.” The information that these organizations are worrying isn’t “intellectual property”–it is the work of scholarship. When one is applying for public funding, it’s difficult to understand how it is that the science you propose to use should remain unpublished. It is of course understandable how someone might want to hold back key stuff in order to win funding. It is understandable even that someone might think that getting funding is more important than publishing–but does that not raise concerns? And is it not of even greater concern that organizations representing universities, rather than individuals, are siding with the idea that funding is more important than publishing?
The applicant (a faculty member) may indeed have invested “hard work and intellectual capital” in the science that underlies a given proposal. It is not generally true, however, that universities can make the same claim for themselves–and yet that is what they do in the above argument, writing themselves in along side the “applicants.” What “hard work” do the administrators do that has anything to do with the content of the application? Whatever they have done, that’s what the indirect cost recoveries–typically 1/3 of the award amount–are for. The institution gets paid for its “hard work.” For all that, most publicly funded research is built on other publicly funded research, so if anyone has invested “hard work and intellectual capital” it is not the universities but the government, on behalf of the public. Making it appear that the public is served by keeping proposals and reviewers and their employers secret from the public makes no sense. It’s one thing to keep everything blind while a review is made, but another to leave the reviewers anonymous and withhold the grant proposals that are funded after their review. Even then, it is often easy for a reviewer to recognize the work of colleagues, and easier still for an applicant to leave markers in the proposal so that recognition is easy when desired.
Whatever the issues with the release of funded proposals and the names and employers of reviewers, it has nothing to do with the theft of intellectual property and worries that others might take “shortcuts” and have the benefit of what has been proposed, reviewed, and funded by the government. Not only is openness a check on problems in the review system, it also provides an opportunity to ascertain the directions of government, which, no matter how much we would like to believe are motivated by an unending pursuit of the truth based on empirical studies and reason, are going to be, in the end, politically motivated. Politics in and of itself is not bad–it is merely a fact of the behavior of government institutions. University administrators have figured this much out, at least–that it is easier to get money from a political organization by playing politics than by pushing science in all its craziness, and by secrecy rather than openness.
Finally, consider the argument for global competitiveness. The university administrators are living in a past that felt good–the US was the leader in science and everyone else waited with bated breath for the next publication to emerge. That isn’t how it is any more. If we are not publishing abstracts and proposals, then there is plenty of research in the world for others to rely on. Our lack publishing will take us further from the mainstream currents of inquiry in the world. That may be a rational position for universities seeking to maximize their government funding, but it is not the way to advance science or to find the ideas from science that will benefit community.
Pingback: Federal Sweep and Swamp Policy | Research Enterprise