I generally agree, but their solution (fund boring research, publish only in journals with high standard) is in direct contrast to what they stated earlier. It's basically saying yes, we have bad incentives, but we could ignore them. That's not going to happen; shiny new research will attract people and funding. Especially more than "boring" what we found before was indeed a finding-research.
Now, I don't have a good solution either, unfortunately. What might work is that we require replication work for a PhD or have a certain percentage of a journal dedicated to verification. That, combined with some meta-studies to reward people with citations for replication, might work without fully swimming against the current.
I think a lot of scientists would actually do "boring research". (a) they actually know such work is useful (b) some know that they don't produce work as good as the best in the field, but are happy to do more grunt work at lower stress level, rather than endlessly chasing "high impact" publications.
Unfortunately, there is no funding for such research. Which is find really sad. Private grants might need to show "impact" but State run grants don't have that many constraints, and they could conceivably offer such grants.
I'd happily have done that grunt work, back when I was in a lab. The work may not have been interesting but at least I could feel as if I was being useful whilst also paying the mortgage.
Instead, I've ended up writing software which is related to assisting journals in the task of getting researchers to share their data.
No problem.
This article might be of some interest:
https://www.go-fair.org/fair-principles/
There are, of course, political challenges in persuading researchers to share data, primarily the fear of other researchers using those data for their own papers or to secure funding, depriving the originator of the work of the chance to do the same. I hear this at conferences quite often.
The part I'm working on is at attempt to catalogue the places were research data may be stored and the requirements various journals and repositories impose on researchers who want to deposit data there. Gathering this information, curating it and making it easily accessible is not trivial, and relies upon a lot of manual curation by specialist "knowledge engineers".
> What might work is that we require replication work for a PhD
I don't think this will work. All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers.
Also, we have to get away from the idea that the scientist's job is to think and write, and literally all of the other work can be shuffled off onto low wage (or no wage), low status workers. This is one of the biggest reasons that science is going through such a crisis. If you want enough papers to consistently get grants you probably need at least 4/5 PHD students every few years. This causes a massive glut in the job market. It also dissociates scientists from their work. I've met esteemed computational biologists who could barely work a computer. All of their code was written, run, and analyzed by graduate students or post docs. They were competent enough at statistics, but that level of abstraction from the actual work is troubling.
Requiring replication work for a PhD seems like a great idea. PhD programs already use a mandatory exercise—the qualifying exam—to check a student's competence, with ambiguous effectiveness. Turning the qualifying exam into a replication study seems like a win: it tests the student's ability to do their actual job rather than pass an abstract test, and produces output that is useful both to the student and the community. The qualifying exam committee (usually ~ 4 PIs from different labs) can do quality control on the replication.
> All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers.
Most studies are done by students regardless, so it seems unlikely that replication studies would be devalued merely because they're done by students. Although disputing the findings of established researchers can be risky, they would be publishing jointly with their PI (or, with the above implementation, multiple PIs), not alone with no support. Few students want to stay in academia, so it usually doesn't matter to them if a professor at some other institution gets offended. Most importantly, if everyone is doing replication studies, there will be so many disputations flying around that any particular person is less likely to be singled out for retaliation.
It sounds like what you're suggesting would be functionally equivalent to PI-led replications, which I would agree is a good idea. There are still some practical problems though.
1. Studies can be much more expensive than most people think. In my field, a moderately sized study can easily cost $100,000+ if you're only accounting for up front cost (e.g. use of equipment, compensating participants). Someone would have to foot the costs of this.
2. Studies can be incredibly labor-intensive. PI's can get away with running studies that require thousands of man-hours because they have a captive market of PHD students, Post-docs, and research assistants all willing to work for low wages or for free. PHD students usually don't have the same amount of man-power.
3. For obvious reasons, studies that require high cost, high man-power work tend to get replicated naturally less. In other words, the least practical studies to replicate happen to also be the most necessary to replicate.
A couple of things I would dispute:
> it seems unlikely that replication studies would be devalued merely because they're done by students
I think academics value work in a particularly skewed way. There is "grant work" and there is "grunt work". Grant work is anything that actively contributes to getting grants for one's institution. Grunt work is everything else. PHD's can do grunt work, but that doesn't mean it will be valued on the job market. For example, software development is actively sought after in (biology) grad students, because it's a very useful skill. However, I've also seen it count against applications as professors because it shows they spent too much time on "grunt work". Software development skills don't win grants.
> Few students want to stay in academia
In some fields there aren't any options except to stay in academia or academia adjacent fields.
Tyler Cowen writes [0] that the most important question in economics, to him, is
> how do differences of culture — however defined — interact with traditional economic mechanisms involving prices, incomes, and simple comparative statics? Are those competing explanations, namely cultural vs. economic?
Richie's answers are mostly focused on changing the culture of science, and while there are lots of ways we could change the incentives, none of them would be pretty.
Example: let's say we want to more closely align research and actionable results, e.g., a product a company can use (Brian Armstrong argues for something like this [1]).
Solution: radically reduce public funding for scientific research and for university education as a whole (in line with Bryan Caplan's arguments in "The Case Against Education" [2]). Academics, who would be many fewer in number, would then have to get more of their funding from companies, who (presumably) would:
A) guide them towards asking market-relevant questions, and
B) have a clear incentive to check the data, re-run the code, etc. -- so that the product they built based on that research didn't flop.
I think most people would recoil at this proposal. But that's what comes to mind when I think about fixing the incentives rather than the culture.
P.S. Small nitpick: Richie gives an _example_ of a perverse incentive in lieu of a definition.
We tried that and got the smoking is not bad for you science of the '70s, sugar is not bad for you science of the '80s, and the fossil fuels aren't causing global warming of today. How many drugs have been "proved" safe for human consumption because it is better for the bottom line?
Science beholden to business just proves whatever is convenent for business owners not the truth.
If you think funding from industry makes academics' results more robust, you might want to look into tobacco-industry-funded research on cancer, oil-industry-funded research on global warming, and ridesharing-industry-funded research on drivers' working conditions.
No one thinks "research on drivers' working conditions" is science. Manifestly self-interested nonsense like that is relatively easily ignored. And if that's the trade we make to get Bell Labs, Xerox Parc and IBM Research, I'll take it.
Some, perhaps. Others would likely choose different careers, or would adapt.
FWIW, and I didn't clarify this enough in my post -- I meant this as an example solution of something that would change the incentives rather than the culture, not necessarily as my own full-throated endorsement; I do personally think that steps in this direction would be for the best, but it's not as though there wouldn't be downsides that would need to be managed/mitigated.
A more radical solution: Basic income for all researchers and provide efficient digital way to see how funds are being spent like open collective for open source. Most problems arise at an individual level where people don't get paid unless they are putting research papers on the new hype thing and the other is expenditure on middlemen and bureaucracy which isn't easily visible. Eliminate the use of research grants for funding people's living.
Bit pedantic but you should use they instead of he/she. People can have other pronouns.
IIRC, Hungary has implemented something like this and it's been floated as an idea, to distribute grant dollars as a fixed amount to everyone who has an H index or citation count above a certain amount.
When I've talked about this with colleagues, they've argued this would make problems worse, and it might, but I think the idea was more that once you reach a certain threshold you should be funded some amount.
Of course, I'd argue that this is basically the idea behind tenure at an R1 institution, that once you reach a certain level the state or private board of trustees is paying your salary to do research. But nowadays tenure at an R1 institution is proxy for external grant dollars, which defeats the purpose and is redundant.
The real problem is indirect funds, which create profit for institutions. The federal government needs to eliminate indirect funds, so that grant dollars aren't perversely incentivised and tenure is based on research quality rather than profit.
>Bit pedantic but you should use they instead of he/she. People can have other pronouns.
One of those other pronouns is literally "they", so you're still risking using the wrong pronoun by using "they" for someone who prefers to be referred to as "he".
Was anyone else taught the rule that when writing about a third-party whose gender is unknown, use your own pronouns?
Oh, you mean tenure? UBI is a general thing for the whole population, and not something just for scientists. Tenure is in general a good, as long as scientists keep working, and there are some controls keeping them producing at least "something". And let's face it, most of them do. This is great, because it releaves scientists of pressure, which means more of them can be creative in their research, which actually speeds up discoveries. This is also a valid approach within tech and IT btw. But it should be said that it has a very poor track record outside of these knowledge-heavy professions.
On the other hand, if you really think UBI is a good solution, then I'm afraid you don't know how an economy really works.
It has been discussed to death, but for the sake of argument, let's review these two completely opposing factions, who both agree that UBI doesn't work:
Your second link mentions "universal basic services", I always supported that idea but didn't know it had a name. It's better than UBI because it sidesteps the problem of cost disease (landlords raising rents in response, etc).
I think they mean something like UBI: non- or less-competitive grants that would fund modest-sized projects (or pilot studies for larger ones).
Tenure doesn't really help with this problem in the lab sciences. While it lets faculty members keep their jobs, it usually doesn't come with enough funding to do experimental work. In fact, while you can "keep" a tenured position without grants, many places find ways to...discourage that (move your office to a shoebox in the sub-basement, crappy teaching and service assignments).
Now, I don't have a good solution either, unfortunately. What might work is that we require replication work for a PhD or have a certain percentage of a journal dedicated to verification. That, combined with some meta-studies to reward people with citations for replication, might work without fully swimming against the current.
It's a hard problem, really.