> First we need to understand what we want to solve. The attention economy and PR war goes on just as much on the open internet. E.g. people only reading papers from big shots, MIT, Stanford, Harvard labs etc.
Well, I think there's two separate problems:
1. Academic journals not publishing "boring" results, which incentivizes scientists to bias toward novelty and not publish null results. I think the solution to this is for journals to accept based on methodology and subject matter of experiments/studies, and to commit to publish before the study is even done, so there's no chance that results can influence the decision to publish.
2. Science journalism being written by people unqualified to analyze the science who instead sensationalize for clicks.
I'm not particularly concerned that the people are "only reading papers from big shots". The problem is that prestige is measured by impact analysis rather than replication. If you incentivize a metric which is largely a popularity contest for ideas, it should be no surprise that what you get is popular lies[1]. One might think that the focus on impact in prestigious journals means there might be higher replication in less prestigious journals, but from what I can tell, the less prestigious journals are no better: they're mostly targeting the same metrics, they're just less successful at reaching their goals.
[1] My intent is not to actually accusing anyone of intent to deceive here--I just can't quite figure out the right word. Untruths doesn't quite match what I'm trying to say.
Well, I think there's two separate problems:
1. Academic journals not publishing "boring" results, which incentivizes scientists to bias toward novelty and not publish null results. I think the solution to this is for journals to accept based on methodology and subject matter of experiments/studies, and to commit to publish before the study is even done, so there's no chance that results can influence the decision to publish.
2. Science journalism being written by people unqualified to analyze the science who instead sensationalize for clicks.
I'm not particularly concerned that the people are "only reading papers from big shots". The problem is that prestige is measured by impact analysis rather than replication. If you incentivize a metric which is largely a popularity contest for ideas, it should be no surprise that what you get is popular lies[1]. One might think that the focus on impact in prestigious journals means there might be higher replication in less prestigious journals, but from what I can tell, the less prestigious journals are no better: they're mostly targeting the same metrics, they're just less successful at reaching their goals.
[1] My intent is not to actually accusing anyone of intent to deceive here--I just can't quite figure out the right word. Untruths doesn't quite match what I'm trying to say.