We sat down with him to ask what convinced a working scientist to get involved with an AI review company, and where he thinks this could take peer review next. 

‍

What’s your relationship with QED Science?

Technically I have a consulting contract, but "consultant" undersells it a bit. In the usual sense, consulting means a business sends you something, you comment, you send it back. That's not really what this is. Every week I spend a day in the office with the team. I sit in on meetings, I talk to everybody, I hang out. I'm not an employee, but I'm not really an outside consultant either. I have a good view of the whole picture. 

At the same time, I have a day job. I'm a professor, I run a lab, I write my own papers and grants. My life doesn't depend on QED Science the way it would if I were an employee. That's actually what makes my position useful, I think. I'm close enough to see everything, but I'm not relying on this for my livelihood so I can point out the issues in a more impartial way. 

‍

What made you want to get involved in an AI tool for scientific discovery?

I'd had some version of the same conversation with a lot of scientists over the years, about how much better review could be if it were less biased, less afraid of being critical, more focused on the actual science instead of everything around it. Then I heard about QED Science, an AI system built to give researchers real, expert-level feedback on their work, and it was the first time I'd actually seen someone building that instead of just talking about it.

I knew pretty quickly I wanted to be part of it. It wasn't a hard decision.

‍

What’s your view of the current scientific publishing system?

This question goes right to the heart of how science actually gets done. At the end of the day, publishing determines what position you can get, what gets funded, who gets promoted. Those are the incentives that drive people doing science, so the publishing system ends up shaping the science itself. That's how influential it is.

And yet within that system, there are two big problems. The first is bias. This has been shown in study after study, it's a system where the strong get stronger and the rich get richer. It's conservative by design, and it punishes work that challenges the status quo. The people already in positions of power have an interest in keeping things the way they are. Planck’s principle has never been more true, that science advances one funeral at a time. Try doing something genuinely innovative and see how easy it is to get it into a major journal. The big journals have a stake in maintaining the field as it is, they're not exactly looking to fund their own disruption. And it's often the same people sitting on grant panels, so a truly novel idea has just as hard a time getting funded as it does getting published.

The second problem is time. It's completely normal for a postdoc to spend four or five years in a lab and still be racing the clock to get their story out. That's gotten worse over the last twenty or thirty years, because high-impact papers now demand more and more data before they're considered a complete story. So it takes longer and longer just to be ready to publish.

‍

How does this impact the scientists?

What often happens is someone gets to the end of their PhD or postdoc with a paper that's ready to submit. Then comes the revision stage, and that can drag on for years. For a high-impact paper, that's not unusual at all. Which means people either have to extend what was supposed to be a short, defined position (a PhD is meant to be around four years), or they're stuck.

In a lot of European systems, a postdoc position legally can't run longer than a few years. It's built as a temporary role, and the benefits and everything around it are designed for someone passing through, not for someone trying to build a life. You want to start a family, you want enough money to actually live, and the system just isn't built for that. We're asking talented people, the people we want doing science, to accept conditions that are far from optimal, for years longer than they should have to.

The revision process itself is why it drags. You can think your paper is done, post it on bioRxiv with no review at all, and then you're still waiting to find a journal willing to send it out for review, waiting for reviewer comments to come back, and then spending time answering them. That whole cycle is where the time really gets lost.

‍

How do you see QED Science changing that status quo?

If the vision actually plays out the way I hope, it looks something like this: You post your paper on bioRxiv, and right next to it sits a review, not months later, right away. That's the first piece. The second is you post the QED Score alongside it. Now people already know where the issues are, and just as importantly, you know how your paper stacks up against everything else out there.

Today the signal we all rely on is the journal impact factor. Some may say we need some kind of ranking system, and I'm not against that in principle. But the problem is that impact factor doesn't actually rank your paper. It ranks the club that let you in. And that acceptance process is deeply biased and inconsistent, we already talked about that. A machine, by contrast, is supposed to be consistent. Standardized. It should score the same paper the same way regardless of who wrote it or where they're from. That's the whole premise. And QED Science goes to a lot of trouble to actually check and prove that consistency, not just claim it. I’ve been impressed by the lengths the team goes to in validating with experts what their data scientists came up with. 

‍

For someone who's never heard of it, what does QED Science actually do?

At its core, it reads a paper the way a good reviewer would, except it's not writing you a paragraph of prose telling you its impressions. It breaks the paper down into its actual claims, what is this paper really asserting, and then for each one it looks at whether the evidence in the paper actually backs it up and builds a comprehensive claim tree. You can look at it and see, this claim is solid, this one has a gap, here's exactly where the logic doesn't hold.

On top of that it gives you a score, the QED Score, which is meant to capture two things: how original the work is, and how valid it is, meaning does the evidence actually support what's being claimed. And the whole thing is anonymized before it's scored, no names, no institutions, so you're not getting a score influenced by whose lab it came out of.

Beyond QED Score, there's a Grant Review tool that asks the same kind of questions but for a grant proposal; will these experiments actually answer what you're asking? Does your preliminary data support the idea? And there's a Reader, which learns what you're interested in from what you've uploaded and recommends papers, including preprints you'd never have found otherwise. So it's not one product, it's really a few different applications of a single underlying idea: judge the science on its own terms.