Peer Review in the Age of Automated Scholarship

I have personally been on both sides of the peer review process, in more than one field. I thus know something of not just its virtues but its limitations and vulnerabilities firsthand. Altogether it seems to me rather a fragile thing, belonging within an earlier, slower-paced era in which the academic world was smaller and "clubbier." (Indeed, on more than one occasion reading my peer reviewer's comments I made guesses about who wrote the feedback--on one occasion having the guess confirmed as correct.)

In an era of chatbot-powered paper mills supplying a much more thoroughly global community of scholars living by "publish or perish" rules content to submit to the explosion of increasingly fee-charging journals that system seems too fragile to survive, as, indeed, absurdities break through its barriers to deluge us all in even worse nonsense than before. Much as elitists whine about the openness of the Internet (which I think a good thing on the whole, of which we unfortunately have had less these many years), the fact remains that even at its worst what we had on that open Internet was at least humans putting up content, rather than very incomplete artificial intelligences slapping words together; and those who put together an academic journal adhered to some minimum standard, especially because they expected subscribers, not authors, to pay for that journal's operation out of finite resources, which meant there was some filtration, far from perfect, but not wholly unhelpful, with all it means for even those journals where peer-review will still be effective (even their efforts tarnished).

I suspect the only real solution will mean effectively automating the examination of all this material--helping us weed out what is illegitimate--in step with the automation of its production. However, just as it seems that the effectual editing of content is a much higher-level skill than just churning stuff out, any such adaptation lies a long way away, leaving us making do as best we can with our old devices in the meantime.