Mmmm nutritions

The idea of a nutrition label on food is well established. In the EU they say how much of various nutrients and food components are in a fixed amount of the product and how much of it it is recommended one should eat in day. With this information in hand, one can wander round the supermarket for hours unable to make a decision, slightly horrified by how much sugar is in even the blandest, least digestible of cereals¹. One might play top trumps, trying to find the packet that contains the most phosphorous. Or of course, one can also use the data to make informed purchases² that add up to a healthy and wholesome diet. That’s the theory anyway.

There is a movement afoot to provide similar labels for academic papers (there’s a more detailed paper too³). Academic papers typically lack substance and nutrients in the traditional, physical sense⁴, so the information provided is slightly different. Also, the aim isn’t to help banish avoidable malnutrition, but to provide the information general readers need to distinguish information from its mis or dis forms as well as to help them to understand and navigate some of the fundamentals of scientific publishing⁵. As scientists, it can be easy to take for granted the basic signs that a publisher is a reputable one and the information that notionally undergirds the integrity of the peer-reviewed literature. There are four suggested components of the label for the journal:

1. Publisher
2. Editorial team
3. Percentage of articles accepted
4. Where the journal is indexed (e.g. web of science, google scholar)

And four for the paper itself:

1. Number of peer reviewers
2. Whether authors have competing interests
3. Data availability
4. Funders

Such information is usually available from reputable publishers, though it’s not always easy to find. As a scientist myself, I would quibble with the usefulness of some of these fields, but it’s nonetheless helpful to have them gathered in one place.

Figure from the paper, Willinsky, J. and Pimentel, D. (2024), The publication facts label: A public and professional guide for research articles. Learned Publishing, 37: 139-146. https://doi.org/10.1002/leap.1599. CC-BY-4.0

The proposed design (above) is based on the look of actual nutritional labelling. The clean black and white design⁶ has been modified somewhat for the internet. On the left we have information from the article and on the right, some kind of average. I suppose the analogy is to Recommended Daily Amounts, but of course that doesn’t quite work here. Indeed, as skeuomorphic interfaces go it’s a bit busy. The many pale blue ‘i’s are awkward and the green “iD” logo probably doesn’t mean a thing to most people. It’s also going to sit awkwardly on the web page of most journals⁷.

Grumbles about the design aside, it would be useful to standardise this kind of information as metadata, which the journal can then display how it wishes, and which a browser add on (or similar) might be able to expose in a useful standardised way alongside other standard information like the paper title, author names and affiliations. Much of this information might already be available. If you view the page source for a journal article, there’s usually a large amount of metadata already, in standard forms. Well-structure metadata ought to say which journal and the metadata for the journal ought to say which publisher… and so on. There’s no problem, usually, to adding more.

As I’m not really the target audience here, I do wonder how effective or useful it might be and I’m not entirely sure what to make of some of the elements myself. Is a high acceptance rate a good thing or a bad thing? Nature rejects a lot of papers, but it also publishes some absolute nonsense. What does it matter if a paper has 5 peer reviewers if the peer reviews all say something like “Paper is good, but too long⁸“? It’s not sufficient to know that data are available. That can mean almost anything. What data are available and are they sufficient to reproduce the analysis are more pertinent but harder to check? Some journals have huge editorial teams and it can often be important exactly who edited a particular paper. An important part of the integrity of the scientific process is that authors and editors put their names to their work⁹.

Some of the amendments to the base design are supposed to guard against journals gaming some of these stats, leaning heavily on Orcid for establishing identities and relationships between people, positions (author/editor/reviewer), journals and so on. Where it is weakest – on peer review – is where publishing is generally weakest. Predatory¹⁰ journals can get away with being predatory (for a while at least) because they can cheaply mimic most of what constitutes scientific publishing. The thing that has, traditionally, been hard to copy is the peer review itself¹¹. By hiding that process away – anonymising referees and keeping the reviews themselves in a locked drawer – some of the most important information is lost: was the person who reviewed the paper qualified and did they do a reasonable job? Similarly, there’s no entry for computer codes. While this might not be relevant in some disciplines, it will be in others. Working code is also difficult to fake convincingly.

Anyway. It’s an interesting idea and prompts one to think of the heuristics that one uses as a scientist when sifting the mountains of information that are out there.

-fin-

1. Seriously. Go find something that seems terribly austere, something that claims to be made out of rye husks and sawdust. It will contain more sugar than Frosties and it won’t even be grrreat. ↩︎
2. As teenagers what we mostly wanted to know was how many units of alcohol per pound were in the drink we were trying to buy in the one shop in town where the shopkeeper was reliably bad at judging age. ↩︎
3. With a 1925 reference to Virginia Woolf and so many footnotes. ↩︎
4. Unless you still consume your journals in hard copy. In which case, it’s mostly fibre and whatever goes into the ink. ↩︎
5. Perhaps it’s to banish malinformation? One can go too far with an analogy. ↩︎
6. The design is described as “lauded” and a “masterpiece“, and also that it “was first introduced in 1993 by the U.S. Food and Drug Administration, represented a recognition of consumers’ ‘right to be informed’, which became a prominent theme in the 1960’s“.  ↩︎
7. Although this is standard for most journal web design. The online journal is the most 19th century thing on the web although they have generated some innovations that are, in many ways, a step backwards that wouldn’t have been possible with 19th C tech: figures that fill the screen and obscure the related text but still manage to be illegible, popups that allow you to annotate the document but only once you sign in (there is no way to get back to the paper from the sign in page), free-form supplementary information that contains most of the important information, etc. ↩︎
8. An actual and complete review I saw on a paper I was reviewing. It was, I should add, lest you think the reviewer was in complete dereliction of their duties, rendered in two different colours and three different fonts suggesting an artistic dimension to reviewing that I had not hitherto considered and had therefore utterly neglected in the seven pages of my own review. ↩︎
9. Popper rather charmingly believed that we’d reached a stage where our ideas could live or die in our stead. Gone were the days when you vanquished an idea by killing everyone who thought it. Then came publish or perish and all the wonderful things that go with that. ↩︎
10. I think predation is the wrong metaphor here. The journals work by a combination of parasitism, mimicry and symbiosis. The notion that the journals prey on researchers suggests that researchers don’t themselves get something out of the process; they do and enough to sustain an industry. Someone somewhere is losing out, but it’s not always easy to say who or how. By framing the situation in these terms, it might help to identify solutions. Mimics use mimicry because it copies the cheap signals of expensive machinery. “Predatory” sounds more exciting though. ↩︎
11. Of course LLMs have come along and buggered that up to. It’s now possible to write reams and reams that look like reasonable reviews to the uninitiated. To ruin the system, it doesn’t have to be wrong or intentional or even well done, it just has to exhaust our capacity to assess it. ↩︎