The Legibility Project: Reversing the dark economy of academic labor

Here is an example of the kind of registry I am thinking of, using my own activity as a starting point.

On-going duties include: Undergraduate Advisor European Studies Minor, Editor Cultural Analytics, Board Member Centre for Social and Cultural Data Science

Over the years I have become aware that a significant portion of my time is spent on tasks for which I am not directly paid, either in the form of money or public credit, and about which no one outside of my chair or dean is aware. I am talking about work known as “peer review.” Typically we associate this term with the reviewing of scientific articles. However, the scope of “peer review” is considerably larger than that understanding implies.

Peer review can encompass:

• Scholarly articles, the most familiar category
• but it can also, especially in the humanities, entail reviewing entire book manuscripts. If the average scholarly article is between five and seven thousand words, then the average academic book is anywhere between ten and twenty-four times as long (not to mention time-consuming to review). Sometimes, I will be asked to review a book proposal, which can be considerably shorter, between 20 and 75 pages.
• Promotion dossiers, either for tenure or full professor. These include publications produced over the course of a career. If someone has published several books and dozens of articles, then the time commitment is now potentially 100x the extent of reviewing a single academic article.
• Faculty recommendation letters. These entail knowledge of their entire scholarly output, which in some cases may be more than promotion to full professor if they have already been a full professor for a while.
• Grant or prize committees. A book prize committee can mean reading upwards of 100 scholarly monographs (i.e. the equivalent of 2,400 academic articles), while a grant committee can mean reading either a single proposal (about as long as an article and just as dense) or adjudicating up to 20-30 proposals at a time.

I should note that I have not included writing letters of recommendation for undergraduate and graduate students because I consider those to be part of my teaching and supervision, for which I am directly paid. Nor have I included my own research writing, again because my assumption is that I am directly paid for this work.

These tasks will be familiar to anyone in the profession. They are almost entirely unknown to those outside of it.

Some might say, ah, come on, that’s part of your job, too. You may not have known about it when you started out, but in addition to teaching, advising, mentoring, researching, writing, and sitting on committees, it was implied that you would also be doing a lot of reviewing of other people’s work. After all, how do you think your articles and books get published? Someone’s got to do it.

Absolutely. But the bigger issue for me is that these activities are almost all confidentially recorded, which means that no one knows you are doing them, except either your chair or dean to whom you might report your yearly activities, or the individual parties that made the request. That’s why you never anticipate doing this work because you don’t see your advisors doing it when you’re training in grad school. It just suddenly appears — and keeps on appearing. I am not opposed to the work. I am opposed to the way we hide it.

Why does this lack of transparency matter?

I think for two reasons. First, it means there is all this work going on — work which has serious consequences in the lives of real people — which is totally inscrutable. How many books or articles were or were not published last year because I, or someone else, reviewed them for a press or journal editor? What kind of biases do I bring to my judgments and do we have any way of assessing that? What individuals, and now more importantly, what social networks are making things happen in the field?

All of these questions are currently unanswerable because of this dark economy of labor. While we have a tremendous amount of freedom in the classroom, I still have to make course proposals, still have to get approvals for new classes, still have to have my performance evaluated by my students, etc. There is an important degree of accountability for what and how I teach. That is totally missing from peer review.

The second reason this matters is purely practical. I am totally exhausted by these requests. If I said yes to every request I would have no time for anything else. It literally could be a reasonable full time job to adjudicate everything people have asked me to read. Period. No teaching, no research, no writing (other than “reports”), no recommendation letters for students, no advocacy on campus for things I believe in, no advising duties. Just “peer review.”

So inevitably I say no, and then yes, and then no, and maybe a few more no’s with some guilt-ridden yes’s dropped in for good measure. I try to create some rationale, but really it’s random. That’s not a good way to make decisions, it’s not a good way for me to apportion my work time, and it’s not a good way for the field to be relying on people.

I also don’t think I’m special. My working assumption is that many, many in the field experience the same thing. I hear this anecdotally all the time when it becomes my turn to ask people to review something for the journal I edit. But it’s hard to know because everything is so invisible. And as the tenure labor market continues to shrink, the problem will only worsen, as fewer people are called upon to do more and more things.

So here is what I suggest: We need a peer-review registry.

We need a place where this work is recorded and made visible. But it’s confidential you’ll say! It’s a fair concern. But we can create a registry that contains minimum information for public consumption, and then contains confidential information for auditing purposes. For example, I can list that I am doing “promotion review” right now. You don’t need to know whom I’m doing it for. But it is important for people to be aware of who is doing this kind of work. Who are the gatekeepers? I can guarantee you will start to see biases and unintended networks appear. It will also help me in my decision making to be able to say to a requestor, look, I’m doing 6 different reviews right now, I really can’t say yes. Many of the reasons why we say yes is we are trying to maintain social bonds. No communicates a lot of negative will. No + I have a very good reason is very different. Right now, it’s hard to know if someone is just dodging work or is legitimately swamped.

But we also need a confidential section for auditing purposes. If all an academic has to do to look busy is check a box, s/he will. We need some way of validating that the public-facing representation is accurate. And we also need some way of further delving into the data. The point wouldn’t be to disclose embarrassing information — did you know that Prof. X was the reviewer for these 20 articles! — but to work with stakeholders to help them understand where problems might lie. We’re seeing a strong network effect here around this group of people, perhaps Editor Y you might consider expanding your pool a bit. Or Grant Agency Z you have traditionally been relying on reviewers with these gender/institutional/ethnic/disciplinary backgrounds. You might want to take steps to address that.