The evidence
Reading a study somebody cites at you
Four checks, about ten minutes, that will tell you whether a paper bears on the claim it was produced for. Most do not.
Claims this part is about, and what each was answered with
New starters take longer to become productive remotely
Empirical→EvidenceThe one claim answered with evidence that measured it
Productivity fell when we went remote
Empirical→DataRemote workers put in longer hours than office workers
Empirical·Nothing; it was never answeredthree claims · three empirical · none of the answered ones was answered in the wrong kind · one was never answered at all
Three empirical claims with three different fates.
Somebody sends you a paper. It supports their position, or it is said to. Before arguing about it, four checks, in this order, about ten minutes in total.
One: the unit
Find what was compared. Firms, teams, individuals, days, tasks?
Write the claim it was offered for and underline its unit. If the two units differ, the paper does not bear on the claim and nothing else matters. On this record this check alone disposed of two of the four most-cited papers.
Two: the outcome
Find what was measured as the result. Not what the abstract says it means — the actual variable.
"Productivity" in these papers is usually revenue per head, self- reported output, lines of some countable thing, or manager ratings. Each of those is a different claim. A manager-rating result quoted as a productivity finding is a finding about manager ratings, which is interesting and is not the same.
Three: who chose
Did the people in the groups choose which group they were in?
Almost always yes, in this literature, because almost nobody randomises working location. Where people chose, the groups differ in every way that influenced the choice — commute, caring responsibilities, seniority, type of work.
A difference between self-selected groups is a real difference and it is not necessarily caused by the thing being studied. Papers that handle this say so prominently; papers that do not, usually have not.
Four: when
Look at the period. If it spans 2020 to 2022, it spans a pandemic, a hiring boom, a hiring freeze, and in many places a change of product and customer base.
A before-and-after across that window is comparing two different worlds, and attributing the difference to working location requires an argument the paper may not have made.
The checklist
| Check | Question | Fails if |
|---|---|---|
| Unit | What was compared? | Different unit from the claim |
| Outcome | What was measured? | Different thing from the claim |
| Selection | Who chose? | Groups differ for other reasons |
| Period | When? | The comparison spans a change of world |
A paper passing all four and addressing the claim is genuinely useful and should be accepted, including when it goes against you. That is the point of doing the checks rather than looking for an escape.
What to say
Not "your study is flawed". Something narrower and almost always accurate:
I think this is about firms and our claim is about teams. Is there anything that looks at it at our level?
It concedes the study, declines the inference, and asks for the thing that would actually help. Most people respond to it well, because it treats them as somebody trying to find out rather than somebody trying to win.
The one that will be used on you
All four checks apply to whatever you cite, and the third is the one that will catch you.
Nearly every piece of evidence available on this subject involves people who chose their own arrangement. That includes the studies that support your position, and it is worth saying so before somebody else does.
The fifth check, for this subject only
One more, specific to working location: what were people comparing it with?
A great deal of the research compares an office arrangement with whatever happened in 2020 and 2021, which was not remote working as anybody would now design it. It was remote working with schools closed, with no equipment, under a public health emergency, among people who had not chosen it.
A finding that offices beat that is not a finding that offices beat remote working.
This is the single most common inferential error in the popular versions of this argument, and it is usually made by people summarising studies rather than by the studies themselves.
What a good-faith citation looks like
For contrast, because it is worth knowing what you are aiming at:
This looks at individuals, measures time to first delivery, covers 2022 to 2024 so it is after the emergency, and the groups were assigned by team rather than chosen. It finds a difference of about this size. It is about new starters only, which is narrower than what we are arguing about.
Five facts and a limitation, in one breath. Anybody who cites like that is worth taking seriously on everything else they say, and on this record exactly one person did it, once.