The evidence
Nine empirical claims
Every claim on the record that evidence could in principle settle, what each one would need, and whether anybody has it.
Claims this part is about, and what each was answered with
New starters take longer to become productive remotely
Empirical→EvidenceThe one claim answered with evidence that measured it
Productivity fell when we went remote
Empirical→DataRemote workers put in longer hours than office workers
Empirical·Nothing; it was never answeredthree claims · three empirical · none of the answered ones was answered in the wrong kind · one was never answered at all
Three empirical claims with three different fates.
The empirical claims, and whether evidence that would settle them exists
Teams that sit together ship faster
The same work, done by co-located and distributed teams, compared on deliveryThe paper cited compared companies, not teams, and not on the same work
Productivity fell when we went remote
Output per head on the same work, before and against afterThe figure quoted was revenue per head across a period in which the product changed
New starters take longer to become productive remotely
Time to first independent delivery, by starting cohortThe company already holds this and it took an afternoon to produce
People are less likely to leave if they feel connected to a place
Leaving rates against hours spent with colleagues, which nobody recordsInnovation happens in unplanned conversations
Ideas traced back to where they started, which nothing records anywhereRemote workers put in longer hours than office workers
Logged hours by working pattern, which the company already holdsNever produced, because nobody asked for it
Hybrid teams make worse decisions than co-located ones
Decisions rated by somebody who does not know how the team workedSickness absence is lower when people can work from home
Absence records by working pattern over the same periodThe study cited is about a different country's sick pay rules
Collaboration tools have replaced what the office used to do
An agreed list of what would have to be replaced, written before the testNobody has ever written the list, so the claim cannot be wrong
nine empirical claims · two of them could be settled today with evidence that exists · three were answered with a study that measures something else · for four there is no evidence either way
The nine empirical claims from the same record of twenty-four.
Nine of the twenty-four claims are the kind evidence could settle. This page is all nine, with what each would actually need and whether anybody has it.
The short version: two could be settled this week with data the organisation already holds, three were answered with studies that measure something else, and four have no evidence anywhere, for or against.
The two that could be settled today
"New starters take longer to become productive remotely." The measurement is time to first independent delivery, by starting cohort. The company holds it. Somebody produced it in an afternoon when finally asked, and it supported the claim.
This is the only claim in the entire record that was both empirical and answered with evidence that measured it. One, out of twenty-four.
"Remote workers put in longer hours than office workers." Logged hours by working pattern. Also held, also a day's work. Never produced, because the claim was made by the side that would have benefited from it and nobody on the other side asked.
That pattern is worth noticing: an empirical claim that nobody checks is usually one that nobody has an incentive to check. The person making it does not need to, and the person doubting it would rather doubt than risk confirming.
The three answered with the wrong measurement
Each of these had a study produced at it, and in each case the study is sound and about something else.
- Teams that sit together ship faster. Answered with a paper comparing companies, not teams, and not on the same work.
- Productivity fell when we went remote. Answered with revenue per head over a period in which the product changed.
- Sickness absence is lower at home. Answered with a study from a country with different sick pay rules.
In all three the citation was made in good faith by somebody who had read the paper. Reading a paper and checking that it addresses your claim are different acts, and only the first one feels like diligence.
The four with no evidence at all
These are the interesting ones, because they are asserted with the most confidence.
| Claim | What would settle it | Why nobody has it |
|---|---|---|
| People stay longer if they feel connected to a place | Leaving rates against time spent with colleagues | Nobody records the second variable |
| Innovation happens in unplanned conversations | Ideas traced to their origin | Nothing anywhere records this |
| Hybrid teams make worse decisions | Decisions rated blind by configuration | Would have to be set up in advance |
| Tools have replaced what the office did | An agreed list of what the office did | The list has never been written |
Each is stated as though established. For each, no observation exists anywhere that bears on it. And for the last one in particular, the absence is self-sealing: without the list, the claim cannot be wrong, which is not the same as being right.
What this adds up to
Of nine empirical claims, two are answerable with existing data and were not answered, three were answered with the wrong thing, and four are unanswerable as stated.
That is the entire empirical content of an argument that both sides believe is about facts.
The pattern in who makes which
Worth noting, because it is not what either side expects.
The four claims with no evidence anywhere divide two and two between the sides. The three answered with mismatched studies divide two and one. The two that were checkable with existing data divide one and one.
There is no side in this record that is noticeably more disciplined about evidence than the other. Both make unevidenced empirical claims at about the same rate, and both fail to check the ones they could check.
What a disciplined version would look like
If both sides applied one rule — state the measurement with the claim — the nine would have sorted themselves immediately.
Two would have been checked within a week, because the data existed. Three would have been revealed as citing the wrong thing, because the measurement named would not have matched the study offered. And four would have been identified as unmeasurable at the moment of being made, which would have moved them straight to the trade column where they belonged.
Nine claims, resolved into three categories, at a cost of one extra clause per claim. On this record it happened once out of nine.