The evidence
The claims with no evidence either way
Four claims that look empirical, are asserted with great confidence, and have never been measured by anybody. Why those two things go together.
Claims this part is about, and what each was answered with
New starters take longer to become productive remotely
Empirical→EvidenceThe one claim answered with evidence that measured it
Productivity fell when we went remote
Empirical→DataRemote workers put in longer hours than office workers
Empirical·Nothing; it was never answeredthree claims · three empirical · none of the answered ones was answered in the wrong kind · one was never answered at all
Three empirical claims with three different fates.
Four of the nine empirical claims on this record have no evidence behind them at all. Not weak evidence, not contested evidence: none, in either direction, anywhere.
They are also, by some distance, the four asserted with the most confidence. That is not a coincidence and the mechanism is worth understanding.
The four
- People stay longer if they feel connected to a place. Would need leaving rates set against time actually spent with colleagues. Nobody records the second variable, here or anywhere.
- Innovation happens in unplanned conversations. Would need ideas traced back to where they started. Nothing records this.
- Hybrid teams make worse decisions. Would need decisions rated by somebody blind to how the team worked. Would have to be set up in advance and nobody has.
- Collaboration tools have replaced what the office did. Would need an agreed list of what the office did, written before the check. The list has never been written by anybody.
Why unmeasured claims are held most confidently
Three reasons, and they reinforce each other.
Nothing has ever contradicted them. A claim that has never been tested has a perfect record. Years of holding it have produced no disconfirming experience, because no experience could have disconfirmed it.
They are consistent with everything observed. Each is compatible with any day anybody has had at work. "Innovation happens in unplanned conversations" fits every good idea you remember having near somebody, and the ideas you had alone do not present themselves as counter-examples.
They sound like things that are known. All four are in general circulation. Having heard a thing from several directions is indistinguishable, from inside, from having evidence for it.
The self-sealing one
The fourth is worth separating because it has an additional property.
"Tools have replaced what the office did" cannot be wrong until somebody writes down what the office did. And writing that list is unattractive to whoever makes the claim, because a specific list invites specific checking, and some items will survive.
The same shape appears on the other side: "something is lost when we are not together" is equally unfalsifiable until the something is named, and naming it is equally unattractive for the same reason.
Both sides have a self-sealing claim and neither has an incentive to open it.
What to do with an unmeasured claim
Not dismiss it. "There is no evidence for that" is true, unhelpful, and invites the reply that there is no evidence against it either, which is also true.
The useful move is to ask what it would take, and to take the question seriously:
If we wanted to know that, what would we have to look at?
Three things can follow. Sometimes the answer is specific and cheap, in which case the claim has just become checkable. Sometimes it is specific and expensive, which is worth knowing. And sometimes there is no answer, which is the most informative outcome of all — it tells both of you that this part of the disagreement is not going to be settled by looking, and can go straight to being traded.
The honest position
Four unmeasured claims out of twenty-four is not a scandal. Most things people believe about how work works are unmeasured, and acting on unmeasured beliefs is unavoidable.
What is worth avoiding is the confidence. An unmeasured claim is a reasonable thing to hold and a poor thing to win an argument with, and the difference between "I think this is how it works" and "this is how it works" is the whole of it.
How these claims behave over time
They do not weaken and they do not strengthen. They accumulate confidence, because confidence in an unexamined belief is a function of how long it has been held.
That is the opposite of how an examined claim behaves. A claim that has been checked, even once, acquires a shape: it holds under these conditions, it did not hold in that case, the effect was about this big. Shape is what makes a claim usable in a negotiation.
An unexamined claim has no shape, only intensity, which is why these four were the ones most often repeated and least often useful.
Making one checkable
The move is to ask for one instance rather than for evidence.
Can you think of a specific time that happened here?
It is a much smaller request than evidence, it is usually answerable, and the answer is informative either way. A specific instance turns an unfalsifiable generalisation into something with edges — that is the sort of thing I mean — and from there a measurement is often visible.
On this record this question was asked once, of "something is lost when we are not together", and it produced three specific things, one of which was then measured.