On the need to measure productivity

    I’ve long said that no-one really knows what knowledge work looks like. It’s easy to see whether or not someone is digging a hole in the ground, but it’s much more difficult to see whether the work that someone is doing on a computer is ‘productive’.

    This is, I think, partly because ‘productivity’ is something that is best thought about for things that can be systematised and made routine. A lot of knowledge work is fundamentally creative, and so quantitative metrics are meaningless. Who cares if you’ve made a million pull requests if they’re all to change a single character?

    This article discusses the complexities of assessing productivity in various fields, the issues with current interviewing processes, and suggests that future evaluations may become more tied to tangible accomplishments rather than arbitrary metrics. That’s presupposing, of course, that hierarchical evaluations are even necessary.

    [E]very potential metric we devise appears woefully inadequate in assessing this holistic outcome. Whether it's pull requests, lines of code, user stories, story points, or ship dates, it seems that every metric can be manipulated or gamed. Ship dates may be advanced, but quality suffers; story points morph in size depending on the project, and lines of code can be bulked up with a test suite. Even pull requests can be sliced and diced to skew the numbers. It's a frustrating conundrum.

    For more fuzzy fields, like product management or marketing or design, it becomes even more hand-wavey. Some fields tend to depend on getting other roles to execute better, but you can’t go rewind history and try things with a different PM to see if things would have been better. Same with design.


    If you give the most productive employee more work, presumably they’d be justified in asking for higher compensation? After all, they are driving greater outcomes for you. Would you be comfortable paying it?

    For example, would you pay a 3x more productive designer 3x the fully loaded cost of the average designer? If 10x engineers truly exist, why do pay scales intra company not cover a 10x spectrum?


    My suspicion is that, like in other fields where performance matters and is financially rewarded, there will be a surge in our capacity to measure and evaluate real-life work performance. Compensation will become more closely tied to tangible accomplishments rather than arbitrary levels or seniority. Interviews will transition to be more real-world scenarios, perhaps within the customer’s actual codebase, addressing a genuine problem the customer faces—possibly even compensating the interviewee for their time.

    Source: Why is it so hard to measure productivity? | fractional.work

    Image: Kelly Sikkema

    Artificial metrics are flying by instrument

    We had a conversation earlier this week about how we’re going to measure the progress of some community work we’re doing. In the end, we decided that there were no metrics that would make sense. It’s a vibe.

    This post says much the same thing. Sometimes there are no  objective measurements for things that matter. And that’s OK.

    Flight deck controls

    Artificial metrics are flying by instrument. They're individual "better/worse" dials that in amalgamation are supposed to tell you which way things are going, as long you are paying attention to the correct combination of them at the correct moment, and don't over-react to the feedback loops and crash the whole thing via a PIO. Instrument-only flight is harder than visual flight, it takes extensive practice, and the mistakes have worse repercussions.

    You can instead choose to just fly visually. It’s easier, it’s safer, and it’ll get you where you’re trying to go. The thing is, your entire industry thinks it’s impossible, and worse, they think it is irresponsible. They’re kinda right. You have to be good at the innate skill of flying, instead of the skill of navigating by instrument. Guess which one the “become a manager in tech” system produces. Bonus points: recognize how that is itself a PIO.

    Bonus Bonus Bonus points: Consider that if you’ve learned the skillset of visual flight poorly, and you don’t use the instruments to correct yourself, how will you ever know it’s going wrong in time?


    What matters for your team/org’s success is the fundamental human relationships, comradery, esprit de corps, support and space-curation, and especially, all of the prior while treating-em-like-adults. Those things make up the totality of why people want to work on your team and are excited about working with and supporting their peers. These are not invisible things. These are things you can pay attention to, structurally. These are not things you can quantify with numbers. You’re going to have to get comfortable with forming, expressing, and defending opinions based on things besides “data.” Not because you don’t have data, but because you don’t have quantifiable numbers that represent themselves, and our industry is poisoned into believing that only such things are data. We’ve got thousands of years of evolution helping us understand how group dynamics are flowing. Yes, using that is a skill set. That’s my point. Build and use that skill set. Learn how to read people’s reactions. Learn how to understand people’s motivations. Learn how to see how people work in groups and as individuals. Do the work.

    Source: How to build orgs that achieve your goals, by absolutely never doing that | Graham says wrong things

    Image: Jp Valery