False Economy of Agile
March 29, 2019
It's impossible to formulate a replicable set of practices that will make a software developer team effective in the absence of a trained personnel manager. Here's why.
Grooming Sick Pets
Agile, as it is often practiced these days (by rote, or by replication of best practices) is a way of cargo-culting behaviours of healthy teams.
Observable behaviours are often symptoms, results of a condition. Healthy animals have thick, shiny fur, but grooming your pet's fur is not actually going to make it healthier.
Is it good when two people talk often? That depends. Maybe they are collaborating on a project. Maybe they're flirting. Maybe they mistrust each other and attempt to figure out the other person's position. Maybe they gossip about a person they both dislike. Maybe they're passing time.
It's their motivations that make the behaviour productive and healthy, and it's motivations that were enshrined in the Agile Manifesto. Unfortunately, motivations are ultimately unobservable. Behaviours are, though. We need to have some way of confirming that a team has become agile, so we use prescriptive lists of behaviours to diagnose it.
We've gotten it all wrong. We're grooming our pets according to prescribed schedule and wonder why they're still sickly, even though their fur is very shiny indeed.
Unconscious Knowledge
There is, apparently, a way to become an unconscious expert, where your brain functions like a pattern-recognition machine but you don't can't verbalise what patterns you're spotting. At the same time, humans are excellent at inventing explanations and rationalisation. If you ask an unconscious expert about the patterns, they may have an explanation, and that explanation may be either wholly wrong, or insufficient.
Humans teach each other how to behave correctly by example. Social skills are a prime candidate for unconscious expertise. We don't actually know why a team functions well. (A certain Google-shaped business commissioned original research and came up with "psychological safety." That's it. People have to trust each other. Mind-blowing, eh? But that's the end result, the symptom. How do you get them to trust each other?)
So maybe we used to have people who taught agile by transference. Maybe they had relationships within the company which allowed them to tell others what to do, adjust teams, without explaining why. "Just trust me." Their disciples didn't have the same relationships, but still thought they could improve the world. What could they do to gain the leverage? They could get certified. Thus agile certifications were invented.
In order to certify, you have to qualify and quantify. And so we're back to the problem of prescriptive lists, but it gets worse: they were formulated by well-meaning unconscious experts who got the right results, but weren't necessarily able to explain why. Sometimes they got it right, and sometimes they rationalised it, and we can't tell which parts are which.
Solving Power Struggles
Agile and other work process methodologies claim to improve how teams work, which means they solve social problems. Either the team members don't know how to work with each other, or the manager doesn't know how to manage the team.
Thing is, it's offensive to tell an adult they don't know how to do their job, and it's even worse to imply that they don't know how to interact with other humans. So we don't. Instead we talk about productivity, and being client-focused, and responsive to change.
Agile may have actually started as a way to make better products, but now it's sold as a solution to power struggles. Workplaces are full of political conflicts over who's right, and whose convenience and comfort takes precedence. You can tell agile is meant to solve that, because usually people introduce agile in hope of fixing frustrating behaviours of others, not their own.
(This is also why nobody can agree on what agile actually is. We all agree that we should do, or be, agile, so criticising people for not being agile is a convenient lever to make them do as you say, just like telling them off for not thinking of the children.)
Since this is a political problem, a checklist of rules won't solve it. Humans are excellent at gaming the rules when it's to their benefit.
Publicly Inadmissible Motives
Every interaction with a second person has three elements: what I think, what you think, and what I think you think. These can change human behaviour in unexpected and inconsistent ways. (We usually label the inconsistency "hypocrisy" and "peer pressure", but really it's the best-effort attempt to make sense out of a world in which we need to compete and cooperate at the same time, often with the same people.)
These motivations are inadmissible. Nobody can come out and say, "You should write extensive reports. I don't understand what you're doing and I need detailed explanations. I think that you noticed this, and that you think my skills are deteriorating, and so I feel anxious and insecure," or, "I should not write extensive reports! It's exhausting because I struggle with expressing myself in English. I think you noticed this, and that you think I'm uneducated, and I feel shame. Also when you supervise me in close detail I think you do this because you doubt my motives, and so I feel angry."
Did I make you uncomfortable? That was on purpose.
We are used to admitting a part of our feelings and motivations, but usually it's not the part that will make us look bad and spell out exactly how the other person can hurt us (which, after all, sits at the heart of conflict.) All the behaviours encoded by agile are part-driven by such hidden motives: struggles over status, resources, power, respect.
Codified practices are insufficient, because you can't codify in publicly available rules solutions for the problems that must never be named.
Debugging Humans
We all arrive in teams with uneven social skills; engineers, apparently, more often than other professions. As human beings we are also permanently experiencing conflict of interest between acting for the good of self and the good of others: the team, or the company.
This is the problem good personnel managers solve. The usual tasks that we think of as managerial - filling out reports, managing projects and resources with Gantt charts, broadcasting the team to the rest of the company, providing information on broader company strategy - are actually administrative tasks. They could (and probably should) be performed by a PA. The manager is there to navigate power relationships and mediate interpersonal conflicts.
A manager is, however, not a therapist, because their role is not to fix people's gaps. Instead, they rearrange employees into teams, and help direct them into positions with the best balance of usefulness to company vs. effort needed to train the person. They have to do all this without admitting that's what they do, because some of the problems they solve are inadmissible.
This is also why it's incredibly hard to level up from a technical employee to a manager: there's a part of the job which nobody can tell you how to do, or even that you're supposed to perform it. Since social skills are best learned by transference (building unconscious expertise), it's best to find a mentor who can give you honest feedback on your personal, social or political skills.
Accidental Performance
If you have a well-functioning team without manager, it is highly likely that it functions well as an accidental side effect of its composition.
Sometimes there is a team member who acts as an implicit manager without the title. This can cause unacknowledged power structures to arise in a theoretically "flat" organisation. Hidden power structures are not a new problem: political activists attempting to run self-governing consensus-driven groups have been struggling with this. Since nobody is supposed to hold power, the team will either resist acknowledging such a hidden leader exists, or, if forced to admit it, they will tear down the leader to save face and substitute them with someone else.
A team may also be highly functioning because all team members are highly skilled in "soft" skills. The industry generally does not recognise, or talk about, that engineering team members do need soft skills, and we do not reward them for improving. Hiring soft skills is currently called "hiring for empathy." Unfortunately, empathy is presented as a character trait rather than as a trainable skill. This limits the available hire pool, and causes people to be unnecessarily shamed. Lack of soft skills is neither a moral nor personal failing. A good manager will compensate for lacks in soft skills, and train employees as necessary.
Lastly, a strong focus on hiring for culture fit can also produce unmanaged but well-functioning teams. Culture fit means selecting people similar to existing team members in anything from sense of humour, cultural background, to social group. When team members can interact with each other in a familiar way, they feel safer and more comfortable. Psychological safety, as previously mentioned, increases team performance.
Conversely, even the most well-meaning people feel anxiety about giving offence by breaking unfamiliar social rules, which they may express as having to "walk on eggshells" when talking to a team member who does not conform to "team normal." These minority members will then be at risk of being excluded from team communication, and later silently isolated and pushed out of the team. It's not something we like to talk about for fear of being called bigots, so the problem goes unnamed and unrecognised. A good manager will be able to offset this problem without explicitly calling it out.
"Hiring for culture fit" is sometimes considered by minorities as code word for "minorities not welcome." Think before including the phrase in your job ad.
The first person who breaks the norm (woman, older person, younger person, parent, remote worker, person requiring physical accommodation) bears the burden of normalising their group to other team members, educating them, and making them comfortable. It's exhausting, unrecognised, unrewarded, and if they don't (or don't know how to) perform the work, they get penalised for it and branded as "not a team player."
You must name this work, recognise it and offset the effort expended on it, otherwise you'll be implicitly penalising the minority person for being a minority. At the same time, you should not shame people for being human and being anxious of unfamiliar. You will have to watch out and intervene if people harass or exclude the minorities.
Summary
Well-functioning unmanaged teams can arise by accident when:
- one of the team members plays a "shadow" manager role
- all team members are highly skilled in soft skills
- all team members are highly similar
However, describing and reproducing practices and habits of these teams is not sufficient to turn any team into a well-functioning team, because:
- lists reproduce symptoms, not causes
- lists omit practices learned by imitation
- a third party is needed to provide conflict mediation
- some motivations and problems are publicly inadmissible and therefore cannot be covered by explicit rules
- conflict resolution, diplomacy, negotiation, understanding of human psychology and group dynamics are not common skills, and are not easy to acquire
Treating agile as a way to get engineers to perform administrative and managerial (interpersonal) tasks is a false economy.