Why you should consider giving more of your money away effectively and why you should talk about it
In an age where individual actions can catalyze global change, the power of giving has never been more evident. Donating to effective charities isn't just an act of generosity; it's a strategic move towards creating substantial, measurable impact in the world. This is my attempt to delve a bit deeper into the rationale behind donating to organizations that maximize the effect of every cent, underscored by my long-standing personal commitment to contribute thousands of pounds of my hard-earned cash every year to hand-selected causes.
The Importance of Effective Charities
Defining Effectiveness in Charity
An effective charity is not just about the goodwill behind it but its ability to produce significant, measurable impacts with the resources it has. Such organizations are characterized by their cost-effectiveness, transparency, and evidence-based approaches to solving problems. They go beyond good intentions, rigorously evaluating their interventions to ensure that donations translate into real-world benefits. The aim is to maximise the impact on the cause the given organisation is focusing on.
The Selection Process
Choosing where to donate involves more than picking a cause close to one's heart. It requires diligent research to identify organizations that not only address pressing needs but also demonstrate a track record of success and efficient use of funds. Organisations like GiveWell and The Life You Can Save offer comprehensive analyses, publish their data, continuously refine their criteria and aim to course correct as best as they can. All this makes it possible for all of us to make much more informed choices about which charities to support.
Emotional vs. Impact-Driven Giving
While emotional responses often drive spontaneous donations (e.g. responding to an appeal by a donkey hospice operating in a nearby city), effective giving advocates for a more analytical and more impact-focused approach. By focusing on outcomes, donors can ensure their contributions are not just felt but are also effective in making a substantial difference. And this difference can be measured and evaluated. The best charities are often orders of magnitude more effective (in terms of impact achieved per dollar invested) than those that don't rigorously focus on effectiveness (which, sadly, are most charities out there).
Personal Journey: A Commitment to Effective Giving
My journey into philanthropy and effective altruism began with a simple realization: I had the means to make a difference and I wanted to make as much of a difference as I could. Over the years, I've spent a lot of time reading and learning about philosophy and ethics and came across Peter Singer's groundbreaking book Famine, Affluence and Morality. From there, one can easily leap into the Effective Altruism movement which considers Singer as one of its philosophical forefathers.
My decision to donate considerable amounts of money every year to hand-picked charities was motivated by a desire to see real change in areas of profound need. I selected each charity I donate to after careful research (which I update every year) with a focus on commitment to transparency and impact, aligning with my belief that giving should be intentional, effective and transformative. At the same time, I tried to concentrate my giving on causes which deeply resonate with me — global health, long-term future, humanism and rationality.
Maximizing Impact
Realising that a dollar of donated funds can have a vastly different impact based on how and where it is spent I felt that if I deeply cared about the impact I was making, donating to effective charities is the only way. Supporting such organisations ensures that each contribution goes towards making the most significant possible difference. Some of the organisations I support include Sightsavers, Against Malaria Foundation, Malaria Consortium and Effective Altruism Funds. These charities use evidence and analysis to deploy resources in ways that can dramatically alter outcomes for the better, whether by saving lives, improving education, or protecting the environment.
Now, is all my giving as effective as it can be? The vast majority of it, I hope, is. But a small part of my donations also goes to charities and organisations that I simply want to support because I really want them to succeed in their mission (e.g. Cancer Research, Humanists, and UNICEF) even though their effectiveness is perhaps not as high as I would like it to be.
Personal Fulfillment & Encouraging Accountability
Beyond the tangible impacts, giving effectively brings profound personal satisfaction. Knowing that your donations are working hard to address critical issues provides a sense of fulfilment unmatched by more passive forms of support. It is a way to connect donors directly to the causes they care about, creating a feedback loop of positivity and purpose.
Supporting effective charities also promotes a culture of accountability within the nonprofit sector. It signals to all organizations the importance of transparency, efficiency, and measurable outcomes, raising the bar for what donors expect and what charities strive to achieve.
Why Being Public About Altruism Is Beneficial
There are many strong reasons to abandon once and for all the idea that charity should be private and that one should not talk about it publicly. If one cares about maximising impact, if one really wants to do as much good as possible, then being open and outspoken about one's giving is imperative.
Inspiring Others
When people share their stories of giving, they do more than just highlight their own contributions; they inspire others to take action and this little article is my way of doing just that. Publicising altruistic efforts can help demystify the process of donating, making it more accessible and appealing to a broader audience. It helps explain why charity matters and may help inform about some of the key considerations involved in choosing where to donate creating a narrative that encourages others to consider how they, too, can make a difference.
Addressing Criticism
While some may view publicising one's charity work as self-serving, my view is that the broader impact of such openness far outweighs potential critiques. The goal is not self-promotion but the promotion of a cause, leveraging personal influence to spur collective action. And the more effective that action can be, the better.
How Everyone Can Contribute
Donating money is a great way to help and drive progress in the cause area you are focused on. But contributing to effective charities doesn't always mean writing a check. Whilst EA popularised the concept of earning to give, there are many other options to make a difference instead of, or in addition to, giving your money away — volunteering time, lending expertise, or even spreading the word about an organization's work can be equally valuable. What matters is the intention and the effort to make a positive impact, regardless of the form it takes. Just remember that choosing the cause and the organisation you are supporting will be the crucial determinant with regards to the impact you can make.
The journey to impactful giving begins with a single step. Even small donations to effective charities can have outsized impacts when used wisely. This can be monthly, yearly or, of course, one-off. The key is to start, to take that initial action towards contributing to the greater good, and to build from there.
Conclusion
In a world rife with challenges, donating to effective charities offers a beacon of hope and an opportunity to make a difference. Do you care about animal welfare (as there is still an unfathomable amount of animal suffering in the world) - there are many great effective charities for you to choose from. Would you rather help with mitigating the risk of AI (perhaps because you share my view that in the long run AI may pose an existential risk to humanity)? Well, there are charities for that too. There are several major causes worthy of your attention and the choice is only yours.
Effective giving represents a commitment not just to giving but to giving wisely, ensuring that each dollar/pound/euro contributes to building a better future. And by sharing our journeys of altruism, we can all inspire others to join in this movement of meaningful change achieved in the most effective way possible. Today, we have the opportunity to turn our compassion into making a real impact. Let's embrace it with open hearts and minds.
Empiricism makes everything better
I am fully aware that making broad and far-reaching proclamations like the one in the title of this post in dangerous. Yet, having thought about this topic long and hard, my credence in the value of empiricism is rather high and my hope is that if you manage to read the whole post you will come to a similar conclusion. Let’s get started.
What is empiricism
Empiricism is an epistemological theory that holds that knowledge or justification comes only or primarily from sensory experience.
As a philosophical theory, it has a long and rich tradition dating back to the the sixth century BCE with many famous thinkers making major contributions to the core concepts and ideas over the last two and a half millennia. The modern version of empiricism that people typically think of originates chiefly from 17th and 18th century empiricists like John Locke, Francis Bacon, Thomas Hobbes and David Hume.
A wise man proportions his belief to his evidence — David Hume
On Hume’s view, our knowledge of the world is based on sense impressions and such “matters of fact” are based on experience. In practice, empirical research would then typically progress along the following path:
Observation — Gather empirical data
Induction — Frame a general conclusion from the gathered data
Deduction — Create a conclusion from the experiment
Testing — Analyse and validate collected data with appropriate statistical methods
Evaluation — Present the gathered data and the conclusions of the experiment and assess their validity, value and consequences
It is worth contrasting empiricism with the philosophy of rationalism which posits that knowledge can also be developed by exploring concepts and through deduction, intuition and revelation and that knowledge can be created even without prerequisite sensory experience. In my experience both rationalism (sometimes known as intellectualism) and empiricism are important avenues through which we acquire new knowledge and understanding. As such both need to work hand in hand in most human endeavours to drive progress.
But enough of philosophy — my aim in this post is not to provide a full and holistic overview of empiricism as a philosophical concept (which I am not qualified to do), but instead I would like to show that having empiricism as a mindset is important and valuable both in life and in how we build, run and grow organisations.
Empiricism as a foundation of progress
Scientific method and empirical research are perhaps the most transformational ideas humanity has ever come up with. These ideas and methods have lifted humanity from poverty, powered the development of technology and medicine and enabled a fundamental transformation of our society resulting in a creation of a completely new epoch, Anthropocene.
It was the mindset, ethos, practices, and methods of empiricism that enabled people to think more deeply and more honestly about what is real, what is true, what they should (and could) believe and why. It is empiricism that also dramatically accelerated society’s departure from the supernatural and the spiritual (recent, hopefully temporary, regression in some countries notwithstanding) and enabled a gradual liberation of whole peoples from the millennia of religious dogma. For all these reasons, it is hard of overstate the impact empiricism had on the society as a whole and on the pace and direction of human progress.
When empiricism is absent
Now, so far all we have been talking about empiricism purely at a conceptual level. Let’s now explore how ideas of empiricism translate into a business context. In my mind, a powerful way to illustrate the value of empiricism is to describe a fictitious organisation, let’s call it Legacy Ltd., where empiricism is entirely absent. At Legacy Ltd, the following is very common:
Most decisions are made exclusively by people with the strongest opinion, those who are most assertive at advocating their views and those who are paid the most
Strategy is often based on hopes, assumptions, gut feelings and ‘strong intuitions’
True level of organisational maturity is not understood or is based on unfounded beliefs
Major structural problems and issues continue to ‘bubble’ deep down in the organisation without senior management ever admitting there’s a problem (even if people are fully aware of them). Issues and incidents are explained away as bad luck, unforeseen circumstances, unexpected human errors or are downplayed as unavoidable.
People’s career growth is very positively correlated with their ability to ‘manage the message upwards’ and ‘tell a story’.
Dubious, irrelevant, unreliable, or incomplete data is used as justification for decisions or investments
Vanity metrics used to measure organisational and team performance
The organisation spends a lot of effort on reinforcing existing beliefs and ideas
The list goes on. In various guises, many of us have experienced one or more of such things in our careers. If any of the above looks familiar, your organisation or team would certainly benefit from more empirical thinking and more empiricism-based practices.
Why is practicing empiricism hard?
So if lack of empiricism tends to result in poor outcomes, why don’t all leaders, teams and organisations fully embrace empiricism and thus gain the many benefits it brings? Well, as is usually the case, there are many reasons. In this post, I will outline only three of them.
Firstly, practicing empiricism requires a culture that values seeing things as they are not as people would like them to be. I am talking about a culture which values truth above everything else regardless of how painful, unpopular, or difficult to accept it might be. Way too often people have incentives to distort reality in a specific direction — perhaps they feel threatened in their role and feel that any admission of a problem may weaken their position. If the cultural context they operate in requires careful ‘management upwards’, navigating complex politics, dealing with power struggles, or being reprimanded for bringing up bad news or challenging things, it is only natural that people prefer to employ creative ‘spin-doctoring’ instead of empiricism.
Empiricism implies facing reality, digesting it, and shaping our actions based on what is reasonable to believe to be true.
Empiricism also requires having means to assess things in some empirical manner. This usually requires data, insights, analyses, feedback, and other types of inputs all of which need to be reasonably reliable. Without such inputs it is not possible to establish what is true and real what is fiction or wishful thinking. Organisations often either lack trustworthy data and insights or, and that could be worse, knowingly or unknowingly work with data that’s misleading, irrelevant or incomplete. This is usually a consequence of measuring what you can vs. measuring what you should.
Finally, empiricism requires intellectual humility, curiosity, absence of hubris and appreciation of one’s fallibility. These are all exceptionally valuable qualities to have and any team formed of people with such qualities is destined to grow and succeed. Alas, the world around us often rewards and incentivises different behaviours.
Empiricism in practice
As we said, empiricism implies facing reality, digesting it, and shaping our actions based on what is reasonable to believe to be true. It means grounding our actions and strategies in what is real and continuously validating if our understanding is correct. Here is a list of basic principles, approaches and ideas that, in my experience, support empirical thinking:
Don’t distort facts — good or bad. See things as they are, even if you don’t like what you are seeing. That means not hiding from bad news but also not mislabelling mediocrity as excellence.
Reward and recognise people who are honest and transparent and who help you see what’s true. Publicly recognising people whose work is grounded in empiricism enables others to gain confidence to do the same.
Promote and foster a culture of open enquiry and curiosity. Lead by example.
Use (the right) data, insights, and feedback to help you establish what is real and what is not. Pay attention to what is measured, why and how and be conscious of Goodhart’s law.
Continuously evaluate how valid and representative your measurements, data and insights are and to what extent your conclusions really follow from the data (premises) — see Inductive Fallacies.
Get used to working with credences and degrees of confidence as opposed to absolutes. Your confidence in something should always be proportionate to the evidence you have and the quality/reliability of that evidence.
Be wary of confirmation bias and Dunning-Kruger effect — nobody is immune to it. Be careful about the questions you are asking yourself e.g. Can I believe it? vs. Must I believe it?
Make Bayesian thinking part of your mental toolbox and use it to update your understanding of reality.
Empiricism is, of course, not a panacea, but I strongly believe that it needs to have a much more prominent place in the minds of leaders both in government and in business. The agile community, for instance, has done a great job at promoting empiricism as one of the foundational ideas of what agile is all about and reminding organisations how valuable it is. Another example might be Ray Dalio whose leadership philosophy is strongly driven by empiricism — just read his ‘Principles’. Yet, I still maintain that we don’t have enough senior leaders whose ethos is grounded in empiricism and related ideas.
I hope I succeeded in making a good case for empiricism as an approach and mindset. So now it is over to you: How can you lead by example and bring a bit of David Hume and Ray Dalio into your team or organisation? Where and how can you best introduce empiricism in your work and your life?
The Yahaba Technique
Introduction
How are our future concerns represented in the decisions we make today? How can the people who will inherit our world influence actions we are taking and the policies we put in place? How shall we balance the needs of today with the needs of tomorrow?
In 2015, representatives of residents of Yahaba, a small town in northeastern Japan, went to their town hall to take part in a unique experiment. Their aim was to develop policies that would influence the future of Yahaba. But this time, there was a twist. Before debating municipal policy, half the participants were asked to put on ceremonial robes and imagine they were from the future, representing the interests of the current citizens' grandchildren. Specifically, they were told to imagine they were from the year 2060 and therefore representing the interests of a future generation. The results were very encouraging. Not only did researchers observe a "stark contrast in deliberation styles and priorities between the groups," the concern for future generations was infectious — among the measures on which consensus could be achieved, more than half were proposed by the imaginary ‘grandchildren’.
Organisations have a similar challenge. Like councils and governments, businesses need to think very carefully about the long-term. This is not only because most businesses hope to be successful not only today but also tomorrow, next year and in five years’ time and thus create a long-term value for shareholders but also because some policies, strategies and investments take many months if not years to yield results.
Concerns of tomorrow are therefore very relevant today. And whilst the time horizon most organisations today consider is typically in years rather than decades, the challenge remains the same: How do we practically bring considerations and perspectives of the future people into our conversations? How do we ensure our decisions reflect and take into account the viewpoint of people who may be in our roles in one, five or even ten years’ time? As I wrote elsewhere, long term success is very closely linked to the organisation’s ability to grow its overall maturity and that often involves making difficult trade-offs between the short-term and the long-term. Only organisations that get this balance right can hope to remain viable in the long-run.
What is the Yahaba technique?
In business and especially in tech there’s very often a tension between things we absolutely must complete here and now (e.g. because our commercial success directly depends on it) and things that we have to focus on in parallel so that we can continue to grow, remain adaptive, scale our operation without impeding agility and be able to innovate also in the medium and long-term.
To turn this tension into an advantage, to make more informed decisions and to properly examine what really matters to us and why, I propose the Yahaba technique.
Here’s how to use the Yahaba technique in practice:
When considering a strategic, organisational, structural, technical or other significant decision, bring together a small group of people whose input is relevant for the decision. In practice these could be experts in the field, senior leaders and managers who will need to lead the implementation of the decision, people who will be impacted by the change the most, customers, stakeholders etc. Keep the group small, ideally between three and eight.
Designate up to a half of the group participants (i.e. typically 1-5 people) to be the Shōrai (将来, Japanese for ‘future’). The rest of the participants stay in their ‘normal’ roles i.e. represent their perspective on the matter as they would normally do.
The role of the Shōrai is to put themselves in the shoes of the people who will have to live with the decision being discussed and to represent the view of these future people during the discussion. The Shōrai are expected to come up with arguments, ideas, challenges and proposals to shape the decision being discussed in a way that they believe would be most favourable from the perspective of people who will be leading the business, enhancing and operating the platform or running a business process in a few years’ time.
The group as a whole explores the decision in question from various perspectives and aim to reach a consensus about a set of specific proposals, actions, ideas and approaches that, in their view, best represents the varied views in the room and balance the different outcomes and concerns voiced during the conversation.
A few things to keep in mind
Running a good Yahaba session is not difficult, but there is a number of considerations you should keep in mind when using this technique:
Avoid making the group too large. My strong recommendation would be to limit the participants to the maximum of ten people, ideally less.
How do you decide who will play the role of a Shōrai? It is best to start with asking people to volunteer. Typically, people who are most concerned about the future implications of the decision(s) in question will happily put their hand up. At the same time, it is also insightful (both for the group and for the individual) to invite someone who typically tends to favour short-term concerns over the long-term ones to be one of the Shōrai. It not only forces them to think differently, but it also makes it possible for the rest of the group to obtain a new perspective on the matter in question.
If are a Shōrai, your job is to be a passionate but considerate and reasonable advocate of the concerns of future people (colleagues, customers, leaders, stakeholders etc). Your task is not to overwhelm the rest of the group with a set of purely ‘self-serving’ demands or to dismiss other people’s concerns for the urgent issues of ‘today’. Also, consider that too much emphasis on the long-run may result in failing in the tasks of here and now. This in turn may mean there will be no ‘long-term’ anymore and thus taking a position that’s too biased towards the future may be self-defeating.
It is a good practice to rotate the role of a Shōrai to different people. This can be done for every decisions/session or even once or twice during the same session. The aim is to come up with the best possible argument from both short and long-term perspective and allow the group to discuss these fully.
It helps to have a facilitator who will ensure the discussion is kept focused, that arguments are presented clearly and that people stick to their roles. The facilitator should also help capture different arguments being presented so that the group can explore them properly as well as actions agreed and decisions made.
Yahaba sessions could be as short or as long as you see fit — it all depends on the number of participants, nature of the decision being discussed and complexity of perspectives. As a rule of thumb though, my recommendation is to keep the session between 30 and 120 minutes. A shorter session is unlikely to enable all perspectives to be meaningfully considered. On the other hand, sessions longer than two hours tend to gradually lose focus and you may find yourselves going in circles.
It may not be practical or desirable to run a full-fledged Yahaba session for every decision. Consider the significance of the decision you are making and the risks involved. Based on that you can determine if and how the session should be run, who should be invited, how long it should be etc.
When running the meeting remotely (e.g. via Teams, Zoom etc) you may find it helpful for the Shōrai to temporarily add the word ‘Shōrai’ to their name. Alternatively, the facilitator may post who the Shōrai are into the meeting chat.
A Yahaba session in practice
Here’s a simple made-up example of how a team-level Yahaba session may look like in practice. In this example session, taking place at a on-line clothing retailer Peak Cloud Ltd., we are observing one of the teams trying to decide about the best approach when implementing a new major capability. At a senior level, the nature of the discussion, the time horizon people are concerned with and the trade-offs involved would, of course, be different. But the example below should illustrate how a simplified Yahaba session may look like and how people in the Shōrai role can play a vital role in illuminating the consequences of decisions being taken.
The participants in this session are:
John — Scrum Master and session facilitator
Claire — Solution Architect
Sandra — Lead Developer
Marcus — Product Owner
Tom — Senior Developer
Anthony — SRE
John: Hi all and welcome to our Yahaba session! Today, we will be talking about a rather significant decision we need to make. As you know, over the next few sprints Marcus would like us to develop a new multi-level filtering capability for customers who are looking to buy one of our products and want to filter based on a range of criteria. We have an existing solution in place and can look to enhance it to support the new requirements but Sandra raised a concern about that due to the complexity of the code in that area. We all know there’s a deadline we are working towards and there’s a lot to do. To make sure we are making an eyes wide open decision about the implementation approach, we decided to run this Yahaba session. We have about 45 minutes so let’s get started. Is everyone ready?
(people nod)
John: Great, thanks! So, first things first, I am happy to be the facilitator today. Is everyone happy with that? Awesome. Next, we need to appoint our Shōrai for today. Any volunteers?
Sandra: Yes, happy to be one of the Shōrai today.
John: Thanks Sandra.
Anthony: Me too.
John: Thanks Anthony! Anyone else? No? Ok, fine, let’s do this. Marcus, do you want to set the context please?
Marcus: Sure. As you know, we have a set of major new product product launched coming up in July and that will significantly increase the number of products and their variants on our website. Having a really good filtering capability is essential and we need to have it on day one to maximise the conversion from the outset. I appreciate there are challenges in implementation of the filtering logic, but we just need to get it done.
Sandra (Shōrai): Thanks, and I get that Marcus. As a Shōrai though, I am now thinking how this will feel like in 6-9 months’ time. Because of the decision being discussed today to simply bolt on a lot of additional logic on top of an already complex filtering mechanism, I am now stuck with a massive amount of complexity in this area of code, it keeps constantly breaking and filtering is now very slow. Our customers complain about it and it is affecting our conversation significantly. The team is now spending a huge amount of time just fire-fighting issues with this logic and we can’t support any more enhancements.
Anthony (Shōrai): I agree Sandra. Operationally, this is nightmare. Because the filter logic is tightly coupled with the rest of our system, any deployment is now very slow and error-prone and we tend to introduce defects which are hard to detect and even harder to fix. The amount of testing we now need to do is huge even for basic changes which have nothing to do with filters!
Claire: Hang on. It can’t be that bad! Yes, things are coupled but it’s working fine now and it’s not causing us that much trouble, is it!
Marcus: Exactly! Plus, as I said, we really can’t afford to delay this — it has to be done by July!
Tom: I agree with Claire that this is not causing issues now, but I want to hear more from Anthony as one of the Shōrai, about the operational issues. Can you please elaborate?
Anthony (Shōrai): Of course. What I am seeing now, speaking as a Shōrai about nine months in the future from the rest of you, is that the code is now so complex that none of the teams are prepared to touch it and the we struggle to test it effectively. Our deployment cycles have increased because of the constant back and forth around broken filtering functionality which nobody can fix. And when John asked us how long it would take to refactor it, the answer was at least three months!
John (facilitator): OK, thanks everyone. Sandra, Anthony, what are you proposing we do?
Sandra (Shōrai): As I said, I now see major issues with the way filtering works because we took the shortcut nine months ago to meet the July deadline…
(Marcus interrupts…)
Marcus: Well, Sandra, if we don’t get this done by July, your concern about complexity etc. will be irrelevant as we will miss our yearly targets, so…
John (facilitator): Hang on, Marcus, let our Shōrai speak please. Sandra, you were saying?
…
(Sandra explains her concerns from a perspective of someone working on the code in the future and together with Anthony adds a number of examples about things that are causing problems)
…
Sandra (Shōrai): …and to avoid this scenario, I propose to split our work over the coming weeks into two streams. Stream one will be about separating the existing filtering logic from the rest of the system which will alleviate a significant part of the operational pain Anthony is talking about as we will remove all the coupling. The second stream will be focused on building the new requirements Marcus mentioned.
Claire: Yes, but that does not solve the code and logic complexity issues you called out Sandra.
Sandra (Shōrai): No, it doesn’t, but it does significantly reduce the operational impact AND means we can still get close to July with the new functionality.
Tom: I am ok with that, but when will we refactor the rest of it? I mean, we will still have a hugely complex code in this area!
Marcus: If it helps, I am fine with blocking up to two months of our roadmap from August to allow for a proper refactoring of this.
John (facilitator): Would that work Sandra? Anthony, Claire — what are your thoughts?
Sandra (Shōrai): Yes, with encapsulation done I think two months works.
Anthony (Shōrai): Works for me. As long as we have an independently deployable and encapsulated module I am happy to go ahead.
Claire: Fine with me too. But I will need time over the next few weeks to work with Tom and Sandra to shape how the design for the new filter will look like so that we can hit the ground running in August. Can we put some stories in for that John?
John (facilitator): Sure. You happy with that Marcus?
Marcus: Yep, I am fine. Thank you.
John (facilitator): OK, great. Seems like we have a way forward. I will capture the actions and the summary of our conversation and share it with you. Thanks all for this session!
Conclusion
As you can see, this technique can be used at various levels of the organisation (from the exec board, to teams and down to individuals) and in many different contexts and to varying degrees. My recommendation is to try to include an element of Yahaba in any decision you are making. After all, considering future concerns is a healthy habit to develop and it will also help you hone your systems thinking skills. I hope I managed to make a good case for the Yahaba technique and encourage you to try it and make it part of your toolkit when trying to make important decisions.
The Yahaba technique is, of course, not a panacea and it will not magically enable you to solve all your problems. For it to work, people need to be engaged, argue in good spirit, be prepared to listen to each other and consider a range of competing perspectives. Having said all that, just the act of giving a ‘voice’ to the future people can be immensely valuable and is very likely going to help you and your team or organisation to make much better decisions and achieve better outcomes. And in my mind, that’s well worth it!
How I think about: Organisational Maturity
Introduction
This is a third post in my ‘How I think about…’ series. If you are interested, feel free to read my previous posts about Uncertainty and Productivity respectively. In this article I hope to illustrate what the concept of organisation maturity means to me, why it matters, what antipatterns I often observe and how growing maturity translates into superior business outcomes.
In the business (and especially IT) context, over the last 20 years a lot has been written on the topic of maturity. CMMI was (Or still is? Anyone still using it?) one of the ‘foundational’ maturity models many organisations adopted during the 1990s. Many agile ‘frameworks’ these days come with some kind of maturity model (including Kanban which, as a community, resisted the idea of a maturity model for a long time) and, of course, there’s a myriad of de-facto proprietary (agile) maturity frameworks and models developed by various organisations, consulting firms and communities. Some are helpful, some are useless, some are effectively harmful.
This post is not about any specific model or framework, or a specific implementation of a given maturity model, its pros and cons. Instead, I will focus on the concept of maturity as such (more precisely growing maturity) and how the way maturity growth is approached influences the overall outcomes at a team and organisational level. Going forward, I will use the term maturity as a generic term for both team and organisational maturity.
What is maturity?
There are, of course, many ways how one can define and think about maturity. In the context of this article I define maturity as follows:
Organisational maturity is a measure of organisation’s ability to attain complex objectives optimally.
I appreciate this may not be how you think about maturity but please bear with me. The key word in this definition is optimally. And by achieving objectives optimally I mean achieving them in the best possible way which integrates a range of organisational priorities and considerations both in the short- and the long-term. There are naturally many aspects that matter, but a couple of examples to bring this to life — an organisation that launches a new product in a record time, but with a series of major security flaws is not mature. Similarly, an organisation whose overly rigid controls and processes prevent it from innovating and create customer and business value at pace is not very mature either.
We can see that, on this definition, maturity requires holistic thinking. And that’s because achieving outcomes optimally requires a good understanding of what factors influence success, what matters to us, what is the hierarchy of these factors, which of them are specific to us and which pertain to the world around us and how we intend to balance them when we are facing tough choices. Only with this in mind, we can start thinking about how we ought to structure our maturity approach, what dimensions it should have, what qualities and factors we optimise for and how this relates to our understanding of success. At this point, using one of the existing models is a good way to stimulate the conversation about all these aspects and examine them from different perspective but be wary on converging on a specific model/framework too soon.
Finally, we ought to recognise that the term (organisational) maturity integrates a range of areas some of which are independent while others are deeply intertwined. Therefore, when we speak about maturity, I really refer to a collection of components including technical (software engineering, architecture etc) maturity, operational maturity, product delivery maturity, financial, governance and process maturity etc.
Crucial point to note — please note that ‘mature’ on my definition does not automatically imply comprehensive, complex, or all-encompassing. Instead, being mature really refers to the overall level of a set of organisational capabilities and as needed (their fitness for purpose) to succeed in the relevant context and market.
Organisational vs. Team Maturity
Organisations usually contain a set of smaller sub-units. Some sub-units can be fairly large (division, department), others can be smaller (team) and others smaller still (individual). When thinking about maturity, we need to carefully consider the remit of our efforts and understand which elements are best tackled at team-level and which are really a departmental or organisational concern. Failing to recognise this difference leads to one or more of the antipatterns listed later.
Why maturity matters
So why is maturity relevant? Why should we really care? And to what extent is focusing on growing maturity justified? A lot of the answers are, of course, related to what we care about. Your approach to growing maturity is, at its heart, about your approach to continuous improvement. Increasing maturity is about investing in continuous improvement across (typically) several dimensions and on multiple levels. How mature is ‘mature enough’ really depends on your definition of optimal and success.
What are the consequences of creating a highly mature organisation? As the definition I suggested above implies, it means such organisation will be really good at attaining its goals optimally. In practice, any highly mature organisation will be very effective in things like:
creating value for its employees, shareholders, and customers in a sustainable way
balancing short-term needs and long-term strategic priorities
responding and adapting to change (externally and internally induced) rapidly
developing services and products that solve real-world problems and/or create new markets and opportunities
maintaining healthy balance sheet and profitability
etc.
These are all highly desirable traits that most organisations care about and that is why growing maturity (as one of the core elements of a continuous improvement mindset) is such a critical element of creating success.
Antipatterns seen way too often
Too much or too little
As with most things, there’s a range of antipatterns that organisations and teams can (mostly unknowingly) fall into. One that probably happens most often is a failure to balance investment in maturity wisely. In most cases this has the form of dedicating too little time and effort to growing maturity. This usually stems from a lack of understanding of the current level of maturity (typically due to being overly optimistic or outright deluded) and tends to be coupled with a culture that is big on hubris and small on introspection and open feedback. This is especially lethal as it tends to produce a gradual and hard-to-revert organisational decline. Usually, such organisations realise there’s a major problem only when it’s too late to do something about it without drastic and painful measures. In extreme cases this leads to the organisation going out of business due to lack of competitiveness and failures in operational capability.
The other variant of this antipattern (i.e., ‘doing too much’) tends to be quite rare and usually has a form of trying to grow maturity in places where only marginal impact on overall org performance can be achieved. It basically means chasing more and more improvement in some capability even though such an improvement cannot translate to a correspondingly large business advantage.
The ‘We will do it later’ fallacy
This antipattern is different from the first one in that the organisation is (largely) aware of the need to grow maturity in one or more areas or disciplines but is unable or unwilling to commit the time, effort and investment needed to do so. Usually, the reasoning behind this stance goes as roughly follows: “We recognise that we have major deficiencies in area X, but addressing these would divert capacity, attention, and resources away from delivering our project/service/product/feature. We cannot afford to do this now as delivering on time is essential. We will, however, invest in improvement at a later stage once this project/service/product/feature is delivered. Then we will have more time and breathing space to focus on improving maturity.”
This is typically a fallacy and that’s due to several factors. Here are some of the main ones:
As long as the org exists and as long as it aims to remain competitive, there will always be another project/product/feature to deliver. Therefore, there will never be an ‘optimal’ time to start focusing on growing maturity as the ‘We will do it later’ argument can always be invoked.
Things almost always take longer than we hope. As a result, what may seem like only few weeks/months until our project/product/feature is done can very easily turn into a much longer period. This then creates more pressure to ‘catch up’ which, in turn, makes it even less likely to find time to grow maturity.
Growing maturity is, by the very definition, about increasing the organisation’s ability to succeed — to attain its objectives optimally. The lower the maturity, the harder it is to succeed. Delaying focus on maturity improvements means that whatever we are doing now is going to be progressing more slowly, with greater risk and less optimally than it could. And whist this may indeed be justified in some scenarios (e.g., when completion of some business-critical initiative is less than few weeks away), in most cases it only means that the thing (project, product, feature etc.) we care about and we have been working so hard on continues to be affected by our immaturity for longer and longer.
Tools and frameworks over outcomes
When thinking about maturity, it is tempting to just pick one of the ready-made maturity models (typically linked to and packed-with specific agile, lean or software delivery and digital ‘frameworks’) and use it as a complete and optimal source of truth and wisdom. Plus, people love check lists. Working towards a set of clearly defined criteria seems like a sensible strategy to grow maturity.
There are real dangers in adopting a maturity model of any kind blindly though. When the model we choose to follow does not match our organisational context, we will not get good outcomes. When we don’t understand what our maturity is today and where we are starting from, we will likely pick the wrong model, focus on the wrong things, and end up in a place that’s far from optimal. When we focus on driving adherence to a specific model and measure success predominantly in terms of model/framework adoption, what we will get is indeed a superficial adoption of a chosen model but usually without the business outcomes we were hoping for.
If we choose to use a specific maturity framework/model, we can’t allow it to dominate the conversation. Instead, it should act as a sort of a guide to enable, encourage, and focus our conversations on things that really matter and that will make a difference to our team and organisational maturity in our context and given our ambitions.
Local optimisations
Local optimisation is a well-known phenomenon from many disciplines from Lean to Systems Thinking. In the context of our discussion here it takes the form of focusing our maturity growth initiatives only on specific, often fairly narrow areas and failing to recognise that we need to look at a system as whole.
A typically example is an over-emphasis on growing team level maturity and obsessing about specific elements of team performance (also see How I think about: Productivity) without noticing that the end-to-end organisational performance may much more be determined by factors that happen ‘outside’ of a team, for instance how work is prioritised, how funding is allocated, how governance works, what controls are used and how, how are incentives aligned etc.
Another example is a situation when organisations focus on growing maturity in areas where they feel they can do so most easily, rather than in areas what they ought to do it most urgently. This may not be an entirely unreasonable position to take, but it does mean that we are giving up on a much larger opportunity to make an impact.
Whilst very common, of the four antipatterns covered in this section, Local Optimisation is arguably the one that ought to be easiest to correct. After all, an organisation that focuses a lot on local optimisation must, at least to some degree, have a continuous improvement mindset and a willingness to invest in maturity. A correction in this context therefore means directing more of organisation’s resources to areas and aspects where maturity growth will yield greatest benefit. This needs to start with establishing a shared understanding of the end-to-end challenges and constraints the organisation is dealing with. Value Stream Mapping is only one of the techniques that can help identify such constraints and then inform where and how maturity growth efforts should be directed.
So, what’s the catch?
If increasing maturity is such a good deal, why don’t all organisations invest more in it? In fact, why don’t we spend most of our effort on growing maturity all the time? As many things in life and in business it comes down to trade-offs. On one hand, as we have seen above, we have strong incentives to want to grow org and team maturity. On the other hand, we often do not have the time to wait until some high level of maturity is reached as we need to act (or deliver something) now. We want to get better, but we have deadlines to meet and products to deliver. And some of the maturity improvements will take months (if not years) to achieve which means we need to find a balance.
How do we resolve this conundrum? The best way to think about it is in terms of Polarities. I find Polarity Thinking (you can read a simple primer e.g., on the Sloww website) quite helpful in situations like this. In a nutshell, growing maturity (or continuous improvement) and ‘getting things done now’ is not a problem to solve, but a polarity to manage. We need to try to get stuff done AND, at the same time, grow our maturity. And we need to do it without continuously flip-flopping between positive and negative sides of both ‘poles’. Doing this well means that growth in maturity enables, supports, and accelerates the product/service/value creation efforts of the organisation without unwanted side effects.
How to grow maturity in practice
If you managed to make it all the way to this point, I hope that by now I managed to make a strong case for focusing on team and organisational maturity growth as a key component of running a successful business. So how do we do this well? What are some of the ways to avoid the common antipatterns? What can we specifically do in practice to enable a sustained maturity growth? How can we manage this polarity effectively?
Know where you (really) are
As is often the case, the first step is to ground our thinking in empiricism. We need to be confident that we understand our current condition well enough.
Questions we ought to be asking ourselves are:
Do we really understand what level of maturity exists in our organisation today? How do we know this this?
How do we guard against hubris, organisational delusion, reality distortion bubbles and other biases driven by past decision, political influences, power dynamics etc?
How do we ensure we have an open, honest and safe conversation about status quo?
What data can we use to corroborate or challenge our qualitative understanding?
Decide what you are aiming for
Assuming we do have a decent understanding of where we are now, we now need to determine what good looks like in our context (perhaps with a help of one or more existing models) and which elements of the ‘organisational landscape’ we will focus on and why. This is not trivial as it is tempting to try to boil the ocean and spread ourselves way too thinly. Not everything can be a priority. Not everything matters in the same way, at the same time and to the same degree. Tough choices need to be made.
At this point we need to be able to link the anticipated growth in maturity to a set of outcomes we are hoping to achieve and a set of indicators that will give us confidence that we are on the right path.
Some questions we should be asking here are:
Which areas do we need to focus on and why?
What outcomes do we hope to achieve?
What level of change/improvement/maturity growth do we feel we need and why?
How will we measure success?
What is the first step we can take in each area?
What are the main organisational, cultural, structural, process and technical obstacles in our way? How do we eliminate them?
Determine what’s needed to get there
The next step might then be determining the scale of investment, priority and focus our maturity growth strategy will require. This is, of course, context specific and the answer will be different in different parts of the organisation and when talking about different aspects of maturity. But being explicit about it and having a clear, ideally data-driven reasoning for our position is vital. It enables us to then tailor next steps accordingly and limit the risk of local optimisations.
Questions we might ask here are:
What is the investment (time, money, people etc.) we ought to be prepared to make?
How do we ensure this investment is available and will be sustained for as long as is needed?
Where do we need a ‘short & sharp’ action and where do we need go ‘slow & steady’?
Which elements of our maturity strategy are team-driven, and which require a broader change/focus/support?
How will senior stakeholders actively support these efforts in practice?
Make it happen
Finally, we need to take clear steps to enable maturity growth to happen. After all, execution really matters and if we can’t establish an environment within which our ambitions can realistically be achieved, our efforts will fail. In practice, this may mean many things including:
changing how our backlogs look like and what our teams are focusing on
changing our product roadmap
stopping or delaying existing initiatives to create space of maturity-focused work
changing existing and developing new incentives for people, teams, and leaders
changing what and how we celebrate and recognise
developing policies to ensure that an appropriate level of funding and focus on maturity growth is always in place
reviewing what we measure, how and why
making organisational and leadership changes
tailored communication to establish visibility of our efforts
changing what we measure and how
Conclusion
It is my strong belief, that most organisations under-value the importance and the benefits of maturity growth and, as a result, do a rather poor job at it. Making trade-offs between value here and now and (much higher value) in the future is challenging. Saying ‘No’ to things we really want in order to create space to drive maturity growth can be painful. Yet, growing maturity in the right way is one of the most impactful organisational capabilities.
Being successful in this endeavour requires a strong alignment and commitment at a leadership level, strong cultural foundations, taking a longer-term outlook, adopting systems thinking mindset and, of course, passionate individuals throughout the organisation who are able and willing to drive maturity growth whilst maintaining balance between building new things and making things better at all times. Luckily, getting it right pays off handsomely in the form of great competitiveness in the market, ability to innovate at pace, operational excellence, engaged people and long-term viability and prosperity of the business. In my mind, these are good enough reasons to think very carefully and in depth about organisational maturity and build it into the fabric of the organisation and to be bold in directing organisational resources to drive maturity growth.
PS: Found an error? Or a typo? Did I get something wrong? Or do you have an idea you’d like to share? Please let me know!
How I think about: Productivity
This is a second post in the How I think about... series in which I am sharing my current views on a range of topics related to leadership, tech, culture, ethics, and strategy. This post is about productivity in knowledge work, common anti-patterns, productivity measurement and how to think about productivity more usefully.
This is a second post in the How I think about... series in which I am sharing my current views on a range of topics related to leadership, tech, culture, ethics, and strategy. I deliberately say ‘current views’ as my thinking continues to evolve and I expect that at some point in the future I may change my view about some of the ideas mentioned in this article — as far as I am concerned that’s a feature, not a bug. If you want to read the first post in this series, have a look at How I think about: Uncertainty.
Why are we so focused on Productivity?
In various shapes and forms and to varying degrees, productivity has been a focus area for businesses, governments, and institutions for centuries. Most of the modern thinking about productivity and its measurement originated during the industrial revolution and is grounded in the ideas of Scientific Management. Overall productivity, seen through the GDP lens, after centuries of stagnation, has been rising exponentially since about mid 1800s mostly as a result of applying technology and scientific method to create more value and accelerate progress. The graphs below from Our World In Data illustrate this beautifully.
Whilst traditionally we tend to think about productivity more in terms of industries like manufacturing in the IT context, ‘engineering productivity’ has become a topic of debates at executive levels of most organisations and a major area of focus for the industry. With so much business value today being dependent partly or completely on technology, it is understandable that making IT as productive as possible is a key enabler for business success. Or is it?
What do we mean by productivity anyway?
In general terms, productivity is the efficiency of creating some valuable output. Typically, it can be expressed as the ratio between inputs (in terms of time, money and other resources) and the desired outputs.
In manufacturing and activities that are based predominantly on a repeatable process, productivity is much easier to measure as both inputs and outputs can usually be quantified well and changes in process translate to changes in output rather clearly. For example, we can easily count the number manufactured items (toothbrushes, cars, steel pipes) and measure how changes in our process, ways of working, technology, knowledge etc. influence the production numbers.
Productivity in knowledge work
The work product engineering teams do is also almost always unique in nature — we rarely do the same thing more than once. We are discovering and learning as we go, we probe, sense and respond. This context is fundamentally different to a repeatable manufacturing process and as such, requires a very different mindset when it comes to thinking about, measuring and optimising productivity.
Since knowledge work and digital product development specifically is what I am most interested in, I will focus the rest of this post on productivity in the context of knowledge work, namely software engineering and digital product delivery. In software engineering and technical knowledge work in general what we understand as ‘productivity’ is indeed multi-faced and encompasses several dimensions like:
Pace: How fast work gets done
Quality: How well work gets done
Satisfaction: How satisfying the work is
Maintainability: How well are long-term considerations taken care of
Communication and collaboration: How well are information and knowledge shared and used
Given all this context, we need to recognise that there isn’t a single metric or indicator that gives us a holistic view of engineering productivity. In fact, reducing productivity to a single number tends to have very negative consequences as we shall see later.
Common anti-patterns
Assuming software engineering is like manufacturing
The biggest anti-pattern I repeatedly observe is organisations applying manufacturing mindset to knowledge work and trying to measure productivity in the same way as one would measure an output of a production line. As we saw above, knowledge work is fundamentally different from a repeatable process we tend to optimise in manufacturing. And while various lean techniques can be very useful in identifying and removing process bottlenecks and thus improving flow, productivity is software engineering takes a range of forms and needs to be thought about very differently.
Trying to measure productivity of individuals
This stems from the myth of 10x developer and the desire to maximise individual performance in hope that this will directly result in an equivalent improvement in the team and org performance. The reality, again, is much more complex and messier. It is true that people’s skills, expertise, added value and impact vary quite significantly and that organisations should always aspire to attract and retain the best talent they can. At the same time, given the nature of productivity in knowledge work, putting a lot of focus on measuring how individuals perform using specific metrics leads to poor outcomes — people feel watched, they tend to start gaming whatever metric is used to measure their performance, they over-focus on demonstrating their own contributions at the expense of the overall output/productivity of the team etc.
Looking for a simple answer
Numbers usually tell only (a small) part of the story. It’s tempting to use whatever metric we can measure as a proxy for productivity. But that is never the full story and over-indexing on any one number is highly undesirable. There are many context-/situation-specific reasons why various productivity-related metrics may go up and down for a given person/team. These need to be understood to truly determine what is happening and why. Over the years, many metrics have been proposed and used to measure productivity. Yet, in my experience, none of them does the job in practice:
Forgetting what we are actually trying to achieve
Peter Drucker and W. Edwards Deming are both credited for saying: “If you can't measure it, you can't manage it.” And as someone who is very focused on organisational performance, I sympathise with this. At the same time, measurement of anything, and productivity is no exception, needs to be seen as a means to an end, not the outcome in itself.
The ultimate outcome we are interested in is NOT how productive a person or a team is, but how much business and customer value we can create at any given time. And to maximise that, we need a good dose of systems thinking and we need to look at the organisation as a whole and examine which factors influence our ability to create value the most. Over-focusing on productivity, especially in the limited and narrow sense outlined in this section, is not going to result in the outcomes we want and will likely cause a range of local optimisations without any major improvement in the top line value creation.
When a measure becomes a target, it ceases to be a good measure - Goodhart's law
Adding people as a go-to method to increase output
Productivity does not scale linearly with the number of people — experience shows that productivity scales roughly following a square root function i.e., to get double the productivity we need to quadruple the team size. But that also means increasing hand-offs, complexity, cognitive load, and coordination. All these aspects will impact productivity in various ways, and we need to consider carefully if increasing number of people in the team or organisation is really the right answer and whether we have the basic structural conditions and technical and organisational maturity to make scaling worthwhile.
Productivity as an IT issue
Way too often I see organisations considering productivity as a challenge that needs to be addressed first and foremost by IT/Engineering as if it was only IT that participates on value creation. This effect is chiefly down to the pre-existing Taylorism-style ideas and also the fact that most IT teams, in some way, tend to measure things related to their productivity. This narrow focus on IT-only productivity inevitably results in local optimisations where a small part of the overall value stream gets all the attention despite the fact that much bigger gains can be achieved by looking at the overall end-to-end flow from idea to operations. As an additional side effect, one part of the organisation feels constantly under pressure and continues to look for more and more ways how
Utilisation = Productivity
This is one of the ideas that, whilst having been debunked a hundred times over, continues to live on and influence decisions, process and measurement of productivity. The intuitive assumption is that in order to be maximally productive, people ought to be maximally utilised. This works well in the manufacturing context where a machine ideally should be running at 100% utilisation all the time as its productive output is directly proportional to its utilisation. This however does not translate to knowledge work. People are not machines. Digital products are not toothbrushes. As shown nicely in this post by Pawel Brodzinski, the optimal utilisation point to maximise value overall is typically way less than 100%.
Should we focus on productivity at all?
In a digital product context, productivity is influenced by a range of factors such as:
the nature and the maturity of the product the person/team is working on
the processes the team and the organisation follow
the maturity of the development environments
people’s levels of clarity and understanding of the desired outcomes
individual skills, experience, and motivation
team and corporate culture and overall work environment e.g., psychological safety, dependability, structure & clarity, meaning & impact of work
In practice, there are many things we care about when it comes to ‘productivity’ as they have real-world effects on the overall performance of the team, the organisation and the amount of business and customer value that gets delivered. Here are a few 'dimensions' all of which contribute to productivity:
Levels and quality of communication within a team and across teams — this results in fewer mistakes, less rework, higher quality, fewer misunderstandings etc.
Adherence to technology principles and standards — this results in less tech debt, less rework, better alignment, better performance etc.
‘Scout mindset’ (I leave the code in a better shape than I found it) — this results in reducing tech debt, lead time getting shorter, fewer defects etc.
New ideas and contributions to team and org improvement — this results in improved team engagement, better collaboration, improved outcomes etc.
Coaching and growing others — this results in improved people retention, engagement, better collaboration, higher quality, better outcomes
Innovation and solution creativity — this results in shorter lead times, solutions fit for purpose and future-proof etc.
Ability to give and receive feedback and reflect on it — this results in better team cohesion, higher team performance and engagement etc.
Ability to learn quickly and apply learnings in practice — this results in better solutions, faster delivery, less rework, better outcomes etc.
As we now see, productivity is not one thing, and we should not fool ourselves into thinking that we can abstract it into a single metric. Instead, if we want to gain better insights about productivity to help direct our actions and team support more effectively, the mindset we need to adopt should be based on:
Focusing on Outcome (business value) as opposed to Output
Paying more attention to progress/trends over time rather than absolute values
Focusing on teams rather than individuals
Focusing team and E2E measurements on flow-based metrics and business OKRs rather than output and ‘productivity’ metrics
Focus on growing team and org maturity (from idea to production) — productivity will follow
Consider using a team-based scorecard based on a range of measures rather than a single metric and make targets team specific
Interpreting all metrics WITHIN the relevant context (i.e., in other words, numbers without context are often meaningless)
Evaluating individuals’ added value based on feedback from peers and relevant domain experts (as opposed to purely data)
With all that said, there are many things that we can and should do to help engineers and teams to become more productive, regardless of what metrics we choose to use, for instance: limiting context switching and keeping levels of work-in-progress relatively low, having a clear understanding of product roadmap, the desired outcomes and customer needs, having time to invest in addressing technical debt on an ongoing basis, managers actively removing impediments which are beyond control/influence of teams, eliminating dependencies etc.
So where does this leave us?
We need to think about productivity in knowledge work very differently and need to leave behind old paradigms which do not translate into the digital context. We need to appreciate that productivity is complex and multi-faceted and as such needs to be evaluated and considered from a range of perspectives and with a multitude of insights. We need to see productivity as a multi-dimensional phenomenon not as a two-sided coin. We must not forget that any measurement is context sensitive and that failing to understand the context will almost certainly lead us astray.
Insights and metrics pertaining to productivity are helpful to understand what is true and what and how we should change, but we must get blinded by them — instead of obsessing about individuals’ productivity, businesses should concentrate more of their energy and resources on the value-creation capabilities of the organisation as a whole.
PS: Thanks to Lewis Jardine, Sandra Atkins, Susanne Franke, Lien DePlancke and Wayne Palmer for their ideas, critique, and contributions to the original version of this post.
PPS: Found an error or a typo? Did I get something wrong? Or do you have an idea you’d like to share? Please let me know!
How I think about: Uncertainty
I hope to provide a perspective on the phenomenon of uncertainty and introduce some basic concepts and approaches which help to navigate uncertainty in business and in life more successfully.
This is part one of a series of posts focused on a range of topics, concepts and ideas that, in my experience, significantly influence the level of overall success in any organisation. Some posts will be more about specific practices and approaches while others, like this one, will be more conceptual and aimed at creating a foundational understanding and a basic mental model of a specific idea. In this post, I hope to provide a perspective on the phenomenon of uncertainty and its different flavours and introduce some basic concepts and approaches which help to navigate uncertainty in business and in life.
The urge to ‘know’
As humans, we like to know. We like to feel we are in control. We want to know that we are in charge of our destiny — be it in our personal lives or in the context of the organisation we are part of. And we go to extraordinary lengths to reduce uncertainty. We analyse and compare scenarios, we devise intricate plans, and we identify and manage risks. We do all these things and more in a hope that we will reduce or ideally eliminate the uncertainty. And yet, despite all this effort, we continue to be surprised by what actually happens and how difficult it is to be truly ‘in control’ in reality.
The human desire to be certain, to be in control is natural and it stems from our evolutionary history. Being able to reduce uncertainty and predict the outcomes of our actions or decisions would be hugely valuable. In many areas of life, we pay experts to help us reduce uncertainty (think investment bankers, financial advisors, pollsters, political advisors etc.). Some people even pay charlatans to make them at least feel that they ‘know’ (think astrologers, psychics, fortune-tellers etc). And most of us spend money to reduce possible negative impacts of uncertainties in life (think insurance).
We crave certainty because the feeling of being uncertain is something deeply uncomfortable, distressing and altogether undesirable. It creates anxiety, makes decision-making harder, and forces us to keep changing our plans. Uncertainty can be so uncomfortable that we are prepared to fool ourselves into feeling confident or certain about something even if such feeling is not grounded in any meaningful data or evidence (think most religions). Moreover, an uncontrolled quest for certainty resulting in holding one’s opinions too strongly hinders progress, impedes innovation, perpetuates falsehoods, ignorance and injustice and shuts down conversations (think religious or cultural dogmas, Taylorism applied to knowledge work, superstitions, creationism etc.)
In a nutshell, feeling uncomfortable with uncertainty is very human. And while working effectively towards reducing uncertainty is logical and can be hugely advantageous in life and business, our discomfort with uncertainty can and does often lead to waste and creates real-world harm. Getting good at living with, managing and exploiting uncertainty, therefore, makes a lot of sense.
Where does uncertainty come from?
Uncertainty is all around us. The more you look, the more uncertainty you will find. In fact, even events that we normally tend to consider ‘certain’ (the sun will rise tomorrow) carry a degree of uncertainty. So where does all this uncertainty come from? At the most basic level, uncertainty is both a physical property of the universe (e.g., quantum mechanical events that can only be described in probabilistic terms) and of human society and the world we live in (e.g., moral uncertainty about what is the right thing to do in a given situation or the impossibility to practically anticipate all effects of all our actions). As we shall see later, while some uncertainties can be reduced, others can’t.
In a business context, uncertainties are inherent to running a business and, as such, they have many forms — political, economic, social, structural, financial, organisation, technical etc. As a result, businesses spend billions on dealing with uncertainty. Yet, in many cases, a lot of this investment is wasted and yields zero or even negative value.
Types of uncertainty
It’s important to recognise that there are several distinct types of uncertainty and dealing with each type of uncertainty requires a different approach and a different mindset. In addition, not all uncertainties can be meaningfully reduced or eliminated, some not even in principle.
The main types of uncertainty include:
Epistemic uncertainty — being uncertain due to a lack of knowledge or understanding e.g., I am uncertain about the colour of the t-shirt my son is wearing today
Aleatory uncertainty — being uncertain in principle e.g., when flipping an (ideal) coin, I am uncertain if I will get heads or tails.
Ontological uncertainty — different parties in the same interactions have different conceptualisations about what kinds of entities inhabit their world, what kinds of interactions these entities have and how the entities and their interactions change as a result of these interactions.
Semantic uncertainty — different participants in the same interactions giving different meanings to the same term, phrase and/or actions. The words and concepts that we are using are inadequate to describe what we are trying to explain.
In the case of epistemic uncertainty, if we gain more information (e.g., by doing research, taking more measurements, conducting tests etc) the uncertainty can be reduced.
Aleatory uncertainty, on the other hand, is irreducible in that there will always be variability in the underlying variables. These uncertainties are characterised by a probability distribution and need to be dealt with as such.
Ontological uncertainty is about different people (teams, organisations, social groups) having different mental models about the state of the world, and/or different perception of how cause and effect work in the given context.
Semantic uncertainty, similarly to Ontological uncertainty, results in communication and interaction issues, confusion, misunderstanding and conflict as people talk cross-purposes. The latter two types of uncertainty can be reduced e.g. by explicit definition of terms, their meaning and public ‘validation’ of these terms to create a shared language and taxonomy across a team or an organisation. One should also openly examine and discuss existing perceptions and mental models that different parts of the organisation have (e.g. using Causal Loop Diagrams and other techniques) and seek a common interpretation.
In organisations, we encounter all four types of uncertainty regularly, yet we rarely think more deeply about what type we are dealing with, and if and how can it be effectively reduced. Way too often we implicitly assume that all our uncertainties are epistemic in nature.
Finally, uncertainty is, of course, very closely related to the concept of complexity. Highly complex systems, organisations, processes, and interactions often tend to create or amplify existing uncertainty as relationships between cause and effect are unclear and emergent and most actors have only a limited understanding of the system as a whole. In such systems, most actions have a range of second, third and higher-order consequences some of which are highly uncertain. Meaningfully reducing such uncertainty often requires a more fundamental rethink of the organisational model, process, structure, technology, the flow of work or other core ‘properties’ of the system as a whole and reducing the overall complexity.
What can we do?
Organisations need to get much better at embracing uncertainty.
By ‘embracing’, I specifically mean taking steps to:
recognise when a specific uncertainty exists
understand the nature of the uncertainty
clearly describe its nature and properties and carefully think about them
assess the cost AND the value of reducing/eliminating the uncertainty
where it is meaningful, effectively reduce the uncertainty (but only to the extent warranted by the cost/value analysis)
establish a culture and technological, organisational, structural, and systemic capabilities which enable working effectively in the presence of uncertainty and respond to change
create and foster organisational ’antifragility’ on multiple levels
The above should be read less as a checklist and more as a set of practices, qualities and organisational traits — a mindset about uncertainty. In this context, I would like to emphasise that reducing uncertainty comes at a cost. To reduce the level of uncertainty, we will need to invest time, effort, money, and other resources. This, of course, promises a reward in a form of greater confidence, better predictability, or more optimal and less risky decisions. However, it also comes at a cost which can be very significant. Way too often costs of reducing uncertainty remain hidden and are not explicitly talked about. This can result in an excessive (and ultimately futile) focus on uncertainty reduction (e.g., through more analysis, planning, detailed specifications etc) even though a focus on organisational, technical and process changes enabling effective uncertainty management and real organisational responsiveness/adaptability/agility would have been a much smarter and more productive strategy. Finding the balance is hard and, as is often the case, it all starts with the right conversation.
In practice, there are many ways, tools and techniques which enable us to work with uncertainty better. Context, as always, matters but, in general, when navigating uncertainty I find ideas like the Cynefin framework, Bayesian inference, Decision Theory (including basic concepts like Minimax, Maximin), Systems thinking, and Probabilistic forecasting very useful. When used in the right way with the right level of understanding of the uncertainty we are dealing with and the overall system or area we operate within, these approaches yield remarkable results and enable us to fundamentally change the conversation and the way we look at dealing with uncertainty.
Uncertainties, like taxes, are here to stay. Trying to completely eliminate them or ignore them is dangerous, can be very costly (and futile) and leads to bad outcomes in the long term. Uncertainties also create opportunities. Exploiting these opportunities requires teaching ourselves to live with, embrace and take advantage of uncertainties around us. It means getting comfortable with the discomfort, thinking more deeply about the uncertainties we need to work with and reframing the conversation we are having. Organisations that embrace uncertainty tend to reliably outperform those that don’t — both in the short and the long run. Good luck!
PS: Found an error? Or a typo? Did I get something wrong? Or do you have an idea you’d like to share? Please let me know!
J.S. Mill: On Liberty - Quotes
Below are some of my favourite quotes from John Stuart Mill’s On Liberty — a truly groundbreaking work of philosophy and ethics which remains very relevant today. Mill was one of the people who were decades ahead of his time and his ideas laid foundations for modern ethics, feminism, equality, free speech and liberal order. Together with his uncle Jeremy Bentham, Mill is also regarded as the father of Utilitarianism.
Below are some of my favourite quotes from John Stuart Mill’s On Liberty — a truly groundbreaking work of philosophy and ethics which remains very relevant today. Mill was one of the people who were decades ahead of his time and his ideas laid foundations for modern ethics, feminism, equality, free speech and liberal order. Together with his uncle Jeremy Bentham, Mill is also regarded as the father of Utilitarianism.
A time, however, came, in the progress of human affairs, when men ceased to think it a necessity of nature that their governors should be an independent power, opposed in interest to themselves. [page 6]
What was now wanted was, that the rulers should be identified with the people; that their interest and will should be the interest and will of the nation. [page 6]
Like other tyrannies, the tyranny of the majority was at first, and is still vulgarly, held in dread, chiefly as operating through the acts of the public authorities. [page 8]
When society is itself the tyrant —society collectively, over the separate individuals who compose it—its means of tyrannising are not restricted to the acts which it may do by the hands of its political functionaries. [page 8]
Society can and does execute its own mandates: and if it issues wrong mandates instead of right, or any mandates at all in things with which it ought not to meddle, it practises a social tyranny more formidable than many kinds of political oppression, since, though not usually upheld by such extreme penalties, it leaves fewer means of escape, penetrating much more deeply into the details of life, and enslaving the soul itself. Protection, therefore, against the tyranny of the magistrate is not enough: there needs protection also against the tyranny of the prevailing opinion and feeling; [page 8]
There is a limit to the legitimate interference of collective opinion with individual independence: and to find that limit, and maintain it against encroachment, is as indispensable to a good condition of human affairs, as protection against political despotism. [page 8]
The rules which obtain among themselves appear to them self-evident and self-justifying. This all but universal illusion is one of the examples of the magical influence of custom which is not only, as the proverb says, a second nature, but is continually mistaken for the first. [page 9]
People are accustomed to believe, and have been encouraged in the belief by some who aspire to the character of philosophers, that their feelings, on subjects of this nature, are better than reasons, and render reasons unnecessary. [page 9]
[…] that the sole end for which mankind are warranted, individually or collectively, in interfering with the liberty of action of any of their number, is self-protection. That the only purpose for which power can be rightfully exercised over any member of a civilised community, against his will, is to prevent harm to others. His own good, either physical or moral, is not a sufficient warrant. [page 13]
In the part which merely concerns himself, his independence is, of right, absolute. Over himself, over his own body and mind, the individual is sovereign. [page 13]
Despotism is a legitimate mode of government in dealing with barbarians, provided the end be their improvement, and the means justified by actually effecting that end. Liberty, as a principle, has no application to any state of things anterior to the time when mankind have become capable of being improved by free and equal discussion. [page 13]
I regard utility as the ultimate appeal on all ethical questions; but it must be utility in the largest sense, grounded on the permanent interests of man as a progressive being. [page 14]
A person may cause evil to others not only by his actions but by his inaction, and in either case he is justly accountable to them for the injury. The latter case, it is true, requires a much more cautious exercise of compulsion than the former. [page 14]
This, then, is the appropriate region of human liberty. It comprises, first, the inward domain of consciousness; demanding liberty of conscience, in the most comprehensive sense; liberty of thought and feeling; absolute freedom of opinion and sentiment on all subjects, practical or speculative, scientific, moral, or theological.
Secondly, the principle requires liberty of tastes and pursuits; of framing the plan of our life to suit our own character; of doing as we like, subject to such consequences as may follow: without impediment from our fellow-creatures, so long as what we do does not harm them, even though they should think our conduct foolish, perverse, or wrong. Thirdly, from this liberty of each individual, follows the liberty, within the same limits, of combination among individuals; freedom to unite, for any purpose not involving harm to others
The only freedom which deserves the name, is that of pursuing our own good in our own way, so long as we do not attempt to deprive others of theirs, or impede their efforts to obtain it. Each is the proper guardian of his own health, whether bodily, or mental and spiritual. Mankind are greater gainers by suffering each other to live as seems good to themselves, than by compelling each to live as seems good to the rest. [page 15]
[…] religion, the most powerful of the elements which have entered into the formation of moral feeling, having almost always been governed either by the ambition of a hierarchy, seeking control over every department of human conduct, or by the spirit of Puritanism. [page 16]
Let us suppose, therefore, that the government is entirely at one with the people, and never thinks of exerting any power of coercion unless in agreement with what it conceives to be their voice. But I deny the right of the people to exercise such coercion, either by themselves or by their government. The power itself is illegitimate. [page 19]
[…] the peculiar evil of silencing the expression of an opinion is, that it is robbing the human race; posterity as well as the existing generation; those who dissent from the opinion, still more than those who hold it. If the opinion is right, they are deprived of the opportunity of exchanging error for truth: if wrong, they lose, what is almost as great a benefit, the clearer perception and livelier impression of truth, produced by its collision with error. [page 19]
We can never be sure that the opinion we are endeavouring to stifle is a false opinion; and if we were sure, stifling it would be an evil still. [page 19]
All silencing of discussion is an assumption of infallibility. Its condemnation may be allowed to rest on this common argument, not the worse for being common. [page 19]
[…] for while every one well knows himself to be fallible, few think it necessary to take any precautions against their own fallibility, or admit the supposition that any opinion, of which they feel very certain, may be one of the examples of the error to which they acknowledge themselves to be liable. [page 20]
Yet it is as evident in itself as any amount of argument can make it, that ages are no more infallible than individuals; every age having held many opinions which subsequent ages have deemed not only false but absurd; and it is as certain that many opinions, now general, will be rejected by future ages, as it is that many, once general, are rejected by the present. [page 20]
There is the greatest difference between presuming an opinion to be true, because, with every opportunity for contesting it, it has not been refuted, and assuming its truth for the purpose of not permitting its refutation. [page 21]
Strange it is, that men should admit the validity of the arguments for free discussion, but object to their being “pushed to an extreme;” not seeing that unless the reasons are good for an extreme case, they are not good for any case. [page 23]
The real advantage which truth has, consists in this, that when an opinion is true, it may be extinguished once, twice, or many times, but in the course of ages there will generally be found persons to rediscover it, until some one of its reappearances falls on a time when from favourable circumstances it escapes persecution until it has made such head as to withstand all subsequent attempts to suppress it. [page 30]
But it is not the minds of heretics that are deteriorated most, by the ban placed on all inquiry which does not end in the orthodox conclusions. The greatest harm done is to those who are not heretics, and whose whole mental development is cramped, and their reason cowed, by the fear of heresy. [page 34]
No one can be a great thinker who does not recognise, that as a thinker it is his first duty to follow his intellect to whatever conclusions it may lead. [page 34]
However unwillingly a person who has a strong opinion may admit the possibility that his opinion may be false, he ought to be moved by the consideration that however true it may be, if it is not fully, frequently, and fearlessly discussed, it will be held as a dead dogma, not a living truth. [page 35]
He who knows only his own side of the case, knows little of that. His reasons may be good, and no one may have been able to refute them. But if he is equally unable to refute the reasons on the opposite side; if he does not so much as know what they are, he has no ground for preferring either opinion. The rational position for him would be suspension of judgment, and unless he contents himself with that, he is either led by authority, or adopts, like the generality of the world, the side to which he feels most inclination. [page 37]
So essential is this discipline to a real understanding of moral and human subjects, that if opponents of all important truths do not exist, it is indispensable to imagine them, and supply them with the strongest arguments which the most skilful devil’s advocate can conjure up. [page 38]
If the teachers of mankind are to be cognisant of all that they ought to know, everything must be free to be written and published without restraint. [page 39]
As mankind improve, the number of doctrines which are no longer disputed or doubted will be constantly on the increase: and the well-being of mankind may almost be measured by the number and gravity of the truths which have reached the point of being uncontested. [page 43]
If there are any persons who contest a received opinion, or who will do so if law or opinion will let them, let us thank them for it, open our minds to listen to them, and rejoice that there is some one to do for us what we otherwise ought, if we have any regard for either the certainty or the vitality of our convictions, to do with much greater labour for ourselves. [page 45]
Christian morality (so called) has all the characters of a reaction; it is, in great part, a protest against Paganism. Its ideal is negative rather than positive; passive rather than active; Innocence rather than Nobleness; Abstinence from Evil, rather than energetic Pursuit of Good: in its precepts (as has been well said) “thou shalt not” predominates unduly over “thou shalt.” [page 49]
If Christians would teach infidels to be just to Christianity, they should themselves be just to infidelity. It can do truth no service to blink the fact, known to all who have the most ordinary acquaintance with literary history, that a large portion of the noblest and most valuable moral teaching has been the work, not only of men who did not know, but of men who knew and rejected, the Christian faith. [page 51]
Human nature is not a machine to be built after a model, and set to do exactly the work prescribed for it, but a tree, which requires to grow and develop itself on all sides, according to the tendency of the inward forces which make it a living thing. [page 58]
To be held to rigid rules of justice for the sake of others, develops the feelings and capacities which have the good of others for their object. But to be restrained in things not affecting their good, by their mere displeasure, develops nothing valuable, except such force of character as may unfold itself in resisting the restraint. If acquiesced in, it dulls and blunts the whole nature. To give any fair-play to the nature of each, it is essential that different persons should be allowed to lead different lives. [page 62]
Genius can only breathe freely in an atmosphere of freedom. Persons of genius are, ex vi termini, more individual than any other people—less capable, consequently, of fitting themselves, without hurtful compression, into any of the small number of moulds which society provides in order to save its members the trouble of forming their own character. [page 64]
Eccentricity has always abounded when and where strength of character has abounded; and the amount of eccentricity in a society has generally been proportional to the amount of genius, mental vigour, and moral courage which it contained. That so few now dare to be eccentric, marks the chief danger of the time. [page 66]
If society lets any considerable number of its members grow up mere children, incapable of being acted on by rational consideration of distant motives, society has itself to blame for the consequences. [page 80]
But there is no parity between the feeling of a person for his own opinion, and the feeling of another who is offended at his holding it; no more than between the desire of a thief to take a purse, and the desire of the right owner to keep it. [page 82]
The maxims are, first, that the individual is not accountable to society for his actions, in so far as these concern the interests of no person but himself. Secondly, that for such actions as are prejudicial to the interests of others, the individual is accountable and may be subjected either to social or to legal punishments, if society is of opinion that the one or the other is requisite for its protection. [page 91]
A person should be free to do as he likes in his own concerns; but he ought not to be free to do as he likes in acting for another, under the pretext that the affairs of another are his own affairs. [page 101]
The almost despotic power of husbands over wives need not be enlarged upon here because nothing more is needed for the complete removal of the evil, than that wives should have the same rights, and should receive the protection of law in the same manner, as all other persons. [page 101]