Let’s start with a brief definition

The Dunning-Kruger effect is the phenomenon whereby people with little knowledge tend to overestimate their abilities, precisely because they ignore how much knowledge is necessary to master them.

What is this got to do with Software Engineering? Well, in software engineering you can broadly spot 3 kinds of people, the ones with super high energy to learn and learn fast; the ones who have answer for everything and think they know it all and the ones who actually know a lot but will always answer conditionally with an “if” and a “maybe”. These are just different stages generally we all go through. Let’s see how it really plays out.

Coding (verb) / Coder (noun)

We all generally start here and this when we are excited to learn about coding. We want to solve a fascinating problem, and will read some cookbooks, look at some videos (read Youtube), browse the web for snippets (stackoverflow/github/anything anyone has shared) and cobble something together to solve the problem. Solving a problem (within hours or days) gives an immense rush and feeling of satisfaction that hooks us into doing more of it and again and again.

This is the first stage of learning to write software and it is generally facilitated by REPL(Read–eval–print loop).

Behaviour characteristics

• Can build POCs at rapid pace
• Can deal with unclear requirements as they are happy to iterator many times
• Would be keen to work on different problems rather than improve solutions for the same problem
• Very confident they can solve any problem and do it fast

Developing (verb) / Developer (noun)

We soon reach a stage when we know we can code and hack things together to make it work. We occasionally come across people and opportunities that educates us about more structured way (sometimes enterprise way) of writing software. We hear about coding standards something we never paid attention to earlier. Now we are told this is essential because someone more experienced tells us so. This is when we start our journey in developing software. We develop it based on some golden standard, some industry best practises and some organisational biases. Gospels of software engineering given to us is finite, especially the one our immediate technical leaders refer to and hence we try to read them all and start quoting them to the coders (juniors) who work with us.

Behaviour characteristics

• When asked to build a POC, they would take a lot longer than a hacker because of their biases
• Would tend to over-engineer solutions or under-engineer solutions
• Would be fussy if the requirements are not very clear
• Would be keen to work on the same problem space over and over again to improve their systems knowledge

Engineering (verb) / Engineer (noun)

While we develop solutions for different problems and see the solutions in actions, we also start seeing the flaws in the approaches we used. We start seeing how the so called best practises are not quite the best. We are confused and we talk, we read and we read more. This is when we start discovering that the so called best practise was created within a “bounded context” and the reason why they didn’t work for us is because our problem wasn’t quite the same. Now we obverse the changes in the industry with keen eyes. We hear people debating on different sides of the fence. This is when we start doubting all the knowledge we have learnt, and we start asking “Why”. We start probing everything that we thought we knew to realign our understanding. We experiment new ideas. We create practises (not best practises) and we measure the impact of those practises to know their effectiveness. This is when we realise engineering is not just about developing solutions but engineering is “building more with less” and engineering is about having the “mechanical sympathy” for the systems.

Behaviour characteristics

• Would be able to build POCs as rapidly as the coders/hackers when convinced problem is worth solving
• Will be on a lookout for complex problems to test their knowledge and skills
• When presented with new requirements, they would want to analyse it from different angles to assess different ways to solve it or even tweak the requirements

The Dunning-Kruger peak

Caution: If you feel this peak doesn’t exist, maybe you are on it right now and best to check in with your peers and mentors

Why are we referring to the Dunning-Kruger peak of ignorance as “The peak”? Well, because there is only one peak in the process of learning and it is not something to be proud of. On the other hand, we can aspire to have it as low as possible and ensure it has a steep descent.

Cases where I have spotted a very high peak (and some will even refer to it as a plateau), is when technologist stay in the same setting for too long. A setting can be the same organisation, the same product or the same industry. There is no one explanation for this but it is important to understand why it happens. Let’s explore this in the case of “the same organisation”. In order for organisations to be in study state (please do not read it as successful), sometimes innovative and disruptive thinking is not promoted because of the high cost it comes with. If the business is not willing to spend on innovation, people do not get a natural playground to explore and learn. This means people are left with options to either use their personal time or stick with what is being told. You can imagine what the latter would lead to. Similar learning restrictions apply when people move projects but within the same product space or even industry though probability of the restriction is lower than sticking in one single less diverse organisation.

When looking for opportunities as a technologist, always ask yourself, what would you learn, does the business have the need to innovate, are there engineers in the organisation or is it run with people standing on the summit.

The End

Having worked in various industries and with diverse group of technologist, I have found these phases to be very consistent (and applies to me too, ofcourse). I had the “Ah ha” when someone shared the Dunning-Kruger graph in a blog. It helped me be more conscious of my biases and look beyond them. In my opinion, a way one can avoid a high “The peak”, is to build a habit of asking yourself “Why”.

And finally (for fun) … if someone says they are at “Their peak of their career”, well, they really might be on one