Evolution in nature
Imagine you have a bunch of cells forming an organism. Each cell has a copy of the organism's DNA. The fitness measure how successful is an organism in its life, . The better the organism does during its life in the environment it is, the higher the fitness. A mutation is defined as cells having different DNA than the parent cells, due to a wrong copy of the DNA.
While the organism is in a stable environment, it will reach an equilibrium fitness value. When the organism is close to its equilibrium with the environment, the cells that mutate fast are suppressed, favoring the ones that mutate slowly.
A chaotic environment is defined as an environment that changes randomly and you can't predict it's future with certainty. It's also been seen that in a chaotic environment the organisms will favor the cells that mutate fast. Those will be picked by the organism in its effort to adapt to the environment. Once the organism starts adapting to the environment, fitness will stabilize, and the organism will go back to favor the less mutating cells.
Having too many mutations does not always mean the mutated cells will increase the organism fitness, it might take it to death too. Some mutations are good, and those should be the ones the organism needs to keep in order to survive to the chaotic environment. There is a cellular mechanism, which attempts to fix bad mutated DNA in order to prevent diseases and increase overall organism fitness.
Actually this is natural selection, the organisms who can adapt will survive more time, being able to reproduce more. While the ones with a lower fitness have lesser chances and time to reproduce, disappearing over time.
Some organism might found a perfect fitness for the current environment, conquering the world very fast. Later on there could be another environment change that could cause that particular organism, to quickly die, or other organism could mutate and feed from the previous organism. Perfect is enemy of good in this case. Perfect means everything is in order, everything works as it's supposed. If something is not in order or does not work, everything falls apart. While good searches for redundancy allowing the organism to survive if something does not work. It's better to be good at many things, than great at only one.
Evolution in software development
Now imagine your development team is the organism. The people in the team are the cells forming the organism. The organism DNA, which has been copied to each cell, is your development process. Each person has a specific copy of the DNA encoding. Management is the cellular mechanism in charge to control the cells do not mutate with the wrong copy of the DNA.
Predefined DNA
Some methodologies allow you to define the development process. This is similar to create a DNA for your organism. They let you know how the DNA should look like. You can make some changes, adapt it to your environment and use it to your organism. The person or team creating the DNA will attempt to create it perfectly. The DNA is in the organism before the cells that make it. If those cells cannot create an exact copy of the DNA, you need a strong cellular mechanism, to enforce the exact copy, or remove the bad cell of your organism, like a cancer.
Once you get into a chaotic environment, the cells will mutate at a faster rate, thus management will have more work to enforce the perfect copy of the DNA. This is because changing the DNA is expensive. It's expensive because it's complex and it was defined by a 3rd party. It takes more effort to change it than enforce it. The energy of the organism will be wasted on preventing mutations of all kind, reducing the ability of the organism to adapt to the changes in the environment.
Fitness is measured by achieving milestones in the schedule. In a chaotic environment those milestones have to change. You have to rework your project plan, your schedule, etc. to keep you milestones up to date. If you are not able to keep up, this causes wrong measurements of the fitness. Often the resulting fitness will only be seen at the end of the project, when it's probably too late to fix it.
Evolutionary DNA
Looking at Agile approaches I can see the DNA is not injected or defined by 3rd parties. It's grown from the cells in the organism. Each cell gives a piece of their DNA, to generate the organism DNA. The way to measure the fitness of the organism, is working production code at the end of the iteration. Once the iteration is done, the cells are able to improve the DNA causing mutations, exploring better alternatives to their particular environment. Once they get to their next iteration, they will be able to check if the mutation improved or reduced their overall fitness. The cellular mechanism should focus on preventing bad mutations, the ones that reduce overall fitness.
Agile methodologies only define a basic ceremony, which initially you need to follow, unless you are very confident of what you are doing. Once your team starts getting close to the equilibrium fitness, means everybody knows what they have to do, without have to introduce more changes in the DNA. This is the point where you will start favoring less mutating cells. This is the point where you want to get to, have a self organized team, which can react to the development needs.
Why Agile rules are so basic and simple? It is because they aim to set the most basic healthy DNA, from which you can evolve. Why it's so hard to implement them? This is because evolution is not easy. At first, all your cells will fight to make it to the organism DNA. You should start from the most basic healthy organism, and evolve to overcome the problems. Do not start from an advanced DNA, because it is complex to change, and you will end up with a powerful cellular mechanism to enforce it instead of focusing on evolving healthy.
Mutating the DNA and enforce it, is no longer a management task. Management can focus on avoiding bad mutations, which could cause diseases in the organism. This is not always easy, because there are a lot of cells in the universe which have a really twisted DNA. Starting from a wrong DNA, could turn your project into failure because you will need lots of iterations to get to a point where things start happening. If you do not have good cells in your team with healthy DNA, you need to invest precious time and money on training, and aim to maintain your good ones.
Healthy evolution
When you start from the basic organism, to evolve in the right path you need to focus on the basic agile principles:
-Focus on improving your fitness with continuous delivery of valuable working software for the customer.
-Focus on decoupled code to prepare your application for changes. Do not create your own internal framework to solve common problems because it increases the complexity when they are not common anymore.
-Focus on testing, to prevent new changes from breaking your application, reducing your fitness in the future.
-Focus on refactoring, change your code base to be simpler, better and avoid duplication. Learn and architect your application through this.
-Focus on having motivated and healthy cells, because that's what makes better the organism as a whole.
-Focus on interactions and knowledge share. If the cells do not know what to do, they will easily mutate in wrong DNA. This will also give you redundancy, allowing your organism to survive when it lost the head.
How Agile helps to evolution?
There are very important pieces in scrum that will allow and even force you to evolve. One of those is time-boxing. This is why you have time-boxed iterations. This is why all your meetings should be time-boxed. This is why you need to set a limit in the time you spend discussing issues. This is another factor that creates a needed amount chaos for the team to speed up evolution. If you couldn't get any production code at the end of your iteration, you need to evolve. If you couldn't get enough information from the users in the meeting to continue, you need to evolve. If you couldn't prioritize all the tasks you needed for the next iteration, you need to evolve. If all your team is locked waiting on the only person being able to do a task, you need to evolve.
You need to remove obstacles as fast as possible. This creates a better environment for the organism to evolve healthy. If your problems are not quickly overtaken, you team will mutate with wrong DNA in an attempt to continue with evolution, causing bad long term effect and diseases.
Have a small and well prioritized list of tasks to do next. Continuous big shifts in the direction of the project will accelerate mutations. This will create constant chaos, too many mutations and bad fitness measurements to ensure things are in the right path. You will end up with a bad DNA, and no compass to guide you in your evolution.
The team is what makes everything. If your cells are not healthy, they will make your organism sick. Fitness will be reduced, not because of wrong DNA, because the organism is sick. Aim to keep them happy. If you want your puppy to do all the tricks you want, you either need a big stick or bacon in your pocket. The big stick never works in IT, people leave their companies just because they don't like a framework the company is using, imagine if you scream at them. A good piece of bacon is what works. It does not mean overpay your employees, there are many ways to find out what would motivate them, spend time in this, it's important.
Why Agile does not solve my problems?
As I said before, Agile focused on the most basic healthy organism which you can start from and principles you have to follow to evolve healthy. It cannot impose a solution to all your problems, because it will not be able so solve others problems. If Agile would present a complex DNA for you to solve everything, you will not be able to change it and you would need again strong management to enforce it.
It's not paradise; you will get into many problems too. Where to go? What do I have to change to be better? You cannot keep changing everything in your DNA after each iteration. In the end, you will get nothing, even less than if you enforced any other more complex DNA.
If you solve problems you don't have, you are wasting energy from the organism in things that do not increase your fitness. Aim to solve the problems that reduce or prevent your fitness to be higher.
I said you need to evolve to overcome your problems. Sometimes the hardest thing is to realize what your problems are. If you don't know which your problems are, you need to spend more time reviewing the roots of those. If you are not able to deliver working code at the end of the iteration, aim to do the smallest thing possible, and grow from there. If testing of current code is impossible, aim to create code that can be tested, and decouple pieces as you can. Doing the simplest thing exposes your problems clearly. Doing complex things will easily hide them.
You know your problems, but they seem hard or impossible to solve. This case is simpler, try to waste as little as possible to overcome them. For this type of problems, seek help in Lean Software development principles.
Conclusion
Start from the most basic DNA and evolve to overcome problems. Do not start from complex solutions, or you will need strong management to enforce it, wasting energy. Aim to reduce waste from your organism. Aim to be healthy and do anything to increase your fitness; this is in the end, what will allow you to last longer.
sources:
http://www.obitko.com/tutorials/genetic-algorithms/index.php
http://www.talkorigins.org/faqs/evolution-definition.html
http://www.jstor.org/pss/3067634
http://evolution.berkeley.edu/evolibrary/home.php
Ramiro,
ReplyDeleteVery well stated. I have been involved in SW dev for years and I am still amazed at how many companies do not use Agile/Scrum methodologies. I am currently looking for a new opportunity and am amazed at how many companies desire PMI/PMP certification instead of a Certified Scrum master. The PMP method may be fine for building bridges, planes and items that are mandated to be heavily documented and bureaucratic in nature, but this is not the path for good, clean, efficient code development and testing.
Paul