The Deadly Silent Debt

I heard many times the concept of technical debt. I also realized it's terribly hard not to acquire it and it will keep increasing in your application all the time as you build it.

I will start with the first question...

What is technical debt?

Technical Debt is a detour from the right system to reduce development time.

It's called technical debt, because it behaves like a real monetary debt. You get a fully functional system without investing all the time and effort it needed to get it right. Just like a monetary debt, you need to return a capital plus interests. Depending on the amount of technical debt you acquire, the amount of interests will grow faster or slower. The sooner you pay your capital debt, the less you spend in interests. The longer it takes you to repay the technical debt, the interests will keep adding, making it at some point economically inviable to be repaid.

How do I acquire technical debt?

Well, as I said before, it's really easy to acquire it, temptation is always there to do the quick thing and keep going. I think there are four ways to increase your technical debt. A mix of intentionally, unintentionally, planned and unplanned.

Planned Intentional Technical Debt

The more formal definition of technical debt, implies it's consciously acquired to get something in the market faster to start making money sooner. That money that is made earlier on, will allow the company to survive or make the difference to repay the technical debt. Technical debt in this case, it's something planned and done intentionally. It's done as part of a business strategy which usually leads to a bigger profit. If it's planned correctly, and the risks are known it's not a bad thing.
Examples:

• We need to release the new version of the Iphone OS today, before Microsoft announces their Windows Phone 7 next week. We will add multitasking for Iphone 3GS now, and someday we will add it to the 3G too.
• We will release our next version of the application using the built-in user repository, and later on we will integrate it with CAS to support single sign on.

Planned Unintentional Technical Debt
Well, this is the the type of technical debt added into the system because of external constraints. It could be because you cannot get senior developers. It could be because another team in the organization needs something working in less than a week. It could be due to a corporate policy: We will only hire junior developers and train them.

We know we will be adding technical debt due to this constraints, so we plan for ways to reduce the capital we are borrowing. Also planning a way to repay the capital once the constraints are gone.


Examples:

• Junior members in the team might add technical debt because the lack of knowledge so we will do code reviews.
• Eventual tight deadlines might require the development team come up with a quick solution, and then refactor it to what we think is the right one.

Unplanned Intentional Technical Debt

In this case, I refer to the technical debt constantly added into the system. The unplanned intentional technical debt, is caused by constantly wrong desitions of management and the development team. This is the technical debt added when we are rushing things all the time. This is the type of debt added by the lack of a well defined development process or because it's not followed intentionally. This is the type of debt that rottens the guts of the systems and breaks the will of the ones who are trying to do the right system.
Examples:

• We will not be writing any test into the code because we are in a hurry this month. 
• Just add another if statement and move on, they want it today and we have a pile of work for 5 months waiting for us tomorrow.
• Submit the code now, I will do a quick check on the test server and then we promote it to production tomorrow.
• I need to leave early today, so I will submit my fix now, and someday I will fix it properly.
• I don't know how to use hibernate, so I will do it with a hardcoded SQL statement now, and hopefully someday somebody fixes it properly.

Unplanned Unintentional Technical Debt

This is very similar to the Planned Unintentional case. It's caused by the same constraints, but the big difference is that nobody plans or wants to spend time fixing them. This is due to resource constraints plus constant bad choices from the management team.
Examples:

• When we ask you to do the invoice module in two days, we thought you were doing it also right. 
• I cannot understand, you have 3 months of experience in java, and you never heard of the proxy pattern?. 
• I just added another IF there, there are already 30 of them in that method so I guess another one will do no harm.
• That java class with 10K lines of code started with just 5 lines a few years ago.

What can I do to reduce technical debt?

The first step would be to try to stop increasing the technical debt. I know, it's already a mess, all the developers complaint about it. Some of them would love to put the latest framework on it as it would be the silver bullet for your technical debt. And also, yes... you are right, each task will take you a lot longer now, as you need to fix countless problems for each thing you have to change. Yes, that's what you get by borrowing all the time and never paying it back. But, be sure that paying back some of it each time you have to do something, as little as it may seem, will make the difference

I can tell you how you can get more than 2 millions in technical debt. Borrowing one buck at a time. I can also tell you how you can repay it. Paying back one buck at a time. Of course, keep in mind it won't happen over night.

It's also very important to make technical debt as visible as possible. Having technical debt visible, will make the team to start taking it into account. Will help the team to stop increasing it.

Conclusions

I think planned debts, are not actually a problem in any company. Making technical debt part of your business strategy, when it's carefully planned, can be the difference of keeping your job in 2 months or not. Planning ways to mitigate technical debt your are not expecting and not doing intentionally is also a healthy way to do software development.

The unplanned debts are the problem. This is when technical debt is not considered and ignored by everybody. It becomes a deadly silent debt. It's deadly because the system will likely be considered to be rewritten even before it's finished, and it's silent because management doesn't want to hear about it. There is no worst deaf, than the one who doesn't want to hear. Without management knowing where you are, they will not plan ways to get you out of it.

Something is part of your behavior when you do it naturally and effortless. Something is part of your character, when it costs you more than you were willing to give for it. Managing and reducing technical debt must be engraved in management character, and not just part of it's behavior. That will be the only way for technical debt not kill your application.

Speed, it's all about speed!

In a previous post, I compared Agile methodologies with evolution in nature (Agile, Chaos and Evolution). There I compare the fitness of and organism with the amount of working software the team can deliver in an iteration. Increasing the amount of software delivered (I will call it velocity from now on) is equivalent to increasing life expectancy in nature.

This means velocity is critical for today's Software Development. If you want to last longer, you need to increase your velocity. The team needs to do everything they can to improve it.

Increasing the velocity, many times turns into a problem, rather than a solution. Instead of finding better and faster ways to do things, they just aim to cut what doesn't seem important, and you end up with thousands lines of code in production without any testing, documentation or even logic. That would be like trying to go faster with all your wheels flat. It might seem you go faster in the beginning, but you are putting in risk everybody on the car if the trip is long.

I will go over some concepts and show you some ways to improve your velocity without putting any life in risk. 

Closest Train Station

Imagine somebody asks you to take him to the train station. You can take him to any of the station of a particular train line. You will just look the closest one and take him there. You would never take him to the train station on the other side of the country! When you are doing software development, you are taking your customers to a train station. You need to find the closest one and go there. If you take them to the station on the other side of the country, you will waste a lot of work and time doing the train job. You need to take your customers to a point where they can keep going without you taking them all the way.

In this case, you are not affecting your speed, but you are traveling less distance, which in the end, achieves the same goal. You need to minimize the distance you have to take your customers. This is the equivalent of "the art of maximizing the amount of work not done" in the agile manifesto. The more things you don't do, the faster the customers get where they want.

Minimize Work in Progress

Work in progress are the tasks you are working at the same time. Any task you started working on and it's not yet used in production is considered work in progress. Having more WIP affects your velocity an quality. Another way to say the same to make it more generic is if "you have to still think or manage that story". Everybody on the team, will have to manage WIP, and waste time on it. If you do not manage it well, your quality and speed will decrease considerably. Also, it prevents people from being focused.

Once you finished working on a task, get it to the users as soon as possible, and stop thinking about it. This is in particular important, when you don't have automated regression tests and a good testing environment. Good testing policy could back up your work done 2 or 6 months later when the users get it.

WIP affects you not only by having to manage it, but also because it implies context switching. Context switching is a great source of bugs, and it reduces a lot your speed. You can easily verify this, with a simple and known exercise.

Prepare 4 pieces of paper and a stopwatch. With the first two pieces of paper write numbers in one and letters in the other one at the same time. Write a "1" in the first paper, and an "A" in the second all the way up to "10" and the letter "J". Measure the time it took you to complete it.
Then, with the other two pieces of paper, grab the first one and write numbers from 1 to 10, then the second and write letters from A to J, and again take the time it took you. You will clearly realize how context switching affects your speed and quality. As an example, it took me 29.82 secs to do the first part and only 16.23 sec the second part. It had also better quality, because I didn't have to rethink the letters every time I was writing them
You can imagine  context switching will be harder with complex development tasks. There will be more bugs because your context will not change completely and it will be harder to maintain it in your brain.

I think reducing WIP, increases your quality and velocity. I understand this it not easy, as there is always too many things to do, and they have to be done yesterday. But as I just showed you, doing all those things at the same time, will make you work more, think more and it will still take you longer. You will also do it with less quality.

Increase Quality to increase reusability and reduce rework
Lately, technology has been going really fast, allowing you to reuse almost any application you have and combine it with others to increase your flexibility and productivity. The only way to achieve this, is by building applications on top of other applications.

I could verify with many years of doing really large development projects, anything you didn't do a year ago, will take you two or three times the effort to correct it today. I will not also take you longer, but it will also have a great amount of bugs. All this because it was poorly requested, poorly developed, poorly tested, and poorly documented, so all the assumptions done a year or two ago to make it work will be lost.

Investing on doing the right thing, and doing it right in software development is the only way you will be able to build things on top of your application. If you don't do the right thing, you will spend a lot of time changing and fixing it, and probably throwing away a lot of what has already been done. This will not allow you to have the time to build something else on top of it.
If you don't do it right, you end with something complex, full of bugs, no documentation, no tests, no knowledge to allow you to build on top of it either. Both cases imply a lot of rework. Reducing rework translates in to doing less and reusing more. The only way to reduce rework is by making quality a pillar of your process.

Conclusions

Increasing your velocity is the only way to last longer on IT. Still, this doesn't mean doing it recklessly. Even an old lady can push the accelerator pedal to the bottom and go up to 150 miles, but I wouldn't like being on the car when it happens. You have to do like race drivers and practice all the time trying to do the perfect turn. Only trying to make it better you will be able to go faster. It will not happen on your first iteration, or the second. First try to get to a stable point, where you are doing the right thing right. Then practice will allow you to go faster and see better ways to do things. If you just stop doing important pieces to go faster, you only foolish yourself.

Value, what is value?

When you get to work with Agile methodologies, you start hearing a magic word all the time. The word "Value" in agile methodologies is what drives everything. What makes "Value" so magical? Lets see what is behind that word by starting with the  dictionary definition. There are many definitions there but I think this is the one that applies in this case:

"relative worth, utility, or importance" 

This is where everything gets hard with an agile methodology. The definition says it's a relative, but relative to what? Everybody will gets a difference sense of value, because everybody measures it with a different stick.

The business user measures value, in something that will allow him to work less and make the same or more money. A developer will see value on using the latest technology. A DBA will see more value on having a well defined DB schema. A Project Manager will see value following a process and become predictable. A QA guy will see value in a testable application. But, which of all those values is the one you should care about when working in software development? We got into a worst place, because everyone seems to have different objectives while they do their work.

Now, which of all the different values is the right one? Well, I think all of those at the same time, but each one with a relative worth. Now lets look at the first principle of the Agile Manifesto to get a better understanding of which value is important

"Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software."

This means the value the customer gets, it the most important one. So we need to build everything around that. Still, that is not something simple, and it takes a lot of effort to satisfy a customer. Who will put in all that effort to achieve this important goal? That's the whole team working together. But as we know, not all teams are able to overcome all the problems in the process of satisfying a customer. Something else should be added to this mix, and that is team commitment. Without the team working at full steam, it will be really hard to satisfy a customer and be profitable at the same time. There is also another principle in the agile manifesto related to this.

"Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done."

Which is the trick for that? is there any recipe to build a committed team? That's something everybody struggles with, when they are building a team. I think the word "Value" comes into play again. To have motivated people in your team, they need to get value from the project.

How can you align each team member's value with the customer's value? I think that is what management should be focused on through the whole project. The best management is the one that achieves that. This is also not an easy task. Because management is not measured by value, it's measured by numbers. They are often so trapped into those numbers, that they forget what really matters.

"Value" can be many things, acts and even just feelings. I could be just being listened or listen. It can be a mountain touching the sky or bytes stored in a hard drive. The only way to find out what it is, it to talk, dig and look for it. This is probably the biggest challenge for a manager, find out where is value for everybody in the team and align that with the value for the customer. Once you get to that point, you will see a successful project all around.




References
http://www.merriam-webster.com/netdict/value
http://agilemanifesto.org/principles.html

The definitive Corporate IT Firefighter guide

A firefighter in real life, is a person dedicated to extinguish fires caused intentionally or unintentionally. The Corporate IT firefighter does the same. Here is a guide for all the IT firefighters around the globe.

The IT Firefighter theme
"All the effort is dedicated to the most important fire. Then we run to the next fire."

Here is the process IT firefighters follow everyday. We start with the most important thing a firefighter needs to know to be the best as his job. That's knowing :

How to prioritize a Fire?

Fires can have multiple sources on IT. The best way to prioritize fires is analyzing the source. The source of the fire is more important than the size of it, that's why the source is vital to determine the fire priority. A good practice is working with an organization chart. The organization chart is like the real life firefighter cloth, it prevents you from getting burnt too fast. The standard procedure is matching the name of the person that triggers the alarm with the names in the org chart. The higher the box matching that name, the higher the priority. All the fires you are not working on are kept in a backlog, which we cleverly named it the Burning backlog. We will focus then on the fire at the top of the backlog, while all the others wait in the burning backlog to be extinguished.

Once you get to be an experienced firefighter, you will develop improved methods to prioritize backlogs. I met a couple of IT firefighters that used the title of the person reporting a fire but unfortunately they are not working on IT anymore. This is because titles lately became meaningless in companies. For example the only tester in the company now is called Vice President of Quality Assurance. So as a general rule, don't pay attention to titles.

Also you might run into the case, where two fire alarms were triggered by two people at the same level. The best way to solve this case, is to look who has more boxes below him in the org chart. It's also highly important if your name appears in one of those boxes, because that could change the more boxes rule.

With this guidelines, you will be able to successfully prioritize fires in your corporation. 

Fire Alarm response procedure

Whenever there is a new alarm, you need to prioritize it. If the priority is higher than your current fire, you need to grab all your stuff and run to fight the new one. Remember our theme, we alway fight the most important fire. Wasting time in the other fires is not well seen in the IT Firefighting Union, and could get as bad as lose your annual bonus if there is one.

Once the new fire has been prioritized, you need to send a response to the person who set the alarm to calm him during this troubling time. Your response should be "We are working on it, and we will have a solution as soon as possible". That will make you look good, even when you are not even close to work with it or know what the problem is.

Make it a learning experience
Corporations usually have two policies. One is the "no training" policy, and the other is the "no training now" policy. Either way, you will not get any training for most of your tasks, if not all. So, what do you need to do to learn from experience?
When you are working on a fire, you need to be really dedicated, and not just look at the fire, but what caused it. So you will start learning what you should never do in your applications. Things like:

• Catching exceptions and never log any error.
• Building your own internal web framework based on struts.
• Your own XML parser or ORM tool.
• Your own IoC lightweight container.

All those might have a reasonable explanation when they decided to do them, but in the end, you learn they become a big source of fires. Once you see a big source of fire, look at it, understand it, and search somewhere what would be the best way to do it, so it doesn't cause more fires. That will be the closest you will get to training unfortunately. If you are lucky you will get a chance to properly fix them.

Live with it and climb the corporate ladder
Being on Corporate IT is not an easy job. Just like a real life firefighter. Nobody wants to pay for your work or care about your well being. But as soon as they get into a fire, they will call you right away and thank you gratefully once you extinguished it.

Once you become popular in the corporation, everybody will call you and talk about you. If you always follow these simple guide and keep up with the hard work, you future is assured and you will get high in the corporate ladder.

Seriously... if you are working as a firefighter in IT, don't sign up for the union. Become a planner, and work on plans. Planning is looking ahead, firefighting is looking backwards.