When I was last employed the Agile movement was just getting started and I didn’t hear about it until probably the mid 2000s. I haven’t really experienced Agile development in a formal sense. But it was happening all the while without us realizing that it needed a name. Since then I’ve been hearing about it, have read articles about it. I have numerous friends and former colleagues who all tell me that Agile (or agile with a little “a”) is pervasive in the software industry. So I thought it’d be interesting to learn a little more about it, as I will probably be returning to work in the industry.
So first off, what came before Agile? The waterfall development model. Basically this is the old methodology where product releases are these big, monolithic entites that take a long time, long development cycles to produce. Heavy on requirements, design, specs. As I understand it this methodology came from the engineering/manufacturing industries. Apparently it was logical for the software industry to adopt. And this is the model I worked with for most of my career.
I gather that the waterfall metaphor comes from the idea that once you get to a certain place on the river, you’re going over the falls no matter what! In other words, there’s no going back. So that, once a customer signs off on a contract, and agrees to specifications and requirements, then there’s no going back. It has to be this way because it was so costly to development these big software releases and they had to be so thoroughly documented, specified etc. that it was too difficult to go back and make changes. Or else the cost of doing so would be so prohibitively expensive there was strong incentive not to do so. And, correspondingly strong incentive to get the specs and requirements right up front. But this is difficult and problematic. Customers don’t know how to articulate what they want with any degree of detail. If they did they’d probably be developing software themselves.
So as the story goes, after a few decades of using the waterfall method things started to change. In the 90s we had what we called “Internet time” – where everything happened like 10 times faster than it used to, in large part due to the Internet bubble. Investors were pouring money into your company and demanded returns. Quick. So you had to get product out and get your stock price up in a hurry before someone else beat you to it and drove your price down. It was a helluva game. So anyway, we were speeding things up. Product cycles were getting shorter, sales cycles, everything. So, we had to develop releases faster one way or another. Eventually a group of guys got together at a ski resort one weekend and had a pow wow. What came out of it is called the Agile Manifesto. It basically formalizes this new methodology in what must be considered a big departure from the old waterfall model.
From what I gather anyone entering into the industry today most likely will be living and breathing one flavor or another of agile development to varying degrees. The movement has grown so that there is a certain amount of backlash against it. Which is probably to be expected.
Recently I sat down with a good friend and former colleague of mine, Mike Pav, who currently runs development for Spanning.com. I asked if he could elaborate a little for me on Agile development. He’s definitely an advocate of agile, but with a little “a”. He says it that way just because he sorta does his own version of agile. He’s been managing development groups for 10+ years with about 15 years of development experience before that.
Mike says it’s about being lighter, smaller, having much more communication between team members, between team and customer. Agile presses on efficiency. Presses hard. It puts a magnifying glass on everything and everyone. Finds weaknesses. He emphasized this point with the magnifying lens. If people or processes are weak, and they might have flown in the old days, they don’t fly in agile.
XP is a part of Agile. Pair programming is a part of it too but they don’t do it all the time. Here’s what Mike said about pair programming;
When starting on a project a developer will grab a peer and talk over overall strategy, design plan etc. Then the one guy goes off and does the coding but keeps open communication with the pair partner. Then at certain points along the way and at the end the partner is part of the process but they don’t sit together and code. They’ll get together go over issues, problems, solutions. Basically the pair are both owners of the component but only one is directly doing the coding.
I remember at Vignette, a version of pair programming we would do late in the release cycle. As the date draws closer to the end of the cycle you enter into different phases of the cycle where you’re doing less and less coding. For example one earlier phase was called “feature freeze” where you are finished adding features and now you’re only doing QA and fixing. Then once you know it’s solid (after sufficient QA) you enter a “code freeze” phase where you’re no longer changing code at all, except for only very critical fixes. You’re still might find some bugs but unless they’re critical they’ll have to wait until the next release. Otherwise you’d never get a release out.
With regards to the larger company and Agile the whole company has to buy into it. Mike said previously when they’d try it (agile) on their (engineering) own they’d run into trouble when the rest of the company expected them to be doing things the old way. So, they learned, adapted. Now everyone knows what’s going on and it works.
Mike described how it was difficult to shift form a layered based approach of development to a vertical one. Normally you develop bottom up in some sense. You build your foundational elements first, get them working at least enough so that you can begin to build the next layer up, and so on. This is the old way.
Now what you have to do is think vertically. Instead of building out that bottom layer broadly you have to think in thin vertical elements. Implement one narrow piece of functionality of that bottom layer that works top to bottom. Then build up on that. Move up to the next layer and so on. What you end up with are these thin (maybe narrow is a better description) components that work from top to bottom but are light in functionality. You produce a single vertical deliverable top to bottom. Then you broaden that “column of functionality” in later releases.
Mike says the key is keeping the customer in the loop. 1 to 4 week deliverable cycles, or sprints as Agile calls ’em. Embedded QA. He has no QA people, developers do it. Production quality small morsels delivered frequently, regularly.
Much more interaction with customer or end user who ever is buying. All along the way. Rather than waiting 6-9 months for monolythic deliverable that might be the wrong thing. With agile it comes out in smaller pieces, everyone can look at it and touch and feel it and decide if it was right nor not. If not then little cost to go back 1 week and redo. End result is much better quality and much more on target with what customer wants cause they’re more involved and deciding along the way.
Mike says he brought in consultants to train his group. He said it was well worth it because it was hard to “switch over”.
Another friend and former colleague, yet another “Mike”, Mike Strong (Open Text) had this to say when I asked him for his top N Agile bullets:
OK, Well, here are some thoughts, my brother …
Cross disciplined teams working concurrently, rather than separate teams stacked end to end, doing their respective roles in phases. Qualification to this is that, often, in large organizations, there may still be a small QA phase at the end, but it has little left to do since you had embedded QA working with dev as they went along. Explanations on how that works available upon request. :)
Time boxed development partitioned into 2 to 4 week sprints. These deliver production quality morsels. This stands in contrast to waiting 6 to 9 months for a monolithic deliverable.
Delivering production-quality deliverables in small chunks permits dynamic realignment of priorities while the project is in flight. Backlog management can be “resorted” as you rack-n-stack priorities continuously per the continuously changing realities in the market, or in the continually changing perception of needs / demands (i.e., early feedback from stake holders, gaining more info than you had earlier, etc.).
Team has a lot more ownership of results, and has a much more direct relationship with the stake holders or at least with a dependable proxy to the real stake holders.
Customer satisfaction is much higher. Team morale is much higher. Productivity has been measured between 35% to 50% higher in teams that get it and execute it according to the vision VS the old water fall or RUP approaches.
Mike
PS – Lot’s of info to back up my points (Gartner, Forrester, Rally, et al).
That’s about the extent of my mini investigation on Agile development. My take on it is that it’s a pretty natural progression for developers. I mean if you think about it, it’s a lot easier to crank out a little bit of code, let the customer see it and see if they like it, let ’em touch, feel and squeeze it and get their reactions. Then move forward with baby steps.
Here are a pair of articles on Agile development. The first is a short and sweet overview and introduction of Agile as compared with the old Waterfall methodolgy. And the second is an interesting article honing the point about automation in the development process.
5 Reasons while Agile won’t scale without Automation
I remember taking Software Engineering over 20 years ago. I don’t remember my instructors name but I remember her saying “automate, automate, automate”. Referring to all the engineering processes. Source code control, build, issue tracking. Everything. “If it’s not automated it won’t get done.” And I’ve always lived and breathed that mantra. It only makes sense.
I enjoyed reading the Joel Test recently. I’ve always been proud of the work we did at Vignette and the team I was a part of. It was very easy for me to give us a perfect 12 score on the Joel Test. We did all of that and more. Automated nightly builds (make, home grown borne shell), one step build, source code control (cvs, perforce), specs, bug tracking (gnats), schedule, quiet working conditions – absolutely. We actually had that article from Peopleware and I have it here. One of our founders made a point of posting it and keeping the environment quiet. So much so that the paging system was not allowed to be used except in rare circumstances, in engineering. Everything was automated that could be.
So at least I have an idea of what Agile will be like, if and when the day comes. It seems natural to me.