Staying Agile in a waterfall [closed]

Question

We have a large enterprise application where projects are scoped designed and finally coded using a formalized waterfall process. I am often given code changes for non related initiatives just because they are in the same section of code. All initiatives must be turned over at the same time. Development team also has little say over scope or delivery time line. We are not able to talk to the users we must go through a group of requirement gathers who don't know the business.

Does any one have any advice on how to implement even the smallest of agile techniques to such a entrenched environment.

Dellora

SO's very own S. Lott has written about something similar in his article "The Waterfall's Not Working -- Even A Client Said So" (homepage.mac.com/s_lott/iblog/architecture/C551260341/…). He's talking about it from a consulting perspective, but some of the ideas should be transferable to an in-house team fighting the same battles. – Rebec

Answer 1

At least you could start writing unit tests, or even - as much as circumstances allow - Test Driven Development yourself (possibly spreading the ideas among your fellow co-developers too). You can change a lot without management even noticing anything ;-)

Of course, in an average or better place, people in the management are not completely stupid. Over time, when you have managed to raise awareness about these issues among the development team, you (as the team, collectively) can talk to upper management too, and convince them to take steps toward the right direction. Start small, get concrete results, and build on them - and build leverage by finding allies both in the dev team and (as much as possible) among management and users.

Very often certain processes are followed only because "we always used to do it like this". If you can show people that there are better ways - and prove it with convincing arguments - you have a good chance of succeeding. Note that management and users need quite different arguments than developers. You can try making rough cost-benefit calculations (or google - I am pretty sure there are lots of stuff on the net about these). I also remember there is good material about this in Kent Beck's first XP book. You can also collect bug statistics which over time (hopefully) show that features developed the agile way have noticeably less defects in later (integration test or production) phases. (For this you need a defect tracking system, if you don't already have one.)

Another useful tool is asking questions. If you something - a document, a way of doing things - you don't understand, dare to ask questions:

• Why are we doing this like we do?
• Is there a better way?
• Do we actually need to this thing at all?
• Who needs this document?
• What information does she actually need out of it?
• Does it contain the right info for her?
• Is it up to date?
• Who updates it?

Often people just take things as "given", but when you start asking for causes, you may find lots of interesting things... and ideas for improvement.

A very useful agile tool is retrospectives. After each iteration (whatever it is called in the actual process) the team gets together and brainstorms about

• what went wrong in this iteration, and how to make sure it doesn't happen again (or at least improve it to some extent)
• what went nicely and how to ensure it will continue like that

This may be fairly easy to get accepted by management, and is a good way to start positive changes. The most important thing is to wake up and activate people - to make everyone realize that the success or failure of the project is (at least to some extent) in their own hands, that they can do something to improve the situation.

Bailiff

Concrete results are key. Find a way to get numbers that show the new techniques work. Managers eat that stuff up. – Unhappy

You make the implicit assumption that Agile/XP is the "right way". I don't recall that being proven at all. All these management styles have good points, "Agile" and "XP" are just trendy buzzwords IMO and will be replaced by something else. – Metaplasm

Unit tests are part of standard software these days and can fit into any methodology. Waterfall can include unit tests without any definition-stretching. – Metaplasm

@John I don't care about buzzwords, only about concrete results. Many of the concrete agile practices have been proven to work well before the term "agile" was invented (yet the majority of SW dev projects still don't use them :-( ). – Celestinacelestine

@John IMO it depends first and foremost on what management means by "Waterfall" - that's not at all obvious. The point is not to prove on theoretical level that the process can include this or that practice, rather to prove to management (and users) that it is practically worth taking that practice into use. – Celestinacelestine

@John unit testing is done by inspection and trust. We have no management buy into creating repeatable reusable tests, so why yes we do unit testing it is a one time throw away effort each time. – Dellora

@Peter my goal is to at least broach the way we develop software. Every time we have a problem we look for a way to add a document for CYA but not to add value. – Dellora

@Dellora I know what you mean :-) :-((( See my update, hope it helps – Celestinacelestine

Answer 2

In my experience, large enterprises are concerned with RISK, PREDICTABILITY, and MEASURABLE RESULTS. You'll have an easier (though maybe not easy) time introducing Agile if show how it aligns with those metrics better than existing practices.

1. Make it possible to ship often, even if you don't do it yet: Leverage CI tools and automated build scripts to build and package your application. This way you're primed to capitalize on any opportunity to incrementally release new code that might come up.

2. Measure your productivity now so that you have a baseline: The more you can measure the better.

   1. Average # of programmer hours per "feature".
   2. Average length of time between code checkin and the discovery of defects against that code.
   3. Average length of time between defect discovery and defect resolution in production.
   4. Average amount of time needed to identify, resolve, and deploy defect fixes.
   5. etc.

3. Project the changes in these metrics under an Agile process: For instance, in most cases the sooner we find a bug the easier/cheaper it is to fix, so benefits from TDD and rapid releases to QA should be easy to quantify.

4. Start small: You may have a waterfall schedule handed to you, but you can still break that down into iterations, so do so. Get your engineering practices in place, then start trying to adjust process. See if you can try Agile on a small auxiliary project as a proof of concept.

5. Find a sponsor: Try and convince someone higher in the pecking order than yourself of the merits of Agile. Engage their help in framing the "Agile vs. Waterfall" arguments in terms familiar to the decision makers.

6. Be patient... it may take time to see results.

7. ... or don't. If you're deeply interested in Agile and are getting zero support, find a new job. Yes it's rewarding to effect change from the belly of the beast, but it's also rewarding to work with people that share your ideas about building software.

Alesiaalessandra

Answer 3

Agile is all about breaking down those waterfall walls. BDUF; simultaneous, multi-component release; lack of communication between the developers and the business process owners; planned iterations - these all stand in your way in the waterfall process.

At my position, we have broken down a lot of these walls - and it started with getting direct access to the customer. When that happened, the customer got a better product. That led to happier customers. That drove away things like BDUF, etc.

We still don't quite have true iterations/sprints, continuous integration, etc., but we're getting there. (Old habits, and all that.)

Rudderpost

Answer 4

You as the development team could still coordinate internally using agile methods (test driven development, pair programming, story cards, CI, common language, etc.)

Agility in my mind is about being able to have confidence in changes to the software and preventing large misinvestments in features which nobody needs three steps down the waterfall route. Testing and refactoring and avoiding over-engineering are key here.

Raceway

Answer 5

To me the problem doesn't seem to be that you're using Waterfall rather than Agile. It's that your waterfall implementation has big problems. Most obviously:

Requirement gathers who don't know the business

Waterfall can and does work well if done properly. I think it sounds like some of the people involved and how they do things are wrong, not the conceptual process.

Metaplasm

Agreed, we suffer from many anti patterns but, a lot of the problem is a function follows form issue. We can't seem to implement any good changes due to the structural impediments. The goal is to see if we can implement some agile methodologies show their value and then slowly gain more control over the entire life cycle of project which is the main issue. – Dellora

Answer 6

Depending on your domain, automated testing and continuous integration should be doable.

Also, consider coming up with your own, highly granular burndown list (inchstones) for your currently assigned tasks. It should help your work estimates be more predictable and make it easier for you to explain any schedule slippage and unplanned tasks.

In general, keep track of some metrics in your system. If you can show some Agile technique adds value (shortened cycle time, lowered defect rate, etc.), it should be easy to sell your leadership on that technique.

...depending on the manager, you may want to avoid actually using the word "Agile" though, especially if you're just looking for a small victory on using one technique.

Unhappy

Something my team did was compile our own bottoms-up list of defects, bugs, missing features, and such. We called it our "Technical Debt" list and started treating it like a burndown chart of sorts. – Unhappy