Having One Big Target

I've been in sprints where there are lots of little things to be done, as well as sprints where there's just one thing that has to happen. My experience is having one big target is much better than lots of little targets.

The reason, I think, is that one big target lacks the context switching that going from target to target. It allows all people to focus on one problem and gradually work to that.

A big target is not an ill-defined target. The same rules apply as apply in all scrum - it should be coherent enough that it can be worked on. Anything less and a big target has a chance of failure.

A big target should still be broken down to small targets. But the package of targets together provides a context to the work. A package of disparate targets has their own context, with the developers needing to adopt that mindset to begin work. Smaller tasks for a greater whole do not have that problem.

Identifying and Eliminating Bottlenecks

"Don't be irreplaceable, if you can't be replaced, you can't be promoted." (from Dilbert's Laws of Work)

The easiest way to identify a bottleneck is to see what happens when time is a factor. If someone (or something) is critical to an operation, then things don't happen. That is obviously a less-than-ideal way to find where a bottleneck is.

Bottlenecks usually present themselves long before a crisis, but aren't seen as critical at the time. They are usually things that only occasionally matter, so general availability is usually enough.

Unusual situations aren't unavoidable situations. Taking steps to eliminate the bottleneck is sensible risk management. It is much better than the alternatives, such as cancelling leave or letting deadlines slip.

For those bottlenecks that are part of everyday development, they should be mitigated and eliminated as quickly as possible. In our team, only a few people had the ability to do deploys. This was liveable until those people went on leave. So we implemented automated deploys from Jenkins so that would never be a problem again.

Book Recommendation: The Nature of Software Development

Development has more than enough to keep anyone occupied on immediate concerns. But it's worth standing back in order to reflect on what it is we are doing.

The Nature of Software Development by Ron Jeffries can best be seen as an expansion on the agile manifesto. Like the manifesto, it's high level but gets right to the point. Like the manifesto, it's concise but profound. And like the manifesto, the aim is to identify what makes for successlooks like.

What we are doing when building software is trying to add value. The word can mean different things in different contexts, but in this case it is a strength rather than a weakness.

We build software because the software will serve a purpose. We should focus on delivering that which serves that purpose, and not waste time on features that are superfluous to that.

We should develop such that there will be value if the project winds up tomorrow. The code should always be ready to deploy. We should fix problems as we go along because that makes development quicker in the long run.

There's much more to the book than I'll cover here. It's a short book, and conversational in tone, and that fits with the message the cook conveys. Deliver what adds value, and don't waste time with filler that doesn't. This book is very valuable.

The Shopping Analogy: Learning to Shift Your Thinking

I don't like to go food shopping often, so what I end up doing is buying more than I need. I'm anticipating my future needs - short term for meat and vegetables, mid-to-long term for everything else.

So when I buy, I'm rarely thinking of a specific dish, but what kinds of foods I may want in the future. This has two apparent consequences. First, I end up getting things that go bad before they're used. Second, I'm susceptible to the specials and whims of possibility.

Any food I don't end up using is a waste of money. I'm paying for the potential it provides rather than the reality of consumption. And anything that goes into a packed fridge or pantry is out of sight (i.e. out of mind) so it can easily be lost. While I always have food around - which may be of psychological comfort - I'm paying for things I don't use.

There is a way of solving this: buy ingredients on a needs basis rather than an anticipatory one.

What this requires, however, is a change of mindset. That when I shop, I buy based on an actual need rather than an anticipated one. So each time I buy less, but make more trips to the shop. This way I know what I'll buy will be consumed, and that my freezer and pantry won't be full of spontaneous anticipated purchases.

In software design, the Big Requirements Up Front approach is similar to trying to shop in anticipation of what one might need in the future. Like with shopping, what sounds like a good idea at the point of sale won't necessarily translate into what's needed now, or even useful in the future.

The agile approach is for the business to value and prioritise what is needed, not what might be nice to have at some stage. A pantry full of uneaten food would rightly be considered a waste, just as software with unused features ought to be considered a waste.

And like shopping, recognising the problem of waste isn't enough without that change of mindset. The process of 'going agile' is not about following scrum (or any other process) to better outcomes, but embracing a mindset that makes the process effective. It's the shift from "I want this someday" through "I'll better plan to use what I buy" to "I'll buy what I need when I need it."

Utilising Continuous Integration

When I came into my current role, we had our repository hooked up to Jenkins. Every time we would commit, it would build the code and run our unit tests. We'd know almost right away if someone had broken the build.

Yet if we wanted to deploy, that was a manual process. Someone would download the EAR and manually log into the Test environment to upload it. Continuous deployment was deemed a bad idea as it would disrupt testing, but there was no good reason to keep it as a manual process. Not least that the power to deploy was limited to a couple of people.

In our new setup, we have three deployment jobs:

1. Branches - builds a maven deploy up to the unit tests, and informs Bitbucket of the results.
2. Master - builds the master branch and stores the artifact on Archiva.
3. Deploy - takes the last successful build and deploys it to the Test server.

The first two are automatic and will run whenever there is a code change. The Branches job is integrated into our code review process, as Bitbucket is configured to only allow merging upon success. The Master job ensures we always have a potentially deployable artefact.

The Deploy job is manual, and controlled by the testing team, who run it whenever they are ready for more features to be tested. This gives the advantage that testers decide when they are ready for new work.

This doesn't solve all deployment problems. Any intervention with databases is manual. The necessity to deploy shared EARs is invisible to the testers and requires intervention on our part. And deploying to hither environments involves the cumbersome change management process. However, it's important to recognise all the wins the current setup gives us over what was there before.

Building a Minimum Viable Product

When developing on a Waterfall project, the criteria by which you develop to are determined by the specifications. This is both a blessing and a curse, in that the decisions about what to do are external to the development team.

The general issue with design by specification is each part is of equal value, in the sense that it's presented as an all-or-nothing proposition. And since everything is designed upfront, it has to cater for all conceived features - no matter the actual value.

With agile, features are organised and developed by value. The idea is to create a minimum viable product, where the features developed are those that will be the those that add maximum business value.

There are two really good reasons for doing this:

1. If development needs to stop at any time, then what's delivered will reflect the greatest value.
2. Most software features turn out to be either never or rarely used.

To me, both issues come down to the problem of trying to sort out the needs of a system upfront. Design, especially on detail, is constrained by the development costs - something difficult to work out in advance. And the value of features is something that can only really be seen once the system is live, so anticipating that is going to involve guesswork.

Keeping in mind the possibility that development could wind up tomorrow puts pressure on the actions of today. Each day a product is not delivered is a day the value of the product is not being realised. By focusing on MVP, the outcome is delivering sooner, delivering something that addresses the needs, and thus represents the best value of the time and effort of everyone involved.

The Cycle of Process

1. The solution to alleviate human error or overcome personal limitations is the addition of process to avoid / minimise it.
2. The longer the timespan, the greater the accumulation of processes.
3. The more processes that are in place, the less intrusive adding new process seems.
4. Working within such an environment entrenches it as the status quo.
5. Eventually the processes become so taxing that any changes in personnel or circumstances pose a problem in itself.
6. Outside perspectives offer an entirely new way of doing things that will eliminate all the wasted effort gone to maintaining the process rather than the outcomes the process was there to manage.
7. The new way of doing things turns out not to eliminate all human error or overcome personal limitations.

The Beauty and Necessity of The Agile Manifesto

Of all the documentation I've read on agile, the one thing I keep referring to in conversation is the Agile Manifesto. For reference, here it is:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Whenever I'm pulled into meetings to discuss the agile process, I think of this manifesto. Whenever someone asks me about how we are collaborating with business, I think of this manifesto. Whenever the very question of agile comes up, I think of this manifesto.

I even have the manifesto printed out and stuck to my desk!

The beauty of the manifesto is how much it says in so few words. The terms are broad enough such that they don't get bogged down in any particular aspect of the SDLC. Yet the terms are also targeted enough that they hit the relevant factors associated with software development.

Furthermore, they are a statement of intent - of what a software developer ought to strive for. The goal is working software, and to get that requires interaction, collaboration, and responding to change.

Those things on the right too often get in the way of creating working software. We've all been in projects where managers put pressure on the team because it deviates off their unrealistic "plan". That bad ideas get coded in because the phase to negotiate that was before coding began. That a effort is put into elaborate documentation that goes out of date the moment a system enhances it.

Even when going agile, the agile manifesto works to dampen the tendency to think any problem can be solved with more process. It can stop those in-grained waterfall tendencies from answering the inevitable "now what?" question.