Kinds of Technical Debt

I think there are three main ways technical debt accumulates. They are:

1. Debt by negligence
   This is the everyday accumulation of technical debt in a project. It's the little things that can really add up over time. Examples of this include...
   • Leaving code in giant indecipherable methods.
   • Not giving variables meaningful names.
   • Bloating an existing class.
   • Leaving in commented out code.
   • Not giving meaningful commit messages.
   • Failing to write or update unit tests.
   All of these are entirely avoidable if a programming team is taking care with what they do. Unless the code is finalised, and the system works so well that no maintenance is ever needed, these practices will hurt the project later on.


2. Debt by constraints
   The best of intentions and due diligence among the developers won't matter if there's too much being asked of them. Examples of this include...
   • The amount of work cannot be achieved in the time allotted.
   • The developers with the wrong skill-set are assigned to the project.
   • Interference in the process from management.
   • The technology stack is less than optimal for the solution.
   • The requirements repeatedly shift.
   These debts are often the most easy to recognise, but the most lamented by developers. It can be an excuse for why a system ends up a fragile kludge, but not an excuse for throwing away good practices.


3. Debt by investment
   There is the very rare occasion that taking on technical debt is the wise decision. Examples of this include...
   • Responding to an urgent deadline.
   • Getting a prototype up and running quickly.
   In these cases, the technical debt should be acknowledged and paid down as soon as possible.

Not all technical debt is bad, and some kinds of technical debt are more or less benign in the scheme of things, but it's worth recognising what factors lead to technical debt, so that the decision to address it can be a informed and deliberate one.

Sometimes we should know better, sometimes we can't do anything about it, and sometimes it's worth it.

The Virtue of Brevity

"If it is possible to cut a word out, always cut it out." - George Orwell

On my last project, there was a substantial guidelines document for how to do code review. I read that document multiple times, but I could not tell you anything that was in there.

This is not to say that the guidelines were disagreeable or poor, but that there was way too much information for the document to be useful.

I contend the guidelines would have been better if they were shorter and more concise.

For the document, and anything else, being brief serves two purposes.

1. It becomes more readable, and thus more useful.
2. It respects the time and abilities of those who read it.

When we write, it's to get something across. Successful communication is the goal, and brevity is the key to successful communication.

Dealing with Waterfall Change Management

No development team can ever just decide to 'go agile'. They can adopt agile practices into development, but the full benefits of agile are realised when that adoption impacts the whole software development lifecycle.

For us, the main sticking point is change management. To get anything approved to even go to preproduction requires seven different approvers and 2 weeks lead-in time, only for the process to be repeated to get into production.

The way change management works affects the way we make our deliverables. From the end of the sprint to production, we have a 3 week lead time, contingent on everything going right. (The product owner makes the final decision whether we are going to production.)

But it also affects how we handle production issues. One thing that cropped up last sprint was a request to change how a screen behaved. Programmatically it needed to flip a boolean on a read-only field, but because of the time and effort to get that change in, multiple avoidable data fixes will accompany the next production release.

We know that there's no getting around this, so we adjust accordingly. It makes us less effective to respond to minor issues, but it's manageable for the day-to-day.

The Heisenberg Product Owner

"We're done when I say we're done" - Heisenberg

From the development's team perspective, product owner sign off is the one stage of the development process they can't control. Ideally, this should a minimal step - all the work has been done before this point. But the product owner can always say no.

As scrum masters, our job is to try to sort this out, as this is indicative of failure somewhere along the way. The product owner is a part of the team, and thus involved in the day-to-day delivery of the project. So the question is where the failure is taking place.

There are many reasons within the team this could happen, and many ways a team could deal with it. One thing to watch out for, however, is a product owner who simply cannot make up their mind.

If this is the case, it's up to the scrum master to intervene. Simply acquiescing to the product owner is not a good idea as it breaks the flow of development. It also erases the trust and enthusiasm of the development, as their work is undermined.

If new aspects of a user story are required, they are to be turned into new user stories, prioritised, and put on the backlog. This gives a path to improvement that ensures those features are added, while not holding up the flow of development. The development team then doesn't have to worry about moving targets or feature creep.

Similarly, if user stories aren't being fleshed out enough, then there's grounds for spending more time in backlog refinement. Or, perhaps, the developers can swarm with the product owner during development.

As scrum masters, it's our job to ensure the process is running smoothly, and that includes how the product owner works with the team.

Following the Boy Scout Rule

"Always leave the campground cleaner than you found it."

In all my years as a software developer, the only times I've ever gotten to start from scratch is when I've worked on personal projects. And even then, starting from scratch is a last resort.

My experience of being a programmer is working in someone else's architecture. I get to expand it out, fix it up, and sometimes refactor it away. And each time I end up having the same question - why is it built that way? And each time there's the same answer - it's because the code needed to be delivered quickly.

It feels at times that all we can do as programmers is try not to make things worse. One thing I've done is tried to fit in with the existing way of doing things. That way the code is consistent for those who maintain it in the future. But it doesn't solve the problem, merely prevents it from getting any worse.

The answer is meant to be refactoring, though it's really not an answer for fragile code. It's a sad irony that the code in most need of refactoring is the code least able to be refactored...

Any complex system is going to seem impenetrable at first glance, but there are ways to improve it. A system doesn't need to be overhauled to be fixed, and something as simple as tidying up the code that's being worked on is enough to make things better.

For this project, we have pretty abysmal unit testing. So one rule we've set for ourselves is to put unit tests in wherever we find ourselves enhancing the code. Coverage is still woeful, but it is less woeful than it was a month ago. Another thing is to make sure no commit has commented-out code.

Code doesn't get to a messy state by a single act, nor can it be fixed in one. The complex mess that is the average codebase (at least average from my experience) is the way it is because of a long path to that state. Making things better is a similarly long path, but thankfully one that be addressed as part of enhancing a system. Fragility is quantitative, not qualitative.

The Shopping Analogy: Learning to Shift Your Thinking

I don't like to go food shopping often, so what I end up doing is buying more than I need. I'm anticipating my future needs - short term for meat and vegetables, mid-to-long term for everything else.

So when I buy, I'm rarely thinking of a specific dish, but what kinds of foods I may want in the future. This has two apparent consequences. First, I end up getting things that go bad before they're used. Second, I'm susceptible to the specials and whims of possibility.

Any food I don't end up using is a waste of money. I'm paying for the potential it provides rather than the reality of consumption. And anything that goes into a packed fridge or pantry is out of sight (i.e. out of mind) so it can easily be lost. While I always have food around - which may be of psychological comfort - I'm paying for things I don't use.

There is a way of solving this: buy ingredients on a needs basis rather than an anticipatory one.

What this requires, however, is a change of mindset. That when I shop, I buy based on an actual need rather than an anticipated one. So each time I buy less, but make more trips to the shop. This way I know what I'll buy will be consumed, and that my freezer and pantry won't be full of spontaneous anticipated purchases.

In software design, the Big Requirements Up Front approach is similar to trying to shop in anticipation of what one might need in the future. Like with shopping, what sounds like a good idea at the point of sale won't necessarily translate into what's needed now, or even useful in the future.

The agile approach is for the business to value and prioritise what is needed, not what might be nice to have at some stage. A pantry full of uneaten food would rightly be considered a waste, just as software with unused features ought to be considered a waste.

And like shopping, recognising the problem of waste isn't enough without that change of mindset. The process of 'going agile' is not about following scrum (or any other process) to better outcomes, but embracing a mindset that makes the process effective. It's the shift from "I want this someday" through "I'll better plan to use what I buy" to "I'll buy what I need when I need it."

Keeping Track of Tasks with SplenDO

A few jobs back, my monitor would be covered in post-it notes. It was my way to keep track of my tasks, or my progress in particular tasks. It's a practice I wish I kept up, and I'm disappointed I let the lack of office-supplied stationary get in the way of doing it. For my development, it was the time I felt the most productive.

Between Notepad, Sticky Notes, and actual notepads, I've never managed to find a sustainable suitable replacement. Memory only goes so far.

For the last few weeks, I've been trialling out the SplenDO on my mobile. This is what I found so far.

The useful:

• Putting rooms on meetings - Knowing when meetings are is useful enough (though mostly taken care of by Outlook), but having the place of the meeting at your fingertips is invaluable.
• Tasks I'd normally delay - Some things I know I tend to procrastinate on, such as anything involving chasing up people. Having it as a task (and one that becomes overdue) compels me to get it out of the way sooner rather than later.
• Temporal tasks - For the occasional task that has a deadline, it's very useful to have a reminder close to when it's needed.
• Tasks that come up in conversation - So much of what needs to happen at work is stated during the course of conversations. This alleviates the need to carry a notepad and pen everywhere, or trying to remember every action item after the conversation is over.

The not-so-useful:

• Habitual tasks - The daily stand-up happens at 9:45am each weekday. While I have the tasklist to remind me of this, it doesn't do much to have it as a task. I find I do it just to tick it off that it is done.
• Aspirational tasks - If the task isn't necessary, then putting it on the list becomes a source of frustration as the deadline passes.
• Setting times on non-temporal tasks - This is a case of over-planning. If a task is not time-dependent, it makes no sense to try to give a deadline to it.

The verdict:
As far as keeping organised goes, this is far superior to my old post-it note system. It's simple to use, and it's notifications keep it relevant. It's great for someone who always has a smartphone around.

Our Training Sprint

I suppose there's never a good way to start an agile project. On the project I was put on, almost everyone was new to the codebase, new to the project, and new to each other. So I called the first sprint Sprint 0 - our training sprint.

The difficulty was to ramp up with learning the existing product, learning the business rules, and learning to work with the tools and processes provided. With this sprint complete, here are some of the lessons I came away with:

• Being on the project doesn't mean buy-in - The standard way to illustrate commitment in agile is the fable of the chicken and the pig. As a development team, the ideal is to get collective commitment, where all individuals work together to get the product over the line. It was clear when individuals were merely involved rather than committed, be it some or most of the time.
• The daily scrum is important - Perhaps the daily stand-up felt too much like a status meeting, but it was really useful for the team to know what was going on with the project as a whole. When information was shared outside of this, it didn't always get to everyone.
• Intervention should be done early - We had one member who only ever turned up for some of the meetings, and missed a lot of the daily stand-ups. Intervention about what was expected as a team member came too late in the sprint to rectify the problem. And even then, it did little to change things.
• Business As Usual consumes time and energy - We had two issues with BAU. First, when production issues arose, it took time out of the sprint to deal with them. Second, there were non-sprint activities for nearly all members dealing with that took focus away from making progress.
• It's easy to revert into a management-like demeanour - The ideal of the scrum master being a servant-leader is really difficult to achieve in practice. At times, it felt like I was driving the process too much, rather than simply guiding a self-organising team.
• The day-to-day can obfuscate accumulated achievement - At times, it felt like the sprint moved really slow. Yet when I added up what we had built by the end of the sprint, we had made quite a number of improvements. And that wasn't including some of the background tasks we did, like setting up automated deployments or getting better code coverage tools.

Effective Bitbucket: 10 Tips to Better Utilisation of the Tool

One can simply use Bitbucket as a basic source repository, configured to behave like SVN. But with a few configurations and proper usage, it can be very powerful. Here are 10 things I've found for getting the most out of it.

1. Use the feature-branch branching model
   Although Bitbucket can be configured to work in master, creating branches should be the default. It allows you to separate out all the muck that happens through the development process from the clean deployable code in the master branch. Merge back into master when the code is ready, and only when it is ready.
   (The feature-branch model is good, the gitflow model is better.)
2. Prevent changes to master except by pull request
   Setting up a feature branch workflow doesn't guarantee people won't develop in master. Putting a rule restricting commits to master will. This can be achieved by going to "Settings -> Branch permissions" then adding a rule that master can only be changed by a pull request.
3. Create branches in JIRA
   Working in branches is good. Working in meaningfully named branches is better. If you are integrating Bitbucket with JIRA, then you can create branches from a ticket. From the JIRA issue, click "Create Branch", then set the right branch type and which branch to branch from.
4. Don't take ownership of branches
   The main point of source control is collaborative development. Creating branches is a good way to isolate changes from the main codebase (until they are ready), but that's no reason to isolate yourself.
5. Write meaningful comments in commit messages
   This is true for any source repository, but it's well worth repeating. Writing a good commit message may seem tedious at the time, but it's invaluable for understanding that code in the future. The code is the how, the commit message should be the what and the why.
6. Turn on mandatory code review approval
   In "Settings -> Pull requests", tick "Requires [1] approvers". Bitbucket has code review built into the web interface, so make sure they happen before the code is merged back into master. Having code reviews at this point dissolves the pre-commit / post-commit dilemma, as the gateway moves from entry into the repository to entry into the main codebase.
7. Turn on mandatory successful build
   All source repositories should be hooked up to a continuous integration tool. Bitbucket goes one better in that it can use the results of the build as a necessary step in the merge process. In "Settings -> Pull requests", tick "Requires a minimum of [1] successful builds. But it also requires some setup on the CI tool. For Jenkins, it required the Stash plugin (the Bitbucket plugin is for the cloud version). And to set the job up right, build branches with the prefix features and bugfix. If you're using Maven, set the goals as clean test as this will compile / run the unit tests, but not build the final artefact.
8. Treat the code reviews as a conversation
   Comments can be put against files or against lines. Multiple commits contribute to a pull request. The history of pull requests can be viewed at any time by changing the filter on the Pull Request screen. So treating the code reviews as a conversation between developers is a way of documenting specific changes in a way that most code review processes lack.
9. Don't hang onto old branches
   If you're finished with a branch, and the code has been successfully merged into master, then clean up the branch right then and there. Deleting the branch signals that there's no work left to be done on it. And even if a defect is found on that feature, just create a new branch in its place. This way, the repository does the job of keeping the branch up to date with all the other code changes.
10. Search is a fast way to find code
    Searching on Bitbucket is much faster than searching in an IDE, so using the Bitbucket interface can assist in being able to navigate through an unfamiliar codebase. If there are multiple projects / repositories on the same server, a global search can be performed. To search, just use the search box located on the top navbar. This, by default, will be a global search. To search within a particular repository, navigate to that repository before performing the search.

From Feature Branch to Gitflow

When we migrated from SVN, one of the first acts was to lock down the master branch. This was mainly to get developers used to the Git way of doing things, so hopefully to reap the benefits that comes with proper usage of branches.

The ideal, however, has been to get to a Gitflow model. The idea is to have easy access to the Production code - should the need arise - and that fits well with the Git branching model. This could have been retrofitted to our migrated codebase, but we didn't do it as we thought the best thing to do was get the team used to Git first.

Here's the path we took to incorporate Gitflow:

1. Create a release branch - This branch incorporated all potential fixes found in the preprod environment while preparing for production. This branch could only be changed by pull request, so the developers would branch off it if needed. A job in Jenkins was set up to build the release branch.
2. Do the release - The latest artefact built from the release branch was promoted to production.
3. Create a production branch - A new branch called production was set up in Bitbucket. On the branching model tab, this branch was set as the production branch. The current release branch was merged into it.
4. Finalise the release - At this point, we put an exception on the restrictions to the release branch: the release manager. The release manager removed the SNAPSHOT from the version number and committed, so Jenkins would build the final version*. The release version was then incremented to the new SNAPSHOT version.
5. Merge back to master - All the release changes were merged back into the master branch. As the branch had served its purpose, it was deleted.

---

*I really don't know why we do this after production rather than immediately before the production deployment. It seems odd to deploy SNAPSHOT code into production.

Embracing Shadow IT

Back in 2009, I was put on a project with a very short time frame and a lot of work to deliver. A couple of the senior developers looked at the resources and technologies at our disposal, and they realised they were inadequate for the task at hand. So we set up a completely open source solution - developed on Eclipse instead of RAD, tested on Tomcat instead of WebSphere, used SVN instead of ClearCase, and hooked into Hudson.

All this was running on a spare machine sitting on my desk. And of course, we had to keep it a secret. It all came to a head two project managers later, but it is a vindication that when I returned to the department years later, many of those tools had become the norm.

There is a term for this: Shadow IT. And it comes about because no enterprise-wide solution is going to cover everything that needs to be done. In 2009, the computers we had weren't powerful enough to effectively develop on RAD or WebSphere. ClearCase didn't give us continuous integration, and is a generally burdensome source repository compared with SVN.

(In late 2010, I switched projects in the department to one that used the official set of tools. The team had just lost 7 days to a ClearCase outage, despite it having paid support. We never lost any time running SVN.)

What surprised me about our switch to the Atlassian suite of tools is we are still in a semi-secretive state about it. It's a tool for our section, that we are meant to deny knowledge of to outsiders. The tools and their relative effectiveness are of interest to others, and better practices only come by demonstrated utility. Otherwise we're beholden to sales pitches to people who aren't directly affected by the the products being bought.

The whole point of going agile is that teams individually can reflect on what works (and what doesn't!), so they can respond for the future. Having severe constraints on their tools limits their capacity for change, thus making the exercise less effective.

For us, we are certainly benefiting from using our own tools. JIRA is far better for collaboration than the physical boards we had. Confluence is more intuitive and powerful than Sharepoint. Bitbucket has many advantages over SVN. But most of all, we've got a development team excited and engaged with the tools - a far better situation than the usual stream of complaints that have been ubiquitous in every workplace I've been in.

Walking for Health and Sanity

One thing I always make sure to do before lunch every day (it isn't raining) is get outside for a walk. I consider this the most important thing I do at the office each day.

The obvious reason to walk is that it's a form of exercise, and that has all sorts of health benefits. It certainly is better than sitting at a desk all day. One of the reasons I started doing it was seeing the occupational consequences for others, and realised I was heading down the same path.

It's not always easy to make time to exercise, so my lunch break serves as a time I can dedicate to it without needing to juggle other things around. If it becomes habitual, one never needs to make time for it.

And as far as exercise goes, walking is not that strenuous. One can walk (and even walk fast) without breaking a sweat, which makes it a convenient form of exercise for the office.

There are benefits besides the fact that it's exercise, and exercise is good for your health. Walking has been found to help with learning - that it can increase memory retention. For my personal experience, I've found it's a great way to collect my thoughts, and even solve problems. Many times I've been stuck on a problem, only to solve it 5 minutes into my lunchtime walk.

If you are walking outside, there's the added bonus of getting vitamin D. Not getting enough vitamin D is an unexpected occupational hazard of working in an office, as incidental exposure to the sun (our primary source of vitamin D) is much less than in other professions. There's even research suggesting that walking through areas with trees has benefits over walking through streets.

Finally, walking provides an excuse to get away from the office, away from coworkers, and away from the work itself. It gives me a break, and a chance to take my mind off work-related matters. And best of all, it's undisturbed.

What does a Scrum Master do?

As a developer, it's always been really easy to know what my responsibilities are (and what they aren't). I do tech-related stuff, answer tech related questions, and try to be as articulate as possible at the intersection of the technical stuff and the wider business processes.

With moving into the scrum master role, this has shifted. I'm still developing, but developing nowhere near as much as before. Rather, my standard tool has switched from RAD to JIRA. I'm even finding that RAD remains unopened until time during the day.

I also find myself doing a lot of coordination tasks. The documentation about scrum masters repeatedly use the word 'facilitator', but it seems like coordinator is more appropriate. Perhaps that's indicative of less-than-complete buy in among the team, or simply the teething issues that come with trying to start from scratch. Either way, a lot of time is trying to get the right information from the right people.

The final big thing for me is the sense of responsibility. On the last project where I was purely in a development role, I had the collective commitment attitude, but I knew a lot of what was being done by my Scrum Master was obscured from me. It's a big step up - I have to be the one to make all the things happen that I took for granted when my focus was purely development.

Utilising Continuous Integration

When I came into my current role, we had our repository hooked up to Jenkins. Every time we would commit, it would build the code and run our unit tests. We'd know almost right away if someone had broken the build.

Yet if we wanted to deploy, that was a manual process. Someone would download the EAR and manually log into the Test environment to upload it. Continuous deployment was deemed a bad idea as it would disrupt testing, but there was no good reason to keep it as a manual process. Not least that the power to deploy was limited to a couple of people.

In our new setup, we have three deployment jobs:

1. Branches - builds a maven deploy up to the unit tests, and informs Bitbucket of the results.
2. Master - builds the master branch and stores the artifact on Archiva.
3. Deploy - takes the last successful build and deploys it to the Test server.

The first two are automatic and will run whenever there is a code change. The Branches job is integrated into our code review process, as Bitbucket is configured to only allow merging upon success. The Master job ensures we always have a potentially deployable artefact.

The Deploy job is manual, and controlled by the testing team, who run it whenever they are ready for more features to be tested. This gives the advantage that testers decide when they are ready for new work.

This doesn't solve all deployment problems. Any intervention with databases is manual. The necessity to deploy shared EARs is invisible to the testers and requires intervention on our part. And deploying to hither environments involves the cumbersome change management process. However, it's important to recognise all the wins the current setup gives us over what was there before.

Building a Minimum Viable Product

When developing on a Waterfall project, the criteria by which you develop to are determined by the specifications. This is both a blessing and a curse, in that the decisions about what to do are external to the development team.

The general issue with design by specification is each part is of equal value, in the sense that it's presented as an all-or-nothing proposition. And since everything is designed upfront, it has to cater for all conceived features - no matter the actual value.

With agile, features are organised and developed by value. The idea is to create a minimum viable product, where the features developed are those that will be the those that add maximum business value.

There are two really good reasons for doing this:

1. If development needs to stop at any time, then what's delivered will reflect the greatest value.
2. Most software features turn out to be either never or rarely used.

To me, both issues come down to the problem of trying to sort out the needs of a system upfront. Design, especially on detail, is constrained by the development costs - something difficult to work out in advance. And the value of features is something that can only really be seen once the system is live, so anticipating that is going to involve guesswork.

Keeping in mind the possibility that development could wind up tomorrow puts pressure on the actions of today. Each day a product is not delivered is a day the value of the product is not being realised. By focusing on MVP, the outcome is delivering sooner, delivering something that addresses the needs, and thus represents the best value of the time and effort of everyone involved.

Busier Than Ever

One thing I'll lament to anyone who is in earshot is how little I get done. What I mean by that is that the Java tasks I've pulled into the development queue.

On the other hand, I now need to remind myself to get water. I haven't even made tea since I started in this role. And it's a deliberate and rare act to go get a coffee with a coworker from the café just outside.

So the rules of inference suggest that I must be doing something. And the most obvious answer is dealing with people.

However autistic I feel at times, dealing with people is natural in a way that development has never been. The years of developed intuitions and heuristics I have about coding don't surmount the weirdness of the task cognitively. We're evolved social interactors, not programmers.

The point I'm getting at is that it doesn't feel like work to communicate. Having a coding task interrupted by a conversation never requires much effort to switch context, but it's very difficult to break free from a conversation to do coding.

The feeling of being busy is not being able to set aside time for the tasks I want to be doing, as what I need to be doing is something that doesn't feel like work. It is work, and it is necessary work, and it sure is time-consuming work.

The Cycle of Process

1. The solution to alleviate human error or overcome personal limitations is the addition of process to avoid / minimise it.
2. The longer the timespan, the greater the accumulation of processes.
3. The more processes that are in place, the less intrusive adding new process seems.
4. Working within such an environment entrenches it as the status quo.
5. Eventually the processes become so taxing that any changes in personnel or circumstances pose a problem in itself.
6. Outside perspectives offer an entirely new way of doing things that will eliminate all the wasted effort gone to maintaining the process rather than the outcomes the process was there to manage.
7. The new way of doing things turns out not to eliminate all human error or overcome personal limitations.

Going Agile: The Elevator Pitch

What we are trying to achieve is to change the way IT interfaces with business. Instead of business trying to work out ahead of time all the details they want, the business collaborates with IT to work out what the requirements are.

This is further broken down into time, with development continuously delivering functionality. The business doesn't need to have every detail set out ahead of time, but just enough of an understanding of the features that development can take place.

The advantage of this is that development becomes a feedback loop, where features can be gradually introduced, changed, enhanced and even removed based on business requirements. In short, it allows business to concentrate on what is needed, and for IT to respond to these needs as they arise.

How to Git: Branch, Code, Merge, Delete, Repeat

Switching technologies under people is a great way to learn about implicit understandings people have about how to use the technology. With the branching model in Git, it brought out just how people see their own development as a solo enterprise.

The common reaction was to think that each person has their own branch to work with, rather than a branch serving as a unit of work. This highlights one barrier to better collaboration, but also can limit the effectiveness that branching in Git provides.

The best way I've found to deal with branching in Git is as follows:

1. Branch - When starting a new feature, branch from the head of the development branch. This ensures you are working with the latest deployable code.
2. Code - Your regular development work. Commit as needed as it backs up your work, as well as allowing for collaboration.
3. Merge - When your coding is done, merge it back into the development branch.
4. Delete - After you merge the branch back to the development branch, delete it. Leaving it just creates clutter, and clutter that goes quickly out of date at that.
(if needed) Repeat - If it turns out steps 1-4 didn't cover the feature, start the process again. Don't worry about maintaining a branch for the possibility that feature still needs to be worked on.

The ideal is that branches are short-lived. The longer a branch lives, the more likely there are to be merge conflicts when it's brought back.

With JIRA and Bitbucket, it's easy to create a branch named for a particular feature. So anyone seeing a branch will at a glance know what unit of work that branch is for. And since this can be done multiple times, having to recreate a branch when more aspects of a user story emerge is simple.

The Beauty and Necessity of The Agile Manifesto

Of all the documentation I've read on agile, the one thing I keep referring to in conversation is the Agile Manifesto. For reference, here it is:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Whenever I'm pulled into meetings to discuss the agile process, I think of this manifesto. Whenever someone asks me about how we are collaborating with business, I think of this manifesto. Whenever the very question of agile comes up, I think of this manifesto.

I even have the manifesto printed out and stuck to my desk!

The beauty of the manifesto is how much it says in so few words. The terms are broad enough such that they don't get bogged down in any particular aspect of the SDLC. Yet the terms are also targeted enough that they hit the relevant factors associated with software development.

Furthermore, they are a statement of intent - of what a software developer ought to strive for. The goal is working software, and to get that requires interaction, collaboration, and responding to change.

Those things on the right too often get in the way of creating working software. We've all been in projects where managers put pressure on the team because it deviates off their unrealistic "plan". That bad ideas get coded in because the phase to negotiate that was before coding began. That a effort is put into elaborate documentation that goes out of date the moment a system enhances it.

Even when going agile, the agile manifesto works to dampen the tendency to think any problem can be solved with more process. It can stop those in-grained waterfall tendencies from answering the inevitable "now what?" question.

The Switch to Git

Apart from 9 months using Rational ClearCase (and a couple of dark years using SAP WCEM), I've used SVN as a source repository. I even use it for my personal development projects. So I feel pretty comfortable in the SVN paradigm.

The strength of SVN is its simplicity. It's fairly easy for any developer to pick up and master its basic functions. That said, it has some pretty major shortcomings in a collaborative environment:

• Everything goes to trunk - All the checks that go into ensuring quality code happen after the code is shared. A bad commit can break the build and hold up deployments.
• Branching is difficult - While it's possible to branch in SVN, it's a non-trivial process that's largely avoided in practice.
• It encourages sitting on code - The flip-side of a central repository and the difficulty of branching is that developers tend to sit on uncommitted code.
• Collaborating in an unfinished state is tricky - Committing unfinished code is risky because it breaks the source of truth. It encourages workarounds like emailing code or zipping up entire repositories.
• Code reviews are after the fact - .

None of those are showstoppers, of course, but something definitely feels amiss when you have to use alternatives to a collaborate tool to do what the collaborative tool is for.

So as part of our switch to a more agile framework, we took on migrating from SVN to Bitbucket. Most of our team hadn't used Git before (myself included!), so one issue with doing this would be the reskilling of developers in a new way of collaborating. So we needed a good reason to disrupt what people are familiar with. And with Bitbucket, these are the advantages I can see:

• Branching is easy - Creating branches in Git is a trivial process. Switching between branches is a trivial process. Working with branches can avoid prematurely merging code before it's ready.
• Integration with JIRA - JIRA Agile tickets can be used to create branches, as well as linking to commits against a number. This is good for chasing history.
• Commits don't need to worry about breaking the build - When working in a branch, checking in code that breaks the build doesn't affect the "source of truth". So developers don't have to keep code on their desk for fear of "polluting" the master code.
• Collaboration is made easier - Since multiple people can use the same branch, there's no need to resort to emailing code between developers.
• Master can be protected - Bitbucket can be set up so the only way into master is through a "pull request", and that can be protected by setting rules about code reviews, as well as ensuring the code will successfully compile.
• Code reviews are post commit, but pre-merge - Bitbucket puts code reviews as part of a pull request, so with the right rules only code that passes review will be put as part of the build.

This is not to say that all of it is easy. Git has a higher learning curve than SVN, and working with branches can be confusing. It's also hard to break out of the silos of development mindset that SVN encourages, thus robbing Git of the collaborative advantage it has over SVN. And no code tool is going to stop poor code from being committed.

That said, Git fits much better into the feature-driven development of scrum than SVN. It encourages keeping features discrete by their creation as branches, and keeps features from being prematurely deployed by keeping a master branch as the endpoint of development.

From Java Dev to Scrum Master

My previous workplace did classic Waterfall, which meant coming on and going to spec review meetings while waiting for the final green light to start coding. And I came on just before development was due to ramp up - in this case, about 6 weeks.

That was 6 weeks of reading elaborately crafted specs set out in a systematic way, and occasionally going to long tedious meetings where minor details were argued over. All this for what turned out to be a few weeks of actual development, before the long wait for testing to be complete and into production.

By the time I left, we were a few months into the design phase for the next project, with coding due to start just as soon as the specs were signed off.

The place I entered had just started taking on an Agile scrum methodology in software development. In the space of a month, I had gone to arguing over the minutiae of 100 page specifications to being asked to size stories based off a two line description and 5 minutes of discussion among the team. It was quite the culture shock!

It took a few sprints, but we slowly established a good team working ethic. When I came on, there was pessimism about whether the agile process could deliver anything other than more process for us to follow. But over the next 6 sprints, we made it work and got a high quality product out on time and with a good working relationship between business and IT.

So that brings me to where I am now. I've come out the other side of a scrum agile project as a scrum master for another. That means taking on a lot more of the organisational side of things, and consequently having a lot less time doing actual development. And it's meant spending time working out how to integrate the tools we have seamlessly into our workflow.

---

The way I see this blog is this - it's a way to collect my thoughts on the agile process as a whole. To write out some of the things I'm learning as I'm learning them, and to reflect on the practical realities of agile software development as I see it. I've been happy to wear the developer hat for close to a decade now, so will be an interesting journey for me to be immersed in a different aspect of the process.