Sustainable Software Development

Software is hard to make, capturing the domain and getting it right with as little bugs as possible. Then comes release time and your users starts using it, things get even harder - now comes all the changes for all the things that you as a developer did wrong, all the bugs, things you didn’t think of, misunderstandings. All that and the users have requirement changes or additional features they’d love to have on top. I’ve often heard people say things like “… if it wasn’t for the users, creating software would be quite joyful …”. It says tons, it does say amongst other things that creating software is not an easy task. Also, it does say a bunch of other things as well, for instance - it says that we’re not very good at talking to our users, or we’ve taken the concept of having your team working in black-box environment too far or too literally.

Even if we did it all right there are tons of external events that could cause the requirement of changing what we already made. The market could change, new competitors arriving to the table or existing competitors taking a completely different approach to the same problem you just solved. Another aspect that can cause changes is that the people we develop the software for that we’ve included in the dialog are learning all the way through the process of what is possible and what is not, causing them to have more knowledge and wanting change.

The purpose of this post is to shed some light on what we think are good practices on their own but put together represents a very good big picture of how you can create software that will easier to change, meeting the requirements of your users better and possibly reduce the number of bugs in your software.

Talk Talk Talk

One of the biggest mistakes I think we do is to think we are able to intuitively understand our users, what they need and want and also how it should take form. Going ahead and just doing things our way does not come from an inherent arrogance from us developers, but rather something I think is closer to an what we consider an intelligent and qualified assumption of what we think we’re making. At the core of this problem sits the lack of dialog with the real users. We as developers are not all to blame for this, of course. On unstructured projects without any particular process applied one finds very often that users or representatives of the users, such as support personell, sales persons or similar have a direct link to the developers. If you’ve been on a project like this, you’ll probably also remember that there was especially one guy in the office these guys went to - he was the “Yes Man” that just jumped and fixed that “critical” bug that came on his desk. This is needless to say a very counterproductive way to work; at times you’re not able to get into the zone at all because of all the interruption. Then on more mature teams they’ve applied something like Scrum, eXtreme Programming or similar and been taught that we run demos for the end users or their representatives at the end of an iteration of X number of weeks and then they can have their input into the process. It gaves us the breathing room we were looking for, phew…

The only problem with this breathing room is that it is often misinterpreted and taken too literally. During an iteration, there is nothing saying you can’t speak to a user. In fact, I would highly recommend we do it as often as possible, so that we’re certain that what we’re making is really what the users want. We don’t know, in fact, unless you’re making software that you’re using yourself - like a dev tool or something else you are using on a regular basis, we really haven’t got a clue whatsoever. Even though opening up a communication channel like this sounds scary, and it should sound scary if you’ve ever been on the an unorganised team. Communication is key; make it so that you as a dev contact the users or their representatives and they can’t go to you directly for anything - unless agreed to because you’re working on a particular feature together. So in conclusion; users don’t speak to the developers about things they aren’t working on with the developer - if they’re working on something together, they should be the best of buddies.

One of the practices that can one could consider applying Domain Driven Design (DDD), which comes with a vocabulary in itself that is very helpful for developers. But the best part of DDD is the establishment of a ubiquitous language, a language representing the elements of your particular domain that both the users speak and the developers. Often this means representing the vocabulary the end users already have in your own code. Getting to this language will help you communicate better with the end users and you don’t have to do mental translations between what the users are talking about and “what it really is” in the code.

The secret sauce of agile

We’ve all been touted our ears full of how we have to become agile. You would be crazy to not do it, we’ve been told. I think its important to clarify what agile means; its all about feedback loops. Get the software into the hands of the users as fast as possible and often as possible, so that we can learn what works and what doesn’t. Find problems as early as possible when it is still fresh in the minds of the developers - leading to saving time in the long run and increased quality of the user experience and code.

I’m all in, keeping the feedback loop as tight as possible. In fact, the part about the feedback loop and keeping it as tight as possible is something I promote all the time as well - for just about everything from execution time of your automated tests to feedback from the real users. One of the promises of being agile is to be able to have software that is changeable and adapt to input from users. But I would argue that there is part of this story that seems to drown in the overall messaging; your software needs be written in a way that it is agile. What does that mean? I think there are a few basic techniques and principles we can apply to make this a reality.

Testing is one of these pieces to the puzzle. Writing code that verifies the production code and its promise is something some of us has been doing for quite a while, in fact I would argue - everyone does this, but not necessarily consciously. Every time one adds a little console app or something to test out a particular feature in the system that you’re developing - just to keep you from having to run the entire app every time, you’re then putting in code that you can more easily debug what you’re working on in a more controlled way. These are in fact tests that could with a small fine-tuning become automated tests that can be run all the time to verify that you’re not breaking anything while moving forward. This gives you a confidence level to be able to change your software whenever change is needed, typically due to requirement changes or similar.

Also, your tests becomes the documentation representing the developers understanding of what is to be created. Applying techniques like specification by example and BDD, your code becomes readable and understandable for other developers to jump in and understand what you originally had intended without having to explain it verbosely to another developer. By being clear in the naming of your specifications / tests, writing them out with the Gherkin language, you add the ability to really look at test / spec signatures and have a dialog right there and then with a user.

But with testing comes a new responsibility, the ability to have code that is testable. All of a sudden it can feel painful to actually test your code, due to complex setup and often one ends up testing a lot more than needed. The sweet spot lies in being able to mock things out, give fake representations of dependencies your system under test might have and test the interaction. Testing in isolation is the key, which leads to a couple of things that you want to consider when writing your code; Single Responsibility Principle and Interface Segregation Principle. Both of these helps in testing, but are on their own good practices and makes your software ready for change. With SRP, your code gets focused and specialised - having a type represent one aspect and a method only do one thing, all of a sudden your code gets more readable as well. Applying ISP, you have contracts between systems that aren’t concrete and these can be swapped out, giving you great opportunity for change but also makes it a lot easier to test interaction between systems as you now can give them fake representations and all you need to test is that the interaction between them are correct. All this again leading to higher focus and in return in the long run giving you as a developer greater velocity and confidence in changing your implementations.

I strongly feel that testing is a necessity for agile thinking, it gives you the power of changeability, and often forgotten as a vital part of the big picture.

Rinse repeat; patterns

When developing software for a while, one finds things that work and things that doesn’t; patterns and anti-patterns. These are often organically appearing by plain and simple experience. And some of these patterns have been promoted and been published in literature. Patterns are helpful on many levels; they give you as a developer a vocabulary - making it easier to talk to other developers. They also provide guidance when you not necessarily know how to do something - knowing a few well known patterns can then become very powerful.

Concerns

Software can be divided into logical parts that are concerned with different things. In its simplest form you can easily define things like what the user sees and uses as its separate concern, it is the UI, frontend or the View. Then you can also pinpoint down to what is the place that performs all the business logic, the thing that represents the vocabulary of your domain. Going back into the UI we can be even more fine-grained, you have the concern of defining the elements that are available in the UI, then you have the concern of styling these elements and making them look like you want to and thirdly the concern of making them come to life, the code that serves information into the UI and responds to the users actions. Right there in the UI we have at least 3 concerns. Translating this into the world of web this means; HTML, CSS and JavaScript. These are three different things that you should treat very differently, not in the same file withand tags all over the place, separate them out!

On all tiers in an application you have different concerns, some of them hard to identify and some just pop up, but they are nevertheless just as important to find and identify them to separate them out. Sometimes though, you run into something that applies to more than one thing; cross cutting concerns. These can be things you just want to automatically be used, or is something your main application does not want to think too much about. The classic example of such things are transactions, threading and other non-functional requirements. Something that we can technically identify and “fix” once and not have to think about again, in fact they often have a nature of being something we don’t have to think about in advance either. But there are other cross cutting concerns as well. Take for instance a scenario were you have the need for a particular information all over, something like tenant information, user information or similar. Instead of having to pass it along on all method calls and basically exposing your code to the possibility of being wrongly used, you can simply take these cross cutting concerns in as a dependency on the constructor of your system and define it at the top level of your application what that means.

Identifying your different concerns and also cross cutting concerns makes life so much simpler. Having cross cutting concerns identified makes your code more focused and easier to maintain and I would argue, more understandable. You’re not getting the big picture when looking at a single system, but nor should you. Keep things focused and let the different systems care about their own thing and nothing else. In fact, you should be very careful about not bleeding things between concerns. If your UI needs to have something special going on, you should be thinking twice about introducing it already in the business logic often sitting on the server. A classic example of this is to model a simple model of a person holding FirstName and LastName and then introduce a computed property holding FullName. FullName in this scenario is only needed in the UI and gives no value in polluting the model sitting on the server with this information. In fact, you could easily just do that in the View - not even having to do anything in the view logic to make that work.

Fixing bugs - acceptance style

There is probably no such thing as software completely without bugs - so embrace fixing bugs. You will have to fix problems that come in, but how you approach the bug fixing is important. If you jump in and just stunt the fix disregarding all the tests / specifications you wrote, you’re really just shooting yourself in the foot. Instead, what you should be doing is to make sure that you can verify the problem by either changing existing tests / specifications to accommodate the new desired behaviour or add tests / specifications to verify the problem. Then, after you’ve verified the problem by running the automated test and seeing that it is “green”, you go back and fix the problem - run the automated test and verify that it has become “green”. A generally good practice to follow is the pattern of “Red”, “Green”, “Refactor”. You can read more about it a previous post here.

Compositional Software

Making software in general is a rather big task, its so big that thinking of your solution as one big application is probably way too much to fit inside your head. So instead of thinking of it as one big application, think of it as a composition of many small applications. They are basically representing features in your larger application composition. Make them focused and keep them isolated from the rest of the “applications” in your composition. Keeping them in isolation enables you to actually perform changes to all the artefacts of the “application” without running the risk of breaking something else. All you need to do then is to compose it all together at the very top. Doing this you need to consider things like low coupling and high cohesion. Don’t take dependencies on other “applications”, keep all the artefacts of each tier close - don’t go separating out based on artificial technical boundaries such as “.. in this folders sits all my interfaces .. “ <- swap out interfaces with things like “controllers, views, viewModels, models”. If the artefact is relevant to your feature - keep it close, in the same folder. Nothing wrong with having HTML, CSS and JavaScript files in the same folder for the frontend part of your solution, in fact - nothing wrong having the relevant frontend code that runs on the server also sitting in the same folder. Partition vertically by feature and horizontally by tier instead and make your structure consistent throughout. All the tiers can have the same folders, named the same. If you have a concept and feature of “Documents”, that is the name of the folder in all tiers holding all the artefacts relevant for the tier in that folder for that feature.

File->New….

It is by far so much easier to create new things rather than changing, fixing or adding to existing code, ask anyone that is or has maintained a legacy system without having the opportunity to do the rewrite. Software do have a tendency to rot which makes it harder and harder to maintain over time. I would argue that a lot of that has to do with how the software is written. Applying the principles mentioned in this post like SRP, SOC, building compositions are all part of making this easier and actually make you for the most part create new things rather than having to fix too much of your old code. In fact, applying these principles you will find yourself for the most part just adding new things, rather than amending existing. This leads to more decoupled software, which is less prone to regressions.

Conclusion

By saying Sustainable Software Development, I mean software that will sustain in time and meet the following requirements:

• Maintainable - spaghetti is not what you’re looking for. Apply practices such as the SOLID principles, decouple things. Make sure you identify the concerns in your software, separate these out. Identify cross cutting concerns, keep them in its own little jar and not bleed it through your entire app.
• Changeable - Single Responsibility do in fact mean single, make the contracts explicit - you should be constantly changing your software, improving it - not just for new features or added value, but also when dealing with bugs - improve the quality of the system through refactoring.
• Debuggable - Make your code debuggable, but not let the tool suck you in - use it wisely
• Testable - Be sure to write code that is singled out on its responsibility, taking dependencies on contracts - making it possible to be focused and your tests small and focused

What I’m realising more and more from the work I do, there is no single magic bullet - there are many things that fit together and doing them together makes the bigger picture better. We’re often told when some new practice is introduced, it will be the magic bullet making all problems go away. They often fail to say that they were already doing a bunch of other practices that needs to be applied as well in order for it to be successful.

It is very important that we as developers focus on the important parts; deliver business value and deliver it so that we can change it when needed without sacrificing quality.

Applying the things in this blog post has proved to us that we get measurably less bugs, less regression, and it all boils down to the simple fact that we’re for the most part just developing new things. After all, focusing on single responsibility and separating things out and putting things in isolation leaves to creating new code all the time, rather than having to constantly just add yet another if/else here and there to accommodate change. Of course, there are much more to it than a post like this can ever capture, but its part of the core values that we believe in.