.net, C#, Code Quality

Avoid code generation if compiler is in error state

One of the things discovered with usage of our proxy generator was that when working in the code and adding things like another property on a class/record. While typing we could see the generator running and spitting out files as we type. For instance, lets say we have the following:

public record DebitAccount(AccountId Id, AccountName Name, PersonId Owner, double Balance);

If I were to now after a build start typing for a fifth property in this, it would start spitting out things. First a file without any name, then as I typed the type I would get a file for each letter I added – depending on how fast I typed. So if this was a string type, I could be seeing s.ts, st.ts, str.ts and so on.

Turns out this is by design. One of the optimizations done for the dotnet build command is that it keeps the compiler process around. It starts a build-server that will handle incremental builds as things is happening and therefor be prepared for when we actually do a dotnet build and be as fast as it can be.

When doing proxy generation, this is obviously less than optimal. To avoid this, we added a check if there are any diagnostics errors from the compiler – if so, do not generate anything.

In our source generator we added a line at the top to avoid this:

public void Execute(GeneratorExecutionContext context)
{
    if (context.Compilation.GetDiagnostics().Any(_ => _.Severity == DiagnosticSeverity.Error)) return;
}

.net, Code Quality, Code Tips, Practices

Domain Concepts

Back in 2015, I wrote about concepts. The idea behind these are that you encapsulate types that has meaning to your domain as well known types. Rather than relying on technical types or even primitive types, you then formalize these types as something you use throughout your codebase. This provides you with a few benefits, such as readability and potentially also give you compile time type checking and errors. It does also provide you with a way to adhere to the element of least surprise principle. Its also a great opportunity to use the encapsulation to deal with cross cutting concerns, for instance values that adhere to compliance such as GDPR or security concerns where you want to encrypt while in motion etc.

Throughout the years at the different places I’ve been at were we’ve used these, we’ve evolved this from a very simple implementation to a more evolved one. Both these implementations aims at making it easy to deal with equability and the latter one also with comparisons. That becomes very complex when having to support different types and scenarios.

Luckily now, with C# 9 we got records which lets us truly simplify this:

public record ConceptAs<T>
{
    public ConceptAs(T value)
    {
        ArgumentNullException.ThrowIfNull(value, nameof(value));
        Value = value;
    }

    public T Value { get; init; }
}

With record we don’t have to deal with equability nor comparable, it is dealt with automatically – at least for primitive types.

Using this is then pretty straight forward:

public record SocialSecurityNumber(string value) : ConceptAs<string>(value);

A full implementation can be found here – an implementation using it here.

Implicit conversions

One of the things that can also be done in the base class is to provide an implicit operator for converting from ConeptAs type to the underlying type (e.g. Guid). Within an implementation you could also provide the other way, going from the underlying type to the specific. This has some benefits, but also some downsides. If you want the compiler to catch errors – obviously, if all yours ConceptAs<Guid> implementations would be interchangeable.

Serialization

When going across the wire with JSON for instance, you probably don’t want the full construct with a { value: <actual value> }, or if you’re storing it in a database. In C# most serializers support the notion of conversion to and from the target. For Newtonsoft.JSON these are called JsonConverter – an example can be found here, for MongoDB as an example, you can find an example of a serializer here.

Summary

I highly recommend using strong types for your domain concepts. It will make your APIs more obvious, as you would then avoid methods like:

Task Commit(Guid eventSourceId, Guid eventType, string content);

And then get a more clearer method like:

Task Commit(EventSourceId eventSourceId, EventType eventType, string content);

Code Quality

Evangelizing

A while back I got into a discussion with another developer. The other guy basically critiqued my code because Angular 1 was so much better than Knockout. It really annoyed me, not because I felt the other person was wrong, or at least not wrong in the sense that I loved Knockout that much. But it annoyed me because its really comparing apples and pears. Sure, both targeting to solve workloads for frontend development on the Web. The arguments presented was very clearly for Angulars ease and against Knockouts idiotic properties being “ko.observable*”. But fundamentally they are different in its underlying core values and concepts. Knockout aiming for a reactive model with its observables, while Angular not having any reactive model at all and being a MVA (Model View Anything) – not opinionated.

This is what bugs the hell out of me. Since these things are so different, fundamentally – how can one be better than the other? And as a consequence, just because one has found something that makes sense for you, why does it have to be true for everyone else? To me for instance; Angular is just completely weird and can’t understand why one would like it at all.

Lets take a step back; this is not a “Foo Framework” against “Bar Framework”. I’m trying to get to the core of something that I think is way much broader than just software development – but probably more visible in software, seing we have all the options we have. Going back to when I grew up for instance, I was a fan of SONY’s HiFi equipment, while one of my best friends was a JVC fan. We had the weirdest discussions. Enter my third friend who was fan of things no one had ever heard of; NAD, Denon and similar. The discussions got even weirder – not from our third friend, but from the others trying to convince him that he was wrong. You can see this with just about everything – people being so convinced their choice of car is the best, TV and of course religion. It seems that this is part of human nature. And we love to evangelize it and try to convince everyone else we’re right.

Trust me, I’m not taking a high road here; because I do this myself. I find something that I enjoy enough to start “evangelizing” it. This is probably something well defined in psychology already; related to wanting to getting a reinforcement of ones own beliefs and decisions.

Core values

There is nothing wrong with a good discussion and if you’ve discovered something you should of course show it off and see if others find it cool. But if you’re in a team, having these discussions around one thing being better than other can truly be poison. In my oppinion it also reflects a symptom of lack of common core values. The discussion of Angular vs Knockout as described above is a weird one because it really is not about a discussion between the two, but rather about the differences in underlying mindset and core values. If you’re used to and enjoy a particular way of thinking, changing to something else is very hard. For instance, if you’re used to object oriented programming, functional can be too far away to understand.

Job protection

A scary thing I’ve seen a few times; people get comfortable in their job – so comfortable that they start making up reasons for keeping their beloved way of doing things. The reasons behind this is not always obvious. The ones I’ve encountered have been in two categories.

I’ve spent a lot of time learning what I know now and don’t see the point of learning anything new.
The company has had success with the way we’ve done things.

For the first reason I’m just going to say “ehh… what…”. That is especially true for our line of work; software development. A relatively young business with so many things changing all the time, and not only changes in software – but the hardware in which the software is running on. Personally I think that is a risky way of going about thinking in our line of business.The second reason; if you’re having success it is really hard to see and even argue that you should change your way. If the business you’re working for is having success, but for instance the software you’re working on hits a snag and you get to the point of getting to rewrite it. Even then, you will have a hard time seeing that you need to do it in a different way; after all – the way you did it worked, because it brought success.

In both cases I think the boiling frog anecdote might be the reason behind. Basically, while building something over time – you won’t notice the things that lead to something not really being done in a good way. You’re having many small successes and feel good periods and you simply aren’t noticing that the project is in fact failing. Most projects do tend to have the best velocity in the beginning and slow down over time. Something the team members executing on it won’t notice, because it is happening slowly. But anyone external to the development team is noticing, because their demands for new things have not declined over time but rather increased and often these are the people dealing with the problems as well – while the development team is having a hard time getting through things at the right pace.

I find what Einstein saying: “Insanity: doing the same thing over and over again and expecting different results.” interesting and so true. One gets into discussions with management of needing to recreate the product from scratch, because you’ve identified something that is holding the development back and is too painful to fix. You get to do “File -> New Project” and do it again using the same kind of thinking we used when we created the problems in the first place (referring to another Einstein quote).

We are so lucky in our profession; there are so many choices – so many ways of doing things. It truly is a luxury.

Mixing it up

At a place I was we were about to embark on a rewrite. A few of us had discovered some new principles and experienced some pain while doing a small 3-4 month project in isolation. From this two of us were tasked to do a small proof of concept for the larger project, to sketch out a direction and a starting point for a discussion. After doing this we decided on the direction for the project and what fundamentals to go for. To get the team familiar with different thinking, we mixed things up a bit by doing a period of Ruby development. Writing Ruby and discovering the culture of rubyists let us write our C# in a different way when we got back to it.

Keeping fresh

So, how do you keep fresh thinking? It is hard, it really is. Everyone suggesting everything from kode katas, pair programming, friday lightning talks and all these cool things must realize that it is hard to get there. They are all good ideas and things that can help your team keep fresh. But getting to a place where you can build this type of culture is hard. You need the trust from the business to be able to do so. Also, you probably need to have a certain critical mass of people working together before it can be accomplished. Lets face it, being a development team of one makes this very hard. But it is very doable. It starts with recognizing that one has to keep fresh and on top of things. If you think when you graduated that you didn’t need to learn new things, you need to start with changing your attitude. That attitude won’t push anything forward in terms of new ways of doing things. Innovation comes from being aware of what is going around out there and combining wizdom and build on top of it or go in a different direction from established wizdom – but nevertheless, its about knowledge.

Breaking it all up…

One way of being able to push forward and try different things to be able to learn new tricks and gain knowledge could be to start breaking your software up into smaller pieces; enter the buzz of the Micro Services. Sure, there is a lot of buzz around this. But cut to the bone; its about breaking monoliths up into smaller digestable pieces. These digestables could be implemented in different ways, in fact – I would argue they should be. Basically because the different parts have different requirements both for business and ultimately also technical. This is a great opportunity to mix things up.

Back to the value thing…

Remember I said something about core values. I’m convinced this sits at the heart of it. If you get to work and it is a predictable environment in terms of you know you share something with your coworkers. Having the thing you share being core values, rather than the belief in something hard and concrete as a specific technology – you move the discussions to a more healthy level. If your discussion is which of Windows, Linux or OS X is the best operating system, you’re probably not solving the real problems. If you can look past this and past implementational details and start having a solid fundamental that you all believe in – you can move up the stack much more rapidly. Chances are that a lot of the decisions will automatically be made for you if you settle the core value debate first. I’ve blogged about this before – I have no problem reiterating the importance of this, I think it is at the heart of a lot of silly discussions that could’ve been resolved without them having started.

Wrapping up…

Is it really important that I drive a Toyota and you drive a Ford? Will the world be a better place if I switched to a Ford? Probably not! Conformity is nothing but just that. It doesn’t really solve any fundamental issues, it covers it up. It is the fundamentals that are different. In software development, these fundamentals are core principles – how you believe software should be written and what makes you productive. This view is a subjective view and it is also a very temporary view. Once you’ve learned new ways of doing things, chances are you look back at your previous knowledge level and laugh or you’re embarressed you ever had that view. If you’re really cool, you even move forward in life and forget you had the view and laugh at people who have the view you had way back… ehh… a whole month ago… Everything is hard until you know it, and then it is the easiest thing in the world. We tend to make a point about it as well. Once we know it, we’re so proud of knowing it that we tell everyone. Knowledge is a moving target, new knowledge emerges – embrace this concept and embrace change. I know I will work for this and aim for getting better at doing better in this area. I have a few comfort zones to break out of and will push forward. Wish me luck!

Bifrost, Code Quality, CQRS

Concepts and more

With Bifrost we’re aligning ourselves more and more with being a platform for doing Domain Driven Design. Introducing more and more artefacts from the building blocks as we go along. When we set out to build Bifrost, we decided early on to be true to not be building anything into it that we didn’t need in a real world scenario. This was after we had started falling into the pattern of what if of software development. We started imagining problems and had to deal with them way before they had actually happened. With the risk of generalising; a fairly common scenario amongst dirty minded tech people. It stems from experience, knowing that there will always be something that can go wrong. Sure, there always is. I digress, I think this could probably be a blogpost on its own. The point being, we were heading down this path and for some reason got jolted back to reality and we started focusing on implementing only the things we needed and rather actually go back and remove things that came out of the “what if game”. On this new path we wanted to also stay focused on implementing things that were aligned with DDD and keep a close eye on the user.

Concepts

With the philosophy of CQRS at heart built with SOLID care we keep a very close eye on being very specific in our modelling. Things that are used in one part of the system is not automatically reused somewhere else, for the DRYness. We don’t believe in DRYing up properties and we favor composition of inheritance. Logic is still kept only once, on the command side of the fence. With all these principles at hand we were missing something that would link it all back together and make things look and feel consistent.

Let’s look at a scenario; say I want to update the address of a person. A command could be something like the following:

using System;
using Bifrost.Commands;

public class UpdateAddressForPerson : Command
{
   public Guid PersonId { get; set; }
   public string Street { get; set; }
   public string City { get; set; }
   public string PostalCode { get; set; }
   public string Country { get; set; }
}

In Bifrost you’d then have a CommandHandler to deal with this and then an AggregateRoot that would probably look like the following:

using System;
using Bifrost.Domain;

public class Person : AggregateRoot
{
   public Person(Guid personId) : base(personId) {}
   public UpdateAddress(string street, string city, string postalCode, string country)
   {
      // Apply an event
   }
}

The aggregate would then apply an event that looks like the following:

using System;
using Bifrost.Events;

public class AddressUpdatedForPerson : Event
{
   public Guid PersonId { get; set; }
   public string Street { get; set; }
   public string City { get; set; }
   public string PostalCode { get; set; }
   public string Country { get; set; }
}

An event subscriber would pick this up and update a read model that might look like the following:

using System;
using Bifrost.Read;

public class AddressForPerson : IReadModel
{
   public Guid PersonId { get; set; }
   public string Street { get; set; }
   public string City { get; set; }
   public string PostalCode { get; set; }
   public string Country { get; set; }
}

That was the artefacts we would typically be dealing with; command, aggregateroot, event and readmodel. For simplicity, these look pretty much the same – but they don’t have to, and in fact; most of the time they don’t. Lets address something here. We’re losing out on a potential in the modelling here. Take the Guid representing the unique identifier for the person. This is in fact something that is part of the domain vocabulary that we’re losing by just making it a Guid directly.

In Bifrost we have something called ConceptAs that we can use to represent this domain concept. This is a base class that we recognize throughout the system and deals with properly during serialisation between the different out of process places it might go.

using System;
using Bifrost.Concepts;

public class Person : ConceptAs<Guid>
{
   public static implicit operator Person(Guid personId)
   {
      return new Person() { Value = personId };
   }
}

What this does is to wrap up the primitive, giving us a type that represent the domain concept. One modelling technique we applied when doing this is to stop referring to it as an id, so we started calling it the noun in which it represented. For us, this actually became the abstract noun. It doesn’t hold any properties for what it represents, only the notion of it. But codewise, this looks very good and readable.

In the ConceptAs base class we have an implicit operator that is capable of converting from the new type to the primitive, unfortunately C# does not allow for the same implicit operator going the other way in the base class, so this has to be explicitly implemented. With these operators we can move back and forth between the primitive and the concept. This comes very handy when dealing with events. We decided to drop the concepts in the events. The reason for this is that versioning becomes very hard when changing a concept, something you could decide to do. It could also make serialization more complex than you’d hope for with some serializers. Our conclusion is that we keep the events very simple and uses primitives, but everywhere else the concept is used.

The way we structure our source we basically have a domain project with our commands, command handlers and aggregates. Then we have a project for our read side and in between these two projects sit a project for holding the domain events. With this model we don’t get a coupling between the domain and the read, which is one of our primary objectives. The concepts on the other hand, they are going to be reused between the two. We therefor always have a concepts project where we keep our concepts.

Our typical project structure:

So, now that we have our first concept, what did it do? It replaced the Guid reference throughout, introducing some clarity in our models. But the benefit we stumbled upon with this; we now have something to do cross cutting concerns with. By having the type of pipelines we have in Bifrost, we can now start doing things based on the type being used in different artefacts. Take the command for instance, we can now introduce input validation or business rules for it that would be applied automatically whenever used. Our support for FluentValidation has a BusinessValidator type that can be used for this:

using Bifrost.FluentValidation;
using FluentValidation;

public class PersonBusinessValidator : BusinessValidator<Person>
{
   public PersonBusinessValidator()
   {
      RuleFor(p => p.Value)
         .Must(… a method/lambda for checking if a person exist …)
         .WithMessage(“The person does not exist”);
   }
}

As long as you don’t create a specific business validator for the command, this would be automatically picked up. But if you were to create a specific validator for the command you could point it to this validator as a rule for the person property.

The exact same thing can then also be used for an input validator, which then would generate the proper metadata for the client and execute the validator on the client before the server.

It opens up for other cross cutting concerns as well, security for instance.

Value Objects

A second type of object, with the same importance in expressing the domain and opening for solving things in a cross cutting manner are value objects. This is a type of object that actually holds information, attributes that have value. They are useless on their own, but often used in the domain and also on the read side. Their uniqueness is based on all the fields in it. We find these in any domain all the time, they are typical things like money, phone number or in our case address. These are just the off the top of my head type of value objects you’d have, but you’ll find these in many forms. Lets tackle address:

using System;
using Bifrost.Concepts;

public class Address : Value
{
   public string Street { get; set; }
   public string City { get; set; }
   public string Postal { get; set; }
   public string Country { get; set; }
}

The Value baseclass implements IEquatable and deals with the property comparisons for uniquness.

With the value object you do get the same opportunities as with the concept for input and business validation, and yet another opportunity for dealing with cross cutting concerns.

If we summarize the sample before with these new building blocks, we would get:

using System;
using Bifrost.Commands;

public class UpdateAddressForPerson : Command
{
   public Person Person { get; set; }
   public Address Address { get; set; }
}

Our event:

using System;
using Bifrost.Events;

public class AddressUpdatedForPerson : Event
{
   public Guid PersonId { get; set; }
   public string Street { get; set; }
   public string City { get; set; }
   public string PostalCode { get; set; }
   public string Country { get; set; }
}

As you can see, we keep it as it was, with the properties all in the event.

Our AggregateRoot:

using System;
using Bifrost.Domain;

public class Person : AggregateRoot
{
   public Person(Guid person) : base(person) {}

   public UpdateAddress(Address address)
   {
      Apply(new AddressUpdatedForPerson {
         Person = Id,
         Street = address.Street,
         City = address.City,
         Postal = address.Postal,
         Country = address.Country
      });
   }
}

The readmodel then would be:

using System;
using Bifrost.Read;

public class AddressForPerson : IReadModel
{
   public Person Person { get; set; }
   public Address Address { get; set; }
}

Conclusion

For someone more familiar with traditional N-tier architecture and modelling your EDM or rather than separating things out like this, this probably raises a few eyebrows and questions. I can totally relate to it, before starting the Bifrost journey – I would have completely done the same thing. It seems like a lot of artefacts hanging around here, but every one of these serves a specific purpose and is really focused. Our experience with this is that we model things more explicitly, we reflect what we want in our model much better. Besides, you stop having things in your domain that can be ambiguous, which is the primary objective of DDD. DDD is all about the modelling and how we reach a ubiquitous language, a language that represent the domain, a language we all speak. From this perspective we’ve found domain concepts and value objects to go along with it to be very useful. With them in place as types, we found it very easy to go and retrofit cross cutting concerns we wanted in our solution without having to change any business logic. When you look at whats involved in doing it, its just worth it. The few lines of code representing it will pay back in ten folds of clarity and opportunities.

Code Quality

Sustainable Software Development

Software is hard to make, capturing the domain and getting it right with as little bugs as possible. Then comes release time and your users starts using it, things get even harder – now comes all the changes for all the things that you as a developer did wrong, all the bugs, things you didn’t think of, misunderstandings. All that and the users have requirement changes or additional features they’d love to have on top. I’ve often heard people say things like “… if it wasn’t for the users, creating software would be quite joyful …”. It says tons, it does say amongst other things that creating software is not an easy task. Also, it does say a bunch of other things as well, for instance – it says that we’re not very good at talking to our users, or we’ve taken the concept of having your team working in black-box environment too far or too literally.

Even if we did it all right there are tons of external events that could cause the requirement of changing what we already made. The market could change, new competitors arriving to the table or existing competitors taking a completely different approach to the same problem you just solved. Another aspect that can cause changes is that the people we develop the software for that we’ve included in the dialog are learning all the way through the process of what is possible and what is not, causing them to have more knowledge and wanting change.

The purpose of this post is to shed some light on what we think are good practices on their own but put together represents a very good big picture of how you can create software that will easier to change, meeting the requirements of your users better and possibly reduce the number of bugs in your software.

Talk Talk Talk

One of the biggest mistakes I think we do is to think we are able to intuitively understand our users, what they need and want and also how it should take form. Going ahead and just doing things our way does not come from an inherent arrogance from us developers, but rather something I think is closer to an what we consider an intelligent and qualified assumption of what we think we’re making. At the core of this problem sits the lack of dialog with the real users. We as developers are not all to blame for this, of course. On unstructured projects without any particular process applied one finds very often that users or representatives of the users, such as support personell, sales persons or similar have a direct link to the developers. If you’ve been on a project like this, you’ll probably also remember that there was especially one guy in the office these guys went to – he was the “Yes Man” that just jumped and fixed that “critical” bug that came on his desk. This is needless to say a very counterproductive way to work; at times you’re not able to get into the zone at all because of all the interruption. Then on more mature teams they’ve applied something like Scrum, eXtreme Programming or similar and been taught that we run demos for the end users or their representatives at the end of an iteration of X number of weeks and then they can have their input into the process. It gaves us the breathing room we were looking for, phew…

The only problem with this breathing room is that it is often misinterpreted and taken too literally. During an iteration, there is nothing saying you can’t speak to a user. In fact, I would highly recommend we do it as often as possible, so that we’re certain that what we’re making is really what the users want. We don’t know, in fact, unless you’re making software that you’re using yourself – like a dev tool or something else you are using on a regular basis, we really haven’t got a clue whatsoever. Even though opening up a communication channel like this sounds scary, and it should sound scary if you’ve ever been on the an unorganised team. Communication is key; make it so that you as a dev contact the users or their representatives and they can’t go to you directly for anything – unless agreed to because you’re working on a particular feature together. So in conclusion; users don’t speak to the developers about things they aren’t working on with the developer – if they’re working on something together, they should be the best of buddies.

One of the practices that can one could consider applying Domain Driven Design (DDD), which comes with a vocabulary in itself that is very helpful for developers. But the best part of DDD is the establishment of a ubiquitous language, a language representing the elements of your particular domain that both the users speak and the developers. Often this means representing the vocabulary the end users already have in your own code. Getting to this language will help you communicate better with the end users and you don’t have to do mental translations between what the users are talking about and “what it really is” in the code.

The secret sauce of agile

We’ve all been touted our ears full of how we have to become agile. You would be crazy to not do it, we’ve been told. I think its important to clarify what agile means; its all about feedback loops. Get the software into the hands of the users as fast as possible and often as possible, so that we can learn what works and what doesn’t. Find problems as early as possible when it is still fresh in the minds of the developers – leading to saving time in the long run and increased quality of the user experience and code.

I’m all in, keeping the feedback loop as tight as possible. In fact, the part about the feedback loop and keeping it as tight as possible is something I promote all the time as well – for just about everything from execution time of your automated tests to feedback from the real users. One of the promises of being agile is to be able to have software that is changeable and adapt to input from users. But I would argue that there is part of this story that seems to drown in the overall messaging; your software needs be written in a way that it is agile. What does that mean? I think there are a few basic techniques and principles we can apply to make this a reality.

Testing is one of these pieces to the puzzle. Writing code that verifies the production code and its promise is something some of us has been doing for quite a while, in fact I would argue – everyone does this, but not necessarily consciously. Every time one adds a little console app or something to test out a particular feature in the system that you’re developing – just to keep you from having to run the entire app every time, you’re then putting in code that you can more easily debug what you’re working on in a more controlled way. These are in fact tests that could with a small fine-tuning become automated tests that can be run all the time to verify that you’re not breaking anything while moving forward. This gives you a confidence level to be able to change your software whenever change is needed, typically due to requirement changes or similar.

Also, your tests becomes the documentation representing the developers understanding of what is to be created. Applying techniques like specification by example and BDD, your code becomes readable and understandable for other developers to jump in and understand what you originally had intended without having to explain it verbosely to another developer. By being clear in the naming of your specifications / tests, writing them out with the Gherkin language, you add the ability to really look at test / spec signatures and have a dialog right there and then with a user.

But with testing comes a new responsibility, the ability to have code that is testable. All of a sudden it can feel painful to actually test your code, due to complex setup and often one ends up testing a lot more than needed. The sweet spot lies in being able to mock things out, give fake representations of dependencies your system under test might have and test the interaction. Testing in isolation is the key, which leads to a couple of things that you want to consider when writing your code; Single Responsibility Principle and Interface Segregation Principle. Both of these helps in testing, but are on their own good practices and makes your software ready for change. With SRP, your code gets focused and specialised – having a type represent one aspect and a method only do one thing, all of a sudden your code gets more readable as well. Applying ISP, you have contracts between systems that aren’t concrete and these can be swapped out, giving you great opportunity for change but also makes it a lot easier to test interaction between systems as you now can give them fake representations and all you need to test is that the interaction between them are correct. All this again leading to higher focus and in return in the long run giving you as a developer greater velocity and confidence in changing your implementations.

I strongly feel that testing is a necessity for agile thinking, it gives you the power of changeability, and often forgotten as a vital part of the big picture.

Rinse repeat; patterns

When developing software for a while, one finds things that work and things that doesn’t; patterns and anti-patterns. These are often organically appearing by plain and simple experience. And some of these patterns have been promoted and been published in literature. Patterns are helpful on many levels; they give you as a developer a vocabulary – making it easier to talk to other developers. They also provide guidance when you not necessarily know how to do something – knowing a few well known patterns can then become very powerful.

Concerns

Software can be divided into logical parts that are concerned with different things. In its simplest form you can easily define things like what the user sees and uses as its separate concern, it is the UI, frontend or the View. Then you can also pinpoint down to what is the place that performs all the business logic, the thing that represents the vocabulary of your domain. Going back into the UI we can be even more fine-grained, you have the concern of defining the elements that are available in the UI, then you have the concern of styling these elements and making them look like you want to and thirdly the concern of making them come to life, the code that serves information into the UI and responds to the users actions. Right there in the UI we have at least 3 concerns. Translating this into the world of web this means; HTML, CSS and JavaScript. These are three different things that you should treat very differently, not in the same file with <style/> and <script/> tags all over the place, separate them out!

On all tiers in an application you have different concerns, some of them hard to identify and some just pop up, but they are nevertheless just as important to find and identify them to separate them out. Sometimes though, you run into something that applies to more than one thing; cross cutting concerns. These can be things you just want to automatically be used, or is something your main application does not want to think too much about. The classic example of such things are transactions, threading and other non-functional requirements. Something that we can technically identify and “fix” once and not have to think about again, in fact they often have a nature of being something we don’t have to think about in advance either. But there are other cross cutting concerns as well. Take for instance a scenario were you have the need for a particular information all over, something like tenant information, user information or similar. Instead of having to pass it along on all method calls and basically exposing your code to the possibility of being wrongly used, you can simply take these cross cutting concerns in as a dependency on the constructor of your system and define it at the top level of your application what that means.

Identifying your different concerns and also cross cutting concerns makes life so much simpler. Having cross cutting concerns identified makes your code more focused and easier to maintain and I would argue, more understandable. You’re not getting the big picture when looking at a single system, but nor should you. Keep things focused and let the different systems care about their own thing and nothing else. In fact, you should be very careful about not bleeding things between concerns. If your UI needs to have something special going on, you should be thinking twice about introducing it already in the business logic often sitting on the server. A classic example of this is to model a simple model of a person holding FirstName and LastName and then introduce a computed property holding FullName. FullName in this scenario is only needed in the UI and gives no value in polluting the model sitting on the server with this information. In fact, you could easily just do that in the View – not even having to do anything in the view logic to make that work.

Fixing bugs – acceptance style

There is probably no such thing as software completely without bugs – so embrace fixing bugs. You will have to fix problems that come in, but how you approach the bug fixing is important. If you jump in and just stunt the fix disregarding all the tests / specifications you wrote, you’re really just shooting yourself in the foot. Instead, what you should be doing is to make sure that you can verify the problem by either changing existing tests / specifications to accommodate the new desired behaviour or add tests / specifications to verify the problem. Then, after you’ve verified the problem by running the automated test and seeing that it is “green”, you go back and fix the problem – run the automated test and verify that it has become “green”. A generally good practice to follow is the pattern of “Red”, “Green”, “Refactor”. You can read more about it a previous post here.

Compositional Software

Making software in general is a rather big task, its so big that thinking of your solution as one big application is probably way too much to fit inside your head. So instead of thinking of it as one big application, think of it as a composition of many small applications. They are basically representing features in your larger application composition. Make them focused and keep them isolated from the rest of the “applications” in your composition. Keeping them in isolation enables you to actually perform changes to all the artefacts of the “application” without running the risk of breaking something else. All you need to do then is to compose it all together at the very top. Doing this you need to consider things like low coupling and high cohesion. Don’t take dependencies on other “applications”, keep all the artefacts of each tier close – don’t go separating out based on artificial technical boundaries such as “.. in this folders sits all my interfaces .. ” <- swap out interfaces with things like “controllers, views, viewModels, models”. If the artefact is relevant to your feature – keep it close, in the same folder. Nothing wrong with having HTML, CSS and JavaScript files in the same folder for the frontend part of your solution, in fact – nothing wrong having the relevant frontend code that runs on the server also sitting in the same folder. Partition vertically by feature and horizontally by tier instead and make your structure consistent throughout. All the tiers can have the same folders, named the same. If you have a concept and feature of “Documents”, that is the name of the folder in all tiers holding all the artefacts relevant for the tier in that folder for that feature.

File->New….

It is by far so much easier to create new things rather than changing, fixing or adding to existing code, ask anyone that is or has maintained a legacy system without having the opportunity to do the rewrite. Software do have a tendency to rot which makes it harder and harder to maintain over time. I would argue that a lot of that has to do with how the software is written. Applying the principles mentioned in this post like SRP, SOC, building compositions are all part of making this easier and actually make you for the most part create new things rather than having to fix too much of your old code. In fact, applying these principles you will find yourself for the most part just adding new things, rather than amending existing. This leads to more decoupled software, which is less prone to regressions.

Conclusion

By saying Sustainable Software Development, I mean software that will sustain in time and meet the following requirements:

Maintainable – spaghetti is not what you’re looking for. Apply practices such as the SOLID principles, decouple things. Make sure you identify the concerns in your software, separate these out. Identify cross cutting concerns, keep them in its own little jar and not bleed it through your entire app.
Changeable – Single Responsibility do in fact mean single, make the contracts explicit – you should be constantly changing your software, improving it – not just for new features or added value, but also when dealing with bugs – improve the quality of the system through refactoring.
Debuggable – Make your code debuggable, but not let the tool suck you in – use it wisely
Testable – Be sure to write code that is singled out on its responsibility, taking dependencies on contracts – making it possible to be focused and your tests small and focused

What I’m realising more and more from the work I do, there is no single magic bullet – there are many things that fit together and doing them together makes the bigger picture better. We’re often told when some new practice is introduced, it will be the magic bullet making all problems go away. They often fail to say that they were already doing a bunch of other practices that needs to be applied as well in order for it to be successful.

It is very important that we as developers focus on the important parts; deliver business value and deliver it so that we can change it when needed without sacrificing quality.

Applying the things in this blog post has proved to us that we get measurably less bugs, less regression, and it all boils down to the simple fact that we’re for the most part just developing new things. After all, focusing on single responsibility and separating things out and putting things in isolation leaves to creating new code all the time, rather than having to constantly just add yet another if/else here and there to accommodate change. Of course, there are much more to it than a post like this can ever capture, but its part of the core values that we believe in.

Code Quality, CQRS

Singling things out…

I was at a client a while back and was given the task to asses their code and architecture and provide a report of whatever findings I had. The first thing that jumped out was that things did more than one thing, the second thing being things concerning itself with things it should not concern itself about. When reading my report, the client recognised what it said and wanted me to join in and show how to rectify things; in good spirit I said:

We approached one specific class and I remember breaking it down into what its responsibilities were and fleshed it all out into multiple classes with good naming telling exactly what it was supposed to do. When we were done after a couple of hours, the developer I sat with was very surprised and said something to the effect of; “.. so, single responsibility really means only one thing..”. Yes, it actually does. A class / type should have the responsibility for doing only one thing, likewise a method within that class should only do one thing. Then you might be asking; does that mean classes with only one methods in them. No, it means that the class should have one subject at a time and each and every method should only do work related to that subject and every method should just have the responsibility of solving one problem. Still, you might be wondering how to identify this.

Lets start with a simple example.

Say you have a system were you have Employees and each and every one of these have a WorkPosition related to it enabling a person to have different positions with the same employer (Yes, it is a real business case. 🙂 ).

The employee could be something like this :

public class Employee
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

A fairly simple class representing an aggregate root.

Then go and add this other thing in our domain; a WorkPosition, a value type that refers to the aggregate:

public class WorkPosition
{
    public int Id { get; set; }
    public int EmployeeId { get; set; }
    public double PositionSize { get; set; }
}

One could argue these represent entities that you might want to get from a database and you might want to call them entities and DRY up your code, since they both have an Id:

public class Entity
{
    public int Id { get; set; }
}

public class Employee : Entity
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

public class WorkPosition : Entity
{
    pubic int EmployeeId { get; set; }
    public double PositionSize { get; set; }
}

Nice, now we’ve dried it all up and the Id property can be reused.

The Id property is a classic pattern, but in domain driven design you would probably not use an integer as Id but rather a natural key that describes the aggregate better. For an Employee this could be the persons social security number. This means that an integer won’t do, but something that can tackle the needs of a social security number. For simplicity in this post, I’ll stick to primitives and do a string, normally I would introduce a domain concept for this; a type representing the concept instead of using generic primitives – more on that in another post.

We want to introduce this, but we’d love to keep the Entity base class, so we can stick common things into it, things like auditing maybe. But now we are changing the type of what identifies an Employee, and it’s not the same as for a WorkPosition; C# generics to the rescue:

public class Entity<T>
{
    public T Id { get; set; }
    public DateTime ModifiedAt { get; set; }
    public string ModifiedBy { get; set; }
}

public class Employee : Entity<string>
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

public class WorkPosition : Entity
{
    public string EmployeeId { get; set; }
    public double PositionSize { get; set; }
}

Great, now we have a generic approach to it and get auditing all in one go.

We’ve just created a nightmare waiting to happen. We’ve made something generic, lost a lot from the domain already just to save typing in one property; Id (yeah I know, there are some auditing there – I’ll get back to that soon). We’ve lost completely what the intent of the Employees identification really is, which was a social security number. At least the name of the property should reflect that; Id doesn’t say anything about what kind of identification it is. A better solution would be going back to the original code and just make it a little bit more explicit:

public class Employee
{
    public string SocialSecurityNumber { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

public class WorkPosition
{
    public int Id { get; set; }
    public string SocialSecurityNumber { get; set; }
    public double PositionSize { get; set; }
}

That made it a lot more readable, we can see what is expected, and in fact we’ve also decoupled Employee and WorkPosition in one go – they weren’t coupled directly before, but the property named EmployeeId made a logical coupling between the two – which we might not need.

Another aspect of this would be bounded contexts; different representations of domain entities depending on the context they are in. In many cases an entity would even have different things that identifies it, depending on the context you’re in. Then Id would be a really bad name of a property and also having a generic representation of it would just make the whole thing so hard to read and understand. Normally what you would have is an underlying identifier that is shared amongst them, but you wouldn’t necessarily expose it in the different bounded contexts. Instead you would introduce a context map that would map from the concepts in the different bounded context to the underlying one.

Back to auditing – don’t we want to have that? Sure. But let’s think about that for a second. Auditing sounds like something you’d love to have for anything in a system, or in a particular bounded context, one could argue its cross cutting. It is a concern of every entity, but I would argue it is probably not something you need to show all over the place; in fact I’ll put forward the statement that auditing probably is an edge case when showing the entities on any dialog. So that means we probably don’t need them on the entities themselves, but rather make sure that we just get that information updated correctly in the database; this could be something we could do directly in the database as triggers or similar, or make sure everything goes through a well-defined common data context that can append this information. Then, for the edge cases were you need the auditing information, model only that and an auditing service of some kind that can get that information for the entities you need.

Fess up

Ok. So I have sinned, I’ve broken the Single Responsibility Principle myself many times, and I will guarantee you that I will break it in the future as well. In fact, let me show you code I wrote that I came across in Bifrost a few weeks back that got the hairs on my back rising, a system called EventStore:

public class EventStore : IEventStore
{
    public EventStore(IEventRepository eventRepository,
                      IEventStoreChangeManager eventStoreChangeManager,
                      IEventSubscriptionManager eventSubscriptionManager,
                      ILocalizer localizer)
    {
        // Non vital code for this sample...
    }

    public void Save(UncommittedEventStream eventsToSave)
    {
        using(_localizer.BeginScope())
        {
            _repository.Insert(eventsToSave);
            _eventSubscriptionManager.Process(eventsToSave);
            _eventStoreChangeManager.NotifyChanges(this, eventsToSave);
        }
    }
}

I’ve stripped away some parts that aren’t vital for this post, but the original class some 60 lines. But looking at the little code above tells me a couple of things:

Its doing more than one thing without having a name reflecting it should do that
The EventStore sounds like something that holds events, similar to a repository – but it deals with other things as well, so…
The API is wrong; it takes something that is uncommitted and it saves it – normally you’d expect a system to take something uncommitted and commit it

The solution to this particular problem was very simple. EventStore needed to do just what it promises to do in its name, all the other stuff is coordination and by the looks of it, it is coordinating streams of uncommitted events. So I introduced just that; UncommittedEventStreamCoordinator with a method of Commit() on it. Its job is then to coordinate the stuff we see above and the EventStore can then take on the real responsibility of dealing with storing things, and in fact I realised that the EventRepository could go at this point because I had tried to solve it all in a generic manner and realised that specialised EventStores for the different databases / storage types we support would be a lot better and not a lot of work to actually do.

Another thing the refactoring did for us was the ability to now turn of saving of events, but still get things published. By binding the IEventStore that we have to an implementation called NullEventStore – we don’t have to change any code, but it won’t save. Also what we also can do is to introduce the ability the EventStore itself to by asynchronous, we can then create something like AsyncEventStore that just chains back to the original EventStore, but does so asynchronously. All in all it adds more flexibility and readability.

Behaviors to the rescue

All good you might think, but how do you really figure out when to separate things out. I’ll be the first to admit, it can be hard sometimes, and also one of them things one can’t just be asked and necessarily be able to identify it within a heartbeat – sometimes it is a process getting to the result. But I would argue that thinking in behaviors makes it simpler, just for the simple fact that you can’t do two behaviors at the same time. Thinking from a testing perspective and going for BDD style testing with a gherkin (given, when, then) also gives this away more clearly; the second you write a specification that has and in it, you’re doing two things.

Why care?

Took a while getting to the why part. It is important to have the right motivation for wanting to do this. Single Responsibility Principle and Seperation of Concerns are principles that have saved me time and time again. It has helped me decouple my code and get my applications cleaner. Responsibility separated out gives you a great opportunity to get to Lego pieces that you can stitch together and really get to a better DRY model. It also makes testing easier, testing interaction between systems and more focused and simpler tests. Also, separating out the responsibility produces in my opinion simpler code in the systems as well, simpler means easier to read. Sure, it leaves a trail of more files / types, but applying proper naming and good structure, it should be a problem – rather a bonus. It is very different from procedural code, you can’t read a system start to finish – but I would argue you probably don’t want to that, it is in my opinion practically impossible to keep an entire system these days in your head. Instead one should practice focusing on smaller bits, make them specialised; great at doing one thing. Its so much better being great at one thing than do a half good job at many things.

Code Quality

Time for a debt management plan?

Lately I’ve been getting negative feedback from talking about technical debt. I’ve been trying to go over it in my head why it would be considered negative, and what spawns the idea of it being a negative thing. I really can’t figure out why it would ever be considered bad to have technical debt, so instead of trying to figure out why people would consider it a bad thing, I’ll try to shed some light on what technical debt in reality is.

I’m going to start off by saying; technical debt is something that all projects have, no matter how recent the project was created. In fact, I would probably argue we put in technical debt on a daily basis on all software projects. You might be screaming WAT at this point, and you’re fighting the urge to completely disregard this post and move long, because you think I’m talking utter nonsense. But before you do that, I’d like to take the time to lay out what i think classified as technical debt.

// Todo

Ever sit in your code and you go up against something you really can’t fix, its too involved at the current point in time, or the architecture in its current state would not permit it, or plain and simple; you don’t have time – you’re having a Sprint demo in 1 hour. You end up putting a // Todo comment in the code, something to revisit at a later stage. You might not even know when. This falls into the technical debt category. One could even argue that any comment put into the code makes the code technical debt, the reason being – the code is too hard to understand on its own and needs a comment to explain what it is doing. I’ll leave the subject of general comments for now, as it should be subject of a different post.

Legacy code

Just about every system being built out there has some legacy they have to deal with, some data model that you can’t get rid of. The art of dealing with legacy code is then to try to come up with an anti corruption layer that will keep the legacy in one place. But even having this in place, sometimes things leak through. So things you really didn’t want in your new model might be leaking in, it could be anything from a simple property with the wrong name going through your new model to large concepts being misrepresented. All these can be categorised as technical debt.

Changing horses midstream

Working in software is in my opinion much like being on a high speed train that really don’t have a destination but have railway switches that makes the software change tracks. This can be new way of doing things that the team want to embrace. It could be new knowledge the team acquires that they didn’t have before that will make the software better moving forward. Everything being left behind at that point represent technical debt. They are things the team would ideally want to have fixed, but often in the interest of time, they can’t go and do it right now.

Complexity

Another warning sign that makes my bells go off is when something is hard to follow. Unless you’re sending spaceships to the moon or other planets, I have a hard time believing that software should be hard to do, understand or follow. Done right, and everything should be very simple. Large methods, large classes, things taking on more than one responsibility, all these are things leading to complexity in the software and making it hard for anyone else but the guy that originally wrote it to understand. In fact, I would even argue that the guy that originally wrote it would have a hard time going back into own complex code. This is typically a situation were the team would like to have it another way, and hence should be categorised as technical debt.

Bugs

So, what about bugs then. This is funnily enough something that does not fall into the category. Bugs are defects in the software, functionality that does not work as promised. Sure, it could be a ripple effect thats causing the bug, or clash of bugs, based off other parts of the system being technical debt. But then you should try to identify the real problem and fix that in isolation, not categorise the bug as technical debt.

Now what?

Don’t jump into panic just yet, just because you’re realising the project you’re on seems to suffer from one or more of the above mentioned issues. How do we deal with it? Well, first of all – be honest. When it comes to software development and life in general, I’ve had great success with just being honest. Be honest with yourself, be honest with your team, be honest with whoever is picking up the bill at the end of the day; you have technical debt. And its fine, its actually a good thing, it means the system is evolving, you as a developer is evolving, and probably for the better. Its called progress. You only need to be aware of the technical debt, write it down somewhere – NOT IN THE CODE – somewhere making it visible for the entire team. Writing it down and glancing at it during planning, having it in the back of every developers mind makes it so much easier to actually deal with the debt, pay back some of it while moving forward and creating new features or fixing bugs. How you register it and make it visible is not important, whatever works for your team. Personally I prefer simple ways of dealing with this. One project I was on, we started by just putting the technical debt in the form of post-its on a wall. This slowly progressed into becoming a Trello board that we had running on a 42″ screen all the time. By doing this we had the technical debt available when not sitting in the office, but by putting it on the 42″ screen displaying only that made it a focal point. Something the team was focused around, all the time; keeping an eye on the debt – slowly paying back.

Another aspect I find important, not only for technical debt but in general; don’t make your code personal. Its not a manifestation of yourself, it is code. The code gets committed into the repository, and it is no longer yours, but the entire teams. The idea that you will be maintaining it for ever is just silly, you might quit your job, or worse things could happen. The code is not yours in that sense, you wrote it, but detach from it!

If for any reason technical debt is a loaded term in your organisation, you might want to consider calling it something else. Don’t pretend you don’t have the elements mentioned in your solution just because it is a loaded term. Call it something like “wall of shame”, or anything that will work within your team and organisation.

Conclusion

Stop letting technical debt scare you, stop trying to avoid admitting to the fact there is technical debt. We all have it, we all contribute to having it, the only important thing is that it is on our agenda, we need a way to pay it back. Establish your own debt management plan, either in the form of work items in your work item tracking software, post-its on a wall or something that fits you. Get it out there, in the open, its not scary – in fact, I would argue the opposite is scary; not knowing what the debt level is.