Software Architecture for Developers in Chinese

Although it's been on sale in China for a few months, my copies of the Chinese translation of my Software Architecture for Developers book have arrived. :-)

Software Architecture for Developers

I can't read it, but seeing my C4 diagrams in Chinese is fun! Stay tuned for more translations.

C4 stencil for OmniGraffle

If you like the look and feel of the C4 software architecture diagrams in my Software Architecture for Developers book (see examples here), Dennis Laumen has created an OmniGraffle stencil that will save you some time. Just download the stencil, install and it will appear in your stencil library.

The C4 stencil is available from Omni Group's Stenciltown. Thanks Dennis!

Software architecture workshops in Australia during August

This is a quick update on my upcoming trip to Australia ... in conjunction with the lovely folks behind the YOW! conference, we've scheduled two public software architecture sketching workshops as follows.

This workshop addresses one of the major problems I've evidenced during my career in software development; namely that people find it really hard to describe, communicate and visualise the design of their software. Sure, we have UML, but very few people use it and instead resort to an ad hoc "boxes and lines" notation. This is fine, but the resulting diagrams (as illustrated below) need to make sense and they rarely do in my experience.

Some typical software architecture sketches

My workshop explores this problem from a number of different perspectives, not least by giving you an opportunity to practice these skills yourself. My Voxxed article titled Simple Sketches for Diagramming Your Software Architecture provides an introduction to this topic and the C4 model that I use. I've been running this workshop in various formats for nearly 10 years now and the techniques we'll cover have proven invaluable for software development teams around the world in the following situations:

  • Doing (just enough) up front design on whiteboards.
  • Communicating software design retrospectively before architecture refactoring.
  • Explaining how the software works when somebody new joins the team.
  • Documenting/recording the software architecture in something like a software architecture document or software guidebook.
"Really surprising training! I expected some typical spoken training about someones revolutionary method and I found a hands-on training that used the "do-yourself-and-fail" approach to learn the lesson, and taught us a really valuable, reasonable and simple method as an approach to start the architecture of a system. Highly recommended!" (an attendee from the software architecture sketching workshop at GOTO Amsterdam 2014)

Attendees will also receive a copy of my Software Architecture for Developers ebook and a year subscription to Structurizr. Please do feel free to e-mail me with any questions. See you in Australia!

Diff'ing software architecture diagrams again

In Diff'ing software architecture diagrams, I showed that creating a software architecture model with a textual format provides you with the ability to version control and diff different versions of the model. As a follow-up, somebody asked me whether Structurizr provides a way to recreate what Robert Annett originally posted in Diagrams for System Evolution. In other words, can the colours of the lines be changed? As you can see from the images below, the answer is yes.

Before Before

To do this, you simply add some tags to the relationships and add the appropriate styles to the view configuration. structurizr.com will even auto-generate a key for you.

Diagram key

And yes, you can do the same with elements too. As this illustrates, the choice of approach is yours.

Diff'ing software architecture diagrams

Robert Annett wrote a post titled Diagrams for System Evolution where he describes a simple approach to showing how to visually describe changes to a software architecture. In essence, in order to show how a system is to change, he'll draw different versions of the same diagram and use colour-coding to highlight the elements/relationships that will be added, removed or modified.

I've typically used a similar approach for describing as-is and to-be architectures in the past too. It's a technique that works well. Although you can version control diagrams, it's still tricky to diff them using a tool. One solution that addresses this problem is to not create diagrams, but instead create a textual description of your software architecture model that is then subsequently rendered with some tooling. You could do this with an architecture description language (such as Darwin) although I would much rather use my regular programming language instead.

Creating a software architecture model as code

This is exactly what Structurizr is designed to do. I've recreated Robert's diagrams with Structurizr as follows.

Before “After"

Before “After"

And since the diagrams were created by a model described as Java code, that description can be diff'ed using your regular toolchain.

The diff between the two model versions

Code provides opportunities

This perhaps isn't as obvious as Robert's visual approach, and I would likely still highlight the actual differences on diagrams using notation as Robert did too. Creating a textual description of a software architecture model does provide some interesting opportunities though.

Unit and integration are ambiguous names for tests

I've blogged about architecturally-aligned testing before, which essentially says that our automated tests should be aligned with the constructs that we use to describe a software system. I want to expand upon this topic and clarify a few things, but first...

Why?

In a nutshell, I think the terms "unit test" and "integration test" are ambiguous and open to interpretation. We tend to all have our own definition, but I've seen those definitions vary wildly, which again highlights that our industry often lacks a shared vocabulary. What is a "unit"? How big is a "unit"? And what does an "integration test" test the integration of? Are we integrating components? Or are we integrating our system with the database? The Wikipedia page for unit testing talks about individual classes or methods ... and that's good, so why don't we use a much more precise terminology instead that explicitly specifies the scope of the test?

Align the names of the tests with the things they are testing

I won't pretend to have all of the answers here, but aligning the names of the tests with the things they are testing seems to be a good starting point. And if you have a nicely defined way of describing your software system, the names for those tests fall into place really easily. Again, this comes back to creating that shared vocabulary - a ubiquitous language for describing the structures at different levels of abstraction within a software system.

Here are some example test definitions for a software system written in Java or C#, based upon my C4 model (a software system is made up of containers, containers contain components, and components are made up of classes).

Name What? Test scope? Mocking, stubbing, etc? Lines of production code tested per test case? Speed
Class tests Tests focused on individual classes, often by mocking out dependencies. These are typically referred to as “unit” tests. A single class - one method at a time, with multiple test cases to test each path through the method if needed. Yes; classes that represent service providers (time, random numbers, key generators, non-deterministic behaviour, etc), interactions with “external” services via inter-process communication (e.g. databases, messaging systems, file systems, other infrastructure services, etc ... the adapters in a “ports and adapters” architecture). Tens Very fast running; test cases typically execute against resources in memory.
Component tests Tests focused on components/services through their public interface. These are typically referred to as “integration” tests and include interactions with “external” services (e.g. databases, file systems, etc). See also ComponentTest on Martin Fowler's bliki. A component or service - one operation at a time, with multiple test cases to test each path through the operation if needed. Yes; other components. Hundreds Slightly slower; component tests may incur network hops to span containers (e.g. JVM to database).
System tests UI, API, functional and acceptance tests (“end-to-end” tests; from the outside-in). See also BroadStackTest on Martin Fowler's bliki. A single scenario, use case, user story, feature, etc across a whole software system. Not usually, although perhaps other systems if necessary. Thousands Slower still; a single test case may require an execution path through multiple containers (e.g. web browser to web server to database).

Thoughts?

This definition of the tests doesn't say anything about TDD, when you write your tests and how many tests you should write. That's all interesting, but it's independent to the topic we're discussing here. So then, do *you* have a good clear definition of what "unit testing" means? Do your colleagues at work agree? Can you say the same for "integration testing"? What do you think about aligning the names of the tests with the things they are testing? Thoughts?

Software Engineering Radio - Episode 228

Software architecture sketches with Simon Brown

A quick note to say that my recent interview/chat with Sven Johann for Software Engineering Radio has now been published. As you might expect, it covers software architecture, diagramming, the use of UML, my C4 model, doing "just enough" and why the code should (but doesn't often) reflect the software architecture. You can listen to the podcast here. Enjoy!

Diagrams for System Evolution

How to show modifications to system architectures?

Every system has an architecture (whether deliberately designed or not) and most systems change and evolve during their existence - which includes their architecture. Sometimes this happens gradually and sometimes in large jumps. Let's have a look at a simple example.

Basic context diagram for a furniture company


Before


The main elements are:

  • Sales Orderbook where sales staff book orders and designers occasionally view sales.
  • Product Catalogue where designers record the details of new products.
  • Warehouse Management application that takes orders and either delivers to customers from stock or forward orders to manufacturing.


The data flow is generally one way, with each system pushing out data with little feedback, due to an old batch-style architecture. For example, the designer updates product information but all the updates for the day are pushed downstream at end-of-day. The Warehouse application only accepts a feed from the Sales Orderbook, which includes both orders and product details.


There are multiple issues with a system like this:

  1. Modification to the products (new colours, sizes, materials etc) are not delivered to the manufacturer until after an order is placed. This means a long lead time for manufacturing and an unhappy customer.
  2. The sales orderbook receives a subset of the product information when updated. This means the information could become out-of-sync and not all of it is available.
  3. The warehouse system also relies on the sales orderbook for product information. This may be out of date or inaccurate.
  4. The designer does not have easy-to-access information about how product changes affect sales.


The users are unhappy with the system and want the processes to change as follows:

  1. The product designer wishes to see historic order information alongside the products in the catalog. Therefore the catalog should be able to retrieve data from the Sales Orderbook when required.
  2. Don't batch push products to the Orderbook and warehouse management. They should access the product catalog as a service when required.


System redesign

Therefore the system is re-architected as follows:


“after"


This is a significant modification to how the systems works:

  • The Sales Orderbook and Product Catalogue have a bi-directional relationship and request information from each other.
  • Only orders are sent from the Sales Orderbook to the Product Catalogue.
  • There is now a direct connection between the Warehouse Manager and the Product Catalogue.
  • The product catalogue now provides all features the Designer requires. They do not need to access any other interfaces into the system.

However, if you place the two diagrams next to each other they look similar and the scale of the change is not obvious.

Before “After"

When we write code, we would use a ‘diff’ tool to show us the modifications.


Before


I often do something similar on my architecture diagrams. For example:


System before modification:

Before


System after modification:

“After"


When placed side-by-side the differences are much more obvious:

Before “After"


I’ve used similar colours to those used by popular diff tools; with red for a removed relationship and green for a new relationship (I would use the same colours if components were added or removed). I’ve used blue to indicate where a relationship has changed significantly. Note that I’ve included a Key on these diagrams.

You may be thinking; why have I coloured before and after diagrams rather than having a single, coloured diagram? A single diagram can work if you are only adding and removing elements but if they are being modified it becomes very cluttered. As an example, consider the relationship between the Sales Orderbook and the Product Catalogue. Would the description on a combined diagram be "product updates", "exchange sales and product information", or both? How would I show which was before and after? Lastly, what would I do with the arrow on the connection? Colourise it differently from the line?

Conclusion

I find that having separate, coloured diagrams for the current and desired system architectures allows the changes to be seen much more clearly. Having spoken to several software architects, it appears that many use similar techniques, however there appears to be no standard way to do this. Therefore we should make sure that we follow standard advice such as using keys and labelling all elements and connections.

Extracting software architecture from code

I ran my Extracting software architecture from code session at Skills Matter this evening and it was a lot of fun. The basic premise of the short workshop part was simple; "here's some code, now draw some software architecture diagrams to describe it". Some people did this individually and others worked in groups. It sounds easy, but you can see for yourself what happened.

Diagrams

There are certainly some common themes, but each diagram is different. Also, people's perception of what architectural information can be extracted from the code differed slightly too, but more on that topic another day. If you want to have a go yourself, the codebase I used was a slightly cutdown version* of the Spring PetClinic web application. The presentation parts of the session were videoed and I'm creating a 1-day version of this workshop that I'll be running at a conference or two in the Autumn.

This again raises some basic questions about the software development industry though. Why, in 2015, do we still not have a consistent way to do this? Why don't we have a shared vocabulary? When will we be able to genuinely call ourselves "software engineers"?

* The original version ships with three different database profile implementations, and I removed two of them for simplicity.

Diagramming microservices with the C4 model

Here's a question I'm being asked more and more ... how do you diagram a microservices architecture with the C4 software architecture model?

It's actually quite straightforward providing that you have a defined view of what a microservice is. If a typical modular monolithic application is a container with a number of components running inside it, a microservice is simply a container with a much smaller number of components running inside it. The actual number of components will depend on your implementation strategy. It could range from the very simple (i.e. one, where a microservice is a container with a single component running inside) through to something like a mini-layered or hexagonal architecture. And by "container", I mean anything ranging from a traditional web server (e.g. Apache Tomcat, IIS, etc) through to something like a self-contained Spring Boot or Dropwizard application. In concrete terms then...

  • System context diagram: No changes ... you're still building a system with users (people) and other system dependencies.
  • Containers diagram: If each of your microservices can be deployed individually, then that should be reflected on the containers diagram. In other words, each microservice is represented by a separate container.
  • Component diagrams: Depending on the complexity of your microservices, I would question whether drawing a component diagram for every microservice is worth the effort. Of course, if each microservice is a mini-layered or hexagonal architecture then perhaps there's some value. I would certainly consider using something like Structurizr for doing this automatically from the code though.

So there you go, that's how I would approach diagramming a microservices architecture with the C4 model.