AuthorDavide Guida

Testing the boundaries of your Web APIs

How do you make sure an entire software you wrote works? And how would you do that if your system doesn’t have a UI? Well, simply by testing the boundaries of course!

From time to time I like to extract pieces of code from what I’m working on and create small repos just to showcase a single functionality or idea. 

This time I’m putting some efforts on TDD on APIs and after few refactorings I came up with a nice structure that you can use as a starting skeleton for a simple system. You can find all the sources here on GitHub.

The demo is very simple, just a single controller that stores and provides user details. Nothing fancy. The user model class exposes only three properties: id, full name and email.

Few points worth noting though:

  • the class is immutable. I wrote a bit about the concept here.
  • I’m adopting the Special Case (or Null Object) pattern a lot these days. Hence the NullUser static property.

Persistence is done in-memory as it’s obviously outside the scope. Moreover, as you can see the Tests project contains only the end-to-end tests, no unit/integration test to cover the persistence layer.

The testing infrastructure is where things gets interesting, even though it’s actually fairly straightforward. An XUnit Fixture is firing up a TestServer and bootstrapping the application using (possibly) the same settings as the real system.

A shared WebHostBuilderFactory class is indeed responsible of building the required IWebHostBuilder instance.

That’s it!

Ok, just to be honest, I got the idea from Mark Seemann : he has a very interesting course on Pluralsight named “Outside-in TDD“. If you have the chance, I strongly suggest you to watch it.

So, now that we have our nice infrastructure ready, all we have to do is write our tests! Being this a Web API, these might be considered either “functional” or “end to end”.

Honestly I think it’s simply a naming thing and doesn’t change the fact that probably these should be the first tests you would write.

Why? Because (and Mark explains it really well in his course) you’re ensuring from the consumer’s perspective that your APIs do what they’re expected to do.

You’ll be “testing the boundaries”.

But most importantly, you’re validating your acceptance criteria and making sure your system works. 

Everything else is just an implementation detail.

So what are we testing here? The routes of course! Our API is managing users, and being it RESTful, we’re asserting that all the http verbs are doing what we expect to do. 

Most of these tests should derive directly from the acceptance criteria written by your Product Owner. In case you don’t have one but instead rely on some (even vague) specifications, a good starting point is simply testing inputs and outputs. 

Happy testing!

List of useful Docker commands

In the last few weeks I’ve been doing several experiments with Docker, just trying to grasp the main idea and maybe even come up with something useful.

As often happens with tools these days, there’s an entire world of command line tools that you should learn.

OR you can just be lazy as me and keep a list of the most common ones 🙂

I’ll be updating this list from time to time, just to keep track of what I’ve used so far and be a quick reference for my sloppy memory.

docker login [HOST] # in case you need auth to pull or push images
docker build -t [TAG]
docker images # shows a list of the local images along with the size
docker push [TAG]
docker system prune -a # this will remove ALL your images and containers. Very useful when you run low on space, but be careful!

The next command will display the Docker daemon logs, extremely useful when you don’t know what happened (basically me most of the time) and you’re looking for an answer, even if obscure.

It depends on the OS you’re running on:

  • Ubuntu (old using upstart ) – /var/log/upstart/docker.log
  • Ubuntu (new using systemd ) – sudo journalctl -fu docker.service
  • Boot2Docker – /var/log/docker.log
  • Debian GNU/Linux – /var/log/daemon.log
  • CentOS – /var/log/daemon.log | grep docker
  • CoreOS – journalctl -u docker.service
  • Fedora – journalctl -u docker.service
  • Red Hat Enterprise Linux Server – /var/log/messages | grep docker
  • OpenSuSE – journalctl -u docker.service
  • OSX – ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/log/d‌​ocker.log
  • Windows – Get-EventLog -LogName Application -Source Docker -After (Get-Date).AddMinutes(-5) | Sort-Object Time, as mentioned here.

(thanks, Scott)

Immutable Builder Pattern

This time we’ll talk about the Immutable Builder Pattern, but with a twist: the resulting instance has to be immutable.

From time to time I need to move away from the routine, just to avoid getting bored. Also, taking short breaks might help viewing things under a different perspective. Anyways, while working on Statifier I felt the need to get back to the roots and dive a little bit into the Design Patterns World ™ .

Straight from Wikipedia:

“the intent of the Builder design pattern is to separate the construction of a complex object from its representation”

This pattern is indeed extremely useful when you have to create an instance of a complex class, eg. with a long list of properties that have to be initialized.

It can be confused sometimes with the Factory pattern, but there’s a very important difference: Factories can be used to create an object instance based on some rules and return an interface. The result will be any concrete implementation of that interface, but consumers won’t (and don’t need to ) know which one.

Builders instead know how to create a single, specific class and the result is exactly an instance of that class.

And why should I be using the Builder if it can create just one thing? Well maybe because the creation is complex and requires several steps. Maybe you cannot just do something like builder.Build(param1, param2…..) , but you need a more structured approach.

The Wikipedia page has already some good examples of how the pattern can be implemented so I won’t talk much about that. 

But what is immutability anyway? Why do I need it?

At its core, immutability is a simple, very important concept: don’t let anyone from the outside update the internal state of your objects. That’s it.

Coding-wise, it translates basically into classes with no setters at all. All the properties will be initialized in the cTor, along with all the dependencies.

And why should I use it? Here’s an excellent article by Martin Fowler (yeah I love quoting him). Take your time, I’ll wait.

Ok so now let’s get back to out Builder. Here’s a sample implementation:

Few points to note:

  1. Vehicle cTor is private. Only the Builder can create an instance, and this is the main reason why the Builder class is declare inside the Vehicle class.
  2. all properties on Vehicle are get-only. Internal state cannot be changed, immutability is ensured. Good luck changing them.
  3. every call to Build() creates a new instance of Vehicle. This is again made to ensure immutability but comes with a twist: you have to be more careful if you want to reuse an instance of the builder to create more vehicles. You’ve been warned.

A static website is better.

Yes, you read that right: a static website is better.

Now take a deep breath and follow me. To read this post you’ve politely asked your browser to fetch data from a url. This has triggered a chain of servers up to the one hosting this blog which has identified the right php script to execute and returned a somewhat well-formatted html string in output. At this point your nice browser can start rendering the page, while at the same time performing some other requests to download images, styles, scripts and so on.

Phew.

That’s a lot of steps.

Some of them cannot be avoided (you have to download something sooner or later), but for sure we can try to make the life of our servers easier by reducing the workload. Maybe by optimizing the database calls. Or maybe by simply serving static html files and nothing else.

At this point some of you might argue that we’re not in the early ’90s. I agree, even though I really love grunge music. 

But…think of those websites that have little to none dynamic content. Maybe not even a contact form. Something very similar to this blog. What’s the ultimate way to speedup a website like this?

I can see some of you mouthing “use a cache”. Yes, I can use a cache. Of course I do. But caching is difficult.

Also, what if I want to completely avoid exposing my admin panel url? I’m a paranoid maniac, absolutely afraid of hackers and don’t want to be pwned. And I’m also too lazy to do nasty tricks like ip whitelisting or VPNs.

So what’s the solution? I said it already: use plain old html files. Make everything static and easier.

Ok, that won’t be a silver bullet but bear with me: I’m talking about a very specific niche of websites, with content that doesn’t change by user and with little to no forms. These should be static already but you know better than me that installing WordPress is incredibly easy and cheap.

So this is why I’ve started working on Statifier! For now it is a simple NodeJs tool that will crawl your website, download pages and assets and perform some very simple string replacements. Nothing else.

But it works. With some hiccups, but it works. Still doubting? Take a look here, you won’t be disappointed.

Yes, I could have used an existing tool like Grav or even better, Jekyll, but why give up on the amazing WP admin panel?

 

Unit, integration, end-to-end tests: do I need all of them?

Yes. I mean, don’t even think about it. You’ll need all of them, probably in different measures, but there is no “we shipped to production without tests”.

Tests are the first rampart separating you from madness and failure.
Why madness? Try to do even a small refactoring after you’ve deployed your app. Without automatic tests you’ll have to manually probe the entire system (or systems if you’re on microservices).

Why failure ? Simple, just think on the long run. Maintenance will quickly become a hell and adding new features will soon bring you to the infamous “it’s better if we re-build this from scratch”.

So! Where should we start? From the pyramid!

the test pyramid

The test pyramid. Image taken directly from Martin Fowler’s article. Thanks, Martin.

Starting from the bottom, you’ll begin with writing the unit tests. “Unit” here means that you’re testing a single small atomic piece of your system, a class, a function, whatever. You won’t connect to any external resource (eg. database, remote services) and you’ll be mocking all the dependencies. 
So, ideally you’ll be checking that under specific circumstances a method is throwing an exception or the cTor is populating the class properties or the result of a computation is a specific value giving a controlled input.
Also, unit tests have to be extremely fast, in the order of milliseconds, giving you a very quick and generic feedback of your system.

Next is the “Service” layer or, more commonly, “Integration”. This is where things start to get interesting. Integration tests check that two or more pieces fit correctly and the cogs are oiled and greased.  So stuff like your Persistence layer, access to the database, ability to create or update data and so on. They might take more time than  unit tests and probably will be in a lesser number, but their value is extremely high.

Then we have the “UI” or “end-to-end” tests. Here we’re making sure that the whole system is working, inspecting from the outside, with little to none knowledge of the inner mechanism. You’ll be checking that your API routes are returning the right HTTP statuses, setting the proper headers and eating the right content types.

In the end it’s all a matter of perception. The point of view is moving from the inside of the system, the developer perspective, to the outside: the consumer perspective.

There are of course other typologies of tests, acceptance, smoke, functional and so on. But if you begin adding the coverage using this pyramid you’ll save an awful lot of headaches and keep your system maintainable and expandable.

© 2018 Davide Guida

Theme by Anders NorenUp ↑