Testing Basics: SUT and DOCs

This blog post is devoted to two unsung heroes of testing: SUT and DOC. Who are they and what is their role in unit, integration and end-to-end tests?

This topic is described in more details in my book:
"Practical Unit Testing
with TestNG and Mockito"

Definition

By SUT, or System Under Test, we understand an entity that is being tested. Depending on types of tests SUT might be of very different granularity - from single class to the whole application.

BTW. SUT acronym is used in few variants: first word is expanded as "system", "structure" or "subject".

A DOC, or Depended On Component, is a collaborator, any entity that is required by SUT to fulfill its duties. Usually DOC is of same granularity as SUT, e.g. if SUT is a class, then it uses other classes, if it is a module, then it depends on other modules, etc.

Examples

The following table presents few examples of SUT and DOCs for different types of tests.

type of tests SUT example DOC example
unit tests UserService UserDAO
MailService
Invoice Product
Client Account
integration tests DAO Layer (ORM-based) Hibernate
DAO Layer (JDBC-based) MySQL
Search module FileStorage module
end-to-end tests Whole application External web service(s)
LDAP repository

As stated previously, SUT and DOCs are usually of same granularity. You can also see, that one SUT might need more than one DOC to work. For example UserService uses UserDAO to save user data and MailService to send emails (that his data were updated).

SUT is Real, DOC Might Be Or Might Be Not

SUT is the main hero. This is what we test, so SUT is real, it is really created and exercised.
DOC might be real or might be not. It might be substituted by test double (of any kind - mock, stub, fake, test-spy - depending on what we test).

SUT Is Alone, DOC Might Be Or Might Be Not

There is only one SUT in your test, because you want to test exactly one class, module or application. It can use one or more DOCs. Number of DOCs used is an important indicator of SUT's health - if there are too many of DOCs, then something is wrong with this SUT (it probably has too much responsibility).

Ok, I admit, I could bring some examples of testing few classes at once (so there is more than one SUT), but still the general rule is, that there is only one. Howgh.

How SUT And DOCs Come To Life

Before testing, one have to put "everyting" in certain state. This is the idea of text fixture, which refers to setting of
well known and fixed environment in which tests are run so that results are repeatable (http://en.wikipedia.org/wiki/Test_fixture)

Unit Tests

In unit tests you use various setUp methods (nowadays testing frameworks use annotations for this) to create all objects in the appropriate state. Usually you create DOCs first (or test doubles of DOCs), then the SUT (because SUT might need some DOCs to be passed to its constructor). Creation of SUT is usually a matter of calling its constructor (or some factory method). Creation of DOCs often involves use of some mocking framework.

The question "to mock DOCs or not to mock them" arises here. In general, you should mock them (so SUT is tested in isolation), but be pragmatic about it. I will not discuss it further, cause that would make this blog post 20 times larger than I intend it to be. All I tell you is:

Integration Tests

In integration tests creation of both SUT and DOCs will be probably taken care of by some framework you use (e.g. Spring or Guice), but they can also be brought to live in similar manner as described for unit tests. In integration tests some DOCs come in form of "external" entities, i.e. database, and setting them up (and tearing down) might be quite complicated. Databases can be set up in your code but also by build tool (e.g. some Maven plugins can help you with this task). Yet another vast topic that is only mentioned here.

Can you use test doubles in integration tests? Sure you can, as long as you know what you do. You test integration of things here. So if you check if integration of X with A works fine, you do not need to worry about integration of X with B, so B can be replaced with some test double. But, because creation of both SUT and DOCs is costly (comparing to unit tests) and done by some framework, and dependencies of tests are quite common, you will probably create all DOCs real and use them in many tests. So, answering the question: yes, you can use test doubles here, but you rarely will. Howgh. Yes I know, this is a simplification.

End-To-End Tests

Because there are so many types of applications, it is very hard to generalize about end-to-end tests. Your application (your SUT) might come in form of a simple war file deployed to Tomcat, it can be distributed as a virtual machine image, or it can be also a huge system deployed to many nodes (clouds?) etc., etc. Limitless possibilities.
And what are DOCs in end-to-end tests? Usually this are some external systems that your application communicates with (e.g. via web services). You are lucky if you can use the real systems in your end-to-end tests, but if not, you will have to fake them.

Credits

Thanks to Gerard Meszaros for introducing a common testing vocabulary. The success is only partial, because number of people who say e.g. "mock" and know exactly what are they talking about is still too low, but at least some standard understanding of basic testing terms seems to be out there in the community. Thanks Gerard!

Links

  • DOC definition at xunitpatterns
  • SUT definition at xunitpatterns
Please comment using