Developers' Testing Dilemma: Aim for Verification or Design?
Few notes on two approaches to testing. Some people seem to be more interested in verification, others aim for better design.
This topic is described in more details in my book:
"Practical Unit Testing
with TestNG and Mockito"
The continuum of testing approaches lies between two contrary beliefs. I will introduce both extremities to make the distinction clear.
Verificators
Some people (I will call them verificators for convenience) want to check that their code works. That is their goal - to make sure it does what it should. They will sacrifice some OO rules if they believe that is what they need to do to achieve their Holy Grail of testability. This approach is well illustrated by this excerpt from a blog post that I found long ago somewhere in the Net:
In case of a code that is hard to test, they will resort to any available techniques to be able to test it. They will modify methods visibility using reflection or use classloading hacks to deal with final classes. This way they are able to test just about everything, including tons of nightmarish legacy code.
Designers
The other group - let us call them designers - believe that following OO rules is the most important and that it leads to easily testable code. They treat tests as an indicator of code health. Easy to write tests denote sound code. Hardship during tests writing indicates problems within code and means that the code should be reworked. They tend to write tests using same techniques as they use for production code, and renounce use of reflection or classloading hacks. Designers especially like TDD approach which guarantees a certain level of code quality. In case of legacy code they will tend to refactor (or rather rewrite) it, in order to make it more testable.
Tools
This distinction is also visible in available testing tools. Some of them (e.g. JMockit and Powermock) are there to test the untestable by giving you power to mock static classes, final classes and constructors or call private methods.
Other avoid using any kind of such hacks. For example JUnit has never introduced any feature that would make testing of private methods easier, even thought such request were mady by many since JUnit’s early days. Another example could be JMock, which offers syntax that in general is regarded as "not so intuitive" (compared to what EasyMock and Mockito offer). They do it on purpose, because they believe that this syntax expresses well important concepts of mocking, and does not allow you to do nasty things easily.
Conclusion
Obviously the conflict between these two approaches will never be solved. The proponents hold different views, have different needs and value different things. Both of them have also some good examples that can "prove" their superiority.
I position myself close to designers, sharing their care of good design, and believing that good code is easily testable. But in the past, I used to be more of a verficator.
...and by the way, what about you?
This used to be my blog. I moved to http://tomek.kaczanowscy.pl long time ago.
Difficult to solve this confit
This problem also exists in computer system design and verification. The source code is difficult to design when you achieve good performance, as well as verification.
- scholarships for african students
"Verificator" x "Designer"
I also position myself as a "designer".
For example, lets say I am TDD-ing a new business service class which has a requirement of sending a notification e-mail. Lets also say that a nice "Email" class is already available (fully developed and tested) in the project, and that I can simply use it in the new business service class ("Email" takes input through constructors and/or setters, and has a "send()" method - ie, as easy and simple as possible). In the unit test(s) for the business service, I obviously would not want to actually send e-mails, but rather to verify that they get sent through this "Email" collaborator. Therefore, in such tests the Email class gets mocked, with a verification at the end that "send()" was called.
Note that the use of "Email" in the business service class is an internal detail which should not be exposed to its clients in production code. An unit test, however, is a white-box test.
Note also that, by following the TDD process, I would write the simplest code possible that can pass all tests. That is, at the point where the notification e-mail needs to be sent, I would simply instantiate "Email" with the correct data, and call "send()".
Now, the problem is that not all mocking tools support such applications of the TDD process and of good OO design. From what you wrote, it seems that we have different understandings of TDD and/or OO design. Thoughts?
- Rogério (JMockit's developer)
Email knows too much
Hi Rogerio,
first of all, my apologies for taking so long to respond!
Before I answer, let me share a general comment.
I do not believe any of us is 100% a "verificator" and "designer". All the devs I know (me included) are somewhere between these two extremes. Being pragmatic is what probably counts in every case. I can only guess, that we both agree on that, and that for both of us "being pragmatic" means different things. :)
Now, I have to admit that I do not like the example you gave. :) The idea of having Email class which represents an email (recipient address, title, body, attachments) and also knows about how to send itself (SMTP stuff etc) is something I would not code myself. From the description of the class it is clear to me that its responsibility is too big - the word "and" in the description ("represents and email (...) AND also knows xyz") tells me that it should be split into two.
So starting with your example, one may come to the conclusions as you did, but for me it only proves, that one bad design decision leads to another. If you had a service (a separate class) for sending Email entities, than you could model your class much better, and there would be no issue at all.
BTW. I wonder how you go with writing a test for such code (with Email class as you described). Do you simply use JMockit and mock "new" constructor without any hesitation? For me this is a point, where I would notice, that something is wrong with my design. What I follow, is a simple heuristic of "if it hard to write a test, then there is something wrong with the design". For me, the need to use JMockit, PowerMock or any similar tool, is the sign that there is probably some bad design there. I wonder what is your heuristic for this.
Thanks for sharing your thoughts, and awaiting your comment!
--
Regards,
Tomek Kaczanowski
Re: Email knows too much
Hello again,
Thanks for answering! (I understand the delay, sometimes I also take some time to respond to JMockit users - too many things to do, so little time.. ;)
The example of an Email class is a common one, when services operate on some specific state and don't need to have an implementation selected at runtime based on external configuration. I like it because it is a "pure OO" design (not that I am an OO purist, but anyway), *and* also a very pragmatic solution.
The Apache Commons Email API is just like that, except that "Email" is a base class (usually, the programmer would actually instantiate a "SimpleEmail" subclass). The Spring framework, on the other hand, divides it in a "data-holder" class (SimpleMailMessage) and a "service" interface (MailSender) which gets injected. I don't like solutions like these because a) it breaks a natural object (the email message to be sent) which has state (the message, the address, etc.) *and* behavior (the "send" operation) in a "dumb" value object plus a stateless object. That is, the Spring design is a procedural, non-OO one, where the only "benefit" is that the service object can be injected by the DI container. Arguably, it's also less pragmatic, since it depends on the injection framework.
Now, about the writing of the unit test where "Email" needs to be mocked, it's not the case that you would need to worry about mocking a constructor or not. You simply mock the class, automatically affecting all of its instances (including *future* instances, if any). The test wouldn't really be different if the email object was injected instead (JMockit explicitly supports DI, in several ways). It's just as easy to write such a unit test for a client that uses the Commons Email *or* the Spring API; if you put both tests side by side, there would hardly be a difference.
I agree with your heuristic of "if it's hard to write a test, then there is something wrong with the design". JMockit is not some tool that you "need to use" because of some characteristic in the code to be tested, but rather a tool to make the creation of unit tests as easy and simple as it should be, *regardless* of the design choices the developer likes to make in the tested code. For example, if I like to follow the common advice (several books...) of "code for inheritance or prohibit it" (see http://stackoverflow.com/questions/218744/good-reasons-to-prohibit-inher..., for one), then being able to mock final classes and methods is a must.
Regards,
Rogerio Liesenfeld
JUnit is my favorite
I still love JUnit and treat it as the best unit testing tools. It's easy and elegant. For some code health tools which don't read the code logic just detect the function calling and class relationship, I don't think they are very useful. - ibs symptoms