Recently I had a chance to start from a scratch. A brand new internal project. Oh yeah! :)

Interested in the whole serie?
part 1
part 2
part 3
takeaways
So I decided to do my best while working on it. Time will tell if I succeed, but right now, I'm quite happy with what I'm doing. I've decided to describe how I approach this project, in order to:

• reflect on my decision,
• get some feedback (well, maybe?)
• share valuable hints (if any)

OK, so let us begin.

Unfortunately I can't give the details of the project, but I think I can share enough so this post is worth reading.

Thinking

First of all, I spent some time (like day or maybe more) mainly thinking about the task. Not that I sat down for X hours and scratched my head. In fact, I spent this time wandering around the office, and also drawing some diagrams and writing a very rough documentation (using Asciidoc of course!). The act of writing definitely helped me to clear my thoughts.

My mind was drifting from the 10000 feet view to the tiny details and back again. So one time I was looking at the modules, and after few minutes I was thnking about tiny unit tests for a specific component. But of course I tried to concentrate on the big picture leaving the details for now (not easy task, at least not easy for me).

Peers Review

Ok, so after this day I had a quite good idea of what I would like the system to look like. I draw a diagram of the planned architecture on a white board and asked my teammates to listen to my description, and to bash any stupidity they spot. And bashed they did, oh yes! :)

And I tell you, that this was a very, very fruitful 30 minutes. In general my plan got approved, but they challenged my ideas on so many levels, that now, it is much, much better. Of course, it appeared that I have left some very important cases, which in turn lead to some significant changes in the design. Oh my, how glad I am that it has happened before the first line of code was written!

So, that would be it, regarding the Big Upfront Design :) After this discussion with my colleagues I'm sure the design is good enough to start. So let's do some coding.

(picture from http://www.marksdailyapple.com)

On the picture you can see my colleagues tearing my initial plan to pieces. ;)

Greenfield is Not Entirely Green

I called this a "greenfield project", but soon I have realized it is more complicated than this. So basically, in the context of my team, and my company, we have plenty of stuff we commonly use. And it was obvious, that a lot of this stuff is so useful that I want to have it. It is various kind of stuff - from some Spring-related utils to some specific elements of web-apps which are used during the deployments (we deploy with our own tools). Another thing that was clear to me, is that it would be stupid not to reuse (meaning copy&paste) some configuration stuff and such from other projects. Really, what point in setting everything from scratch (meaning Spring stuff: transactions, properties configuration, view handlers and such).

And BTW, as Pablo Picasso said "Good Artists Copy; Great Artists Steal" :)

But Where To Start?

I have to admit I have committed the crime of starting with the database model. Yes, that was the first thing I did. It was a very, very simple model, but nevertheless, this is where I started. It took me some time before I mentally slapped myself, and dropped this task. No, database model is not important at this point.

(picture from http://malialitman.files.wordpress.com)

(This was really weird what I did. The thing is, that I do not even know what kind of database will fit - it might be NoSQL or SQL, so what the hell was I thinking?)

So what is important? The flow. The whole stack. The layers. The connections. The message going end-to-end. In short, I ought to write an end-to-end test!

(picture from http://www.gopixpic.com)

There and Back Again

So I started with tests. A very simple test. Its goal was to send a request to the system via its REST API (using rest-assured), and then check whether the message goes through the whole system to the very end. But to make such test pass, you need to prepare quite a lot of stuff. You need a web application, for example, and this means the web.xml file, some controllers, configuration of JSON to POJO mapping etc. Then you need lower layers, so your controller has someone to talk to and so on. Thanks to my diagrams and the discussion, I had quite a good idea of where I'm going, and how the parts of the system should look like.

I kept everything as simple as possible and spent few hours creating maven subprojects, taking care of the dependency issues (Maven Enforcer Plugin and Maver Versions Plugin to the rescue), creating Spring configuration files and so on. On the second thought, I might have postpone some of this work, but on the other hand, having separate subprojects allows me to keep things separated from the very beginning. In other case, it would be possible to make a mess at this very early stage of development.

Because the project acts as an intermediary between 3rd party systems, I had to create some fakes which could be verified. At the end I decided where my e2e tests will end - on the adapter layer. So what they check is whether an adapter for a specific system got the right message. Separate tests will check whether the adapter works well with the 3rd party system (using probably some sandbox environment). But this is yet to be decided. I will discuss it with our QA, and surely he will tell me something interesting.

(Almost) No Code, (Almost) No Tests

When I think about it now, there are few things worth noticing:

1. there is almost no code,
2. there are almost NO TESTS,
3. ...but the system looks like a system already!

No code. A lot of "glue" (Spring configuration, Maven subprojects and pom.xml files) but no real code. Not zero lines of code, but close to zero. Mostly interfaces. And no tests. At least no unit tests. Not very surprising given that there are only few fake classes pretending to be doing something (I decided to replace the database with simple classes which keep some data in memory using hash maps). And since I was prototyping, checking whether this interface fits there, or maybe it should look completely different, or maybe not interface, but a class, and so on, well, since I was playing with the code, I felt no urge to write tests. They would be useless at this point. They would only hinder my progress.

But I had my e2e test which guided me all the time. All I did was because of this test, to make it pass. Well, not all, because sometimes when I copied & pasted some config from another projects I also took things which I'm pretty sure I will need soon. Maybe I'm right, maybe I'm wrong, will see.

Stuck

At some point I got stuck. After the server was up and running I was sending some JSON data to it with curl. And somehow I couldn't make the simplest possible controller work. The JSON-to-POJO mapping was refusing to cooperate. I changed version of Jackson, browsed through Spring documentation, Stackoverflow and numerous blogs. All to no avail. Some stupid bug was still there.

(picture found on list25.com)

It took me an hour to realize the bug was in another place: the JSON data I tried to POST was invalid! :) One check with jsonviewer helped me to realize this. Well, that sucked. I took almost the simplest JSON possible and still I was able to make a mistake which hindered my progress for quite a long time. Eh.

The Plans

I'm quite happy with what I have achieved today. I can already see the system working so I know the design makes sense. It does not matter that some parts of it are fakes (for example there are some business rules involved, and now there is only a fake implementation of the "always-return-true" kind).

During the coding I have improved the naming. After I wrote some interfaces and looked at the way they interact with each other, I have realised that the names that I have chosen initially could be well improved.

At the current stage of development I call my system "a walking skeleton". I believe it is time to add some muscles. :) My plan for tomorrow is to discuss the integration/end-to-end tests with Dawid (our QA). This will allow me to write some more serious stuff.

I also wonder what happens if I write some unit tests for the database replacements (for the silly services which keep some date in memory using hash maps). How much work will be required later to use the same tests for real database implementation?

Stay tuned, I will continue this. ...unless my coding ends with some disaster so that I will be ashamed of blogging about it. ;)