Sunday 28 February 2010

Fooling the ODR (and the compiler)

I just discovered a technique that is prety convenient but rather not self-explaining, nonportable (?), and possibly a maintenance nightmare. So you should never use it, should you? Except, as it's sort of cool, if you'd like to go a little on the geeky side...

1. The need for speed

Imagine you are writing some quick classes in the Java style (i.e. no header files), locate them in some C++ source file, and forget about them. Then you discover that you need to refactor your program (in my case it was a custom test framework) and to move some classes out to a separate file (a.k.a. compilation unit). Now you shun to do the right thing (i.e. to refactor) because you'd have to cut up your classes in two: the .h file and the .cc file, and to duplicate some (non always trivial) amount of code. So yo aren't doing that...

But stop, following trick did it for me: I defined the classes in a separate .h file and kept the .cc file with the extracted classes unchanged. I included the new .h file in the rest of the old .cc file where the classes were removed from. Then I just linked the the whole thing, et voilá, everything worked like a breeze - a refactoring as we like it!

2. Why does it work?

But wait, we spared some editing work, but didn't we violate the One Definition Rule of C++? I'd say we did. So why didn't the compiler complain? Well, as to answer it, we've got to look at the example a little closer.

The extracted classes looked like that:

TestThreads.cpp:

  #include "WorkerThread.hpp"

// test thread A
class ThreadA : public WorkerThread
{
public:
...
private:
...
} g_workerA;
  // test thread B
class ThreadB : public WorkerThread
{
public:
...
private:
...
} g_workerB;
Note that there's no TestThreads.hpp definition file here, thus we don't have a direct ODR violation for the compiler to complain! The newly created definitions file TestThreads.hpp looked like this:
  #include "WorkerThread.hpp"
  class ThreadA : public WorkerThread
{
public:
virtual void addRequest(Request& r);
...
};
  class ThreadB : public WorkerThread
{
public:
virtual void addRequest(Request& r);
...
};
The WorkerThread.hpp definition file looks like this:
  class WorkerThread
{
public:
virtual void addRequest(Request& r);
...
};
And the rest of the TestMain.cpp file where the TestThread classes were previously placed:
  #include "TestThreads.hpp"
  extern TestThreadA g_workerA;

extern TestThreadB g_workerB;
  g_workerA.addRequest(req);
Can you see why we could fool the ODR? Well, in this special case we didn't used the constructors directly in the TestMain.cpp file, only the methods of the WorkerThread superclass which then call the appropriate virtuall methods defined in the ThreadA class.
The linker then generates a reference to the virtual ThreadA::addRequest() method. Well, guess what, a method with the very name can be found in TestThreads.cpp, and the linker can resolve it no probs!

Although we clearly have two definitions of the ThreadA class, we get away with it! The ultimate reason for that is that the C++ compiler doesn't have the global knowledge of the whole program - it works only on the compilation unit level, and the relinquishes the responsibility for the complete programm to the linker! I.e. there's no whole programm optimization possible (at least not in the standard model, although Visual C++ compiler can do it).

3. How to extend the "hidden" classes?

Suppose we'd like to call a new method in the TestThread.cpp file from the outside. You cannot call this method directly, as we do not include the real definitions of the TestThread classes, but only the "fake" ones. Alas, there is a price to pay for fooling the compiler and for beeing lazy! The price here is, that we have to modify the base class (TestWorker.hpp) by adding a new public virtual method, if we need a new TestThreadA method to be called in the TestMain.cpp file!

Not pretty, uh? You know, sometimes you can go away with breaking the rules, but in the end you'll have to face the consequences. In the art of software design, your skills are measured at your ability to estimate for how long you will go away with laziness and tricks, and when you have to resort to the known, sound and boring "best practices".

But nonetheless, the price I had to pay was OK for the deeper knowledge of the compiler-linker cooperation and the increased knowlede of my own code.

Sunday 14 February 2010

Do you GoF - the belated anniversary and the ladder

Last year we had the 15th (or so?) anniversary of the seminal "Gang of Four" book. Is it that old already? So maybe discussing the design patterns is all an old and boring stuff now? Well, shame on me, I wanted to write up my ideas on that matter for such a long time that I run the risk of them beeing irrelevant. But on the other side it's a kind of unfinished business for me, and I have a to kind of close it up.

Some time ago I wondered - why didn't I use the design patterns that much in my programming and design work? I came to the conclusion then, that the design patterns are useful as a common vocabulary as to speak about designs, but then some new ideas surfaced in my mind. By the way, why should I use a pattern vocabulary if I don't use any patterns in my designs?

1. Ideas from chess and math

If you're playing chess (need a Wikipedia link here? ;-) you are accustomed to seeing the board in patterns: check-mate patterns it is! You try to imagine what kind of a check-mate setup could be possible in a given situation, and then work towards that by moving your pieces and trying to chase away your opponent's pieces as to complete the pattern you have in your mind. I must admit that I never had this feeling when writing software. Some pattern in my mind which is the solution for some design problem? Nope.

Read the "Refactoring to Patterns" book by Joshua Kerievsky? When I first heard of the book I though he struck the right idea: only use patterns to refactor code which is a mess. But then when actually reading the book, it was rather a feeling (in most cases) of doing something wrong to the code, well, like overengineering it! So the book didn't reconcile me with patterns too, as it didn't give me the chess-like feeling of a pattern just matching a given situation.

The other problem with patterns is their presentation. This overly formal template with "Forces" (why not just problems?), "Intent", "Consequences", etc, etc... This makes me think about "The mathematicians lament"* - where a matematician reproaches his fellow scientists of over-formalization of mathematics with the effect of its incomprehensibility. What he is saying is basically the following: yes we need the formalism, but only in some very difficult cases where our intuitions don't work. But not in every and one case: "nobody's dying here!"*, you can relax a bit!

It's the same with patterns: the stern formality of presentation is both somehow comical and making the contents rather indigestible, and an dry, uninspiring read. I can only say, relax, we are not deliberating about the meaninig of infinity like Cantor did! We are just describing some not so complicated techniques of object composition! Could you stop to be so pompous?

2. Dreyfus et al.

Do you know the Dreyfus model of knowledge acquisition**? It defines several phases in learning a specific skill: beginner, advanced beginner, competent, proficient and expert. Beginners need rules and recipies, competents can continue learning on their own, proficients and experts see the big picture and fly by the seat of his pants.


Where do the design patterns fit in this scheme? Well, at the beginner level they aren't useful, as they are not "step by step" recipies. On the other side, for the proficients and experts, they are some kind of rules, and experts do not like rules and recipies by definition!

I personally think that patterns appeal rather to the lower ranges of the skill spectrum. Did you see the hopelessly overengineered software full of design patterns written by the beginners? I've seen it only too often!

So maybe they are for the competent? And the usage by competent and proficient users? Let me tell you a little story here. It's not entirely true that I don't use patterns! Lately, I thought that I could warrant the use of the visitor pattern in my current product, as to extract the reporting aspect out of several content managing classes. Well, it has a kind of worked at first, but now, as this whole aspect has to be redesigned, I noticed that this design is rather unintuitive and I cannot simplify the reporting without dumping visitor first. Ok, experiment failed, but my initial problem was that I didn't really search for a solution of my problem, but rather stopped at the point where I found the (as I thought then) matching design pattern! But I didn't think it trough as thoroughly as I should!

3. Summing up

So maybe the thing with patterns is that they are like the "Wittgenstein's Ladder"? You know, what he said in his "Tractatus Logico-Philosophicus" (I cite from memory):

...this book is like a ladder, use it, but then throw it away.
Isn't it the same with patterns? Use them to expand your experience, but don't use them in your software! And as to make the discussion complete, let us state a completely different point of view, a Lisper's critique***:
When I see patterns in my programs, I consider it a sign of trouble. The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I'm using abstractions that aren't powerful enough-- often that I'm generating by hand the expansions of some macro that I need to write.

This is the typical smug Lisp programmer assertion, as they mean that all other languages will in limes converge to Lisp, but newly I tend to think along the lines of the Lispers: the design patterns are a fix for lacking feature in language A, which you know from language B. But as we cannot change the language in most cases, and we are doing OO in the mainstream, just mind the ladder parable!

Update (June 2010): There's another explanation which I didn't think of before. I came across that when reading some programming book (which I cannot come up with now, sorry for the author!). Recall the mid 90-ties: it was the time of the "object craze", everybody wanted to be part of OO. But nobody really understood OO, and as the book went "Stroustrup gave us the C++ but he didn't tell us how to use it"! Her come the patterns, a kind of missing OO manual! This would explain my feelin that half of them was trivial (or not so difficult to figure them up by yourself) and the other half was implemeting language features missing from C++. Very sober diagnose, but at the moment it's my favourite.

---
* Paul Lockhart, "A Mathematician’s Lament", www.maa.org/devlin/LockhartsLament.pdf

** I found it in the book of Andy Hunt, "Pragmatic Thinking and Learning", Pragmatic Bookshelf, Oct. 2008, but here's a blog post link, and here another one

*** Paul Graham, "Revenge of the Nerds", http://paulgraham.com/icad.html