C++ Annotations not so pointless? Plus some UMTS history.

Let me start this post with a retrospection: long long time ago I promised to write a blogpost about multithreaded testing, temporal logic and (even) the boundaries of knowledge. Pretty high claims, you'd say, and you'll be of course right, as I didn't write it despite of a few false starts. So though the wholesale treatment of this theme isn't possible now because I plain and simply forgotten much of the details, I'll try to catch the opportunity to pay off my "blogging debt" and use that stuff in a condensed form as intro for this post.

1. Past struggles

So then, long long ago I was writing an multithreading application (or rather a framework* for other programmers to use) under Linux/pthreads against an underspecified multithreading interface of a "General Telco Systems Programming Platform" ;) - let call it GTSPP;). It was underspecified because it was still being developed while we were writing production code to be deployed on backplane UMTS switches.

Well, what can I say, the regular multinational project's madness, alleged synergy, and all that stuff... But, perhaps surprisingly to you, I wanted to have that thing thoroughly tested! And I mean tested! After I found some unexpected behaviour of GTSPP;) and resulting deadlocks, I didn't want to allow it to happen again.

As at that time I was interested in logic, automatic theorem proving and generally in foundational speculations, I had that brilliant (or at least I thought that...) idea - why don't we just describe a multithrerading system in terms of modal logic, like that:

• []T (read: box T) meaning: T alway happens
• <>T (read: diamond T) meaning: T sometimes happens

 and then use automatic derivation do derive some theorems of thatt system? Like "Theorem 1: there will never be a deadlock here".  [In the originally planned post a lot of explanantions about Kripke, possible worlds, Buchi automata... and model checking would follow here... BUT this isn't the planned post, sorry!]

At last some sensible use of maths! Can't believe it, so these whole math classes weren't a total waste of time?

Well, yes and no. As I tried several open source products, none of them was able to read C++ code, annotate it with modal logic, and infer the needed theorems. You call me naive? But that's exactly what SPIN is doing! The only problem was that SPIN doesn't accept C or C++ code, but only it's own description language: Promela. So I had to recode my system in Promela. That was out of the question, as I didn't have so much time. Writing a general translator C++ to Promela translator was beyond question as well - Clang wasn't there yet!

SPIN's author did some interesting work for AT&T on translating C code to SPIN format and even open-sourced it, but as I tried to install it on my Linux machine I failed. Some package dependency was missing, and our sysadmin asserted that there's no no such package for the RedHat distribution... At that point I gave up** thinking: the real boundaries of knowledge are that of human laziness...

2. New hope?

You may remember that I wrote a post "Two C++ curiosities..." where I expressed my inability to grasp the reason for the introduction of attributes in C++ (apart from Microsoft's influence). Recently, the same Microsoft organized a C++ conference  with quite a good video coverage. As I happened to listen to the Clang presentation i noticed some interesting usage of that language feature. It looked roughly like that:

or in gcc's syntax:

Do you see that? Now you can add an attribute to multithreaded code and let the compiler check if it's deadlock and starvation free!

That's what I call a sensible vendor specific extension! Well, it could be, as at the present all this isn't available yet (it's a research project at Microsoft's) but at least now I learned one case where the attributes could do quite a good job!

PS: More information on that topic: the boost::thread and thread safety annotations post or LLVM docs.

--

* I know, programmers hate frameworks, we were hating GTSPP;) too, but we were merging an another telco framework used in the previous project with the hapless GTSPP;) as not to expose the application programmers to too much madness - you know, brains melting, people screaming, teams disintegrating... So it was a Good Thing in the end!

** You don't really think I gave up on this, do you? So what was my solution? Right! I wrote a Python test driver repeating testcases by 1000s and grepping the output for specified error messages. The classic and proven multithreading testing strategy. Maybe a little brute-force and old-fashioned, but you know what? It did the job!

Coding styles.

As I was looking at my mate's C++ code I spotted the following curious lines:

  TreeDataCIterator it  ( m_treeDataList.begin() ), 
                    end ( m_treeDataList.end() );

Don't you think it's rather unusual for C++? Myself would rather writhe something more mundane like:

  TreeDataCIterator it = m_treeDataList.begin(); 
  TreeDataCIterator end = m_treeDataList.end();

complying with some old coding guidelines of the dark ages (aka 90-ties) which were imposed on the unhappy programmers in some big, iternational, waterfall project. But, you know, habit is the second nature... So I was somehow perplexed by the originality of the code, as I'd rather expected more dull style (like mine) considered C++'s strong C-legacy. But then I rememebred that my mate was originally a LISP programmer (but then his LISP-based company didn't really take off) and everything became clear! Look at this snippet uf LISP code from the SICP book:

  (define (count-leaves x)
      (cond ((null? x) 0)
            ((not (pair? x)) 1)
            (else (+ (count-leaves (car x))
                     (count-leaves (cdr x))))))

isn't it similiar? The love of parenthesis cannot be cured of easily ;-).

On the other hand, this could be as well an exmple if the C++ initalization syntax, and in C++11 we could write as well:

  TreeDataCIterator it  { m_treeDataList.begin() }, 
                    end { m_treeDataList.end() };

Not that LISPy at all (instead maybe a little confusing)! But seriously, don't you think arguing about this is a loss of time? Well, not really, good code should be readable, and if we care about our code, it's does matter. But then I find myself increasingly obsessed with code formatting and visual appeal of my code. It's certainly not wrong, but it robs my brain of its preciuos ressources. Remember? Your brain can hold in memory only 6-7 items at the same time, so decreasing mental load is a blessing for it. Not mentioning that formatting and reformatting your code takes time.

And the worst is, that your favourite formatting style at the moment won't be that in the next year. Myself, I was in projects where the coding styles looken at first insane to me! And I arguend and ranted but complied to them for the greater (i.e. project's) good. An guess what, now I'm using elements of these coding styles in my own code! Ok, not everything, but some parts weren't so nutty at all. What does that mean? Yes, you guessed it: coding styles doen't matter at all. But switching between them does!

So why we just don't stop doing this? Why don't let wo the computer do this and concertrate on more important aspects oif code? There are so many of them! As I first read about Go's policy about code formatting I wasn't too enthusiastic about it. But meanwhile I find it a good idea: only one prescribed layout style automatically imposed on your source code! Let's forget about coding standards and concentrate on problem solving!

Static vs Dynamic Debate Decided

As I said, it's decided (at least for me) now!

A couple of years ago it became fashionable to be ecstatic about dynamic (i.e. dynamically typed*, aka scripting) languages, this sentiment is maybe best exposed by Steve Yegge here. People started to talk about "freedom languages", and it all started getting rather emotional. Like in this tweet:

"Strong types are the hobnailed boot of the Enterprise Man on the neck of the Agile Code Poet"  http://goo.gl/aeEcn

It is understandable as an aftershock of the J2EE meltdown by Ruby and Rails, but I was always rather suspicious with that. I was always rather suspicious because of the following gut feeling - why should I throw away things which can possibly help me? If you are skiing, you know that: you were falling and you'll fall, the question is only when (hope not in an icy couloire or over a cliff...). It's the same in programming - you will fall.

1. Theory 

A proof written as a functional program (Wikimedia)

As at that time I was interested in mathematical logic and automatic theorem proving, I happened to know about the Curry-Howard isomorphism. It roughly states that a proof in intuitionistic logic has its equivalent type expression and vice versa - a type is equivalent to some proof.

So if a type system is equivalent to a (constructive!) proof of correctness, that's cannot be bad. We are getting some limited guarantees from the compiler - limited because the type system doesn't describe all of the program's behaviour - and that's one thing less we have to worry about!

Well, OK, maybe that's all theoretical babbling -  who knows if all that logic/foundational thing in mathematics isn't contradiction-free? And even if it's not, it could be not that relevant in practice?

Because when confronted with such a question (i.e. about the correctness guarantees), the standard answer was: "You gotta write more tests!" or "Strong Testing not Strong Typing"** or  "I only proved the correctness of this program, I didn't test it yet" ;). And because tests are a good thing, you cannot deny that this maybe could be true in practice...

2. Own Experiences

So let me share some of my own experiences with dynamic typing.


A couple of years ago, as I read the excellent "Higher Order Perl" book , I got through the first and middle chapters and the code was understandable and clear.

At this point it's maybe appropriate to give the book some praise, as it's well written, and shows an unknown side of Perl - it's functional self. As the title reveals, the book is about functional programming in... Perl! Yes, that's possible, it even seems to match the language rather well. Of course the language wasn't designed for that, and you will miss all the compiler support and type inference of OCaml but then, Perl isn't statically typed, OK? The only fly in the ointment is that Perl has't been fashionable for quite a couple of years yet...

But I'm digressing here, so back to the main theme: at first it wasn't a problem that you don't know the types of the parameters in Perl. It's all convention, you have to retrieve the parameters from the argv and use them according to their type like that:

   my($top, $code) = @_;  # @_ is Perl's argv! 

So all the type information is so to say in the code that uses it later on. But in the last chapter, when the author is developing a linear equation DSL-based drawing system, I sometimes got lost in code. I simply didn't know what the inputs are, and what the function will be doing with it. How I wished I could see the parameter types in the function declarations!

Another example: if you are writing Python code using a library you aren't that accustomed to, and you don't want first to read the documentation for an hour or so, but just quickly code something small you need just now, you'll very likely just to get a bunch of runtime exceptions and have to guess what is exactly going on now. What is the exact type the library is expecting here? I did everything right, didn't I? Nope, you didn't. Please look up the docs.

So AFAIK, the types are for human understanding, even if you don't need the type checking guarantees of the compiler. So common sense dictates using it - you know 80% (my wild guess here) of programming is maintenance, the code is written once and read 1000 times, and so on. We all know these old programming adages, and they don't just apply to telling variable names and insightful comments. Type information is in my eyes another part of writing understandable (as opposed to "genius"-) code.

3. Other people's experiences

 OK, I'm not that much experienced in dynamic languages, so maybe I've got the wrong impression, or didn't understand something correctly, or maybe I'm just not clever enough?

But then, when I started reading the "Beginning Scala" book (well, I didn't finish reading it in the end because I got bored, but that's another story...) the following lines struck me:

As my Ruby and Rails projects grew beyond a few thousand lines of code and as I added team members to my projects, the challenges of dynamic languages became apparent. We were spending more than half our coding time writing tests, and much of the productivity gains we saw were lost in test writing. Most of the tests would have been unnecessary in Java because most of them were geared toward making sure that we’d updated the callers when we refactored code by changing method names or parameter counts.

As you see, that practical experiences are rather confirming my assertions about static typing: a) why write test which can be (kind of) automatically generated by compiler, and: b) why not let automatically document and check the interfaces?

But there's another angle on that problem which I didn't think of:

Also, I found that working on teams where there were mind melds between two to four team members, things went well in Ruby, but as we tried to bring new members onto the team, the mental connections were hard to transmit to new team members.

This goes more into the psychology of programming on the first sight, and maybe it's a cultural thing altogether, but it relates a little with: c) why not document the interfaces as you are writing code to make obvious things obvious? I mean, if the interfaces are better described then maybe the new team members will absorb the knowledge better?

Another Ruby guy tried the Go language and wrote an interesting post about his experiences with static (although implicit) typing he encountered in GoLang:

I guarantee that no matter how much I look at the Ruby code, if it runs for any length of time I'll eventually encounter a typing error because of a bug in the code…

OK, the same old story about interface documentation and checking as above. But read on, there's more to come:

I've learned that I vastly prefer the cost of slower development time to gain reliability and safety, as long as the cost isn't too high. Haskell is even safer, but it pushes the boundaries of how much pain I'm willing to endure trying to get code to pass through the type checker

Here he's making a good point, kind of reiterating my main argument: there's a trade-off between the developer time needed to type a piece of code and it's reliability. In the past we opted for ease of construction, but if we can have the reliability for very small additional cost? With type inference we hit a sweet spot here!****

So it's decided, there are real problems with dynamic typing in real-world projects, and static typing can help out here. Why would anyone want to use a dynamic language (beside for scripting some small pieces of code) then?

But wait, what about Clojure? I like this language... and I'm somehow in 2 minds here - the reason says ML but the heart says Lisp. So is this an emotional thing in the end? But in business you have to follow the path of reason, not coolness, to deliver results to your clients. And there aren't that much Lisp projects out there... So the decision isn't difficult: I'll opt for statically typed languages in general, but when offered a Clojure project (which will happen very seldom I guess) I'll jump to it! Don't you think it's a good tradeoff?

Goodie:

PS: do you remember pre-ANSI C? Probably not, but it wasn't type safe:

Within the function, the declaration of a is visible, so the compiler knows it's a char*. For a call, the compiler doesn't know what type of argument the function expects, so it's up to the caller to pass the right type. It's similar to what happens with printf-like functions.*** 

Here an example for those not remembering the "stone-age":

int 

ParseGLFunc (interp, argc, argv, nArg)

    Tcl_Interp *interp;

    int argc;

    char *argv [];

    int *nArg;

{

    ....
}

--

* I'm sure you know it already, but statically typed means for a programming language to check the types at compile time whereas dynamically typed means checking of type correctness at runtime.
** An article by Bruce Eckel: "Strong Typing vs. Strong Testing"

*** found it here: "pre-ansi declarations"
**** here I  must admit that I'm just impressed with the ML family of languages (i.e. OCaml, F# and well, Haskell too) and the automatic type deduction the compiler is doing! The code is just flowing from your fingers as in Python or Ruby, but it's type checked!

ASSERT(AfxGetThread() == NULL) with VisualStudio 2010, or the DLL-hell, revamped

Everybody thinks DLL-hell is a thing of the past. Me included: #WTF, just start the Dependency Walker, look what libraries got loaded, correct it and be happy*. Not so, with the advent of Visual Studio 2010 everything got a bit more complicated!

Let me describe the problem which I encountered working in a project for one of my customers. Let's start with a...

Disclaimer:

This is only a short technical note - for those who (like me) first check the Internet for solution to weird programming questions!

The Problem:

is maybe best described by this desperate post:

Hi all,

I have a question to solve, the situation is I use a exe to load a dll(MFC regular dll),but when I use LoadLibrary() to load the dll and in the constructor of the object whose base class is CWinApp in the dll, It have a assert error, and the code line is ASSERT (AfxGetThread() == NULL).Is there anyone have the similar situation with me and have thoughts about it can tell me ,we can talk about it.

thanks in advance.

Well, the same happened to me when I was porting a C++ application from Visual Studio 2008 to VS 2010. To be honest, the situation was a little more complicated: a mixed mode C++/C# program using C++/CLI as a glue. But back to the problem: the standard explanation for that ASSERT is that you have forgotten to include the AFX_MANAGE_STATE macro (Reason 1) in exported DLL functions as described here.

Sadly, this wasn't my problem. It applies to non-MFC DLLs only**, which mine wasn't, so the whole module state management business should be cared automatically in DLL initalization code the linker is generating. This again is controlled by #define macros used whith the compiler (like _USRDLL, _AFXDLL and their hellish likes). So maybe this could be the reason? Nope, everything was correct :-(.

After more web-searching and finding only tons of happless cries for help I stumbled upon the 2nd useful link

Sounds like you have two or more CWinApp objects, and they're using the same AfxGetThread data. I suppose this could happen if your libraries are linking to different versions of MFC, including mixing of debug and release versions. You might use Dependency Walker (depends.exe) to determine the DLLs you link to implicitly, or the Sysinternals Process Explorer to determine the DLLs you're using at runtime

Well, that sounds interesting, mybe I've mixed debug and release DLLs? Nope (although I made that error later on, so it's defintely a good point to keep in mind (Reason 2)). What could it possibly be then???

The Solution:

The solution is more complicated, but somehow both typical and symptomatic for real-life nontrivial system using Microsoft technology... As my client had a really old codebase growing steadily with the years, all possible Microsoft technologies could be found there; beginning with plain old DLL, through MFC-based ones, further using C++/CLI and at last arriving at C# and .NET. This last one was burdened with a constraint: we had to use .NET v.2.0. The problem is, that Visual Studio 2010 doesn't suppport .NET 2.0! Because of time shortage they decided that only .NET 4.0 will be supported out of the box, and that the 2008 VS-compiler will be inegrated in Visual Studio 2010 to be used for older .NET versions.

Well, in itself it wouldn't pose any problems, if there wouldn't be the C++/CLI code which we used as glue between pure C++ MFC and the newer C# parts. It has to be compiled for the .NET 2.0 as well, thus should use the 2008 VS-compiler. But it is put in a DLL which int turn is loaded by a traditional C++/MFC one (created with  2010 VS-compiler!). See it? Two different compilers, 2 diffrent runtimes, 2 CWinApp objects as in (Reason 2) (or maybe the loader gets misleaded, I'm not sure about it). In any case, Visual Studio 2010 is forcing us to use 2 compilers in one application here (Reason 3).
 

The Mores:

I'm not sure about that one. How about this: stay away from MFC and use Qt? Or: don't migrate your code to VisualStudio 2010 (as it's rather slow to boot)? You're damned if you have to be backwards compatible? 

--

* see here  for author's previous letter from DLL-hell

** more hellish details: 3 kinds of DLLs you can use on Windows: ... TODO: link (but that's just plain boring!)

DSLs not so pointless?

... or: busting myths and fallacies, this time - the DSL-s!

Until recently I'd only sneer at the idea of DSL's: what the @!%*, either we have a soild API for developers, or a specialized higher level language for the specific task. And the higher level language best shouldn't be  done at all, because the non-technical user just wants a smooth GUI to click his requests together! So spare your DSL hype on me, go and find another gullible middle management person.

But then I read the following passage and the fallacy behind this reasoning became apparent to me:

In other environments I could write tests together with the product owner. Together, we could create and discuss tests that expressed what we wanted our code to do without getting buried in the details. The tests I wrote in C++ were quite unreadable for a non-C++ developer.

What was my error? DSL isn't meant as a programming language, neither for the developer, nor for the notechnical user - it is meant to document things!

Look at the following example of a testing DSL in C++:

  #include <igloo.h>  // testing framework*
  #include <cell.h>   // tested class
 
  using namespace igloo;

  Describe(A_Cell)
  {
      It(should_have_coordinates)
      {
          Cell cell(3,4);
          Assert::That(cell.getX(), Equals(3));
          Assert::That(cell.getY(), Equals(4));
      }

      It(can_detect_if_its_a_neighbour)
      {
          Cell cell(1,5);
          Cell neighb(2,4);

          Assert::That(cell.isNeighbourTo(neighb), IsTrue());
      }
  };

You must say it's a piece of C++ code** you could show to anyone and discuss it as a kind of formal spec. You and your partner don't have to learn Z or some other specialized notation. What both of you know already is just enough to bridge the gap. And it's cool code as well!

--
* see: http://igloo-testing.org/
** it's not a joke, all this is possible in C++

Geeky Halloween costumes...

Matching nicely my last post abot the Go language: dress up as a programming language mascot:

(as appeared in  Golang blog).

Any other languages to follow? For Perl it's simple (a camel), for OCAML too (a supercharged camel), but what about C++? Or Assembler?

PS: My favourite C++ pic is this:

. But you really can't dress up like that, can you?

What I did like about Go.

Last week I've read the "Learning Go" book. I did that by sheer coincidence: it happened that I had the (free) book loaded on my eBook reader and just looked for light but interesting read. Why not an intro to a new programming language? So I started to with it and I didn't regret my decision. It was an entertaining and interesting lecture. I'd like to describe here what I liked about the language.

See, normally reviewers concetrate on what they didn't like in a language, but I think that's way too easy! The things we don't like are most probably the things we aren't accustomed with, and with some experience they wouldn't annoy us any more. So don't gripe, just be easy on Go!

As I learned from the book, Plan 9 project*, the (supposed to be) successor of Unix, included a language called Limbo which was a lot like Go already, which built on an earlier langauge called Newsqueak. So surprisingly, there's a lot of history showing up here!

Go itself is like it's mascot (look to the right!): small and quick. It's compiled and statically typed but does type deduction (like OCAML or Haskell). It main claim to fame are (beside garbage collection) goroutines and channels, but I have too little experience with them, so I'd rather concentrate on basic day to day traits of the language. I mean, the small things are the things that annoy us the most, and should a language will be anoying, it will screw them. Additionally, this way we can somehow transmit the "feel" of the language.

So let's start with:

1. multiple assignement

...a la Python:

  x, y := 11, 12
  _, y := 22, 23

As you can see the semicolons are not needed too - gives you a nice Python / F# / Clojure feeling!
If you have mutliple assignament, you get multiple return values for free, as in the following function**:

  func Write(file *File, b []byte)(int, Error)

See? You just specify the returns in parens after the parameters. And that leads us to the next goodie connected to the multiple assignment:

1a. the ok-idiom

it goes like that:

  if ok, x = foobar(y); ok {
    // do sth
    ...
  }

So no more if(foo() == -1)! Instead you can write:

  if nr, err = write(f, buf); err == nil {
    // do sth
    ...
  }  

You can assign and test a value inside of the "if "clause. Additionally the parens can be omitted too with "if" - more Python / F# feeling! Nice, isnt it?

2. named return parameters

We can assign names the return parameters:

  func write(file *File, b []byte)(n int, err Error){
    ...
    
    return
  }

When we name the returns, the compiler will initialize them with defaults (i.e. their zero-values)! We can eithet refer to them inside of the function (!!) or even omit them from the return statement, and then they will be used automatically with the initialized values. One thing less to worry about.

3. maps (and strings)

They are part of the language and not in the library. I wish they'll be that way in C++ too. Come on, D has done that!. Add this to the {} initalization syntax and you can write (almost LISP, Python or Perl style)***:

  var opmap = map[int]string{
    ADD: ”+”,
    SUB: ”-”,
    MUL: ”*”,
    DIV: ”/”,
  }

Well, speaking frankly, the {} initialization is possible in C++ too, alas only as of C++11.

4. defer for destructors

You can define Deferred actions, i.e. actions which are executed on the end of scope only:

  file.Open(”filename.in”)
  defer file.Close()

It's a nice replacement for destructors (see my previous post here). And it looks better that some "using" constructs from  other languages.

5. duck typing

Go has duck typing, but it's not C++ type of template duck typing. You don't have to mention a specific interface (yes, Go has a notion of interfaces!), it suffices to implement them ***:

  type I interface {
    Get() int
    Put(int)
  }

And then you just implement the required functions for a data type (see note ** below), and it will automaticall comply with the interface I:

  type S struct { i int }
  func (p *S) Get() int { return p.i }
  func (p *S) Put(v int) { p.i = v }

like that:

  func f(p I) { p.Put(1) }
  var s S 
  f(&s)

OK, so whats the difference to C++ type duck typing? C++ doesn't care about types: a template with two parameters simply takes two things, and then tests if they have the methods the template code requires:

  template <type T> void foo(T& t)
  {
    if(t.canBark()) t.bark();
  }

How's that different from Go? C++ gets problems with overloading in that context: it sees only a function that takes 1 thing. A totally generic thing! So you cannot define a second template with the same name and a different argument type, because at the definition level there aren't types, only things! That's why there's no range or complete container overloads for SLT algorithms (see H.Sutter's blogpost).

6. generic switch

Just like Groovy, Go has a very flexible switch statement:

  switch c {
    case ’ ’, ’?’, ’&’, ’=’, ’#’, ’+’:   
      return true
    case c < 'a': fallthrough
    case c > 'A':
      return true
    default:
      return false
 }

I mean, ever language needs something like that: case switch for strings and other types! And did you notice that fallthrogh must be stated explicitly? I like that.

7. the go keyword.

I liked the idea of introducing the word "go" as keyword to start a parallel "goroutine":

  int i
  go foobar(i)

I just find it funny :), that's all...

Note:
I admit, I didn't describe the features that make Go to stand out, only a couple of  those which particularly appealed to me. For more info on: function definitions, val notation, control structures, function literals (i.e. lambdas), goroutines, etc please read the book by yourself!

---
* a note aside: as pointed out several times on the web, there's a notable similarity between the logos of Plan 9 and Go. BTW: don't you think they are cute?

** more precise, you'd rather define a function like this:

  func (file *File) Write(b []byte)(int, Error) 

i.e. a function working on the File type, and invoke it like:

  nr, err = f.Write(buf)

But this is the part of the OO stuff so I skipped it in this post.

*** code example from the "Learning Go" book.