From DLL- to XML-hell

This is one of the original articles I wanted to write when I started this blog. Well, it's taken some time, it's a little outdated now, but at last I've kept my promise (or rather one of them) !

1. The Lament

As I switched from C++ to Java in some previous project I thought I plunged directly from the DLL- to the XML-hell! Well, not exacltly (as I didn't programm under Windows in the preceeding project) but a little hyperbole makes a good start! The XML part of the title however, is true. The very first thing I noticed, was the ubiquitous XML - you had to use it for compilation (Ant scripts), you had to use it for for deployment (web.xml), you had to use it for your application (struts2.xml, spring.xml, log4j.xml).

The most annoying part of this was the change from makefiles to Ant scripts: so much superfluous verbiage! Look, compare the Ant script with the equivalent Ant-builder script in Groovy, taken from the "Groovy in Action" book, first the XML file:

<project name="prepareBookDirs" default="copy">
  <property name="target.dir" value="target"/>
  <property name="chapters.dir" value="chapters"/>
  <target name="copy">
    <delete dir="${target.dir}" />
    <copy todir="${target.dir}">
      <fileset dir="${chapters.dir}"
        includes="*.doc"
        excludes="~*" />
    </copy>
  </target>
</project>

Then the builder script:

  TARGET_DIR = 'target'
  CHAPTERS_DIR = 'chapters'

  ant = new AntBuilder()
  ant.delete(dir:TARGET_DIR)
  ant.copy(todir:TARGET_DIR){
      fileset(dir:CHAPTERS_DIR, includes:'*.doc', excludes:'~*')
  }

Well, that's at least readable (and writeable), and it states exactly what you want! When I see such a comparison I think that the XML code should be used only as Ant-internal, assembly-code level data representation! You could maybe find it interesting that even the creator of Ant himself admitted that using XML was "probably an error"*.

The second problem was that, (as another programmer put it):

Developers from other languages often find Java's over-reliance on XML configuration annoying. We use so much configuration outside of the language because configuration in Java is painful and tedious. We do configuration in XML rather than properties because...well, because overuse of XML in Java is a fad.**

I can only confirm this. You can get a feel of how much of XML "code" was needed in a Java web application I was then writing from the following (addmittedly pretty old) table*** :

     Metric               Java +              Ruby +
                     Spring/Hibernate         Rails 

  Time to market     4 months, approx.      4 nights,
                      20 hours/week       5 hours/night
  Lines of code          3,293                1,164
  Lines of config        1,161                113
  Number of classes/     62/549               55/126
  methods                  

Do you see? The configuration, it's 30% of the code (!!!), and that's 10 times more than was needed in Rails! That's some dependency! By the way, note that Rails needs only 3 times less code that Java, despite the fact that Java libraries can be sometimes pretty low level!

Now, for me the Spring 2 platform was somehow an apogeum of the XML overdependency - the endless configuration files weren't even human readable, you should better use a graphical Eclipse plugin:

to edit the wirings. That's like you need something like UML to tame the complexity of the XML files (and of Spring's configuration of course):


A rhetoric question: isn't it too much of a good thing for something as simple as configuration files? I found the Spring-2 MVC (Spring's web GUI application framework) particularly bad in that respect. Comparing it to the Struts 2 configuration, I had an impression that every obvious fact must be expressed in XML, and that there's no defaults altogether:

<bean name="userListController"
    class="de.ib-krajewski.myApp.UserListController">
    <property name="userListManager">
        <ref bean="userListManager" />
    </property>
    ....
</bean>

<bean id="userListManager"
    class="de.ib-krajewski.myApp.UserListManager" />

and so on, and on, and on.

But wait, it comes even better: some people are so enamoured with XML that they are using it as a programming language****, either doing patterns with Spring and XML configuration files, or even inventing an XML-based programming language and writing its interpterer in XSLT (I'm not joking!).

A word of caution: Java community noticed the problem and is trying to reduce the amount of XML needed. Unfortunately, I haven't got any hard numbers, but each new release of a Java framework seems to claim a "reduced XML config file size" - take for example Spring MVC v.3. However, as the DLL hell is still a reality despite of Microsoft's efforts and the introduction of manifests (really, look here!), equally, for my mind the overdependence on XML won't vanish overnight from the Java world - it's at its core!

2. The Analysis

2.1 XML data

Why are we all using XML? The received standard answer is: because it's a portable, human-readable, standardized data format with a very good tool support!

So why does it all feel so wrong? The first reason I see is that XML is too low level for human consumption. It's claiming to be human-readable while it's only human-tolerable - see the Groovy AntBuilder example above. There are tons and tons of verbiage (although that's something a Java programmer will be rather accustomed to ;-)) which is what machines need, but it's not how human mind is working. It needs high level descriptions because it's only all to easy distracted by unneeded details.

When I recall my own config file implementations of yore, I conclude that I never needed more constructs than single config values ans lists of them. And I never needed hierarchical config data for my systems. So if people are using XML for configuration, they are maybe after something more than pure config settings, maybe they are looking for a scripting solution for the application (as mentioned in*) along the lines of the late Tcl? But with Tcl the model was entirely different: the Tcl scipts glueing together passive modules of code, whereas now, the modules are active and use XML as a database of sorts... So is it that at last? People using XML data as a poor man's, read only database? It's a good, ad-hoc solution, and you can emulate hierarchical, network and OO databases without much of a hassle, can't you? So people are using XML as a kind of qiuck and dirty hack, and hacks aren't normally very pretty ;-).

2.2 XML in Java

The second part of the question is: why are we using so much XML in Java? The answer is probably that we need XML config files to increase modularity an require the loose coupling of our systems. The Spring framework allows us to pull the parts of the system together at the startup or run-time, and that's supposedly a good thing, isn't it.

Coming to this part of the question: I think that the loose coupling thing is overrated. As Nicolai Yossutis said in his book about SOA (we reported ;-)):

"...loose coupling has its price, and it's the complexity."

So firstly, a little bit of coupling isn't that bad if it decreases complexity and increases readability of code! To my mind, it's a classical tradeoff, but as some developers are only too prone to introduce tight coupling all over the place, the received wisdom is to avoid the coupling at every price! But be honest, you can't decouple everything, if the parts are ment to be working together! Secondly, I thing that the "inversion of control" pattern is a bit of overkill too.

Personally, I'd like to glue together the parts of the system directly in code, so that I can see how my software is composed, and that in my source code - it should be the sole point of reference! Thus I like much more the approach Guice or Seam are taking (and nowadays the Spring framework itself) - that of annotations. At last we have the relevant information where it belongs - in code! The problem here is, that the annotations aren't really code, they are metadata, and I don't like the idea of using metadata where you could do things directly by using some API. But hey, that's probably the way we are doing such things in Java...

3. A Summary

That's what I wanted to write approximately 2 years ago. Meanwhile I use XML everywhere in my architectures and designs, mainly because it excludes any discussions about formats! No one has ever said anything against XML!

But programming it (in C++ and Qt) is a real pain - I first tried DOM, than SAX - and it was an even greater mess (well, there's a C++ data binding implementation á la Java's Apache XMLBeans but it's not used at my client's :-((). XPath's selectors are supposed to be better, but Qt doesn't have it by now. The good thing is, everyone will think the system is sooo cool because it uses XML, and I have to make my users happy!

PS: And you know what? The new dm application server by SpringSource uses JSON for configuration files!!! I told you so ;-)

--
* "The creator of Ant excercising hit own daemons", this article was originally stored under http://x180.net/Articles/Java/AntAndXML.html, but this link seems to be dead, so I'll host it on my website for a while (hope it's OK...): http://www.ib-krajewski.de/misc/ant-retrospect.html

** as a matter of fact, he's a Ruby programmer too, so he isn't that unpartial: http://www.brainbell.com/tutorials/java/About_Ruby.htm

***taken from the book "Beyond Java" by Bruce Tate: http://commons.oreilly.com/wiki/index.php/Beyond_Java/Ruby_on_Rails, unfortunately the original link stated there isn't working: Justin Gehtland, Weblogs for Relevance, LLC (April 2005); http://www.relevancellc.com/blogs. "I *heart* rails; Some Numbers at Last."

**** "Wiring The Observer Pattern with Spring": http://www.theserverside.com/tt/articles/content/SpringLoadedObserverPattern/article.html, and "XSL Transformations. A delivery medium for executable content over the Internet" in DDJ from April 05, 2007: http://www.ddj.com/architect/198800555 . I cite from the latter:

XIM is an XML-based programming language with imperative control features, such as assignment and loops, and an interpreter written in XSLT. XIM programs can thus be packaged with an XIM processor (as an XSLT stylesheet) and sent to the client for execution.

See, I wasn't joking...

Some distributed computing news: cuddly elephans, pigs, nymphs and couch computing

As I did some distributed computing research in my previous life, and then tried to get involved with Google, I'm still somehow interested in the subject. If you are into distributed computing, you'd know that Google is using a distributed C++ system called MapReduce* for indexing of the crawled webpages. It happens that there's an open-source Java implemenation of Google's system (based on published Google's research papers) called Hadoop . Up to now I didn't know about any serious usage of Hadoop in big distributed systems, but as it comes, I learned that Yahoo! is using Hadoop internally** for spam detection!

Yahoo's researchers have also designed a new language called PIG Latin (?!)*** for distributed programming on the Hadoop platform. The idea is rather an interesting one: they pointed out that the existing way of writing programs for the mapper and reducer part of a distributed application is too low-level and not reusable, but somehow familiar to programmers. In contrast, a declarative language like SQL poses some problems, so the paradigm there is completely diffrent:

At the same time, the transformations carried out in each step are fairly high-level, e.g.,filltering, grouping, and aggregation, much like in SQL. The use of such high-level primitives renders low-level manipulations (as required in map-reduce) unnecessary.
...
To experienced system programmers, this method is much more appealing than encoding their task as an SQL query, and then coercing the system to choose the desired plan through optimizer hints.

So it provides a middle ground, where we can avoid SQL and write down the computation steps, but can do that at higher level!

Unsuprisingly, Microsoft already has a proprietary distributed computing platform called Dryad, which tries to do something like this as well, but is using C# and LINQ (i.e. an SQL-like query language):

DryadLINQ generates Dryad computations from the LINQ Language-Integrated Query extensions to C#.

But I don't know how much this is a research project and if it's really used internally. On the other side, Microsoft recently aquired a search company called Powerset, which is using Hadoop at its core, and as the gossip runs, should be used to bring Microsoft's LiveSearch up to speed.

Facebook, in their turn, are using Hadoop, but added a Business Intelligence tool called Hive (now released as open source!) on top of it. It uses an SQL-like query language, as BI-users understand it best. Beneath the surface Hive leverages Hadoop and translates SQL-like imperatives into MapReduce jobs.


Not bad, cuddly elephant! It seems everyone is loving you!


Accidentally, when speaking about SQL and non-SQL: there's a non-SQL database by Apache called Couch DB. It is written in Erlang (we reported already**** ;-)) and implements a distributed document database. It's designed for lock-free concurrency using Erlang's "share-nothing" philosophy. To my mind it resembles Linus' Thorvalds git source configuration system, as it holds all the versions of documents determining the "winner" to be visible in defined views. Hear, hear! Erlang's alive, distributing and kicking!

Another thing worth noticing is that's not only SQL anymore, there are attempts to do things differently, athough others are sticking with the ol 'n reliable. Paradigm change on its way? It's interesting times I think.

PS (6 Okt. 2009): Look, look, now even the Big Blue has jumped the wagon too! They offer the M2 enterprise data analysis platform, based on Hadoop and using PIG. See: http://www.sdtimes.com/link/33808. So it seems Hadoop is definitely mainstream and business analyst thingie now, and as such it has lost much of its appeal, as far as I'm concerned.

---
* Jeffrey Dean, Sanjay Ghemawat: "MapReduce: Simplified Data Processing on Large Clusters", OSDI 2004, http://labs.google.com/papers/mapreduce.html
** unfortunately it's a two-parts video: http://developer.yahoo.net/blogs/hadoop/2009/03/using_hadoop_to_fight_spam_-_part_1.html
*** Christopher Olston et al: "Pig Latin: A Not-So-Foreign Language for Data Processing", SIGMOD 2008, http://www.cs.cmu.edu/~olston/publications/sigmod08.pdf
**** http://ib-krajewski.blogspot.com/2007/08/erlangs-change-of-fortunes.html

One cute microdesign example and some thoughts on language design.

This time I'll begin with a citation of another blogger*:

I made the switch from C++, I'd wished that Java allowed operator overloading (i.e. custom implementations of operators such as '+', or the array brackets). However, some argue that this feature adds to the complexity of C++ with little advantage, and that there is no need to pass this complexity on to the Java language. Someone who never coded in C++ would more than likely agree with that.

OK, someone who never coded C++ won't understand that, and will take the argument about the undue complexity as received wisdom. So let's show you a little piece of design, a microdesign actually, so you can (maybe) appreciate the feature. If you're a Python/Perl/Groovy (or C++ for that matter!) programmer, I must apologise, but maybe you'll find this small example nonetheless instructive.

1. A piece of microdesign

It's a piece of code from my old project I wrote several years ago. I wanted a class for interthread and interprocess messages (you know, for my favourite pattern: share-nothing, message-passing actors). I wanted to be able to construct a message using parameters of the constructor like this:

    AbstractData notif;
    SocketMsg msg(DispInterface::crReqNotifType, &notif, father);

And then simply send the formatted message as binary data:

    m_connSocket.send(msg, msg.buffsize())

On the other side, I wanted to receive the binary data from socket directly into the message object, which would then parse it and provide high level access to the received data, just like:

    SocketMsg msg; // empty
    if(!m_connSock.receive(msg.writebuff(), msg.buffsize()))
    {
        // if too much errors, better bail out here!
        TRACE_AND_CONT("Cannot read from socket in an FD event handler!",
                       E_ID_Q3A_SOCK_ERR); // try to escalate
        return;
    }

    // look what message received
    if(msg.getType() == cmdOkMsgType)
        handleCmdResp(msg.getAbstrData());

Pretty high level, eh? Just hide away all that low level data parsing and formatting and use the message!

It's design on its lowest level, you cannot possibly make a smaller piece of meaningful design, but I liked that piece! Now a question for the astute reader: what pattern is that? Proxy, bridge or a facade? Or a builder maybe? Perhaps none? Well, I don't think in pattrens, I think in abstractions: what I needed was an abstraction of a self-formatting message. Don't you like it too? Unfortunately, you need some operator overloading magic to do this. Look at the following class:

class SocketMsg
{
public:
    SocketMsg(int type, AbstractData* message, DsetId dsetId = nillDsetId,
              int cmdId = 0, int userCode = 0);
    ... etc

    // conversions
    operator const char*() const { return m_charRepr; };
    inline SocketMsgBufferProxy writebuff(); // see inline section 

private:
    // buffer
    static const int c_buffsize = sizeof(SocketMsgStruct) + 1;
    char m_buff[c_buffsize];

    // decoded data: don't remove in case we use different encoding!
    AbstractData* m_data;
    int m_type;
    DsetId m_dsetId;
    int m_cmdId;
    int m_userCode; 

    friend class SocketMsgBufferProxy;
};

You can see in the above definition that the class has a buffer for the coded binary message and several fields for the decoded message values. In the char array context, it exports its internal message buffer for reading (through the operator const char*()) so the socket's send function can read the raw data from there. In the other direction, i.e. for receiving data into the internal buffer a different mechanism is used: we are returning a SocketMsgBufferProxy object (this time through the writebuff() function) , which is defined as follows:

class SocketMsgBufferProxy
{
public:
    SocketMsgBufferProxy(SocketMsg* sm) : m_sm(sm) {};
    operator char*();
    ~SocketMsgBufferProxy();
private:
    SocketMsg* m_sm;
}

What it that proxy object doing? It's forwarding the address of the internal raw data buffer and revealing it through the operator const char*() again. As a matter of fact I could offer an even cleaner interface:

    m_connSock.receive(msg, msg.buffsize())

So why didn't I simply expose the internal buffer with the char* conversion operator and introduced a SocketMsgBufferProxy object inbetween? Simply because of the SocketMsgBufferProxy's destructor. When its destructor is invoked, it rescans the raw contents of the receive buffer so after the receive() line of code is finished, the message object is parsed and ready! Of course there is another possibility to achieve this: lazy parsing, but I was somehow more excited with that cool proxy object automatically coming inbetween and doing all the work! I'm only a human too and sometimes looking for the unusual...

2. And now some more general discussion

So as you can see, the operator overloading is a pretty useful little feature. So why didn't the designers of Java include it into the language definition? My opinion is that they overreacted. I can still remember the times of the “object craze” when everybody tried to be object oriented without knowing what that really was, and to use every single feature C++ offered without thinking if it's needed or not for the particular problem at hand. That was somehow similar to the “patterns craze” I can see even today: people asking me “so what pattern could I use here?” - you don't need a pattern here, you need a solution of the programming problem, or do you want just a cool name to stick it on your code? But more about that in future entry.

You wouldn't believe how stupid the people got with the operator overloading, maybe it was something like the ”powder rush” of the skiers, an incredible infatuation with the until-then unheard-of possibilities! You must consider that a fresh-made C++ programmer came from a much smaller language (namely C) and normally didn't have exposure to any higher level language concepts form Smalltalk, Lisp or Simula. So the Java designers simply thought “Hey folks, it's time to cool down!” and banned the feature. To my mind, a fatal case of underestimation of people's capabilities and of patronizing. Because the result of it is that “you cannot do any magic in Java!” (well, actually you can with introspection and bytecode insertion, but only so much, and that's simply not enough). Look at the following citation**:

In fact, I'd say that many of today's current hot trends in programming are a direct result of a backlash *against* everything that Java has come to represent: Lengthy code and slow development being the first and foremost on the list. In Java you generally need hundreds of lines of code to do what modern scripting languages do in dozens (or less).
...
So what's the solution? Well, this is the problem. People have been tinkering with Java for years now, and there's still no hope in sight. There's something about the Java culture which just seems to encourage obtuse solutions over simplicity. As a Java developer, I was always so amazed at how difficult it was to use the standard Java Class Libraries for day-to-day tasks.

And you know what? My pet idea is, that the reason why Java code is so dull and repetitive (and that the Java culture is, ehm..., you've just read it...) is the lack of operator overloading! You think I'm overly generalizing now? Well, the fact is, you simply cannot hide the gory details as conveniently as I was able to do it in the SocketMsg class! The best indication that the people in Javaland are thinking the same, is the Groovy JVM programming language: it has the operator overloading at its core, and is capable of doing things you wouldn't dream possible in Java. Well, that's no virtue in and by itself, but as result it gives you much leaner code! And we all need it: less code!

3. Now something even more general

When am already at it, Java programmers I met wouldn't understand the idea of the h-files (aka include files) too. Why are they needed for Heaven's sake? Isn't it an awful bore? I admit that the origin of the h-files lies in the limitations of the early C compilers, but with C++ it gained another significance: a “table of content” of the code proper. I give the word to another man from Java(FX)land***:

From my experience with C++ and Java, having method bodies in the class declaration clutters it with a mass of implementation details which is detrimental to getting an overview of the actual relationships and operations embodied by the class. It was for this reason that I decided to define the bodies of class operations and functions outside the class declaration.

Exactly what I think! You need an overview for the class, so you don't have to skip every one int and bool declaration until your head gets dizzy. You'll retort that this will be done with Javadoc comments. Well... The Javadocs... It's quite a theme of its own. Let me give only a small example here: if I've already written the function declaration like:

    void sendMessage(SocketMsg& msg, Receiver dest);

why should I repeat all this in a trivial Javadoc header? Is it "don't repeat yourself" or what? Don't get me wrong, I wouldn't check in any uncommented code, but I don't use them in (C++) header files - they only clutter up and make the definitions unreadable! For me they are well suited as implementation descriptions. And seconly, you have to run the javadoc tool first, you cannot just have a quick glance at the code! I must admit, sometimes I'm annoyed about having to edit the h-files, but when I'm reading the code afterwards, I'm happy I did that. Pretty non-progressive, isn't it?

4. Summing up

Maybe you'll say that it's not ethical to lambast a slightly ageing, slightly demodée, mainstream language, and that all is only too well known for several years now, but hey, it's just what I always thought! And like with my other favourite subjects (Agile, XML, Python, patterns), it simply takes time till I write it down, and it's no so insightful then, sorry...

--
* http://www.ddj.com/blog/javablog/archives/2007/08/java_and_potato.html
** http://www.russellbeattie.com/blog/java-needs-an-overhaul
*** Chris Oliver, Creator of JavaFX, in his presentation about JavaFX: http://jazoon.com/jazoon07/en/conference/presentationdetails.html

Fibers, at last!

As I'm recently doing Windows programming again after some 10 years of a pause, I keep learning about new things. One of them are fibers. Come again? What's that??

A fiber, as the name implies, is a piece of a thread. More precisely, a fiber is a unit of execution within a thread that can be scheduled by the application rather than by the kernel. *

So fibers are just user space threads, aka cooperative multitasking, which I knew since the stone-age DOS times and which weren't supposed to be cool at all!

It were native kernel threads which were supposed to be cool back then! But I must admit that in the past as I was working with cooperative multitasking in a rather big, multithreading system, I was quite happy with it! It gave a nice, predicatble feel to the system; you just new at what times what threads are supposed to run! The sole problem was (of course) that a long running user task couls block all the system. But then we moved on to native threads and I forgot about cooperative multitasking.

But something has changed since then: first, as it was already said, Windows has got it, although allegedly only as a good-enough implementation to support the SQL-Server (fibers have severe restrictions on the kinds of Win32 API’s they can call!). Second, Oracle 10g has got it too, I cite**:

Fiber model support
- An extension of the thread support already in place since Oracle7. Users may now run the database in fiber mode, which employs Oracle-scheduled fibers instead of O/S scheduled threads.
- For CPU intensive apps, this will provide a performance boost and reduce CPU utilization.

Moreover, recently I discovered a fiber library for Windows XP to remedy the unsufficient native Windows fibers support named FiberPool***. And then there are Green Threads (user space level, cooperative multitasking threads) in Java's JVM for some time now! The recent Windows 7 beta has introduced User Mode Scheduled Threads (UMS Threads)**** , which are like Win32 Fibers, as they can be scheduled by hand, but are backed by real bona-fide kernel level threads! Sounds rather interesting!

So at last it's the old, good cooperative mutitasking rediscovered? There's a growing evidence! Will be the ol' good times of DSET maybe back then Stefan?

---
* Johnson M. Hart: Windows System Programming Third Edition, Addison Wesley 2004
** http://download.oracle.com/owsf_2003/40171_colello.ppt
***http://www.fiberpool.de/en/research.html
**** http://blogs.msdn.com/nativeconcurrency/archive/2009/02/04/concurrency-runtime-and-windows-7.aspx

C++ Server Pages

In my prevoius blogs I wondered if there is/could be a C++ implementation of the Java's Servlet interface(s)*, and recently a friendly fellow blogger** pointed out to me that there already is one, viz the CPPSERV: http://www.total-knowledge.com/progs/cppserv/!

Can you see C++ it its logo?

I cite*:

CPPSERV is a web application server that provides Servlet-like API and JSP-like functionality to C++ programmers.
...
CSP (C++ Server pages), the JSP equivalent for CPPSERV, are currently implemented as a precompiler that converts CSP source to C++ source of a servlet, which, in turn needs to be compiled into CPPSERV servlet.
...
Current CSP implementation supports basic parsing, as well as compile-time taglibs.

Well, that's it! Or perhaps not? What we have is the equivalent of the Java's JSPs, which generally suck, and standing alone are too week a framework for serious work. But hey, it's not that bad, as we have got some choices: the CPPSERV for the oldskool JSP-like programming, the late Bobcat for more of the same, or the Wt framework*** for more modern, GWT-like work!

And with some discipline (I mean the model 1 ;-))) servlets, do you remember???) it's possible to get working application in C++ quickly! And all that Java frameworks out there are only trying to hide the JSP processing model, and they aren't that beautiful themselves. So for smaller things a JSP-like solution will be probably sufficient, and for more modern applications (including AJAX support) I'd choose the Wt framework.

There remains the question of the OO-Relational mapping, an the creators of CPPSERV suggest using their SPTK classes****, but it seems to be only a JDBC-like DB access layer. I suppose, I'd rather take Qt's DB classes for that, but on the other side, they are somehow heavyweight. But the OO-Relational thing for C++ is a different question in itself. But JDBC style access isn't that horrible either, and in a whole I think there are some viable options for C++ web applications out there!

Now I guess, I should try out the 2 frameworks, but ... ehm ... you know, so little time, so many books...

PS (07 Oct. 2009): meanwhile I have detected two another C++ and C "Servlets". Well they aren't really Servlets, but are rather one step back on the evolutionary path, and are somehow JSP-equivalent. The first of them is named KLone and allows you to include C scripts in the HTML page in the exact JSP manner. The other is GWAN - it's free and it allows you to write "servlet" scripts in C, which will be executed by the server and generate the HTML output - so in this case we are almost there, except for the very limited set of functions you can use in the script. Nonetheless, very interesting!

--
* see: http://ib-krajewski.blogspot.com/2008/10/c-servlets-again.html
** Gavino, unfortunately his profile isn't available :-(((, but nonetheless, thanks bloke!
*** The Wt-framework ("witty" - http://www.webtoolkit.eu/wt/)
**** SPTK: mainly a GUI framework, but it has some DB access support as well: http://www.sptk.net/index.php

Reading the new C++ Standard

Recently I've read on my local German C++ forum* that's there was a new draft document for C++09 Standard** and immediately I rushed out to read it. I mean, something tangible at last! Well, actually it wasn't really a read, but rather a very superficial skimming-over. There are good reviews of C++09 on the Web***, so I didn't actually want to read it in depth, but rather to get an overall impression of the changes: just how it all would feel like in C++0x: x < a.

And guess what? It was last year in November (sic!!!) when I read it and then wrote some things up :-(((. Somehow I'm too busy with my current project! Of course I'd like to read the proposal in depth and analyze it here thoroughly, but alas, it's not possible. So let's start with just few of my very personal impressions:

1. Ranges and initializers

What I perhaps liked best of the new features! Now I can write things like:

    map<type-expr> m = {{}, {}, {}};
    int a[] = {1};
    vector<type-expr> v = { a, b, c };
    SomeClass obj({ a, b, c, d });

and that for any class I want! How is that done? You have to implement the initializer_list constructor for your class! Wow, it feels just (like) Groovy to me :-). A great improvement over the oldskool tricks used to force initialization where it wasn't possible by the language definition!

A realated feature I liked are ranges:

    for(int& x: array)
    {
        x = 0; // for example!
    }

This is accomplished internally by a Range<> class being internally instantiated with the array instance: Range<type-expr>(array). It gives you a nice, scripting language like feeling when iterating; I mean, even Java's got it, and the boost::FOREACH macro was in my personal opinion rather an ugly hack than anything else!

When talking about ranges: actually I was expecting that the ranges will be included for the STL algorithms, so we don't have to write the annoing:

    find(v.begin(), v.end(), not_zero());

each time. I was quite positive that we'll going to have something like:

    find(v, not_zero());

as an overload of the traditional STL signature using the new Range<> class (there is a boost::range library after all!), so I must say I'm a bit disappointed here.

2. Some nice utilities I found

There were suprisingly large number of small interesting things to be discovered, like:

• error support: error_code, system_error - well, I can definitely use some of it!
  
• compile time (i.e. template) rational arithmetic - suprise! BTW do we really need it?
  
• clock and time_point classes - useful!
  
• stoi: string to int conversion - a suprise!, so don't we take the road of lexical_cast<> as I was expecting all the time?
  
• alignment control - at last I can say what the aligment is and not the machine! Will then the machines revolt?
  
• unique_ptr - just what we need most often, or auto_ptr fixed!
  
• addressof - even if the actual & operator is overloaded!
  
• reference_closure as Callable - what was that again??? But I wrote it up in my original notes, so it must be something funny!
  
• delegating constructors - we can reuse one constructor in another one, like in this example from the draft document:

    struct C {
        C(int){}     // 1: non-delegating constructor
        C(): C(42){} // 2: delegates to 1

• nullptr as a keyword - no more the annoing (XXX*)0 casts!

I liked these little things, and there is probably more like that to be discovered. It's refreshing to see that the smaller problems of day-to-day programming are addressed too!

3. Some expected things

As expected we've got the hash tables at last, as actually the whole TR.1 stuff like regular expressions, type traits for templates, tuples, general binders and function objects, is in there too. Plus the move constructors and rvalue references.

Moreover the auto keyword and the lambda functions are there, so now we can write

    auto iter = m.begin();
    // and not
    map<string,string>::const_iterator iter = m.begin();

at last! Just think how many keystrokes/copy and pastes it will save!

Speaking of lambdas - as I read it, there isn't one example of a lambda function in the draft! And moreover, in "Chap. 14.3.1 Template type arguments" there's something missing: namely the old restictions about the non-global linkage of the template parameters! This means, I can write a functor in local scope and pass it to the STL algorithm like that:

    void XXX::someFunc()
    {
        struct my_not_zero {
            bool operator() (const X& p) { return p != 0; }
        };

        // use the local functor 
        find(v.begin(), v.end(), my_not_zero());
        // OR use a lambda function:
        find(v.begin(), v.end(), [] (const X& p) -> bool { return p != 0 });
    }

I'd say, what was the reason for lambdas again (at least for me)? To be able write a functor which doesn't have to have global linkage! Are lambdas obsolete then? Well, both yes and no. Just look at the code example above, writing a local functor still requires quite a lot of plumbing, and a lambda expression is somehow more terse (we could even spare the return type of bool in our example!). IMHO both of them are not wholly satisfactory. Using a lambda library solution I could write it in a much more terse way:

    // OR using my pocket lambda library:
    find(v.begin(), v.end(), _$1 != 0 });

Another expected (and somehow dreaded) new thing are the Concepts. I must say thery are everywhere in the standard library section, like in this new std::list move constructor:

    requires AllocatableElement<Alloc, T, T&&> list(list&& x);

I mean, now the standard library header files will get even less readable, but the compiler errors will be (hopefully) more so. So I'm in two minds about this feature: isn't it too much of the type system? What about the duck typing in templates? Are we giving it up? For me it was a very nice feature. I guess it won't come that bad, and the templates without Concepts will be working as before but are Concepts only for compiler support or are we supposed to use meta-typing everywhere?

The next expected thing, which however deserves its own section is:

4. Multithreading support

Sorry, I didn't read it yet... It's two whole chapters (29 and 30) ant they aren't exactly short.

But instead it started me thinkig: how was I writing multithreading applications before there was the C++ memory model defined? How on earth was this possible? Well, it was all vendor (i.e. compiler) specific. When a compiler supported the POSIX thread library (pthreads) for example, it would pose some memory fences around the pthread calls****. It would treat some pieces of code in a special way - just like in Java's String magic. The local static initializes would be automatically guarded by locks (Gnu compiler) or maybe not (Sun compiler). The volatile keyword would be given an extra visibility guarantee (Microsoft compiler). And I never initialized global static variables out of the multithreading part of the program, only in the main thread portion. An the best of it is, that all these techniques wasn't guaranteed to work****! But hey, the 1st Java memory model wa buggy too, wasn't it?

Now it's all supposed to be portable.

One thing which I did read up (however on the Web and not in the proposal itself), and what was a littel unexpected for me, is about the volatile keyword. The proposal didn't stengthen the volatile keyword in the way it's been done in Java or C#. I suppose that out of the reasons of backward compatibility with C's volatile remained only a compiler (dis)optimisation directive, while for the visibility guarantees between threads we are supposed to use the atomic<> library!

How will the old programs using volatile for visiblity between threads be running now? Are they broken? I suppose so, they probably must be rewritten using the atomic<> types.

5. Summing things up

My impression so far is, that there are many small things you wouldn't expect, and that they are fun! I was expecting the big new chunks of functionality to be added (and they were added indeed) to be the most impressive ones, but it's rather the small things which make the difference. Look at that code snippet in the new, cool layout style (which I picked up in this Ferbruary's issue of the MSDN Magazine) :

    for_each(values.begin(), values.end(), [](int& value)
    {
        // now we can nest STL at last! (or can we???)
        for_each(value.begin(), value.end, DoSomething);
    });

I'd dare to say that C++ seems to have put away its 90-ties image, as it allows to write programs more in style of the scripting languages than the old and venerable K&R C! I can only add "at last" and "Amen".

---
* Thanks Jens!
** Herb Sutter's announcement and some discussion with the inevitable C++ bashing: http://herbsutter.wordpress.com/2008/10/28/september-2008-iso-c-standards-meeting-the-draft-has-landed-and-a-new-convener/, the Draft document itself is here: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2798.pdf
*** like the article in the Wikipedia, which I found to be surprisingly good: http://en.wikipedia.org/wiki/C%2B%2B0x
**** Hans Boehm: "Threads cannot be implemented as a library" Technical Report HPL-2004-209, HP Laboratories Palo Alto, 2004, http://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf

Do we need destructors?

This post is for my honoured, old-time, old-skool hacker colleague Stefan Z. As we were discussing the new and (then) hot Java language, we couldn't accept that it didn't have destructors. I cite Stefan:

The constructor/destructor pair is an incredibly powerful concept!

Well, you can't have everything: either we support garbage collection or destructors, isn't it? But it just one more point where Java sucks: the ugly try/finally block and then the explicit close() call like in the following code:

    BufferedReader reader =
            new BufferedReader(new FileReader(aFileName));

    try {
      String line = null;
      while ( (line = reader.readLine()) != null ) {
        // process the line...
      }
    }
    finally {
      reader.close();
    }

Ugly? You bet! And what I really don't like is that you cannot hide all the required handling in a library! In C++ you'd just write:

    ifstream f(aFileName);
    string line;

    while(f)
    {
      getline(f, line);
      // process the line...
    }    

That's all, the plumbing is hidden in the destructor* and it is there automatically! It's the reason why Stefan called this concept an "incredibly powerful" one. But that powerful concept can cause problems in a multithreading and garbage-collected environment. As a matter of fact, a recent C++ standard proposal for the multithreading execution model opted for removing destructors from the language (!!!), or at least for not executing the static destructors in a multithreading setting! Of course, it's a shortcut in order to solve a rather complicated problem, but you get the idea, right?

So maybe the destructors are a little bit outdated, what do you think Stefan? All the more was I pleased as I recently stumbled on a Smalltalk pattern called "Execute Around Method" pattern** (to be true, I don't do any Smalltalk and have seen it in some Groovy example code). It's another possiblity to hide the plumbing in the library: you just define a static method doing all the dirty work and accepting your "payload" code - just like in an IP packet: we have the framing and the payload, and the user is only delivering the payload! Well, an example explains it best:

    def static use(closure)
    {
      def r = new Resource()
      try
      {
        r.open()
        closure(r)
      }
      finally
      {
        r.close()
      }
    }

The closure parameter stands for our "payload" code. This code is the hidden in the library. An application of this is the following Groovy code:

    new FileReader(aFileName).withReader
    { reader ->
        reader.readLine(line)
        // process the line...
    }
    // no need to close()!!!

We create a new reader, give it to the static withReader() library method, and provide a "code block" (as you'd call it in Perl) for execution. This code block (called closure in Groovy) gets as the parameter the ressource which will be closed at the end, just like the use() method shown above!

A destructor for the modern times! So the "incredibly powerful" idea can be saved?

---
* this is called a RAII-pattern in C++, see: http://www.hackcraft.net/raii/
** Kent Beck: "Smalltalk Best Practice Patterns", Prentice Hall, Englewood Cliffs, NJ, 1996.