What can you learn from the bad guys?

Really, is there something you can learn from them? Confucius said some time ago: "I'm always glad to meet another voyager: if it's a pleasant man, I can learn what I can do better, if it's an unpleasant man, I can learn what I shouldn't do!". But can we learn something in apositive sense? In order to be able to answer this, let us return to the realm of IT.

What do the really bad IT-guys do? They break in into systems! They use a host of techniques, and one which is particularly interesting is the "SQL injection" (I personally prefer the more colourful - if slightly incorrect - name "SQL hijacking"). Let's explain it using an SQL example. If we build an SQL query string like this:

query = "SELECT * FROM users WHERE name = ' " + userName + ' ";

Now, if we supply an user name string like, say "dummy' or name not null --", the resulting SQL query looks like*:

SELECT * FROM users WHERE name = 'dummy' or name not null --'

And this will get you the list of all users in the system! If the submitted username is like "xxx'; DROP TABLE users; --" the attacker can even get nasty on the database! Simple but effective!

So what, you'd say, how many bad guys will have direct access to my SQL database? Well, you'd be surprised! If you have a Web application which accesses an SQL database the following scenario is conceiveable. The bad guy invokes your SQL database indirectly requesting an URL in the form:

your.site.de/your_app/ListUsersAction.do?ID=11;UPDATE+USER+SET+TYPE="admin"+WHERE+ID=23

which, when appended to the "SELECT * FROM users WHERE id=" string will grant the attacker the admin privileges! Or, when he requests the URL in the form:

your.site.de/your_app/ListUsersAction.do?ID=11;GO+EXEC+cmdshell('format+C')

the disaster is looming! I don't want to discuss this in detail, as it's not a hack instruction page, but I hope you've got the general idea: this a real problem for the Web application development. Well, that's all very bad behaviour and all, but the question is: why should we learn anything about this?

Recently I was charged with extension of a J2EE web application for one of my clients.They were using a proprietary custom tag JSP library**, whose sources were not available (unfortunately, in a big company that's not so unusual). The custom tag library was rather nice, but as the code was gone, it was not extensible. For example, I couldn't specify the onClick attribute for the custom button tag:

<somelib:button action="someapp/SomeAction.do" image="..." ... ⁄>

Can you see it? Think, think... Yes! Let us try some JSP/Javascript injection:

<somelib:button action="someapp/SomeAction.do \" onClick=\"alert('Gotcha!'); \" " image="..." ... ⁄>

which produces, when iterpreted to HTML, a wonderful:

... url="someapp/SomeAction.do" onClick="alert('Gotcha!')" image="..."

The reason why this is working*** ist the fact that \" isn't interpreted as the end of string sign, as it is escaped with a "\" ! So we have a kind of double bottom here - on one level we have a regular string, which, when interpreted, is transformed into something different sematically - it's the second bottom. Isn't it beautiful? On one level (tags) we have a senseless string, which on the lower level (HTML) gets injected into a page code and starts living its own life! Like a seed coming to life after a hibernation phase. Since then I used this technique in several other places, and it never ceases to amaze me. Look at that: <somelib:help key="start_task" tooltip="Help \" align=\"right\" "/>.

So now we can answer the title question: from the bad guys, we can learn how to extend a non-extensible JSP custom tag library! And yes, Confucius was right: we can learn from every person. Indeed.

---
* here "--" comments out the last apostrofe, but we might as well add some ineffective condition like "and name='yyy" to balance the apostrophes.
** for those of you not doing J2EE: a custom tag is a sort of macro generating HTML code when interpreded on the server side
*** this works on Tomcat, and as Tomcat is the reference implementation of a Servlet container... guess what.

Some plans, or what's to be expected

...from this blog? As this blog is stalling a bit recently (I began several articles in parallel and they are each in a different phase of completion) I felt like it would be a good idea to reinstate the purpose of this blog. For me: to motivate myself to complete at last some of the most promising articles, and for my readers (yes, don't laugh, I know that nread >= 1!): to get some perspective if it'd worth reading this blog for the next time.

When I started, I had two purposes on my mind:

1. to write about some of my technical work which I found interesting
2. to give some interesting vistas on some general topics in SW developement - provided I can find some ;-)

So you can see this blog wasn't intended as uncoordinated blatherings, but rather a limited collection of articles: I simply haven't done that many interesting things or found that many new ideas. This can be explained by the fact that this blog is a sort of replacement for my old, nonfunctional, dilapidated homepage, which I really should update sometimes soon. But these days starting a blog is much simpler than re-working an old homepage (it's Web 2.0 now!). So let's extemporize a list of topics I've got in the pipeline:

1. Technical topics

• design and implementation of my own small lambda expression library in C++ (as it was fun)
• modal logic and tesing of multithreaded application (as it shows the limits of knowledge)
• using lock-free synchronization (as C++ programmers always worry about performance, maybe to much?)
• subtle Java vs C++ differences (as I like language-design discusssions)

2. General SW-engineering topics

• why is everybody keen on scripting languages (sociology applied ;-)
• Python's language design (coming to terms with Python hype)
• waterfall and XP methods (a somewhat different comparision)
• web services, SOAP an the (r)REST
• does speed really kill in SW-engineering?
• the new antipattern (some ruminations on singletons)
• C++ myths in the Java community (as most of the Java people know C++ only from fairy tales, I think)
• from DLL to XML hell (sometimes it must be a rant)
• Idiot's sorts (I mean sort algorithms...)

etc...

OK, let's get real. I'd be happy to complete the blogs on the three first topics from the list 1. and the four first topics from the list 2. In addition I'll sometimes do a "follow up" on some earlier blog entry. The one I must definitely write is a follow-up on the "Google Solvers"* blog: there is defintely something that has to be added to that one! Sometimes I'll simply extend some older article if it isn't too long, for example the "iPhone presentation"** blog: at the time of its writing I purposely omitted examining any of my guesses, now it's time to compare my intuitions with reality. And there are some other topics I've forgotten, but I may remember again sometime. As my distinguished colleague Stefan Z.* would say "Software engineering is like war, ...", so I'll keep on fighting.

----
* http://ib-krajewski.blogspot.com/2007/07/google-solvers.html
** http://ib-krajewski.blogspot.com/2007/08/iphone-presentation-with-afterthoughts.html

shell as a "modern" programming language

I don't know how many of you is reading ACCU's Overload magazine, but I do. In the last issue* we find an article by Thomas Guest with the flippant title: "He Sells Shell Scripts to Intersect Sets", which demonstrates implementation of the set-theoretic operations (i.e. union, intersection, difference, etc) in UNIX shell language. As interesting pastime as these techniques are, I was struck by the final words of the article:

"For me, it’s not just what the shell tools can do, it’s the example they set. Look again at some of the recipes presented in this article and you’ll see container operations without explicit loops. You’ll see flexible and generic algorithms. You’ll see functional programming. You’ll see programs which can parallel-process data without a thread or a mutex in sight; no chance of shared memory corruption or race conditions here.

.....the intent shines through as clearly as ever: we have a compact suite of orthogonal tools, each with its own responsibility, which cooperate using simple interfaces. We would do well to emulate this model in our own software designs."

The second part of the quote was not new to me: in fact I've heard it uncountable many (pun intended) times from my distinguished colleague Stefan Z.: set of simple tools which can be combined through a common set of interfaces. You've probably read this many times before. But what I really liked, is the mercilessly pointed out (timeless?) modernity of the shell: its high level programming model, its extensibility, its focusing on concepts and hiding details away. This stands in sharp contrast to the self proclaimed "modern" programming languages of the day: Java and C# (or is this already Python?), which rather don't excel in this regard.

We so often take these features of shell programming for granted, that it must be sometimes pointed out to make us think about it. These features are not given as a matter of course! After day-to-day struggle with low level aspects of the "modern" programming languages it sometimes pays off to turn your attention to shell and contemplate it's design once again (and dream of a better world ;-).

If you are Java programmer, read it. You'll like it. It's a lesson in abstraction.

---
* Overload 80, August 2007: http://accu.org/index.php/journals/c78/

Reprise:

Speaking about set operations: relational databases are built around a set algebra (as relations are sets in, ehm, set theory), so the shell file operations could be in principle implemented with SQL! If only we could turn the Apache logs from the abovementioned article into realtional tables...

Well, why not! Look at SQLite - an embeddable, zero-configuration, C-language database (it's used in GoogleGears for example). This database has one pretty nifty feature known as "virtual tables"*: you only have to write some callback methods which will be called by the SQLite database engine to read data which are stored in different formats. For example you could treat the filesystem as a database table(s), i.e. fire SQL queries on it! So, leaving out all technical details, it's possible with the help of virtual tables to scan our logfile using SQL! I guess it's rather cool.

---
* http://www.sqlite.org/cvstrac/wiki?p=VirtualTables

Do you GoF?

My first encounter with design patterns in action was a memorable one. It was about 10 years ago, shortly after the GoF book appeared and everybody who thought to be somebody was supposed to speak about it. I too pored briefly over the book, but was not a very dedicated student of it, and did it rather out of the sense of obligation. Not by any means to be able to solve a design-patterns crossword (yes, things like that exist in the vast open spaces of the web*)!

But what about the announced encounter? Well, there was a guy on the project I was in at that time, and he used nearly every pattern he could. The trouble was, his program didn't work! This gave me the first impression of the design patterns: it's something for weirdos, something to brag about, nothing for the people who really want to get the job done!

Then I have re-read the GoF book two or three times and found some of the techniques interesting, some not so, and some rather dull. Now, 10 years later, rather suprisingly for myself, I arrived at the opinion that I never really used the design patterns that much!!! I realized for example, that the much beloved patterns of the Java community** had never much appeal to me. For example I never used a factory pattern: why should I do it though? If I need an object, I simply create it using its constructor. Nothing simpler than that! In that way I never needed an Inversion of Control framework - I simply used Dependency Inversion, i.e. I parametrized the lower layer objects from the upper layer ones! I used the singelton pattern, but only to sort out the C++ problems with static object's initialization, not to ensure a single instance of something. If you need it once, create it once as Uncle Bob rightly said***. Ok, I must confess my sins: I used singletons instead of global objects :-((, but it was my laziness, and - yes, you guess it - nobody dared to gripe about this, as it was a "design pattern"! I never used the visitor pattern: it's simply too complicated, too confusing, and its variants (like asynchronous visitor) didn't make the situation any better. It's an "aha" moment when you realize that the visitor pattern is not about visiting but about adding new methods to classes!

An yes, I admit, there could be some problems where I'd use the abstract factory pattern, and there is always a problem where I will use the observer pattern, but here it is, my main point about GoF-ing: it is not that difficult to make out the equivalent solution on your own, really! Moreover, having a pattern given to you and ready to use (i.e. ready to copy and paste) is actually pernicious, as you won't have the deeper understanding of what you are doing, which can only arise from intensive occupation with the subject at hand. I know, the "deeper understanding" phrase sounds rather Harry Potter'esque, but I think it contains some truth.

So maybe the real value is that they establish a "pattern language" for the solutions of SW problems? I think it is so. You can refer to a portion of your code like "I use here an observer for this and a singleton for that" and don't have to describe the solution in detail - only its main idea.

Alas, the design patterns are presented to the developers rather as a catalogue of ready-made, off the shelf solutions to be copy-and-pasted every time you have a problem, or worse, every time you want to look smart to some other people. And this way they are stifling the creativity, stopping you from starting your own journey. But they were invented (if I'm interpreting Ch. Alexander's notion of architectural pattern correctly) to let you start your journey from a higher level.

PS: There is another opinion on design patterns out there: Mark Dominus maintains that a design pattern is really an indicator for a flaw in a programming language, i.e. that a language of next generation will make it obsolete. His example is the iterator pattern. I must admit I still haven't given it enough thought. On the other hand Rod Johnson makes a case again the design patterns as well - in this case again J2EE design pattens. He accuses them of being a mere workarounds for the design flaws of the J2EE platform.

---
* design patterns crossword: http://www.vokamis.com/products/cword/app/enterGame.php?ns=/a/a&or=V&amp;amp;amp;amp;amp;h=128&pub=2&ex=http://www.softwaresecretweapons.com/jspwiki/Wiki.jsp?page=GangOfFourSoftwareDesignPatternsJavaScriptCrossword
** see for example Rod Jonson's book, Expert One-on-one J2EE Design and Development, Chap. 4: http://www.theserverside.com/tt/articles/article.tss?l=RodJohnsonInterview
*** Uncle's Bob blog: http://butunclebob.com/ArticleS.UncleBob.SingletonVsJustCreateOne

servlets in C++ or 2 small C++ ideas

As I was recently reading the article of Stefan Wörthmüller "C++ Versus JEE"*. Basically Stefan argues that:

"If you want to write a server application, you need control. Maybe even control about your failures. But you need control."

And that of course is what C++ will give you. For the details refer to the article. However another sentence got stuck in my mind:

"I still wonder why there are so few approaches for Web frameworks/toolkits or application servers written in C++. C++ is made for such a task."

So I've got that big ;-) idea: why shouldn't I take the Servlet specification, restate the interfaces in C++ and code the Servlet classes and (perhaps) a servlet container! Why not use the language which is known to me (and many other server-side programmers) and combine it with an iterface which is known to many web-developers (i.e. Java programmers) to get the best of the two worlds? This will be a big success!!! Ok, I tell you why: no spare time.

But I still think this idea is not bad. It goes in the same vein as my other idea about C++: why should I use scripting languages for scripting? I can do everything just as well in C++. Ok, not everything, I would have to write some template libraries to plug the holes, but in principle I could. I didn't pursue this idea very long, but after some time I found the same reasoning explained more lucidly in the rationale of Boost's "Filesystem" library:

"The motivation for the library is the need to perform portable script-like operations from within C++ programs. The intent is not to compete with Python, Perl, or shell languages, but rather to provide portable filesystem operations when C++ is already the language of choice."

Then there is the Boost::String library which makes possible the Perl-like manipulations of strings... We are almost there.

But end of digression! The main theme of this entry are C++ servlets! So back to the web programming.

For some time I tried to find some C++ web frameworks by googling, but nothing useful emerged. So I a sort of gave up. Till today. I was googling for documentation of the OSE RTOS by Enea when I stumbled upon the OSE library. OSE, whassat? Never heard of it! Another legacy do-it-all framework? It turned out that this library was long time popular in Australia, and more: it has a HTTP Servlet support! At last!! It supports HTTP Session objects, XML, and its HTTP Daemon class seems to be a sort of servlet container. From what I've seen, its interface is different form Java's HTTPServlet class, but it is a start! Maybe one could combine it with a non-JSF view technology (Velocity?) to enhance it further? So maybe I will be able to pursue my idea nonetheless? But is it worth the candle? I would be only a copy of an existing technonogy, certainly not an "killer app" and not a new approach. Or maybe not?

----
* Dr. Dobb's Journal Jun 21, 2007, http://www.ddj.com/dept/cpp/199905990

Erlang's change of fortunes

Today morning I had a conversation with my distinguished colleague Stefan Z. while hanging out at the coffee-vending machine in the tea-kitchen at our client's office. As we are both working for the telecom industry, we came to discuss Erlang. No, not that celebrated guy, but a rather obscure programming language designed at the Ericsson Corporation.

For the fun of it, let's first look at some snippets of Erlang code!

define a data structure:

  Student={person,{name,{first,yakkety},{last,yak}},{footsize,1}}.

access a member of data struct (by pattern matching, cool!!!):

  {_,{_,{_,HisName},_},_} = Student.

printing yakkety:

  io:fwrite("his name is: ~w~n", [HisName]).

define a list:

  Fruits = [{apples,10},{pears,6},{prunes,3}].

append something to a list:

  Fruits1 = [{oranges,4},{lemons,1}  Fruits].

get the head of the list (by pattern matching, cool!!!):

  [ListHeadFruits2] = Fruits1.

simple function:

  incr(Num) -> Num + 1.

recursive function:

  fac(0) -> 1;
  fac(Num) when Nun > 0 -> N * fac(Num-1).

infinite loop receiving messages (by pattern matching, cool!!!) and sending responses back (by the !operator):

  loop() ->
      receive
          {From, {request, Param1, Param2}} ->
              From ! {self(), Param1 + Param2},
              loop();
       end.

a foreach loop (note the lambda function definition between fun and end!):

  extract2ndElements(List) ->
      lists:map(fun({_,Second}) -> Second end, List).

Ok, it was fun, but we mustn't digress too much...

Erlang in itself is rather a paradox: you normally associate Telco equipment programming with the lowest of the lowest: asm, C99 or EC++ (castrated C++). Now, there come some guys in the 80-ties and to the corporate management they can sell a functional language for this task! This alone is incredible! But wait, there's more! Up to now Erlang was a niche language used quite exclusively at Ericsson. Now this seems to be changing. But first we need some history.

In the 80-ties Erlag was designed to be slow but reliable. This is due to the very nature of the functional languages: you don't have shared state, no side effects, all data is copied between functions. For safety's sake even the variables can be assigned only once! If you apply that to parallel computing, you don't have to worry about synchronization and critical sections - the "share nothing semantics". But it will be rather slow. In this case it was a deliberate decision: they wanted not simply a reliable language, but a "highly reliable language"! And a distributed one. So to speak a natural fit for the Telco environment. As an example: the AXD301 switch achieved the incredible 99,9999999% realiability! It's 9 nines availability! And we were struggling to achieve 5 nines* availability with C++ on CarrierGrade, High Avalability (HA) Linux and a custom HA-middelware platform in my last project! By the way, the AXD301 software has 1.7 million lines of Erlang, making it the largest functional program ever written**! Really impressive stuff IMHO.

I guess you have read (or heard about) Herb Sutter's article titled "The free lunch is over"***. So you know that the "next big thing" in software is supposed to be multicore scalability. So hear up now: as the multicores are comming pretty cheap now, Erlang can be at last both fast and reliable as it can be easily distributed on several cores! It's definitely good news! We've seen the incredible reliability which can be achieved, and have a natural programming model for distributed computation. Why? In short, as there is no shared data:

1. programs are easily distributable,


2. easily made fault tolerant by composing them from worker and observer processes,


3. easily made scalable (because of 1. we simply add more processors).**

So could distributed computing be the saviour of the functional programming? Think about Google's MapReduce!

---
* For those not working with HA systems:

      Availability%   Downtime per year
      99.9999            30 seconds !!!!
      99.999              5 minutes
      99.99                50 minutes
      99.9                  9 hours

** says Phillip Wadler: http://wadler.blogspot.com/2005/05/concurrency-oriented-programming-in.html

*** on DDJ or www.gotw.ca/publications/concurrency-ddj.htm

iPhone presentation with afterthoughts

Yesterday me and my distinguished colleague Stefan Z. both saw the iPhone for the first time (over here in Germany it isn't available easily yet, it'll come sometime in November). It was presented to us by our co-consultant in the telco field, a long-time Mac and Objective-C freak. He is pushing Apple technology wherever he can, and has got considerable succes with his Mac-based telco testbed infrastrustructure recently. 

He's the archetypical nerd - totally enthusiastic about technonolgy. To help you to go into the mood of the conversation I offer a couple of juicy citations: "...y'all haven't got a clue!", "...here ends your tether but I'm only beginning.", "...all the telco companies can shut down now!". 

He showed us the iPhone GUI and it was impressive: it was just as it always should be! You can operate it holding it in one hand and just using your thumb on the touch screen! His basic message was this: no other company can duplicate this on a mobile phone (or on any other operating system for that matter)! Ok, sometimes he tends to massive exaggeration but this one started me thinking. 

I guess my coleague is essentially in the right, and it boils down to the technology (not management). As I don't know much about iPhone I'll be shamelessly wallowing in conjectures now, but read on. 

The first thing I thought about was: yeah, that wouldn't be certainly possible with JavaPhone! They are doing it in Objective C and Cocoa as "Java and GUI don't mix" and "Friends don't let friends Swing" (guess who said this*). And Java is the programming language of choice today! You can even do real-time programing with JRockit's deterministic garbage collection of late! Java proponents do not conceal that Java's stronghold is more and more the enterprise application computing**. 

On the other side Java critics may say that this stronghold is rather a kind of ghetto (Java is the next COBOL - where did I hear that?). Steve Jobs said: "Java's not worth building in. Nobody uses Java anymore. It's this big heavyweight ball and chain..."*** Ok, an exaggeration, we are using Java on a project here, but you get the idea: Java is now a language for big, heavyweight, corporate applications. Nothing exciting to be expected here. 

Is there another language/system which can do something that cool? Yes, on the desktop (well browser...) you can do some cool things with Flash, maybe with JavaScript. But for the regular GUI there's nothing comparable I fear. So the Objective-C freak can be in the right. 

---

 * it was the creator of Tomcat and Ant

 ** for example "Why Java?" on http://www.wantii.com/wordpress/?p=5

*** see http://www.informit.com/discussion/index.asp?postid=d1e63fde-10d5-404b-8a14-6ff0b92c1ee1 for some lively discussion on that phrase, you can google for "Java? It's so 90-ties" for some more Java critique

Reprise:

As I said, I didn't know first thing about iPhone at the time I wrote this entry. But gradually I've learnt some new things, and now I can compare my then guesses to the facts. 

1. Yes, I was basically right: iPhone uses Cocoa*, and you'll program Cocoa with Objective-C. So you cannot program it with Java - the iPhone is just 100% anti-Java. But curiously, you can do it anyway, only through an detour. Just take the GWT (Google Web Toolkit), write an Web application in Java and GWT will translate it to JavaScript, which will be executed by iPhone's Safari browser!** I find it somehow a strange twist of fate! BTW, the Google Phone*** is rumored to be 100% pro-Java... 

2. Yes, I was partially right about GUI programming: yes, Java isn't up to the task, but Sun seems to have noticed this and works on Java-FX: "a new family of Sun products based on Java technology and targeted at the high impact, rich content market"****. The reasons why a new take was needed were summed up as a following series of questions:****

• Why does it take a long time to write GUI programs?
• How can we avoid the “Ugly Java technology GUI” stereotype?
• Why do Flash programs look different than Java platform programs?
• Why does it seem easier to write web apps than Swing programs?
• How can I avoid having an enormous mass of listener patterns?

So these are some problems! As it seems, Java isn't anymore the one-fits-all language of yore, but is complemented by a host of new languages using the Java-VM (FX, Groovy, JRuby...). So maybe it's really: "Java is the new COBOL"? 

---

 * see: http://cocoasamurai.blogspot.com/2007/01/how-i-think-iphone-software-works.html

** http://groups.google.com/group/Google-Web-Toolkit/web/gwt-iphone-pretty-tasty

*** for some cool pics of Google Phone see: http://www.google-phone.com/ or - even better - http://www.seekxl.de/blog/2007/08/05/google-phone-fotos-vom-google-phone/

**** Sun's Java-FX presentation: http://jazoon.com/jazoon07/en/conference/presentationdetails.html?type=sid&detail=2161