shell as a "modern" programming language

I don't know how many of you is reading ACCU's Overload magazine, but I do. In the last issue* we find an article by Thomas Guest with the flippant title: "He Sells Shell Scripts to Intersect Sets", which demonstrates implementation of the set-theoretic operations (i.e. union, intersection, difference, etc) in UNIX shell language. As interesting pastime as these techniques are, I was struck by the final words of the article:

"For me, it’s not just what the shell tools can do, it’s the example they set. Look again at some of the recipes presented in this article and you’ll see container operations without explicit loops. You’ll see flexible and generic algorithms. You’ll see functional programming. You’ll see programs which can parallel-process data without a thread or a mutex in sight; no chance of shared memory corruption or race conditions here.

.....the intent shines through as clearly as ever: we have a compact suite of orthogonal tools, each with its own responsibility, which cooperate using simple interfaces. We would do well to emulate this model in our own software designs."

The second part of the quote was not new to me: in fact I've heard it uncountable many (pun intended) times from my distinguished colleague Stefan Z.: set of simple tools which can be combined through a common set of interfaces. You've probably read this many times before. But what I really liked, is the mercilessly pointed out (timeless?) modernity of the shell: its high level programming model, its extensibility, its focusing on concepts and hiding details away. This stands in sharp contrast to the self proclaimed "modern" programming languages of the day: Java and C# (or is this already Python?), which rather don't excel in this regard.

We so often take these features of shell programming for granted, that it must be sometimes pointed out to make us think about it. These features are not given as a matter of course! After day-to-day struggle with low level aspects of the "modern" programming languages it sometimes pays off to turn your attention to shell and contemplate it's design once again (and dream of a better world ;-).

If you are Java programmer, read it. You'll like it. It's a lesson in abstraction.

---
* Overload 80, August 2007: http://accu.org/index.php/journals/c78/

Reprise:

Speaking about set operations: relational databases are built around a set algebra (as relations are sets in, ehm, set theory), so the shell file operations could be in principle implemented with SQL! If only we could turn the Apache logs from the abovementioned article into realtional tables...

Well, why not! Look at SQLite - an embeddable, zero-configuration, C-language database (it's used in GoogleGears for example). This database has one pretty nifty feature known as "virtual tables"*: you only have to write some callback methods which will be called by the SQLite database engine to read data which are stored in different formats. For example you could treat the filesystem as a database table(s), i.e. fire SQL queries on it! So, leaving out all technical details, it's possible with the help of virtual tables to scan our logfile using SQL! I guess it's rather cool.

---
* http://www.sqlite.org/cvstrac/wiki?p=VirtualTables