cat out.txt grep ERRORWell, as I'm doing Windows programming now, I recently found out that it's possible after all! Incredible but true! Hovever the syntax is somehow different:
type out.txt findstr ERROROn things like that I see how fast the time is passing! A couple of years before, I'd never believe that Windows would ever catch on. I think, the tide change came with XP.
try {
Query q = new Query ();
q.parse(
"SELECT *" +
"FROM java.lang.String " +
"WHERE length <= :stringSize " +
"AND toString LIKE '%e%' " +
"ORDER BY length DESC");
q.setVariable ("stringSize", "5");
List<String> names = new ArrayList<String>();
String[] n = {
"alpha", "beta", "gamma", "delta", "epsilon", "zeta", "pi", "chi"
};
Collections.addAll(names, n);
QueryResults qres = q.execute(names);
List res = qres.getResults();
}
catch(Exception e) ...
Here I got { "beta", "delta", "epsilon" } as result. The only problem I can see as far is that the library doesn't use generics. The other one is the somehow unintuitive way it's treating the "SELECT *" query. Nomally the JoSQL library treats a collection of Java objects as a table which columns are formed by the values returned by the class' methods. However, in case of a "SELECT *" query, we won't get all of the object's attributes, but just the whole object as it is. On the other side, it's the equivalent, isn't it.JXPathContext context = JXPathContext.newContext(employees);Pretty nifty, isn't it? But while SQL query syntax seemed obvious to me, the XPath search expressions seems plain ugly and non-intuitive. I don't know both SQL and X-Path very good, but I find myself always forgetting pretty everything about XPath while SQL gets stuck. Maybe because XPath is too low-level an too much a hack?
Employee emp =
(Employee)context.getValue("/employees[name='Susan' and age=27]");
Employee emp =I assume you can do the equivalent in JoSQL, but you'd like to use joins, an that isn't that convenient for simple hierarchical structures like that above. So, in the end, maybe the XPath isn't so bad after all?
(Employee)context.getValue("/departments/employees[name='Johnny']");
<project name="prepareBookDirs" default="copy">Then the builder script:
<property name="target.dir" value="target"/>
<property name="chapters.dir" value="chapters"/>
<target name="copy">
<delete dir="${target.dir}" />
<copy todir="${target.dir}">
<fileset dir="${chapters.dir}"
includes="*.doc"
excludes="~*" />
</copy>
</target>
</project>
TARGET_DIR = 'target'
CHAPTERS_DIR = 'chapters'
ant = new AntBuilder()
ant.delete(dir:TARGET_DIR)
ant.copy(todir:TARGET_DIR){
fileset(dir:CHAPTERS_DIR, includes:'*.doc', excludes:'~*')
}
Well, that's at least readable (and writeable), and it states exactly what you want! When I see such a comparison I think that the XML code should be used only as Ant-internal, assembly-code level data representation! You could maybe find it interesting that even the creator of Ant himself admitted that using XML was "probably an error"*.
The second problem was that, (as another programmer put it):
Developers from other languages often find Java's over-reliance on XML configuration annoying. We use so much configuration outside of the language because configuration in Java is painful and tedious. We do configuration in XML rather than properties because...well, because overuse of XML in Java is a fad.**
I can only confirm this. You can get a feel of how much of XML "code" was needed in a Java web application I was then writing from the following (addmittedly pretty old) table*** :
Metric Java + Ruby +
Spring/Hibernate Rails
Time to market 4 months, approx. 4 nights,
20 hours/week 5 hours/night
Lines of code 3,293 1,164
Lines of config 1,161 113
Number of classes/ 62/549 55/126
methods
Do you see? The configuration, it's 30% of the code (!!!), and that's 10 times more than was needed in Rails! That's some dependency! By the way, note that Rails needs only 3 times less code that Java, despite the fact that Java libraries can be sometimes pretty low level!
Now, for me the Spring 2 platform was somehow an apogeum of the XML overdependency - the endless configuration files weren't even human readable, you should better use a graphical Eclipse plugin:
to edit the wirings. That's like you need something like UML to tame the complexity of the XML files (and of Spring's configuration of course):
A rhetoric question: isn't it too much of a good thing for something as simple as configuration files? I found the Spring-2 MVC (Spring's web GUI application framework) particularly bad in that respect. Comparing it to the Struts 2 configuration, I had an impression that every obvious fact must be expressed in XML, and that there's no defaults altogether:
<bean name="userListController"
class="de.ib-krajewski.myApp.UserListController">
<property name="userListManager">
<ref bean="userListManager" />
</property>
....
</bean>
<bean id="userListManager"
class="de.ib-krajewski.myApp.UserListManager" />
and so on, and on, and on.
But wait, it comes even better: some people are so enamoured with XML that they are using it as a programming language****, either doing patterns with Spring and XML configuration files, or even inventing an XML-based programming language and writing its interpterer in XSLT (I'm not joking!).
A word of caution: Java community noticed the problem and is trying to reduce the amount of XML needed. Unfortunately, I haven't got any hard numbers, but each new release of a Java framework seems to claim a "reduced XML config file size" - take for example Spring MVC v.3. However, as the DLL hell is still a reality despite of Microsoft's efforts and the introduction of manifests (really, look here!), equally, for my mind the overdependence on XML won't vanish overnight from the Java world - it's at its core!
2.1 XML data
Why are we all using XML? The received standard answer is: because it's a portable, human-readable, standardized data format with a very good tool support!
So why does it all feel so wrong? The first reason I see is that XML is too low level for human consumption. It's claiming to be human-readable while it's only human-tolerable - see the Groovy AntBuilder example above. There are tons and tons of verbiage (although that's something a Java programmer will be rather accustomed to ;-)) which is what machines need, but it's not how human mind is working. It needs high level descriptions because it's only all to easy distracted by unneeded details.
When I recall my own config file implementations of yore, I conclude that I never needed more constructs than single config values ans lists of them. And I never needed hierarchical config data for my systems. So if people are using XML for configuration, they are maybe after something more than pure config settings, maybe they are looking for a scripting solution for the application (as mentioned in*) along the lines of the late Tcl? But with Tcl the model was entirely different: the Tcl scipts glueing together passive modules of code, whereas now, the modules are active and use XML as a database of sorts... So is it that at last? People using XML data as a poor man's, read only database? It's a good, ad-hoc solution, and you can emulate hierarchical, network and OO databases without much of a hassle, can't you? So people are using XML as a kind of qiuck and dirty hack, and hacks aren't normally very pretty ;-).
2.2 XML in Java
The second part of the question is: why are we using so much XML in Java? The answer is probably that we need XML config files to increase modularity an require the loose coupling of our systems. The Spring framework allows us to pull the parts of the system together at the startup or run-time, and that's supposedly a good thing, isn't it.
Coming to this part of the question: I think that the loose coupling thing is overrated. As Nicolai Yossutis said in his book about SOA (we reported ;-)):
"...loose coupling has its price, and it's the complexity."
So firstly, a little bit of coupling isn't that bad if it decreases complexity and increases readability of code! To my mind, it's a classical tradeoff, but as some developers are only too prone to introduce tight coupling all over the place, the received wisdom is to avoid the coupling at every price! But be honest, you can't decouple everything, if the parts are ment to be working together! Secondly, I thing that the "inversion of control" pattern is a bit of overkill too.
Personally, I'd like to glue together the parts of the system directly in code, so that I can see how my software is composed, and that in my source code - it should be the sole point of reference! Thus I like much more the approach Guice or Seam are taking (and nowadays the Spring framework itself) - that of annotations. At last we have the relevant information where it belongs - in code! The problem here is, that the annotations aren't really code, they are metadata, and I don't like the idea of using metadata where you could do things directly by using some API. But hey, that's probably the way we are doing such things in Java...
That's what I wanted to write approximately 2 years ago. Meanwhile I use XML everywhere in my architectures and designs, mainly because it excludes any discussions about formats! No one has ever said anything against XML!
But programming it (in C++ and Qt) is a real pain - I first tried DOM, than SAX - and it was an even greater mess (well, there's a C++ data binding implementation á la Java's Apache XMLBeans but it's not used at my client's :-((). XPath's selectors are supposed to be better, but Qt doesn't have it by now. The good thing is, everyone will think the system is sooo cool because it uses XML, and I have to make my users happy!
--
* "The creator of Ant excercising hit own daemons", this article was originally stored under http://x180.net/Articles/Java/AntAndXML.html, but this link seems to be dead, so I'll host it on my website for a while (hope it's OK...): http://www.ib-krajewski.de/misc/ant-retrospect.html
** as a matter of fact, he's a Ruby programmer too, so he isn't that unpartial: http://www.brainbell.com/tutorials/java/About_Ruby.htm
***taken from the book "Beyond Java" by Bruce Tate: http://commons.oreilly.com/wiki/index.php/Beyond_Java/Ruby_on_Rails, unfortunately the original link stated there isn't working: Justin Gehtland, Weblogs for Relevance, LLC (April 2005); http://www.relevancellc.com/blogs. "I *heart* rails; Some Numbers at Last."
**** "Wiring The Observer Pattern with Spring": http://www.theserverside.com/tt/articles/content/SpringLoadedObserverPattern/article.html, and "XSL Transformations. A delivery medium for executable content over the Internet" in DDJ from April 05, 2007: http://www.ddj.com/architect/198800555 . I cite from the latter:
XIM is an XML-based programming language with imperative control features, such as assignment and loops, and an interpreter written in XSLT. XIM programs can thus be packaged with an XIM processor (as an XSLT stylesheet) and sent to the client for execution.See, I wasn't joking...
At the same time, the transformations carried out in each step are fairly high-level, e.g.,filltering, grouping, and aggregation, much like in SQL. The use of such high-level primitives renders low-level manipulations (as required in map-reduce) unnecessary.So it provides a middle ground, where we can avoid SQL and write down the computation steps, but can do that at higher level!
...
To experienced system programmers, this method is much more appealing than encoding their task as an SQL query, and then coercing the system to choose the desired plan through optimizer hints.
DryadLINQ generates Dryad computations from the LINQ Language-Integrated Query extensions to C#.
But I don't know how much this is a research project and if it's really used internally. On the other side, Microsoft recently aquired a search company called Powerset, which is using Hadoop at its core, and as the gossip runs, should be used to bring Microsoft's LiveSearch up to speed.
Facebook, in their turn, are using Hadoop, but added a Business Intelligence tool called Hive (now released as open source!) on top of it. It uses an SQL-like query language, as BI-users understand it best. Beneath the surface Hive leverages Hadoop and translates SQL-like imperatives into MapReduce jobs.
Not bad, cuddly elephant! It seems everyone is loving you!
Accidentally, when speaking about SQL and non-SQL: there's a non-SQL database by Apache called Couch DB. It is written in Erlang (we reported already**** ;-)) and implements a distributed document database. It's designed for lock-free concurrency using Erlang's "share-nothing" philosophy. To my mind it resembles Linus' Thorvalds git source configuration system, as it holds all the versions of documents determining the "winner" to be visible in defined views. Hear, hear! Erlang's alive, distributing and kicking!
Another thing worth noticing is that's not only SQL anymore, there are attempts to do things differently, athough others are sticking with the ol 'n reliable. Paradigm change on its way? It's interesting times I think.
---
* Jeffrey Dean, Sanjay Ghemawat: "MapReduce: Simplified Data Processing on Large Clusters", OSDI 2004, http://labs.google.com/papers/mapreduce.html
** unfortunately it's a two-parts video: http://developer.yahoo.net/blogs/hadoop/2009/03/using_hadoop_to_fight_spam_-_part_1.html
*** Christopher Olston et al: "Pig Latin: A Not-So-Foreign Language for Data Processing", SIGMOD 2008, http://www.cs.cmu.edu/~olston/publications/sigmod08.pdf
**** http://ib-krajewski.blogspot.com/2007/08/erlangs-change-of-fortunes.html
I made the switch from C++, I'd wished that Java allowed operator overloading (i.e. custom implementations of operators such as '+', or the array brackets). However, some argue that this feature adds to the complexity of C++ with little advantage, and that there is no need to pass this complexity on to the Java language. Someone who never coded in C++ would more than likely agree with that.OK, someone who never coded C++ won't understand that, and will take the argument about the undue complexity as received wisdom. So let's show you a little piece of design, a microdesign actually, so you can (maybe) appreciate the feature. If you're a Python/Perl/Groovy (or C++ for that matter!) programmer, I must apologise, but maybe you'll find this small example nonetheless instructive.
It's a piece of code from my old project I wrote several years ago. I wanted a class for interthread and interprocess messages (you know, for my favourite pattern: share-nothing, message-passing actors). I wanted to be able to construct a message using parameters of the constructor like this:
AbstractData notif;And then simply send the formatted message as binary data:
SocketMsg msg(DispInterface::crReqNotifType, ¬if, father);
m_connSocket.send(msg, msg.buffsize())On the other side, I wanted to receive the binary data from socket directly into the message object, which would then parse it and provide high level access to the received data, just like:
SocketMsg msg; // empty
if(!m_connSock.receive(msg.writebuff(), msg.buffsize()))
{
// if too much errors, better bail out here!
TRACE_AND_CONT("Cannot read from socket in an FD event handler!",
E_ID_Q3A_SOCK_ERR); // try to escalate
return;
}
// look what message receivedPretty high level, eh? Just hide away all that low level data parsing and formatting and use the message!
if(msg.getType() == cmdOkMsgType)
handleCmdResp(msg.getAbstrData());
class SocketMsg
{
public:
SocketMsg(int type, AbstractData* message, DsetId dsetId = nillDsetId,
int cmdId = 0, int userCode = 0);
... etc
// conversions
operator const char*() const { return m_charRepr; };
inline SocketMsgBufferProxy writebuff(); // see inline section
private:
// buffer
static const int c_buffsize = sizeof(SocketMsgStruct) + 1;
char m_buff[c_buffsize];
// decoded data: don't remove in case we use different encoding!
AbstractData* m_data;
int m_type;
DsetId m_dsetId;
int m_cmdId;
int m_userCode;
friend class SocketMsgBufferProxy;
};
You can see in the above definition that the class has a buffer for the coded binary message and several fields for the decoded message values. In the char array context, it exports its internal message buffer for reading (through the operator const char*()) so the socket's send function can read the raw data from there. In the other direction, i.e. for receiving data into the internal buffer a different mechanism is used: we are returning a SocketMsgBufferProxy object (this time through the writebuff() function) , which is defined as follows:
class SocketMsgBufferProxyWhat it that proxy object doing? It's forwarding the address of the internal raw data buffer and revealing it through the operator const char*() again. As a matter of fact I could offer an even cleaner interface:
{
public:
SocketMsgBufferProxy(SocketMsg* sm) : m_sm(sm) {};
operator char*();
~SocketMsgBufferProxy();
private:
SocketMsg* m_sm;
}
m_connSock.receive(msg, msg.buffsize())So why didn't I simply expose the internal buffer with the char* conversion operator and introduced a SocketMsgBufferProxy object inbetween? Simply because of the SocketMsgBufferProxy's destructor. When its destructor is invoked, it rescans the raw contents of the receive buffer so after the receive() line of code is finished, the message object is parsed and ready! Of course there is another possibility to achieve this: lazy parsing, but I was somehow more excited with that cool proxy object automatically coming inbetween and doing all the work! I'm only a human too and sometimes looking for the unusual...
In fact, I'd say that many of today's current hot trends in programming are a direct result of a backlash *against* everything that Java has come to represent: Lengthy code and slow development being the first and foremost on the list. In Java you generally need hundreds of lines of code to do what modern scripting languages do in dozens (or less).And you know what? My pet idea is, that the reason why Java code is so dull and repetitive (and that the Java culture is, ehm..., you've just read it...) is the lack of operator overloading! You think I'm overly generalizing now? Well, the fact is, you simply cannot hide the gory details as conveniently as I was able to do it in the SocketMsg class! The best indication that the people in Javaland are thinking the same, is the Groovy JVM programming language: it has the operator overloading at its core, and is capable of doing things you wouldn't dream possible in Java. Well, that's no virtue in and by itself, but as result it gives you much leaner code! And we all need it: less code!
...
So what's the solution? Well, this is the problem. People have been tinkering with Java for years now, and there's still no hope in sight. There's something about the Java culture which just seems to encourage obtuse solutions over simplicity. As a Java developer, I was always so amazed at how difficult it was to use the standard Java Class Libraries for day-to-day tasks.
From my experience with C++ and Java, having method bodies in the class declaration clutters it with a mass of implementation details which is detrimental to getting an overview of the actual relationships and operations embodied by the class. It was for this reason that I decided to define the bodies of class operations and functions outside the class declaration.Exactly what I think! You need an overview for the class, so you don't have to skip every one int and bool declaration until your head gets dizzy. You'll retort that this will be done with Javadoc comments. Well... The Javadocs... It's quite a theme of its own. Let me give only a small example here: if I've already written the function declaration like:
void sendMessage(SocketMsg& msg, Receiver dest);why should I repeat all this in a trivial Javadoc header? Is it "don't repeat yourself" or what? Don't get me wrong, I wouldn't check in any uncommented code, but I don't use them in (C++) header files - they only clutter up and make the definitions unreadable! For me they are well suited as implementation descriptions. And seconly, you have to run the javadoc tool first, you cannot just have a quick glance at the code! I must admit, sometimes I'm annoyed about having to edit the h-files, but when I'm reading the code afterwards, I'm happy I did that. Pretty non-progressive, isn't it?
Maybe you'll say that it's not ethical to lambast a slightly ageing, slightly demodée, mainstream language, and that all is only too well known for several years now, but hey, it's just what I always thought! And like with my other favourite subjects (Agile, XML, Python, patterns), it simply takes time till I write it down, and it's no so insightful then, sorry...
--
* http://www.ddj.com/blog/javablog/archives/2007/08/java_and_potato.html
** http://www.russellbeattie.com/blog/java-needs-an-overhaul
*** Chris Oliver, Creator of JavaFX, in his presentation about JavaFX: http://jazoon.com/jazoon07/en/conference/presentationdetails.html
A fiber, as the name implies, is a piece of a thread. More precisely, a fiber is a unit of execution within a thread that can be scheduled by the application rather than by the kernel. *
So fibers are just user space threads, aka cooperative multitasking, which I knew since the stone-age DOS times and which weren't supposed to be cool at all!
But something has changed since then: first, as it was already said, Windows has got it, although allegedly only as a good-enough implementation to support the SQL-Server (fibers have severe restrictions on the kinds of Win32 API’s they can call!). Second, Oracle 10g has got it too, I cite**:
Fiber model support
- An extension of the thread support already in place since Oracle7. Users may now run the database in fiber mode, which employs Oracle-scheduled fibers instead of O/S scheduled threads.
- For CPU intensive apps, this will provide a performance boost and reduce CPU utilization.
Moreover, recently I discovered a fiber library for Windows XP to remedy the unsufficient native Windows fibers support named FiberPool***. And then there are Green Threads (user space level, cooperative multitasking threads) in Java's JVM for some time now! The recent Windows 7 beta has introduced User Mode Scheduled Threads (UMS Threads)**** , which are like Win32 Fibers, as they can be scheduled by hand, but are backed by real bona-fide kernel level threads! Sounds rather interesting!
So at last it's the old, good cooperative mutitasking rediscovered? There's a growing evidence! Will be the ol' good times of DSET maybe back then Stefan?
---
* Johnson M. Hart: Windows System Programming Third Edition, Addison Wesley 2004
** http://download.oracle.com/owsf_2003/40171_colello.ppt
***http://www.fiberpool.de/en/research.html
**** http://blogs.msdn.com/nativeconcurrency/archive/2009/02/04/concurrency-runtime-and-windows-7.aspx
In my prevoius blogs I wondered if there is/could be a C++ implementation of the Java's Servlet interface(s)*, and recently a friendly fellow blogger** pointed out to me that there already is one, viz the CPPSERV: http://www.total-knowledge.com/progs/cppserv/!
Can you see C++ it its logo?
I cite*:
CPPSERV is a web application server that provides Servlet-like API and JSP-like functionality to C++ programmers.
...
CSP (C++ Server pages), the JSP equivalent for CPPSERV, are currently implemented as a precompiler that converts CSP source to C++ source of a servlet, which, in turn needs to be compiled into CPPSERV servlet.
...
Current CSP implementation supports basic parsing, as well as compile-time taglibs.
Well, that's it! Or perhaps not? What we have is the equivalent of the Java's JSPs, which generally suck, and standing alone are too week a framework for serious work. But hey, it's not that bad, as we have got some choices: the CPPSERV for the oldskool JSP-like programming, the late Bobcat for more of the same, or the Wt framework*** for more modern, GWT-like work!
And with some discipline (I mean the model 1 ;-))) servlets, do you remember???) it's possible to get working application in C++ quickly! And all that Java frameworks out there are only trying to hide the JSP processing model, and they aren't that beautiful themselves. So for smaller things a JSP-like solution will be probably sufficient, and for more modern applications (including AJAX support) I'd choose the Wt framework.
There remains the question of the OO-Relational mapping, an the creators of CPPSERV suggest using their SPTK classes****, but it seems to be only a JDBC-like DB access layer. I suppose, I'd rather take Qt's DB classes for that, but on the other side, they are somehow heavyweight. But the OO-Relational thing for C++ is a different question in itself. But JDBC style access isn't that horrible either, and in a whole I think there are some viable options for C++ web applications out there!
Now I guess, I should try out the 2 frameworks, but ... ehm ... you know, so little time, so many books...
--
* see: http://ib-krajewski.blogspot.com/2008/10/c-servlets-again.html
** Gavino, unfortunately his profile isn't available :-(((, but nonetheless, thanks bloke!
*** The Wt-framework ("witty" - http://www.webtoolkit.eu/wt/)
**** SPTK: mainly a GUI framework, but it has some DB access support as well: http://www.sptk.net/index.php
