RDF, printf() Debugging, Testing

When the large distributed system going gets tough, the tough add printf calls. And when the tough can no longer keep track of all the printf calls, the tough start calling the printfs a "Trace System" and add it to the development schedule. We're using log4net instead of printf, but the principle is the same.

Large program traces can get tough to read, so you generally end up writing little tools to help process them. It's enough of a problem that the supercomputing people (big fans of parallel programming) have come up with trace-file format standards and a slew of utilities for dealing with them.

Sadly, what I needed (dumping application state as well as program trace info) didn't fit well into the existing standards, so I was stumbling along with ad-hoc log messages that I post-processed with awk. I was in the middle of writing a tiny app to graph the post-processed output and show object linkages when I realized I was being silly. There was already a perfect data format with tons of associated filtering tools and visualizers: RDF.

Although I sometimes rant against the more extreme goals of the Semantic Web effort, I find some of the underlying technology useful. RDF can be a pretty nifty tool if you don't let it go to your head.

It worked great. Using RDF doesn't allow anything I couldn't have in theory done before, but it means I can take advantage of tons of existing tools and expertise. (You can buy a book on RDF, but you're never going to buy a book on whatever internal debug format you come up with.) I've been processing the RDF with cwm and viewing it with IsaViz and Welkin, and there are many other tools I haven't had a chance to try yet.

I'm certainly not the first person to think of this (it looks like the W3C uses RDF as the format for test results for conformance suites), but I haven't seen a lot of buzz about it. I think that's a shame.

I've imposed a one-week moratorium on any further RDF research (it was getting a little out of hand), but I think when the time's up I'm going to investigatge using N3 rules to validate test output (dump state, do an operation, re-dump state, compare states with reasoning engine, etc).

Has anybody out there heard of any existing efforts along these lines? Google turns up tons of references, but none seemed to match exactly...

tags: printf, debugging, trace, parallel, distributed, software, development, rdf, semantic, web, september, 2005, n3