Today is Document Freedom Day: Roughly 200 teams from more than 60 countries worldwide are organising local activities to raise awareness for Document Freedom and Open Standards.

What does this mean for mean personally? Less than one would expect. I have been advocating the use of Open Document formats (such as ODF) for the past two years already, and try to do so whenever possible.

People react very differntly when I raise this issue. Some appreciate being informed that there are Open Document formats, which guarantee interoperability with everyone, but others tend to tell me “everyone uses [Microsoft] Office, isn’t that format a standard?“. The answer is always the same: NO.

Neither the old proprietary Microsoft Office format, nor the new format, OOXML are standards in my opinion and here is why:

The old format is not documented at all, and no international standards body, such as the ISO, have ever made this format a standard.

The new format, OOXML, which is in the news quite often lately, is being pushed to be made an ISO standard. People often think that, as documentation (which is said to be of poor quality) is available, making this format an international standard would be a good thing.
I am afraid I have to say NO once again here. There are too many references to the old proprietary format, which is a huge no-go for something that should become an international standard.

Also, there already is an international standard for office documents, ODF. In my opinion there is no point in having two separate standards for the same thing and the chance of such a situation causing a lot of havoc is quite good.

So, personally I have to say that I quite often suggested people to switch to OpenOffice.Org lately, instead of buying Microsoft’s latest Office suite. Document Freedom and the use of Free Software are not my main arguments lately, but rather that people switching to OpenOffice.Org now do not have to learn how to use a new user-interface. People are lazy, and this argument works perfectly.

And there is yet another point for using Open Standards in IT:
Think of the Internet and where it would be without Open Standards (and also Free Software). Think of how everything on the Internet would work together. Think of one browser supporting only its own network protocol (which of course would be proprietary) and other browsers only supporting theirs. The Internet would not be what it is today without Open Standards and guaranteed interoperability.

More information about the Document Freedom day can be found in the last news entry over at documentfreedom.org.

To sum the problem I was experiencing up: My anti-spam system (namely Spamassassin) did not detect spam mails anymore.

Now here is the reason it did not: After some more investigation of the problem I noticed that spam emails were received via a local connection (forwarded from fetchmail). However, one of my Exim ACLs says not to scan emails from localhost for spam.

So, the solution might be a hack, but it worked out perfectly. Starting fetchmail with the -S <servername> argument causes it to send emails to the given SMTP server rather than localhost. Using the real hostname of my server caused the “do not scan local mails” not to kick in and all mails received via fetchmail to be scanned again.

Problem fixed.

Now I am wondering why someone sets up SPF for his mail domain when he is in fact sending emails from other IP addresses as well. Time to update your SPF rules Technorati…

Now that my mail server is configured to do sender verification and a few very old mails came from domains or systems which are non-existent today about 50 mails ended up being frozen.

But how to remove all frozen mails from Exim’s queue? I ended up using mailq | grep frozen to get a list of all messages (and more importantly their message IDs) and saved that to a file. I then wrote a minimalistic Python script attached to this article to delete all those messages. Consider the script a quick and dirty hack, but it might come in handy for some of you. Get it here.

I set up my mail server in February and noticed a decrease in spam mail delivered to my mailbox compared to my old system. However, in the past two weeks more and more spam mail has been delivered to my mailbox again. So is it just me, my system or the system’s configuration or is everyone else receiving more spam again?

Anyways, it’s about time to inspect the configuration of my mail system again…

Well, I decided it was time to give a few IDEs a try. There were only two requirements I had: the IDE has to be Free Software and it has to run on GNU/Linux.

If you are planning to read on please be aware that this was no real test, but rather contains my observations regarding the IDEs I have tested, what I liked and did not like and if one surprised me enough to actually use it instead of my good old plain GNU Emacs.

Eclipse

As Java development in school is done with Eclipse and all teachers are more than happy with that program I gave it a try first. I head that there was some sort of Python IDE plugin and so I downloaded Eclipse 3.3. After a few problems keeping the bugger running for more than 5 minutes (seems like the default memory-usage configuration did not provide Eclipse with enough memory) I started downloading the PyDev plugin using the internal plugin download manager. This worked quite smoothly, however, it seemed a bit slow.
Now PyDev looks quite neat, but without the proprietary PyDev extensions it is rather useless and GNU Emacs gives me pretty much the same features.

CONCLUSION: Bloated, using a huge load of memory (Eight-Megabyte-And-Constantly-Swapping joke comes to my mind again), not offering a lot more features than GNU Emacs without proprietary PyDev Extensions.

OpenKomodo

After reading this post on lwn.net about OpenKomodo (note: the post says Komodo Edit, but that’s proprietary software) and how it supports Python I gave it a try. I built it from Subversion trunk, which took some time. OpenKomodo is based on Mozilla and Gecko and if you ever built Firefox from source you should know that you can go and grab quite a few coffees while waiting for the build to finish.
The build system seems to be one specifically written for this application and so is a bit weird to use for people use to either GNU Autotools or Python’s distutils. After the build process finished I was unable to find a way to install the application. The documentation only contains a note about using the build tool (black, “bk”) with the “run” argument to start OpenKomodo.
At first everything looked quite nice. It supports Python quite well, including limited auto-completion support and so on and also supports, just like you would expect, tabbed-editing. After playing around in the source tree of one of my projects and trying to get used to “normal” keyboard shortcuts, such as Ctrl+s for saving a file, I had quite a few tabs open.
You probably know that having a lot of tabs open just leads to confusion and so I wanted to close all tabs but the currently active one and oops: that feature does not exist.
I then digged into the OpenKomodo source, added that feature, prepared a patch and tried to get it into the trunk: without luck as it seems. As noted in a comment to my bug report such features should go into extensions. As I am too lazy to write an extension just for this small patch and basic feature I am still trying getting the changes into trunk.
However, I abandoned OpenKomodo, as I found something better. First to my conclusion though:

CONCLUSION: Nice editor, but like Eclipse, quite bloated as it is based on Mozilla (memory leaks anyone?). Compile time is bad, again, because it is based on Mozilla. Getting simple patches into its trunk also seems to be a problem.

PIDA

I do not remember how or where I stumbled accross a reference to PIDA, but it sounded interesting. PIDA is a Python IDE, built using Python, with a lot of features.
Even though you cannot see this on the screenshots on the PIDA homepage it does not include its own editor. It rather makes use of an existing editor. It currently can embed either vim or GNU Emacs (you need CVS version 23.x or newer). As I was using GNU Emacs before this really caught my attention. I downloaded PIDA from mercurial and built it. Build time is less than 5 minutes on my machine, which is more than acceptable.
When starting PIDA for the first time it asks which editor you want to embed. I obviously chose Emacs there.
It seems like embedding Emacs is in an early stage right now. Even though everything seems to work PIDA embeds the whole GNU Emacs (GTK version) window, including the menu bar and the toolbar. This generates a weird look, as you have two menu- and toolbars, one belonging to PIDA and one belonging to GNU Emacs. No problem for me though, as it is rather a style-problem than a real one.
After opening up one of my projects I immediately noticed one thing: version control integration. I can confirm that Subversion is properly supported and works perfectly. Including reverting of files, updating the local copy, committing changes and viewing differences. This indeed is a great feature and I like it.
I played around a bit more and stumbled across the plugins. There are quite a few neat plugins, like a Trac integration plugin which allows you to view tickets inside the IDE or a TODO parser plugin, which parses comments containing “TODO:” or “XXX:” from files and gives you references to them.
Another useful plugin seems to be the Python Source Viewer, which displays all functions, classes and methods present in the current python file in a tree view.

CONCLUSION: The IDE I am most likely going to use for now. Why? Because it seems to be lightweight, uses GNU Emacs as embedded editor and comes with a proper feature set. I suggest everyone, even hardcore GNU Emacs users, to give PIDA a try. It looks worth it.

For those of you who are now wondering what nwu is or could be, I did write about nwu on this weblog already and the ‘nwu – an introduction’ post should give you a good idea of what it is.

So, what has changed recently? Basically I merged my changes back into trunk, which means that most of these things are going to be used now. This means that the application framework, the scheduler, the APT “Packages” file parser, support for gzip compression in both the SecureXMLRPC client and server and the brand-new RPC framework are either already being used, or are going to be used soon.

Except for the RPC framework, which would need to be adapted, and the application framework, which depends on nwu.common.config, all these pieces of code also work stand-alone and can be used in other python applications too.

This article is going to explain the stand-alone nwu.common.* modules and their function.

nwu.common.SecureXMLRPC

I would like to start out introducing my “baby”: SecureXMLRPC. As the name suggests it provides support for XML-RPC over https. You might think “but that’s present in Python’s SimpleXMLRPCServer and xmlrpclib already” and you are right about that. However, the Python implementation uses OpenSSL, whilst SecureXMLRPC provides both a server and client implementation using GnuTLS. Even though the OpenSSL version works perfectly fine you cannot link it to GPL-licensed code due to the OpenSSL license being incompatible to the GPL.
Furthermore SecureXMLRPC supports compression of the XML-RPC payload and as of today is aware of multi-threading.

nwu.common.config

The config module provides a single class, “Config”. It basically is a slightly improved version of Python’s SafeConfigParser class and allows passing a “default value” argument to its get() method which is returned in case the setting is not present in the config.

nwu.common.app

This is a simple application framework. It makes use of “nwu.common.config” for reading the config file. However, its main feature is a simple-to-use command-line parser, which parses not only arguments from the command-line (such as –do-something, -d or –configfile=filename), but also supports “commands” as they can be found in several applications, such as aptitude. This way it is easy to not only create top-level commands (like “aptitude search <package name>”), but also nested command structures (like “program computer get <computer name>”, “program computer list“, etc.).

nwu.common.certtool

The certtool module is a wrapper around GnuTLS’ certtool application. It allows you to create private keys, certificate authorities, sign certificates, created certificate-signing-requests (CSRs)  and sign such.
However, please note that it currently does not implement the full functionality certtool provides but rather only the pieces nwu needs.

nwu.common.aptmethod

nwu.common.aptmethod simplifies implementation of an APT transport/method in Python. It takes care of all communication done with apt itself and thus gives you the opportunity to implement a new method in a simple way.

nwu.common.apt

Finally there is also the apt module. This module currently contains code to parse APT Packages files.

nwu.common.scheduler

One of the latest additions to the common module. It provides a way of scheduling tasks for execution at a certain date/time and executing a task in a given interval.