epic fail!

When it comes to application development, almost every time there is the need to store data persistently between application launches. Setting up a relational database schema is one thing, but maintaining the relation between the python objects and the database tables is a serious task. Luckily for me being a opportunistic developer, there are many frameworks which aim at scratching that itch. In fact there are so many that I am having trouble chosing.

I tried one or two, but always found myself somewhat constrained, especially when it comes to schema evolution or trying to trigger gui updates from data events. So a little search over google code tells me that basically all the projects that make use of these frameworks are web applications. Canonical even developed their own for the use with launchpad et all.

So my question to the world is twofold:

1) What do existing python desktop applications use? Do they all take care of the “M” in the MVC themselves? I find that hard to believe. It appears to me as I take one step back that actually I do not a database. I need persistent objects. Exaile uses shelve, but is does not look very handy to use.

2) What shall I as an opportunistic developer use? Well… the opportunistic echo is couchDB, but even in the projects intro page it says quite clearly that it is not what I want.

It hasn’t gotten much attention in the ubuntu universe yet, but Google announced ChromeOS. The sparse information available tells you that it will be a google chrome browser running on top of a linux kernel running webapps. It will be free. As in speech.

It is targetet towards netbooks and “web oriented use”. All the apps will not be tied to the chrome browser, but will run in any other standard conformant browser.

Now we all know that google has already a decent set of web applications that mean to replace long existing tools (Office, webmail, calendar, instant messaging). The (german) online media sees this as the next big attack on microsoft, targeting windows.

This is where they lose me. For ChromeOS to replace windows (or any other currently used desktop OS), it either has to offer the same set of applications or all the users will use their PCs “web oriented” exclusivley in the future. I don’t see any of this happening.

So what is in it for “us”? Just some additional webapps that we can use or not? A fast OS that we can run on our netbooks? A new approach to computing that will make our current OS obsolete?

The coders among us geeks will probably agree: The fuel that gets us going is the passion the create something new and see something grow that I created myself. Fuelled by that energy, you forget time, space and basic human needs while hacking on a cool feature. One catchy thing about passion is that it cannot be forced or bought. That is why software development for a commercial product is often not as fulfilling as hacking at home, in your private space and by your rules. What is the consequence of this?

You only contribute your time to the area that you enjoy most, maximizing the scratch for your geek itch. The outcome?

You start several promosing projects, only to find yourself switching to the next one when your previous milestone has been reached or simply another topic offers something more “sexy” to work on than your current one. This behavior is perfectly okay. You don’t get paid, you do it in your free time and for fun. So if you maximize your fun, you use coding exactly in the way you wanted to, you have the most fun possible.

So by this you get to know new technology, your software design skills emerge and you replenish your energy with rewarding “it works!!” moments. Unfortunately, this does not ultimately lead to better software, because good software is far more than coded features.
You have to do documentation, bug triaging (filing first would be good), give some love to your release page/space.. Oh, no, wait! I went too fast. *Before* doing all that you need to define release critical milestones, agree on a common goal for your release and drive the development into a road where it is actually targeted towards your release milestones, i.e. not working on features that are for the next release (because they are so big they essentially break the whole functionality until they are finished). And there is our main point! You have to *drive* the development. That means, restraining it from its pure hedonistic purpose of maximizing the fun of the coder towards getting the coder to code something that is not in his focus of interest.

Now you could argue that the focus of the coder is to produce quality software that people can use, but out of my experience I have to say that this is too optimistic. You want fast and sexy results, rushes of adrenaline and “whuhuuu!” moments rather than editing 200 docstrings or writing an tutorial on how to use a special function.

So why am I whining about this? The reason is that I have hit rock-bottom with my dearest pet project. It started of quick and we could “release” 0.1 quite early. The reason for that is that it was clear that the choices where either to redesign the codebase of stop doing it. We did the former and were quickly back to what 0.1 provided with beeing able to extend it so easily and really “get going”. It was then that there was a magical moment where the enthusiasm just faded. We were near to getting 0.2 ready, but deep inside i wanted to start hacking on post 0.2 features rather than finishing up that release. On the other hand it was clear that “release early, release often” (metalink ) is not just phrase, but a necessity if you want to draw peoples attention to what we are doing. Especially if we are talking about a framework.
So i tried *really* hard to get to the point:

• I created bug reports for the 2 main showstopper bugs
• I set up a webpage for the project
• I wrote tutorials, documented the code, posted screenshots, I even released the thing although I know that there is still much work to be done.

Now I just don’t know what else to do. When you browse the intarwebs you often come along webistes of projects that are deprecated or no longer beeing developed. But they were active once. They had a (vibrant) community around them pushing the next release.

I want to learn something out of this:
What can you do to prevent this? I mean there are many examples where small groups of people or even single persons are producing great software for many release cycles.
How do you actually drive the release of an starting open source project? There is no pressure. There is no way telling people what to do.
Are the great projects that we see and use every day really just the gems that survived the strong selection of time and motivation? That would be a hard lesson for me to learn…

Normally I don’t like those memes because they don’t deliver any valuable information, but that’s not my point. This one is different because you can look at the very recently taken photos of the people and compare them to their hackergotchi on the planet. We notice a few things:

Richard Johnson apparently never smiles.
Greg Grossmeier apparently only owns one wool cap.

What else do you notice?

[Update]I just figured out I had a terrible bug in the gedit plugin which caused the wizardy to work only once. This should be fixed now in 1.0b2[Update]

[Update]I find this script useful enough, so I created a gedit plugin for it and hosted is on google code[Update]

I work on a few little projects. When I get going I focus solely on the new feature or on squashing that bug, but not on publishable code. This is a flow in my working process but just who I am. The consequence of that behavior is: I neglect proper documentation. And by proper documentation I mean epydoc conforming docstrings.

When the time comes and you want to release something, especially when you want people to look at your code and work with it, you realise that you need proper documentation. So I sat down and started to write the docstrings. Writing documentation can actually be fun! What certainly is NO fun is writing the overhead: “”" to start the docstring and end it and @param and @type for each variable. There must be a way to avoid this boring and purely repetitive task. So I searched the intarwebs and, to my surprise, found nothing. While I still refuse to believe that everybody writes their docstrings on the fly, or uses an IDE das generates the stubs, my buddy and me also refused to write all the donkey code ourselves. So we started what I want to share with you:

A python script that inserts docstring stubs into a file. We used the oppurtunity to revive our regular expression skills. (Kudos to Kodos! If you want to develop a regex on the fly, Kodos is for you…).

So here goes:

import re

# fileNames to process
baseFile = "Game.py"
outputFile = "Game_enhanced.py"

# configuration parameters
# method parameters to skip
skip = ["self"]
# number of spaces to insert for identation
ident = 4

# we have to keep track of how many chars we already inserted because
# we are modifying the string while processing it
inserted = 0

def insert(original, new, pos):
    “”"
    inserts new into original at pos
    @type original: string
    @type new: string
    @type pos: integer
    “”"
    return original[:pos] + new + original[pos:]

def newline(current_ident):
    “”"
    returns \n with current_ident + global ident spaces attached
    @type current_ident: integer
    @param current_ident: the identation of the line we are wrapping
    “”"
    global ident
    res = “”
    for i in range(0,current_ident + ident):
        res += ” ”
    return “\n” + res

# regular expressions
method_pattern = re.compile(’.*def .*.’)
variable_pattern = re.compile(’[(].*[)]‘)
variable_name_pattern = re.compile(’,')
docstring_pattern = re.compile(’\s*”"”‘)

# read in the file to be processed
f = open(baseFile, “r”)
text = f.read()

# find all methods
method_iter = method_pattern.finditer(text)

# process each method
for item in method_iter:
    # the position of the docstring
    # here we have to take into account what has already been inserted
    pos = item.end() + inserted
    # the identation of the method is the distance from the beginning of the
    # line till the “d”
    current_ident = item.group().find(”d”)
    # test if there already is a docstring: the next non-whitespace should be “”"
    if docstring_pattern.match(text[pos:]):
        # debug/info notice on the konsole
        #print item.group(), ” already has a docstring”
        pass
    else:
        variables = []
        #search vor the (…) section
        vars = variable_pattern.search(item.group())
        vars = vars.group()
        # get rid of the paranthesis
        vars = vars.replace(’(',”)
        vars = vars.replace(’)',”)
        # split the variables by commata
        for var in variable_name_pattern.split(vars):
            # get rid of spaces
            var = var.strip()
            # and use only the first word, we get rid of standard values
            var = var.partition(” “)[0]
            # test for exclusion
            if not var in skip:
                variables.append(var)
        # generate the docstring
        docstring = newline(current_ident) + ‘\”\”\”‘ + newline(current_ident)\
                    + ‘FIXME’ + newline(current_ident)
        # add the @ lines
        for var in variables:
            docstring += ‘@param ‘+var+’:’ + newline(current_ident)\
                        +’@type ‘+var+’:’ + newline(current_ident)
        # close it
        docstring += ‘\”\”\”‘

        #insert the docstring into the text
        text = insert(text, docstring, pos)
        # record how many chars have been inserted
        inserted += len(docstring)

# open the output file
fo = open(outputFile, ‘w’)
# write the output
fo.write(text)

Now the trained eye notices a couple of things:

• It is neither very clever or complicated. All it does is search for a method declaration, check if a docstring is present, parse the variable names (stripping of keywords), generate a docstring with @param and @type lines and paste it to the text.
• It completely ignores @return and @rtype stuff because when considering nested functions we quickly came to the insight that detecting returns is not trivial. The “return” of a function does neither have to be before the next “def” nor does the identation give any sufficient info. As this is the only information our script gives us, we skipped that. In order to properly detect the return stuff we would have to parse the whole method body (including any inner methods, i.e. we would have to parse everything till the ident matches the ident of the def again) and make a judgement which return belongs to which def based on all the idents in between those two. If between a def and a return there is another def, then the return belongs to the first if the ident of the return is lower than the ident of the second def.
• It uses spaces and no tabs to do idents
• It is not a usable module. Ideally, the thing would be a module which would provides a method to pass a list of files and some configuration stuff to.
• Again: It’s simple!

Comments and suggestions are very welcome. And I would love to have a “D’Oh!!” moment when somebody points me to where we could have achieved that in 5 minutes time…

I like to play an occasional game of dota, the famous Warcraft III - The Frozen Throne mod, with my friends. It works perfectly with wine since I can remember (so, probably feisty). You don’t even have to install it using wine, just launch your windows installation and your fine. In fact, I kept my initial Warcraft III installation all the time in my ~/.wine directory and never had any problems.

Yesterday, a buddy of mine asked me to play a round. Not to note that my perfomance would be rusty at best, upon starting up the game my screen went black and told me it was “out of range”. This usually happens when the refresh rate is higher than the monitor can handle. I tried several tweaks in the registry, but no luck. I then stumbled upon this bug in launchpad against xorg that is excactly the same problem.

This is where I get confused: This bug has been openend quite some time ago and also some people have said that they are experiencing the same problem. Yet, the bug is in status “incomplete”. I commented that I have the same issue and attached my XOrg log. Now my question: Is that all I can do? What is the normal lifecycle of a bug? Is this problem just too uncommon, too little? What could I (as an unexperienced “bug triager”) do to help?