Artikel getaggt mit programming

PyGObject 3.7.92 released

I just released a new PyGObject for GNOME 3.7.92. This fixes a couple of crashes and marshalling errors again, but most importantly got a change to automatically mute the PyGIDeprecationWarnings for stable versions. Please run pythonX.X with the -Wd option to still be able to see them.

We got through all our bugs that were milestoned for GNOME 3.8 and don’t want to or plan to introduce any major behavioural change at this point, so barring catastrophes this is what will be in GNOME 3.8.0.

Thanks to all contributors!

  • Fix stack smasher when marshaling enums as a vfunc return value (Simon Feltman) (#637832)
  • Change base class of PyGIDeprecationWarning based on minor version (Simon Feltman) (#696011)
  • autogen.sh: Source gnome-autogen to fix out of source builddir (Alban Browaeys) (#694889)
  • pygtkcompat: Make gdk.Window.get_geometry return tuple of 5 (Simon Feltman)
  • pygtkcompat: Initialize hint to zero in set_geometry_hints (Simon Feltman)
  • Remove incorrect bounds check with property helper flags (Simon Feltman)
  • Fix crash when setting property of type object to an incorrect type (Simon Feltman) (#695420)
  • Remove skipping of object property tests (Simon Feltman) (#695420)
  • Give more informative error when setting property to incorrect type (Simon Feltman) (#695420)

Tags: , , , , , , , ,

Automatically generating documentation from GIR files

Many libraries build a GObject introspection repository (*.gir) these days which allows the library to be used from many scripting (Python, JavaScript, Perl, etc.) and other (e. g. Vala) languages without the need for manually writing bindings for each of those.

One issue that I hear surprisingly often is “there is zero documentation for those bindings”. Tools for building documentation out of a .gir have existed for a long time already, just far too many people seem to not know about them.

For example, to build Yelp XML documentation out of the libnotify bindings for Python:

  $ g-ir-doc-tool --language=Python -o /tmp/notify-doc /usr/share/gir-1.0/Notify-0.7.gir

Then you can call yelp /tmp/notify-doc to browse the documentation. You can of course also use the standard Mallard tools to convert them to HTML for sticking them on a website:

  $ cd /tmp/notify-doc
  $ yelp-build html .

Admittedly they are far from pretty, and there are still lots of refinements that should be done for the documentation itself (like adding language specific examples) and also for the generated result (prettification, dynamic search, and what not), but it’s certainly far from “nothign”, and a good start.

If you are interested in working on this, please show up in #introspection or discuss it on bugzilla, desktop-devel-list@, or the library specific lists/bug trackers.

Tags: , , , , ,

PyGObject 3.7.2 released

just released a new PyGObject, for GNOME 3.7.2 which is due on Wednesday.

In this version PyGObject went through some major refactoring: Some 5.000 lines of static bindings were removed and replaced with proper introspection and some overrides for backwards compatibility, and the static/GI/overrides code structure was simplified. For the developer this means that you can now use the full GLib API, a lot of which was previously hidden by old and incomplete static bindings; also you can and should now use the officially documented GLib API instead of PyGObject’s static one, which has been marked as deprecated. For PyGObject itself this change means that the code structure is now a lot simpler to understand, all the bugs in the static GLib bindings are gone, and the GLib bindings will not go out of sync any more.

Lots of new tests were written to ensure that the API is backwards compatible, but experience teaches that ther is always the odd corner case which we did not cover. So if your code does not work any more with 3.7.2, please do report bugs.

Another important change is that if you build pygobject from source, it now defaults to using Python 3 if installed. As before, you can build for Python 2 with PYTHON=python2.7 or the new --with-python=python2.7 configure option.

This release also brings several marshalling fixes, docstring improvements, support for code coverage, and other bug fixes.

Thanks to all contributors!

Summary of changes (see changelog for complete details):

  • [API change] Drop almost all static GLib bindings and replace them with proper introspection. This gets rid of several cases where the PyGObject API was not matching the real GLib API, makes the full GLib API available through introspection, and makes the code smaller, easier to maintain. For backwards compatibility, overrides are provided to emulate the old static binding API, but this will throw a PyGIDeprecationWarning for the cases that diverge from the official API (in particular, GLib.io_add_watch() and GLib.child_watch_add() being called without a priority argument). (Martin Pitt, Simon Feltman)
  • [API change] Deprecate calling GLib API through the GObject namespace. This has always been a misnomer with introspection, and will be removed in a later version; for now this throws a PyGIDeprecationWarning.
  • [API change] Do not bind gobject_get_data() and gobject_set_data(). These have been deprecated for a cycle, now dropped entirely. (Steve Frécinaux) (#641944)
  • [API change] Deprecate void pointer fields as general PyObject storage. (Simon Feltman) (#683599)
  • Add support for GVariant properties (Martin Pitt)
  • Add type checking to GVariant argument assignment (Martin Pitt)
  • Fix marshalling of arrays of struct pointers to Python (Carlos Garnacho) (#678620)
  • Fix Gdk.Atom to have a proper str() and repr() (Martin Pitt) (#678620)
  • Make sure g_value_set_boxed does not cause a buffer overrun with GStrvs (Simon Feltman) (#688232)
  • Fix leaks with GValues holding boxed and object types (Simon Feltman) (#688137)
  • Add doc strings showing method signatures for gi methods (Simon Feltman) (#681967)
  • Set Property instance doc string and blurb to getter doc string (Simon Feltman) (#688025)
  • Add GObject.G_MINSSIZE (Martin Pitt)
  • Fix marshalling of GByteArrays (Martin Pitt)
  • Fix marshalling of ssize_t to smaller ints (Martin Pitt)
  • Add support for lcov code coverage, and add a lot of missing GIMarshallingTests and g-i Regress tests. (Martin Pitt)
  • pygi-convert: remove deprecated GLib → GObject conversions (Jose Rostagno)
  • Add support for overriding GObject.Object (Simon Feltman) (#672727)
  • Add –with-python configure option (Martin Pitt)
  • Do not prefer unversioned “python” when configuring, as some distros have “python” as Python 3. Use Python 3 by default if available. Add –with-python configure option as an alternative to setting $PYTHON, whic is more discoverable. (Martin Pitt)
  • Fix property lookup in class hierarchy (Daniel Drake) (#686942)
  • Move property and signal creation into _class_init() (Martin Pitt) (#686149)
  • Fix duplicate symbols error on OSX (John Ralls)
  • [API add] Add get_introspection_module for getting un-overridden modules (Simon Feltman) (#686828)
  • Work around wrong 64 bit constants in GLib Gir (Martin Pitt) (#685022)
  • Mark GLib.Source.get_current_time() as deprecated (Martin Pitt)
  • Fix OverflowError in source_remove() (Martin Pitt) (#684526)

Tags: , , , , , , ,

libxklavier is now introspectable

On my 8 hour train ride to Budapest last Sunday I finally worked on making libxklavier introspectable. Thanks to Sergey’s fast review the code now landed in trunk. I sent a couple of refinements to the bug report still, but those are mostly just icing on the cake, the main functionality of getting and setting keyboard layouts is working nicely now (see the example script).

Tags: , , , , ,

Improved PyGI documentation

As a followup action to my recent Talk about PyGI I now re-used my notes to provide some real wiki documentation.

It would be great if you could add package name info for Fedora/SUSE/etc., and perhaps add more example links for porting different kinds of software! Please also let me know if you have suggestions how to improve the structure of the page.

Tags: , , , , , , , ,

Na zdraví PyGI!

(Update: Link to Tomeu’s blog post, repost for planet.gnome.org)

Last week I was in Prague to attend the GNOME/Python 2011 Hackfest for gobject-introspection, to which Tomeu Vizoso kindly invited me after I started working with PyGI some months ago. It happened at a place called brmlab which was quite the right environment for a bunch of 9 hackers: Some comfy couches and chairs, soldering irons, lots of old TV tubes, chips, and other electronics, a big Pirate flag, really good Wifi, plenty of Club Mate and Coke supplies, and not putting unnecessary effort into mundane things like wallpapers.

It was really nice to get to know the upstream experts John (J5) Palmieri and Tomeu Vizoso (check out Tomeu’s blog post for his summary and some really nice photos). When sitting together in a room, fully focussing on this area for a full week, it’s so much easier to just ask them about something and getting things done and into upstream than on IRC or bugzilla, where you don’t know each other personally. I certainly learned a lot this week (and not only how great Czech beer tastes :-) )!

So what did I do?

Application porting

After already having ported four Ubuntu PyGTK applications to GI before (apport, jockey, aptdaemon, and language-selector),
my main goal and occupation during this week was to start porting a bigger PyGTK application. I picked system-config-printer, as it’s two magnitudes bigger than the previous projects, exercises quite a lot more of the GTK GI bindings, and thus also exposes a lot more GTK annotation and pygobject bugs. This resulted in a new pygi s-c-p branch which has the first 100 rounds of “test, break, fix” iterations. It now at least starts, and you can do a number of things with it, but a lot of functionality is still broken.

As a kind of “finger exercise” and also to check for how well pygi-convert works for small projects now, I also ported computer-janitor. This went really well (I had it working after about 30 minutes), and also led me to finally fixing the unicode vs. str mess for GtkTreeView that you got so far with Python 2.x.

pygobject and GTK fixes

Porting system-config-printer and computer-janitor uncovered a lot of opportunities to improve pygi-convert.sh, a big “perl -e” kind of script to do the mechanical grunt work of the porting process. It doesn’t fix up changed signatures (such as adding missing arguments which were default arguments in PyGTK, or the ubiquitous “user_data” argument for signal handlers), but at least it gets a lot of namespaces, method, and constant names right.

I also fixed three annotation fixes in GTK+. We also collaboratively reviewed and tested Pavel’s annotation branch which helped to fix tons of problems, especially after Steve Frécinaux’s excellent reference leak fix, so if you play around with current pygobject git head, you really also have to use the current GTK+ git head.

Speaking of which, if you want to port applications and always stay on top of the pygobject/GTK development without having to clutter your package system with “make install”s of those, it works very well to have this in your ~/.bashrc:

export GI_TYPELIB_PATH=$HOME/projects/gtk/gtk:$HOME/projects/gtk/gdk
export PYTHONPATH=$HOME/projects/pygobject

Better GVariant/GDBus support

The GNOME world is moving from the old dbus-glib python bindings to GDBus, which is integrated into GLib. However, dbus-python exposed a really nice and convenient way of doing D-Bus calls, while using GDBus from Python was hideously complicated, especially for nontrivial arguments with empty or nested arrays:

from gi.repository import Gio, GLib
from gi._gi import variant_type_from_string

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

vb = GLib.VariantBuilder()
vb.init(variant_type_from_string('r'))
vb.add_value(GLib.Variant('s', 'test'))
vb.add_value(GLib.Variant('u', 1))
vb.add_value(GLib.Variant('s', 'gtk-ok'))
vb.add_value(GLib.Variant('s', 'Hello World!'))
vb.add_value(GLib.Variant('s', 'Subtext'))
# add an empty array
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('as'))
vb.add_value(eavb.end())
# add an empty dict
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('a{sv}'))
vb.add_value(eavb.end())
vb.add_value(GLib.Variant('i', 10000))
args = vb.end()

result = notify.call_sync('Notify', args, 0, -1, None)
id = result.get_child_value(0).get_uint32()
print id

So I went to making the GLib.Variant constructor work properly with nested types and boxed variants, adding Pythonic GVariant iterators and indexing (so that you can treat GVariant dictionaries/arrays/tuples just like their Python equivalents), and finally a Variant.unpack() method for converting the return value of a D-Bus call back into a native Python data type. This looks a lot friendlier now:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

args = GLib.Variant('(susssasa{sv}i)', ('test', 1, 'gtk-ok', 'Hello World!',
    'Subtext', [], {}, 10000))
result = notify.call_sync('Notify', args, 0, -1, None)
id = result.unpack()[0]
print id

I also prepared another patch in GNOME#640181 which will provide the icing on the cake, i. e. handle the variant building/unpacking transparently and make the explicit call_sync() unnecessary:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

result = notify.Notify('(susssasa{sv}i)', 'test', 1, 'gtk-ok', 'Hello World!',
            'Subtext', [], {}, 10000)
print result[0]

I hope that I can get this reviewed and land this soon.

Thanks to our sponsors!

Many thanks to the GNOME Foundation and Collabora for sponsoring this event!

Tags: , , , , , , , , , ,

Creating an HTTPS server in Python

For a test suite I need to create a local SSL-enabled HTTPS server in my Python project. I googled around and found various recipes using pyOpenSSL, but all of those are quite complicated, and I didn’t even get the referenced one to work.

Also, Python has shipped its own built-in SSL module for quite a while. After reading some docs and playing around, I eventually got it to work with a remarkably simple piece of code using the builtin ssl module:

import BaseHTTPServer, SimpleHTTPServer
import ssl

httpd = BaseHTTPServer.HTTPServer(('localhost', 4443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, certfile='path/to/localhost.pem', server_side=True)
httpd.serve_forever()

(I use port 4443 so that I can run the tests as normal user; the usual port 443 requires root privileges).

Way to go, Python!

Tags: , , , , , ,

GTK 3.0/GIR application porting: Successes and problems

GNOME 3.0 and Ubuntu Natty are currently undergoing a major architectural shift from GTK 2.0 to 3.0. Part of this is that the previous set of manually maintained language bindings, such as PyGTK, are being deprecated in favor of GObject Introspection, a really cool technology!

For us this means that we have to port all our PyGTK applications from PyGTK 2 to gobject-introspection and GTK 3.0 at the same time. I started with that for my own projects (Apport and Jockey) a few days ago, and along the way encountered a number of problems. They are being fixed (particular thanks to the quick responsiveness of John Palmieri!), so I guess after a few of those iterations, porting should actually become straight forward and solid.

So now I’m proud to announce Apport 1.16 which is now fully working with GTK 3.0 and pygobject-introspection. I just uploaded it to Ubuntu Natty, where it can get some wider testing. I also have a pygi/GTK3.0 branch for Jockey, which is also mostly working now, but it’s blocked on the availability of a GIR for AppIndicator. Once that lands, I’ll release and upload the Jockey as well.

For other people working on porting, these are the bugs and problems I’ve encountered:

  • [pygobject] GtkMessageDialog constructors did not work at all. This was fixed in pygobject 2.27, so it works fine in Natty, but there is little to no chance of making these work in Ubuntu 10.10.
  • [pygobject] Gtk.init_check() crashes (upstream bug). It’s not strictly necessary, though, just avoids crashes (and crash reports) when calling it without a $DISPLAY. I’ll put that back once that gets fixed.
  • [pygobject] You can’t pass unicode objects as method arguments or property values (upstream bug). Method arguments were recently fixed in upstream git head, and I backported it to the Natty package, so these work now. Property values are still outstanding. The workaround for both is to convert unicode values to bytes with .encode('UTF-8') everywhere.
  • [pygobject] pygi-convert.sh is a great help which automates most of the mechanical rewriting work and thus gets you 90% of the porting done automatically. It’s currently missing MessageDialog constants, and is a bit inconvenient to call. I sent two patches upstream (upstream bug) which improve this.
  • [GIR availability] libnotify did not build a GIR yet (upstream bug). This was fixed in upstream git head with some contributions of mine, and I uploaded it to Natty. This works quite well now, although add_action() crashes on passing the callback. This is still to be investigated.
  • [GIR availability] There is no Application Indicator GIR, as already mentioned. Our DX team and Ken VanDine are working on this, so this should get fixed soon.
  • [GTK] Many dialogs now scale in a very ugly manner: Instead of resizing the contents, the dialog just grows huge outer padding. This is due to a change of the default “fill” property, and apparently is not fully understood yet (upstream bug). As a workaround I explicitly added the fill property to the top-level GTKBox in my GTKBuilder .ui files. (Note that this happens in C as well, it’s not a GI specific bug.)
  • [GTK] Building GtkRadioButton groups is currently a bit inconvenient and requires special-casing for the first group member, due to some missing annotations. I sent a patch upstream which fixes that (upstream bug).
  • [GTK] After initial porting, a lot of my dialogs got way too narrow with wrapped labels and text, because GTK 3 changed the behaviour and handling of widget and window sizes. This is intended, not a bug, but it does require some adaptions in the GtkBuilder files and also in the code. The GTK 2 → 3 migration guide has a section about it.

Despite those, I’m impressed how well gobject-introspection already works. It was relatively painless for me to generate a GIR from a libnotify (thanks to the annotations already being mostly correct for building the documentation) and use it from Python. In another couple of months this will be rock solid, and porting our bigger pygtk projects like ubiquity and software-center will hopefully become feasible.

Tags: , , , , , , , , , ,

Simple udisks based automount daemon

For an embedded/thin client project without GNOME, KDE, or even full XFCE I needed a small daemon to automount USB sticks. Using the full gvfs/gdu/nautilus or Thunar stack is too heavyweight for my purposes, but a simple udev rule just doesn’t cut it — I need to mount these USB sticks for a particular user (permissions), and also want to do an action like pop up a window with the contents.

This finally provided me the opportunity to write something bigger than just a 10 line demo in Vala (well, it’s not that much bigger admittedly :-) ). Since that is my first real Vala project, it took quite a lot longer than anticipated; some areas of Vala are still a bit underdocumented, e. g. I spent some half an hour trying to find out how to set a result callback for an asynchronous function invocation. Mikkel Kamstrup Erlandsen suggested to just use a wrapper instead which uses yield, which works fine indeed. Mikkel, thanks for bearing with me!

Anyway, here it is: https://launchpad.net/udisks-automounter, complete with a first release, bzr branch (lp:udisks-automounter), and a package for Ubuntu 10.04 LTS in my PPA.

For avoidance of doubt, this won’t ever make sense on a GNOME/KDE/XFCE desktop, which already have their (much better developed) automounting services. But perhaps it’s useful for someone else with similar constraints.

Tags: , , , , , ,

What I do

It’s been a decade ago when I did my first steps with contributing to Free Software, about seven years when I joined Debian, and about 6 with Canonical and Ubuntu. Time for some reflection what I have done over these years!

Distribution Packaging and Maintenance

My first sponsored Debian upload ever was cracklib2, which seriously needed some love and was looking for a new maintainer. So in that upload I managed to close all outstanding bugs. Thanks to my mentor Martin Godisch about providing a lot of guidance for this!

Since then I’ve maintained various packages, where the most popular ones are certainly the free database server “PostgreSQL” (see next section) and the e-book management software “Calibre”.

“Maintaining” by and large means “making it really easy to get and use this software”. This decomposes to:

  • Packaging it in a way that a simple apt-get install makes the software work out of the box (as far as possible)
  • Provide a default configuration/customizations so that it integrates and plays well with the rest of the system; this includes the paths and permissions for log files, log rotation, debconf, configuration file standards, etc.
  • Be the front line for bug reports from users, sort, answer, and de-duplicate them, and either fix them myself, or forward useful bugs to upstream.
  • Providing security updates for stable releases
  • To some extent, help with the development of the software; this gets mostly driven by user demand, and of course my personal interests.

In August 2004 I got employed by Canonical to work full time on Ubuntu, which pretty much turned a hobby into a profession. I never regretted this in the past years, it’s an awesome job to do!

In principle I’m doing the same thing in Ubuntu as well: Bring stuff from developers to the people out there. Except with a different focus, in Ubuntu my daily bread and butter is the GNOME desktop and stuff around (and immediately below) it. And even though after a long day of bug triaging and debugging I feel a bit low-hearted (“50.000 bugs away from perfection”), when I take a step back and see how much the usage of Free Software in the world has grown since 2004, I am very proud of being part of Ubuntu, which certainly has its fair contribution and share in this growth. So what seemed like a crazy idea from Mark back in 2004 actually has made a remarkable progress.

In the beginning of Ubuntu I mostly sent back patches to the Debian bug tracker, but this evolved quite a bit on both sides: These days I try to keep “my” packages in sync and commit stuff directly to Debian, which works very well with e. g. the pkg-utopia team, which is responsible for HAL, udisks, upower, PolicyKit, and related packages. At this point I want to thank Michael Biebl for being such an awesome guy on the Debian side! Also, it seems that Debian has moved a fair bit away from the strong “Big Maintainer Lock” towards team based maintenance, so these days it is easier than ever to commit stuff directly to Debian for a lot of packages, without much fuss.

PostgreSQL

I have done a handful of changes to PostgreSQL, but these mostly concerned easy packaging and crash fixes, nothing out of the ordinary. I’m not really a PostgreSQL upstream developer.

The thing I am really proud of is the postgresql-common package, which is a very nice example what a distro can provide on top of upstream: If you install the upstream tarball, you have to manually care for creating clusters, providing a sensible configuration for them, set up SSL, set up log rotation, etc. With postgresql-common, this is all done automatically. The biggest feature it provides is a robust and automatic way of upgrading between major releases with the pg_upgradecluster tool, which takes care of a dozen corner cases and the nontrivial process of dumping the old cluster and reloading the new one. Also, you can effortlessy run several instances of the same version in parallel, so that you can e. g. have a production and a development instance, or try the new 9.0 RC1 while still running the 8.4 production one. (more details)

Crash Reporting with Apport

This has been a pet peeve of mine pretty much from day 1. Back in the old days, crashes in software were a pain to track down: many crashes are hard to reproduce, it takes ages to get useful information from bug reporters, and a lot of data cannot be recovered any more when you try to reproduce and analyze a crash after the fact.

With the growing demand for QA from both Canonical and Ubuntu, in 2006 I finally got some time to start Apport, which would make all this a lot easier: It intercepts crashes as they happen, collects the data that we as a developer need, and makes it very easy for the user to submit them as a bug report. This is accompanied by a backend service (called “retracers”) which would reprocess the bug reports by taking the core dump, reproducing a chroot with the packages and versions that the reporter had, installing the debugging symbols, and re-running gdb, to produce a fully symbolic stack trace.

See this bug report for how this looks like. Since then, tons of crashes were fixed, way more than we could ever have done “the old way” with asking users to rebuild with “-g -O0″, running gdb, etc.

By today, Apport has grown quite a bit: rich bug reports, per-package hooks, automatic duplication of crashes, interactive GUI elements in hooks, etc.

Plumbing Development

Handling hotpluggable hardware has always interested me, since the day when I got my first USB stick and it was ridiculously hard (from an user perspective) to use it:

    $ su -
    # mount -t vfat -o uid=1000 /dev/sda1 /mnt

My first go at this was to write pmount which would allow normal users to mount hotpluggable storage without root privileges and worrying about mount paths and options, and then integrate it into GNOME and HAL. Personally I abandoned it years ago, but it seems other people still use it, so I’m glad that Vincent Fourmond took over the maintenance now.

Since then, the entire stack evolved quite a bit: HAL grew to something useful and rather secure, and finally into some monstrous unmaintainable beast, which is why it was declared dead in 2008, and replaced with the “U” stack: udev, udisks, upower, etc. I enjoy hacking on that a lot, and since it’s part of my Desktop Team Tech Lead/Developer role in Canonical, I can spend some company time on it. so far I worked on bug fixes, small new features, and writing a rather comprehensive test suite for udisks (see my udisks commits so far). I also did a fair share of porting stuff away from HAL to the new stack, including some permanent commitments like maintaining the keymaps in udev.

GNOME

Before I started with Canonical I was never much of a GUI person: I was fully content with fvwm and a few xterms around it. But as an Ubuntu developer I do dogfooding, and thus I switched to GNOME for my day to day work. It didn’t take long before I really fell in love with it!

Similar to my Debian packages, my upstream involvement with GNOME is mostly integration and bug fixing. As already explained, we get a looooot of bug reports, so my focus is mostly on bug fixing. To date, I sent 93 patches to bugzilla. Since January 2010 I became a committer, so that it’s easier for me to get patches upstream.

Debugging problems and fixing bugs is a pretty tedious task, but I still enjoy the rewarding nice feeling when you finally tracked down something and can close a bug with 50 duplicates, and you have made people’s life a little bit easier from now on.

Tags: , , , , , , ,