Artikel getaggt mit conference

Ramblings from LinuxCon/Plumbers 2014

I’m on my way home from Düsseldorf where I attended the LinuxCon Europe and Linux Plumber conferences. I was quite surprised how huge LinuxCon was, there were about 1.500 people there! Certainly much more than last year in New Orleans.

Containers (in both LXC and docker flavors) are the Big Thing everybody talks about and works with these days; there was hardly a presentation where these weren’t mentioned at all, and (what felt like) half of the presentations were either how to improve these, or how to use these technologies to solve problems. For example, some people/companies really take LXC to the max and try to do everything in them including tasks which in the past you had only considered full VMs for, like untrusted third-party tenants. For example there was an interesting talk how to secure networking for containers, and pretty much everyone uses docker or LXC now to deploy workloads, run CI tests. There are projects like “fleet” which manage systemd jobs across an entire cluster of containers (distributed task scheduler) or like project-builder.org which auto-build packages from each commit of projects.

Another common topic is the trend towards building/shipping complete (r/o) system images, atomic updates and all that goodness. The central thing here was certainly “Stateless systems, factory reset, and golden images” which analyzed the common requirements and proposed how to implement this with various package systems and scenarios. In my opinion this is certainly the way to go, as our current solution on Ubuntu Touch (i. e. Ubuntu’s system-image) is far too limited and static yet, it doesn’t extend to desktops/servers/cloud workloads at all. It’s also a lot of work to implement this properly, so it’s certainly understandable that we took that shortcut for prototyping and the relatively limited Touch phone environment.

On Plumbers my main occupations were mostly the highly interesting LXC track to see what’s coming in the container world, and the systemd hackfest. On the latter I was again mostly listening (after all, I’m still learning most of the internals there..) and was able to work on some cleanups and improvements like getting rid of some of Debian’s patches and properly run the test suite. It was also great to sync up again with David Zeuthen about the future of udisks and some particular proposed new features. Looks like I’m the de-facto maintainer now, so I’ll need to spend some time soon to review/include/clean up some much requested little features and some fixes.

All in all a great week to meet some fellows of the FOSS world a gain, getting to know a lot of new interesting people and projects, and re-learning to drink beer in the evening (I hardly drink any at home :-P).

If you are interested you can also see my raw notes, but beware that there are mostly just scribbling.

Now, off to next week’s Canonical meeting in Washington, DC!

Tags: , , , , , , , ,

PyGObject hackfest at GUADEC

Yesterday, GUADEC hosted a PyGObject hackfest. I was really happy to see so many participants, and a lot of whom who are rather new to the project. I originally feared that it would just be the core crew of four people, as this is not exactly the shiniest part of GNOME development.

So I did not work on the stuff I was planning for, but instead walked around and provided mentoring, help, and patch review. Unfortunately I do not know all the results from the participants, hopefully they will blog some details themselves. But this is what I was involved in:

  • Manuel Quiñones added an gtk_tree_view_column_set_attributes() override (the original C function uses varargs and thus is not introspectable). Most time was spent figuring out an appropriate test case.
  • I showed Didier Roche some tricks about porting a pygtk application to PyGI/GTK3. He gave a shot to porting Meld, but unfortunately it uses a lot of pygtk hacks/tricks, most of which are obsolete now. So this proved too big a project for one day eventually :-(
  • Paolo and I guided Marta Maria Casetti, one of this year’s GNOME GSoC students, through her first pygobject patch. The test case still needs some love (again, nothing regarding GtkTreeView is easy), but the actual patch is good. Thanks Marta for participating, and not getting intimidated by all the new stuff!
  • While working on above patch, Marta encountered a rather curious TypeError: Expected Gtk.TreeViewColumn, but got GObjectMeta when writing the override. What seemed to be a trivial problem at first quickly turned into an one-hour debugging session involving grandmaster John Palmieri and me, with others chipping in as well. In the end it (of course!) turned out to be a trivial four-character change in Marta’s patch, but it was fun to get to understand the problem (a loong-forgotten special case of overrides resolution in overrides code). Now pygobject gives a proper error message which is actually helpful, i. e. which argument causes the problem and which module/class/method is provided, which should prevent us from being misguided into the totally wrong direction the next time this happens.
  • John Stowers got the Windows build working again, and showed off the gtk-demo under Windows. This is really amazing, I hope we can get that into trunk soon and not let it bitrot again for so long. Thanks!
  • Simon and Manuel worked on porting some Sugar extensions. Together with Paolo we also discussed the GStreamer 1.0 API a bit, which parts can become API additions and which need to become overrides.
  • Michal Hruby debugged a leak in the handling of GVariant arrays when using libdee.

Thanks everyone for participating! I hope everyone enjoyed it and got to learn a new thing or two. See you at the next one!

PyGObject hackfest at GUADEC 2012

PyGObject hackfest at GUADEC 2012

Tags: , , , , , ,

My impressions from GUADEC

I have had the pleasure of attending GUADEC in full this year. TL;DR: A lot of great presentations + lots of hall conversations about QA stuff + the obligatory be{er,ach} = ♥.

Last year I just went to the hackfest, and I never made it to any previous one, so GUADEC 2012 was a kind of first-time experience for me. It was great to put some faces and personal impressions to a lot of the people I have worked with for many years, as well as catching up with others that I did meet before.

I discussed various hardware/software testing stuff with Colin Walters, Matthias Clasen, Lennart Poettering, Bertrand Lorentz, and others, so I have a better idea how to proceed with automated testing in plumbing and GNOME now. It was also really nice to meet my fellow PyGObject co-maintainer Paolo Borelli, as well as seeing Simon Schampier and Ignacio Casal Quinteiro again.

No need to replicate the whole schedule here (see for yourself on the interwebs), so I just want to point out some personal highlights in the presentations:

  • Jacob Appelbaum’s keynote about Tor brought up some surprising facts about how the project has outgrown its past performance problems and how useful it was during e. g. the Arab revolution
  • .

  • Philip Whitnall’s presented Bendy Bus, a tool to mock D-Bus services for both unit and fuzz testing. He successfully used it to find and replicate bugs in Evolution (by mocking evolution-data-server) as well as libfolks (by mocking the telepathy daemons). It should work just as well to mock system services like upower or NetworkManager to test the UI bits that use it. This is a topic which has been on my wishlist for a long time already, so I’m happy that there is already an existing solution out there. We might have to add some small features to it, but it’s by and large what I had in mind, and in the discussion afterwards Philip said he’d appreciate patches against it.
  • Christophe Fergeau showed how to easily do Windows builds and installers from GNOME tarballs with MinGW-w64, without having to actually touch/use Windows (using cross-building and running tests etc. under wine). I found it surprising how easy that actually is, and it should not be hard to integrate that in a jhbuild-like setup, so that it does not keep breaking every time.
  • Colin Walters gave an introduction to OSTree, a project to build bootable images from kernel/plumbing/desktop upstream git heads on a daily basis. This is mostly to avoid the long delays that we otherwise have with doing upstream releases, packaging them, and getting them into a form that can safely be tested by users. In an afterwards discussion we threw some ideas around how we can integrate existing and future tests into this (something in spirit like our autopkgtest). This will be the area where I’ll put most focus on in the next time.
  • Adam Dingle of yorba fame shared his thoughts about how we can crowdfunding of Free Software Projects work in practice, comparing efforts like codefoundry and kickstarter. Of course he does not have a solution for this yet, but he raised some interesting concerns and it spun off lots of good discussions over lunchtime.
  • Last but not least, the sport event on Saturday evening was awesome! In hindsight I was happy to not have signed up for soccer, as people like Bastian or Jordi played this really seriously. I participated in the Basketball competition instead, which was the right mix of fun and competition without seriously trying to hurt each other. :-)

There were a lot of other good ones, some of them technical and some amusing and enlightening, such as Frederico’s review of the history of GNOME.

On Monday I prepared and participated in a PyGObject hackfest, but I’ll blog about this separately.

I want to thank all presenters for the excellent program, as well as the tireless GUADEC organizer team for making everything so smooth and working flawlessly! Great Job, and see you again in Strasbourg or Brno!

Tags: , , ,

PyGObject hackfest at GUADEC

I just received confirmation that my request for a PyGObject hackfest has been approved by the GUADEC organizers.

If you are developing GObject-introspection based Python applications and have some problems with PyGObject, this is the time and place to get to know each other, getting bugs fixed, learn about pygobject’s innards, or update libraries to become introspectable. I will prepare a list of easy things to look into if you are interested in learning about and getting involved in PyGObject’s development.

See you on July 30th in A Coruña!

GUADEC Badge

Tags: , , , , , , , , ,

Happy Birthday Ubuntu!

7 years ago, The Ubuntu 4.10 “The Warty Warthog” was announced. A huge congrats to the community, Canonical, and especially Mark for getting so far from “there” to “here”.

This brings back old memories of my first conference in Oxford in August, the great-great-grandfather to what is UDS these days. Back then, there was no company, no Launchpad, no Blueprints, no work items, no detailled plans, just a bunch of ideas, BoFs, and this was a third of the entire crowd:

Warty Hack Room

Back then we worked on the famous TRLS technology (“Totally Rad Laptop Support”) and were proud when we got the ThinkPads to suspend once. During that conference I wrote pmount to provide automatic mounting of USB sticks in a safe manner. Those were the days… :-)

But I can also safely say that there are some things that haven’t changed. Even though both the community and the company (which changed away from www.no-name-yet.com recently) grew by two magnitudes since then, we still have the same serious attitude, stern look, and formal attire as we had back then:

We are professionals, really!

We are professionals, really!

Tags: , , , , , ,

Na zdraví PyGI!

(Update: Link to Tomeu’s blog post, repost for planet.gnome.org)

Last week I was in Prague to attend the GNOME/Python 2011 Hackfest for gobject-introspection, to which Tomeu Vizoso kindly invited me after I started working with PyGI some months ago. It happened at a place called brmlab which was quite the right environment for a bunch of 9 hackers: Some comfy couches and chairs, soldering irons, lots of old TV tubes, chips, and other electronics, a big Pirate flag, really good Wifi, plenty of Club Mate and Coke supplies, and not putting unnecessary effort into mundane things like wallpapers.

It was really nice to get to know the upstream experts John (J5) Palmieri and Tomeu Vizoso (check out Tomeu’s blog post for his summary and some really nice photos). When sitting together in a room, fully focussing on this area for a full week, it’s so much easier to just ask them about something and getting things done and into upstream than on IRC or bugzilla, where you don’t know each other personally. I certainly learned a lot this week (and not only how great Czech beer tastes :-) )!

So what did I do?

Application porting

After already having ported four Ubuntu PyGTK applications to GI before (apport, jockey, aptdaemon, and language-selector),
my main goal and occupation during this week was to start porting a bigger PyGTK application. I picked system-config-printer, as it’s two magnitudes bigger than the previous projects, exercises quite a lot more of the GTK GI bindings, and thus also exposes a lot more GTK annotation and pygobject bugs. This resulted in a new pygi s-c-p branch which has the first 100 rounds of “test, break, fix” iterations. It now at least starts, and you can do a number of things with it, but a lot of functionality is still broken.

As a kind of “finger exercise” and also to check for how well pygi-convert works for small projects now, I also ported computer-janitor. This went really well (I had it working after about 30 minutes), and also led me to finally fixing the unicode vs. str mess for GtkTreeView that you got so far with Python 2.x.

pygobject and GTK fixes

Porting system-config-printer and computer-janitor uncovered a lot of opportunities to improve pygi-convert.sh, a big “perl -e” kind of script to do the mechanical grunt work of the porting process. It doesn’t fix up changed signatures (such as adding missing arguments which were default arguments in PyGTK, or the ubiquitous “user_data” argument for signal handlers), but at least it gets a lot of namespaces, method, and constant names right.

I also fixed three annotation fixes in GTK+. We also collaboratively reviewed and tested Pavel’s annotation branch which helped to fix tons of problems, especially after Steve Frécinaux’s excellent reference leak fix, so if you play around with current pygobject git head, you really also have to use the current GTK+ git head.

Speaking of which, if you want to port applications and always stay on top of the pygobject/GTK development without having to clutter your package system with “make install”s of those, it works very well to have this in your ~/.bashrc:

export GI_TYPELIB_PATH=$HOME/projects/gtk/gtk:$HOME/projects/gtk/gdk
export PYTHONPATH=$HOME/projects/pygobject

Better GVariant/GDBus support

The GNOME world is moving from the old dbus-glib python bindings to GDBus, which is integrated into GLib. However, dbus-python exposed a really nice and convenient way of doing D-Bus calls, while using GDBus from Python was hideously complicated, especially for nontrivial arguments with empty or nested arrays:

from gi.repository import Gio, GLib
from gi._gi import variant_type_from_string

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

vb = GLib.VariantBuilder()
vb.init(variant_type_from_string('r'))
vb.add_value(GLib.Variant('s', 'test'))
vb.add_value(GLib.Variant('u', 1))
vb.add_value(GLib.Variant('s', 'gtk-ok'))
vb.add_value(GLib.Variant('s', 'Hello World!'))
vb.add_value(GLib.Variant('s', 'Subtext'))
# add an empty array
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('as'))
vb.add_value(eavb.end())
# add an empty dict
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('a{sv}'))
vb.add_value(eavb.end())
vb.add_value(GLib.Variant('i', 10000))
args = vb.end()

result = notify.call_sync('Notify', args, 0, -1, None)
id = result.get_child_value(0).get_uint32()
print id

So I went to making the GLib.Variant constructor work properly with nested types and boxed variants, adding Pythonic GVariant iterators and indexing (so that you can treat GVariant dictionaries/arrays/tuples just like their Python equivalents), and finally a Variant.unpack() method for converting the return value of a D-Bus call back into a native Python data type. This looks a lot friendlier now:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

args = GLib.Variant('(susssasa{sv}i)', ('test', 1, 'gtk-ok', 'Hello World!',
    'Subtext', [], {}, 10000))
result = notify.call_sync('Notify', args, 0, -1, None)
id = result.unpack()[0]
print id

I also prepared another patch in GNOME#640181 which will provide the icing on the cake, i. e. handle the variant building/unpacking transparently and make the explicit call_sync() unnecessary:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

result = notify.Notify('(susssasa{sv}i)', 'test', 1, 'gtk-ok', 'Hello World!',
            'Subtext', [], {}, 10000)
print result[0]

I hope that I can get this reviewed and land this soon.

Thanks to our sponsors!

Many thanks to the GNOME Foundation and Collabora for sponsoring this event!

Tags: , , , , , , , , , ,

Linux Plumbers Conference

I spent the last week in Portland, Oregon, at the Linux Plumbers Conference. Since several people asked, here is my travel report:

Tuesday

This was not an official conference day yet, I just arrived early due to flights being cheaper.

I spent the entire day with the LinuxFoundation driver backports work group, with Ram Pai (IBM), Jon Masters (Red Hat), and Andreas Gruenberger (Novell). Novell’s orginal member (Susanne Oberhauser) works on other projects now, so we gave a quick summary of the status quo, and the results from Austin to Andreas. We discussed some outstanding questions and the next steps, including

  • online driver DB: per-distro and/or central, synchronization between them
  • various ways of using DKMS to build driver packages
  • various problems in drivers why they can’t/wouldn’t use the standard KBuild system
  • we finally need to get some whitepapers written to provide an end-to-end vision/tutorial about the entire driver workflow

The afternoon was spent on hacking on the online device DB, which we almost, but not quite, got to talk correctly with Jockey.

Wednesday

I met a lot of people in the morning and had some casual chatting.

The keynote was done by Greg KH about an overview of the Linux ecosystem, which was by and large how many patches various contributors got into Linux, binutils, gcc, and other low-level stuff (e. g. dbus was already too high in the stack to be considered), and half of his presentation was bashing Canonical for how they not submit patches upstream. According to some critical questions from the audience, they took it with the right grain of salt. Nothing technically interesting here really, and the matter has been discussed at length in public now.

In the morning I attended half of the audio track. Lennart Poettering started with an overview of the current audio APIs and Pulseaudio, which was a nice summary. That was followed by a presentation of Takashi Iwai about current problems and plans in ALSA, and a following discussion; this was pretty much over my head, so I returned to the main room. Had some interesting one-to-one discussion there, amongst it with Simon McVittie about a dbus-glib bug which has annoyed me a looong time and is a major blocker for Jockey/PackageKit integration. We did not find an easy workaround, unfortunately.

In the afternoon I listened to the presentation of various tracing and debugging technologies in current and future Linux. Roland McGrath pointed out why ptrace() is both badly written and hard to maintain, as well as not very convenient and flexible for userspace programs, and thus should get a replacement. He introduced “utrace”, a low-level kernel API which provides an abstraction of hardware capabilities and register access, and other low-level basics for tracing engines. It is not a replacement for ptrace itself; quoting him, “utrace is for a new tracing engine like the block device layer for file systems; it makes writing the latter tractable”. Next talk demonstrated improvements in “dynamic ftrace”, for run-time instrumentation of code to track kernel syscalls without measurable overhead; veeery deeply technical and utterly fast (time pressure), so it was very hard to follow. The following systemtap was more interesting for non-kernel-hackers like me; it is a very powerful system to track down bugs, which would probably help us in lots of situations (if only it would work out of the box on Ubuntu, we need to fix that).

In the early evening I listened to Linus Torvald’s git tutorial, mostly because it happened in the main hall and I hoped to understand some of the rather weir^Wunique design concepts. It was quite helpful for git noobs like me, who just have used git for and entire three commits so far. Honestly, this just reinforced my love to bzr, though.

After that we went to the Portland Art Museum for a cocktail/dinner party.

Thursday

Thursday’s keynote started off with Jonathan Corbet giving a summary of recent kernel development. Was nice to start us up in the morning, although nothing surprising here.

The following “boot and init” miniconf was really great. It started off with Arjan van de Ven demonstrating how to boot a netbook with Linux and XFCE in just 5 seconds. His general approach is fairly generic, although of course he has an advantage there by being able to make assumptions about the hardware, partitioning, and use cases. He was followed by a talk of Kyle McMartin who looked in to optimizing various stages of the current general disto boot process, which was very applicable to Ubuntu as well. There was a great amount of discussion going on, inter-distro cooperation at its finest :-)

It is nice to see that work and those talks fall into a time when we want to massively clean/speed up Ubuntu’s boot process as well.

I followed half of the afternoon session about power management on the OLPC in particular and Linux in general, but stopped when the more intricate implementation discussion happens. I rather had some random conversations and did some email catchup.

In the evening we had a dinner with us Canonical folks in an old brewery, which I enjoyed a lot.

Friday

Conference wise this only was a half day, which I spent in the desktop plumbing track. This consisted of William McCann introducing session management and the challenges of the ancient PAM/VT/NSS layers we still have to deal with, which resulted in some discussion of use cases and possible replacements. Scott followed with a mixed presentation/discussion session about upstart’s design goals, current status, and current ideas, which was received very well. In the end Marcel Holtman surprised me quite a bit with his annoucement of his work about integrating the D-BUS protocol and server functionality into the kernel itself, which would both help latency, ease implementation on the userspace side, and makes it more appropriate to rely on it even in upstart itself.

In the afternoon I met with Ram Pai again to work on the driver database. I got it up to the point where it would actually succeed in talking to Jockey \o/.

Conclusion

Although the conf was interesting for me, I could not contribute much, and the outcome/effort ratio was not too good for me personally. So next year I’ll most probably not go again, and rather concentrate on the LF collaboration summit, which I found much more productive for the things I am working on. Seems that this is much more a home for people like Scott, who had a wealth of discussions and contributions.

Meeting my colleagues and other Linux hackers in person again was great, though, we had a good time together (including the jet boat tour on the Willamette river on Saturday :-) ).

Tags: ,

Getting ready for Austin

I’m really looking forward to go to Austin next Saturday, for my first LinuxFoundation collaboration summit. I’m particularly interested in bringing forward the work of the Driver Backport workgroup, where my focus is on delivering drivers to the user.

Tags: ,