Artikel getaggt mit announcement

New fatrace released, Debian package coming

Paul Wise poked me this morning about uploading fatrace (“file access trace”, see the original announcement for details) to Debian, thanks for the reminder!

So I filed an Intent To Package, and will upload it in a few days, unless some discussion evolves.

I also took the opportunity to do some modernization: The power-usage-report script now uses the current PowerTop 2.x instead of the old 1.13, uses Python 3 now, and includes the “process device activity” in the report. I released this as 0.5. The actual fatrace binary didn’t change its behaviour, it just got some code optimizations; thanks to Yann Droneaud for those.

Tags: , , , , ,

Urgent PostgreSQL security updates for Debian/Ubuntu

PostgreSQL just released security updates. 9.1 (as found in Debian testing and unstable and Ubuntu 11.10 and later) is affected by a critical remote vulnerability which potentially allows anyone who can access the TCP port (without credentials) to corrupt local files. If your PostgreSQL database exposes the TCP port to any potentially untrusted location, please shut down your servers and update now!

PostgreSQL 8.4 for Debian stable (squeeze) and Ubuntu 8.04 LTS and 10.04 LTS also got an update, but these are much less urgent.

Debian and Ubuntu advisories for all stable releases, as well as Debian testing are going out as we speak. The updates are already on security.debian.org and security.ubuntu.com.

I also uploaded updates for Debian unstable (8.4, 9.1, and 9.2 in experimental) and the Ubuntu backports PPA, but it will take a bit for these to build as we don’t have embargoed staging builds for those. Christoph updated the apt.postgresql.org repository as well.

Warning: If you use the current Ubuntu raring Beta-2 candidate images, you will still have the old version. So if you do anything serious with those installations, please make sure to upgrade immediately.

Update: Debian and Ubuntu security announcements have been sent out, and all packages in the backports PPA are built.

Please see the official FAQ if you want to know some more details about the nature of the vulnerabilities.

Tags: , , ,

python-dbusmock 0.6 released

I just pushed out a new python-dbusmock release 0.6.

Calling a method on the mock now emits a MethodCalled signal on the org.freedesktop.DBus.Mock interface. In some cases this is easier to track than parsing the mock’s log or using GetMethodCalls. Thanks to Lars Uebernickel for this.

DBusMockObject.AddTemplate() and DBusTestCase.spawn_server_template() can now load local templates from your own project by specifying a path to a *.py file as template name. Thanks to Lucas De Marchi for this feature.

I also wrote a quite comprehensive template for systemd’s logind. It stubs out the power management functionality as well as user/seat/session objects, and is convincing enough for loginctl. Some bits like AttachDevice is missing, as this sounds unlikely to be required for D-BUS mock tests, but please let me know if you need anything else.

The mock processes now terminate automatically if their connected D-BUS goes down, as advertised in the documentation.

You can get the new tarball from Launchpad, and I uploaded it to Debian experimental now.

Enjoy!

Tags: , , , , , , ,

PyGObject 3.7.92 released

I just released a new PyGObject for GNOME 3.7.92. This fixes a couple of crashes and marshalling errors again, but most importantly got a change to automatically mute the PyGIDeprecationWarnings for stable versions. Please run pythonX.X with the -Wd option to still be able to see them.

We got through all our bugs that were milestoned for GNOME 3.8 and don’t want to or plan to introduce any major behavioural change at this point, so barring catastrophes this is what will be in GNOME 3.8.0.

Thanks to all contributors!

  • Fix stack smasher when marshaling enums as a vfunc return value (Simon Feltman) (#637832)
  • Change base class of PyGIDeprecationWarning based on minor version (Simon Feltman) (#696011)
  • autogen.sh: Source gnome-autogen to fix out of source builddir (Alban Browaeys) (#694889)
  • pygtkcompat: Make gdk.Window.get_geometry return tuple of 5 (Simon Feltman)
  • pygtkcompat: Initialize hint to zero in set_geometry_hints (Simon Feltman)
  • Remove incorrect bounds check with property helper flags (Simon Feltman)
  • Fix crash when setting property of type object to an incorrect type (Simon Feltman) (#695420)
  • Remove skipping of object property tests (Simon Feltman) (#695420)
  • Give more informative error when setting property to incorrect type (Simon Feltman) (#695420)

Tags: , , , , , , , ,

PyGObject 3.7.91.1 released

I just found out that PyGObject 3.7.91 as released yesterday breaks GEdit plugins. I just pushed out 3.7.91.1 to unbreak this again, sorry about that!

Tags: , , , , , , ,

PyGObject 3.7.91 released.

I just released a new PyGObject for GNOME 3.7.91. This brings some marshalling fixes, plugs tons of memory leaks, and now raises a Python DeprecationWarning when your code calls a method which is marked as deprecated in the typelib. Please note that Python hides them by default, so if you are interested in those you need to run python with the -Wd option.

Thanks to all contributors!

  • Fix many memory leaks (#675726, #693402, #691501, #510511, #672224, and several more which are detected by our test suite) (Martin Pitt)
  • Dot not clobber original Gdk/Gtk functions with overrides (Martin Pitt) (#686835)
  • Optimize GValue.get/set_value by setting GValue.g_type to a local (Simon Feltman) (#694857)
  • Run tests with G_SLICE=debug_blocks (Martin Pitt) (#691501)
  • Add override helper for stripping boolean returns (Martin Pitt) (#694431)
  • Drop obsolete pygobject_register_sinkfunc() declaration (Martin Pitt) (#639849)
  • Fix marshalling of C arrays with explicit length in signal arguments (Martin Pitt) (#662241)
  • Fix signedness, overflow checking, and 32 bit overflow of GFlags (Martin Pitt) (#693121)
  • gi/pygi-marshal-from-py.c: Fix build on Visual C++ (Chun-wei Fan) (#692856)
  • Raise DeprecationWarning on deprecated callables (Martin Pitt) (#665084)
  • pygtkcompat: Add Widget.window, scroll_to_mark, and window methods (Simon Feltman) (#694067)
  • pygtkcompat: Add Gtk.Window.set_geometry_hints which accepts keyword arguments (Simon Feltman) (#694067)
  • Ship pygobject.doap for autogen.sh (Martin Pitt) (#694591)
  • Fix crashes in various GObject signal handler functions (Simon Feltman) (#633927)
  • pygi-closure: Protect the GSList prepend with the GIL (Olivier Crête) (#684060)
  • generictreemodel: Fix bad default return type for get_column_type (Simon Feltman)

Tags: , , , , , , ,

umockdev 0.2.1 release

Hot on the heels of yesterday’s big 0.2 release, I pushed out umockdev 0.2.1 with a couple of bug fixes:

  • umockdev-wrapper: Use exec to avoid keeping the shell process around and make killing the subprogram from outside work properly.
  • Fix building with automake 1.12, thanks Peter Hutterer.
  • Support opening several netlink sockets (i. e. udev monitors) at the same time.
  • Fix building with older kernels which don’t have the EVIOCGMTSLOTS ioctl yet.

This fixes the “bind: address already in use” errors that were popping up in X.org and upower when running under umockdev, and finally gets us working packages for Ubuntu 12.04 LTS (in the daily-builds PPA).

Tags: , , , , , ,

umockdev 0.2: record/replay input devices

I just released umockdev 0.2.

The big new feature of this release is support for evdev ioctls. I. e. you can now record what e. g. X.org is doing to touchpads, touch screens, etc.:

  $ umockdev-record /dev/input/event15 > /tmp/touchpad.umockdev
  # umockdev-record -i /tmp/touchpad.ioctl /dev/input/event15 -- Xorg -logfile /dev/null

and load that back into a testbed with X.org using the dummy driver:

  cat <<EOF > xorg-dummy.conf
  Section "Device"
        Identifier "test"
        Driver "dummy"
  EndSection
  EOF

  $ umockdev-run -l /tmp/touchpad.umockdev -i /dev/input/event15=/tmp/touchpad.ioctl -- \
       Xorg -config xorg-dummy.conf -logfile /tmp/X.log :5

Then e. g. DISPLAY=:5 xinput will recognize the simulated device. Note that Xvfb won’t work as that does not use udev for device discovery, but only adds the XTest virtual devices and nothing else, so you need to use the real X.org with the dummy driver to run this as a normal user.

This enables easier debugging of new kinds of input devices, as well as writing tests for handling multiple touchscreens/monitors, integration tests of Wacom devices, and so on.

This release now also works with older automakes and Vala 0.16, so that you can use this from Ubuntu 12.04 LTS. The daily PPA now also has packages for that.

Attention: This version does not work any more with recorded ioctl files from version 0.1.

More detailled list of changes:

  • umockdev-run: Fix running of child program to keep stdin.
  • preload: Fix resolution of “/dev” and “/sys”
  • ioctl_tree: Fix endless loop when the first encountered ioctl was unknown
  • preload: Support opening a /dev node multiple times for ioctl emulation (issue #3)
  • Fix parallel build (issue #2)
  • Support xz compressed ioctl files in umockdev_testbed_load_ioctl().
  • Add example umockdev and ioctl files for a gphoto camera and an MTP capable mobile phone.
  • Fix building with automake 1.11.3 and Vala 0.16.
  • Generalize ioctl recording and emulation for ioctls with simple structs, i. e. no pointer fields. This makes it much easier to add more ioctls in the future.
  • Store return values of ioctls in records, as they are not always 0 (like EVIOCGBIT)
  • Add support for ioctl ranges (like EVIOCGABS) and ioctls with variable length (like EVIOCGBIT).
  • Add all reading evdev ioctls, for recording and mocking input devices like touch pads, touch screens, or keyboards. (issue #1)

Tags: , , , , , , , , ,

PyGObject 3.7.90 released

I just released a new PyGObject for GNOME 3.7.90, with a nice set of bug fixes and some internal code cleanup. Thanks to all contributors!

  • overrides: Fix inconsistencies with drag and drop target list API (Simon Feltman) (#680640)
  • pygtkcompat: Add pygtk compatible GenericTreeModel implementation (Simon Feltman) (#682933)
  • overrides: Add support for iterables besides tuples for TreePath creation (Simon Feltman) (#682933)
  • Prefix __module__ attribute of function objects with gi.repository (Niklas Koep) (#693839)
  • configure.ac: only enable code coverage when available (Jonathan Ballet) (#693328)
  • Correctly set properties on object with statically defined properties (Jonathan Ballet) (#693618)
  • autogen.sh: Use gnome-autogen.sh (Martin Pitt) (#693328)
  • Fix reference leaks with transient floating objects (Simon Feltman) (#687522)

Tags: , , , , , ,

umockdev: record and mock hardware for debugging and testing

What is this?

umockdev is a set of tools and a library to mock hardware devices for programs that handle Linux hardware devices. It also provides tools to record the properties and behaviour of particular devices, and to run a program or test suite under a test bed with the previously recorded devices loaded.

This allows developers of software like gphoto or libmtp to receive these records in bug reports and recreate the problem on their system without having access to the affected hardware, as well as writing regression tests for those that do not need any particular privileges and thus are capable of running in standard make check.

After working on it for several weeks and lots of rumbling on G+, it’s now useful and documented enough for the first release 0.1!

Component overview

umockdev consists of the following parts:

  • The umockdev-record program generates text dumps (conventionally called *.umockdev) of some specified, or all of the system’s devices and their sysfs attributes and udev properties. It can also record ioctls that a particular program sends and receives to/from a device, and store them into a text file (conventionally called *.ioctl).
  • The libumockdev library provides the UMockdevTestbed GObject class which builds sysfs and /dev testbeds, provides API to generate devices, attributes, properties, and uevents on the fly, and can load *.umockdev and *.ioctl records into them. It provides VAPI and GI bindings, so you can use it from C, Vala, and any programming language that supports introspection. This is the API that you should use for writing regression tests. You can find the API documentation in docs/reference in the source directory.
  • The libumockdev-preload library intercepts access to /sys, /dev/, the kernel’s netlink socket (for uevents) and ioctl() and re-routes them into the sandbox built by libumockdev. You don’t interface with this library directly, instead you need to run your test suite or other program that uses libumockdev through the umockdev-wrapper program.
  • The umockdev-run program builds a sandbox using libumockdev, can load *.umockdev and *.ioctl files into it, and run a program in that sandbox. I. e. it is a CLI interface to libumockdev, which is useful in the “debug a failure with a particular device” use case if you get the text dumps from a bug report. This automatically takes care of using the preload library, i. e. you don’t need umockdev-wrapper with this. You cannot use this program if you need to simulate uevents or change attributes/properties on the fly; for those you need to use libumockdev directly.

Example: Record and replay PtP/MTP USB devices

So how do you use umockdev? For the “debug a problem” use case you usually don’t want to write a program that uses libumockdev, but just use the command line tools. Let’s capture some runs from libmtp tools, and replay them in a mock environment:

  • Connect your digital camera, mobile phone, or other device which supports PtP or MTP, and locate it in lsusb. For example
      Bus 001 Device 012: ID 0fce:0166 Sony Ericsson Xperia Mini Pro
  • Dump the sysfs device and udev properties:
      $ umockdev-record /dev/bus/usb/001/012 > mobile.umockdev
  • Now record the dynamic behaviour (i. e. usbfs ioctls) of various operations. You can store multiple different operations in the same file, which will share the common communication between them. For example:
      $ umockdev-record --ioctl mobile.ioctl /dev/bus/usb/001/012 mtp-detect
      $ umockdev-record --ioctl mobile.ioctl /dev/bus/usb/001/012 mtp-emptyfolders
  • Now you can disconnect your device, and run the same operations in a mocked testbed. Please note that /dev/bus/usb/001/012 merely echoes what is in mobile.umockdev and it is independent of what is actually in the real /dev directory. You can rename that device in the generated *.umockdev files and on the command line.
      $ umockdev-run --load mobile.umockdev --ioctl /dev/bus/usb/001/012=mobile.ioctl mtp-detect
      $ umockdev-run --load mobile.umockdev --ioctl /dev/bus/usb/001/012=mobile.ioctl mtp-emptyfolders

Example: using the library to fake a battery

If you want to write regression tests, it’s usually more flexible to use the library instead of calling everything through umockdev-run. As a simple example, let’s pretend we want to write tests for upower.

Batteries, and power supplies in general, are simple devices in the sense that userspace programs such as upower only communicate with them through sysfs and uevents. No /dev nor ioctls are necessary. docs/examples/ has two example programs how to use libumockdev to create a fake battery device, change it to low charge, sending an uevent, and running upower on a local test system D-BUS in the testbed, with watching what happens with upower --monitor-detail. battery.c shows how to do that with plain GObject in C, battery.py is the equivalent program in Python that uses the GI binding. You can just run the latter like this:

  umockdev-wrapper python3 docs/examples/battery.py

and you will see that upowerd (which runs on a temporary local system D-BUS in the test bed) will report a single battery with 75% charge, which gets down to 2.5% a second later.

The gist of it is that you create a test bed with

  UMockdevTestbed *testbed = umockdev_testbed_new ();

and add a device with certain sysfs attributes and udev properties with

    gchar *sys_bat = umockdev_testbed_add_device (
            testbed, "power_supply", "fakeBAT0", NULL,
            /* attributes */
            "type", "Battery",
            "present", "1",
            "status", "Discharging",
            "energy_full", "60000000",
            "energy_full_design", "80000000",
            "energy_now", "48000000",
            "voltage_now", "12000000",
            NULL,
            /* properties */
            "POWER_SUPPLY_ONLINE", "1",
            NULL);

You can then e. g. change an attribute and synthesize a “change” uevent with

  umockdev_testbed_set_attribute (testbed, sys_bat, "energy_now", "1500000");
  umockdev_testbed_uevent (testbed, sys_bat, "change");

With Python or other introspected languages, or in Vala it works the same way, except that it looks a bit leaner due to “proper” object semantics.

Packages

I have a packaging branch for Ubuntu and a recipe to do daily builds with the latest upstream code into my daily builds PPA (for 12.10 and raring). I will soon upload it to Raring proper, too.

What’s next?

The current set of features should already get you quite far for a range of devices. I’d love to get feedback from you if you use this for anything useful, in particular how to improve the API, the command line tools, or the text dump format. I’m not really happy with the split between umockdev (sys/dev) and ioctl files and the relatively complicated CLI syntax of umockdev-record, so any suggestion is welcome.

One use case that I have for myself is to extend the coverage of ioctls for input devices such as touch screens and wacom tablets, so that we can write some tests for gnome-settings-daemon plugins.

I also want to find a way to pass ioctls back to the test suite/calling program instead of having to handle them all in the preload library, which would make it a lot more flexible. However, due to the nature of the ioctl ABI this is not easy.

Where to go to

The code is hosted on github in the umockdev project; this started out as a systemd branch to add this functionality to libudev, but after a discussion with Kay we decided to keep it separate. But I kept it in git anyway, given how popular it is today. For the bzr lovers, Launchpad has an import at lp:umockdev.

Release tarballs will be on Launchpad as well. Please file bugs and enhancement requests in the git hub tracker.

Finally, if you have questions or want to discuss something, you can always find me on IRC (pitti on Freenode or GNOME).

Thanks for your attention and happy testing!

Tags: , , , , , , , , ,