Cagibi 0.1.1 released; network:/ kio-slave freezes kded in 4.5.0

Version 0.1.1 of Cagibi
Release often and early! Hadn’t done so with Cagibi, the experimental cache/proxy system for SSDP, for a while, as I was only experimenting. But with the upcoming 4.5.0 release of KDE’s fine products (the platform, the workspaces and the apps), I rushed 0.1.0 as a first version which was basically working, to be used by the network:/ kio-slave (included in the runtime part of KDE’s platform). Well, basically working in my test environment. But not for the first brave packager who gave it a try, ouch.

So time for version 0.1.1 of Cagibi!
New in this version is basically only the fix for the D-Bus service description file, which now points to the right location of the daemon executable.
There is now also an initial pkg-config file, but it seems pkg-config is just for libs, not D-Bus-only services.

The 0.1.1 tarball can be downloaded from stable/cagibi at the KDE ftp server (please consider using one of the mirrors).

network:/ kio-slave freezes kded of KDE’s platform 4.5.0
But there was more, more bad news: the network:/ kio-slave will freeze kded, a central daemon of KDE’s platform, without and with Cagibi. This bug only got discovered during the creation of the platform packages for 4.5.0, so the fix will be only included in 4.5.1, unless your packager/distribution backports it.

Workaround until 4.5.1
Yet there is a workaround for you if you don’t mind some config file editing. Open the file `kde4-config --prefix`/share/kde4/services/kded/networkwatcher.desktop in your text editor and make this changes:

# This line needs to be added
X-KDE-Kded-autoload=true
# This line needs to be commented
#X-KDE-Kded-load-on-demand=true

This will make kded load the module networkwatcher already at the start, so the deadlock caused by the on-demand loading can be circumvented.

Now, still with KDE’s platform 4.5.0, this workaround and Cagibi 0.1.1 you must make sure that the Cagibi daemon is already running before kded (and the networkwatcher) starts, if you want to also have UPnP devices/services listed, as delivered from Cagibi. So e.g. hack the startkde script to start cagibid in the background pretty early.
Best please just wait for the 4.5.1 version of KDE’s platform and Cagibi 0.1.1 to be available as packages.

UPnP support also for EFL
In related notes I just saw that still alive’n’kicking Enlightenment (rumored to now be used by Samsung for their mobile devices’ platform Bada, well, looks like) now also has a library to wrap around UPnP, named Eupnp. Another candidate to make use of Cagibi as central SSDP cache/proxy daemon 🙂

First release of Cagibi, prototype of cache/proxy daemon for UPnP device listening and publishing

What?
Cagibi is the prototype (in progress) of a cache/proxy daemon for SSDP (the discovery part of UPnP).
Starting with upcoming KDE SC 4.5 the network:/ kio-slave (from kdebase/runtime) will optionally use this daemon to also show UPnP devices/services in its listing of the local network: If the UPnP (root)device (e.g. a router or a media server like on the PS3) offers a web interface, the network:/ kio-slave will forward to that url on a click of the item in the listing.
(There is no build-time check for it in the network:/ kio-slave code yet, as well as no pkg-config support in Cagibi. It’s an optional run-time D-Bus dependency, if not existing just silently ignored by the network:/ kio-slave system)

Where?
Cagibi 0.1.0 tarballs can be downloaded from stable/cagibi at the KDE ftp server (please consider using one of the mirrors).

Why?
As the network:/ kio-slave so far only supported services as reported from DNS-SD/Zeroconf via the Avahi daemon on GNU/Linux (as wrapped by KDNSSD in the KDE platform), it was obvious to me that such a daemon approach should be also done for SSDP, both for listening to service/device announcements and publishing our own.
So far these FLOSS implementation of (parts of) the UPnP specs have passed my eyes, the UPnP protocol stack frameworks/libs

the media servers

and the stuffed UPnP device racks/server suites

  • Coherence (custom code stack)
  • Rygel (GUPnP)
  • Just, they all did the SSDP stuff inside a lib/the program itself, so in case multiple processes which are using that lib will be doing UPnP stuff, they will duplicate resource usage a little. Especially unneeded as there is usually already a daemon like Avahi running, which could do that.

    It may be a small optimization, but IMHO still valid the work, e.g.:

    • a cache is faster, you do not have to wait for the broadcast answers on the startup of your SSDP client, where answers might be late as in 3 secs
    • the daemon as the only one interfacing to network regarding SSDP would run as system user with almost no rights, so exposing less of a risk
    • no need to run a UPnP device server process all the time just to send the alive message once in a while (wakeup on port activity by inetd or daemon)
    • would be a single network/cpu heartbeat instance for publishing alive messages for multiple local UPnP device offerings or receiving broadcast messages not interested in
    • ?

    And now?
    If you are a packager, please package Cagibi and add it to your distributions, so at least users of programs built on the KDE platform >= 4.5 can also see UPnP devices in the KIO system and open their web interfaces.

    If you are a developer of a UPnP stack module/lib, consider to use Cagibi as backend to your SSDP API. Even better, pick up the code and run with it, make a better Cagibi yourself, also adding the support for publishing. And even better, go for Avahi and beg for an extension for SSDP 🙂

    If you are me, you make Cagibi for the next version install on the system bus, so the daemon instance gets used by all users on the device, also change description data in D-Bus API to a string map of variant data types for better backward compatibility, in the future 🙂

    And if you have a comment, post it 🙂 (or “Leave it”, like WordPress says)

    Akademy, development catalysator

    While most of my fellows at Openismus are currently in The Hague for the GUADEC (after all those who are there are Gnome users and developers 😉 ) I got aware that this year’s edition of the conference for users and developers of KDE software, Akademy, is already a few weeks ago again, time to complete and dump my blog entry about it. (And time to welcome next year’s joint event, so all Openismus people will be at the same place at the same time 🙂 ).

    After more than two month with almost no work on KDE stuff (due to broken backlight on old laptop, starting as employee at Openismus and for this moving to and settling temporarily in Helsinki, getting a new shiny laptop but with completely broken kernel drivers for the GPU) I was happy to be able to dedicate my full time to KDE stuff again, by rushing some development setup onto the laptop I temporarily have for work and start hacking on the things that had been stuck over the recent month, especially with all the people around I need to talk to for that:

    Okteta plugin to KDevelop
    I had started that plugin at the Kate-KDevelop sprint this February. It was basically working already, but there was no way to actually trigger the loading of any file as byte array, and without any sufficient clues about the architecture and not much time to do the needed communication via email and irc this integration plugin just bitrotted in playground. Now with Alexander Dymo and Millian Wolff sitting just at one of the other tables I was a matter of preparing my questions for them (and solving part of the problem myself, as you do usually) and some little conversation, and done it was. The only showstopper are a bug in the KDevelop platform itself, worked on, and needing to wait for the SC 4.5 release next month, as only then the headers of the Okteta libs are published, which these plugin uses.

    network:/ kio-slave
    This kio-slave is part of the SC since 4.3 (in kdebase/runtime, try network:/), but since then lacking further polish and better integration with Places, bookmarks and the remote:/ kio-slave. At least one one of the biggest hurdles, the overbending of the KIO concepts, by introducing mimetypes for the service items (for a nice representation in the view), which on a click forward to a proper url of that service, which may deliver an item of another mimetype, but that is not expected.
    I was lucky to find time slots at one of the clones of David Faure present who showed his deep code knowledge and quick solution ability. So starting with the RC3 the network:/ kio-slave finally starts to get usable and can be promoted a little more 🙂

    Cagibi
    Claiming that the network:/ kio-slave was just sitting there and lacking further polish is only telling half of the truth. Last october at a sprint in Barcelona with the nice boys from Coherence I did an initial implementation of a UPnP backend, so also the devices/services as announced within the UPnP system are listed in the network, at least their root device with a link to the device’s webinterface if present (presentation url).
    Since then some time has passed, and as I needed only the SSDP part of UPnP for the network:/ kio-slave, running a feature-full daemon like Coherence was not the best solution, so under the name Cagibi I threw together some custom code for a prototype of a SSDP cache/proxy, as also once upon a time proposed by Armijn Hemel, the long-time pusher for UPnP support in KDE.
    With Armijn present, I could demo it to him, so he could see first fruits of his (and Adriaan’s) sprint sponsoring efforts related to this. How and when you can see it (like in SC 4.5 ?), will be topic of the next post.

    I also was glad to meet, amongst others I was glad to meet of course 🙂 , Nikhil Marathe, who works to finally bring the UPnP mediaserver kio-slave into existence as his GSoC 2010 project, being mentored by Bart, who started on that during the sprints with Coherence. And so Nikhil can demo more fruits, and they look tasty. The days are counted at which you cannot access your files on your X-Box (or whatever closed system you have exposing a UPnP mediaserver) from Amarok, Dolphin or any KDE platform filedialog.

    And I got a lot of inspiration from all the workshops, for going on with going Mobile, for getting our personal and open share in the Cloud, for linking our minds and data models with Telepathy, and what not.

    This year’s Akademy was great, thanks again for everyone involved to make it happen as it did!

    GSoC ideas I dream somebody would do…

    While commenting on one of the application drafts to work on UPnP (MediaServer) support for Amarok I wondered why this proposal is mainly about support for Amarok and not the whole of KDE Platform based programs, especially as the solution is considered to be done basically as a kio-slave. To be honest, I don’t use Amarok (because I only seldomly listen to music, rather make myself some… uhm, noise), but still I would like to be able to access the content on UPnP MediaServer devices from Dragon Player, Kaffeine or Gwenview. And Okteta 😉
    Same is true for other protocols only Amarok supports. Why no DAAP (from a Firefly Media Server of course) access from Kaffeine, too? Well, there once (KDE3) was a DAAP kio-slave which, besides not being ported, seems to have been dropped due to the latest, unknown protocol version (this proprietary restrict system called iTunes should not be supported anyway). But the simple list-stat-get kio-slaves have the problem that they do not expose all of the data which is interesting to Amarok. So the Amarok hackers had to invent their own abstraction layer for the media collections you can handle in Amarok.

    Collections, filtering, tagging, merging content from different sources… isn’t there some dedicated general system for that? I think there is, you guess which I mean.

    So: Couldn’t someone please go and try to port the media access layer from Amarok to Akonadi/Nepomuk/Kio in a GSoC project? And if she is done also give the media access layer from Digikam the same treatment?

    Why shouldn’t the envisioned Plasma Media Center also use the same engine/platform like the “plain” programs do? Currently it looks they are just going to repeat some of the work others have done before. And Picasa galleries, Youtube, Flickr, etc., would be so nice to have transparent access to that content in all KDE Platform based programs, wouldn’t it?

    Just imagine showing your photos from the last Hacksprint tagged Sleep as a slide show directly from the flickr storage system in Gwenview. Or Digikam if you like. Or your own custom-made presentation program which you quickly did because you just needed to concentrate on the UI, as the elaborated storage and query system is already provided with Akonadi/Nepomuk/Kio.

    Oh, just dreaming… wait, am I still sitting in front of the computer? Not laying in the bed? I should be doing that. If only because it is dreaming time…

    Metalworks for UPnP devices at Tokamak4

    As Kevin, KDE’s Solid master, already reported, the Tokamak 4 opened the possibility of the fusion of more particles into Solid (yet without turning it into Plasma, too 😉 ).

    I had been working for two days mainly on porting the service and status framework of Khalkhi to the KDE 4 Platform including a proper Plasma widget, which will take some time to be completed, but I miss it very much now that my workspace is based on Plasma where the Kicker applet does not work. Three years ago I even started such a port already, but intermixed it with my personal ideas about a contact/person API to the Akonadi system. Now this port is of no use, so I have to redo the porting. Huh. Four years old code which I could not even remember/follow why I did its design like I did. Fun, not.

    Just, the opportunity of having Kevin sitting directly on the other side of the table here at Tokamak 4 and him, other than planned, already having added the support for multiple backends to Solid the day after the meeting, also my wish to do something different yesterday, resulted in a first working prototype for a UPnP backend for Solid, at the same time providing a proof-of-concept work for the multiple backend solution.

    I simply relied on my UPnP work started at the second Coherence/KDE sprint last october. Coherence is a Python-based server which… does a lot of UPnP related stuff, e.g. caching all information about the UPnP devices present on the local network and making this information accessible via D-Bus. Kind of like what Avahi does for DNS-SD/Zeroconf, with regard to this.
    While the Coherence-driven backend for the network:/ kio-slave did not make it for KDE SC 4.4, I recently pushed the code into trunk, so it doesn’t bitrot and gets a final brush for 4.5, now that there is a released version of Coherence available which has the needed D-Bus interface as developed during that Coherence/KDE sprint. There is only a run-time dependency, and it is optional. It will fail silently if no Coherence server with a compatible version or one at all can be started/reached.
    Currently the private wrapper lib named KUPnP is simply duplicated between kdebase/runtime and now kdelibs/solid, need to find out a solution. And like KDNSSD does not expose what backend it uses, KUPnP is not bound to Coherence, so it could be adapted if there are proper solutions, without any problems.

    KUPnP, both as backend for Solid and the network:/ kio-slave, needs Coherence v0.6.5 at least (including the D-Bus service file, some packages miss this, e.g. the default one for OpenSUSE 11.2, instead use the Coherence package “python-coherence” from obs://build.opensuse.org/GNOME, from software.opensuse.org).

    See here a screenshot of solidhardwarebrowser listing the UPnP devices found in the local network next to the ones provided from the HAL backend for the built-in hardware. The Coherence upnp-inspector at least shows the same 😉

    Spoiler: Currently there is not anything you can do with the Solid devices from
    the KUPnP backend, besides getting them listed in solidhardwarebrowser. But
    it’s a start and might inspire other people to join efforts, hint… hint…

    E.g. think about if you want to tackle a GSoC project for Amarok to work on a kio-slave to access the content of UPnP MediaServers.

    UPnP devices coming to the network:/ kioslave

    Since the Coherence/KDE sprint (Spain, 26 °C, sun), where work was done for some UPnP bindings for KDE, a week has passed (Germany, 4 °C, rain/hail). Besides the weather some other things have changed that week, too, but for the good 🙂

    While on Day 1 of the sprint I already managed to have first UPnP devices show up in the network:/ kioslave, it was done with hacky prototype code, to have some things working to start from. Day 2 didn’t change much in that regard.

    The days since then Frank Scholz, the main developer of Coherence, has been very supportive and has continued adding needed features to the D-Bus API, and I have hammered a little more onto the code for the UPnP backend to the network:/ kioslave. To much success! UPnP devices are now coming and going in the network:/ listing, matching the real devices’ behaviour, with proper name and type.

    All that is missing to my full pleasure is the usage of each UPnP device’s custom icon (So if you know how to set custom icons (i.e. not by name, but delivered from remote, cmp. favicons) for UDSEntries in a kioslave, please tell me!). And forwarding to the presentation url of the UPnP device, if available, but that looks like it needs some discussions with the KIO developers, because of the “abuse” of KIO for the network:/ kioslave.

    No screenshot today, if you are interested please try it yourself:
    The UPnP backend for the network:/ kioslave is based on the utility libkupnp which cares for the wrapping of the D-Bus connection to Coherence. So fetch both
    libkupnp at
    http://websvn.kde.org/trunk/playground/network/kupnp/
    and the UPnP backend at
    http://websvn.kde.org/branches/work/network-kioslave/adding-upnp-backend/
    and compile as usual.

    You also need a current version (>=#1440) from trunk of Coherence:

    svn co https://coherence.beebits.net/svn/trunk/Coherence
    sudo python ./setup.py install
    

    (No idea how to install Coherence to a user-only prefix, hints welcome!)

    If you are interested in Bart's work on UPnP support for Amarok, do not miss his last blog entry (which seems to have missed planet KDE).

    The coming weeks I will see if libkupnp can be extended to be useful for other code interacting with UPnP devices (at least from a UPnP control point's view). For a start I will try a InternetGatewayDevice proxy class, which could be used by programs wanting NAT traversal (Konversation and KTorrent already have some custom implementation, but this could help others). Next would be a MediaServer proxy class, perhaps that could simplify Bart's upnp:/ kioslave.

    Coherence/KDE sprint Day 2

    Exactly one week already has passed since I arrived in Barcelona for the Coherence/KDE sprint, and almost four days since I left again, so before it is out-of-date here a short summary of Day 2 (see also Day 1 for some introduction):

    Everyone continued hacking on his stuff, e.g. Bart on his kioslave for UPnP MediaCollections, to be used especially by Amarok, and me on the UPnP backend for the network:/ kioslave. Which also involved discussions with Frank about shaping the D-Bus API of Coherence to our needs a little. And how to merge best the UPnP device/service model and objects into the network classes model of the base library of the network:/ kioslave, along with the ones from DNSSD and Co. This merging needs more thoughts definitely.

    Lunch was cared for by our host Edward Hervey from Collabora Multimedia leading us to another non-tourist restaurant with good food and local atmosphere.

    Other stuff that day was done was demoing the things now working, like the Coherence plugin for the Spykee robot delivering pictures from its built-in webcam via UPnP. Or things that have been working for a while, like each one playing his favourite songs via UPnP on Frank’s computer, in a kind of battle style. Including controlling the volume from remote.

    Zaheer, Konqui and me looking what Frank does that makes him wear the K-scarf this way
    Zaheer, Konqui and me looking what Frank does that makes him wear the K-scarf this way

    In some way Coherence is to UPnP what Avahi is to DNS-SD, a demon wrapping the specific system and making it accessible via D-Bus. Just that Coherence does a lot more, as UPnP is not only about discovery of services, but also of types of devices and the interfaces/services they implement and how to access them.

    As the developers of Coherence are using GNOME-centric desktops the current desktop integration is basically done for GNOME programs, like Totem, Rhythmbox, and Nautilus.

    Let’s hope KDE-Libs based programs can catch up to these 🙂 This was the very purpose of this sprint, and it gave a good foundation. It just left us with quite some work to do. Especially Bart might appreciate help with his kioslave, as he is short on time. So if you want Amarok to as well access other UPnP MediaCollections as well as export it’s own collection as such in the next time, get in contact with him.

    I think I might be able to get the integration into the network:/ kioslave done in October (promises, what promises?). Surely there will be the need for feedback. So please stay tuned, more detailed info about approach and progress to follow…