Give us a proper mimetype name for OpenCL C files!

KDevelop, your cross-platform IDE, since version 5.1 has initial OpenCL language support.

If you are into OpenCL and interested, we need your help for some improvement to that:

What is the mimetype name to use for OpenCL C files?

Due to KDevelop needs some weeks ago a mimetype for OpenCL has already been added to the shared-mime-info database, with text/x-opencl-src as mimetype name.
Though if we reflect, that entry was quickly requested and made without consulting a lot of OpenCL users. It currently reads:

<mime-type type="text/x-opencl-src">
    <_comment>OpenCL source code</_comment>
    <acronym>OpenCL</acronym>
    <expanded-acronym>Open Computing Language</expanded-acronym>
    <sub-class-of type="text/x-csrc"/>
    <glob pattern="*.cl"/>
</mime-type>

For one, “OpenCL source code” should be rather “OpenCL C source code”. Because there are now both C and C++ variants (see below) which would need to be reflected in the name.

Next, text/x-opencl-src was made up and seems to be not used by anyone before. Asking your search engine, you will see that people in their desperation had already started to use all kind of other mimetype names before, e.g.:

  • text/x-opencl
  • text/x-clsrc
  • text/x-opencl_c
  • application/x-opencl_c

So text/x-opencl-src might not have been the best choice. Even more as again the C and C++ variants need to be reflected in the name.

Is there no-one at the Khronos Group who had time to look at deciding on a mimetype name and registering that with IANA as MIME type (or now Media type)?

Can you poke them, or hint us whom to talk to?

OpenCL C++

And then there is OpenCL C++ (since OpenCL 2.1), yet to reach the masses.
What mimetype name shall it have? What will the file extension(s) be?

Advertisements

Wanted: your wishes on managing author identity data in KDevelop

If you are a software developer who likes IDEs, please help to shape a new feature that is currently worked on for KDevelop, your attractive cross-platform IDE for currently C, C++, Python, JavaScript and PHP:

support for management of multiple author identities (name, email, …)

Please help by telling your user stories around data about your author identity in your development process. Add the things you would like to see supported (soon or one day) here:
https://community.kde.org/KDevelop/AuthorIdentities#User_stories

Other contributions to that page are welcome, too. It is a wiki 🙂
(If a wiki scares you, replying in the comments to this blog post is also fine, everything will be picked up.)

To give you an idea what kind of things are wanted, read on:

Some initial user stories

Do not get in the way:

As developer working alone on a private project, I do not want to have to deal with any identity stuff, any defaults are fine with me.

Multiple identities in same system:

As developer working with the same computer/system on different projects for different customers or different project groups, I want KDevelop to always inject the correct/matching copyright & author information on generating code from file & app templates for different projects/customers, instead of having to manually post-edit the generated code.

Multiple identities in same project:

As developer working on the same project with the same computer system both for job and privately, I want to quickly switch identity data to be used depending on whether I work for job or privately, to mark copyright of contributions accordingly.

Pick up identity on import:

As developer importing a project into KDevelop, I want to have the identity automatically derived from account/server/fetched project and have the option to create a new identity from the found data if there is none, or overwrite with an existing identity.

Highlight my contributions in VCS history:

As developer working on a project with VCS support, I want my easily see my own commits in the vcs history, e.g. by being highlighted.

Update existing identity data:

As developer working on a project, I want to have assistance when identity properties had been changed (like email address or name) and this should be reflected in all existing copyright/contributor notions of a project, where possible.

Following some more background info why work has been started on this feature:

Problem

When it comes to copyright notions in code/files and authorship/committer id in version control systems, the name and the contact information, like email address, can be a few different ones for oneself. E.g. when working in different FLOSS projects, with each email address dedicated to the project, or when working on a FLOSS project both in name of an employer, but also privately in free times.

Currently KDevelop is hard-coded to use whatever is set for the default profile of KEMailSettings when generating new code from file templates, the only place right now using authorship metadata. KMail, the well-known email client, seems to be feeding KEMailSettings via the KIdentityManagement module, but otherwise there might be no data in there. So the developer has to manually complete the fields in the generated code.

The integration of KDevelop with git also ignores the author and commiter metadata used by git for the repo. They need to be set manually and are nowhere linked to what KDevelop knows.

For a truely integrated development experience this needs to be improved.

Solution

KDevelop could know about the concept of author identity, could allow to have multiple identities available to select from for a project, and could allow plugins to make use of identity details e.g. on doing VCS commits or generating code.

Here a screenshot of some proof-of-concept code:
Draft of KDevelop settings: Configure Author Identitites

So, what do you also think in general? Anything related which is worth looking
into for this? Other comments?

Marble in your CMake or qmake based project, now more easy

Marble, the virtual globe and world atlas, at its core is a library intended for reuse, libmarblewidget. This library is done with minimal dependencies to ease usage (e.g. just needs a few Qt5 modules).

The new version of Marble, coming with libmarblewidget v0.26 and released as part of KDE Applications 16.12, is now making it more easy to integrate libmarblewidget in your own software project, when using CMake or qmake as buildsystem.

To link your project to libmarblewidget and find its headers, you can have that in 1-2 lines:

For qmake a qt_Marble.pri file is installed. So in the pro file of your project just add:

QT += Marble

For this to work you have to do one out of these:

  • install Marble into the same prefix as your Qt5 installation (e.g. /usr)
  • pass -DMARBLE_PRI_INSTALL_USE_QT_SYS_PATHS=ON to cmake when building Marble
  • use the environment variable QMAKEPATH to point to the installation prefix of Marble when calling qmake on your project

For CMake some CMake config files are installed. So in the CMakeLists.txt of your project just add:

find_package(Marble REQUIRED)
target_link_libraries(my_project  Marble)

No more need for some own FindMarble.cmake file.

The CMake variant is already possible since libmarblewidget v0.25 (part of KDE Applications 16.08).

Learn more about how to use Marble in your project, e.g. by simple examples, at marble.kde.org.

Kate, Gwenview & Co. unwantedly trading loss of file meta data against loss of content data, due to QSaveFile

tl;dr Code using QSaveFile (so Kate, KWrite, KDevelop, Gwenview & others) can result in loss of file meta data with extended file attributes, a change of file ownership and loss of hard links on existing files. Cmp. QTBUG-56366.

Wanted: Atomic changes to file content

When you are looking for a way to do transactional writes of files, QSaveFile looks like a simple and clear solution. But, this is 2016, so support for transactional updates of file content with mainstream file systems seems to be still rocket science? At least QSaveFile does a dirty hack as work-around: it creates a new temporary file to which all data is written into, and once all the content has arrived on the permanent storage, on the call of commit() it tries to replace the original file with that temporary file, by registering the new file with the name of the old in the file entries as replacement, which is an atomic system call.

Content data safely arrived, but meta data lost on the way

Just, and this is where things get dirty, this registration as replacement also will result in a replacement of all meta data for the old file with that of the new. And if the old file was registered as a hard link, that link will be dropped as well, the new file registered for the given name is now a file on its own decoupled from the formerly linked ones which also never will learn about the new content just written.
So the hack with the separate temporary file needs some more efforts to simulate a real transactional update of the content of that file identified by its name. Sadly some things are impossible and some things seem not yet implemented. Impossible is e.g. to set the ownership in most cases because for obvious reasons this needs special rights (cmp. API documentation of chown()). And e.g. not yet implemented seems to be copying of extended file attributes.

You who makes a difference?

There is no big warning sign yet on the API documentation of QSaveFile, so some developers might not be aware of these traps when using QSaveFile, like e.g. all over code in KDE projects. Or they are, but the users of their products are not (cmp. related bugs for Kate 333577, 354405, 358457).

So if you desire transactional updates of file contents in your code, consider if there are cases where the loss of the mentioned file meta data can be an issue, e.g. when those files are candidates to be shared on multi-user systems or when extended file attributes are being made use of with those files.

If you are a developer experienced with file system API, please consider writing a fix or improvement for QSaveFile and adopting QTBUG-56366: “QSaveFile results in loss of extended file attributes and change of file ownership”.

If you are a developer of file systems, please consider writing some system API which covers the use cases of transactional access to files. So there could be a proper implementation variant of QSaveFile and also any helpless hacks like .lock files would also no longer be needed.

And if you found wrong claims in this text, please shout out in the comments, as I only explored this issue quick and superficially for now, so chances are I missed something.

Adding API dox generation to the build by CMake macros

Online documentations of APIs are a fine thing: by their urls they provide some shared universal location to refer to and often also are kept up-to-date to latest improvements. And they serve as index trap for search engine robots, so more people have the chance to find out about some product.

But the world is sometimes not an online heaven:

  • “my internet is gone!”
  • documentation webserver is down
  • documentation webserver is slooo….oo…ow
  • documentation webserver no longer has documentation for that older version

So having also access to offline documentation as alternative can make some developers using your library happy. If you are using Qt in your projects, you yourself might even prefer to study the offline Qt API documentation installed on your local systems, either via the Qt Assistant program or as integrated in IDEs like Qt Creator or KDevelop. Even better, the offline documentation comes together with the library itself (e.g. via packages on Linux or *BSD distributions), so the documentation is always matching the version of the library.

Being a good developer you have already added lots of nice API documentation in your code. Now how to turn this into a separate offline documentation manual for the consumers of your library, without too much effort?
If your project uses CMake for the build system and your target audience is using documentation viewers like Qt Assistant, Qt Creator or KDevelop, read on for a possible solution.

Deploying doxygen & qhelpgenerator

Very possibly you are already using doxygen to generate e.g. an HTML version of the API dox for your library project. And you might know doxygen can output to more formats than HTML. One of them is the format “Qt Compressed Help” (QCH). Which is a single-file format the above mentioned documentation viewers support well. QCH support by doxygen even exists since 2009.

So, with doxygen & qhelpgenerator installed and a respective doxygen config file prepared, the build script of your library just needs to additionally call doxygen as part of another target, and there is the API documentation manual as part of your project build, ready to be deployed and packaged with your library.

A CMake macro for convenience

To not have everyone duplicate the needed build setup code, like checking for the needed tools being installed, and to not have to learn all the details of the doxygen config file, some CMake macro doing the dull work recommends itself to be written and to be distributed for reuse.

Before doing that though, let’s consider another aspect when it comes to documenting libraries: including documentation for code constructs used from external libraries.

Linking between offline API dox manuals

Especially for C++ libraries, where subclassing across classes from different libraries is common, one wants to have direct links into the documentation of external libraries, so the reader has transparent access to any info.

The Qt help system has an abstract addressing system which works independently from the actual location of the QCH files e.g. in the filesystem. These addresses use the scheme qthelp: and a domain namespace for identifying a given manual (e.g. org.qt-project.qtcore). Additionally there is the concept of a virtual folder which is also set as root directory within a manual (e.g. qtcore). So the address of the document for the class QString would be qthelp://org.qt-project.qtcore/qtcore/qstring.html, and the documentation viewer would resolve it to the manuals registered by the matching metadata.

Doxygen supports the creation of such links into external QCH files. For a given manual of an external library, the references are created based on the domain name, the virtual folder name and a so-called tag file for that library. Such a tag file contains all kind of metadata about classes, methods and more that is needed to document imported structures and to create the references.

The QCH files for Qt libraries are coming with such tag files. But what about your own library? What if somebody else should be able to create documentation for their code with links into the documentation of your own library?
Doxygen helps here as well, during the creation of the API dox manual it optionally also creates a respective tag file.

Extending CMake Config files with API dox info

So for generating the references into some external library documentation, the location of its tag file, its name of the domain and its name of the virtual folder need to be known and explicitly added to the doxygen config file. Which is extra work, error-prone due to typos and might miss changes.

But it can be automated. All the information is known to the builder of that external library. And with the builder storing the information in standardized CMake variables with the external library’s CMake config files, it could also be automatically processed into the needed doxygen config entries in the build of your own library.

So a simple find_package(Bar) would bring in all the needed data, and passing Bar as name of the external library should be enough to let the to-be-written CMake macro derive the names of the variables with that data.

So with these theoretic considerations finally some real code as draft:

Patch uploaded for your review and comments

As possible extension of the Extra-CMake-Modules (ECM) the review request https://phabricator.kde.org/D2854 proposes 2 macros for making it simple and easy to add to the build of own libraries the generation of API dox manuals with links into external API dox manuals.

The call of the macro for generating the tag and qch files would e.g. be like this:

ecm_generate_qch(
    Foo
    VERSION "2.1.0"
    ORG_DOMAIN org.kde
    SOURCE_DIRS
        ${CMAKE_SOURCE_DIR}/src
    EXTERN
        Qt5Core
        Bar
    QCH_INSTALL_DESTINATION ${KDE_INSTALL_FULL_DATADIR}/qch
    TAGS_INSTALL_DESTINATION ${KDE_INSTALL_FULL_DATADIR}/qch
)

This would create and install the files ${KDE_INSTALL_FULL_DATADIR}/qch/Foo.qch and ${KDE_INSTALL_FULL_DATADIR}/qch/Foo.tags.

The call for generating the CMake Config file would be:

ecm_generate_package_apidox_file(
    ${CMAKE_CURRENT_BINARY_DIR}/FooConfigApiDox.cmake
    NAMES Foo
)

This generates the file FooConfigApiDox.cmake, whose content is setting the respective variables with the API dox metadata:

set(Foo_APIDOX_TAGSFILE "/usr/share/docs/Foo.tags")
set(Foo_APIDOX_QHP_NAMESPACE "org.kde.Foo")
set(Foo_APIDOX_QHP_NAMESPACE_VERSIONED "org.kde.Foo.210")
set(Foo_APIDOX_QHP_VIRTUALFOLDER "Foo")

This file would then be included in the FooConfig.cmake file

# ...
include("${CMAKE_CURRENT_LIST_DIR}/FooApiDox.cmake")
# ...

and installed together in the same directory.

Tightening and extending integration

In the current draft, for defining what source files doxygen should include for generating the API documentation, the macro ecm_generate_qch() takes simply a list of directories via the SOURCE_DIRS argument list, which then together with some wildcard patterns for typical C and C++ files is written to the doxygen config file.
Though being part of the build system, there might be more specific info available what files should be used, e.g. by using the PUBLIC_HEADER property of a library target. Anyone experienced with the usage of that or similar CMake features are invited to propose how to integrate this into the API of the proposed new macro.

Currently there is no standard installation directory for QCH files. So documentation viewers supporting QCH files need to be manually pointed to any new installed QCH file (exception are the QCH files for Qt, whose location can be queried by qmake -query QT_INSTALL_DOCS).
Instead it would be nice those viewers pick up such files automatically. How could that be achieved?

It would be also good if this or some similar API dox metadata system could be upstreamed into CMake itself, so there is a reliable standard for this, helping everyone with the creation of cross-project integrated API manuals. At least the metadata variable naming patterns would need to get wide recognition to get really useful. Anyone interested in pushing this upstream?

For proper documentation of public and non-public API perhaps the doxygen development could also pick up this old thread about missing proper support for marking classes & methods as internal. Or is there meanwhile a good solution for this?

On the product side, what manual formats beside QCH should be supported?
On the tool side, what other API documentation tools should be supported (perhaps something clang-based)?

Comments welcome here or on the phabricator page.

WIP: Plasma World Map Wallpaper & World Clock Applet, powered by Marble

The core of Marble, the virtual globe and world atlas, is a library, intended for reuse. Next to exposing its native C++ interface (see API dox of development version of the Marble library), there is also the option to use it in a QtQuick variant.

The Marble code repository itself holds a few application and tools based on the library. Additionally it also has extensions & plugins for other applications & workspaces, like the KIO thumbnailer plugins for previews of KML, KMZ, GPX & GEOJSON files in KIO-using file manager or file dialogs, a Plasma Runner plugin for looking up geo coordinates or a world clock Plasma applet.

Moving to Plasma5, Marble KRunner plugin arrives first

Having extensions for external applications & workspaces in your repo requires yourself to keep up-to-date with those externals. When Plasma5 appeared with the big switch to QML-only plugin architecture, that left the Marble-based plugins a little behind for now.

Good news is that the KRunner/Plasma Runner plugin was not affected by that, and with “KDE Applications 16.08” and the new Marble release will now also be out in a version working in all locales this month.

Reviving the Marble world clock plasmoid from 2009

The Marble world clock plasmoid, introduced already in 2009 (see bottom of page), though needs a full rewrite, from C++ to QML, which is a bigger undertaking. It also needs a port from KTimeZone & KTimeZoneWidget to QTimeZone, and in that course a cleanup of the mapping between time zones to geocoordinates.

Good news here is that some initial work has been done now and is up for review. A lot of the features are still missing, and there are some glitches (e.g. vertical offset, license info displayed). But it is a start, and the current state might be even fancy enough for some users already 🙂 So it should see a first release with “KDE Applications 16.12”.

It is a small mutation from a Plasma Applet to a Plasma Wallpaper

With the applet initially working, the idea came to have the map also shown fullscreen as wallpaper. While I could not find documentation how to do a wallpaper plugin, the existing code and examples though gave enough hints, and one hour later there was the first incarnation of a Marble-powered Plasma Wallpaper plugin.

See here a screenshot with both the World Map Wallpaper and the World Clock Applet in their current state:
Plasma World Map wallpaper & world clock applet, powered by Marble

Tell what kind of world maps you want to see as wallpaper

Because the World Map wallpaper can make full use of all Marble plugins, a lot of different kind of setups are possible:

  • Plain political map view
  • Globe view with stars in the background
  • Map view of just a continent/country
  • Satellite-photos-based view with live cloud rendering

There could be also plugin variants for the Moon, Venus or Mars.

So start Marble, play around with the settings to learn what is possible. Then tell in the comments what other kind of maps (or globe views) you would like to use for a wallpaper, so we could consider them as predefined settings at least.
As with the rewrite of the World Clock applet, a first release of the World Map wallpaper(s?) should happen with “KDE Applications 16.12”.

Create your own Plasma Wallpaper quickly by a new template

BTW, if you have other fancy ideas for Plasma “Wallpapers”, as by-product I did a template for Plasma Wallpaper plugins. It is currently up for review, so hopefully will be part of future KF5 Plasma frameworks releases. With that template, your first own wallpaper plugin is just a few clicks away e.g. in KAppTemplate or KDevelop.
Update: Until it arrives in releases, for now you can download the template file plasma-wallpaper.tar.bz2 from here and manually install it by moving that file into the folder /usr/share/kdevappwizard/templates/.
When playing around with the wallpaper settings, be aware there might be at least the issue of the wallpaper config page not reading current/default settings when switching the wallpaper type. As workaround, after switching the wallpaper type, one first has to close and reopen the dialog to be able to adjust the settings of the new wallpaper type.

Randa Meetings 2016 Part II: Marble

User Interface Needs Work For Facing More Users

The Randa Meetings 2016 were centered on bringing KDE technology on every device.

With regard to applications created by the KDE community, this means two things:

  1. Getting the source code to build for the operating systems on the target device
  2. Adapting the application UI to the specifics of the target device

The first point is by good parts covered by using the cross-platform library Qt and the cross-platform build system tool CMake, which eases enabling the build on many mainstream and also non-mainstream operating system. And the add-ons to these developed in the KDE community as shared elements, the Qt5 extensions KDE Frameworks (KF5) and the Extra CMake Modules (ECM), see lots of good work to cover more platforms as well.

The second point though is the larger hurdle here, when it comes to non-workstation-like devices, like tablets, smartphones, smartclocks or also TVs, fridges, in-vehicle entertainment screens. There especially the input options are quite different, which is also reflected in the UX of those devices in general.

Most existing applications created in the KDE community though have a UI done and optimized for the classic workstation-like setting. A setting, where input is done with a keyboard and a mouse. Where the main application window follows the concept of a menubar with submenus at the top, one or more toolbars below the menubar, a statusbar at the bottom, and the content in the middle.
And the source code architecture of most applications is often strongly coupled to this one UI, here and there even using the UI classes to store data belonging to the content that is viewed or edited, so no clean separation between UI layer and content model.

So adapting the application UI to the specifics of the target device most often means a big refactoring of the source code or perhaps even a complete rewrite. Something which is not done in a few weeks, but needs a lot of work over quite some time. Something for which people need motivation e.g. by owning and using such target devices and thus having the desire to also use those applications on them.

This showed to be a big hurdle even with devices which came with Qt pre-installed as the native UI toolkit of the operating system. Like the Meego devices, the BlackBerry 10 devices or the Sailfish OS devices. Only a few applications from the KDE community ever made an appearance there.

New: Direct Shipping To Devices By Developers

Another, though smaller issue to tackle for application developers often is to have to become a packager and distributor as well. Because other than with Linux distributions or the BSD port system, where the application developer just releases a tarball with the current sources which then is picked up the distributions’ packagers or the port system maintainers to make the software available in the package system of the respective operating systems, on other (usually proprietary) operating systems the end-user is to get their software from the application developers themselves directly or indirectly in a ready-to-install format, either by download from their homepages or via the so-called “app stores”. Which means more work to do at least.

Marble, a globe app going globally

One of the applications created in the KDE community whose developers managed to get it ported to more than the classic desktop computer, with regard to both operating system and UI, is the virtual globe and world atlas Marble. The reasons for that possibly can be seen in the capable and hard-working developers, but also in these:

  • Structuring the application in a modular way (so ports could be composed specifically for the target device use case, also some module being unported might not block the others being ported)
  • Avoiding dependencies on external libraries & modules if possible (so less things that also need to be available for the target platform)
  • Keeping dependencies on external libraries & modules optional (so if missing they do not block the rest of the application to be made working)

Myself I am more a happy user of the Desktop app Marble, not a core-contributor. So far I only did work on integration of Marble with the Plasma workspace, by writing some KRunner plugins to start Marble for geo coordinates or thumbnailer plugins for previews of KML, GPX & Co. in the file manager or file dialog.

Marble Maps on SailfishOS

Now, currently the Marble developers are working especially on Marble Maps, as a dedicated version of Marble for navigation purposes, with a first release planned for autumn this year. The main target platform right now is Android. As owner of a Jolla phone, which uses SailfishOS and thus Qt as part of the platform, last autumn I gave it a try to write a SailfishOS native variant of Marble Maps, to some quick first success. The uncertain future of Jolla then and thus of SailfishOS though kept me from investing more time into it. With Jolla these days still not sub-surface luckily and still sailing, I picked up those efforts now again, with the goal to merge my Maps variant soon into the Marble main code repository and have it part of the upcoming release, with feature-parity where possible.

With Qt being part of the operating system (and even more, slightly tuned there), there is some challenge though: SailfishOS is based on Qt 5.2 (with some backports from Qt 5.3), whereas either the typical unixoid target platforms themselves are on a more recent Qt or, with self-contained packages for Android, Windows & Co., one for good reasons picks the latest released Qt to ship along. Luckily so far the Marble core code could be kept working for all Qt versions. And there is some hope for more ease in the future, as work on updating SailfishOS to Qt 5.6 has been spotted.

So during this Randa Meetings I joined the Marble developers in their room for many days. For talking about roadmaps, features and plans with Marble Maps, when they had their heads not deep in the code. And learning more about the technologies behind navigation in Marble Maps, e.g. about how the turn-to-turn navigation is done.

More, I got a few hours long introduction into Marble’s elaborated rendering architecture by Torsten. Given my interest in static rendering of maps/globes for printing purposes or video frames generation, after nagging for that all the years I possibly have to scratch my itch here myself finally 🙂 Though that knowledge will also be helpful when perhaps starting a plugin for OpenTripPlanner, another thing I would like to use on my device with Marble Maps.

What excites me most about Marble currently is the on-going work on vector tile rendering, which will allow many new or improved features. See here for a sneak-peak some current WIP state when it comes navigating yourself around Trafalgar Square in London with Marble Maps for SailfishOS, partially rendered in-device from vectors already (you can spot the pixel-based tiles still used as base layer) :
Trafalgar Square in London rendered from Vector tiles in Marble Maps on SailfishOS

Marble, it is a wonderful toy not only for the kid in the developer 🙂 Get involved, either by making use of the Marble lib and plugins in your own app (and follow e.g. Subsurface and Digikam) or contribute directly to Marble’s own apps, lib and plugins.
To experience the wide set of possibilities with Marble, install the Marble desktop app (which IMHO should be renamed Marble Gallery 🙂 ) and walk through all the features, e.g. different sky objects as central globe, satellite display, time-lapses through sun or moon ellipses, interactive routing, or map editing. All waiting for you to compile your own special use-case app from them. Like happened with the Behaim globe app for Android.

Support us

The Randa Meetings and other sprints bring our software forward, and also to more people and more platforms. Please check out the fundraiser for the Randa Meetings, and consider to do your little contribution to get things going: