Adding API dox generation to the build by CMake macros

Online documentations of APIs are a fine thing: by their urls they provide some shared universal location to refer to and often also are kept up-to-date to latest improvements. And they serve as index trap for search engine robots, so more people have the chance to find out about some product.

But the world is sometimes not an online heaven:

• “my internet is gone!”
• documentation webserver is down
• documentation webserver is slooo….oo…ow
• documentation webserver no longer has documentation for that older version

So having also access to offline documentation as alternative can make some developers using your library happy. If you are using Qt in your projects, you yourself might even prefer to study the offline Qt API documentation installed on your local systems, either via the Qt Assistant program or as integrated in IDEs like Qt Creator or KDevelop. Even better, the offline documentation comes together with the library itself (e.g. via packages on Linux or *BSD distributions), so the documentation is always matching the version of the library.

Being a good developer you have already added lots of nice API documentation in your code. Now how to turn this into a separate offline documentation manual for the consumers of your library, without too much effort?
If your project uses CMake for the build system and your target audience is using documentation viewers like Qt Assistant, Qt Creator or KDevelop, read on for a possible solution.

Deploying doxygen & qhelpgenerator

Very possibly you are already using doxygen to generate e.g. an HTML version of the API dox for your library project. And you might know doxygen can output to more formats than HTML. One of them is the format “Qt Compressed Help” (QCH). Which is a single-file format the above mentioned documentation viewers support well. QCH support by doxygen even exists since 2009.

So, with doxygen & qhelpgenerator installed and a respective doxygen config file prepared, the build script of your library just needs to additionally call doxygen as part of another target, and there is the API documentation manual as part of your project build, ready to be deployed and packaged with your library.

A CMake macro for convenience

To not have everyone duplicate the needed build setup code, like checking for the needed tools being installed, and to not have to learn all the details of the doxygen config file, some CMake macro doing the dull work recommends itself to be written and to be distributed for reuse.

Before doing that though, let’s consider another aspect when it comes to documenting libraries: including documentation for code constructs used from external libraries.

Linking between offline API dox manuals

Especially for C++ libraries, where subclassing across classes from different libraries is common, one wants to have direct links into the documentation of external libraries, so the reader has transparent access to any info.

The Qt help system has an abstract addressing system which works independently from the actual location of the QCH files e.g. in the filesystem. These addresses use the scheme qthelp: and a domain namespace for identifying a given manual (e.g. org.qt-project.qtcore). Additionally there is the concept of a virtual folder which is also set as root directory within a manual (e.g. qtcore). So the address of the document for the class QString would be qthelp://org.qt-project.qtcore/qtcore/qstring.html, and the documentation viewer would resolve it to the manuals registered by the matching metadata.

Doxygen supports the creation of such links into external QCH files. For a given manual of an external library, the references are created based on the domain name, the virtual folder name and a so-called tag file for that library. Such a tag file contains all kind of metadata about classes, methods and more that is needed to document imported structures and to create the references.

The QCH files for Qt libraries are coming with such tag files. But what about your own library? What if somebody else should be able to create documentation for their code with links into the documentation of your own library?
Doxygen helps here as well, during the creation of the API dox manual it optionally also creates a respective tag file.

Extending CMake Config files with API dox info

So for generating the references into some external library documentation, the location of its tag file, its name of the domain and its name of the virtual folder need to be known and explicitly added to the doxygen config file. Which is extra work, error-prone due to typos and might miss changes.

But it can be automated. All the information is known to the builder of that external library. And with the builder storing the information in standardized CMake variables with the external library’s CMake config files, it could also be automatically processed into the needed doxygen config entries in the build of your own library.

So a simple find_package(Bar) would bring in all the needed data, and passing Bar as name of the external library should be enough to let the to-be-written CMake macro derive the names of the variables with that data.

So with these theoretic considerations finally some real code as draft:

Patch uploaded for your review and comments

As possible extension of the Extra-CMake-Modules (ECM) the review request https://phabricator.kde.org/D2854 proposes 2 macros for making it simple and easy to add to the build of own libraries the generation of API dox manuals with links into external API dox manuals.

The call of the macro for generating the tag and qch files would e.g. be like this:

ecm_generate_qch(
    Foo
    VERSION "2.1.0"
    ORG_DOMAIN org.kde
    SOURCE_DIRS
        ${CMAKE_SOURCE_DIR}/src
    EXTERN
        Qt5Core
        Bar
    QCH_INSTALL_DESTINATION ${KDE_INSTALL_FULL_DATADIR}/qch
    TAGS_INSTALL_DESTINATION ${KDE_INSTALL_FULL_DATADIR}/qch
)

This would create and install the files ${KDE_INSTALL_FULL_DATADIR}/qch/Foo.qch and ${KDE_INSTALL_FULL_DATADIR}/qch/Foo.tags.

The call for generating the CMake Config file would be:

ecm_generate_package_apidox_file(
    ${CMAKE_CURRENT_BINARY_DIR}/FooConfigApiDox.cmake
    NAMES Foo
)

This generates the file FooConfigApiDox.cmake, whose content is setting the respective variables with the API dox metadata:

set(Foo_APIDOX_TAGSFILE "/usr/share/docs/Foo.tags")
set(Foo_APIDOX_QHP_NAMESPACE "org.kde.Foo")
set(Foo_APIDOX_QHP_NAMESPACE_VERSIONED "org.kde.Foo.210")
set(Foo_APIDOX_QHP_VIRTUALFOLDER "Foo")

This file would then be included in the FooConfig.cmake file

# ...
include("${CMAKE_CURRENT_LIST_DIR}/FooApiDox.cmake")
# ...

and installed together in the same directory.

Tightening and extending integration

In the current draft, for defining what source files doxygen should include for generating the API documentation, the macro ecm_generate_qch() takes simply a list of directories via the SOURCE_DIRS argument list, which then together with some wildcard patterns for typical C and C++ files is written to the doxygen config file.
Though being part of the build system, there might be more specific info available what files should be used, e.g. by using the PUBLIC_HEADER property of a library target. Anyone experienced with the usage of that or similar CMake features are invited to propose how to integrate this into the API of the proposed new macro.

Currently there is no standard installation directory for QCH files. So documentation viewers supporting QCH files need to be manually pointed to any new installed QCH file (exception are the QCH files for Qt, whose location can be queried by qmake -query QT_INSTALL_DOCS).
Instead it would be nice those viewers pick up such files automatically. How could that be achieved?

It would be also good if this or some similar API dox metadata system could be upstreamed into CMake itself, so there is a reliable standard for this, helping everyone with the creation of cross-project integrated API manuals. At least the metadata variable naming patterns would need to get wide recognition to get really useful. Anyone interested in pushing this upstream?

For proper documentation of public and non-public API perhaps the doxygen development could also pick up this old thread about missing proper support for marking classes & methods as internal. Or is there meanwhile a good solution for this?

On the product side, what manual formats beside QCH should be supported?
On the tool side, what other API documentation tools should be supported (perhaps something clang-based)?

Comments welcome here or on the phabricator page.