The best new feature in the soon to be released version 0.4 of Okteta, the KDE hex editor, coming with the KDE SC 4.4, is surely the Structures tool (see also the “All new Okteta features for KDE SC 4.4 in a picture”). Because this tool is just a few months old, his author Alex concentrated on the code and not on the documentation, so the Okteta docs in the 4.4 branch have been missing any note of it. But Burkhard Lück, one of the heros of KDE documentation, pushed Alex and me to update the documentation in trunk at least with some quickly written text, so he might be able to backport it to 4.4, with success. If all works out, the updated doc will be part of the 4.4.1 release. So spend Burkhard a big $favourite_drink if this makes you happy 🙂
(Update: The updated manual is now part of the releases starting with 4.4.1 and also online available.)
Until then, find following the excerpt regarding the creation of structure definition. It is lazily copy-pasted from the meinproc4 generated html, hopefully doesn’t break too much of webpages embedding this. Please give this tool and the docs a try (e.g. if testing the RC1, where Okteta is part of kdeutils), feedback is very much welcome:
The Structures tool enables analysing and editing of byte arrays
based on user-creatable structure definitions, which can be built from
arrays, unions, primitive types and enum values.
It has an own settings dialog, which can be reached by using the
Settings button. There are various options that can be
configured, like the style (decimal, hexadecimal or binary) in which the
values are displayed. Moreover it is possible to choose which structure
definitions get loaded and which structures are shown in the view.
Structures are defined in Okteta Structure Definition files (based
on XML, with the file extension .osd).
Additionally a .desktop file contains
metadata about that structure description file, such as author, homepage
and license.
Currently there is no built-in support for creating, editing or
installing structure definitions, so this must be done manually as
described in the next sections.
Installing structure definitions
The Structures tool looks for structure descriptions in the
subdirectory share/apps/okteta/structures/
of the user’s directory for program data (find that by executing
kde4-config --localprefix).
You may need to create this directory if there are not any structure
definitions installed yet.
In that subdirectory there is an own subdirectory for each structure
definition, which contains both the .desktop
file and the .osd file of that definition.
E.g. with the program data directory .kde
and a structure definition named ExampleStructure there is the directory
.kde/share/apps/okteta/structures/ExampleStructure,
which contains a file ExampleStructure.desktop and a
file ExampleStructure.osd.
If you have installed a new structure definition by creating such a
subdirectory with the two files or edited them, you need to restart Okteta
and open the settings dialog of the Structure tool. There select the
Structures Management tab and make sure the relevant
structure definition is checked. Then use the button Apply
Changes, switch to the Structures tab and
make sure the desired element is listed on the right-hand side.
Sharing structure definitions
For common structures you may not need to create a definition yourself,
but instead can reuse an already existing definition from places like
kde-files.org store.kde.org (Updated in 2016).
You also may want to share a definition yourself. To do so, create a
file archive (e.g. a zipped tar archive, .tgz)
containing just the subdirectory with the .osd
file and the .desktop file. From the
example in the last section this would be the subdirectory
ExampleStructure.
If you adhere to this format future versions of the Structure tool
will be able to provide automatical download and install of these definitions.
Creating your own structure definitions
A first structure definition
To start we create a definition for a very simple test structure
containing only integral data types (one char and one 32-bit int).
They would be expressed in C/C++ as:
struct simple {
char aChar;
int anInt;
};
The first step is writing the .osd file,
which we will call simplestruct.osd:
which is fairly similar to the C/C++ definition.
The type attribute can be any of the following:
| Char |
| Int8 |
| UInt8 |
| Int16 |
| UInt16 |
| Int32 |
| UInt32 |
| Int64 |
| UInt64 |
| Float |
| Double |
| Bool8 |
| Bool16 |
| Bool32 |
| Bool64 |
The next step is writing the .desktop
file for the metadata, which is just a set of key-value pairs. It
should be named simplestruct.desktop.
In this example it looks as such:
[Desktop Entry]
Encoding=UTF-8
Icon=arrow-up
Type=Service
ServiceTypes=KPluginInfo
Name=Simple test structure
Comment=A very simple test structure containing only two items
X-KDE-PluginInfo-Author=Alex Richardson
X-KDE-PluginInfo-Email=alex.richardson@gmx.de
X-KDE-PluginInfo-Name=simplestruct
X-KDE-PluginInfo-Version=1.0
X-KDE-PluginInfo-Website=http://www.plugin.org/
X-KDE-PluginInfo-Category=structure
X-KDE-PluginInfo-License=LGPL
X-KDE-PluginInfo-EnabledByDefault=false
These fields should all be pretty much self-explanatory, except for
X-KDE-PluginInfo-Name. This one must have the name
of the .osd file as its value
(without the .osd suffix).
Now create a subdirectory simplestruct
as described above, copy the two files to this directory, and restart
Okteta.
Next we create a definition of a more complex structure which we will
call “complex” and save in a file named complex.osd.
This structure will contain two arrays (one with fixed length and one
where the length is determined at runtime) as well as a nested structure
and a union.
This would correspond to the following in pseudo C/C++
struct complex {
uint8_t size;
union aUnion {
int8_t fourBytes[4];
};
struct nested {
char string[size] //not valid C++, references value of the uint8 size
};
};
Note
You can obviously only have dynamic length arrays
reference fields before the array.
Next we create the complex.desktop file just as
in the example before (make sure you set X-KDE-PluginInfo-Name
correctly) and also do the same to install both files.
A few example structure definitions can be found here.
This include for example the file header for PNG files.
An XML schema describing the structure of the .osd file can be found here.
If more information is needed feel free to contact me at (alex.richardson AT gmx.de)
Attracted by virtual constructs 
Hi,
This is like THE best feature for a hex editor!
I will definitely use it, I sometimes need to analyze binary files so this will really be nice. Thanks a lot!
In the project msoscheme there is a big structure description for streams in PowerPoint files. It would be great if okteta could support the simple structure definition language used there.
Wouldn’t it be possible for Okteta to parse c99 style structure definitions instead of xml?
That was originally my intent, but due to time limitations it was not possible. I started this end of October and feature freeze was in November. Since it was easier to parse XML, I chose that. But it is still on my TODO list. I will have to check if I have to write my own parser or if there is any AST-Builder I can reuse.
It might be interesting to support GCCXML files, which are XML representations C++ headers generated by GCCXML.
Alternatively, instead of Octeta supporting various structure definition formats directly, it might make sense to have standalone tools to convert other formats into .osd files, which could be written in Python and other languages, and maybe even included in a project’s build script to generate .osd definitions whenever a header is modified.
Here is the structure definition:
http://gitorious.org/msoscheme/msoscheme/blobs/master/src/mso.xml
Seems quite straightforward and fairly similar to the current XML-Definitions, I’d say it’s not unlikely that that feature might be in the next Okteta version. I’ll have a look at the schema and see if I can implement it.
This is a great feature. I’m just wondering whether it is possible to describe more complex structures like file formats where the flow of data is not necessarily sequential? I’m especially thinking about an ELF format structure definition, that would be a great tool to have for embedded systems hackers.
It is not possible yet, but I’m working on it. Please keep in mind that I only had 1,5 months until feature freeze for 4.4
Hopefully it will be possible with the next Okteta release or the one after that.
Cool, I’ll be looking forward for that feature then, and will probably write a couple of file format inspectors.
Please keep up the good work!
Is there any way to describe bitfields?
ATM not, but you should check trunk in 1-2 Months. All the necessary infrastructure is in place, only 1 more class has to be added.
Do I need root access to select my structure in the Settings dialog?
No, of course not. You only need read permissions if it is installed system-wide. If you install it locally to your home directory you obviously have read permission, so its no issue.
For some reason, when I bring up the Structure settings dialog and select the Structures tab and try to moved my installed structure into the used structures by highlighting my structure and clicking the right arrow, nothing happens.
Maybe there is an error in the structure. The latest version (which will be with 4.9 has a new dialog which shows all errors. In older versions error messages were just printed to the console. Maybe it helps if you start okteta in a terminal.
That was it!
I didn’t have the .osd file in the proper directory.
Thanks!
[…] a small set of features was added once in a while, most impressively Alexander Richardson’s Structures tool in 2010,. Until then in 2013 the port to Qt5/KF5 was done (also to a good degree by Alexander). […]