Name needed for single-doc-per-window Okteta variant?

Wuuuush – and DesktopSummit is over. What, already? I would like some more weeks of this kind! Although being slightly ill, I still enjoyed it as much as possible. A big “Thank you!” also from me to all the organizers, helpers and supporters!

While hearing and seeing and talking about a lot of interesting things, I also managed to use the time to push that simple hex editor named Okteta, or rather the component system named Kasten it is based on, a little further, so the difference between a so-called “Multiple document interface” (MDI) and a “Single document interface” (SDI) is almost only by the set of used components in the code. I am actually in the camp of the people disliking the MDI approach, as it can be considered a non-integrated workspace in a workspace, so pretty much broken. The SDI variant of Okteta provides all the tools and features of the MDI variant, just the tools Filesystem and Documents and the related entries in the menu have been removed/are not added.

But this poses a question:
Should this be just a difference in the settings? Or should this variant instead be shown as a different program, like users see Kate and KWrite as different programs?
For some reasons I guess these should be two different programs, but then I need a new name for the SDI variant Or should the MDI version be renamed, e.g. to “Okteta Studio” or similar?
Please post/discuss proposals in the comments, or send me your idea by email (see About dialog of Okteta)!
Final code is not yet commited to the central repo, needs some cleanup/finishing and also depends on the decision above, so sorry, cannot yet be tried out (but then besides MDI vs. SDI there is also nothing else new to see yet).

During the Desktop Summit it was interesting to hear how many people actually already have made use of Okteta. Thank you, good to know Kasten code to work in real life
I even was told feature requests on the hall ways. The one late on the last day was even easy to do, it was having a sane default count of bytes per line, which would be 16, without an adaption to the current view width. That was fixed by a single line addition and quickly tested and commited, so at least an Italian user will be more happy with 0.7.1

All new Okteta features for KDE Apps 4.7 in a picture

KDE Apps 4.7 is out, and with it the hex editor Okteta 0.7!

For 0.6 and 0.5 I missed to give updates, like done for 0.4, 0.3, and 0.2.

So what is new since 0.4?

Not that much as in the early versions, since life has been busy for both Alex and me. So the biggest issues are yet to be solved: no smart handling of very large files (still completely loaded into memory), no storing of bookmarks. But besides that there is still nice progress:

(Eh, small glitch with the otherwise awesome new KWin in KDE Workspaces 4.7 is that shadows are not working for non-Oxygen styles, please ignore)

First thing to notice (besides the very first start of the program after updating to 0.7):

• much faster program start-up thanks to Alex’ analysis and improvements
  (what, you cannot see this in the picture? )

Other stuff:

• More support for char encoding/decoding:
  • New charset conversion tool: rewrites the bytes so the respective chars are the same as with the other charset
    (only 8-bit charsets supported, and unmatched chars are currently substituted with a value hardcoded to 0, setting is disabled)
  • New encodings: ISO-8859-14, ISO-8859-16, Codepage 874
  • Fixed: show bytes without a defined chars as such
    (missed before that some of the supported charsets are incomplete)
• Many improvements in the Structures tool by Alex:
  • Structure definitions can now also be “described” in JavaScript (see the manual)
  • Begin of a structure can be pinned to a specific offset
  • Structure definitions can be installed using KNewStuff directly e.g. from kde-files.org
  • Support for strings (various Unicode encodings) in definitions
• New formats available for “Export to file”/”Copy as”:
  • Base32 (classic, Base32Hex, z-base-32)
  • Base85 (Ascii85 for now)
  • Uuencoding (historical, Base64)
  • Xxencoding
  • S-Record (16,24,32-bit address size)
  • Intel Hex (8,16,32-bit address size)
• Added embedded terminal
  (might be useless to most, but only needed a few code lines to implement )
• Menu entries to set the count of bytes per line and per grouped bytes
• File info tool now estimates the mimetype also for the unstored/edited data in the working memory
• Bytearray views and tools are updated on change of default fixed font in System Settings

Oh, and headers for the Okteta libs and Kasten libs are finally installed since version 0.5, including the Qt Designer plugin. So if you are interested to make use of these libs in your own code, see for Kasten and OktetaKasten e.g. this blog entry (hm, Binspekt code needs updating to current API, will not compile) and the code of the plugin for KDevelop, for the Qt Designer plugin this blog entry and the example code.
But beware: the next version of the libs, coming with Okteta 0.8, will already have a different API/ABI, but that will be protected by proper namespacing.

For future versions there are still plenty of ideas for Okteta left, let’s see what gets realized

So for now “Happy bytes peeping and pushing!” also with the 0.7 version of Okteta!

Okteta and Kasten on MeeGo

Having been caught by some cold two weeks ago, I was still lucky to have recovered enough in time to go to the MeeGo conference in Dublin, sponsored by my employer openismus GmbH, joining a bunch of my colleagues. It was a really great conference, will be a nice challenge e.g. to match it with the Desktop Summit in Berlin next year (remember the date, August 6-12th). Having arrived there the evening before, we added our part to the success by helping out with putting up the large stickers to the windows, which gave directions and set MeeGo theming to the rooms (Openismus – your (now) also experts in applying large stickers to windows).

It was nice to see how many of the people on the floors were real life animations of the avatar images on Planet Gnome and Planet KDE. And even more great to see and hear KDE oldie and now director of the AMD Operating System Research Center, Chris Schläger, announcing that AMD is going to join the MeeGo project. Things are getting together which ought to get together!

A very welcome idea of Nokia and Intel to improve the motivation of the people to contribute to the MeeGo project was the give-away of a Lenovo ideapad for each conference participant. While the MeeGo Netbook release 1.1 is still anything but capable to make use of the multitouch feature of the ideapad’s screen, it for sure will be a nice device to experiment with new UI principles.

Which I am looking forward to do. While the work on my experimental Kasten framework has pretty much stalled this year, I hope I can pick that up again once my transition to Berlin has been completed (which could have been more smooth ). At least I took two days off and joined the KDE Mobile sprint starting the day after the MeeGo conference, hosted by KDAB in their Berlin office, thanks to them (incl. the delicious orange-tiramisu on sunday from Till’s wife’s birthday party, even if it was just a left-over ). While I happened to be part of the team which started the profile-oriented modulization of kdelibs at Tokamak 4 in the begin of this year, my only real contribution so far has been to propose (talk is cheap ) the removal of the deprecated api+code for the “mobile” profile, to help a little in shrinking the size of the packages of the KDE Platform/Framework, then lost track of this topic in general. The more I was surprised now how much has been done already, great effort by whoever was involved (mostly Kevin Ottens, I guess?).

With no real touch UI oriented code for Okteta and Kasten yet (okteta-mobile is mainly main() atm ) I used the sprint to join the “porting” efforts to MeeGo, which is basically rpm packaging. And removing some deprecated stuff here and there, as the kdelibs packages for now are created using the Mobile profile, thus having none of the deprecated APIs, quite nice to see how clean all the code is already regarding usage of these. Of course I worked on the packaging of kdeutils and also kdesdk, now that Okteta is part of kdesdk. The few hidden deprecated calls left in Okteta were quickly patched away, and there is was running on my new ideapad:

Sure, nothing as useful and shiny as others are doing, but then Kasten still has way to go Hopefully there soon will be long and cold winter nights and a new flat in Berlin. I know what I will do!

And while it is great to see GTK and now Qt being used and developed by the top companies, it is even greater to see other UI related FLOSS making their way into normal industry products as well, including upstream contributions, e.g. EFL on fridges. Things are really moving in good directions

Synchronisation with Kasten, just to filesystems now

Sometimes it seems with faster and faster JavaScript-JIT-compilers and Gears/HTML5 the current software stacks, written in other languages, like e.g. Qt/KDE are going to be legacy. Google Docs & Co. show what is possible, with the browser as the generic adapter to the platform.
But the current software stacks are fighting back. Things like the openSUSE Build Service make it possible to have your code compiled and ready to be delivered for several platforms, run natively by the hardware you actually bought to do so, instead of having some resources abused for virtual layers. It just needs code which does not (over-)use platform-specific constructs. The expansion of Qt/KDE to more and more architectures is a fine example here what is possible.
Curious what will be next, perhaps Javascript programs using alternatively binary blocks of code compiled for popular hardware, given the OS has built-in access control? Things are merging.

With this in mind I congratulate the people from AbiWord on their new version 2.8, which not only improves the support for the open spec’ed on-disk format OpenDocument (.odt), but especially improves on the collaboration features which integrate the Web 2.0. That looks really great. My only nitpick is that this seems to be a non-FLOSS webservice. I just hope they find a good business model so that the software for the server can be shared, too. Others, like Inkscape or hopefully soon KOffice, might be interested in such a server system, too.

I also congratulate the people from Étoilé, who just have started to send objects over XMPP. This all sounds so promising.

Realtime collaboration and such data model synchronisation is something I hope Kasten, my WIP framework for composing programs, one day supports out of the box, too. From the very beginning this has been considered in the design, even if it is not yet too visible. And it still will take some time to get there.

There is other stuff to do first, like making Okteta work with very large files. This isn’t done yet, I am still working on it, but have hopes this feature will make it into the next release. Working with very large files means only loading the parts which are currently interesting (e.g. solved by memmapping the file). Currently Kasten, the framework Okteta is built on, only has support for the concept of loading document objects completely from the remote storage (like the filesystem on disk) to the working memory, done by subclasses of AbstractModelSynchronizer. There is a specialized subclass AbstractModelFileSystemSynchronizer, which takes care for the common stuff on dealing with the filesystem, including loading/storing to remote filesystems with KIO::NetAccess. So the/your document-specific subclass has just to implement the de- and encoding of the data from a QFile to the working data model.

Now, AbstractModelFileSystemSynchronizer was not really finished, so before adding a new construction site in Kasten for the partial-loading of files I want to finish the old one for the complete loading. E.g. there was no support for a property State of the remote storage. Thanks to KDirWatcher there is now some easily done initial one. Investigations are still needed how to make sure there is no race-condition where another process might break into writing to the file, reading the file’s timestamp of the own modification and starting to watch for changes again.
Also, KDirWatcher does not work for Kio-slaves. So files loaded from them are just marked to be in Unknown state. Besides, if the network connection is down, they are set to Unreachable. Solid::Networking serves this with just a few lines again

See in the screenshot for the two state symbols now in the Documents tool: the left icon shows the working data state (saved changes, unsaved changes), the right one the remote storage state. The yellow flag means Unknown, in case you wonder, and turns into the Network-disconnected icon if you disconnect from the network, like pulling the plug (looks impressive if you have several files loaded e.g. by sftp:// ). Well, seems I need to apply for some more icons from our icon heros.

Other changes in this area:

• The action Revert is renamed to Reload. Revert will not do what the name says if the storage copy was modified since the loading of the document. Revert would rather mean undoing all changes, so that action semantic might rather appear in the Edit menu, for some quicker navigation through the change history.
• The actions Reload and Save are only available if they are useful and usable. E.g. they are disabled if there are no known changes in both the working copy of the document data and the storage copy or the connection to the remote storage is lost.
• Also new documents which are generated from random data or user input (like parameters) are set to modified, so closing them without adding any further content will still trigger the question if you really want to discard this perhaps unique data.

Okteta: Fighting the crisis with productivity from generators

(And if it is just random data)

There has been some productivity-related productivity in Kasten and Okteta development the last days, as can be seen by commit logs like “added: factories to create generators” :

Kasten, my WIP framework for composing programs, gained some AbstractModelDataGenerator class to serve as base class for data generating stuff. It also got a general Insert controller which plugs into the menu, offers all available data generators and has the chosen one generate its data (if parameterized after showing a config dialog) and then inserts it into the current document. And with the AbstractModelDataGenerator around, I extended the AbstractDocumentFactory to also create new documents from QMimeData and reworked the general New controller: it now not only offers to create an empty document, but also to create one prefilled with data from one of the data generators. Or by the data from the clipboard, which also serves QMimeData. And while I was at it, the shell now no longer rejects D’n'D data other than file urls to non-document areas (like next to tabs or the empty space on start) but passes it to the document factory to see if it can construct a new document from it, all by just a few lines of codes. Kasten may look bloated to some (including myself sometimes) but it has it’s value here and there

To see if it is working Okteta, the KDE 4 hex editor, which is built on Kasten, has got some generators:
For one Okteta since the first release had some simple Insert pattern controller, which now got ported to a plain data generator plugin (+ config editor). Then two weeks ago I found myself for the x-th time in life in the need for data in the whole range of all bytes, from 0 to 255. So I had done a clone of the pattern controller and already added as a Insert sequence controller (which started the idea for these general data generators). Got ported to a plain generator plugin, too.

Another generator perhaps useful might be a random data generator. So now there is one. It is based on srand(time(0)) and rand(), which may be of different quality depending of the platform’s libc. If you instead want a certain algorithm, please contribute and send me some code for it filling a QByteArray, licensed like the rest of the file, I am happy to add some alternative RNG to select from

Lots of TODOs and ideas coming with this, but works already. If you like screenshots, have a look:

And there has been even more productivity with regard to Okteta: First external contribution is coming with a new tool More on this soon…

Call for action: Write your own plugins for Okteta

Up to now I have written Okteta, the KDE 4 hex editor, completely on my own. This has pros and cons:
The biggest pro is that I do not have to argue a lot about design decisions, especially as I have some big plans with the co-developed framework Kasten, which Okteta is built on (to serve as testing area) and which might not yet be understood/followed by others (could just be a crazy idea, after all) as I have implemented only parts of it and also added unmarked intermediate shortcuts to get things running already.

The biggest con is that Okteta doesn’t grow as fast as it could.* So there might be some features you, dear Okteta user, are still missing from it. Additionally, remember that I do not really use Okteta myself, it really is almost only a testing area of Kasten for me (I still am happy if Okteta is already a useful tool to many, as this indirectly results in more testers for Kasten). And so I concentrate on things which help to develop Kasten further. I now turned to think about support for very large files, meaning working with data not completely copied to the working memory (just do not hold your breath for this ).

So, what about the binary filters (e.g. a bit switcher), your export plugins (e.g. Base64) or even complete tools (I can not even imagine) you are missing from Okteta and hoping for in a future release?

For the coming weeks I am going to show you how simple it should be to help yourself, in a series of short tutorials.

Okteta/Kasten has been designed very modular, with a lot of interfaces and plugin-prepared structures. For now everything is hardcoded, but as we have readwrite access to the sources this is of no great problem here

There is almost one month left for you to decide if you want to make your hands and keyboard dirty with an addition to Okteta, then the soft features freeze will hit the development branch for KDE 4.4. And only November 11th would set the barrier for a basically working solution, so should be enough time to get your plugin done, given it doesn’t include rocket science. Or you could be still pleased if it only will be part of KDE 4.5 next summer (well, still not assumed to include rocket science). In any case I will be happy to help you where doable.

The first tutorial will appear on sunday (September 27th) and teach you how to write a binary filter (Update: online now). So stay tuned if you are in need of a hex editor like Okteta and interested to give the implementation of your needed features yourself a try.

* For sure, the pace of development for Okteta has been quite good, just have a look at the automatic analysis done for Okteta at ohloh.net. 48,042 lines of code have been gathered in the last two or so years. Which, using LOC while admitting it’s an evil metric for code work, is estimated to have, taking the given default average salary of $ 55,000 / year, a “Project Cost” of some $ 641,343. I just hope I never will write myself this bill. Because the moment I do I am for one in big debt to someone and also in trouble because someone doesn’t pay my bill.
Another interesting metric seems behind the assigned property “Well-commented source code”: Well, I have spread the code over quite some files, and each and every file has 20 lines comment from the license header. The other comment I often leave is done with this pattern:

// TODO: some improvement

currently around 560 times. Instead almost all API is missing proper apidox. So, rather not well-commented. Still, I use expressive, thus pretty long names for classes and variables, so comments are not often needed (it’s obvious what ByteArraySourceCodeStreamEncoderConfigEditorFactory does, or?). But I wonder if this metric takes this into account

Polishing the tools for Okteta 0.4

While still avoiding to tackle the next big task for Okteta, supporting very large files, thanks to a strain in my right leg I have found some time to polish the tools a little.

Last night I worked on the input widgets for addresses/offsets and byte strings. Those enable the user to select one from several formats in which to do the input (auto detection still a TODO). While I wanted to make a widget similar to the one of the searchbar for Konqueror, with the current format to be selectable like the current engine in the left of the searchbar widget, I found this not easy to implement and the searchbar implementation not pleasing my needs (e.g. not focusable with keyboard, no method QComboBox::setEditedIcon() or similar). So I stayed with my old implementation where the format is selected in an external combobox, but which is now placed to the left of the input field and only shows an abbr. of the format type (“Hex”, “Dec”, …). The input field itself is a combobox which can and is used to store previous inputs (the popup still lacks a delegate to also show the corresponding format). These widgets are now used all over Okteta, just cmp. the screenshot below.

Already in trunk for some time are support in Kasten for inlined dialogs, with Okteta’s Goto dialog and the (new for 0.4) Select-range dialog (see screenshot) ported to it. Additionally all tool views are ported to the new AbstractToolWidget in Kasten which has support for default buttons which are activated on the press of the Return key, like you know from the dialogs. So if any widget in a tool view has focus (and doesn’t eat the Return key itself) one now longer needs to switch the focus to the button but can simply press Return (just seems that the Oxygen style does not give an extra style to default buttons?).

Also switched to use QFormLayout where possible. And found the great KIntNumInput, so any units are shown inside the input field (including bytes, as in number of bytes )

Just the Find and Replace dialogs may stay popup dialogs for a while. For one I have no idea how to best place all the options into an inline-dialog (e.g. the solution in Kate does not follow my workflow) without getting a monster dialog. Even with the Go-to and the Select-range inline-dialogs the window size is sometimes changed, which is a no-no IMHO. And for another on this change I want to implement search-as-you type, which is still lacking a proper foundation for worker threads and locking of document objects. So, support for very large files has a chance to be done before that.

New in Okteta: Splitted Views

Konqueror can do it. Kate can do it. KDevelop can do it.
And possibly others, too. So, Okteta should not stand back and also be able to split a view and have two or more of them at the same or multiple documents. Can be useful in times. And afterall, the Okteta library can do MVC since some years (back then had classes named KHexEdit, even if hasn’t been used by and never shared anything with the program named KHexEdit, besides living in a subdir of it).

Sadly, all programs have their own implementation, which are also to a certain degree coupled with the rest of the program or the framework it is built on. So there is now another (draft of an) implementation, again coupled to a framework but this time to one with ambitions for more: the MultiViewAreas (working title) in Kasten.

While there are some glitches left most of the basic stuff is working, one can split views vertically and horizontically, here with the same approach like Sublime/KDevelop has:

See how on splitting also the settings are copied to the new view. Even better, it is tried to set the visible part in the new view aligned to the one in the old one, so it appears as if just the splitter and the new tabbar cut into the old view, and the user must not research the parts she has seen before.
(Tried as in: Currently broken for the use in Okteta, as some ensureCursorVisible() in the bytearray view widget moves the visible section around on the resizing after the split, workaround pending…)

The Kasten::MultiViewAreas is just another implementation of Kasten::AbstractGroupedViews, though hard coded for use in Kasten::ShellWindow temporarily. An alternative implementation of a multi view areas setup for a design like Kate has (no tabs, statusbar per view) will be the next to be done, so a developer using Kasten has some choice.

One more step on a long way yet for Kasten… But one less to be done

Binspekt KPart for reuse in KDevelop

Some days ago I wrote a mini-tutorial how to use the Okteta KPart in Konqueror/KDevelop. With the mind turned to the Okteta KPart I after that have taken on a long waiting task and removed some code duplication by porting it to the Kasten classes, now that these are available in shared libs. Which means, with KDE 4.4 you should have the same readonly actions available with the Okteta KPart that you also have in the Okteta program (not all done yet). Even better, I have just written a first code line to turn it into a ReadWritePart, so that you could also do the editing and saving in KDevelop. More on this later hopefully.

After having done this first wrapping of Kasten classes in a KPart I did another one, to learn more about how a generic KPart Kasten module might be best designed, to get the Kasten plug&play experience I am aiming at. The other program I am writing using the Kasten framework is Binspekt, a viewer (and perhaps editor) for binary files, that is executables, libraries, object files etc. It was basically copy,paste&replace, so here see the small and dirty Binspekt KPart inside the (amazing and great BTW) KDevelop, fresh from the KDE repository:

Binspekt is still prealpha (yet doesn’t crash, because it doesn’t do a lot) if you are interested. But it all has to start somewhere, no?

Kasten: WIP framework for composing programs

Since a few days kdeutils (from trunk) also installs some libraries, including public headers:

• Okteta libraries
• Kasten libraries

The Okteta libraries oktetacore and oktetagui provide some model and view classes for hex views of bytearrays. If you have some special data provider you could also subclass your own Okteta::AbstractByteArrayModel. So using the libs directly is now an option, if the KHexEdit interfaces from kdelibs don’t do it for you.

The Kasten libraries are far more interesting, at least to me. Kasten (former project title was Kakao), that is a project of mine where I try to develop a framework for composing programs from modules and interfaces by the model-view-controller pattern, with the pattern recursively applied to the view and the controller. Think KPart 2.0. So far the development was driven by the needs I had with Okteta (that program serving as the testing ground, now you know my motivation for Okteta. Byte arrays are just the most simple data structure.). The Kasten framework is still far from what I have in mind, but things at least are moving I try to express my ideas directly in code, so the ideas meet reality as soon as possible. Might be some more years before I am done, oh well.

Both libraries are still far from elaborated, especially Kasten, given that there has been only one usecase, that is the program Okteta. But you are invited to see if you already can make use of them. Just be prepared for changes. Which could also be done for your needs

The reason I turned to install these libraries now is that I want to reuse them in another project, working title Binspekt: It is about viewing and editing of binary files, that is executables, libraries, object files etc. I always forget about the parameters of objdump, objcopy, nm & Co., so I thought a GUI variant might be nice, and here we go (of course using the very same bfd library from the GNU binutils).

Obligatory screenshot of the current state, which is pre-alpha:

Find the code at playground/devtools/binspekt in the KDE repository, imported there a few hours ago.