Help on TechBase, KDE Community Forums

From time to time techbase readers ask questions in the associated talk pages, e.g. with regard to bugs in the tutorials. Usually, they get no reply. This is mainly because noone gets even noticed about this. I've added a link to the KDE Community Forum now on top of each techbase page (MediaWiki:sidenotice). Maybe this helps that developers new to KDE get more help. A little...

News from the Holy Kate Land

Since we now all know that Kate is holy (thanks to rms. By accident, he obviously confused Kate with emacs, though) let's have a look at what's going on. In the last months Kate development is quite active, so here is a quick update:

• new: on-the-fly spell checking thanks to Michel Ludwig. Highlights include e.g. spell checking in comments of source code or latex parts. Also, constructs like sch\"on work in latex.
• extended scripting support in the command line, more on that later
• more and more mature vi input mode
• lots of bug fixing. quite impressive bug squashing by Pascal Létourneau for more than 4 months now
• lots of refactoring and code cleanups thanks to Bernhard!
• "Find in Files" appears by default again in the tool view,
• "File Browser" uses UrlNavigator, huge code cleanup
• convenience updates of syntax highlighting
• delayed highlighting of code folding ranges to prevent flickering on mouse move
• new command line commands: 'toggle-header' in the Open Header plugin. 'grep' and 'find-in-files'
• haskell and lilypond indenter
• much, much more, see commits for details.
  

Thanks to all contributors involved in Kate development. Keep it up :)

Followup on Kate's on-the-fly spellchecking

As there was a lot of feedback in the last blog, I'll answer some of the questions here.

> Where can I get this patch?
The relevant subversion revisions are r992778, r992779, r992780, r992784.

> Will it be available in 4.3, or only in 4.4?
As KDE 4.3 will be released end of the month, this feature will be available in KDE 4.4 and not earlier.

> Please, please tell me that it's powered by Sonnet, one of the most awaited KDE4 pillar by me...
Yes, it uses Sonnet :)
The old spellcheck dialog however still uses the old spellchecking code without Sonnet. Any volunteers to port this? Also, the on-the-fly spellchecking needs to be more configurable in Kate's config dialog, e.g. selecting the correct language.

> Thanks so much, this was the feature I was mostly longing for in kate.
Yes, it is one of the oldest reports with 1245 votes!

> What languages are supported support?
On-the-fly spellchecking works if the specific itemDatas are marked with spellChecking="true" or "1" in the xml highlighting definition files. UPDATE: On-the-fly spellchecking is enabled for all itemDatas by default in the xml highlighting defintion files. To disable it, you have to add spellChecking="false" or "0" to an itemData, e.g. at the end of the cpp.xml file:

<itemDatas>
<itemData name="Normal Text" spellChecking="0" />
...
<itemData name="String" />
<itemData name="String Char" spellChecking="0" />
<itemData name="Comment" />
<itemData name="Symbol" spellChecking="0" />
...
</itemData>

So we have to go through all .xml files and change the relevant parts in the itemDatas section. And that's where we need your help, as we don't know all the languages :) ...and if you want to test this feature, you first have to enable it in Tools > On-the-fly spellchecking.
PS: Is there a better name? Maybe Inline spellchecking? Any linguistic experts around? :)

On-the-fly spellchecking in Kate

Christoph just added an awesome and long awaited feature: on-the-fly spellchecking. 'Kate's xml highlighting files now have an additional attribute in the itemData section: spellChecking="true/false". C++ comments and strings can be spellchecked now :) Same for all other languages such as Latex. Really big thanks to Michel Ludwig for the patch, good work! Screenshot for latex highlighting:

Kate Test Regression Suite

This blog is mainly about documenting the process how to run Kate Part's test regression suite and probably not much use for other developers.

Step I: Setup the environment

1. create the file ~/.testkateregression. The content of this file is a single line pointing to the folder of the checkout of the test regression suite data. For me this is
   /home/dh/kde/tests/katetests/regression/
2. Create the folder where you checkout the suite
   mkdir /home/dh/kde/tests
3. Change to this folder
4. Checkout the data:
   svn co svn://anonsvn.kde.org/home/kde/trunk/tests/katetests
   

Now all the data is there for testing.

Step II: Run the test suite

1. go to your build directory of kate (e.g. kdelibs/build/kate/tests/
2. run the complete suite:
   ./testkateregression.shell
3. run specific tests, e.g. for the c indenter:
   ./testkateregression.shell -t indent/csmart

That's it.

Again System Load Viewer

It's in kdeplasma-addons now will be part of KDE 4.3. For those who can't wait we have released version 0.6 for KDE 4.2. I encourage developers to get their hands dirty and make it look even better. Another idea is to automatically use the plasma colors so it visually integrates better into the desktop. It's more than good enough though, as it has all the features of KDE3's System Monitor and some more (like support for showing all CPUs' usage). Have fun :)

Creating a Plugin System

On techbase we have a tutorial about extending your application with a plugin system. However, it's unfinished and some people need it as you can see on the talk page. So if you feel like passing your knowledge on, please fill in the missing sections to complete this tutorial. Just make sure to keep it as simple as possible, of course you can also change what's already there. Btw, we once had such a tutorial for KDE3, maybe you can copy some text from there. It would be awesome to have it for KDE4 as well :)

Feature Plans for KDE 4.2 / 4.3

Hi KDE developers: As KDE 4.2 is about to be released you should move all your features that did not make it into the KDE 4.2 release to the KDE 4.3 feature plan. Thanks! :)

Compile Time

Recompiling kdelibs, kdepimlibs, kdebase, kdesdk, kdeplasma-addons, playground/base/plasma all in all takes about 2h 13min on my t41p (1.7GHz, 1.5GB ram). It's still quite fast imo, but I imagine with today's hardware it's maybe 30 minutes? See below for detailed infos. I'm using mpyne's excellent kdesvn-build scripts to build KDE from trunk. I even believe this should be the default way for building trunk instead of the rough road we propose on techbase right now. Are there any reasons not to use kdesvn-build?

Building kdelibs (1/6)
       Preparing build system for kdelibs.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 44 minutes, and 56 seconds.
       Installing kdelibs.
       Overall time for kdelibs was 48 minutes, and 49 seconds.

Building kdepimlibs (2/6)
       Preparing build system for kdepimlibs.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 7 minutes, and 53 seconds.
       Installing kdepimlibs.
       Overall time for kdepimlibs was 8 minutes, and 14 seconds.

Building kdebase (3/6)
       Preparing build system for kdebase.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 52 minutes, and 35 seconds.
       Installing kdebase.
       Overall time for kdebase was 57 minutes, and 44 seconds.

Building kdesdk (4/6) # note: several subdirs commented out
       Preparing build system for kdesdk.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 8 minutes, and 26 seconds.
       Installing kdesdk.
       Overall time for kdesdk was 8 minutes, and 45 seconds.

Building kdeplasma-addons (5/6)
       Preparing build system for kdeplasma-addons.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 6 minutes, and 15 seconds.
       Installing kdeplasma-addons.
       Overall time for kdeplasma-addons was 6 minutes, and 24 seconds.

Building plasma (6/6) # this is: playground/base/plasma
       Preparing build system for plasma.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 3 minutes, and 25 seconds.
       Installing plasma.
       Overall time for plasma was 3 minutes, and 35 seconds.

Quickie: Faster compilation with cmake

It happens again and again that developers complain about cmake being too slow when compiling after a quick change. The solution is to use

• make $target/fast
• make install/fast

This way, cmake skips the dependency checking. As example, whenever I change code in kdelibs/kate, I type make katepart/fast and make install/fast.

System Load Viewer

Last year I've blogged about the missing system monitor with the three bars for the panel and about its port to Plasma. Meanwhile other developers also did a port called System Status. In a collaboration with them we finally have the applet back in KDE's subversion, the name is now "System Load Viewer" and it uses the data engine "systemmonitor" that already exists in KDE 4.2.
So if you want to have the plasmoid for your KDE4.2 desktop, it should be straightforward to compile/install.
On the screenshot you can see the plasmoid in action. There are two instances, one on the panel and one on the desktop. The one on the left is the KDE3 one.

It's worth to mention that the plasmoid already supports more featues than the KDE3 version. Features include:

• show all cpus (for computers with multicores)
• tooltip updates continuously
• nicer visualization (maybe needs some more tweaks)
  

As soon as the KDE 4.2 freeze is over we'll have to see where we can put the plasmoid to make sure it's available for KDE 4.3 :)

Kate Internals: The Undo/Redo System

The Kate Editor Component (also called KatePart) has its own undo/redo system. It did not change much since KDE2 and basically it is very simple. Meanwhile there are classes for undo/redo support in Qt as well. In fact both systems are very similar. Let's focus on KatePart's system for now.

Text Operations

First we have to take a look at what actions need to be saved. In KatePart this basically comes down to

• insert text or line
• remove text or line
• selection changes
• (and a few others like wrapping a line)

When typing text, each keystroke inserts a character. This is exactly one undo/redo item. As example, typing a character 'x' creates a new undo item:

• the content is 'x'
• the type is 'insert text'

Undo in this case means 'remove x'. Redo means 'insert x (again)'. The undo/redo history is just a list (more like a stack to be precise) of simple edit actions.

KateUndo Items

In KatePart, an undo item is represented by the class KateUndo:

class KateUndo {
 public:
   KateUndo (KateUndoGroup::UndoType type, uint line,
             uint col, uint len, const QString &text);
   ~KateUndo ();

   bool merge(KateUndo* u);

   void undo (KateDocument *doc);
   void redo (KateDocument *doc);

   inline KateUndoGroup::UndoType type() const;

   inline uint line () const;
   inline uint col () const;
   inline uint len() const;

   inline const QString& text() const { return m_text; }
};

Item Merging

Note the function KateUndo::merge(KateUndo* u); This functions merges two undo items of the same type if possible. For instance, typing 'hello world' inserts one undo item for every character, i.e. 11 undo items of type 'insert text'. Kate merges those 11 items into only 1 item with the string 'hello world'. Merging leads to less KateUndo items (less memory) and faster undo/redo replaying.

Item Grouping

What's still missing is the possibility to group several undo items together. Imagine you have selected the text 'hello world' and paste the text 'cheers' from the clipboard. What happens is this

1. remove selected text
2. insert text from clipboard

So there are two undo items of different type. They cannot be merged into only one KateUndo item. Though, we want to support undoing both items in one go, that's why we add several undo items into undo groups. In KatePart, this is done by the class KateUndoGroup:

class KateUndoGroup
{
  public:
    explicit KateUndoGroup (KateDocument *doc);
    ~KateUndoGroup ();

    void undo ();
    void redo ();

    enum UndoType { ... };

    void addItem (KateUndoGroup::UndoType type, uint line, uint col, uint len, const QString &text);

    void setUndoSelection (const KTextEditor::Range &selection);
    void setRedoSelection (const KTextEditor::Range &selection);

    void setUndoCursor(const KTextEditor::Cursor &cursor);
    void setRedoCursor(const KTextEditor::Cursor &cursor);

    bool merge(KateUndoGroup* newGroup,bool complex);

    void safePoint (bool safePoint=true);
};

Every KateUndo item belongs to one KateUndoGroup. A KateUndoGroup can have an arbitrary count of KateUndo items. In the example above we want to group 'remove selected text' and 'insert text' together. Grouping can be explicitely done in the code as follows (simplified version):

void KateDocument::paste ( KateView* view, QClipboard::Mode mode )
{
  QString s = QApplication::clipboard()->text(mode);

  editStart();
    view->removeSelectedText();
    insertText(pos, s, view->blockSelectionMode());
  editEnd();
}

Grouping: editStart()/editEnd()

The call of editStart() tells the document that an edit operation is running. All text operations are added to the current KateUndoGroup, until editEnd() is called. editStart() and editEnd() do reference counting, i.e. editStart() can be called nested as long as for each call of editStart() there is (finally) a call of editEnd().

Grouping: Cursors and Selections

Undoing the paste-action above should restore the selection if there was one previously. Redo (i.e. paste again) should remove the selection again. So there are two different types of selections: one before the undo group, and one after. That's why each undo group has the functions setUndoSelection() and setRedoSelection(). The same applies for the cursor position: We have to store two different cursor positions, one for undo and one for redo.
For instance, imagine we removed the text 'world'. Undo (i.e. insert 'hello') should set the cursor position to the end of 'hello'. Redo (i.e. remove 'hello') should set the cursor position to the start of it.

Luckily a programmer does not have to set the undo/redo cursor positions and text selections manually. undoStart() is called the first time editStart() is called. The closing editEnd() finally calls undoEnd(). So undoStart() sets the undo cursor position and undo text selection, while undoEnd() sets the redo cursor position and redo text selection.

Group Merging

The careful reader might have noticed KateUndoGroup::merge(). So merging of two groups is also supported. Whether text operations should be merged into an existing undo group can be controlled with KateDocument::setUndoDontMerge(). Pasting text for example set's this flag.

Undo and Redo

Every document in KatePart has two lists: An undo list, and a redo list. Suppose we have 10 KateUndoGroups in the undo list and the user invokes undo 4 times. Then the undo list only contains 6 items and the redo list 4. Now it is also possible to redo. However, typing text clears the redo list.

Document Modified Flag

KateDocument::updateModifed() is called to update the modified flag of a file. This update also relies on the current active undo group. Saving the file saves a pointer to the current undo group, and later we simply can check whether the current undo group is the active one. Pretty simple mechanism. However, there right now seems to be a bug you can reproduce as follows:

1. save doc [not modified]
2. type a character [modified]
3. undo [not modified]
4. type a character [modified]
5. undo [still modified]

Step 5 is incorrect, as the document is not modified here. Maybe now you have enough knowledge to fix this. Any takers? :)

Akademy 08: Kate Flashback

This Akademy's Kate changes include

• fix: drag & drop of text
  
• code completion: only show group header if the group name is not empty
• reintroduction of buffer blocks in Kate's document buffer (one buffer contains up to 4096 lines). The blocks build a linked list. Editing a 500 MB file kind of works now again. It's still rather slow, though.
• more speedup in Kate's document buffer
• Kate is using KEncodingProber instead of KEncodingDetector now
• generate internal version of KatePart automatically, so developers don't have to adapt it manually each release
• python encoding detection plugin that warns if encoding is not correct while saving
• new plugin: Backtrace browser, mainly for developers
• find in files: several speed optimizations
• find in files: progress indicator while search is active
• find in files: redesign of workflow. Search happens with non-modal dialog and results are shown in toolviews.
• lots of vi mode changes
• lots of bugs closed, mainly old ones
• some real bug fixes...
• things I forgot
  

Kate: Fast backtrace navigation

I've added a new plugin to kdesdk/kate/plugin: a backtrace browser. It's meant for developers and probably of no use for users. What does it do? It shows a backtrace delivered by gdb in a listview in a Kate toolview. Clicking on an item opens the selected file and jumps to the correct line number. It works for backtraces generated on your own machine, but it will also work for backtraces from other people, i.e. with /home/dummy/qt-copy/.../qwidget.cpp will still be found on other machines. For that to work, you have to index the directories where the source code is located.
Sometimes there are several files with the same name, e.g.

• trunk/kdegraphics/okular/generators/dvi/config.h
• trunk/kdepim/mimelib/mimelib/config.h

To pick the right choice, the plugin picks the last two parts of the url, in this case this would be

• dvi/config.h
• mimelib/config.h

and then usually finds the correct one. Indexing trunk/KDE and branches/KDE/4.1 of course will lead to a clash, now way to fix it. Maybe I could present a list of valid files to the user and let the user pick the right one. I don't think that's necessary though for now.

How to configure

1. Enable the plugin: go to Settings > Configure Kate > Application Plugins and enable 'Kate Backtrace Browser'
2. A config page appeared, so click on it and add the directories containing the source code
3. Clicking OK will start indexing. It will take some time (the index of kdesupport + kdelibs + kdepimlibs + kdebase + kdesdk + playground/plasma + plasma-addons + kdevplatform + kdegraphics is about 6MB)

When indexing is finished, open the toolview "Backtrace Browser". Now you can load a backtrace from the clipboard (e.g. when you clicked "Copy to Clipboard" in Dr. Konqi) or from a file.

Hope it's useful :)

C++ Template magic

Now that Johan talked about why the STL simply rocks I have to add a quick note about C++ templates in general, to be precise about template specialization. I've recently written a ring buffer template, something like

template <class T, int TSize>
class RingBuffer
{
public:
  RingBuffer();
  // ...
  void append(int count, T* first);
private:
  std::vector<T> buffer;
  int start;
  int end;
};

template <class T, int TSize>
RingBuffer<T,TSize>::RingBuffer()
{
  buffer.resize(TSize);
  start = 0;
  end = 0;
}

template <class T, int TSize>
void RingBuffer<T,TSize>::append(int count, T* first)
{
  // code omitted: make sure count elements fit, otherwise return
  // now there are two cases: either count elements fit completely,
  // or we have to wrap around at the end of the ring buffer.
  // the case of a wrap around is ignored here, too

  copyHelper(&buffer[start], first, count);
}

Now the function copyHelper looks like this:

template <class T, int TSize>
static inline void copyHelper(T* dest, T* src, int count)
{
  while (count--) {
    *dest++ = *src++;
  }
}

copyHelper simply assigns count elements with the operator=(), as T can also be a class. But in the case of T being e.g. a simple char this code part performs really bad. However, there is a solution: Template specialization. That means, we add another function copyHelper() and the compiler then is clever enough to pick the correct one:

template <>
static inline void copyHelper<char>(char* dest, char* src, int count)
{
  memcpy(dest, src, count);
}

Now the code is really fast in the case of char. In other words, with template specialization we can heavily tune template code in a lot of cases. And ideally, STL implementations do exactly this, don't they? Does Qt use template specialization for its template classes?

"I miss the applet which shows system usage"

...is one of the comments on Sune's blog. "I won’t call it a a must-have, but I’d love to have it back :D". Well, I call it a must have, that's why I've ported it to KDE4 a week ago:The code is not in svn yet (here is the code for now). I still had issues with correct resizing and believe there was also a bug in the plasma lib when I developed it which seems to be fixed now. I didn't have time for more testing, but finally this has to go in for KDE 4.2 :)
Besides that, I'm not yet content with how the data engine publishs data. The situation is that all system monitor data is extracted in one tick, but I still would like to publish it in three categories: cpu, mem and swap. But if I do so, an applet's dataUpdated() function is called 3 times and that's not what I want. So for now I put the data into the category systemmonitor and named the items cpu-total, cpu-kernel, ..., swap-total, swap-used, mem-total, mem-kernel, mem-cached, ... However the other way of categorization would be better :) Any ideas?

Offline for 3 months

I don't have internet the next 3 months. Maybe here and there at some point. That means I can not review the techbase changes. Also, I'll be rather silent wrt kate/kwrite.

Do you understand the word HTML?

During the Kate developer meeting we also thought about simplifying KWrite and how to make the decision whether KWrite should be launched in full featured mode or in a stripped version. ...well, and we found a really funny idea:


Note, that this would even work, the question would be rather annoying, though :) The solution right now is to always start KWrite in a simple mode. Mostly only actions are hidden in the menus (@distributors: kdelibs/kate/data/katepartsimpleui.rc), but you can also change c++ code at Kate part level, as there are some functions:

• bool KateDocument::simpleMode() (kate part internal), and
• bool KTextEditor::Editor::simpleMode() (along with setSimpleMode())

This way config dialogs can be adapted depending on the mode as well. Ah, and if you want the normal mode back, go into the KWrite settings and enable [x] Enable Developer Mode. Then restart KWrite.

PS: Tackat, we came up with this image before our phone call. That's why it was really funny when you said HTML is something that should not be removed. hehe... :)

Kate Meeting: Day 1 and 2

The Kate Developer Meeting was a productive weekend and once more shows how important the developer sprints are. The summary will be on the dot shortly. Work already started on several topics. As everyone want screenshots, here we go: The new annotation interface available now in KTextEditor can be use to e.g. show svn annotations directly in kate:
basysKom's coffee maching is simply the best: It can do everything, you just have to press the right key combos:

Kate Meeting: Day 0

Finally it all begins: Anders and Joseph arrived at basysKom and we've started to discuss some things we want to do for KDE 4.1. Later, we are going to meet with the rest of the attendees in a restaurant to get to know each other. The official start of the meeting is tomorrow morning. If you are interested in contributing to Kate, just join #kate on irc.kde.org. I'm looking forward to the next two days :)