Scripting Kate

In my last blog I explained Kate's scripting features in KDE 4.4. To better understand how scripting can be used let's look at some use cases.

• join lines: This feature request wants the action "join lines" to not join different paragraphs, i.e. not remove empty lines. We have not implemented this wish, as there are probably users who prefer the current behaviour. This request can be fixed by writing a small script that joins the lines according to the user's wishes.


• reformat paragraph: An intelligent reformatter for paragraphs. Should be rather straight forward to implement.


• XML tools: In KDE3, Kate once had a xmltools plugin. Unfortunately noone ported it to KDE4. The plugin provided lots of very useful features for xml editing. For example, you could select text and then wrap it with xml elements, e.g. "text" would become "<para>text</para>". This is a perfect example for a command line script as well. Any volunteers? :)

Scripting also brings us closer to fixing the following reports:

• macro system: Kate still does not have a macro system. A macro can be interpreted as a group of scripts, executed in a sequence (more or less). The vi input mode already supports pretty complex commands, and the code for scripting is all there. It's just a matter of putting this together so it's usable for users.


• word count: maybe the word count features can be implemented by a script (too slow?). Problem is, that a script cannot show dialogs etc.

To make scripting an even better experience, we still need to implement binding shortcuts to scripts. Again: any volunteers? :)

Extending Kate with Scripts

As mentioned in one of my last blogs, there has been quite some work for JavaScript scripting support in trunk recently. So what will Kate 3.4 (KDE 4.4) will bring? This is explained in detail in the sections:

• Indentation Scripting
  
• Command Line Scripting
• Some Remarks
  

The scripting allows to extend Kate with lots of little helper functions. All users can add scripts as they wish. The documentation here is copied from the official Kate handbook. So instead of using this blog as reference, please use the handbook later for an up-to-date version. To script something useful, you'll need to know the scripting API. All the available functions are documented in the section Scripting API in the handbook as well. Enough said, here we go:

Extending Kate with Scripts

Since Kate 3.4 in KDE 4.4 the Kate editor component is easily extensible by writing scripts. The scripting language is ECMAScript (widely known as JavaScript). Kate supports two kinds of scripts: indentation and command line scripts.

Indentation Scripts

Indentation scripts - also referred as indenters - automatically indent the source code while typing text. As example, after hitting the return-key code the indentation level often increases.

The following sections describe step by step how to create the skeleton for a simple indenter. As first step, create a new *.js file called e.g. javascript.js in the local home folder $KDEHOME/share/apps/katepart/script.

The Indentation Script Header

The header of the file javascript.js is embedded in a comment and is of the following form

/* kate-script
* name: JavaScript
* author: Example Name 
* license: BSD
* revision: 1
* kate-version: 3.4
* type: indentation
* required-syntax-style: javascript
* indent-languages: javascript
* priority: 0
*
* A line without colon ':' stops header parsing. That is, you can add optional
* text here such as a detailed license.
*/

Each entry is explained in detail now:

• kate-script [required]: This text string has to appear in the first line of the *.js file, otherwise Kate skips the script.
• name [required]: This is the indenter name that appears in the menu Tools->Indentation and in the configuration dialog.
• author [optional]: The author's name and contact information.
• license [optional]: Short form of the license, such as BSD or LGPLv3.
• revision [required]: The revision of the script. This number should be increased whenever the script is modified.
• kate-version [required]: Minimal required Kate version.
• type [required]: The type must be “indentation”, otherwise Kate skips this script.
• required-syntax-style [optional]: Comma separated list of required syntax highlighting styles. This is important for indenters that rely on specific highlight information in the document. If a required syntax style is specified, the indenter is available only when the appropriate highlighter is active. This prevents “undefined behavior” caused by using the indenter without the expected highlighting schema. For instance, the Ruby indenter makes use of this in the files ruby.js and ruby.xml.
• indent-languages [optional]: Comma separated list of syntax styles the indenter can indent correctly, e.g.: c++, java.
• priority [optional]: If several indenters are suited for a certain highlighted file, the priority decides which indenter is chosen as default indenter.

Kate reads all pairs of the form “key:value” until it cannot fine a colon anymore. This implies that the header can contain arbitrary text such as a license as shown in the example.

The Indenter Source Code

Having specified the header this section explains how the indentation scripting itself works. The basic skeleton of the body looks like this:

triggerCharacters = "{}/:;";
function indent(line, indentWidth, ch)
{
  // called for each newline (ch == '\n') and all characters specified in
  // the global variable triggerCharacters. When calling Tools->Align
  // the variable ch is empty, i.e. ch == ''.
  //
  // see also: Scripting API
  return -2;
}

The function indent() has three parameters:

• line: the line that has to be indented
• indentWidth: the indentation width in amount of spaces
• ch: either a newline character (ch == '\n'), the trigger character specified in triggerCharacters or empty if the user invoked the action Tools->Align.

The return value of the indent() function specifies how the line will be indented. If the return value is a simple integer number, it is interpreted as follows:

• return value -2: do nothing
• return value -1: keep indentation (searches for previous non-blank line)
• return value 0: numbers >= 0 specify the indentation depth in spaces

Alternatively, an array of two elements can be returned:

• return [ indent, align ];

In this case, the first element is the indentation depth like above with the same meaning of the special values. However, the second element is an absolute value representing a column for “alignment”. If this value is higher than the indent value, the difference represents a number of spaces to be added after the indentation of the first parameter. Otherwise, the second number is ignored. Using tabs and spaces for indentation is often referred to as “mixed mode”.

Consider the following example: Assume using tabs to indent, and tab width is set to 4. Here, <tab> represents a tab and '.' a space:

1: <tab><tab>foobar("hello",
2: <tab><tab>......."world");

When indenting line 2, the indent() function returns [8, 15]. As result, two tabs are inserted to indent to column 8, and 7 spaces are added to align the second parameter under the first, so that it stays aligned if the file is viewed with a different tab width.

A default KDE installation ships Kate with several indenters. The corresponding JavaScript source code can be found in $KDRDIR/share/apps/katepart/script.

Developing an indenter requires to reload the scripts to see whether the changes behave appropriately. Instead of restarting the application, simply switch to the command line and invoke the command reload-scripts.

If you develop useful scripts please consider contributing to the Kate Project by contacting the mailing list.

Command Line Scripts

As it is hard to satisfy everyone's needs, Kate supports little helper tools for quick text manipulation through the built-in command line. For instance, the command sort is implemented as script. This section explains how to create *.js files to extend Kate with arbitrary helper scripts.

Command line scripts are located in the save folder as indentation scripts. So as first step, create a new *.js file called myutils.js in the local home folder $KDEHOME/share/apps/katepart/script.

The Command Line Script Header

The header of each command line script is embedded in a comment and is of the following form

/* kate-script
* author: Example Name 
* license: BSD
* revision: 1
* kate-version: 3.4
* type: commands
* functions: sort, format-paragraph
*
* A line without colon ':' stops header parsing. That is, you can add optional
* text here such as a detailed license.
*/

Each entry is explained in detail now:

• kate-script [required]: This text string has to appear in the first line of the *.js file, otherwise Kate skips the script.
• author [optional]: The author's name and contact information.
• license [optional]: Short form of the license, such as BSD or LGPLv3.
• revision [required]: The revision of the script. This number should be increased whenever the script is modified.
• kate-version [required]: Minimal required Kate version.
• type [required]: The type must be 'commands', otherwise Kate skips this script.
• functions [required]: Comma separated list of commands in the script.

Kate reads all pairs of the form “key:value” until it cannot fine a colon anymore. This implies that the header can contain arbitrary text such as a license as shown in the example. The value of the key functions is a comma separated list of command line commands. This means a single script contains an arbitrary amount of command line commands. Each function is available through Kate's built-in command line.

The Script Source Code

All functions specified in the header have to be implemented in the script. For instance, the script file from the example above needs to implement the two functions sort and format-paragraph. All functions have the following syntax:

function (arg1, arg2, ...)
{
  // ... implementation, see also: Scripting API
}

Arguments in the command line are passed to the function as arg1, arg2, etc. In order to provide documentation for each command, simply implement the 'help' function as follows:

function help(cmd)
{
  if (cmd == "sort") {
      return "Sort the selected text.";
  } else if (cmd == "...") {
      // ...
  }
}

Executing help sort in the command line then calls this help function with the argument cmd set to the given command, i.e. cmd == "sort". Kate then presents the returned text as documentation to the user.

Developing a command line script requires to reload the scripts to see whether the changes behave appropriately. Instead of restarting the application, simply switch to the command line and invoke the command reload-scripts.

If you develop useful scripts please consider contributing to the Kate Project by contacting the mailing list.

Final Remarks

Right now, it's not possible to assign shortcuts to command line commands. Thus, there is no way of executing scripted commands with shortcuts. Volunteers wanted!

The command line scripting can be accessed for all KTextEditor users through the KTextEditor::CommandInterface. That is, you can query a specific command and execute it with arbitrary parameters (The parameter cmd contains the command itself including all arguments. Example: cmd = "goto 65").

Kate's command line itself is actually a quite powerful tool. It's a little bit sad that it's rather unknown. If you want to know more, just invoke "View -> Swith to command line" (shortcut: F7) and start typing text. More details are in the Kate handbook as well.

The Kate scripting API can be found here.

Help on TechBase, KDE Community Forums

From time to time techbase readers ask questions in the associated talk pages, e.g. with regard to bugs in the tutorials. Usually, they get no reply. This is mainly because noone gets even noticed about this. I've added a link to the KDE Community Forum now on top of each techbase page (MediaWiki:sidenotice). Maybe this helps that developers new to KDE get more help. A little...

News from the Holy Kate Land

Since we now all know that Kate is holy (thanks to rms. By accident, he obviously confused Kate with emacs, though) let's have a look at what's going on. In the last months Kate development is quite active, so here is a quick update:

• new: on-the-fly spell checking thanks to Michel Ludwig. Highlights include e.g. spell checking in comments of source code or latex parts. Also, constructs like sch\"on work in latex.
• extended scripting support in the command line, more on that later
• more and more mature vi input mode
• lots of bug fixing. quite impressive bug squashing by Pascal Létourneau for more than 4 months now
• lots of refactoring and code cleanups thanks to Bernhard!
• "Find in Files" appears by default again in the tool view,
• "File Browser" uses UrlNavigator, huge code cleanup
• convenience updates of syntax highlighting
• delayed highlighting of code folding ranges to prevent flickering on mouse move
• new command line commands: 'toggle-header' in the Open Header plugin. 'grep' and 'find-in-files'
• haskell and lilypond indenter
• much, much more, see commits for details.
  

Thanks to all contributors involved in Kate development. Keep it up :)

Followup on Kate's on-the-fly spellchecking

As there was a lot of feedback in the last blog, I'll answer some of the questions here.

> Where can I get this patch?
The relevant subversion revisions are r992778, r992779, r992780, r992784.

> Will it be available in 4.3, or only in 4.4?
As KDE 4.3 will be released end of the month, this feature will be available in KDE 4.4 and not earlier.

> Please, please tell me that it's powered by Sonnet, one of the most awaited KDE4 pillar by me...
Yes, it uses Sonnet :)
The old spellcheck dialog however still uses the old spellchecking code without Sonnet. Any volunteers to port this? Also, the on-the-fly spellchecking needs to be more configurable in Kate's config dialog, e.g. selecting the correct language.

> Thanks so much, this was the feature I was mostly longing for in kate.
Yes, it is one of the oldest reports with 1245 votes!

> What languages are supported support?
On-the-fly spellchecking works if the specific itemDatas are marked with spellChecking="true" or "1" in the xml highlighting definition files. UPDATE: On-the-fly spellchecking is enabled for all itemDatas by default in the xml highlighting defintion files. To disable it, you have to add spellChecking="false" or "0" to an itemData, e.g. at the end of the cpp.xml file:

<itemDatas>
<itemData name="Normal Text" spellChecking="0" />
...
<itemData name="String" />
<itemData name="String Char" spellChecking="0" />
<itemData name="Comment" />
<itemData name="Symbol" spellChecking="0" />
...
</itemData>

So we have to go through all .xml files and change the relevant parts in the itemDatas section. And that's where we need your help, as we don't know all the languages :) ...and if you want to test this feature, you first have to enable it in Tools > On-the-fly spellchecking.
PS: Is there a better name? Maybe Inline spellchecking? Any linguistic experts around? :)

On-the-fly spellchecking in Kate

Christoph just added an awesome and long awaited feature: on-the-fly spellchecking. 'Kate's xml highlighting files now have an additional attribute in the itemData section: spellChecking="true/false". C++ comments and strings can be spellchecked now :) Same for all other languages such as Latex. Really big thanks to Michel Ludwig for the patch, good work! Screenshot for latex highlighting:

Kate Test Regression Suite

This blog is mainly about documenting the process how to run Kate Part's test regression suite and probably not much use for other developers.

Step I: Setup the environment

1. create the file ~/.testkateregression. The content of this file is a single line pointing to the folder of the checkout of the test regression suite data. For me this is
   /home/dh/kde/tests/katetests/regression/
2. Create the folder where you checkout the suite
   mkdir /home/dh/kde/tests
3. Change to this folder
4. Checkout the data:
   svn co svn://anonsvn.kde.org/home/kde/trunk/tests/katetests
   

Now all the data is there for testing.

Step II: Run the test suite

1. go to your build directory of kate (e.g. kdelibs/build/kate/tests/
2. run the complete suite:
   ./testkateregression.shell
3. run specific tests, e.g. for the c indenter:
   ./testkateregression.shell -t indent/csmart

That's it.

Again System Load Viewer

It's in kdeplasma-addons now will be part of KDE 4.3. For those who can't wait we have released version 0.6 for KDE 4.2. I encourage developers to get their hands dirty and make it look even better. Another idea is to automatically use the plasma colors so it visually integrates better into the desktop. It's more than good enough though, as it has all the features of KDE3's System Monitor and some more (like support for showing all CPUs' usage). Have fun :)

Creating a Plugin System

On techbase we have a tutorial about extending your application with a plugin system. However, it's unfinished and some people need it as you can see on the talk page. So if you feel like passing your knowledge on, please fill in the missing sections to complete this tutorial. Just make sure to keep it as simple as possible, of course you can also change what's already there. Btw, we once had such a tutorial for KDE3, maybe you can copy some text from there. It would be awesome to have it for KDE4 as well :)

Feature Plans for KDE 4.2 / 4.3

Hi KDE developers: As KDE 4.2 is about to be released you should move all your features that did not make it into the KDE 4.2 release to the KDE 4.3 feature plan. Thanks! :)

Compile Time

Recompiling kdelibs, kdepimlibs, kdebase, kdesdk, kdeplasma-addons, playground/base/plasma all in all takes about 2h 13min on my t41p (1.7GHz, 1.5GB ram). It's still quite fast imo, but I imagine with today's hardware it's maybe 30 minutes? See below for detailed infos. I'm using mpyne's excellent kdesvn-build scripts to build KDE from trunk. I even believe this should be the default way for building trunk instead of the rough road we propose on techbase right now. Are there any reasons not to use kdesvn-build?

Building kdelibs (1/6)
       Preparing build system for kdelibs.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 44 minutes, and 56 seconds.
       Installing kdelibs.
       Overall time for kdelibs was 48 minutes, and 49 seconds.

Building kdepimlibs (2/6)
       Preparing build system for kdepimlibs.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 7 minutes, and 53 seconds.
       Installing kdepimlibs.
       Overall time for kdepimlibs was 8 minutes, and 14 seconds.

Building kdebase (3/6)
       Preparing build system for kdebase.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 52 minutes, and 35 seconds.
       Installing kdebase.
       Overall time for kdebase was 57 minutes, and 44 seconds.

Building kdesdk (4/6) # note: several subdirs commented out
       Preparing build system for kdesdk.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 8 minutes, and 26 seconds.
       Installing kdesdk.
       Overall time for kdesdk was 8 minutes, and 45 seconds.

Building kdeplasma-addons (5/6)
       Preparing build system for kdeplasma-addons.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 6 minutes, and 15 seconds.
       Installing kdeplasma-addons.
       Overall time for kdeplasma-addons was 6 minutes, and 24 seconds.

Building plasma (6/6) # this is: playground/base/plasma
       Preparing build system for plasma.
       Running cmake...
       Compiling, attempt 1...
       Build succeeded after 3 minutes, and 25 seconds.
       Installing plasma.
       Overall time for plasma was 3 minutes, and 35 seconds.

Quickie: Faster compilation with cmake

It happens again and again that developers complain about cmake being too slow when compiling after a quick change. The solution is to use

• make $target/fast
• make install/fast

This way, cmake skips the dependency checking. As example, whenever I change code in kdelibs/kate, I type make katepart/fast and make install/fast.

System Load Viewer

Last year I've blogged about the missing system monitor with the three bars for the panel and about its port to Plasma. Meanwhile other developers also did a port called System Status. In a collaboration with them we finally have the applet back in KDE's subversion, the name is now "System Load Viewer" and it uses the data engine "systemmonitor" that already exists in KDE 4.2.
So if you want to have the plasmoid for your KDE4.2 desktop, it should be straightforward to compile/install.
On the screenshot you can see the plasmoid in action. There are two instances, one on the panel and one on the desktop. The one on the left is the KDE3 one.

It's worth to mention that the plasmoid already supports more featues than the KDE3 version. Features include:

• show all cpus (for computers with multicores)
• tooltip updates continuously
• nicer visualization (maybe needs some more tweaks)
  

As soon as the KDE 4.2 freeze is over we'll have to see where we can put the plasmoid to make sure it's available for KDE 4.3 :)