Decentralised Regulations and XML

By Marin van Sprang and Laurens van den Oever

(This article originally appeared in <!ELEMENT, the magazine published by the Dutch SGML/XML user group)

The Dutch government wants to provide citizens with better access to official government information and has therefore decided to place the regulations of decentralised (local) government bodies on the internet by means of the project ‘Decentrale Regelgeving' (Decentralised Regulations). With this aim in mind the 'Centrale Voorziening Decentrale Regelgeving' (Centralised Facility for Decentralised Regulations) (CVDR) has been created, and all local government bodies will be required to enter their regulations in this system. For its maintenance and management an application has been developed on behalf of Unisys by the Deventer-based Stipp company on the basis of Xopus XML Editor. This is part of the programmeGovernment has the Answer(c)” by the ICTU Foundation

There are two ways via which local government bodies will be able to enter their regulations into the programme:

  1. Via a web service. This option is intended for local authorities that already publish their regulations on the internet in accordance with the developed standard. Via the web service, the regulations in XML are exported to the CVDR. The authorities concerned are responsible for enabling their systems to make use of the web system.
  2. For government bodies that do not have a system for publishing regulations on the internet, the CVDR offers a data entry module. 

The centralised facility publishes the regulations on the website Overheid.nl, but can also provide an XML export to government bodies that want to put their regulations on their own website.

In addition to a data entry module and web service, the application also consists of a workflow and conversion module. Xopus Company and Eden Design & Communication have developed a user-friendly entry module for this complex subject matter.

Stipp was responsible for the umbrella application and expertise regarding the managing and structuring of the legislation and regulations. In the meantime, the application has been developed and went live on 1 October 2008. This article describes some of the “challenges” that were encountered and the solutions that were chosen.

Technology versus reality

The entry module needed to comply with the following requirements, which at first glance seem somewhat incompatible: 

  • The authors do not need to have any affinity with technology.
  • XML is produced which is valid according to the XML Schema.
  • The application must function in a web browser.

There are different ways to comply with all these requirements. One of them is to choose an HTML editor like TinyMCE or FCKeditor, but this means a conversion from HTML to XML needs to be developed. A conversion always includes the risk of loss of information, and important information cannot be recorded because no such equivalent exists in HTML. As regular readers of <!ELEMENT may already know, HTML codes only describe the text on the screen, but not the legal meaning.

Word and OpenOffice are also frequently mentioned in this context, as nowadays they can also produce XML. The problem however is that the XML produced does not validate against XML Schema that was developed for the CVDR. In developing a conversion, it quickly becomes clear that the XML, in the same way as HTML, does not include enough meaningful information.

The choice for an XML editor like XMLmind seems obvious. The Java-basis enables browser integration and with the right configuration most of the buttons in the interface that are so confusing for non-technical authors, can be switched off. However, validation remains a problem as these type of editors only validate the XML at the author’s request. This is not a problem as long as the document does not contain errors, if it does however, then the editor will indicate a validation error and expect the author to solve the problem. However, this cannot be expected of the CVDR target group.

Xopus solves this validation problem by switching off all options in the interface that would make the document non-valid. This technique is called prevalidation (as opposed to postvalidation as in XMLmind). Prevalidation helps to create a friendly environment in which the author cannot make errors and does not encounter cryptic error messages. Additionally, Xopus is a browser-based application (Xopus is an HTML page with JavaScript) which supports the XML Schema 1.0, XSLT 1.0 and DOM Level 3 W3C standards. This means Xopus offers a good basis for the CVDR process interface.

Figure 1: Screenshot of the CVDR interfaceFigure 1: Screenshot of the CVDR interface
A web application has been chosen because it is much easier to manage than a desktop application, and in this project with over 400 potential users throughout the Netherlands, this is a huge benefit. On the other hand, the technical capabilities and speed of the browsers is very limited. This made it very difficult to implement the CVDR’s requirements, which are already difficult in combination.

A bit of WYSIWYM

User-friendliness is one of the key points of the CVDR project. A WYSIWYG (What You See Is What You Get : the document should be viewable in the same way during processing as after publication) interface was therefore a requirement. In the XML schema for decentralised regulations, text components with different meanings can end up having the same form. In this case WYSIWYG can cause confusion. The WYSIWYM interface was created as an alternative (What You See Is What You Mean : the interface shows the structure and the meaning of the XML and not the presentation). A WYSIWYM interface does require that the author clearly indicates the structure in the text and tries to imagine the end result. However, most people are visually oriented and do not have a clear idea of the text structure in their heads. That is why a fully WYSIWYM interface is not suited for the CVDR target group.

In cooperation with Eden Design & Communication, a solution was found in the form of a hybrid between WYSIWYG and WYSIWYM. The document is displayed virtually identically during processing as during the online publication, but the structure is shown in the left margin (see figure 2).

Figure 2: Screenshot of the left marginFigure 2: Screenshot of the left margin
The left margin also shows the features with which the text structure can be expanded. The toolbar is used to make adjustments to the text. In this way the interface provides a visual aid for starting users, and at the same time provides a simple way via which to adjust the document structure.

Restructuring

Regulations have been made for many years, and this process has been modernised many times. The current generation of regulations is almost completely stored in unstructured Word files, whereby templates are sometimes used. In order to make these legacy documents suited for the CVDR, Stipp developed a conversion that produces XML, which is valid according to the CVDR schema. However, in nearly all cases this conversion requires adjustments to be made by an editor. Sometimes these are only limited, but sometimes it is so difficult to deduce a consistent structure from the Word file, that it is simply better to just restructure text.

 A separate interface has been specially created for this, the restructuringinterface. This is because the standard process interface is mainly suited for processing new regulations or expanding correctly structured regulations.

In the restructuringinterface the author cannot change the text, only the structure. The processes that he can execute are limited to entirely removing the structure and marking paragraphs and bullet points as a structure element. If, for example, an author marks a paragraph as a chapter heading (see figure 3), then Xopus uses the XML Schema to convert the paragraph and any following text into a chapter structure and places it in the correct position in the document hierarchy. With the help of a clear interface and with just limited insight in the structure, the author is able to create a good structure for the text of the regulation.

Figure 3: Screenshot of the restructuring interface whereby a paragraph is marked as a chapter heading.Figure 3: Screenshot of the restructuring interface whereby a paragraph is marked as a chapter heading.
Expectations are that the restructuringinterface will no longer be used once all (legacy) regulations have been entered into the CVDR.

Change tracking

Local regulations are altered as a result of council decisions. It is important to be able to check that the changes in the council decision are processed correctly in the CVDR, and to be able to check who made the changes. This can be done by comparing two successive versions of the regulation with each other, but this approach creates a number of fundamental problems:

  • A comparison between two versions of a text does not precisely show what has changed. For example, suppose that the order of items in a list is changed. If an author has only placed the last of four items in the second place, the comparison software might think that the second item is new and that the last item has been deleted, or that the second through to the one-but-last item has been moved down.
  • A comparison of versions is unable to distinguish between changes made by different people or several successive changes. This can be overcome by saving many versions, however this can easily lead to administrative problems.
  • If versions in the XML source are compared with each other, a method is needed to present the result, preferably in a presentation that the author is able to recognise, for example by adapting an existing XSL stylesheet. This is often very difficult.

A trick to prevent this problem is to compare the presentation, e.g. the HTML presentation, but this means the side effects of the stylesheet are also compared, like sorting or showing the same information multiple times. Comparing information that is not explicitly shown, like metadata, is also problematic.

If only a few changes are made, then this does not cause problems. However, if large numbers of changes are made the result can easily become obscure or even misleading. The only way to be able to show the required information is to track the changes in the process interface while the author is making them. Xopus did not offer this option, so this generic expansion was developed with the CVDR application in mind.

Figure 4: Screenshot of a change and change tracking panel.Figure 4: Screenshot of a change and change tracking panel.
The presentation should also be taken into account the tracking of changes in the processing interface. The author sees XML changes via HTML that was produced with XSLT. That is why changes in Xopus are shown in a transparent layer that lies on top of the HTML, and that is adjusted after each change. To position the cursor, Xopus follows the position of all visible HTML nodes on the screen. Additionally, the XML node that corresponds with the HTML node is recorded. This information is used to determine the correct position on the screen for each change in the XML. This approach is reasonably processor intensive, but is totally XSLT-independent and it does not limit the HTML and CSS designer in any way.

The changes must also be saved, without changes being allowed to the XML Schema. During processing, the changes are tracked in a separate data structure, which is constantly synchronised with the XML document. When saving changes, the information can be saved in the document via a chosen method. Xopus currently supports the PTC Arbortext Editor and JustSystems XMetaL formats. The CVDR uses the last mentioned format.

The practice

The users’ first impressions are very positive, and in the meantime the first decentralised government bodies have started entering their regulations. Expectations are that in 2010 it will be obligatory for all local regulations to be published via the CVDR. For an example of regulations published with the CVDR, see http://www.overheid.nl/decentraleregelgeving/ and go to Capelle aan den IJssel.

Marin van Sprang is an ICT project manager at Stipp and Laurens van den Oever is the CEO of Xopus Company.)

  • Xopus Blog
  • › Decentralised Regulations and XML

Modified: February 12th 2009
By: Laurens van den Oever