Declarative Meta-programming

Johan Brichau (PROG/VUB)
Kim Mens (INGI/UCL)

Contents:


Introduction and Motivation

The software development process deals with many phases, such as: analysis, design, coding, maintenance, evolution, etc... Adopting the classic approach to software engineering (e.g., the waterfall model), all these phases are performed separately, where the results of one phase serve as a basis for the start of another phase (e.g., designs are used to start the implementation). But in most cases, phases that occur at a later time in the software development process will require to make changes to the results of earlier phases (feedback). For example, problems that only become apparant during the implementation phase, may require changes to the design. Or even worse: software programmers can deviate from the original design without notifying the designers or maybe even without knowing that they do so. But even if programmers are aware about the impact of their change to the design, updating the design may be a time-consuming process and therefore, they might not do it. All these factors lead to enormeous problems in the long run because the entire design documentation of a software artifact is no longer up-to-date with the actual implementation.

Our research on declarative meta-programming (DMP) investigates how this technique can be used to build state-of-the-art software development support tools that help overcome these problems. The aim is to try to capture and formally express the interaction between the higher phases of software development (design, analysis, etc...) and the actual implementation level. To express this interaction between the program code and the (higher-level) documentation of a software system, we believe that a declarative programming language is highly suited. This is because documentation has a declarative nature and because we can use meta-programming to link this information to the actual code. As such, the design-information of a system is meta-information of the program itself that can be actively used by the programming environment. The result is that we can automate the co-evolution of the different phases of software development. Besides actively guiding the co-evolution of phases such as design and implementation, we also conduct research on actively generating design from implementation and vice-versa. Or the use of high-level information to support higher-level software engineering tasks (such as refactoring, evolution, guiding reuse, aspect-oriented software development, generative programming, etc...)

Logic meta-programming is an instance of declarative meta-programming, where a logic programming language is used. Hence, we express all high-level information of a software artifact in logic facts and rules. As such, we use a logic meta-level language with a standard (object-oriented) base language. For this, we require that the logic meta-programming language lives in symbiosis with the object-oriented base language, allowing base level programs to be manipulated as terms, facts or rules in the meta level. This can be achieved in various ways. In the case of Java, which is a statically typed language, this meta layer might be implemented as a preprocessor or even an extension of the Java compiler itself. In the case of Smalltalk, we can use the reflection facilities to implement a logic meta layer on top of the Smalltalk meta-object protocol. A third approach is the use of a separate logic language and base language with provisions of the logic language to externally access the repository of the base language.

Here are some useful documents to read in order to learn more about declarative meta-programming in general:


What is DMP?

In this section, we try to provide a more precise definition of what exactly we mean by DMP. DMP stands for Declarative Meta Programming. So the short, one-line, definition of DMP would be:

"Declarative Meta Programming is the use of a Declarative Programming language at Meta level to reason about and manipulate programs built in some underlying base language."

Of course, this definition does not help us out much. First of all, it doesn't make clear what we mean by "declarative" and secondly it doesn't tell us what kind of reasoning at meta level we are interested in: what is the goal we want to use DMP for? So let us elaborate on this definition a bit more.

  1. To start with, we restrict the scope to
    1. those kinds of DMP that provide support for (building) tools that support or facilitate the software development process in one way or another;
    2. those tools that address source code.

  2. The "MP" part of DMP should be quite clear now, especially in the light of 1.1:
    1. it is meta programming because the tools are at a meta level with respect to the source code they want to reason about or manipulate
    2. what we can and want to do at this meta level can be almost anything, but belongs to one (or more) of the following categories (this categorization is not supposed to be restrictive, though we hope its more or less complete):
      • verification of source code to some higher-level description (for example, conformance checking to coding conventions, design models, architectural descriptions and so on)
      • extraction of information from source code (for example, visualisation, software understanding, browsing, generation of higher-level models or documentation, measurements, quality control and so on)
      • transformation of source code (for example, refactoring, translation, re-engineering, evolution, optimization and so on)
      • generation of source code

  3. The "D" of DMP is maybe the most difficult one to characterize, but let us make an attempt anyway. In general, what is understood by "declarative" programming is that it is focussed on what the program is supposed to do rather than on how it will achieve its tasks.
    Unfortunately, because of the large variety of possible DMP applications (even if we take the restrictions of 1.1 and 1.2 into account), this still does not clarify much. However, it does become a bit more clear when we refine the definition for each of the categories of 2.2 separately:
Okay, we admit, although this definition may have made some things a bit more clear, it is now much too large for a good working definition, so let us try to summarize it:
  1. RESTRICTION: We only use dmP for building tools that
    1. support or facilitate the software development or maintenance process
    2. deal with source code
  2. dMp = reasoning about and manipulation of source code in a meta language:
  3. Dmp = the meta language is declarative => focusses on WHAT the program does rather than on HOW it works:

Achievements

Declarative meta-programming is an emerging technique that is not quite out of the lab as yet. However, it has already shown to be very expressive and remarkably powerful. A number of succesful experiments with logic meta-programming have been carried out in the context of code generation, aspect-oriented programming, code optimalisation, conformance checking between software architectures and implementation, measuring software quality and codifying coding and design patterns. Logic meta-programming is also being used to determine how inherent domain knowledge in software applications can be described in a localised and declarative manner. Below we present a nonlimitative list of experiments that have been performed in the context of declarative meta-programming.


Ongoing Work


Artifacts

The main artefacts and tools that have been produced in the context of this common research project are:

SOUL

SOUL is an acronym for Smalltalk Open Unification Language. It was originally developed by Roel Wuyts in VisualWorks/Smalltalk in the context of his PhD thesis (see above). It features a hybrid logic language which is integrated with the object-oriented language Smalltalk. SOUL is now used for most of the declarative meta-programming experiments at PROG/VUB and INGI/UCL.

LiCoR

(2001) SOUL comes with a set of primitive logic predicates that can be found in most Prolog-like logic languages. In addition to these logic primitives, specific logic predicates are needed for reasoning about (Smalltalk) source code. Together with the re-implementation of SOUL, this Library of predicates for Code Reasoning was factored out and baptised LiCoR. LiCoR is now provided as a separate package on top of SOUL. This allows one to use SOUL without LiCoR as just an ordinary logic language, or to use SOUL with an alternative library for code reasoning.

(2002) NewLiCoR, a through reimplementation and restructuring of LiCoR is currently underway, but a public release is probably not to be expected anytime soon.

SOULJava

(2002) SOULJava is an add-on to SOUL which allows SOUL to reason about Java code. It is implemented by Johan Fabry.

C2C (deprecated)

(2000) C2C is an acronym for "Classes To Clauses". It is a tool that generates logic clauses from Java bytecode, to allow declarative reasoning about Java code. The tool was implemented by Philippe Demaecker. Though promising, this research has been discontinued.

TyRuBa

(1999) TyRuBa is a logic-meta-programming system which was designed to be used for generating Java code. It was implemented as part of Kris De Volder's Ph.D. work as a system to experiment with the idea of type-oriented logic meta programming. TyRuBa's name derives from this origin, it is an acronym for 'Type Rule Base'. However, the system has no specific built-in features which make it focus on types. Rather it is a general, simple logic programming language with a few peculiarities to facilitate the manipulation of Java code for the purpose of code generation.

Soul/Aop - SoulWeaver - Babel (name should still be determined)

(2002) This is an almost finished experimental tool, implemented by Johan Brichau, to build and compose aspect-specific languages: SOUL/Aop page

JQuery

(2002) JQuery, developed by Kris De Volder and Gregor Kiczales, is a UBC research effort which provides developers with an effective means to navigate and explore source code using a flexible, query-based browser model. JQuery enhances a developer's ability to work with crosscutting concerns in code. An essential component of the JQuery tool is the TyRuBa logic query language, which has been integrated closely with the Eclipse API's.

Intentional View Browser

(2002) Kim Mens' Intentional View Browser is a browser that facilitates the browsing and editing source code that is classified using intentional source-code views. Because 'intentional views' can be seen as an advanced kind of 'software classifications', the intentional view browser is not offered as a stand-alone tool, but rather as a plug-in for Roel Wuyts' Star Browser. The integration of the Intentional View Browser in the Star Browser is joint work by Kim Mens and Roel Wuyts.
Here is a high-resolution screenshot of the Star Browser and Intentional View browser in action.


Workshops

2002


Tutorials, Lectures and Presentations

2002

2001

2000

1999


Publications

An extensive (but not necessarily complete) list of papers related to DMP can be found in the list below. You can also search the publications of the PROG lab or the publications of the INGI department.

Journals

2002

Proceedings

2002

2001

2000

1999

1998

Workshops

2002

2001

2000

Tech reports


Dissertations

PhD Theses

Masters Theses

Graduation Theses


People

Here is a list of people that is currently actively involved with DMP:


DMP related research

And here is a non-exhaustive list of research topics that are related to DMP:


Page last modified on Wednesday, 15 January, 2003 16:24 by KM