Report climb-gui

Building an interface for and with Climb

Laurent Senta

Technical Report no1210, June 2012revision 2304

Climb is a generic image processing library. Still considered as a prototype, Climb already provides toolsthat simplify the building of image processing chains, such as morphers and the $ operator. In order toextend this aspect of Climb, a GUI has been developed. This GUI allows the construction of processingchains with no knowledge in programming while providing a visual and interactive feedback to the user.

This technical report describes the various components of this interface. Our approach uses the algo-rithms and tools defined in the library to construct the back-end logic and the interactive aspects of ourgraphical application. Since genericity is a key feature of Climb, we have been able to extend its scopebeyond image processing, and thus show that we are able to use it in different situations such as theconstruction of a GUI.

Climb est une bibliothèque de traitement d’image générique. Encore à l’état de prototype, Climb fournitdéjà un certain nombre d’outils, comme les morphers et l’opérateur $ qui permettent de simplifier la dé-finition de chaînes d’algorithmes. Afin de prolonger cet aspect, une interface graphique a été développée.Celle-ci permet de construire des chaînes de traitement d’image sans connaissance en programmation,tout en offrant un retour à la fois visuel et interactif.

Ce rapport technique décrit les différents composants de l’interface. Notre approche utilise les algo-rithmes et outils définis dans la bibliothèque pour construire la logique interne de l’application ainsique ses différents aspects interactif. La généricité étant l’une des caractéristiques principales de Climb,nous avons été capable d’étendre son champ d’application au-delà du traitement d’image et ainsi mon-trer qu’elle peut être utilisée dans des situations différentes telles que la construction d’une interface gra-phique.

Keywordsclimb, lisp, image processing, genericity, interface

Laboratoire de Recherche et Développement de l’Epita14-16, rue Voltaire – F-94276 Le Kremlin-Bicêtre cedex – France

Tél. +33 1 53 14 59 47 – Fax. +33 1 53 14 59 [email protected] – http://www.lrde.epita.fr/

[email protected]

http://www.lrde.epita.fr/

2

Copying this document

Copyright c© 2012 LRDE.Permission is granted to copy, distribute and/or modify this document under the terms of

the GNU Free Documentation License, Version 1.2 or any later version published by the FreeSoftware Foundation; with the Invariant Sections being just “Copying this document”, no Front-Cover Texts, and no Back-Cover Texts.

A copy of the license is provided in the file COPYING.DOC.

Contents

1 Introduction 4

2 Required extensions for the GUI 52.1 Image definition extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Image Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Graph images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 Generic graphical preview using morphers . . . . . . . . . . . . . . . . . . . . . . 132.2.1 Climb Morphers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2.2 Morpher Preview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Building a GUI using Climb functionalities 173.1 MVC design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.3 View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.4 Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4 Conclusion 25

5 Bibliography 26

Chapter 1

Introduction

Climb is a generic image processing library written in Common Lisp. It comes with a set ofbuilt-in algorithms such as erosion, dilation and other mathematical morphology operators,thresholding and image segmentation.

The idea behind genericity is to be able to write algorithms only once, independently from thedata types to which they will be applied. In the Image Processing domain, being fully genericmeans being independent from the image formats, pixels types, storage schemes (arrays, ma-trices, graphs) etc. Such level of genericity, however, may induce an important performancecost.

In order to reconcile genericity and performance, the LRDE has developed an Image Process-ing platform called Olena. Olena is written in C++ and uses its templating system abundantly.Olena is a ten years old project, well-proven both in terms of usability, performance and gener-icity [7].

In this context, the goal of Climb is to use this legacy for proposing another vision of the samedomain, only based on a dynamic language. Common Lisp provides the necessary flexibilityand extensibility to let us consider several alternate implementations of the model provided byOlena from the dynamic perspective.

Climb also provides several tools that are built to simplify the use of the library such as mor-phers, that ease dynamic transformations applied on images and the $-operator which allows auser to define complex processing chains using a dedicated syntax.

This technical report describes the graphical user interface (GUI) that gives user without pro-gramming knowledge the ability to use the library using a graphical representation of the pro-cessing chains. Our approach is based on the reuse of the tools provided by the library itself inorder to simplify the writing of such an application. By adapting Climb’s image concept, we areable to define data and computation tailored to our needs. Thus, we show that using Climb asa Generic Processing Library reveals that the library may be an interesting tool to solve commonsoftware engineering issues.

To do so, we start by defining two extensions to the Climb library, the support for graph-based images and a new morpher interacting with a graphical toolkit such as gtk. These twoextensions allow us to check that our model, already well-tested with 2 dimensional images,is able to handle new image structures and that functionalities orthogonal to image processingalgorithms can be built using this model. We then describe the GUI implementation and ourapproach using Climb tools to simplify the construction of such an application.

Chapter 2

Required extensions for the GUI

2.1 Image definition extension

The Climb GUI is a tool that allows a user to build a processing chain graphically. This pro-cessing chain can be expressed with a graph of interconnected algorithm calls that would easilyallows dynamic transformation of the structure, depending on user interactions. Since graphcan be expressed as images, we have chosen to implement them as a Climb type rather thanbuilding a specific tool developed for the GUI. By adding a new image structure into the li-brary, this approach allows us to check that we are able to easily define new images structures.

2.1.1 Image Definition

sites: The pixels of an image are usually represented aligned on a regular 2D grid. In this case,the term “pixel” can be interpreted in two ways, the position and the value, which we need toclearly separate. Besides, digital images are not necessarily represented on a 2D grid. Valuescan be placed on hexagonal grids, 3D grids, graphs etc.

In order to be sufficiently generic, we define an image as a function from a position, calleda site, to a value: img(site) = value. This definition allows us to provide a generic wayof transforming images by avoiding to specify the underlying type. An example is given infigure 2.1, showing how to compute the inverse value for any compatible image and site.

(setf (iref image site)(value-inverse (iref image site)))

Figure 2.1: Generic Site Access

This model allows us to avoid specifying different implementations of the same algorithmfor every combinations of sites and values. Instead, by providing a small, predetermined set ofhigh level operations we are able to define algorithms that can work on very different images.

2.1 Image definition extension 6

site-set In order to apply an operation on all the sites of an image, the image domain is definedby a site-set object representing the set of all the sites of the image. site-set are used asiterators with the three following functions:

• (site-set-next site-set-a) returns the next site of the set or nil if it has reachedthe end.

• (site-set-reset site-set-a) restores the initial state of the iterator

• (do-sites (site set) body) calls body for each site of the set

site-set can also be used to represent a neighborhood. Small sets, called windows, canbe centered on a site and return all the sites within the center neighborhood. Using the twofollowing functions we construct sets representing neighborhoods on a 2 dimensional image(4-connectivity, 8-connectivity, etc.), and center them on a site.

• (box2d size) constructs a 2 dimensional window

• (site-set-window site-set center) centers the site-set at a given position

algorithms figure 2.2 shows a generic implementation of the dilation algorithm using Climbfunctionality. Since there is no information about the underlying types, this algorithm may beapplied on any combination of compatible images and windows.

(defun dilation (image window);; for each site of the image domain(do-sites (site image)

;; center the window on the current site(let ((neighbors (site-set-window window site))

(value (iref image site)));; browse all the neighbors, pick the maximal value(do-sites (neighbor neighbors)

(setf value (value-max value (iref image neighbor))));; update the value at the current site(setf (iref image site) value))))

Figure 2.2: Generic dilation algorithm

In the current Climb implementation, a restricted set of types are defined:

sites: The point object that represents a spatial coordinate in any given dimension.

values: The rgb, grayscale and bool values, used to manipulate respectively, colored, levelof gray and binary images.

images: 2 dimensional images in common format (png, bmp, etc.) can be loaded and saved.

7 Required extensions for the GUI

2.1.2 Graph images

Description Graphs are a common image representation in Generic Image Processing, used tosimplify the representation of a segmented image for example (figure 2.3). From another pointof view, they also act as a generalization of images with regular grid. Even if the underlyingtypes are very different, algorithms implemented in a generic way on 2 dimensional imagesshould work on graphs without any modification (figure 2.4). Their direct usability in Climb isan indication of the library’s genericity level.

A

D C

B

Figure 2.3: Segmented image to graph

...

Figure 2.4: Common image types to graph

Definition A graph image is defined by a set of nodes (or vertices) and a set of edges.Since complex algorithms can use nodes as well as edges to hold values, they both are consid-ered as sites and therefore used to access values with the (iref image site) method. The


following interface, defined on graphs, allows us to provide various underlying implementa-tion forms, from the most naive ones to the most specialized or optimized ones:

• (graph-node-add image node) and (graph-node-remove image node)

• (graph-edge-add image edge) and (graph-edge-remove image edge)

• (graph-node-succ image node) and (graph-node-pred image node)

• (graph-edge-from image edge) and (graph-edge-to image edge)

Where image is a graph-based image, node and edge are the corresponding graph com-ponents. Starting from an empty graph we are then able to progressively construct a complexstructure without prior knowledge of the underlying implementation. Using this interface,generic algorithms and tools manipulating graphs can be built. As an example, figure 2.5 showsthe gdef macro that provides a syntax dedicated to graph construction.

(gdef graph; create two nodes with the labels :A and :B, link :A to :B(:A -> :B); create the nodes :B, :C and :D, add the corresponding links(:B -> :C -> :D); create :E and :F, add the link :D to :C, :C to :E and :C to :F(:D -> ((:C -> :E)

(:C -> :F))))

Figure 2.5: The gdef macro used for graph construction

Once the structure is built using tools dedicated to graph, algorithms should be able to use itas any other image type. In the following we describe how Climb’s interface, providing iterationand value manipulation on images, is generalized in order to allow this genericity.

Differences with previous images As described previously, algorithms implemented in Climbare meant to be generic, by using high level concepts exposed the same way between differentimage structures. The dilation algorithm, detailed in figure 2.2, is one example among others. Amajor concept in Image Processing is neighborhood browsing, represented by a site-set shiftedon the image with the site-set-window function (figure 2.6).

On images with a fixed grid, this set of sites can be defined independently of the image itself,algorithms are able to infer neighbors position on 2 dimensional image using the informationprovided by the current center and the windows. A common pattern when implementing al-gorithms is to give the user the ability to change the algorithm behavior by using different sets:4-connectivity, 8-connectivity or even more complex masks, heavily used in mathe-matical morphology[9].

On image with graph structure however, such window can’t be defined independently fromthe image topology. Neighbors are dependent of the center site considered and the processedimage, the same node having different set of neighbors in different images for example.

Seeing that the iteration process over a neighborhood defined on 2 dimensional image couldnot be applied to graphs, we considered two approaches to fix this incompatibility: modifythe way we browse neighborhoods and thus, rewrite current algorithms, or extend the Climbgeneric model in order to see if it is already powerful enough to handle new concepts. In the


3x3 window

site(4; 5)

(site+ site-i site(4; 5))

-1;-1 0;-1 1;-1

-1;0 0;0 1;0

-1;1 0;1 1;1

Figure 2.6: Shift using site− set− window

following, we describe such an extension, unbound-sites, that are a new kind of sites usedto represent abstract concepts on sites such as the ith neighbor.

Implementation unbound-site is a class inheriting from site. An unbound site representsa concept that must be bound to an image in order to produce a concrete coordinate. Theseconcepts, such as the ithneighbor, can be constructed and manipulated by algorithms withoutprior knowledge of the processed image.

By defining the centering behavior as an abstract concept, it become possible to use the samedefinition with 2-dimensional image and graph based image, making algorithms using thisconcept able to operate on both image types.

We start by defining the binding process, site-bind. This method binds a site to an imageand compute the concrete coordinate described by an unbound-site. In order to hide thisprocess from the algorithm implementation, the iref method, used to access values in images,is also generalized to operate on abstract sites.

Using the Common Lisp Object System, we are able to use generalized dispatch polymor-phism to easily add a new behavior to the irefmethod. Unlike most OOP languages, CommonLisp methods are defined outside the class and can be specialized on any parameters. Duringexecution, the language implementation picks the most specialized method matching the objecttypes.

This functionality allows us to define a generic irefmethod that handles abstract sites andcalls the binding method when it is needed while keeping unchanged already defined behaviorson concrete sites (figure 2.7).


(defgeneric site-bind (site image)(:documentation "Get the bound site when dereferencing SITE on IMAGE."))

;; iref generalization(defmethod iref (image (s site-unbound)) ; read

(iref image (site-bind s image)))

(defmethod (setf iref) (image (site site-unbound) value) ; write(setf (iref image (site-bind site image)) value))

Figure 2.7: The iref method for unbound sites

Using generic functions, we are also able to provide several implementations for the site-bindmethod, depending on the abstract site and concrete image considered. It becomes possible todefine the neighbor and the centering concepts as objects inheriting from unbound-site (fig-ure 2.8).

(defclass unbound-site (site)()(:documentation"A site that defines an abstract concept untilit is bound to an image."))

(defclass site-neighbor (unbound-site)((ith :initarg :ith :reader ith))(:documentation"An unbound site site that defines \"ith neighbors\"."))

(defclass site-centered (unbound-site)((site :initarg :site :reader site)(center :initarg :center :reader center))

(:documentation"An unbound site centered on another."))

Figure 2.8: the neighbor and centering abstract sites

We can now give a generalized implementation of the site-set-window function that willconstruct a site-centered around the original sites rather than calling the specific site+method.

The centering behavior is thus moved to the binding function. Different site-bind spe-cializations are provided. One able to bind a centered site on an image with a regular grid,using the site+ method. The second implementing an equivalent behavior when operating ongraph. Figure 2.9 summarizes the centering process.


graph image 2dneighbor ith neighbor (-1; -1), (-1; 0), etc.

center vertex from the graph domain coordinate from the image domaincentering pick the ith neighbor of the center shift the neighbors by the center coordinates

Figure 2.9: Centering process on Climb images

Since the binding process is implicitly called by Climb when its needed, concepts over sitescan be added and extended by providing the corresponding site-bind implementation.

Iteration on a graph neighborhood We now describe a way to implement the iteration pro-cess on all neighbors of a given vertex with the site-set-first-neighbors set and theextended do-sites macro.

The site-set-first-neighbors allows a piece of code to iterate on a neighborhood us-ing the interface described in 2.1.1. Since the site-set-next procedure must not depend onthe image currently processed, it simply produces a site-neighbor at each iteration, incre-menting the ith value of the unbound site produced each time (figure 2.10).

site-set

centering

binding

1st neighbor

1st neighbor of node AA

B

D

Cimage

B

Figure 2.10: site− set− first− neighbors centering and binding process

The stop condition is handled by the binding method that raises a site-binding-failcondition when the current site-set object has gone too far.

Finally, the do-sites macro, used to iterate over a given site-set, is modified to setupan environment able to handle these conditions. By capturing the binding failure caused bythe current site, we can stop the iteration when the abstract set is fully covered. When thesite-binding-fail condition is raised by the site used in the iteration, we stop the currentdo-sites iteration (figure 2.11).


(defun dilation (image window);; for each site of the image domain(do-sites (site image)

;; center the window on the current site(let ((neighbors (site-set-window window site))

(value (iref image site)));; browse all the neighbors, pick the maximal value(LET ((#:G993 (DOMAIN NEIGHBORS)))

(SITE-SET-RESET #:G993 NIL)(LOOP :FOR NEIGHBOR := (SITE-SET-NEXT #:G993)

:WHILE NEIGHBOR:DO (HANDLER-CASE

;; Call body on each site(PROGN (SETF VALUE (VALUE-MAX VALUE (IREF IMAGE NEIGHBOR))))

;; Capture the error if it’s controlled by the iteration(SITE-BINDING-FAIL (ERR)

(IF (EQL (SITE ERR) NEIGHBOR)NIL(ERROR ERR))))))

;; update the value at the current site(setf (iref image site) value))))

Figure 2.11: the dilation algorithm, do− sites expanded

Usage Using the implementation described previously, we are able to use images based ongraphs with the algorithms already available in the library such as the morphological algo-rithms (example shown in figure 2.12).

With this abstraction on neighbors, we are also able to construct sites, and their associatedset, that represents higher level concepts such as the leaves in a graph, or borders on images, aslong as these sites provide a binding method.


command input output

(dilation input(box2d-1 1))

(dilation input(site-set-first-neighbors))

Figure 2.12: Dilation algorithm on different images structures

2.2 Generic graphical preview using morphers

Before building the complete GUI, we first wanted to be able to display Climb images in agraphical toolkit such as GTK or WxWidgets. Since a Climb image can be composed of verydifferent structures and values, interactions with the toolkit have to be generic enough to ensurecompatibility.

2.2.1 Climb Morphers

Climb already provides the morpher concept. Introduced first in Olena[1] with SCOOP2[6],morphers are objects that change the way an image is seen by the outside world, by dynamicallymodifying the values and/or structures that the image exposes[3]. These transformations areinteresting during processing chains construction, by allowing the user to convert an imagevery easily in order to apply a specific algorithm.

For example, the grayscale morpher transforms an image with colored values to a black &white one (figure 2.13), and the mask morpher restricts the domain exposed by the morphedimage. (figure 2.14).

2.2 Generic graphical preview using morphers 14

Figure 2.13: Morpher Grayscale

Figure 2.14: Morpher Mask

2.2.2 Morpher Preview

By using morphers as observers[5], we define the preview morpher that catches modifica-tions applied on the morphed image in order to update a dynamic visualization of it. Thisobserver holds a small context that constructs a new windows in the user’s operating system.The associated rendering context is then used to display a real-time visualization on screen(figure 2.15).

Performance Execution speed is not our primary concern here, however it must be noted thatsince the previewed image can also be subject to other morphers, the value written on the mor-pher may be different from the final value exposed by the image, when an invert-morpheris applied for example. This means that we have to explicitly read the coordinate after writing,in order to preview the value finally exposed by the image, slowing down execution.

Usage This morpher can then be applied on an image at any point of the processing chainwith the (make-image-morpher-gtk image) function that displays the image with real-time updates. The figure 2.16 shows the images displayed in the graphical interface for the


GTK Context

Climb ImageMorpher Preview

Writing

Reading

Figure 2.15: Morpher Preview

given command sequence. The last one being captured before the end of the algorithm, showingthe real-time update in progress.

This tool is the first step towards a fully interactive GUI and allows us to show that Climbimages can be displayed in a common graphical toolkit such as GTK. Furthermore, previewsare also an interesting tools when fixing algorithms implementations and processing chainsby allowing the developer or image processing researcher to visualize image manipulation inreal-time.

2.2 Generic graphical preview using morphers 16

command real-time preview

; load the image(setf morphed

(image-load ’images/lena128gray.jpg’))

; apply the dilation algorithm on the morpher(dilation morphed (box2d-1 1))

; apply the erosion algorithm on the morpher(erosion morphed (box2d-1 1))

Figure 2.16: Preview morpher demonstration

Chapter 3

Building a GUI using Climbfunctionalities

Purpose Climb already provides the $-operator that allows a user with little knowledge inprogramming to build complex processing chains[4]. This operator defines a domain specificlanguage that implicitly applies common operations in such piece of code, like variables ma-nipulation, through a dedicated syntax (figure 3.1).

($ (image-load "images/lena.png");; each result is passed on to the next algorithm(to-gray)(otsu);; use the image in two parallel branches(// ((dilation (4-connectivity))

(image-save "dilation.png"))((erosion (4-connectivity))))

;; use the branch results using $<branch number>(diff $1 $2)(image-save "diff.png"))

Figure 3.1: $ operator

In order to extend this aspect of Climb, the Graphical User Interface is provided as a way toconstruct complex chain of algorithms with no knowledge in programming, based on a graph-ical and real-time visualization of the current chain. This interface is built with the aim ofsimplifying the creation and exploration of new operators.

Functionality Following the $-operator syntax, the GUI is a graphical application allowing auser to chain smalls blocks in order to construct complex algorithms.

In the GUI, these blocks are provided under the form of graphical boxes that can be movedand interconnected as shown in figure 3.2. Each of these blocks represents a Common Lispfunction and can be parametrized using the bar at the bottom of the interface. This parameterlist is dynamically updated depending on the box selected by the user. Following the usualreading direction, input parameters that have to be images are represented as small connectors

3.1 MVC design 18

load-image

algorithm1

algorithm2

algorithm3

Section1 algorithm1 algorithm2 algorithm3Section2 load-image save-image...

setting1

setting2

setting3

Figure 3.2: Interface Draft

on the left of each boxes. The right connectors represent the output. This construction allows usto provide an easy way to build and understand an operator by examining the algorithm graphfrom left to right.

Finally, each box contains a preview surface updated each time the algorithm is ran by theuser. This surface displays in real time the transformation process along the algorithm, provid-ing the user with a clear feedback.

These functionalities are provided using a model-view-controller architecture describedbelow.

3.1 MVC design

Model-View-Controller is a concept introduced in Smalltalk[8]. It is a design pattern that definesthree main components in an application[2]:

The Model That represents knowledge, encapsulates the data together with its processing

The Controller The link between the user and the system that transforms the model dependingon the user input

The View The visual representation of the model that generates outputs to the user (on-screendisplay)

This design pattern is especially interesting when building graphical interfaces since it pro-vides an explicit separation between the application intelligence, interactions from the user andthe system and the graphical presentation (figure 3.3). When building a GUI for a library suchas Climb, this design allows us to maintain independently the library itself in the model, thetoolkit-dependent graphical export in the view and the GUI behavior, kept in the controller.

Using the MVC pattern, we are also able to switch between graphical toolkit and other waysof interacting with the library very easily. This possibility may be used to explore other displaymodel and fields of applications (web, desktop, cli, etc.).

In the following section, we describe each part of the GUI and how Common Lisp and Climbfunctionality are used to simplify the interface implementation.

19 Building a GUI using Climb functionalities

Model

ViewController

dataquery

notifications

transformations

user interactions

view selection

Figure 3.3: MVC pattern

3.2 Model

As described previously, the GUI’s model encapsulates the data with its processing. Here, thedata is obviously the Processing Chain, composed by the set of operations built by the user andtheir links. By interpreting Processing Chains as graphs of algorithms connected by parametersdependencies, we are able to use the Climb genericity model to store our data. Since graph canholds data on node and edges, we can define new values, tailored to our needs, while usingClimb functionalities on graph and images. The model also holds the available algorithms inorder to expose them and their description to the rest of the application.

Available algorithm We start by representing a Climb algorithm with the algorithm classthat holds the minimal description for an algorithm:

name A readable version of the algorithm name

function The associated function

input the image(s) the algorithm works on

params The parameters controlling the algorithm’s behavior

default The default settings for non-image parameters

This class is used to store all available algorithms in the model so that we can dynamicallyadd and remove algorithms like the threshold morpher detailed in figure 3.4.

(make-algorithm "Threshold"’make-image-morph-threshold’(image (threshold "120")) ; input parameters and default values’(image) ; parameters identifier used as box input

Figure 3.4: Algorithm description

3.2 Model 20

Since an algorithm may be used several times with a different set of parameters, we define thealgorithm-call object that associate the general algorithm description with the parametersdefined by the user.

Chain of algorithms The chain of algorithm is constructed as a directed graph image. Sincegenericity is one of the main goals of the Climb library, this approach allows us to determine ifour image description model is generic enough to be used in different contexts.

In this context, each vertex holds an algorithm definition, described previously as the algorithm-callobject. The connection between two algorithms is defined as an edge between two vertices andadditionally holds the destination parameter.

We can redefine the chain of algorithm in figure 3.1, a simplified representation is given infigure 3.5.

image-load

file=”image.png”

value:

erosion

window=(4-connectivity)

value:

to-parameter: ‘image1

value:

Figure 3.5: Processing chain representation as a Climb graph image

Execution Once the processing chain is built, its execution is defined as a Climb algorithm ongraph that starts by finding the entry point of the chain, looking for boxes with no predeces-sor. Using these vertices as starting point for a breadth-first traversal, the execute function,figure 3.6, calls each algorithm and updates its sons with the resulting image. The controller isalso provided in order to raise notification of the updated images for the rest of the application.

(defun execute (graph initials controller)(do-sites (site (make-node-set-breadth graph initials))

:for call = (iref graph site) ; retrieve the node value:for results = (algo-call call controller) ; call the algorithm:do (algo-sons-update graph site results)))

Figure 3.6: Processing chain execution using Climb algorithm


Modification Using Climb functionalities we are also able to define integrity validation on theprocessing chain. A specific interface is used by the controller to check the correctness of a linkoperation between two boxes for example. This interface allows the controller to warn the userand cancel an operation when the view transmitted an incorrect request such as connecting twoalgorithms outputs.

3.3 View

In the Climb GUI, the view is the visual representation of the processing chain and the compo-nents of the interface. As shown in figure 3.2 the graphical interface is divided in three majorpart:

Algorithm selection On the right of the interface, allows the user to select an algorithm ex-posed by the model and add the corresponding operation to the chain.

Algorithm call setting At the bottom of the interface, used to set the parameters for a givenoperation.

Chain of algorithm Displays the current chain of operation as an interactive graph that can beedited by the user.

The first two parts being mainly composed of gtk representation of the model, they are notdescribed in this report. As we saw in the model description, the chain of algorithms is definedas a graph connecting different operations. The work of the view is here to provide a visual andinteractive visualization of this graph.

Event system using CLOS The view also provides an abstraction layer between the graphicaltoolkit used to communicate with the user and the rest of the application. This layer allowsus to keep the events specific to a given view implementation bound to this implementation.Events transmitted to other components of the application can be correlated from the toolkit,making it easy to replace the toolkit used or even explore very different visualizations.

This event system allows the user to build the processing chain with common paradigms ingraphical user interface like drag&drop. The following capture (figure 3.7) shows the tree viewin which the user can select algorithm that will be added when clicking on the processing chainsurface. He is also able to move a box around and connect its inputs or outputs to another box.

3.3 View 22

Figure 3.7: Processing chain edition

First, events received from the gtk toolkit are translated into interactions on the view, forexample a mouse-click signal with the given coordinates, is interpreted by the view as a clickon a box. After this translation, events are divided into three main categories with additionaldata as shown in figure 3.8.

(defgeneric on-click (view selected from x y)(:documentation "Handle the mouse click event for the given VIEW with

SELECTED, the object under the mouse,FROM, the previously selected object,X and Y the mouse position."))

(defgeneric on-release (view selected from x y)(:documentation "Handle the mouse release event."))

(defgeneric on-move (view selected from x y)(:documentation "Handle the mouse move event."))

Figure 3.8: Generic definition of mouse events

The behavior provided as reaction for an event depends especially on the type of the con-cerned objects. For example, when the user releases the mouse, different behaviors will betriggered if he was dragging a box or if he was creating a link. Using generic functions, theview provides such behavior definition using Common Lisp capabilities.

As we saw in 2.1.2, CLOS’ generalized dispatch polymorphism allows us to specialize ageneric function for any combination of parameters type. Figure 3.9 shows simplified def-initions for various events:


; click on a box, start the drag&drop(defmethod on-click (view (selected gtk-box-call) from x y)

(setf selected-box selected ; store the selected boxrelative-x (- x (x selected)) ; store the relative positionrelative-y (- y (y selected)))) ; to the mouse

; moving the mouse when holding a box(defmethod on-move (view (from gtk-box-call) x y)

(setf (x from) (- x relative-x) ; keep the relative distance(y from) (- y relative-y))) ; between the box and the mouse

; dragging a link from a gtk-io (a box input or output) to another(defmethod on-release (view (selected gtk-io) (from gtk-io) x y)

(link-add (controller view) ; connect the two selected io(id (mother from)) (name from)(id (mother selected)) (name selected)))

Figure 3.9: Different on− click specializations

This approach using native capabilities of the language allows us to dynamically define andredefine view behaviors in order to fix an error or provide new functionality during run-time,simplifying greatly the development process.

We also define a defminimal macro that allows us to write event reaction with the minimalset of used parameters. In figure 3.9, we may replace:(defmethod on-release (view (selected gtk-io) (from gtk-io) x y) ...)by the minimal definition:(defminimal on-release (view (selected gtk-io) (from gtk-io)) ...).In practice this simplification permits to extend an event reaction while keeping the alreadydefined ones functional.

Visual representation In order to keep the visual interpretation of the processing chain in-dependent from the model representation, the view defines its own graph holding the visualcharacteristics of the chain. In this graph, each vertex holds an identifier to the correspondingalgorithm call together with the box’s spatial coordinates and the drawing surface. Since modeltransformations are applied via events sent to the controller, described in the following section,the two graphs are kept synchronized and specific algorithms can be applied on each of them.

Figure 3.10 also shows the use of the morpher-resize object, defined as a Climb morpher,this tool automatically resize a 2 dimensional image to fit a given size. This shows once againthat the generic tools provided by the library may be used effectively for different purposes.

3.4 Controller 24

Figure 3.10: Visual display of the processing chains

3.4 Controller

The controller is the part of the GUI that translates events raised by the view into model trans-formations. Currently, the controller is limited to a small set of interactions between the viewand the model (figure 3.11).

(defun call-preview-output (controller algo-call results)"Triggered when the algorithm call ALGO-CALL produced the RESULTS values.")

(defun call-add (controller name)"Add a new algorithm call NAME.")

(defun call-del (controller algo-call) ...)

(defun call-parameter-update (controller algo-call parameter value)"Update the algorithm call ALGO-CALL with the given PARAMETER VALUE.")

(defun link-add (controller algo-from algo-to param-to)"Add a new link between the algorithm ALGO-FROM and ALGO-TO,PARAM-TO is the parameter id.")

(defun link-del (controller algo-from algo-to param-to)...)

Figure 3.11: Controller definition

Leaving the management of “spatial interactions”, between the mouse and the interactivesurface, to the view allows us to define the controller independently from the graphical toolkitused. With this design, we can build a tool able to handle interactive edition of processingchains without being tied to a given graphical toolkit (GTK, Qt, etc.) or even mean of interaction(graphical, CLI, etc.).

Since we are using Climb’s generic representation, the controller will be able to easily usefunctionalities added to the library. For example, adding graph save and load in Climb may beused in the Controller to save and load the current processing chain.

Chapter 4

Conclusion

In this report, we first showed two extensions to the Climb library. The first one is the imple-mentation of a new image structure, graphs, which led us to define the concept of unboundsites. This concept allows us to use graphs with morphological algorithms initially definedon 2D image. We hereby show that Climb is able to handle new image structures and conceptseasily.

Secondly, we have detailed ways of interacting with a graphical toolkit like gtk using theconcept of morphers and the morpher-preview object. These objects provide a real-time vi-sualization of images and are the first steps toward the Graphical User Interface mainly basedon Climb tools.

Finally, the GUI was described through its model-view-controller design. We showedhow each components of the GUI is based on the tools defined by the library. It must be notedthat even though the GUI is a part of the library, it only uses it externally, as a mere library,providing feedback on the library state while avoiding dangerous interdependency.

By adding new structures to the Climb model and using the tools defined in the library indifferent contexts, the approaches detailed in this report are intended to explore new use casesfor the library and can largely be improved.

Future work Future work based on this report will focus on the stabilization of the newly de-fined components in the library and the stabilization of the GUI implementation. Using Climbas a Generic Processing Library revealed that the library is an interesting tool to solve softwareengineering issues, such as the construction of a GUI. Thus, this approach may be used to findnew opportunities to use Climb in other contexts.

Chapter 5

Bibliography

[1] Ballas, N. (2008). Image taxonomy in Milena. Technical Report 0812, EPITA Research andDevelopment Laboratory (LRDE).

[2] Burbeck, S. (1987). Applications programming in smalltalk-80(tm): How to use model-view-controller (mvc).

[3] Chedeau, C. (2010). Functionnal approach of image processing genericity. Technical Report1001, EPITA Research and Development Laboratory (LRDE).

[4] Chedeau, C. (2011). Component trees and chaining operators in climb. Technical Report1108, EPITA Research and Development Laboratory (LRDE).

[5] Gamma, E., Helm, R., Johnson, R., and Vlissides, J. (1994). Design Patterns: Elements ofReusable Object-Oriented Software. Addison-Wesley Professional, 1 edition.

[6] Géraud, Th. and Levillain, R. (2008). Semantics-driven genericity: A sequel to the static C++object-oriented programming paradigm (SCOOP 2). In Proceedings of the 6th InternationalWorkshop on Multiparadigm Programming with Object-Oriented Languages (MPOOL), Paphos,Cyprus.

[7] Levillain, R., Géraud, Th., and Najman, L. (2010). Why and how to design a generic andefficient image processing framework: The case of the Milena library. In Proceedings of theIEEE International Conference on Image Processing (ICIP), pages 1941–1944, Hong Kong.

[8] Reenskaug, T. (1979). Models-views-controllers. Technical note, Xerox PARC, December.

[9] Soille, P. (2003). Morphological Image Analysis: Principles and Applications. Springer-VerlagNew York, Inc., Secaucus, NJ, USA, 2 edition.

List of Figures

2.1 Generic Site Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Generic dilation algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Segmented image to graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.4 Common image types to graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.5 The gdef macro used for graph construction . . . . . . . . . . . . . . . . . . . . . 82.6 Shift using site− set− window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.7 The iref method for unbound sites . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.8 the neighbor and centering abstract sites . . . . . . . . . . . . . . . . . . . . . . . . 102.9 Centering process on Climb images . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.10 site− set− first− neighbors centering and binding process . . . . . . . . . . . . 112.11 the dilation algorithm, do− sites expanded . . . . . . . . . . . . . . . . . . . . . . 122.12 Dilation algorithm on different images structures . . . . . . . . . . . . . . . . . . . 132.13 Morpher Grayscale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.14 Morpher Mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.15 Morpher Preview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.16 Preview morpher demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.1 $ operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2 Interface Draft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3 MVC pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.4 Algorithm description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.5 Processing chain representation as a Climb graph image . . . . . . . . . . . . . . 203.6 Processing chain execution using Climb algorithm . . . . . . . . . . . . . . . . . . 203.7 Processing chain edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.8 Generic definition of mouse events . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.9 Different on− click specializations . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.10 Visual display of the processing chains . . . . . . . . . . . . . . . . . . . . . . . . . 243.11 Controller definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Documents

Report climb-gui