1

Shortest Paths with Higher-Order RegularizationJohannes Ulen1 Petter Strandmark1 Fredrik Kahl1,2

1Lund University, Sweden 2Chalmers University of Technology, Sweden{ulen, fredrik, petter}@maths.lth.se

Abstract—This paper describes a new method of findingthin, elongated structures in images and volumes. We useshortest paths to minimize very general functionals of higher-order curve properties, such as curvature and torsion. Ourmethod uses line graphs to find the optimal path on a givendiscretization, often in the order of seconds on a singlecomputer. The curves are then refined using local optimizationmaking it possible to recover very smooth curves. We areable to place constraints on our curves such as maximumintegrated curvature, or a maximum curvature at any pointof the curve. To our knowledge, we are the first to performexperiments in three dimensions with curvature and torsionregularization. The largest graphs we process have over ahundred billion arcs. Experiments on medical images and inmulti-view reconstruction show the significance and practicalusefulness of higher order regularization.

Index Terms—Curve extraction, Optimization, Curvature,Torsion, Line graphs.

I. INTRODUCTION

In differential geometry, the fundamental theorem ofcurves states that any regular curve in three dimensionswith non-zero curvature has its shape completely deter-mined by its curvature and torsion [26]. Therefore, forcurve reconstruction problems, it makes sense to regularizehigher-order properties such as curvature and torsion. So,why is it that the majority of approaches in the literatureonly consider length regularization? The reason is of coursedue to computational complexity. In this paper, we demon-strate that with modern CPUs and careful implementationchoices, it is actually possible to solve inverse problems in-volving higher-order geometric regularization at reasonablerunning times.

Curvature regularization has been shown to be importantfor a number of applications in computer vision. In [22],it is shown that Euler’s elastica – the line integral of thesquared curvature – conforms better to intuitive completionof boundary curves. Other applications where curvatureplays an important role include saliency [31], inpaint-ing [18], stereo [40], region-based image segmentation [30]and surface reconstruction [34]. It has also been used asa basis for constructing connectivity priors for tree-shapebased segmentation [36].

Torsion measures how much a curve deviates from itsosculating plane. In many applications, the reconstructedcurve should follow closely the shape of an underlyingsurface which is locally planar. In such situations, it may beadvantageous to penalize torsion. Except for our previousconference paper [35] we are not aware of any work thathas previously used this as a prior. Fig. 1 shows a drawing

Right coronary artery

Left coronary artery

Fig. 1: A drawing of a heart. The thinner coronary vessels are longwith high curvature, but have low torsion as they lie approximatelyin a plane.

of a heart. The coronary arteries are neither short nor havelow curvature, but they do have low torsion. Penalizing hightorsion would be an ideal prior.

Our framework also supports subsidiary constraints onthe minimizing curve using so-called resource constraints.The shortest path problem with resource constraints [13]enforces global constraints on the solution. This formula-tion is employed in, for example, vehicle routing and crewscheduling problems [13]. The global constraint can be usedto limit the maximum length of the curve or the integratedcurvature.

A. Related work

We follow a classic approach for computing the globalminimum of an active contour model based on shortestpaths [7], [10]. This is a flexible and frequently used toolfor extracting 1D structures in both 2D and 3D images,especially in medical image analysis [16]. Most of thesemethods use only weighted length regularization, eventhough the idea of applying shortest paths to higher-orderfunctionals involving curvature is not new. One early exam-ple of minimizing a curvature-based energy can be found in[1]. However, in order to obtain reasonable running times,only a small band around the initial contour is considered,making the approach resemble a local refinement technique.Both [34] and [9] perform volume segmentation with thecurvature of the surface boundary as a regularization prior.However, many problem instances of the resulting non-submodular third-order pseudo-boolean function may beintractable to optimize as noted by [34]. In [29], theelastic ratio functional is proposed for edge-based 2D imagesegmentation. Using a line graph construction the ratio isable to penalize the curvature of the segmentation boundary

2

and finds the global optimum on a discrete mesh, a problemmuch harder than finding the shortest path. The runningtimes are up to several hours for medium-sized imageson the CPU. One advantage of this approach is that it ispossible to implement it in a parallel manner on a GPU,which drastically reduces execution times.

An extension of the live-wire framework with curvaturepriors is presented in [38]. The algorithm is applied toimage squares of up to 80× 80 pixels with 8-connectivity.In [25], the standard 2D shortest path is lifted to 4D byincorporating 12 discrete orientation angles and radii in theestimation, and hence implicitly penalizing curvature. Manynice examples of the benefits with curvature are given inboth [25] and [38], but no running times are reported. Incontrast, our method is run on significantly larger reso-lutions of both the image and orientations with tractablerunning times. In [15], a heuristic method is developedfor minimizing a pseudo-elastica for 2D segmentation. Thecomputational complexity of their algorithm is attractive,but at the price of approximate solutions. Our algorithmhas comparable running times and the same asymptoticcomplexity O(d2n log nd) where n is the number of pixelsand d depends on the neighborhood size. In [20] a methodis presented which is able to penalize an arbitrary functionof a curve, including curvature. In contrast to the line graphformulation used in this paper, the optimization scheme in[20] may fail to recover the optimal solution on a givendiscretization.

Line graphs have been used before to add “turning costs”to shortest path problems [5], [39]. Our paper continues thiswork by showing how to optimize curvature, going beyondrelatively small graphs, refining the solution with local op-timization, and showing applications to medical problems.In particular, [39] mentioned increased performance as anopen question, an issue we will study in detail in this paper.In [6] line graphs are used to find closed curves, with aninitial direction, with low geodesic curvature on triangu-lated surfaces. The paper reports attractive running times;however, many computer vision problems are substantiallylarger and require higher connectivity than used by thisapplication.

The resource constrained shortest path problem has beenoptimized using Lagrangian relaxations previously in theliterature [3], [19]. We show how this easily generalizesto our functionals and optimize the relaxation using thecutting-plane method. There is also previous work on con-straining local properties of a curve. Dubins paths are usedfor path-planning in robotics [27], [37] as they approximatethe mechanical limitations of robots, yielding physicallyfeasible trajectories. In our framework these types of locallimitations can be implemented in a straightforward man-ner.

In [17] a curvature dependent function is optimized usingLimited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS). The derivatives are calculated by hand and theauthors note that it was “tedious to implement and debug.”We also perform local optimization using L-BFGS but ourderivatives are determined using automatic differentiation,

`1`2

`3`4 ∫ `

0 D(γ(s))ds =∑4

i=1 `iD(i)

Fig. 2: The data term is interpolated using nearest neighbor.

which is simpler to implement and can be computationallymore efficient than manual differentiation [4].

B. Contributions

Our main contribution is a new global method for findingcurves with higher-order geometric regularization, such ascurvature and torsion. The running times of our frameworkare tractable even for 3D models using curvature regular-ization.

In addition, we show that within the same frameworkit is possible to incorporate other priors and variants. Thecurvature, or other properties of the curve, can be con-strained both locally disallowing sharp turns and globallyby restricting the integrated curvature cost over the fullcurve.

Finally, we show how to perform local optimization ona discretized curve, giving curves better fitting the under-lying continuous model. This enables us to find smoothcurves even though our initial search space may be heavilydiscretized.

The framework has been implemented in highly opti-mized C++ code and has been made publicly available tofacilitate further research.

C. Outline

The remainder of the paper is organized as follows. InSection II we define the general type of curve functionalswe are interested in. It is also shown how to specializethis functional to penalize length, curvature, and torsion.In Section III we describe how higher-order regularizationcan be penalized using line graphs, and how to optimizethe functional by restricting it to lie on a predefine mesh.In Section IV global constraints are incorporated and it isshown how to optimize the resulting functional. In SectionV we decouple the curve from the mesh and performlocal optimization. In Section VI the implementation detailsnecessary in order to perform efficient optimization forlarge data are given. In Section VII experiments exploringdifferent aspects of our method are presented. The effectsof different regularizations and the efficiency of our methodare analyzed.

II. PROBLEM FORMULATION

We are interested in finding a curve γ in two or threedimensions which minimizes a data-dependent functional

3

of the following type,

minimizeγ,L

∫ L

0

(f(γ(s)) + g(γ′(s))

+h(γ′′(s)) + `(γ′′′(s)))ds

subject to γ(0) ∈ Estart, γ(L) ∈ Eend,

(M)

where γ : [0, L] → R2 or 3 is parametrized by arc-length [26]. The functions f, g, h, and ` are all restrictedto be non-negative. The curve has to start in the set Estartand end in Eend.

The focus of our paper is

f(γ(s)) = V (γ(s)) (1)g(γ′(s)) = ρ (2)

h(γ′′(s)) = σκ(s)2 (3)

`(γ′′′(s)) = ντ(s)2, (4)

where V is a data-dependent function. Here κ(s) and τ(s)denote the curvature and the torsion, respectively. Thescalars ρ, σ and ν are weighting factors of length, curvatureand torsion, respectively.

If we like to constrain f at some point s to be at mostm we can enforce this replacing it with

f(γ(s)) =

{∞ f(γ(s)) > m

f(γ(s)) otherwise.(5)

Any other term in (M) can be constrained in a similarmanner. In Section IV we show how the integral of any fcan be constrained.

In this paper we optimize (M) by considering piecewiselinear curves γ. In order to approximate the k:th derivativeof γ we use k+1 points. In the remainder of this section weshow how we approximate all costs discussed as a functionof either 2, 3, or 4 points.

A. Data-dependent term, two points

We approximate the integral of V (γ(s)) by nearestneighbor interpolation,

∫ `

0

V (γ(s))ds ≈n∑i=1

`iV (i), (6)

where `i is the curve segment length for which voxel i isclosest to γ(s), see Fig. 2 for an example.

B. Curvature, three points

Every line segment has zero curvature, so we will needthree points to approximate curvature. Let (xi, yi, zi) fori = 1, 2, 3 be three ordered points. The quadratic B-splineassociated with these points is

γ(t) =1

2

x1 y1 z1x2 y2 z2x3 y3 z3

T 1 −2 1−2 2 0

1 1 0

t2t1

. (7)

−10

1−1

0

10

0.5

1

xy

z

Fig. 3: Example curve (x, y, z) = (cos 6πt, sin 6πt, t).

The curvature is defined as κ(t) = ‖γ′(t)×γ′′(t)‖‖γ′(t)‖3 [26].

It is straightforward to analytically compute the squaredcurvature as well as the length element ds = ‖γ′(t)‖dt ofthe spline using symbolic mathematics software.

The integral∫ 1

0κ(t)2ds(t) has no closed form, but can

be approximated with numerical integration. We use thetrapezoidal rule for this purpose. We will cache all thesevalues so the execution time of the numerical scheme is oflittle importance.

C. Torsion, four points

Every curve contained in a plane has zero torsion and anygiven triplet of points always lies in a plane. This meanswe need at least four points, (xi, yi, zi), i = 1 . . . 4, toapproximate torsion. Torsion uses derivatives of the thirdorder, which require cubic B-splines. The cubic B-splineassociated with these points is

γ(t) =1

6

x1 y1 z1x2 y2 z2x3 y3 z3x4 y4 z4

T −1 3 −3 1

3 −6 3 0−3 0 3 0

1 4 1 0

t3

t2

t1

.(8)

Similar to curvature, the torsion [26]

τ(t) = det[γ′(t) γ′′(t) γ′′′(t)

]/ ‖γ′(t)×γ′′(t)‖2, (9)

of the B-spline can also be analytically calculated.Fig. 3 shows an example curve (a helix), for which

the exact curvature and torsion are well known: they areboth constant. Fig. 4 shows the error when approximatingthe curvature and torsion using three and four points,respectively.

III. THE SHORTEST PATH IN LINE GRAPHS

In this section we will show how optimize (M) byrestricting γ to lie on a predefined mesh represented bya graph G. We will assume that the reader is familiar withbasic graph theory, see e.g [24]. Given a data volume Veach voxel is represented by a node in G. Edges connectingthe nodes in G represent possible line-segments for γ, seeFig. 6. Given a start and end set we can then find theshortest path very efficiently using Dijkstra’s algorithm [8].

Definition 1: We refer to the number edges pointing to anode as the connectivity of that node. If all nodes have the

4

10−4 10−3 10−2 10−1 100 10110−8

10−6

10−4

10−2

100

102

Distance between points

Rel

ativ

eap

roxi

mat

ion

erro

r

|τ ||κ|

Fig. 4: Error when computing the curvature and torsion using themiddle point of a spline fit to three or four points on the curve inFig. 3.

1

2 3

4

(a) Graph, G.

1

2 3

4

1,2

2,3

3,4

1,4

1,3

(b) L(G) of the graph in (a).

Fig. 5: Example construction of a line graph.

same connectivity we will refer to that as the connectivityof the problem instance.For example, adding all edges for a 2D problem withdistance less than or equal to 2.5 gives a connectivity of 16.

This construction connects two nodes and makes itpossible to penalize functions of at most two points. Inorder to penalize higher-order regularization like curvatureand torsion another graph construction is needed.

Definition 2: For a graph G the line graph, L(G), isgiven by a graph where each edge in G is a node in L(G),and each pair of edges connected by a node in G is an edgein L(G). See Fig. 5 for an example construction.

In the rest of this section we describe the simplest graphconstruction able to penalize the functional (M).

1) h = ` = 0, two points: In this case all cost functionsare functions of two points and the original graph suffices,see Fig. 6a.

2) h 6= 0, ` = 0, three points: In this case h requiresat least three points to be calculated. We form L(G) ofthe original graph. The start set consists of all nodes inL(G) whose corresponding edge e = (u, v) in G connectsa node in Estart, and equivalently for the end set. Each edgein L(G) connects three points in our original graph G, seeFig. 6b.

(a) Using nodes (b) Using edges (c) Using edge pairs

Fig. 6: The same path through a graph (edges shown in black)represented in three different ways. The graph nodes (shown incolor) correspond to (a) nodes, (b) edges, and (c) edge pairs inthe graph.

3) ` 6= 0, four points: In this case ` requires at least fourpoints to be calculated. We repeat the graph constructionfor the previous case and form L(L(G)). Now each edgeconnects four points in the original graph G, see Fig. 6c.

4) k points: It is possible to penalize any function of kpoints by nesting the line graph construction k − 2 times.The main issue with penalizing higher-order functions isthat the size of the nested line graph grows very rapidly.

Suppose we would like to penalize a function of k pointsand thereby for instance approximate the k−1:th derivativeof γ. Given a problem with a total of n points and aconnectivity of c, the corresponding line graph would haveO(nck−2) nodes and O(nck−1) edges.

IV. THE RESOURCE CONSTRAINED SHORTEST PATH

It is possible to extend (M) by introducing hard con-straints such as maximum length, curvature or torsion ofthe curve. This problem is known as resource constrainedshortest path, and we solve this using Lagrangian relaxation[3].

Let

minimizeγ,L

∫ L

0

(f(γ(s)) + g(γ′(s))

+ h(γ′′(s)) + `(γ′′′(s))))ds

subject to γ(0) ∈ Estart, γ(L) ∈ Eend∫ L

0

g (γ′(s)) ds ≤ u1∫ L

0

h (γ′′(s)) ds ≤ u2∫ L

0

` (γ′′′(s)) ds ≤ u3.

(P)

We will refer to this as the primal problem. Furthermore

5

define the Lagrangian dual as

L(λ1, λ2, λ3) =

minimizeγ,L

∫ L

0

(f(γ(s)) + g(γ′(s))

+h(γ′′(s)) + `(γ′′′(s))))ds

+λ1

(∫ L

0

g(γ′(s))ds− u1

)(L)

+λ2

(∫ L

0

h(γ′′(s))ds− u2

)

+λ3

(∫ L

0

`(γ′′′(s))ds− u3

)subject to γ(0) ∈ Estart, γ(L) ∈ Eend,

and the Lagrangian dual problem as

maximizeλ1,λ2,λ3

L(λ1, λ2, λ3) (D)

subject to λi ≥ 0 ∀i.

For a feasible solution to the dual problem we have

L(λ1, λ2, λ3) ≤∫ L

0

(f(γ(s)) + g(γ′() (10)

+ h(γ′′(s)) + `(γ′′′(s)))ds.

This result is known as weak duality. Using this result wecan instead of minimizing the primal problem maximize thedual problem. The difference between the primal and dualfunction value is known as the duality gap and is an upperbound on how far away we are from the global optima.L is a concave function and each evaluation of L gives a

linear constraint for the maximum of (D). By successivelyevaluating the maximum (λ1, λ2, λ3) given by these linearconstraints we get an algorithm for solving (D) known asthe cutting-plane method [2].L can be efficiently evaluated by updating functions f ,

g, and h and running shortest path on this Lagrangianrelaxation.

V. LOCAL OPTIMIZATION

So far we only discussed finding an optimal solution on adiscrete mesh. Ideally we would like a larger search space.This can be achieved by first finding the optimal solution onthe discrete mesh and then performing local optimization,for example using gradient descent, on the discrete solution.The major concern with most local optimization methodsis that we need to find the derivatives of the function.Approximating them via finite differences can lead to issueswith numerical stability. Deriving them analytically by handcan be very cumbersome and is prone to human error.

In this paper we use automatic differentiation [4] to per-form limited-memory Broyden–Fletcher–Goldfarb–Shanno(L-BFGS) descent [23], [33]. This allows us to locallyoptimize our curve without calculating any derivativessymbolically or numerically. Since the problem is relatively

low-dimensional, local optimization typically only requiresa fraction of the time required to solve the shortest pathproblem using Dijkstra’s algorithm.

VI. OUR IMPLEMENTATION

Dijkstra’s algorithm for computing the shortest path iswell known. An important property is the fact that it doesnot require the graph to be stored.

Two things are required:1) The number of nodes along with a start set and an

end set.2) An oracle that given a node returns its neighbors and

edge weights.Using an oracle means that the weight does not have to beprecomputed for every edge in the graph. As Section IIshows, computing the edge weights can become quiteinvolved.

Creating the line graphs explicitly in memory is notneeded; they can be derived on the fly from the originalgraph. This observation is crucial for higher-order regu-larization as the number of edges in nested line graphsexplodes. Furthermore, assuming that each node has thesame type of neighborhood system, the graph itself doesnot have to be stored. This is true for all our experiments.

We use the same implementation of Dijkstra’s algorithmfor all experiments in the paper. It uses the C++ standardlibrary for all data structures.

The limiting factor memory-wise is the priority-queue;each entry requires 12 bytes and consists of an int indexand double weight.

We use a cache in order to avoid performing the samecomputations twice. For example, the curvature cost be-tween two edges is the same even if both edges are trans-lated by the same amount. Thus, the integrated curvatureneeds only be computed once for each configuration. Thecache has to be fast, since when working with torsionthere are millions of different configurations of two edgepairs. For general connectivity our implementation usesa std::map with the coordinates (mean subtracted) askeys. For a fixed connectivity all costs are precomputedand stored in a std::vector for more efficient lookup.

A. The A?-AlgorithmA common improvement to Dijkstra’s algorithm is A?,

which changes the node visitation order. It requires afunction l, which for each node returns a lower bound onthe distance to the end set. Any such l yields a correctalgorithm, and if l ≡ 0, then A? becomes the standardDijkstra’s algorithm.

In some cases, finding a good lower bound is easy. Forexample, when computing the path with shortest length,the distance “as the crow flies” (ignoring V ) can be usedas a lower bound, ignoring any obstacles in the graph. Forhigher order regularization, however, computing a usefullower bound is more difficult (it has to be done really fast tomake a difference). One option is to set h = ` = 0 and solve(M) for all nodes (this is fast), and use the result as a lowerbound. We will evaluate this heuristic in Section VII-A.

6

B. Parallelization

Computing the shortest path between two nodes effi-ciently in parallel is not a trivial task. Therefore Dijkstra’sis sequentially run and the neighbors of each node aresequentially computed. The oracle can compute the edgeweights for all neighbors in parallel on a multi-core CPU.Since these computations are a large portion of the totalcomputational cost (even with the above-mentioned cache),some parallelization is obtained.

VII. EXPERIMENTS

A. Synthetic Experiments

1) Torsion: In Fig. 7 an experiment indicating the dif-ference between a curve with curvature regularization andtorsion regularization is given. High torsion regularizationforces the curve to stay within a plane, but within theplane, the curve may have high length and curvature. Theline graph used for the torsion regularization had about280 billion arcs. The technique of not storing the line graphexplicitly is of course imperative for such a graph.

Curvature TorsionStart set End set

Fig. 7: Synthetic 3D experiment for torsion with volume-rendereddata cost. Darker regions correspond to lower cost. We run thesegmentation with either very high curvature or very high torsionregularization. The underlying graph has 91,000 nodes (35×130×20), 13,286,000 edges (146-connectivity) and 1,939,756,000 edgepairs. Runnings times are 0.1670 s for curvature and 6.8991 s fortorsion (harder problems will take longer time to solve).

2) River segmentation: This experiment highlights thedifference between length and curvature regularization. Wemanually sampled the river image in Fig. 8 at differentlocations and calculated the data cost for every pixel as theshortest (Euclidean) distance in L*a*b* space to any of thecolor samples of the river delta. Estart is the upper boundaryand Eend is the lower boundary of the image.

Fig. 8 presents the results. With curvature regularizationit is possible to find a long path which does not turn much.No amount of length regularization is able to find this pathin Fig. 8c. Local optimization is able to improve on theshortest path solution and is most effective when curvatureregularization is used. In Fig. 9 the improvements of localoptimization are highlighted, note that the solution givenafter local optimization is much smoother, resulting in muchlower curvature cost.

The running times are heavily dependent on the regu-larization as shown in Fig. 10. For length regularization

the running time increases with the regularization strength.In the extreme case ρ = ∞ all lines up to the Euclideanshortest path needs to be considered before the algorithmterminates. The running times for curvature penalty behavedifferently. For low regularization, a strong data togetherwith the A? lower bound will leave most of the graphunvisited when the shortest path is found. For σ =∞ onlythe straight lines are considered which also leaves most ofthe graph unvisited. The longest running time occurs withmedium strong regularization as most of the graph needsto be visited.

The running times are compared in Table I for Dijkstra’sand A?, and it is evident that large speed-ups are obtainedwith A?.

B. 2D Retinal Images

Automated segmentation of vessels in retinal images isan important problem in medical image analysis, which, forexample, can be used when screening for diabetic retinopa-thy. We have investigated whether curvature regularizationis useful for this task on a publicly available dataset [32].Fig. 11 shows the experiment. We iteratively computedshortest paths from a user-provided start point to any pointon the image boundary. A curve could start anywhere alongthe previously found curves, but not end close to a previousend point.

We performed experiments with different amounts oflength regularization (Figs. 11b–11e). No or low lengthregularization resulted in noisy, wiggling paths. This prob-lem expectedly disappeared for medium regularization, butsharp turns were still present in the solution (sharp turnsusually change direction in the vessel tree). When using toohigh regularization (Fig. 11e), the solution almost ignoresthe data term and prefers paths along the image boundary.

In contrast, curvature regularization is able to morecorrectly capture the vessel tree. We knew this would bethe case from a theoretical analysis; this experiment showsthat the segmentation is feasible in practice.

C. Multi-View Reconstruction of Space Curves

Curvature has previously been used to reconstruct spacecurves from multiple calibrated views of a static scene [14],but only using local optimization. Given start and endpoints, we are now able to reconstruct the curves optimally.Fig. 12 shows an experiment on the same data as [14].We reconstructed a tree iteratively in the same way as inFig. 11 and, as expected, length regularization introducessimilar artifacts. Integrating the image along the projected3D edge gives the edge cost for a single view. The dataterm we used for an edge is simply the maximum of thecost over all views.

D. 3D Coronary Artery Centerline Extraction

Finding the centerlines of coronary arteries is of highclinical importance [28], but is very time consuming whenperformed manually. We use a public data set Rotterdam

7

(a) Image. (b) Data cost.

(c) Only length regularization,ρ.

(d) Only curvature regulariza-tion, σ.

(e) (c) After local optimization. (f) (d) After local optimization.

(g) A close-up of (e). (h) A close-up of (f).

(i) Visit order using Dijkstra’s. (j) Visit order using A?.

ρ : 0.25 1 1000 0 0 0σ : 0 0 0 0.25 50 500

Execution time (seconds)

A? 0.37 0.42 0.47 0.82 6.55 8.00Local opt. 0.12 0.03 0.01 18.16 9.01 14.4

Local optimization decreased function value by

1.5% 1.1% 0% 14% 48% 13%

Fig. 8: Example segmentation highlighting the difference betweenlength and curvature segmentation. The start set Estart is the upperboundary of the image and the end set Eend the bottom. Theimages to the left only uses length regularization and the imagesto the right only curvature regularization. No amount of lengthregularization is able to find the long and smooth river path. Theunderlying graph has 220,443 nodes (373 × 591) and 3,509,756edges (16-connectivity). Figs. (i) and (j) show the order in whichthe nodes were visited for medium curvature with and withoutA?, from blue (early) to red (late).

σ = 0.25 σ = 50 σ = 500

886,966(3.45 s) 1,328,045(9.70 s) 916,061(9.14 s)196,553(0.86 s) 885,415(6.67 s) 740,683(7.54 s)

TABLE I: The number of nodes visited using Dijkstra’s (firstrow) and A? (second row) for the curvature experiments inFig. 8. Naturally, the heuristic works best for low curvatureregularization.

Data cost Curvature cost∫ L0D(γ(s))ds

∫ L0κ(s)2ds

Shortest path 68.71 65.66Local optimization 64.21 5.61

Fig. 9: Example showing the effect of refining the shortest pathsolution using local optimization. The image shows a close-up ofthe shortest path problem solved in Fig. 8, With regularizationsettings ρ = 0 and σ = 50.

10−510−410−310−210−1 100 101 102 103 104

0.2

0.3

0.4

0.5

ρ

Tim

e(s

econ

ds)

(a) Only length regularization.

10−4 10−2 100 102 1040

5

10

σ

Tim

e(s

econ

ds)

(b) Only curvature regularization.

Fig. 10: Execution time as a function of regularization strength forthe example in Fig. 8. For purely length regularization the runningtime is an increasing function with regularization strength. Forpurely curvature regularization the longest running times occurfor moderately strong regularization.

8

(a) Data term. (b) ρ = σ = 0. Mean time: 0.41 s. (c) Details of (b) and (f), respectively.

(d) ρ = 0.1, σ = 0. Mean time: 0.56 s. (e) ρ = 1, σ = 0. Mean time: 0.66 s. (f) ρ = 0, σ = 100. Mean time: 10.1 s.

Fig. 11: Segmenting a vessel tree with 8 leaves in a 2D retinal image from the drive [32] database. The green dots show the computedbest starting points for the new branches. The underlying mesh has 329,960 points (584×565) and 10,496,766 edges (32-connectivity).The arrows point at sharp turns where the segmented vessel changes direction in the vessel tree and at particularly noisy parts. Alllength regularizations (b-d) have various issues and curvature regularization (e) finds a reasonable tree.

(a) ρ = 10, σ = 0. (b) ρ = 100, σ = 0. (c) ρ = 0, σ = 10.(d) The reconstructedtree.

Fig. 12: Reconstructing a tree in 3D from 4 different views (only one shown). Using length regularization gives similar artifacts(sharp turns) as in Fig. 11. The underlying mesh has 125,000 points and 24,899,888 edges (218-connectivity). Execution times are4.8 minutes for length and 13.8 hours for curvature. Data from [14].

9

Coronary Artery Algorithm Evaluation Framework [28]consisting of 32 CT angiography scans to evaluate ourmethod. Frequency analysis of the CT volumes in [28] showthat the resolution is artificially high in the dataset [12].In the experiments the volumes are down sampled to halfthe original size in each dimension without any loss ofinformation.

Most of the methods which have submitted results to theRotterdam challenge use some sort of shortest path formu-lation in some part of their algorithm. Reported results inthe database vary between 70-98% overlap, with differentamounts of human interaction and model complexity.

For these experiments we use the data cost proposed byMetz et al. [21], which is based on Frangis’ vesselnessmeasure [11]. The vesselness measure is constructed byanalyzing the eigenvalues of the Hessian matrix in eachvoxel of the volume. As is usual for these measures thevolume is first smoothed with a Gaussian kernel withstandard deviation σ allowing the Hessian and Gaussiansmoothing to be combined into one filter. The eigenvaluesof the Hessian at voxel p are sorted as

|λ1(p)| ≤ |λ2(p)| ≤ |λ3(p)|. (11)

The vesselness is then defined as

V(p) =

{0 λ2(p) > 0 or λ3(p) > 0(1− eA

)eB(1− eC

)otherwise,

(12)

where

A = − 1

2α2

(λ2(p)

λ3(p)

)2

(13)

B = − 1

2β2

λ1(p)2

|λ2(p)λ3(p)|(14)

C = − 1

2c2(λ1(p)2 + λ2(p)2 + λ3(p)2

). (15)

The motivations for the different functions are as follows.A distinguishes between plate-like and line-like struc-

tures.B accounts for deviations from a blob-like structure.C captures that the norm of the eigenvalues should be

small in the background.First the voxel volume undergoes a soft thresholding

T (p) =1

2

(erf (b (I(p)− a1) + 1)

)× (16)(

1− 1

2(erf (b (I(p)− a2)) + 1)

). (17)

The voxel intensity I(p) is measured in Hounsfield andthe error function is defined as

erf (x) =2√π

∫ x

0

e−t2

dt. (18)

The parameters are later tuned to avoid bronchi andcalcium intensities. The data cost for voxel p is definedas

D(p) =1

V(p)T (p) + ε. (19)

α β c σ a1 a2 b ε

12

12

230 0.92mm −24 576 0.1 10−4

TABLE II: The parameters used in the experiments. The param-eters were chosen as in [21].

Table II lists the parameters used for the experiments.The start set Estart and end set Eend needs to be manually

chosen and is extracted from the ground truth data. In allexperiments the connectivity is 96. Each coronary arteryvolume has a resolution of about 200× 200× 150 voxels.

We compare two different regularization schemes:• LENGTH: vary ρ and keep σ = 0• CURVATURE: vary σ and keep ρ = 0.

Grid search gives an optimal value of ρ = 0.08 andσ = 0.05 for the two approaches. In Table III the resulton the training set of [28] is given. The running timesare attractive compared to other methods reported in [28].The error is measured as overlap, which is defined asthe percentage of the extracted curve which lies insidea ground truth artery given by both a centerline and aradius. Performing local optimization improves the resultsslightly and is fast compared to the shortest path runningtime. LENGTH and CURVATURE are best at about the samenumber of arteries whereas LENGTH has slightly highermean overlap. The resulting curves’ data cost, length, andcurvature are compared to the ground truth in Table IV.The curves recovered with CURVATURE have curve charac-teristics closer resembling to the ground truth compared toLENGTH.

To further investigate the advantages and disadvantagesof the different regularizations we took out the two arterieswhich performed worst for LENGTH and CURVATURE andcompared the results when we varied the regularization. InFig. 13 the coronary artery which yielded the best resultfor LENGTH is shown. High curvature regularization isable to find the best possible solution, but still, there isno regularization which gives a result close to the groundtruth due to a weak data cost. In Fig. 14 the coronary arterywhich yielded the best result for CURVATURE is shown. Thecenterline in this artery is very smooth which curvaturemodels very well.

Now suppose that we have some natural limitations onthe curve like maximum length or maximum curvature. Wecan incorporate this into our model and solve the resourceconstrained shortest path problem. In Fig. 15 an exampleis given showing the added benefits.

VIII. CONCLUSIONS

This paper has demonstrated the possibility of incorpo-rating higher-order regularization such as curvature and tor-sion to shortest path problems. The underlying frameworkhas been applied to several different applications and inseveral different settings showing the general applicability.

The fact that the discretized problem converges to theunderlying continuous problem follows from the fact that

10

Name Solver Overlap (%) Regularization Wins Execution time (s)

Mean±std Length (ρ) Curvature (σ) Mean±std

LENGTH Shortest path 91.90±15.10 0.08 0.00 12 15.62±8.04CURVATURE Shortest path 87.51±22.09 0.00 0.05 10 81.91±10.62

LENGTH Shortest path and local optimization 91.94±15.04 0.08 0.00 11 16.59±8.16CURVATURE Shortest path and local optimization 87.80±22.14 0.00 0.05 11 89.35±11.76

TABLE III: Quantitative results for coronary artery centerline extraction on the training set of [28]. The running time includes thetime it takes to calculate the data cost.

Name Solver Data cost Length Curvature∫ L

0D(γ(s))ds

∫ L

01ds

∫ L

0κ(s)2ds

Ground truth Human 8.05±10.38 130.08±47.67 2.59±1.46

LENGTH Shortest path 3.29±2.91 131.96±48.96 18.55±9.92CURVATURE Shortest path 3.23±2.78 140.90±61.96 9.50±5.51

LENGTH Shortest path and local optimization 2.87±2.33 128.57±47.84 359.58±422.61CURVATURE Shortest path and local optimization 2.89±2.47 137.89±60.72 3.76±3.54

TABLE IV: Curve characteristics for all arteries in the training set of [28] given as mean ± one standard deviation.

10−2 10−1 100 101 1020

50

100

ρ, (σ = 0)

Ove

rlap

%

Ground truth∫ L

0D(γ(s))ds 47.77 7.81 9.14 12.59∫ L

01ds 220.14 271.23 238.69 184.71∫ L

0κ(s)2ds 2.88 15.80 8.85 7.81

10−2 10−1 100 101 1020

50

100

σ, (ρ = 0)

Ove

rlap

%

Ground truth∫ L

0D(γ(s))ds 47.77 8.14 10.83 42.92∫ L

01ds 220.14 302.16 243.26 138.18∫ L

0κ(s)2ds 2.88 16.55 2.78 0.44

Fig. 13: The coronary artery with largest advantage for length regularization using ρ, σ from Table IV. To the left and right onlylength and curvature regularization is used respectively. The ground truth artery is shown as the black wireframe.

10−2 10−1 100 101 1020

50

100

ρ, (σ = 0)

Ove

rlap

%

Ground truth∫ L

0D(γ(s))ds 3.18 1.20 2.18 18.78∫ L

01ds 90.34 89.56 86.61 84.30∫ L

0κ(s)2ds 0.92 11.68 1.54 0.19

10−2 10−1 100 101 1020

50

100

σ, (ρ = 0)

Ove

rlap

%

Ground truth∫ L

0D(γ(s))ds 3.18 1.18 1.38 4.64∫ L

01ds 90.34 91.19 89.98 89.15∫ L

0κ(s)2ds 0.92 1.24 0.39 0.34

Fig. 14: The coronary artery with largest advantage for curvature regularization using ρ, σ from Table IV. To the left and right onlylength and curvature regularization is used respectively. The ground truth artery is shown as the black wireframe.

11

Ground

truth

Overlap 100% 35.24% 78.43% 99.23%

Length limit (u1) ∞ 120 120

Curvature limit (u2) ∞ ∞ 2∫ L

0D(γ(s))ds 11.78 5.17 6.16 9.69∫ L

01ds 111.55 123.83 118.49 111.17∫ L

0κ(s)2ds 1.37 15.08 12.34 1.81

Execution time (s) 18.10 288.75 657.00

Fig. 15: Failure case when using only length regularizationρ = 0.08. For the red curve we add more knowledge about thecurve and constrain the length to be less than 120, the algorithmterminated with duality gap 8.8 · 10−11. For the green curve weadd even more constraints and limit the curvature to be less than 2,the algorithm terminated with duality gap 1.9 · 10−9. The groundtruth artery is shown as the black wireframe.

quadratic and cubic splines approximate second and third-order derivatives arbitrarily well. We have also demon-strated convergence in practice, see Fig. 4. Further, by usinglocal optimization, we are able to improve on our initialsolution obtained from shortest path. The benefits of usinglocal optimization should be clear from Table III.

In Fig. 15 we show that constraining the maximumlength and integrated curvature can qualitatively improvethe results. These constraints are useful if the user knowsa priori that a vessel cannot be longer than x or wants tolimit the total amount of turning of a robot arm.

Although all of our methods run in polynomial time, us-ing torsion regularization cannot be considered practical aswe have only used it for small problems. On the other hand,using curvature is not that expensive for many problems andwe have demonstrated its usefulness for medical imagingproblems, both in 2D (Fig. 11) and in 3D (Fig. 14).

The quality of the solution depends a lot on theconnectivity of the mesh. Previous works have used 8-connectivity [38] and the highest we have seen is 16-connectivity [15] with approximate solutions. In contrast,we think that 16-connectivity is the bare minimum and wehave used 32-connectivity for our two-dimensional medicalexperiments.

The source code is available for download fromhttps://github.com/PetterS/curve extraction.

A. Acknowledgments

We gratefully acknowledge funding from the SwedishFoundation for Strategic Research (FFL), the European

Research Council (grant no. 209480) and the SwedishResearch Council (grant no. 2012-4215).

REFERENCES

[1] A. Amini, T. Weymouth, and R. Jain. Using dynamic programmingfor solving variational problems in vision. IEEE Transactions onPattern Analysis and Machine Intelligence, 12(9):855–867, 1990.

[2] M. Bazaraa, H. Sherali, and C. Shetty. Nonlinear Programming:Theory And algorithms. John Wiley & Sons, 2013.

[3] J. Beasley and N. Christofides. An algorithm for the resourceconstrained shortest path problem. Networks, 19(4):379–394, 1989.

[4] C. Bendtsen and O. Stauning. FADBAD, a Flexible C++ Package forAutomatic Differentiation. Department of Mathematical Modelling,Technical University of Denmark, 1996.

[5] T. Caldwell. On finding minimum routes in a network with turnpenalties. Communications of the ACM, 4(2):107–108, 1961.

[6] M. Campen and L. Kobbelt. Dual strip weaving: Interactive designof quad layouts using elastica strips. ACM Transactions on Graphics,33(6):183:1–183:10, 2014.

[7] L. Cohen and R. Kimmel. Global minimum for active contour mod-els: A minimal path approach. International Journal of ComputerVision, 24(1):57–78, 1997.

[8] E. Dijkstra. A note on two problems in connexion with graphs.Numerische mathematik, 1(1):269–271, 1959.

[9] N. El-Zehiry and L. Grady. Vessel segmentation using 3d elasticaregularization. In International Symposium on Biomedical Imaging,pages 1288–1291, 2012.

[10] M. Fischler, J. Tenenbaum, and H. Wolf. Detection of roads andlinear structures in low-resolution aerial imagery using a multisourceknowledge integration technique. Computer Graphics and ImageProcessing, 15(3):201–223, 1981.

[11] A. Frangi, W. Niessen, K. Vincken, and M. Viergever. Multi-scale vessel enhancement filtering. In International Conferenceon Medical Image Computing and Computer Assisted Intervention(MICCAI), 1998.

[12] O. Friman, C. Kuehnel, and H. Peitgen. Coronary centerlineextraction using multiple hypothesis tracking and minimal paths. InThe MIDAS Journal - Grand Challenge Coronary Artery Tracking(MICCAI Workshop), 2008.

[13] S. Irnich and G. Desaulniers. Shortest Path Problems with ResourceConstraints. Springer, 2005.

[14] F. Kahl and J. August. Multiview reconstruction of space curves. InInternational Conference on Computer Vision, 2003.

[15] M. Krueger, P. Delmas, and G. Gimel’farb. Robust and efficientobject segmentation using pseudo-elastica. Pattern RecognitionLetters, 34(8):833–845, 2013.

[16] D. Lesage, E. Angelini, I. Bloch, and G. Funka-Lea. A reviewof 3d vessel lumen segmentation techniques: models, features andextraction schemes. Medical Image Analysis, 13(6):819–845, 2009.

[17] J. MacCormick and A. Fitzgibbon. Curvature regularization forresolution-independent images. In International Conference onEnergy Minimization Methods in Computer Vision and PatternRecognition, pages 165–179. Springer, 2013.

[18] S. Masnou. Disocclusion: A variational approach using level lines.IEEE Transactions on Image Processing, 11(2):68–76, 2002.

[19] K. Mehlhorn and M. Ziegelmann. Resource constrained shortestpaths. In Algorithms-ESA 2000, pages 326–337. Springer, 2000.

[20] N. Merlet and J. Zerubia. Auxiliary functions in dynamic program-ming for smoothed road detection. In Lasers, Optics, and Visionfor Productivity in Manufacturing I, pages 204–212. InternationalSociety for Optics and Photonics, 1996.

[21] C. Metz, M. Schaap, T. Van Walsum, and W. Niessen. Two pointminimum cost path approach for CTA coronary centerline extraction.In The MIDAS Journal - Grand Challenge Coronary Artery Tracking(MICCAI Workshop), 2008.

[22] D. Mumford. Elastica and computer vision. In Algebraic Geometryand its Applications, pages 491–506. Springer, 1994.

[23] Jorge Nocedal and Stephen J Wright. Conjugate Gradient Methods.Springer, 2006.

[24] C. Papadimitriou and K. Steiglitz. Combinatorial Optimization:Algorithms and Complexity. Courier Dover Publications, 1998.

[25] M. Pechaud, R. Keriven, and G. Peyre. Extraction of tubularstructures over an orientation domain. In Conference on ComputerVision and Pattern Recognition, 2009.

12

[26] A. Pressley. Elementary Differential Geometry. Springer, secondedition edition, 2010.

[27] J. Reeds and L. Shepp. Optimal paths for a car that goes both for-wards and backwards. Pacific Journal of Mathematics, 145(2):367–393, 1990.

[28] M. Schaap, C. Metz, T. van Walsum, A. van der Giessen,A. Weustink, N. Mollet, C. Bauer, H. Bogunovic, C. Castro, X. Deng,et al. Standardized evaluation methodology and reference databasefor evaluating coronary artery centerline extraction algorithms. Med-ical Image Analysis, 13/5:701–714, 2009.

[29] T. Schoenemann, S. Masnou, and D. Cremers. The elastic ratio:Introducing curvature into ratio-based globally optimal image seg-mentation. IEEE Transactions on Image Processing, 20(9):2565–2581, 2011.

[30] T. Schoenemann, S. Masnou, and D. Cremers. A linear frame-work for region-based image segmentation and inpainting involvingcurvature penalization. International Journal of Computer Vision,99(1):53–68, 2012.

[31] A. Shashua and S. Ullman. Structural saliency: The detection ofglobally salient structures using a locally connected network. InInternational Conference on Computer Vision, 1988.

[32] J. Staal, M. Abramoff, M. Niemeijer, M. Viergever, and B. vanGinneken. Ridge based vessel segmentation in color images ofthe retina. IEEE Transactions on Medical Imaging, 23(4):501–509,2004.

[33] P. Strandmark. A Library for Unconstrained Minimization of SmoothFunctions using Newton’s Method Or L-BFGS. https://github.com/PetterS/spii, 2013.

[34] P. Strandmark and F. Kahl. Curvature regularization for curvesand surfaces in a global optimization framework. In InternationalConference on Energy Minimization Methods in Computer Visionand Pattern Recognition, 2011.

[35] P. Strandmark, J. Ulen, F. Kahl, and L. Grady. Shortest paths withcurvature and torsion. In International Conference on ComputerVision, 2013.

[36] J. Stuhmer, P. Schroder, and D. Cremers. Tree shape priors withconnectivity constraints using convex relaxation on general graphs.In International Conference on Computer Vision, pages 2336–2343,2013.

[37] M. Vendittelli, J. Laumond, and C. Nissoux. Obstacle distance forcar-like robots. Robotics and Automation, IEEE Transactions on,15(4):678–691, 1999.

[38] H. Wang. G-wire: A livewire segmentation algorithm based ona generalized graph formulation. Pattern Recognition Letters,26(13):2042–2051, 2005.

[39] S. Winter. Modeling costs of turns in route planning. GeoInformat-ica, 6(4):345–361, 2002.

[40] O. Woodford, P. Torr, I. Reid, and A. Fitzgibbon. Global stereoreconstruction under second-order smoothness priors. IEEE Trans-actions on Pattern Analysis and Machine Intelligence, 31(12):2115–2128, 2009.

Johannes Ulen received his MSc degree inengineering physics in 2010 and his PhD inmathematics in 2014, both from Lund University.

Petter Strandmark received his MSc degree inengineering mathematics in 2008 and his PhD inmathematics in 2013, both from Lund University.After graduating, he worked as an optimizationexpert for Boeing, solving scheduling problemsfor airlines. He is currently working for Googlein Stockholm.

Fredrik Kahl received his MSc degree in com-puter science and technology in 1995 and hisPhD in mathematics in 2001. His thesis wasawarded the Best Nordic Thesis Award in patternrecognition and image analysis 2001-2002 atthe Scandinavian Conference on Image Analysis2003. In 2005, he received the ICCV Marr Prizeand in 2007, he was awarded an ERC StartingGrant from the European Research Council. Heis currently a joint Professor at Chalmers Uni-versity of Technology and Lund University.

13

APPENDIX

A. Calculating CurvatureCurvature uses second order derivatives, which requires

a quadratic B-spline to be approximated. Let (xi, yi, zi) fori = 1, 2, 3 be three ordered points. The quadratic B-splineassociated with these points is

γ(t) =1

2

x1 y1 z1x2 y2 z2x3 y3 z3

T 1 −2 1−2 2 0

1 1 0

t2t1

. (20)

The curvature is defined as κ(t) = ‖γ′(t)×γ′′(t)‖‖γ′(t)‖3 [26].

κ(t)2 =

(c∆y − b∆z)2

+ (c∆x− a∆z)2

+ (b∆x− a∆y)2(

(a t−∆x)2

+ (b t−∆y)2

+ (c t−∆z)2)3 ,

(21)where

a = x1 − 2x2 + x3,

b = y1 − 2y2 + y3,

c = z1 − 2z2 + z3,

∆x = x1 − x2,∆y = y1 − y2,∆z = z1 − z2.

(22)

With the same notation, the length element ds = ‖γ′(t)‖dtis equal to

ds =

√(a t−∆x)

2+ (b t−∆y)

2+ (c t−∆z)

2. (23)

B. Calculating TorsionTorsion uses third order derivatives, which requires a

cubic B-spline to be approximated. Let (xi, yi, zi) fori = 1, 2, 3, 4 be four ordered points. The cubic B-splineassociated with these points is

γ(t) =1

6

t3

t2

t1

T −1 3 −3 1

3 −6 3 0−3 0 3 0

1 4 1 0

x1 y1 z1x2 y2 z2x3 y3 z3x4 y4 z4

.(24)

The torsion [26]

τ(t) = det[γ′(t)T γ′′(t)T γ′′′(t)T

]/ ‖γ′(t)× γ′′(t)‖2,

(25)of the B-spline can also be expressed in closed form; let

N = y3

(x2z4 − x2z1 + x4(z1 − z2)

)− y4

(x2z3 − x2z1 + x1(z2 − z3)

)− x3

(y1(z2 − z4) + y4(z1 − z2)

)− y2

(x1(z3 − z4)− x3(z1 − z4) + x4(z1 − z3)

)+ y1(x2z3 − x2z4) + x1y3(z2 − z4) + x4y1(z2 − z3)

(26)

and

D = (s5s3−s2s6)2+(s4s3−s1s6)2+(s2s4−s1s5)2, (27)where

s1 =1

2(z1 − 3z2 + 3z3 − z4)t2

+ (2z2 − z1 − z3)t+z12− z3

2,

s2 =1

2(y1 − 3y2 + 3y3 − y4)t2

+ (2y2 − y1 − y3)t+y12− y3

2,

s3 =1

2(x1 − 3x2 + 3x3 − x4)t2

+ (2x2 − x1 − x3)t+x12− x3

2,

s4 = z1 − 2z2 + z3 − t(z1 − 3z2 + 3z3 − z4),

s5 = y1 − 2y2 + y3 − t(y1 − 3y2 + 3y3 − y4),

s6 = x1 − 2x2 + x3 − t(x1 − 3x2 + 3x3 − x4).

(28)

Then τ = N/D and ds =√s21 + s22 + s23.