Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
The Pennsylvania State University
The Graduate School
College of Engineering
INTERACTIVE VIRTUAL ENVIROMENT FOR DESIGN AND
MANIPULATION OF TOOLPATHS FOR USE WITH 3 DEGREE OF FREEDOM AND
5DOF BIOPRINTING PLATFORMS
A Thesis in
Engineering Science and Mechanics
by
Adomas Povilianskas
©2019 Adomas Povilianskas
Submitted in Partial Fulfillment
of the Requirements
for the Degree of
Master of Science
May 2019
ii
The thesis of Adomas Povilianskas was reviewed and approved* by the following:
Ibrahim T. Ozbolat
Associate Professor of Engineering Science and Mechanics
Thesis Advisor
Elzbieta Sikora
Associate Teaching Professor of Engineering Science and Mechanics, Graduate Programs
Coordinator
Corina Drapaca
Associate Professor of Engineering Science and Mechanics, Graduate Programs Officer
Judith A. Todd
Professor of Engineering Science and Mechanics
Department Head of Engineering Science and Mechanics
Chair of Committee
*Signatures are on file in the Graduate School
iii
ABSTRACT
Designer software interface simplifies toolpath manipulation tasks by abstraction of
complicated calculations through an interactive 3D virtual environment. Implementation of
complete bioprinting toolchain enables researchers without technical background such as doctors
and biologists to produce bioconstructions rapidly using 3 Degrees of Freedom (DOF) and 5
DOF bioprinting platforms.
iv
TABLE OF CONTENTS
List of Figures .............................................................................................................. v
List of Tables ............................................................................................................... vi
Chapter 1 Introduction ................................................................................................. 1
Development of 5-axis motion stage development .............................................. 3
Development of motion control software framework .......................................... 4 Motion control software description ..................................................................... 6
Chapter 2 Current Toolpath Generation Software Used in Bioprinting ...................... 7
Conventional 3D printing toolpath generation ..................................................... 7
2D bioprinting cell culture specific toolpath generation ...................................... 8 3D bioprinting toolpath generation ....................................................................... 9
Chapter 3 Development Requirements of Interactive 3D User Interfaces .................. 10
A-Code implementation and lessons learned ....................................................... 10 Design requirements of a virtual environment for toolpath planning ................... 11
Chapter 4 Toolpath Editor Development ..................................................................... 14
Toolpath editor framework development ............................................................. 15
Toolpath waypoints and paths .............................................................................. 16 Toolpath manipulation .......................................................................................... 16
Visual toolpath representation .............................................................................. 20 Toolpath generation of common geometry ........................................................... 23 3D scanning .......................................................................................................... 26
Chapter 5 Advanced Features ...................................................................................... 28
Toolpath generation within convex surface geometry .......................................... 28 Toolpath generation on free-form surfaces ........................................................... 31
Chapter 6 Toolpath Designer Application Examples .................................................. 37
Designs based on imported geometrical data ....................................................... 37
Toolpath generation based on 3D scan data of live specimen .............................. 38
Chapter 7 Conclusions ................................................................................................. 39
BIBLIOGRAPHY................................................................................................. 40
v
LIST OF FIGURES
Figure 1.1 Bioprinting Toolchain Implementation. ..................................................... 2
Figure 1.2 5DOF bioprinter CAD model and design. .................................................. 3
Figure 1.3 Motion control software framework implementation. ................................ 4
Figure 1.4 Motion control software user interface ....................................................... 5
Figure 2.1 PetriPrinter g-code editing interface. .......................................................... 8
Figure 3.1 A-code interface software (A) and algorithm implementation (B). ........... 10
Figure 4.1 Toolpath Editor, (A) editor UI, (B) final render of resulting structure. ..... 14
Figure 4.2 Example G-code output .............................................................................. 15
Figure 4.3 Waypoint and Path translation using graphical user interface. .................. 17
Figure 4.4 Properties and configuration of designer workspace.................................. 20
Figure 4.5 Parametrization strategies of Catmull-Rom curves. ................................... 21
Figure 4.6 Barry and Goldman’s cubic Catmull-Rom curve formulation. .................. 22
Figure 4.7 Waypoint Generators .................................................................................. 23
Figure 4.8 Example procedure for 3D scanning of a 5mm defect during surgery....... 27
Figure 5.1 Toolpath generated based on 3D scan obtained data.................................. 28
Figure 5.2 2nd degree 3x6 grid NURBS surface formulation23. ................................... 31
Figure 5.3 Knot vector generation algorithm. ............................................................. 33
Figure 5.4 Example of 10x10 (A) and 3x10 (B) NURB surfaces of degree 2. ............ 33
Figure 5.5 NURBS surface positioned tangent to the surface of an imported mesh. .. 34
Figure 5.6 NURBS surfaces of degree 2, 3 and 4. ....................................................... 35
Figure 5.7 Path generation using NURB surfaces. ...................................................... 36
Figure 6.1 Design of semilunar heart valve based on imported STL geometry. ......... 37
Figure 6.2 Testing convex toolpath generator for bioprinting on live specimen. ........ 38
vi
LIST OF TABLES
Table 3.1 Variables required for one waypoint using 3DOF and 5DOF bioprinters. .. 12
Table 3.2 Virtual environment interaction requirements. ............................................ 12
Table 3.3 Virtual environment implementation aspects. ............................................. 13
Table 4.1 Waypoint and path manipulation command list. ......................................... 18
Table 4.2 Available toolpath generators. ..................................................................... 24
1
Chapter 1 Introduction
Bioprinting is a rapidly growing multidisciplinary field of study. Bioprinting enables
researchers to produce biostructures cheaper and faster while allowing for greater flexibility in
bioconstruction design when compared to traditional manufacturing methods such as casting1.
This technology allows patterning of living cells, biological macromolecules, and other
biomaterial without significant reduction in cell viability2. Recent advances in additive
manufacturing are being adopted for bioprinting applications. According to the study conducted
by Grand View Research, this industry is projected to reach $2.6 billion market share by 20243.
Extrusion-based bioprinting originated from the fused deposition modeling (FDM)
fabrication technique created in the late 1990s. FDM was adopted to produce porous scaffolds
which provide the cells with a structure to grow and attach within, prolonging their lifespan4.
Tissue models and bioconstructs are bioprinted by slicing a solid mesh which can be
designed using Computer Aided Drafting (CAD) software or obtained by imaging techniques
such as MRI, CT or 3D scanning 5 6. Obtained slices of the geometry are used to generate
toolpaths, which the bioprinter executes layer by layer to build the desired bioconstruct.
Since bioprinting technology is still relatively new, researchers have adopted advances in
hardware and software development directly from the 3D printing industry. While large scale
models can be easily handled by conventional 3D printing slicing software7, they lack precision
control of the toolpath which is essential when printing small submillimeter scale features of
biodevices. In order to produce complicated designs, while maintaining precision control of the
2
bioprinters end effector, researchers resort to writing the G-code themselves2. This requires
calculation and definition of every waypoint for the bioprinter to execute. While this approach
works for small devices composed of few paths and waypoints, this technique requires
specialized knowlage of G-code specification, tends to be time consuming, and is prone to
mistakes. In order to simplify the biostructure design process, making bioprinting technology
accessible to researchers without technical backgrounds, new software tools are required, as
shown in the Bioprinting Toolchain Implementation schematic shown in Figure 1.1.
Figure 1.1 Bioprinting Toolchain Implementation.
3
Development of 5-axis motion stage development
Bioprinting complex scaffolds and biostructs using conventional 3-axis bioprinting
platforms is a challenging and sometimes even an impossible task8. In order to overcome these
constraints, a 5-axis 5 degrees of freedom (DOF) motion stage was designed and fabricated as
shown in Figure 1.2.
Figure 1.2 5DOF bioprinter CAD model and design.
The 5 DOF bioprinting platform features an open build surface allowing for bioprinting
directly on large specimens, by placing the device above the deposition target, such as an animal
or a human head. This thesis is an extension of ongoing efforts to implement a complete
bioprinting hardware platform and software toolchain designed specifically for bioprinting
applications. Continued work focuses on improving the software implementation of toolpath
planning and generation algorithms.
4
Development of motion control software framework
Motion control software was implemented using Python programing language enabling
the use of advanced features including loops, conditional statements, real-time monitoring of the
motion stage and peripheral device such as sensors, servo motors and solenoid valves. The
structure of the motion control software framework is summarized Figure 1.3.
Figure 1.3 Motion control software framework implementation.
Hardware subsystem handles communication with motion stages and sensors.
Automation subsystem generates G-code commands and monitors the hardware subsystem state.
Data objects contain models of reusable data. Utilities subsystem provides common functions. UI
subsystem provides graphical user interface to control and monitor the framework.
5
This framework was implemented in order to expand bioconstruction design freedom and
facilitate automation of common tasks. The G-code standard is interpreted sequentially by the
motion control card where G-code commands are translated to electrical pulses which move the
motion stage motors. The G-code standard is limited and lack logic commands such as loops and
conditional statements. A-code programing language was designed and implemented, extending
the capabilities of the G-code standard. A-code acts as an intermediate execution layer which
generates G-code commands in real-time implementing execution logic and allowing creation of
algorithm used for automation of common tasks. However, lack of an interactive user interface
and high level of complexity limited the functionality to users with technical backgrounds who
are familiar with the G-code and A-code standards. In order to reduce these technical barriers
and usage complexity; a toolpath designer virtual environment was implemented extending the
capabilities of motion control software shown in Figure 1.4.
Figure 1.4 Motion control software user interface
6
Motion control software description
Motion control software implements 3DOF, 5DOF and secondary motion stage
navigation interfaces displayed in Figure 1.4(1,2,3). These functions provide the ability to
control the position of the end effector within the build volume of the bioprinter. End effector
position may be recorded, providing quick navigation to regularly used locations as displayed in
Figure 1.4(4). Sensor and camera interfaces pictured in Figure 1.4(5,6) allow for real-time
monitoring and control of peripheral sensor devices. Automation interface shown in Figure
1.4(7) monitors the A-code subroutine execution and allows for development of A-code
algorithms in order to automate commonly performed tasks. Diagnostics interface display
information, warning and error messages. In addition, it provides a command terminal of the
primary motion stage hardware controller as shown in Figure 1.4(8).
7
Chapter 2 Current Toolpath Generation Software Used in Bioprinting
Conventional 3D printing toolpath generation
Slicing software designed for 3D printers can be used for bioprinting applications by
applying postprocessing algorithms to enable operation of pneumatic syringe dispensers7,
pressure sensors, pneumatic solenoid valves, and other peripheral devices. These software
packages tend to be complicated allowing for over 200 customizable settings9 10 11 which control
the shape of the toolpath produced by selecting layer height, path spacing, outer shell layer
count, infill direction and density to name a few. Some of the most popular slicers packages used
by the bioprinting community include Skeinforge 11, KISSlicer 12 and Slic3r 9.
While these slicers are excellent for producing toolpaths of relatively large models using
thermoplastics, they lack precision control over individual paths and waypoints. Bioprinting of
low viscosity bioinks like polydimethylsiloxane (PDMS)7 may require curing in order to obtain
optimal mechanical properties. This may result in structure buckling and printing failure during
unoptimized travel moves and path execution sequences.
8
2D bioprinting cell culture specific toolpath generation
Recently developed PetriPrinter software package aims to address some of these issues13.
PetriPrinter features Python application programming interface (API) layer in order to extend the
G-code functionality by providing toolpath generators of common shapes, live monitoring of
bioprinter states, ability to print on petri dish arrays, and to edit the g-code text imported from
third-party slicers. This software package is aimed at 2D cell culture work and does not provide
an interactive interface to manipulate toolpaths in 3D enviroment. Toolpath editing interface is
displayed in Figure 2.1.
Figure 2.1 PetriPrinter g-code editing interface.
9
3D bioprinting toolpath generation
Bioprinting specific toolpath generation tools are available, however they are designed
for specific bioprinter hardware such as Tissue Structure Information Modeling TSIM 14
developed by Advanced Solutions LLC. TSIM provides a graphical user interface for toolpath
creation, manipulation and post-processing of medical image data. However, this software works
only with a BioAssemblyBot bioprinter which costs over $100,000. High price and hardware
restrictions limit the software accessibility to a small portion of the bioprinting community.
10
Chapter 3
Development Requirements of Interactive 3D User Interfaces
A-Code implementation and lessons learned
Motion control software has been under development since 2017 and was extended to
accommodate 5-axis bioprinter platform’s motion control system8 in 2018. Motion control
software features an intermediate execution layer which allows for manipulation of toolpaths and
automation of common tasks. Automation is accomplished by implementing an intermediate
bioprinter control language A-code which interprets the commands and generates the G-code in
real-time for execution by the bioprinter. User interface displayed in Figure 3.1(A) was
implemented using (windows, icons, menus, pointing) WIMP UI model.
Figure 3.1 A-code interface software (A) and algorithm implementation (B).
11
A-code enables the use of advanced functions such as loops, conditional statements, real-
time monitoring of the motion stage and computer vision systems. In addition, it provides the
ability to interface peripheral devices such as solenoid valves and pressure sensors allowing the
user to implement automation algorithms for performing common tasks quickly. Example of A-
code algorithm flow chart is displayed in Figure 3.1(B)
While this functionality provided the means of automating everyday tasks, the resulting
system became too complicated to be useful. This was due to the lack of an interactive user
interface. Even technically inclined users would prefer to perform the tasks manually, rather than
defining an algorithm to automate repeatable tasks.
In order to overcome this issue, a user-friendly, interactive virtual environment VE was
required. Abstraction of the complicated software features was essential for broadening the user
base, making advanced features accessible for people without a technical background in
engineering such as surgeons and biologists, who dominate the bioengineering field.
Design requirements of a virtual environment for toolpath planning
The WIMP (windows, icons, menus, pointing) user interface (UI) model was developed
in the early 1980s. This UI model is excellent for applications requiring text entry and
manipulation of variables but cannot relay information such as a visual representation of
waypoints and geometry in 3D space. Toolpath planning for bioprinting applications requires the
user to define five to eight variables for every waypoint contained within the path as shown in
Table 3.1. Complicated bioconstructs may contain hundreds of paths resulting in an
overwhelming number of waypoints and variables which must be defined by the user.
12
Table 3.1 Variables required for one waypoint using 3DOF and 5DOF bioprinters.
3 DOF Bioprinting 5 DOF Bioprinting
Position vector (x,y,z) End effector velocity Volumetric flow of material
Position vector (x,y,z) End effector normal orientation vector (x,y,z) End effector velocity Volumetric flow of material
By combining the WIMP UI model with an interactive 3D virtual environment, this
complexity can be reduced significantly. A virtual environment enables the user to manipulate
and validate changes in the toolpath visually. By dragging and clicking the waypoints and paths
within the 3D space, the user’s productivity can be increased significantly. Since waypoints are
related though the parent path, all the variables defining the waypoints within the path can be
calculated automatically. The resulting paths can be inspected visually reflecting the changes
based on the users input by translating a waypoint or path within the virtual environment. Great
care must be considered when designing applications which use 3D virtual environments. People
have difficulty understanding virtual 3D spaces presented on 2D screens, due to the lack of
visual cues and depth present in the real world15. The interaction requirements are outlined in
Table 3.2.
Table 3.2 Virtual environment interaction requirements.
Requirements Description
Orbital camera control User is free to rotate, pan and zoom the perspective camera freely to inspect the design within the virtual build volume of the bioprinter.
Translation controls User is free to precisely translate waypoints, paths, and imported 3D geometry along individual axis: X, Y, Z as well as planes XY, YZ, XZ within the build volume with respect to world coordinates.
13
Table 3.2 Virtual environment interaction requirements (continued)
Requirements Description
Rotation controls User is free to orient the imported 3D geometry
Toolpath generator configurators User is free to place commonly used structures within the virtual environment with minimal effort and instant visual feedback to validate the generator’s parameters.
Based on these requirement, different aspects of the implementation should be considered
and include: Input/Output (I/O), navigation, toolpath manipulation, and selection. These features
should follow the general guidelines of interactive design maintaining simplicity, consistency
and error prevention16. These user interface aspects are outlined in Table 3.3.
Table 3.3 Virtual environment implementation aspects.
Aspect Description
IO Software should be platform independent in order to broaden the target audience and work on both desktop and mobile devices. It is expected that the device will have the following discrete inputs: mouse; and keyboard. Featuring a semi-immersive resolution independent display as an output.
Navigation and toolpath manipulation User should have the freedom of exploring the environment16 while transitioning smoothly between different interaction modes such as manipulation of waypoints, paths and imported geometry.
Selection User should be able to select specific objects within the environment as well as direction/plane of action while manipulating the objects within the virtual environment.
14
Chapter 4 Toolpath Editor Development
Figure 4.1 Toolpath Editor, (A) editor UI, (B) final render of resulting structure.
The toolpath editor allows for precise manipulation of waypoints and paths to achieve
precise and repeatable control of material deposition using a bioprinting platform. Sets of
waypoints are grouped in separate paths which are converted to A-code or G-code commands to
be executed sequentially by the bioprinter to produce the desired structure (Figure 4.1).
15
Toolpath editor framework development
Toolpath editor is written in JavaScript. JavaScript was chosen for toolpath editor
interface development due to platform independence, ease of distribution and vast availability of
common / scientific libraries17. Rendering of 3D geometry is facilitated through Three.js18
library. Mathematics libraries Verb.js19 and Math.js20 are utilized for performing mathematical
calculations. In addition, the toolpath editor may be hosted on a stand alone web server, or
served directly by the bioprinter’s interface software, where a real-time communication channel
between the bioprinter control software and toolpath editor is established through WebSocket
protocol. WebSocket allows for instantaneous monitoring of the bioprinter’s current position in
relation to the rest of the toolpaths to be executed. Currently, the designer interface is closely
coupled with the custom hardware implementation of 5 DOF motion stage, and 5 DOF toolpaths
are produced using A-code commands. 3 DOF toolpaths may be generated using G-code
standard and may be executed by any G-code compliant CNC motion stage as shown in Figure
4.2.
Figure 4.2 Example G-code output
16
Toolpath waypoints and paths
Waypoints are parametricized with a position vector p and an orientation vector n, to
orient the bioprinter’s end effector within the build volume. Waypoints also contain additional
information describing the bioink and end effector’s motion. Additional parameters include:
deposition method, end effector speed, material properties, and sequential execution information.
Sets of waypoints are represented as paths, the bioprinter will deposit the material
between the waypoints contained within the path. These paths are visualized in the editor
interface by extruding a circular shape along the path defined by a Catmull-Rom spline
constrained through the waypoints within the path.
The resulting design may contain many paths which are separated by travel moves during
which material deposition is turned off. Waypoints, paths and travel moves are displayed in
Figure 4.1.
Toolpath manipulation
Waypoints and paths can be translated, reversed and deleted within the 3D environment
to design and modify the desired structure (Figure 4.3). In order to simplify the design process,
newly appended waypoints snap to the previously placed waypoints and build platform surface.
The build platform surface is divided by an equally spaced grid pattern spanning the diameter of
the filament strand. The Filament diameter can be changed at any point during the design process
in order to accommodate different bioinks and biostruct design requirements. This approach
makes it easy to prototype symmetrical and dimensionally accurate devices quickly. Rendering
17
of the resultant toolpath is instantaneous; providing the user with visual feedback reflecting the
changes within the toolpath geometry.
Figure 4.3 Waypoint and Path translation using graphical user interface.
Toolpath tree manipulation and sorting techniques are summarized below in Table 4.1.
While the order of executed commands may be arbitrary, the final structure depends on the
command execution order.
18
Table 4.1 Waypoint and path manipulation command list.
Blue – start, Gray – middle, Red – ending
Structure reversal
Reverses all paths within the device, where the starting waypoint becomes the final waypoint.
Path path reversal Reverses a specific path, where the starting waypoint becomes the final waypoint.
Path deletion
Deletes a specific path within the bioconstruct.
Waypoint material Sets path parametrization and visual appearance for specific bioink type.
Path translation Translates all the waypoints contained within the path to the new location.
19
Table 4.1 Waypoint and path manipulation command list. (continued)
Blue – start, Gray – middle, Red – ending
Waypoint deletion
Deletes specific waypoint within a path.
Waypoint translation Translates the waypoint to the new location.
Waypoint speed Sets new end effector velocity for all further paths and waypoints, including the selected waypoint.
Travel move configuratrion
Prescribes the travel move strategy between the paths. Hop mode – bioprinter lifts the nozzle between the ending waypoint of the previous path and starting waypoint of sequential path. Flat mode - bioprinter does not lift the nozzle between the ending waypoint of the previous path and starting waypoint of sequential path.
20
Visual toolpath representation
The graphical user interface enables configuration of the designer workspace to
accommodate the biostruct design requirements. In order to reduce the complexity of workspace
and tool configuration, most of the variables and functions are derived from the following
independent variables: nozzle diameter; hop height for travel moves; build platform size;
material type; and starting position. These settings may be changed at any point throughout the
design process. Different configurations are displayed in Figure 4.4.
Figure 4.4 Properties and configuration of designer workspace
Configuring the hop height and starting position (A). Configuring the nozzle diameter (B).
21
Approximation of resulting structure is performed by constructing a circular extrusion
along the path by fitting a cubic centripetal Catmull-Rom spline through the waypoints contained
within the path. The tightness of the spline curve may be controlled through knot parametrization
variable α defined between 0 and 1. In addition, the overall shape of the path is also dependent
on waypoint density as displayed in Figure 4.5. Mathematical formulation of a Catmull-Rom
spline is presented in Figure 4.6.
Figure 4.5 Parametrization strategies of Catmull-Rom curves.
22
Figure 4.6 Barry and Goldman’s cubic Catmull-Rom curve formulation.
Curve segment C is defined between control points 𝑃0−3 and knot values 𝑠0−3 which are
parametrized through variable α. Intermediate coefficients L are used to produce interpolating 3rd Degree
polynomial curve segments which are valid between neighboring control points P.21
Low values of α produce a tighter curve where α=0, α=1 corresponds to uniform (red)
and chordal (blue) parametrization strategies as shown in Figure 4.5. Centripetal parametrization
of a Catmull-Rom spline was chosen, because it produces a smooth curve between the waypoints
without cusps and self-intersections21. This parametrization strategy provides greater control
over the shape of the path in order to match the shape of different bioinks with varied material
properties.
Values of α between 0.05 and 0.2 were determined to be most appropriate for
representing the extrusion of low viscosity materials such as PDMS and Pluronic by visual
inspection. Color and opacity of the filament may be adjusted to match the visual characteristics
of the bioink.
23
Toolpath generation of common geometry
Toolpath generation for common parametric shapes such as planes, cylinders, cones,
spirals and boxes are provided through an interactive user interface (Figure 4.7).
Figure 4.7 Waypoint Generators
The desired shape’s dimensions are defined by selecting the starting and ending points of
the shape. Additional shape dependent parameters such as height, wall thickness and toolpath
direction must be defined though WIMP UI interface. Toolpath generators allow the user to
prototype quickly, automating the waypoint placement process. The filament diameter is used to
calculate optimal waypoint placement, which is compatible with other tools and manually placed
waypoints allowing them to be used in combination to produce complex designs. Before the
waypoints are generated, a convex hull of the shape is displayed to validate the selected
parameters. Currently available generators are listed below in
Table 4.2. Generators are implemented to produce toolpaths which are continuous and do
not require any travel moves to produce the generated shape. This is important, because travel
moves have a potential of introducing defects and can cause buckling of the structure as
discussed previously.
24
Table 4.2 Available toolpath generators.
Generator Shape parameters Use cases
Snake
Snake generator produces a snake pattern within a rectangular bounding box.
•Pattern direction •Evenly cover large areas •Top and bottom faces of rectangular structures
Box
Box generator produces a rectangular box pattern.
•Height •Wall count
•Perfusion boxes •Outer shells •Channels •Solid square beams
Spiral
Spiral generator produces a spiral pattern.
•Start point •Direction
•Top and bottom faces of circular structures •Even filling of circular areas •Channels •Contact points •Solid rods
25
Table 4.2 Available toolpath generators. (continued)
Generator Shape parameters Use cases
Cylinder/Cone
Cylinder generator produces a cylindrical pattern.
•Start point •Top taper •Bottom taper •Wall segment count •Height
•Circular containers •Channels •Contact points •Hollow rods
Convex Mesh
Convex Mesh generator produces toolpaths outlining sliced mesh geometry.
•Mesh, position, orientation •Minimum and Maximum radius of geometry •Layer count •Initial nozzle size •Loop count •Neighboring point trim •Waypoint spacing
•Even filling of simple arbitrary shapes •Repeatable complex toolpath generation
Surface Generator
Surface Generator produces toolpaths along a NURBS surface.
•Grid size, spacing •Degree of surface •Waypoint spacing
•Curved geometry •Channels •Coating of arbitrary surfaces
26
3D scanning
The ability to visualize, manipulate and produce toolpaths based on captured mesh
surfaces is a desirable feature in bioprinting community.22 In order to produce anatomically
correct devices and models, a precise virtual representation of their internal structures and
surface topologies may be obtained through medical imaging techniques including CT, MRI and
3D scanning as shown in Figure 4.8.
3D scanners produce point cloud representations of captured surfaces. Point cloud data
can be noisy and contain artifacts as displayed in Figure 4.8(B). Post-processing techniques must
be selected on an individual basis to accommodate the present scanning conditions in order to
produce a valid mesh comprised of triangular faces defined by a set of vertices as shown in
Figure 4.8(D). One of the most commonly used standards for captured geometry representation
is stereolithography (STL) format specification.
First, the specimen and targeted scanning areas must be cleaned thoroughly as presented
in Figure 4.8(A). Any liquid contained within the concavities of the scan target area will produce
surface artifacts due to diffraction of the scanner’s laser beam, as shown in Figure 4.8(D).
Immediately after cleaning, the subject is scanned using a handheld Artec Spider 3D scanner
(Artec3D, usa). The obtained point cloud is registered, aligned and converted to a solid mesh
composed of triangles and vertices. This mesh is exported in STL format to be used within the
toolpath designer environment. An imported mesh can serve as a reference while designing the
biostruct, or can be used to produce toolpaths within the concavities using convex mesh
generator.
27
Figure 4.8 Example procedure for 3D scanning of a 5mm defect during surgery.
In this example live rat specimen is prepared by the surgeon. Exposed cranium of the rat
is 3D scanned before bioprinting of hydrogel bioink containing cells which promote bone
regeneration. Four regions are shown in Figure 4.8. Region A show the exposed skull of a rat
with two critical (5mm) defects. Region B show the initial point cloud captured using 3D
scanner. Region C show the point cloud data after application of post-processing techniques.
Region D show the final geometry ready for import within designer virtual environment.
28
Chapter 5 Advanced Features
Toolpath generation within convex surface geometry
Bioprinting directly on live specimen is challenging due to variables such as surface
topology, contour and convexity of the surface, a bioprinter’s nozzle should not collide with the
specimen. In addition, there is a time constraint, because the live specimen is under anesthesia
during bioprinting operation and may become alert. While it is feasible to perform the waypoint
placement manually, the number of required waypoints grow quickly when the size of the defect
increases. In order to overcome these issues an automatic generation algorithm was developed
and implemented as a shape generator, as shown in Figure 5.1. The principles of operation and
use requirements for this shape generator are explained below.
Figure 5.1 Toolpath generated based on 3D scan obtained data.
Seven regions are shown in Figure 5.1. Region A show the 3D scan of a rat’s cranium
imported within designer virtual environment. Region B show the obtained contour of the
defect’s geometry. Region C show the visual validation of the toolpath produced using convex
mesh generator. Region D show the waypoints produced using convex mesh generator. Region E
show the final complete toolpath ready for bioprinting. Regions F and G show the toolpath
produced using convex mesh generator within complex defect boundary with different
parametrization strategies.
29
First, subject’s geometrical representation is obtained using a handheld 3D scanner. Solid
mesh is imported within a virtual environment, where it is prepared for toolpath generation by
aligning the mesh within the printers build volume as shown in Figure 5.1(A).
Next, the mesh geometry is sliced using an xy-plane defined at the origin of build
surface, as shown in Figure 5.1(B). The slicing is performed by checking for intersections
between the plane and the lines connecting vertices of triangular faces. Intersection points are
aggregated into an unordered list defining the wall boundary of the mesh along the slicing plane.
The desired contour is then chosen by clicking the boundary outline as shown in Figure
5.1(B). Since meshes may contain numerous independent islands, the user must select the desired
contour. A nearest-member search algorithm is used for a selected contour point to define an
ordered list of points along the contour. A Catmul-rom spline is fit through the ordered set of
points to obtain a smooth curve.
Finally, the generator’s parameters can be adjusted in order to modify the generated
waypoint density until the desired shape of the toolpath is obtained and visually validated as
shown in Figure 5.1(C). The previously obtained smooth curve defining the boundary of the
geometry is used to generate the first path along the geometry’s border by offsetting the curve
inwards in relation to the direction of the path. Sequential loops use the previous path’s smooth
curve to generate up to Lc contours. While this algorithm is capable of handling convex circular
curves very well as shown in Figure 5.1(E), self-intersections begin to arise along concavities
throughout the path as seen in Figure 5.1(G). This problem is addressed by adjusting waypoint
spacing and checking for neighboring points before subsequent waypoint placement. This may
be adjusted through algorithm parametrization as shown in Figure 5.1(F). A pseudo code for the
base toolpath generation algorithm is provided in Figure 5.2.
30
Pseudo code:
1 𝐶𝑝 = 𝐶𝑖
2 for each 𝐿𝑐 3 for each 𝑊 in 𝐶𝑝
4 𝑾𝒑 = 𝑾𝒑 + 𝑛𝑜𝑟𝑚(𝑪𝒅 × 𝑺𝒏 ) ∗ 𝐷𝑛
5 end 6 𝐶𝑝 = 𝐶𝑐
7 end
Variable definitions:
𝐶𝑖 = 𝐼𝑛𝑖𝑡𝑖𝑎𝑙 𝑝𝑎𝑡ℎ,
𝐶𝑝 = 𝑃𝑟𝑒𝑣𝑖𝑜𝑢𝑠 𝑝𝑎𝑡ℎ,
𝐶𝑐 = 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑝𝑎𝑡ℎ, 𝑪𝒅 = 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑝𝑎𝑡ℎ 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛 𝑣𝑒𝑐𝑡𝑜𝑟,
𝐿𝑐 = 𝐿𝑜𝑜𝑝 𝑐𝑜𝑢𝑛𝑡, 𝐷𝑛 = 𝐷𝑖𝑎𝑚𝑒𝑡𝑒𝑟 𝑜𝑓 𝑡ℎ𝑒 𝑛𝑜𝑧𝑧𝑙𝑒,
𝑊 = 𝑊𝑎𝑦𝑝𝑜𝑖𝑛𝑡, 𝑾𝒑 = 𝑃𝑜𝑠𝑖𝑡𝑖𝑜𝑛 𝑜𝑓 𝑐𝑢𝑟𝑟𝑒𝑛𝑡 𝑊𝑎𝑦𝑝𝑜𝑖𝑛𝑡,
𝑺𝒏 = 𝑁𝑜𝑟𝑚𝑎𝑙 𝑣𝑒𝑐𝑡𝑜𝑟 𝑜𝑓 𝑠𝑙𝑖𝑐𝑖𝑛𝑔 𝑝𝑙𝑎𝑛𝑒,
Figure 5.2 Pseudo code of toolpath generation algorithm within convex surface geometry
31
Toolpath generation on free-form surfaces
Bioprinting along complex surface geometries is challenging, since a stereolithography
(STL) geometry of a surface representation may contain an unordered list of vertices and faces,
and there may be discontinuities across the surface, missing geometry, roughness, and acute
interfacial angles, which impact the stability of toolpath planning algorithms. These issues may
result in large changes of surface normal vector producing paths outside the capabilities of the
bioprinter’s hardware. In order to overcome this problem, we need a surface representation
which is defined by smooth differentiable functions. These functions are used for finer
segmentation of paths to produce waypoints positions and normal vectors within the capabilities
of the bioprinters hardware.
Nonuniform rational B-spline (NURBS) representation of a surface 𝑆(𝑢, 𝑣) can satisfy our
requirements23. In order to implement a dynamic 𝑁𝑤 x 𝑁ℎ grid surface with desired spacing (𝑆𝑝),
we must generate an appropriate set of control points (P), weights (w), degrees (n) and knot
vectors (k) to satisfy the basis functions (N), Graphical representation of mathematical 2nd degree
NURBS surface formulation is shown in Figure 5.2.
Figure 5.2 2nd degree 3x6 grid NURBS surface formulation23.
32
𝑆(𝑢, 𝑣) = ∑ ∑ 𝑅𝑖,𝑛(𝑢, 𝑣) 𝑃𝑖,𝑗
𝑙
𝑗=1
𝑘
𝑖=1
𝑅𝑖,𝑗(𝑢, 𝑣) = 𝑁𝑖,𝑛(𝑢)𝑁𝑗,𝑚(𝑣)𝑤𝑖,𝑗
𝑁𝑝,𝑛(𝑢)𝑁𝑞,𝑚(𝑣)𝑤𝑝,𝑞
𝑁𝑖,𝑛 = 𝑓𝑖,𝑛𝑁𝑖,𝑛−1 + 𝑔𝑖+1,𝑛𝑁𝑖+1,𝑛−1
𝑓𝑖,𝑛(𝑢) =𝑢−𝑘𝑖
𝑘𝑖+𝑛−𝑘𝑖, 𝑔𝑖,𝑛(𝑢) =
𝑘𝑖+𝑛−𝑢
𝑘𝑖+𝑛−𝑘𝑖
𝑓𝑖,𝑛(𝑣) =𝑢−𝑘𝑖
𝑘𝑖+𝑛−𝑘𝑖, 𝑔𝑖,𝑛(𝑣) =
𝑘𝑖+𝑛−𝑢
𝑘𝑖+𝑛−𝑘𝑖
Function 𝑆(𝑢, 𝑣) represents the NURBS surface which is bivariate vector-valued piecewise
rational function defined in region (𝑢, 𝑣) ∈ [0, 1]. where 𝑅𝑖,𝑗(𝑢, 𝑣) define piecewise rational basis
functions of desiered degree 𝑛 defined on knot vectors 𝑘𝑢 𝑎𝑛𝑑 𝑘𝑣 which are nondecreasing sequences of
real numbers. Knot vectors control the span of peacewise functions though control point net defined by
𝑃𝑖,𝑗 where i,j corespond to 𝑁𝑤 x 𝑁ℎ grid surface with desired spacing (𝑆𝑝) 23.
The knot vector must be uniform while clamping the surface at the surrounding edge
control points. Since the shape and size of the captured geometry may vary, flexibility of the
surface size and control point density is required. Therefore, the knot vector must be computed
automatically and depend on the grid size. The Pseudo code for knot vector generation is
presented in Figure 5.3.
33
Pseudo code:
1 2 3 4 5 6
7 8
𝑓𝑜𝑟 𝑒𝑎𝑐ℎ 𝑘𝑢 𝑖𝑓 𝑖 ≤ 𝑛𝑢 𝑘𝑖 = 0 𝑒𝑙𝑠𝑒 𝑖𝑓 𝑖 ≥ 𝑁𝑤 𝑘𝑖 = 1 𝑒𝑙𝑠𝑒 𝑘𝑖 = 𝑖/ 𝑁𝑤 end
𝑓𝑜𝑟 𝑒𝑎𝑐ℎ 𝑘𝑣 𝑖𝑓 𝑖 ≤ 𝑛𝑣 𝑘𝑖 = 0 𝑒𝑙𝑠𝑒 𝑖𝑓 𝑖 ≥ 𝑁ℎ 𝑘𝑖 = 1 𝑒𝑙𝑠𝑒 𝑘𝑖 = 𝑖/𝑁ℎ end
Figure 5.3 Knot vector generation algorithm.
Selected examples of different surfaces and control point configurations are provided in
Figure 5.4. 2nd Degree surfaces feature high local surface dependence on control point position as
shown in Figure 5.4(A). Four neighboring control points are enough to position the surface
through the plane defined by the four control points. In addition, small grids spanning only three
control points may be used as shown in Figure 5.4(B).
Figure 5.4 Example of 10x10 (A) and 3x10 (B) NURB surfaces of degree 2.
Example 1
10x10 surface of degree 2 is presented in Figure 5.4(A):
𝑘𝑢 = [0, 0, 0, 0.23, 0.30, 0.38, 0.46, 0.53, 0.61, 0.69, 1, 1, 1]
𝑘𝑢 = 𝑘𝑣 (due to symmetry)
Example 2
3x10 surface of degree 2 is presented in Figure 5.4(B):
𝑘𝑢 = [0, 0, 0, 1, 1, 1]
𝑘𝑣 = [0, 0, 0, 0.23, 0.30, 0.38, 0.46, 0.53, 0.61, 0.69, 1, 1, 1]
34
The basis functions of NURBS surface are a piece wise polynomial curves calculated
once for the given surface. Basis functions are used for evaluation of any point along the surface
and depends on the knot vector k and control point position.
Control points are initialized as an equally spaced grid along xy-plane. These control
points may be manipulated in order to position the surface tangent to the mesh geometry as
shown in Figure 5.5 where green regions represent the generated surface and red patches display
the surface roughness of the imported mesh.
Figure 5.5 NURBS surface positioned tangent to the surface of an imported mesh.
Designer virtual environment is capable of handling surfaces of 2nd,3rd and 4th degree.
The degree of the surface corresponds to the local surface dependence on position of surrounding
control points and corresponds to the degree of piecewise polynomial curves which compose the
NURBS surface. 2nd degree surfaces are prefered in order to reduce the computational
complexity, while featuring high local dependence of surface geometry with respect to local
35
control point position when compared to higher degree surfaces. This parametrization strategy
allows the use of small surface grids with minimal number of control points. Surfaces of various
degrees are compared in Figure 5.6.
Figure 5.6 NURBS surfaces of degree 2, 3 and 4.
The differentiability properties of NURBS surfaces allows us to obtain the normal vector
along a surface23. Normal and position vectors are sampled across the surface allowing for
toolpath waypoint placement on top of the surface. The pitch of surface sampling may be
adjusted to accommodate the design parameters. Sampled points display normal vectors (yellow
arrows) and provide the user with interactive spheres which can be selected within virtual
environment for waypoint placement. Waypoints are generated on top of the surface by
offsetting the waypoint position by the nozzle diameter in the direction normal to the surface as
shown in Figure 5.7. In addition, the missing geometry patches do not affect the toolpath
placement and allow incomplete geometry to be used for toolpath generation.
36
Figure 5.7 Path generation using NURB surfaces.
37
Chapter 6 Toolpath Designer Application Examples
Several examples of designer virtual environment (VE) usage are presented below.
Designs based on imported geometrical data
Geometrical objects may be imported within the VE in order to produce dimensionally
accurate designs without having to manually measure the object. In this example, the geometry
of a semilunar heart valve was obtained from 3dprint.nih.gov design database. The STL file was
obtained, imported and positioned within the application environment. A cylinder generator was
used four times to build the outer shell of the heart valve as shown in Figure 6.1.
Figure 6.1 Design of semilunar heart valve based on imported STL geometry.
38
Toolpath generation based on 3D scan data of live specimen
Precise deposition of hydrogels in surgical setting can be accomplished by 3D scanning
the specimen and using a convex toolpath generator to produce the toolpaths for the bioprinter to
execute as shown in the following example pictured in Figure 6.2.
Figure 6.2 Testing convex toolpath generator for bioprinting on live specimen.
First, the surgeon exposes the cranium of the specimen and cleans the excess liquid to
prepare the surface for 3D scanning procedure (Figure 6.2A). The exposed cranium is scanned
using a handheld 3D scanner (Figure 6.2B), and the STL geometrical representation of the
surface is imported within designer VE (Figure 6.2C). Toolpaths are generated using the convex
toolpath generator (Figure 6.2D) and are executed by the bioprinter (Figure 6.2E, F).
39
Chapter 7 Conclusions
This thesis has shown how a complete software toolchain for 3DOF and 5DOF
bioprinting platforms can be developed and implemented. This toolchain enables fast prototyping
of bioconstruct design, use of bioprinting platforms in surgical setting, generation of 5DOF
toolpaths on freeform surfaces and enabling users without a technical background to generate
toolpaths for bioprinting applications. This was accomplished by:
• Implementing an interactive 3D toolpath designer virtual environment.
• Development of algorithms which abstract the calculation of waypoint variables
for toolpath creation and manipulation tasks through interactive virtual
environment.
• Development of toolpath generators, which allow fast generation of waypoints
and paths for common shapes.
• Development of bioprinting specific slicer, capable of generating toolpaths within
the concavities of mesh geometry.
• Development of freeform surface generation algorithms in order to bioprint using
5DOF motion stages based on imported mesh data.
40
BIBLIOGRAPHY
1. Au, A. K., Huynh, W., Horowitz, L. F. & Folch, A. 3D-Printed Microfluidics. Angew.
Chemie Int. Ed. 55, 3862–3881 (2016).
2. He, Y. et al. Research on the printability of hydrogels in 3D bioprinting. Nat. Publ. Gr.
(2016). doi:10.1038/srep29977
3. Grand View Research. Grand View Research Updates on the 3D Bioprinting Market.
Entertainment Close - Up (2018).
4. You, F. et al. Application of Extrusion-Based Hydrogel Bioprinting for Cartilage Tissue
Engineering. Int. J. Mol. Sci. 18, 1597 (2017).
5. Derakhshanfar, S. et al. 3D bioprinting for biomedical devices and tissue engineering: A
review of recent trends and advances. Bioact. Mater. 3, 144–156 (2018).
6. Dababneh, A. B. & Ozbolat, I. T. Bioprinting Technology: A Current State-of-the-Art
Review. J. Manuf. Sci. Eng. 136, 061016 (2014).
7. Ozbolat, V. et al. 3D Printing of PDMS Improves Its Mechanical and Cell Adhesion
Properties. ACS Biomater. Sci. Eng. 4, (2018).
8. Povilianskas, A. Development and Implementation of 5 Degree of Freedom Bioprinting
Motion Stage, Control Software and Toolpath Generation Algorithms. BS Honor thesis
(Penn State, 2018).
9. Ranellucci, A. & Lenox, J. Slic3r - About. Available at: https://slic3r.org/about/.
(Accessed: 4th April 2019)
10. Braam, D. Ultimaker Cura: Advanced 3D printing software, made accessible. Available
at: https://ultimaker.com/en/products/ultimaker-cura-software. (Accessed: 4th April 2019)
41
11. Enrique. Skeinforge - RepRap. (2015). Available at: https://reprap.org/wiki/Skeinforge.
(Accessed: 4th April 2019)
12. KISSlicer LLC. KISSlicer. Available at: http://www.kisslicer.com/. (Accessed: 4th April
2019)
13. Gulyas, M., Csiszer, M., Mehes, E. & Czirok, A. Software tools for cell culture-related 3D
printed structures. PLoS One 13, 1–11 (2018).
14. Advanced Solutions. TSIM Software. Available at: https://lifesciences.solutions/tsim/.
(Accessed: 4th April 2019)
15. Kruijff, E., LaViola, J., Poupyrev, I. & Bowman, D. An Introduction to 3D User Interface
Design. Presence Teleoperators Virtual Environ. 10, 96–108 (2001).
16. Biström, J., Cogliati, A. & Rouhiainen, K. Post-WIMP User Interface Model for 3D Web
Applications. 8, (2005).
17. DiPierro, M. The Rise of JavaScript. Comput. Sci. Eng. 20, 9–10 (2018).
18. MrDoob. White paper. gitHub.com (2012).
19. Boyer, P. verb. Available at: http://verbnurbs.com/docs/. (Accessed: 6th April 2019)
20. math.js | an extensive math library for JavaScript and Node.js. Available at:
https://mathjs.org/. (Accessed: 6th April 2019)
21. Piegl, L. Algorithms for Computing Conic Splines. J. Comput. Civ. Eng. 4, 180–198
(1990).
22. Mahmoudi, S. E. et al. Web-based interactive 2D/3D medical image processing and
visualization software. Comput. Methods Programs Biomed. 98, 172–182 (2009).
23. Piegl, L. A. & Tiller, W. The NURBS book. (Springer, 1997).