8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 1/26
Aesthetics of Computation
Visualizing Software
Presented by Ray Garcia
March 7, 2007
University of Arizona
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 2/26
Noesis Challenge
Noesis = Perception + Learning + Reasoning
Software is not yet a mature engineeringprofession and needs to strive to be an
artistic craft that satisfies utility (function) as
well as beauty (form) as co-evolving goals
while enabling anyone to learn the craft.
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 3/26
Software as Art
Produces Artifacts, sometimes unpredictably Malleable Material but limited by the tools Variety of interactive and temporal expressions of a concept User creates their own meaning
The process requires Understanding, abstraction, insight, and interpretation Mastery of a craft Representational patterns and forms Creativity and inventiveness
For general public appreciation it lacks Inaccessible as a communications medium Visibility of computation Aesthetic Sense and Critical Review Cultural constructs
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 4/26
Visible computation
“Programs should be written for people to read,
understand, and manipulate, and only incidentally
for computers to execute”
Visible computation allows for direct observationand interaction with the code execution supporting: Ease of consumption (reading)
Ease of comprehension (thinking)
Ease of construction (writing)
Software should be a communications medium of
design and its intended utility
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 5/26
Computation is invisible
Programming is the craft of manipulatingcomputation
Programming systems enable abstract specificationof the solution in text and diagrams (code & UML)
The problem, intent, and context is lost in thespecification
The specification is not the execution The diagrams and the text are not inter-changeable
and require annotation IDE’s and Modeling tools are not enough Execution is what matters most yet it is obscured
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 6/26
Programming Systems
Programming tools prevent computation from being
visible and accessible by the novice or curious
spectator
Programming concepts are generally easy tounderstand Data structures, state manipulation, conditionals, recursion,
algorithms, …..
Incomplete set of tools and representation for dealing with computation
The mysteries of computation should be exposed
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 7/26
Current Tools are Incomplete
Specification is not computation Anatomy is not a not alive
Blueprints are not a building
A score is not music Code is not computation
What is visible are the remnants or output of
computation UI, reports Debuggers, tracing, footprints
Logs files, profilers
Static snapshots are not enough
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 8/26
Thinking of Computation
Human skim/scan faster than reading
Human process images and sound faster than text
People think in visual spatial terms for many
technical problems
The reader needs to reconstruct a mental map that
existed in the creators mind
Novice programmer construct their own internalrepresentation of computation through trial and error Need to think of computation as a dynamic process
instead of as a static specification language
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 9/26
Unnatural Code
Computation is a multi-dimensional and non-linear
dynamic structure
Code may have parallel or multiple execution paths
Navigating through one dimensional text is likeunraveling a contorted knot
Code relies on understanding written language
Even commented code is difficult to understandwithout the reader exerting a lot of effort
Code strays too far from natural language
Annotated diagrams don’t go far enough
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 10/26
Representations of
Computation
Why isn’t programming more like interactivehypertext with diagrams and animation that can bemanipulated directly?
The syntax of code has no relationship to theruntime process, it is a constraint of the parser Generating names for identifiers is simpler than
creating a picture of the same information
Diagrams can more easily represent structure andflow The representations should be designed to conform
to the limits of Human perception
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 11/26
Programming as a Craft
Requires easy tools which makes the craft possibleand accessible to the novice
The craft needs to conform to the limitations of humancognition instead of forcing people to think likecomputers
Lack of visible computation negatively impact ease of use and communication
Visible computation would facilitate learning An aesthetic of computation is needed to advance the
craft of programming Craft = Literate programming + Visual Modeling +
Visible Computation + Aesthetics
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 12/26
Literate Programming
Program expressively to communicate to anaudience instead for a computer to execute
Tool Support Syntax directed IDE Text Mapping Code Navigation Code Hypertext Documentation Testing and tracing frameworks Code coverage Cross hatching with Aspect Oriented programming
Meta Coding Editorial Code Refactoring Code generators Domain Specific Language development Evolutionary genetic programming
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 13/26
Visual Programming
Visual programming Computation
material data
process
Visual Interface visual language
Construction
navigation Aesthetics
Patterns
critical theory
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 14/26
Computational Processes
Object oriented – interactions between objects thatcontain state and procedures (smalltalk) Imperative – execution of sequential commands with state Procedural – combination of imperative and functional can
affect state
Functional – calculation of mathematical functionswithout state (ocaml)
Constraint based – rules of truth and logicstatements (prolog)
Multi-paradigm languages (mozart) NLP = ontology + behaviors + natural language
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 15/26
Visual Design
Human Perception needs continuity Human mind is not configured to follow discrete
transitions and needs smooth continuous
representations i.e.. animations Gradual updates are easier to perceive
Computation is quantized therefore poses a
design challengeClear spatial relationships
Minimal context switching
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 16/26
Visual language
The visual language should be used for
specification, interaction, and execution
Show cause and effect
Computation should appear as an animated
interactive film that may be edited
Challenge to invoke rhythm and drama in the
animation of computation
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 17/26
Visual Modeling (UML)
Remaining Problems UML is limited to a static representation of mixed
perspectives on the specification
Managing scale and levels of abstraction
Context and transparency of components
On the Horizon
Executable UML Pattern Languages
Model Driven Architecture
Model Driven Development
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 18/26
Possible Solutions
Invent a new visual language design and execution
environment
Extend UML to support code generation, execution
and interactive animation Parse and instrument or probe existing code to
generate animation code to view the computation
with annotation links to the source code
Create a language design environment that
supports human cognition using the full multimedia
capabilities
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 19/26
Complexity, Crowds, Chaos
Size and complexity
Software maintenance and evolution
History through versioning
Open source failure rate fosters innovation
Peer review and collaborations
Chaos due to rate and impact of change
Software development as a social phenomenon Learning theory applied to software development
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 20/26
Scope, Scale, and Perspective
Amplified perception Augmented human & social intelligence Audience support for users and creators Information hiding and abstraction Software comprehensibility through visualization
& sound synthesis of computation execution,interaction, and navigation
Algorithm animation Software Aesthetics Critical Theory for software
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 21/26
Algorithm Animation Examples
cyclic queue
train sorter
dijkstra shortest path
Further study
http://www2.hig.no/~algmet/animate.html
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 22/26
Visual Programming Examples
Marten http://www.andescotia.com [Mac osx]
National Instruments LabView http://www.ni.com/labview/
[device control]Pictorial Janus http://users.encs.concordia.ca/~haarslev/pjmovies/
ToonTalk http://www.toontalk.com [Kids visual
programming]
PureData PD http://puredata.info/ [Media graphical
programming environment]
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 23/26
Pictorial Janus Syntax
Call Configuration
RuleAgentConstraint
Function
Port emptyList
Constant
Link Channel
abc List ([a,b,c | [])
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 24/26
Pictorial Janus Example
Append 2 List
Distributed Queue
8/14/2019 Aesthetics of Computation
http://slidepdf.com/reader/full/aesthetics-of-computation 25/26
Animated UML
OOP Eggs display – using biomimicry to
visualize software
Visio Example of animated UML diagram
Future possibility UML X3D