1 / 21

CodeHint: Dynamic and Interactive Synthesis of Code Snippets

Joel Galenson, Ras Bodik, Koushik SenUC Berkeley

2 / 21

Motivation

Pictures by Mangpo Phitchaya Phothilimthana

Test cases

Desired type

Specification

3 / 21

CodeHint: Autocomplete for the modern age

● Autocomplete is useful but very limited.● Our improvements:

– Being dynamic

– General specifications

– Synthesis

4 / 21

Overview

CodeHint

Source code

Test case

Specification

CandidatesFind locals Evaluate Candidates

Combine

Refine

5 / 21

Example: The problem

6 / 21

Example: First step

“Find a JMenuBar”

7 / 21

Demo

8 / 21

Basic algorithm

● Find local variables, do BFS over Java statements, show user those that pass spec.– Actually evaluate these statements, including file

I/O, reflection, etc.

x

y

Iteration 1

x

+

y

x

max

y

Iteration 2 Iteration 3

x

max

y

average

x

+

y

...

... ......

9 / 21

First iteration

● Find local variables.

x y tree nullthis ...

10 / 21

Second iteration

● Get each expression's type and combine it with others in type-correct ways.

xx

+

y x

<

y

...

tree(JTree)

getPathForLocation

x

tree

y (JTree)

getPathForLocation

y

tree

x tree

getTopLevelAncestor

...

tree

getWindowAncestor

SwingUtilities...

11 / 21

Side effects

● Handle in-memory and external effects.

makeTree()

add

tree

children...

renderer makeTree()

...

tree

tree.children changed from torenderer renderer makeTree()

file

delete

SecurityManager

Breakpoints

In-memory:

External:

12 / 21tree

getTopLevelAncestor

getJMenuBar

Equivalence classes

● Group equivalent code to avoid unneeded work.

tree

getRoot

SwingUtilitiestree

getTopLevelAncestor

tree

getTopLevelAncestor

getName

tree

getRoot

SwingUtilities

getName

tree

getRoot

SwingUtilities

getJMenuBar

tree

getTopLevelAncestor

getJMenuBar

Next iteration:

End of search:

13 / 21

Probabilistic model

● Mined 10MLOC to build model of likely code.

● Use to avoid unlikely expressions.

Probability of accessing method/field m on type T:

Probability of using constant c as argument i to method m:

tree

getNextMatch

... KeyEvent.VK_ENTER

getComponent

tree

14 / 21

Third iteration and result

tree

getTopLevelAncestor

(JFrame)

getJMenuBar

(JTree)

getRowForLocation

x

tree

y

+

1

...Third iteration:

Result:(JFrame)

getJMenuBar

tree

getRoot

SwingUtilitiestree

getTopLevelAncestor

(JFrame)

getJMenuBar

15 / 21

Refinement

● Users can give another demonstration in a different state to refine the results.

(JFrame)

getJMenuBar

tree

getWindowAncestor

SwingUtilitiestree

getTopLevelAncestor

(JFrame)

getJMenuBar

...

16 / 21

Synthesis from specifications

StrengthWeak Strong

Context-dependence

Context-independent

Context-dependent

APIexploration

Programsynthesis

Programmingby demonstration

Static typerv instanceof JMenuBar

Correctness conditionisSorted(array)

Value demonstrationsx == 42

x.toString().contains(“Eve”)

17 / 21

Empirical results

In real-world code, ~95% of expressions need ≤3 iterations and ~99% need ≤4.

Our algorithms workwell in practice.

18 / 21

User studies

● Completed two user studies with 28 subjects.● Found statistically-significant productivity

improvements.– Fewer bugs and more tasks completed in less time.

CodeHint makes programmersmore productive.

Lower is better Higher is better

19 / 21

Future work

● Improving the probabilistic model● CodeHint for JavaScript

– https://github.com/jgalenson/codehint.js

–

● Integrating symbolic techniques

20 / 21

Probabilistic search

foo

;

bar

tree

getWindowAncestor

SwingUtilities

...

Model

Candidates

tree

getWindowAncestor

SwingUtilities

(JFrame)

getJMenuBar

21 / 21

Summary

● Dynamic and interactive synthesis● Autocomplete for the modern age● User studies showed productivity improvements

Thanks!https://jgalenson.github.io/codehint/