Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

John Altidor

Taming the Wildcards:Combining Definition- and Use-Site Variance

2Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Outline

Motivation for Variance. Existing Approaches.

• Java only supports use-site variance.• C# and Scala only support definition-site variance.

What Is New Here:Combine Definition-Site and Use-Site Variance.• Both in a single language, each using the other• Case Study – Inferring Def-Site for Java.• Insights into Formal Reasoning.

Summary.

3Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Software Reusability

Writing reusable software is hard; many factors:• Generality• Adaptability• Modularity• Simplicity of interface / Ease of use• …

Programming language support• Enable reusable code without introducing bugs.• Focus on integrating two flavors of genericity:

• Generics – Parametric Polymorphism• Subtyping – Inclusion Polymorphism

4Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Subtyping – Inclusion Polymorphism

Example: Java inheritance.

class Animal { void speak() { }}class Dog extends Animal { void speak() { print(“bark”); }}class Cat extends Animal { void speak() { print(“meow”); }}

5Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Subtyping – Example Client

void performSpeak(Animal animal){ animal.speak();}

Can be aDog,Cat,

or any other Animal.

6Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Common Code Structure

class ListOfDogs{ void add(Dog num) { … }

Dog get(int i) { … }

int size() { … }}

7Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Common Code Structure (cont.)

class ListOfAnimals{ void add(Animal num) { … }

Animal get(int i) { … }

int size() { … }}

8Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Generics – Parametric Polymorphism

class List<X>{ void add(X x) { … }

X get(int i) { … }

int size() { … }}

type parameter

write X

read X

no X

List<Animal> ≡ List of AnimalsList<Dog> ≡ List of Dogs

CustomizedLists

9Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Generics and Subtyping

Dog <: Animal (Dog is an Animal). Cat <: Animal (Cat is an Animal).

List<Dog> <: List<Animal>

No!

A List<Animal> can add a Cat to itself.A List<Dog> cannot.

10Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Variance Introduction

When is C<Expr1> a subtype of C<Expr2>?

class RList<X>{ X get(int i) { … }

int size() { … }}

Can read from

but not write to.

It is safe to assumeRList<Dog> <: RList<Animal>.

Why?

11Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Flavors of Variance - Covariance

Generic<Dog> Generic<Animal>

Covariance

<:

Assuming Dog <: Animal (Dog is an Animal).

12Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Flavors of Variance - Contravariance

Generic<Dog> Generic<Animal>

Contravariance

<:

Assuming Dog <: Animal (Dog is an Animal).

13Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Four Flavors of Variance

How do programmers specify variance?

Covariance: T <: U ⇒ C<T> <: C<U>.

Contravariance: T <: U ⇒ C<U> <: C<T>.

Bivariance: C<T> <: C<U>, for all T and U.

Invariance: C<T> <: C<U>, if T <: U and U <: T.

14Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Definition-Site Variance (C#/ Scala)

Programmer specifies variance in definition as in Scala and C#.

Variance of a type position.• Return types: covariant.• Arguments types: contravariant.

class RList<+X> { X get(int i) { … } int size() { … } // no method to add}

class WList<-X> { void add(X x) { … } int size() { … } // no method to get}

15Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Use-Site Variance (Java Wildcards)

class List<X>{ void add(X x) { … }

X get(int i) { … }

int size() { … }}

16Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Use-Site Variance (Java Wildcards)

class List<X>{ void add(X x) { … }

X get(int i) { … }

int size() { … }}

List<? extends X>

17Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Use-Site Variance (Java Wildcards)

class List<X>{ void add(X x) { … }

X get(int i) { … }

int size() { … }}

List<? super X>

18Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Use-Site Variance (Java Wildcards)

class List<X>{ void add(X x) { … }

X get(int i) { … }

int size() { … }}

List<?>

19Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Definition-Site: Pros

Conceptual Simplicity• Simpler Type Expressions – no use-site annotations.• Subtype policy stated in class definition.

Burden on library designers; not on users.• Classes declare variance once and for all uses.

class RList<+X> { ... }class WList<-X> { ... }

class List<X> extends RList<X>, WList<X>{ ... }

covariant

contravariant

invariant

20Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Definition-Site: Cons

class RList<+X> { ... }class WList<-X> { ... }

class List<X> extends RList<X>, WList<X>{ ... }

covariant

contravariant

invariant

Redundant Types• scala.collection.immutable.Map<A, +B>• scala.collection.mutable.Map<A, B>• Generic with n parameters ⇒ 3n interfaces (or 4n if

bivariance is allowed)

21Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Use-Site: Pros

Flexibility:• co-, contra-, bivariant versions on the fly.

Easier on library designers• Design classes in natural way• No need for fractured classes.

class List<X>{ void add(X x) { … } X get(int i) { … } int size() { … }}

22Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Use-Site Cons:

Burden shifts to users of generics.• Library designers are often users as well.

Type signatures quickly become complicated. Heavy variance annotation required for

subtyping. From Apache Commons-Collections Library:

Iterator<? extends Map.Entry<? extends K, V>> createEntrySetIterator( Iterator<? extends Map.Entry<? extends K, V>>)

23Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Wildcards Criticism

Iterator<? extends Map.Entry<? extends K, V>> createEntrySetIterator( Iterator<? extends Map.Entry<? extends K, V>>)

“We simply cannot afford another wildcards” – Joshua Bloch.

“Simplifying Java Generics by Eliminating Wildcards” – Howard Lovatt.

24Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Our Approach: Take Best of Both Worlds

Take advantages. Remove disadvantages.• Simpler type expressions than in Java

(burden off clients).• Less redundant type definitions in C# and Scala.

Inferring definition-site variance from only use-site annotations. [1]• Added notion of def-site variance without extending

Java.• VarLang Calculus:

• Denotational and language neutral approach.• [1] PLDI 2011 (Altidor, Huang, Smaragdakis)

25Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Fewer Wildcard Annotations

Iterator<Map.Entry<K, V>> createEntrySetIterator( Iterator<Map.Entry<K, V>>)

Iterator<? extends Map.Entry<? extends K, V>> createEntrySetIterator( Iterator<? extends Map.Entry<? extends K, V>>)

26Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

VarLang Calculus Language neutral approach. Simplifies and generalizes all previous work. Sample Applications:

• Infer definition-site variance for Java.• Add use-site variance to C#.

27Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Variance Lattice

Ordering details later

28Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Java to VarLang Calculus

class C<X> { X foo(C<? super X> arg1) { ... } void bar(D<? extends X> arg2) { ... }}class D<Y> { void baz(C<Y> arg3) { ... } }

module C<X> { X+, C<-X>-, void+, D<+X>- }module D<Y> { void+, C<oY>- }

29Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Infer Definition-Site Variance

Def-Site Variance of C<X> = c = +. (C is covariant) Def-Site Variance of D<Y> = –. (D is contravariant)

module C<X> { X+, C<-X>-, void+, D<+X>- }module D<Y> { void+, C<oY>- }

30Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Case Study: Definition-Site Inference for Java How much benefit if Java’s type system inferred

definition-site variance? Mapped Java classes to VarLang modules. Applied inference to large, standard libraries

• e.g., Sun’s JDK 1.6 Example inferences

• java.util.Iterator<E> is covariant.• java.util.Comparator<T> is contravariant.

31Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Sample Results from Case Study

Analysis was conservative (e.g. ignored method bodies).• “foo(List<Animal> arg)” could have been

“foo(List<? extends Animal> arg)”.

majority

32Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Insights into Formal Reasoning

Variance composition:v1 ⊗ v2 = v3

Variance binary predicate:v(T; T’)

Variance lattice:v1 ≤ v2

Variance of a type:var(X; T)

Relating variance to subtyping:Subtype Lifting Lemma

Variance of a position

33Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Standard Modeling: Variance Lattice

Ordered by subtype constraint

binary predicate

34Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Variance of a Type

When is C<Expr1> a subtype of C<Expr2>? What about existential types?

∃X->[ -String].Stack<X>⊥ We answer a more general question:

When is [U/X]T <: [U’/X]T? Key: defined very general predicate:

var(X; T)= variance of type T with respect to type variable X.

35Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Subtype Lifting Lemma

If:(a) v ≤ var(X; T)(b) v(U; U’) Then: [U/X]T <: [U’/X]T

var(X; Iterator<X>) = +and +(Dog; Animal) ≡ Dog <: Animal

implies Iterator<Dog> <: Iterator<Animal>

We generalize Emir et al.’s subtype lifting lemma. Goal property of var.

36Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Variance Composition

Variance of variable X in type A<B<C<X>>>? In general, variance of variable X in type C<E>?

v1 ⊗ v2 = v3. If:

• Variance of variable X in type expression E is v2.

• The def-site variance of class C is v1.

• Then: variance of X in C<E> is v3.

Transform Operator

37Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Deriving Transform OperatorExample Case: + ⊗ – = – Class C is covariant. Type E is contravariant in X. Need to show C<E> is contravariant in X. For any T1, T2:

Hence, C<E> is contravariant in X.

38Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Summary of Transform

Invariance transforms everything into invariance. Bivariance transforms everything into bivariance. Covariance preserves a variance. Contravariance reverses a variance.

39Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Definition of var predicate

Definition-site variance annotations are type checked using var predicate.

For further details:• [2] ECOOP 2012 (Altidor, Reichenbach, Smaragdakis)

40Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Variance composition:v1 ⊗ v2 = v3

Variance binary predicate:v(T; T’)

Variance lattice:v1 ≤ v2

Variance of a type:var(X; T)

Relating variance to subtyping:Subtype Lifting Lemma

Variance of a position: See [2] for derivation.

General Theory – Template for Adding Variance

A<B<C<X>>>

Subtype Policy

Comparing Variances

Computing Variances

Variance Soundness

Check Def-Site Variance

41Taming the Wildcards: Combining Definition- and Use-Site Variance – Altidor

Summary of Contributions

Generics and subtyping coexist fruitfully.• Subtyping between two different instantiations.

Combine def-site and use-site variance to reap their advantages and remove disadvantages.

Generalize all previous related work. Resolve central questions in the design of any

language involving parametric polymorphism and subtyping.• Variance of a type.• Variance of a position.

To my collaborators:• Smaragdakis, Reichenbach, Huang, Palsberg, Tate,

Cameron, Kennedy, Urban To CLC for hosting.