Hoogle+: Program Synthesis by Type-Guided Abstraction Refnement

Zheng Guo, Michael James, David Justo, Jiaxiao Zhou, Ziteng Wang, Ranjit Jhala, Nadia Polikarpova University of California San Diego Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Component-Based Synthesis Example

Default value List of optional values Desired result

‘a' [ Nothing, Just ‘b', Nothing, Just ‘c', Just ‘d'] ‘b’

0 [ Nothing, Nothing, Nothing, Nothing ] 0

d: a xs: [Maybe a] a Component-Based Synthesis Example

Default value List of optional values Desired result

‘a' [ Nothing, Just ‘b', Nothing, Just ‘c', Just ‘d'] ‘b’

0 [ Nothing, Nothing, Nothing, Nothing ] 0

d: a xs: [Maybe a] a Component-Based Synthesis Example

Here is a DEMO video Component-Based Synthesis Example

Solution: \d xs -> fromMaybe d (listToMaybe (catMaybes xs))

catMaybes [ Nothing, Just ‘b', Nothing, Just ‘c', Just ‘d'] [ ‘b', ‘c', ‘d' ]

‘a’ fromMaybe ‘b’ listToMaybe Just ‘b’ Component-Based Synthesis Example

d: Int -> xs: [Maybe Int] -> Int ?

Solution: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Component-Based Synthesis Synthesis to the rescue

??

Type Query

Synthesizer Result Program

Components Feng et al. POPL ’17 Previous Solution Petri net-Based Search

fromMaybe :: a -> Maybe a -> a

fromMaybe Types

Goal type

Components

Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search

fromMaybe :: a -> Maybe a -> a a

fromMaybe Types

Goal type

Maybe a Components

Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search

fromMaybe :: a -> Maybe a -> a a

fromMaybe Types

Goal type

Maybe a Components

Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a

a head [a]

fromMaybe listToMaybe catMaybes Types

Goal type

Maybe a head [Maybe a] Components

Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a

a head [a]

fromMaybe listToMaybe catMaybes

Maybe a head [Maybe a] Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a

catMaybes xs

a head [a]

fromMaybe listToMaybe catMaybes

Maybe a head [Maybe a] Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a

listToMaybe (catMaybes xs)

a head [a]

fromMaybe listToMaybe catMaybes

Maybe a head [Maybe a] Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a

SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs))

a head [a]

fromMaybe listToMaybe catMaybes

Maybe a head [Maybe a] Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Challenge Polymorphic components

a head [a] fromMaybe :: a -> Maybe a -> a

fromMaybe listToMaybe catMaybes fromMaybe :: ∀α. α -> Maybe α -> α

Maybe a head [Maybe a] Challenge Polymorphic components

fromMaybe :: ∀α. α -> Maybe α -> α

a fromMaybe … Maybe (Maybe (Maybe a)) fromMaybe Maybe (Maybe a)

Maybe a fromMaybe Challenge Polymorphic components Maybe (Maybe a)

fromMaybe fromMaybe …

Maybe (Maybe (Maybe a))

fromMaybe May head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe [Maybe a]

head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Challenge Polymorphic components Maybe (Maybe a)

fromMaybe fromMaybe …

Maybe (Maybe (Maybe a))

fromMaybe May head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe Infnite[Maybe a]graph!

head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Type-Guided Abstraction Refnement Abstract types

τ [τ] Int [a] [Maybe a] a [[a]]

Maybe τ Maybe a Maybe (Maybe a) Type-Guided Abstraction Refnement Abstract petri net Maybe (Maybe a)

fromMaybe fromMaybe …

Maybe (Maybe (Maybe a))

fromMaybe May head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe [Maybe a]

head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Type-Guided Abstraction Refnement Abstract petri net Maybe (Maybe a)

fromMaybe fromMaybe …

Maybe (Maybe (Maybe a)) fromMaybe τ head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe [Maybe a]

head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

head fromMaybe τ / listToMaybe / catMaybes

fromMaybe fromMaybe listToMaybe / catMaybes / head

a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs))

τ head fromMaybe / listToMaybe / catMaybes

fromMaybe fromMaybe listToMaybe / catMaybes / head

a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs))

τ head fromMaybe / listToMaybe / catMaybes

fromMaybe fromMaybe listToMaybe catMaybes / head

a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs))

τ head fromMaybe listToMaybe / catMaybes

fromMaybe fromMaybe listToMaybe catMaybes / head

a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs))

τ head fromMaybe listToMaybe / catMaybes

fromMaybe fromMaybe listToMaybe catMaybes / head

a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

τ head fromMaybe / listToMaybe / catMaybes

fromMaybe fromMaybe listToMaybe catMaybes / head

a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

SOLUTION: \d xs -> fromMaybe d (catMaybes xs)

τ head fromMaybe / listToMaybe / catMaybes

fromMaybe fromMaybe listToMaybe catMaybes / head

a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

d: Int -> xs: [Maybe Int] -> Int

Solution: \d xs -> fromMaybe d (catMaybes xs) Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

Spurious Program: \d xs -> fromMaybe d (catMaybes xs)

fromMaybe

catMaybes :: . [Maybe ] -> [ ] xs :: [Maybe a] d catMaybes [a] ∀α α α catMaybes xs :: [a]

xs [Maybe a]

AST of the program Type checking of the program Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net

Spurious Program: \d xs -> fromMaybe d (catMaybes xs)

fromMaybe ⊥ fromMaybe :: ∀α. α -> Maybe α -> α d catMaybes a [a] Type Error! τ

xs [Maybe a] [a] Maybe a

AST of the program Type checking of the program Type-Guided Abstraction Refnement TyGAR Workfow

Initial Type Check OK Result Abstraction Checker Program Path Query Type Abstract Type Error Reachability Components New No Path Abstraction Refnement Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Type abstraction refnement

Spurious Program: \d xs -> fromMaybe d (catMaybes xs)

fromMaybe ⊥ fromMaybe :: ∀α. α -> Maybe α -> α d catMaybes a [a] τ [τ]

xs [Maybe a] [a] Maybe a

AST of the program Type checking of the program Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net

listToMaybe listToMaybe fromMaybe /head /head a

Spurious Program: fromMaybe fromMaybe \d xs -> fromMaybe d (catMaybes xs) τ listToMaybe fromMaybe catMaybes /head [Maybe a]

catMaybes [τ] catMaybes Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net

listToMaybe listToMaybe fromMaybe /head /head a

Spurious Program: fromMaybe fromMaybe \d xs -> fromMaybe d (catMaybes xs) τ listToMaybe fromMaybe catMaybes /head [Maybe a]

catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net

listToMaybe listToMaybe fromMaybe /head /head a

Spurious Program: fromMaybe fromMaybe \d xs -> fromMaybe d (catMaybes xs) τ listToMaybe fromMaybe catMaybes /head [Maybe a]

catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net

listToMaybe listToMaybe fromMaybe /head /head a

Solution: fromMaybe fromMaybe \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ listToMaybe fromMaybe catMaybes /head [Maybe a]

catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net

listToMaybe listToMaybe fromMaybe /head /head a

Solution: fromMaybe fromMaybe \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ listToMaybe fromMaybe catMaybes /head [Maybe a]

catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net

listToMaybe listToMaybe fromMaybe /head /head a

Solution: fromMaybe fromMaybe \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ

fromMaybe listToMaybe catMaybes

[Maybe a]

catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net

listToMaybe listToMaybe fromMaybe /head /head a

Solution: fromMaybe fromMaybe \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ

fromMaybe listToMaybe catMaybes

[Maybe a]

catMaybes catMaybes [τ] Type-Guided Abstraction Refnement TyGAR Workfow

Initial Type Check OK Result Abstraction Checker Program Path Query Type Abstract Type Error Reachability Components New No Path Abstraction Refnement Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Type-Guided Abstraction Refnement Evaluation

Components Benchmarks

291 components 24 benchmarks from Hoogle 012 popular Haskell library modules 06 benchmarks from StackOverfow 14 benchmarks curated by us Type-Guided Abstraction Refnement Evaluation

50 44 40 Too many refnements Too large petri net 30 34

20 # of benchmarks

10

0 Abstraction w/ Refnement Type-Guided Abstraction Refnement Evaluation

50 44 40 37 Too many refnements Too large petri net 30 34 No refnement Poor at hard queries

20 # of benchmarks

10 Query: f: (a -> b) -> g: (a -> c) -> x: a -> (b, c) Solution: \f g x -> (f x, g x) 0 Abstraction w/ Abstraction w/o TIMEOUT! Refnement Refnement Type-Guided Abstraction Refnement Evaluation

50 44 40 43 37 Too many refnements Too large petri net 34 30 No refnement Poor at hard queries

20 # of benchmarks

10 Query: f: (a -> b) -> g: (a -> c) -> x: a -> (b, c) Solution: \f g x -> (f x, g x) 0 Abstraction w/ Abstraction w/o Abstraction w Refnement Refnement Bounded Refnement Bounded refnements 2s! Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Hoogle+ Support for real-world Haskell

More advanced Haskell features

higher-order functions, type classes, etc.

Filter out uninteresting solutions

e.g. \d xs -> fromLeft d (Right xs) always returns d Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet

02 Challenge: Polymorphism Search space explosion

03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features