Hoogle+: Program Synthesis by Type-Guided Abstraction Refnement Zheng Guo, Michael James, David Justo, Jiaxiao Zhou, Ziteng Wang, Ranjit Jhala, Nadia Polikarpova University of California San Diego Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet 02 Challenge: Polymorphism Search space explosion 03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet 02 Challenge: Polymorphism Search space explosion 03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Component-Based Synthesis Example Default value List of optional values Desired result ‘a' [ Nothing, Just ‘b', Nothing, Just ‘c', Just ‘d'] ‘b’ 0 [ Nothing, Nothing, Nothing, Nothing ] 0 d: a xs: [Maybe a] a Component-Based Synthesis Example Default value List of optional values Desired result ‘a' [ Nothing, Just ‘b', Nothing, Just ‘c', Just ‘d'] ‘b’ 0 [ Nothing, Nothing, Nothing, Nothing ] 0 d: a xs: [Maybe a] a Component-Based Synthesis Example Here is a DEMO video Component-Based Synthesis Example Solution: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) catMaybes [ Nothing, Just ‘b', Nothing, Just ‘c', Just ‘d'] [ ‘b', ‘c', ‘d' ] ‘a’ fromMaybe ‘b’ listToMaybe Just ‘b’ Component-Based Synthesis Example d: Int -> xs: [Maybe Int] -> Int ? Solution: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet 02 Challenge: Polymorphism Search space explosion 03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Component-Based Synthesis Synthesis to the rescue ?? Type Query Synthesizer Result Program Components Feng et al. POPL ’17 Previous Solution Petri net-Based Search fromMaybe :: a -> Maybe a -> a fromMaybe Types Goal type Components Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search fromMaybe :: a -> Maybe a -> a a fromMaybe Types Goal type Maybe a Components Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search fromMaybe :: a -> Maybe a -> a a fromMaybe Types Goal type Maybe a Components Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a a head [a] fromMaybe listToMaybe catMaybes Types Goal type Maybe a head [Maybe a] Components Tokens Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a a head [a] fromMaybe listToMaybe catMaybes Maybe a head [Maybe a] Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a catMaybes xs a head [a] fromMaybe listToMaybe catMaybes Maybe a head [Maybe a] Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a listToMaybe (catMaybes xs) a head [a] fromMaybe listToMaybe catMaybes Maybe a head [Maybe a] Feng et al. POPL ’17 Previous Solution Petri net-Based Search Type Query d: a -> xs: [Maybe a] -> a SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) a head [a] fromMaybe listToMaybe catMaybes Maybe a head [Maybe a] Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet 02 Challenge: Polymorphism Search space explosion 03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Challenge Polymorphic components a head [a] fromMaybe :: a -> Maybe a -> a fromMaybe listToMaybe catMaybes fromMaybe :: ∀α. α -> Maybe α -> α Maybe a head [Maybe a] Challenge Polymorphic components fromMaybe :: ∀α. α -> Maybe α -> α a fromMaybe … Maybe (Maybe (Maybe a)) fromMaybe Maybe (Maybe a) Maybe a fromMaybe Challenge Polymorphic components Maybe (Maybe a) fromMaybe fromMaybe … Maybe (Maybe (Maybe a)) fromMaybe May head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe [Maybe a] head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Challenge Polymorphic components Maybe (Maybe a) fromMaybe fromMaybe … Maybe (Maybe (Maybe a)) fromMaybe May head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe Infnite[Maybe a]graph! head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet 02 Challenge: Polymorphism Search space explosion 03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Type-Guided Abstraction Refnement Abstract types τ [τ] Int [a] [Maybe a] a [[a]] Maybe τ Maybe a Maybe (Maybe a) Type-Guided Abstraction Refnement Abstract petri net Maybe (Maybe a) fromMaybe fromMaybe … Maybe (Maybe (Maybe a)) fromMaybe May head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe [Maybe a] head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Type-Guided Abstraction Refnement Abstract petri net Maybe (Maybe a) fromMaybe fromMaybe … Maybe (Maybe (Maybe a)) fromMaybe τ head listToMaybe Maybe a Maybe [a] Maybe [[a]] a a listToMaybe [Maybe a] head catMaybes listToMaybe listToMaybe [a] [[a]] head head … [[[a]]] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net head fromMaybe τ / listToMaybe / catMaybes fromMaybe fromMaybe listToMaybe / catMaybes / head a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ head fromMaybe / listToMaybe / catMaybes fromMaybe fromMaybe listToMaybe / catMaybes / head a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ head fromMaybe / listToMaybe / catMaybes fromMaybe fromMaybe listToMaybe catMaybes / head a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ head fromMaybe listToMaybe / catMaybes fromMaybe fromMaybe listToMaybe catMaybes / head a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net SOLUTION: \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ head fromMaybe listToMaybe / catMaybes fromMaybe fromMaybe listToMaybe catMaybes / head a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net τ head fromMaybe / listToMaybe / catMaybes fromMaybe fromMaybe listToMaybe catMaybes / head a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net SOLUTION: \d xs -> fromMaybe d (catMaybes xs) τ head fromMaybe / listToMaybe / catMaybes fromMaybe fromMaybe listToMaybe catMaybes / head a [Maybe a] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net d: Int -> xs: [Maybe Int] -> Int Solution: \d xs -> fromMaybe d (catMaybes xs) Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net Spurious Program: \d xs -> fromMaybe d (catMaybes xs) fromMaybe catMaybes :: . [Maybe ] -> [ ] xs :: [Maybe a] d catMaybes [a] ∀α α α catMaybes xs :: [a] xs [Maybe a] AST of the program Type checking of the program Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Abstract petri net Spurious Program: \d xs -> fromMaybe d (catMaybes xs) fromMaybe ⊥ fromMaybe :: ∀α. α -> Maybe α -> α d catMaybes a [a] Type Error! τ xs [Maybe a] [a] Maybe a AST of the program Type checking of the program Type-Guided Abstraction Refnement TyGAR Workfow Initial Type Check OK Result Abstraction Checker Program Path Query Type Abstract Type Error Reachability Components New No Path Abstraction Refnement Outline 01 Problem: Component-Based Synthesis Running example Previous solution: SyPet 02 Challenge: Polymorphism Search space explosion 03 Solution: Type-Guided Abstraction Refnement Abstraction Refnement Evaluation 04 Hoogle+: More Features Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Type abstraction refnement Spurious Program: \d xs -> fromMaybe d (catMaybes xs) fromMaybe ⊥ fromMaybe :: ∀α. α -> Maybe α -> α d catMaybes a [a] τ [τ] xs [Maybe a] [a] Maybe a AST of the program Type checking of the program Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net listToMaybe listToMaybe fromMaybe /head /head a Spurious Program: fromMaybe fromMaybe \d xs -> fromMaybe d (catMaybes xs) τ listToMaybe fromMaybe catMaybes /head [Maybe a] catMaybes [τ] catMaybes Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net listToMaybe listToMaybe fromMaybe /head /head a Spurious Program: fromMaybe fromMaybe \d xs -> fromMaybe d (catMaybes xs) τ listToMaybe fromMaybe catMaybes /head [Maybe a] catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net listToMaybe listToMaybe fromMaybe /head /head a Spurious Program: fromMaybe fromMaybe \d xs -> fromMaybe d (catMaybes xs) τ listToMaybe fromMaybe catMaybes /head [Maybe a] catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a -> xs: [Maybe a] -> a Refned abstract petri net listToMaybe listToMaybe fromMaybe /head /head a Solution: fromMaybe fromMaybe \d xs -> fromMaybe d (listToMaybe (catMaybes xs)) τ listToMaybe fromMaybe catMaybes /head [Maybe a] catMaybes catMaybes [τ] Type Query Type-Guided Abstraction Refnement d: a