EUC 2015 Sean Cribbs Background Macros Techniques for Metaprogramming in Erlang eunit Parse Tr a n s f o rm s lager Sean Cribbs parse_trans Comcast Cable (T+PD) Syntax Trees erl_syntax @seancribbs Neotoma mochiglobal merl erlydtl Conclusion Erlang User Conference Stockholm 12 June 2015 About me EUC 2015 Sean Cribbs Background Macros eunit Parse Tr a n s f o rm s lager Senior Principal Engineer at Comcast Cable parse_trans Syntax Trees Former Te c h n i c a l L e a d for Riak at Basho erl_syntax Neotoma Creator of neotoma,Erlangpackrat-parser toolkit mochiglobal merl erlydtl Conclusion EUC 2015 Sean Cribbs Background Macros eunit Parse Tr a n s f o rm s lager parse_trans Syntax Trees erl_syntax Background Neotoma mochiglobal merl erlydtl Conclusion Run-time Compile-time What is Metaprogramming? EUC 2015 Sean Cribbs Background Macros eunit Parse Tr a n s f o rm s lager parse_trans Code writing code Syntax Trees Programs as data erl_syntax Neotoma Reflection / reflexivity mochiglobal Homoiconicity merl erlydtl Conclusion Compile-time What is Metaprogramming? EUC 2015 Sean Cribbs Background Macros eunit Parse Tr a n s f o rm s lager parse_trans Code writing code Run-time Syntax Trees Programs as data erl_syntax Neotoma Reflection / reflexivity mochiglobal Homoiconicity merl erlydtl Conclusion What is Metaprogramming? EUC 2015 Sean Cribbs Background Macros eunit Parse Tr a n s f o rm s lager parse_trans Code writing code Run-time Syntax Trees Programs as data Compile-time erl_syntax Neotoma Reflection / reflexivity mochiglobal Homoiconicity merl erlydtl Conclusion Why Metaprogram? EUC 2015 Sean Cribbs Background Macros eunit Parse Reduce duplication Tr a n s f o rm s lager Inject optimization parse_trans Syntax Trees Simplify APIs erl_syntax Neotoma Improve tools mochiglobal merl Implement DSLs erlydtl Conclusion Metaprogramming Erlang EUC 2015 Sean Cribbs Background Source substitute Macros eunit Parse tokenize Tr a n s f o rm s lager parse_trans Tokens rewrite pretty-print Syntax Trees erl_syntax Neotoma parse mochiglobal merl erlydtl Abstract form transform Conclusion compile evaluate Bytecode load Runtime EUC 2015 Sean Cribbs Background Macros eunit Parse Tr a n s f o rm s lager parse_trans Te c h n i q u e 1 Syntax Trees erl_syntax Neotoma mochiglobal Macros merl erlydtl Conclusion Macros EUC 2015 Sean Cribbs Generates code in preprocessor (epp) Background Operates over Tokens (mostly) Macros eunit % static term Parse ⌥ ⌅ Tr a n s f o rm s -define(TIMEOUT,5000). lager parse_trans Syntax Trees ⌃ ⇧ erl_syntax Neotoma mochiglobal merl erlydtl Conclusion Macros EUC 2015 Sean Cribbs Generates code in preprocessor (epp) Background Operates over Tokens (mostly) Macros eunit % static term Parse ⌥ ⌅ Tr a n s f o rm s -define(TIMEOUT,5000). lager parse_trans % parameterized Syntax Trees -define(THUNK(A), fun() -> (A) end). erl_syntax -define(IF(B,T,F), Neotoma mochiglobal begin merl ( (B) true (T); false (F) ) erlydtl case of -> -> end Conclusion end). ⌃ ⇧ Macros EUC 2015 Sean Cribbs Generates code in preprocessor (epp) Background Operates over Tokens (mostly) Macros eunit % static term Parse ⌥ ⌅ Tr a n s f o rm s -define(TIMEOUT,5000). lager parse_trans % parameterized Syntax Trees -define(THUNK(A), fun() -> (A) end). erl_syntax -define(IF(B,T,F), Neotoma mochiglobal begin merl ( (B) true (T); false (F) ) erlydtl case of -> -> end Conclusion end). %% escaped arguments -define(Quote(A), io_lib:format("~s",[??A])). ⌃ ⇧ ⌥Nope =?THUNK(launch(missiles)). ⌅ %% Nope = fun() -> (launch(missiles)) end. ⌃ ⇧ ⌥io:format("The value of ~s is ~p.",[?Quote(Foo), Foo]). ⌅ %% io:format("The value of ~s is ~p.",["Foo", Foo]). ⌃ ⇧ Using Macros EUC 2015 Sean Cribbs Background Macros eunit ⌥gen_server:call(?MODULE, ping,?TIMEOUT). ⌅ Parse %% gen_server:call(mymodule, ping, 5000). Tr a n s f o rm s lager parse_trans ⌃ ⇧ Syntax Trees erl_syntax Neotoma mochiglobal merl erlydtl Conclusion ⌥io:format("The value of ~s is ~p.",[?Quote(Foo), Foo]). ⌅ %% io:format("The value of ~s is ~p.",["Foo", Foo]). ⌃ ⇧ Using Macros EUC 2015 Sean Cribbs Background Macros eunit ⌥gen_server:call(?MODULE, ping,?TIMEOUT). ⌅ Parse %% gen_server:call(mymodule, ping, 5000). Tr a n s f o rm s lager parse_trans ⌃ ⇧ Nope =?THUNK(launch(missiles)). Syntax Trees ⌥ ⌅ erl_syntax %% Nope = fun() -> (launch(missiles)) end. Neotoma mochiglobal merl ⌃ ⇧ erlydtl Conclusion Using Macros EUC 2015 Sean Cribbs Background Macros eunit ⌥gen_server:call(?MODULE, ping,?TIMEOUT). ⌅ Parse %% gen_server:call(mymodule, ping, 5000). Tr a n s f o rm s lager parse_trans ⌃ ⇧ Nope =?THUNK(launch(missiles)). Syntax Trees ⌥ ⌅ erl_syntax %% Nope = fun() -> (launch(missiles)) end. Neotoma mochiglobal merl ⌃ ⇧ io:format("The value of ~s is ~p.",[?Quote(Foo), Foo]). erlydtl ⌥ ⌅ Conclusion %% io:format("The value of ~s is ~p.",["Foo", Foo]). ⌃ ⇧ Macros - eunit EUC 2015 Sean Cribbs ⌥-define(assert(BoolExpr), ⌅ Background begin Macros ((fun () -> eunit Parse Tr a n s f o rm s lager parse_trans Syntax Trees erl_syntax Neotoma mochiglobal merl erlydtl Conclusion end end)()) end). ⌃ ⇧ Macros - eunit EUC 2015 Sean Cribbs ⌥-define(assert(BoolExpr), ⌅ Background begin Macros ((fun () -> eunit case (BoolExpr) of Parse true -> ok; Tr a n s f o rm s lager parse_trans Syntax Trees erl_syntax Neotoma mochiglobal merl erlydtl Conclusion end end)()) end). ⌃ ⇧ Macros - eunit EUC 2015 Sean Cribbs ⌥-define(assert(BoolExpr), ⌅ Background begin Macros ((fun () -> eunit case (BoolExpr) of Parse true -> ok; Tr a n s f o rm s lager __V -> erlang:error({assertion_failed, parse_trans [{module,?MODULE}, Syntax Trees {line,?LINE}, erl_syntax Neotoma {expression,(??BoolExpr)}, mochiglobal merl {expected, true}, erlydtl {value, case __V of false -> __V; Conclusion _ -> {not_a_boolean,__V} end}]}) end end)()) end). ⌃ ⇧ 1> eunit:test(mymodule). mymodule: fizzbuzz_test (module ’mymodule’)...*failed* in function mymodule:’-fizzbuzz_test/0-fun-3-’/0 (mymodule.erl, line 18) **error:{assertion_failed,[{module,mymodule}, {line,18}, {expression,"10 =:= fizzbuzz ( 10 )"}, {expected,true}, {value,false}]} Macros - eunit EUC 2015 fizzbuzz_test() -> Sean Cribbs ⌥ ⌅ ?assert(fizz =:= fizzbuzz(3)), Background ?assert(buzz =:= fizzbuzz(5)), Macros ?assert(fizzbuzz =:= fizzbuzz(15)), eunit ?assert(10 =:= fizzbuzz(10)). Parse Tr a n s f o rm s lager parse_trans ⌃ ⇧ Syntax Trees erl_syntax Neotoma mochiglobal merl erlydtl Conclusion Macros - eunit EUC 2015 fizzbuzz_test() -> Sean Cribbs ⌥ ⌅ ?assert(fizz =:= fizzbuzz(3)), Background ?assert(buzz =:= fizzbuzz(5)), Macros ?assert(fizzbuzz =:= fizzbuzz(15)), eunit ?assert(10 =:= fizzbuzz(10)). Parse Tr a n s f o rm s lager parse_trans ⌃ ⇧ Syntax Trees 1> eunit:test(mymodule). erl_syntax mymodule: fizzbuzz_test (module ’mymodule’)...*failed* Neotoma mochiglobal in function mymodule:’-fizzbuzz_test/0-fun-3-’/0 (mymodule.erl, line 18) merl erlydtl **error:{assertion_failed,[{module,mymodule}, Conclusion {line,18}, {expression,"10 =:= fizzbuzz ( 10 )"}, {expected,true}, {value,false}]} Cons: Limited expressivity Appearance Good for: Small API wrappers like in eunit or eqc Naming constants Compile-time feature-switching (OTP upgrades) Debugging statements Macros - Summary EUC 2015 Sean Cribbs Background Pros: Macros Easy and familiar eunit Parse Inline with program Tr a n s f o rm s lager Syntax draws attention parse_trans Syntax Trees erl_syntax Neotoma mochiglobal merl erlydtl Conclusion Good for: Small API wrappers like in eunit or eqc Naming constants Compile-time feature-switching (OTP upgrades) Debugging statements Macros - Summary EUC 2015 Sean Cribbs Background Pros: Cons: Macros Easy and familiar Limited expressivity eunit Parse Inline with program Appearance Tr a n s f o rm s lager Syntax draws attention parse_trans Syntax Trees erl_syntax Neotoma mochiglobal merl erlydtl Conclusion Macros - Summary EUC 2015 Sean Cribbs Background Pros: Cons: Macros Easy and familiar Limited expressivity eunit Parse Inline with program Appearance Tr a n s f o rm s lager Syntax draws attention parse_trans Syntax Trees erl_syntax Neotoma Good for: mochiglobal merl Small API wrappers like in eunit or eqc erlydtl Conclusion Naming constants Compile-time feature-switching (OTP upgrades) Debugging statements EUC 2015 Sean Cribbs Background Macros eunit Parse Tr a n s f o rm s lager parse_trans Te c h n i q u e 2 Syntax Trees erl_syntax Neotoma mochiglobal Parse Transforms merl erlydtl Conclusion $erlc-Pmymodule.erl $catmymodule.P Parse Transforms EUC 2015 Sean Cribbs Generates or transforms code after parsing Background Operates over Abstract Form (AST) Macros eunit %% In your module: Parse ⌥ ⌅ Tr a n s f o rm s -compile([{parse_transform, the_transform_module}]). lager parse_trans Syntax Trees %% In the parse transform module: erl_syntax parse_transform(Forms, _Options) -> Neotoma mochiglobal %% ’Forms’ is the AST. ’Options’ are the compiler options. merl %% Traverse/modify ’Forms’ and return it erlydtl Forms. Conclusion ⌃ ⇧ Parse Transforms EUC 2015 Sean Cribbs Generates or transforms code after parsing Background Operates over Abstract Form (AST) Macros eunit %% In your module: Parse ⌥ ⌅ Tr a n s f o rm s -compile([{parse_transform, the_transform_module}]). lager parse_trans Syntax Trees %% In the parse transform module: erl_syntax parse_transform(Forms, _Options) -> Neotoma mochiglobal %% ’Forms’ is the AST. ’Options’ are the compiler options. merl %% Traverse/modify ’Forms’ and return it erlydtl Forms. Conclusion ⌃ ⇧ $erlc-Pmymodule.erl $catmymodule.P Parse Transforms - lager github.com/basho/lager EUC 2015 Sean Cribbs Rewrites calls