Silverchain: A Fluent API Generator Tomoki Nakamaru Kazuhiro Ichikawa The University of Tokyo The University of Tokyo Japan Japan [email protected] [email protected] Tetsuro Yamazaki Shigeru Chiba The University of Tokyo The University of Tokyo Japan Japan [email protected] [email protected]
Abstract fluent APIs such as Hamcrest15 [ ] and jMock [21]. Method This paper presents a tool named Silverchain, which gener- chaining is a coding style that concatenates consecutive ates class definitions for a fluent API from the grammar of method calls together like a chain. By method chaining, pro- the API. A fluent API is an API that is used by method chain- grammers can group semantically related method calls into ing and its grammar is a BNF-like set of rules that defines a single expression and keep code easy to understand. Fluent method chains accepted in type checking. Fluent APIs gen- APIs are a popular way of designing embedded domain- erated by Silverchain provide two styles of APIs: One is for specific languages (EDSLs), a form of DSLs that is imple- building a chain by concatenating all method calls in series. mented as a library in a general-purpose host language [17]. The other is for building a chain from partial chains by pass- Using a fluent EDSL, DSL sentences can be emulated well ing child chains to method calls in the parent chain as their without any change of the host language. For instance, Java arguments. To generate such a fluent API, Silverchain first programmers can compose an SQL query with jOOQ [12] as translates given grammar into a set of deterministic push- follows: down automata without ϵ-transitions, then encodes these //SELECT*FROM book //WHERE book.published_in= 2016; automata into class definitions. Each constructed automata String query = create.select() corresponds to a nonterminal in given grammar and recog- .from(BOOK) .where(BOOK.PUBLISHED_IN.eq(2016)).getSQL(); nizes symbol sequences produced from its corresponding nonterminal. Fluent APIs usually support subchaining style, which is a style of method chaining that passes a chain to a method call CCS Concepts • Software and its engineering → Do- in another chain as its argument. In the case of the example main specific languages; Source code generation; above, subchaining style is used in the where clause. This Keywords Fluent API, Embedded DSL feature improves the usability of fluent APIs in that program- mers can build an entire chain from semantically grouped ACM Reference Format: Tomoki Nakamaru, Kazuhiro Ichikawa, Tetsuro Yamazaki, and Shigeru partial chains. Note that subchaining style is effective for a Chiba. 2017. Silverchain: A Fluent API Generator. In Proceedings long or complex chain but not for a small or simple chain. of 16th ACM SIGPLAN International Conference on Generative Pro- Non-subchaining style, a style that concatenates all method gramming: Concepts and Experiences (GPCE’17). ACM, New York, calls in series, is useful when building a small or simple chain. NY, USA, 13 pages. https://doi.org/10.1145/3136040.3136041 To improve the usability further, fluent APIs are frequently designed to be type-safe by setting the return type of each 1 Introduction method based on which methods can be written next [2, 10]. A fluent API [10] is an API that is used by method chaining The users of a type-safe fluent API can find misuse easily and a number of well-known class libraries provide their from thrown type errors and autocompletion can provide necessary and sufficient candidate methods to the users. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not Creating a well-designed fluent API, however, is not as made or distributed for profit or commercial advantage and that copies bear easy as using it. To make a fluent API type-safe, API devel- this notice and the full citation on the first page. Copyrights for components opers need to define a lot of classes and choose the return of this work owned by others than ACM must be honored. Abstracting with type of each method carefully. Further, each class definition credit is permitted. To copy otherwise, or republish, to post on servers or to gets complicated when making fluent APIs support both redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. subchaining and non-subchaining style. In fact, manually GPCE’17, October 23–24, 2017, Vancouver, Canada written fluent APIs often fail to follow their expected rules © 2017 Association for Computing Machinery. perfectly. The users of such incomplete APIs accidentally ACM ISBN 978-1-4503-5524-7/17/10...$15.00 https://doi.org/10.1145/3136040.3136041
199 GPCE’17, October 23–24, 2017, Vancouver, Canada T. Nakamaru, K. Ichikawa, T. Yamazaki, and S. Chiba build a chain causing runtime errors, and autocompletion With a fluent API, DSL sentences are embedded without any provides unnecessary or insufficient candidates to the users. change of the host language while those embedded sentences In this paper, we present a tool named Silverchain for gen- maintain domain-specific syntax. When using a non-fluent erating type-safe fluent APIs that support both subchaining API, the document is emulated in a different way as follows: and non-subchaining style. The input to Silverchain is the IDoc idoc = new IDoc (); grammar of a target fluent API, a BNF-like set of rules that idoc.addItem(new Item ( " Item 1 " )); Item item2 = new Item ( " Item 2 " ); defines method chains accepted in type checking. From given item2.addItem(new Item ( " Item 2 . 1 " )); grammar, Silverchain first constructs a set of deterministic item2.addItem(new Item ( " Item 2 . 2 " )); idoc.addItem(item2); pushdown automata without ϵ-transitions (called real-time String tex = idoc.toTeX(); deterministic pushdown automaton, or RDPDA for short), Without method chaining, code tends to contain meaning- each of which recognizes symbol sequences produced from less intermediate variables and does not resemble original its corresponding nonterminal in the grammar. These con- DSL sentences. String encoding and syntax extension are also structed RDPDAs are then encoded into class definitions for techniques to embed DSL sentences into Java source code. By the target fluent API. Silverchain can generate a fluent API string encoding, DSL sentences are directly written as string for any context-free grammar (CFG) but cannot generate literals in Java. Those sentences maintain domain-specific an API that fully supports non-subchaining style for some syntax but are often unreadable especially when they are kinds of grammars. This limitation is due to our construction long or contain escape sequences. Furthermore, no syntax method of RDPDAs. Silverchain differs from other generators checking is applied to DSL sentences at compile-time. By (or generating algorithms) in that generated APIs support extending Java syntax, programmers can compose a DSL subchaining and non-subchaining styles. We describe those sentence in Java as it is but introducing a syntax extension other generators later in Section 5. mechanism is still a research topic [9, 19]. Fluent APIs can be activated easily by importing them since they are just 2 Motivation libraries. As a motivating example, consider creating a Java fluent From the viewpoint of usability, it is essential to support API for IDoc, a small DSL for writing itemized documents. both subchaining and non-subchaining style in one fluent The following code shows an example use of IDoc and its API. Without subchaining, the example document is emu- lated as follows: translation into TEX: String tex = idoc() begin . begin () " Item 1 " . text ( " Item 1 " ) " Item 2 " . text ( " Item 2 " ) begin . begin ()// Nesting structure is flattened. " Item 2 . 1 " . text ( " Item 2 . 1 " ) " Item 2 . 2 " . text ( " Item 2 . 2 " ) end .end () end .end().toTeX(); // Translation to TeX \ begin { itemize }% Just"begin" in IDoc The readability gets worse as the length or the complexity \item Item 1 of a chain increases if a fluent API does not support sub- \item Item 2% Quoted string in IDoc \ begin { itemize } chaining style. The readability gets better by indentation \item Item 2.1 that reflects original nesting structures of the sentence, but \item Item 2.2 \end{ itemize }% Just"end" in IDoc such indentation is not naturally obtained with support from \end{ itemize } an integrated development environment (IDE). Subchain- In IDoc, itemized documents can be written more simply ing style is also effective when changing a part of a chain since IDoc is specialized to itemized documents. dynamically as follows: Using a Java fluent API for IDoc, programmers can com- List list1 = begin() ... .end(); List list2 = begin() ... .end(); pose an itemized document in Java as if they write it in IDoc. String tex1 = idoc() For instance, the IDoc sentence above is emulated in Java by . begin () . text ( " Item " ) method chaining as follows: .list(bool? list1 : list2) .end().toTeX(); String tex = idoc() . begin () // If only non-subchaining style is supported. . text ( " Item 1 " ) IncompChain1 c1 = idoc().begin().text( " Item " ); . text ( ) " Item 2 " IncompChain2 c2; .list(begin() i f ( bool ) { . text ( ) " Item 2 . 1 " c2 = c1.begin() ... .end();// list1 . text ( " Item 2 . 2 " ) } e l s e { .end ()) c2 = c1.begin() ... .end();// list2 .end().toTeX(); }
200 Silverchain: A Fluent API Generator GPCE’17, October 23–24, 2017, Vancouver, Canada
String tex2 = c2.end().toTeX(); 1 idoc -> list* ; 2 list -> " b e g i n " item + " end " ; When subchaining style is supported, the users can build a 3 item -> text list? ; 4 chain by combining the base chain and semantically grouped 5 START = idoc ; subchains according to satisfied conditions. On the other 6 text : STRING ; hand, when subchaining is not supported, intermediate vari- ables get meaningless and the users cannot make good use of Figure 1. The grammar of the fluent API for IDoc method chaining. Non-subchaining style, however, is also im- portant as well as subchaining style. Non-subchaining style can reduce redundant parts in a chain while maintaining the readability if a chain is simple enough: The incomplete type-safety not only is unsafe but also worsen the user experiences in that autocompletion suggests unnec- // Simple chain in non-subchaining style String tex1 = idoc() essary or insufficient methods for hints. .begin().text( " Item " ).end().toTeX(); // If only subchaining style is supported. 3 Silverchain String tex2 = idoc() .list(begin().item(text( " Item " )).end()).toTeX(); The input to Silverchain is the grammar of a fluent API As seen from the above, which style should be used de- written in a BNF-like DSL such as shown in Figure1, which pends on the situation and two styles are often mixed in is the grammar of a fluent API for IDoc. In the grammar, a a chain. Fluent APIs with only subchaining support (or only valid chain of methods (a chain accepted in type checking) non-subchaining support) prevent their users from utilizing is defined by a set of production rules. Each production rule method chaining effectively. specifies a valid method chain that begins with the method Fluent APIs achieve safety by setting the return type of corresponding to the left-hand side of the rule. For example, each method based on which methods can be written next. three production rules in Line 1 to 3 in Figure1 specify that The users of such type-safe fluent APIs can find misuse easily the following chains in subchaining style are accepted (or from thrown type errors: rejected) by a type checker: //"..." representsa subchain. // Missing"end()" call causesa type error. // Line1 String tex = idoc() idoc().list(...).list(...);//OK . begin () . text ( )// Returnsa type that " Item " // Line2 // does not have"toTeX()". list().begin().item(...).end();//OK .toTeX(); list().begin().end();// Error
The type-safety also enhances the usability in that autocom- // Line3 pletion of an IDE can provide necessary and sufficient hints. item().text( " Item " ).list(...);//OK The users can build correct chains easily just by following item ().list(...);// Error provided hints and do not have to pay attention to accidental Those rules also specify that the following chains in non- runtime errors. In other words, an existing IDE for a host lan- subchaining style are accepted (or rejected): guage serves as an IDE for the DSL if a fluent API is type-safe. // Line1 and Line2 Here, type-safety indicates the syntactic correctness from idoc () .begin().item(...).end() the perspective of method arrangement, not of the semantics .begin().item(...).end();;//OK of written chains. // Line2 and Line3 Although well-designed fluent APIs have many advan- list().begin().text( " Item " ).end ();//OK tages, creating such APIs is a tedious task. API developers list().begin().list(...);// Error need to define a lot of classes and connect them appropriately // Line 1, Line 2, and Line3 by choosing the return type of each method. Moreover, the idoc () .begin().text( " Item 1 " ).end () developers need to repeat this tedious task for each language .begin().text( " Item 2 " ).end ();//OK in which they provide the API. In fact, many well-known Line 5 and Line 6 are supplementary rules. The rule in Line fluent APIs fail to follow their desired rules perfectly. For 5 specifies that only a chain beginning with idoc() can be instance, the following chains written with popular fluent a top-level chain. A top-level chain can be terminated by APIs are accepted by Java type checker but those should eval() as follows: not be accepted from the viewpoint of the validity of DSL idoc().list(...).eval();//OK sentences: list().begin().item(...).end().eval();// Error //
with j2html item().text( " Item " ).eval();// Error div().withSrc( " ./image.png" ).render(); Here, eval() is a method call to evaluate (or materialize) a //"SELECT*ORDERBY book.id" with jOOQ chain. The rule in Line 6 specifies that the symbol text is select().orderBy(BOOK.id).getSQL(); translated into a method call taking a string as its argument.201 GPCE’17, October 23–24, 2017, Vancouver, Canada T. Nakamaru, K. Ichikawa, T. Yamazaki, and S. Chiba
list // Class grouping the classes related to idoc. c l a s s Idoc { void eval () {} start D0 // Groups static methods to begina chain. c l a s s StartingMethods { (a) For idoc s t a t i c State0 idoc() { ... } } item // An accepting state is encoded // intoa subclass of Idoc. State0 Idoc { begin c l a s s extends item end // Takesa subchain for list. start L0 L1 L2 L3 State0 list(List l) { ... } } (b) For list }
c l a s s List { text list c l a s s StartingMethods { start I0 I1 I2 s t a t i c State0 list() { ... } } c l a s s State0 { (c) For item State1 begin() { ... } } c l a s s State1 { Figure 2. DFAs for nonterminals in Figure1 State2 item(Item i) { ... } } c l a s s State2 { State2 item(Item i) { ... } Our translation from grammar to a fluent API consists State3 end() { ... } } of four steps. First three steps are for constructing a set c l a s s State3 extends List { of real-time deterministic pushdown automata (RDPDAs). } } Each RDPDA corresponds to a nonterminal in given gram- mar and recognizes all symbol sequences produced from c l a s s Item { c l a s s StartingMethods { its corresponding nonterminal. To construct such RDPDAs, s t a t i c State0 item() { ... } we construct a set of DFAs from given grammar and then } c l a s s State0 { incrementally extend those DFAs to RDPDAs. At the final State1 text(String t) { ... } step, we encode constructed RDPDAs into class definitions. } c l a s s State1 extends Item { Silverchain can generate a fluent API for any context-free State2 list(List l) { ... } grammar but, for some kinds of grammars, cannot generate } c l a s s State2 extends Item { an API that fully supports non-subchaining style. In these } cases, Silverchain generates a fluent API that requires sub- } chaining style to build certain parts of a chain. This limitation is due to the third step of our translation and it is discussed Figure 3. Java classes generated by encoding Figure2 in Section 3.3.
3.1 Constructing DFAs from Production Rules Figure2 by this encoding scheme. (The encoding scheme is described again in Section 3.4.) The users of the API defined At the first step, Silverchain constructs a DFA for each non- in Figure3 can compose an itemized document by method terminal in given grammar by converting the right-hand chaining as follows: side of the production rule for the nonterminal. In the case // Subchaining style of the grammar in Figure1, Silverchain constructs three idoc().list( DFAs in Figure2 at this step. Each DFA is constructed so list().begin().item( item().text( " Item " ) as to recognize symbol sequences produced directly from ).end () the corresponding nonterminal. For example, the DFA for ). eval ();// begin"Item" end list recognizes the sequence “begin item item end” as seen from Figure 2b. Since the right-hand side of a production 3.2 Extending DFAs rule is a regular expression consisting of terminals and non- The fluent API obtained at the previous step does not support terminals, such a DFA can be constructed from a production non-subchaining style as follows: rule by well-known algorithms (e.g. Brzozowski’s [4] and // Partially non-subchaining style Thompson’s [28]). idoc().list( By encoding each state of DFAs into a class and each list().begin().text( " Item " ).end()).eval(); transition to a method, a type-safe fluent API is obtained To support method chaining above, Silverchain extends the though it only supports subchaining style. For example, the DFAs that were constructed at the previous step. A DFA Java classes in Figure3 are generated from the DFAs in is extended so as to recognize symbol sequences produced
202 Silverchain: A Fluent API Generator GPCE’17, October 23–24, 2017, Vancouver, Canada
text idoc list item
L2 text end Figure 6. The initial reference relation graph for Figure1
begin list start L0 L1 text or L4 item item end idoc list item idoc list item
L 3 (b) Examining (list, item). (a) Examining (idoc, list). This edge is selected since the This edge is not selected since item SCC containing item consists the SCC containing list only of one node without a consists of two nodes. Figure 4. New DFA for list self-loop. c l a s s List { Figure 7. Selecting an edge for inline expansion c l a s s StartingMethods { s t a t i c State0 list() { ... } } c l a s s State0 { State1 begin() { ... } } idoc list item c l a s s State1 { State2 text(String t) { ... } State3 item(Item i) { ... } } Figure 8. The updated graph after expanding item in the c l a s s State2 { State2 text(String t) { ... } rule for list State3 list(List l) { ... } State3 item(Item i) { ... } State4 end() { ... } } c l a s s State3 { State2 text(String t) { ... } idoc list item idoc list item State3 item(Item i) { ... } State4 end() { ... } } (a) Expanding (idoc, list). (b) Expanding (idoc, item). c l a s s State4 extends List { } } Figure 9. Without decomposition into SCCs
Figure 5. Java classes generated by encoding Figure4 // Before expansion list -> " b e g i n " item+ " end " ; not only directly but also indirectly from the corresponding // After expansion nonterminal. For example, the DFA for list (Figure 2b) is list -> " b e g i n " (item | (text list?))+ " end " ; extended as shown in Figure4 at this step. As seen from A nonterminal on the right-hand side of a production rule Figure4, the extended DFA recognizes sequences produced is replaced by the union of that nonterminal and its directly indirectly from list such as “begin text end” while it also producing sequence. In fact, Figure4 is constructed from recognizes sequences produced directly from list such as the expanded rule above. Note that the inline expansion “begin item end”. The fluent API generated from the extended is applied to nonterminals only when they have not been DFAs supports non-subchaining style. The classes in Figure expanded yet. Redundant inline expansion of nonterminals 5 are generated from the DFA in Figure4 and they allow the does not extend the corresponding DFA. following code: To avoid infinite regression caused by inline expansion, // Newly supported style by the extension Silverchain selects a nonterminal to be expanded as follows. List l1 = list().begin().text( " Item " ).end (); It first examines reference relations among nonterminals, // Subchaining style is also supported. where a reference from a nonterminal ns to a nonterminal List l2 = list().begin().item(...).end(); nd represents that the rule for ns contains one or more un- The extension of a DFA can be regarded as inline expan- expanded nd on its right-hand side. For example, Figure1 sion of nonterminals in the production rule that corresponds contains a reference from list to item. Silverchain consid- to the DFA. For example, the extension from Figure 2b to ers these reference relations when selecting an expanding Figure4 is modeled as follows: nonterminals:
203 GPCE’17, October 23–24, 2017, Vancouver, Canada T. Nakamaru, K. Ichikawa, T. Yamazaki, and S. Chiba
(1) Create a graph G. A node in G corresponds to each Table 1. Constructing a set of RDPDAs nonterminal in given grammar. An edge (ns ,nd ) in G corresponds to a relation from ns to nd . Figure6 shows (a) The table representation of DFAs after the second step a graph created from the grammar in Figure1. begin end list item text (2) Select an edge (ns ,nd ) such that the following subpro- cess returns true for (ns ,nd ): idoc ⇌ D0 -- D0 -- a. Create a subgraph ′ of by removing and the G G ns list → L L ---- edges with on their source or destination. 0 1 ns → L --- L L b. Decompose ′ into strongly connected components 1 3 2 G → L - L L L L (SCCs) and let the component containing be . 2 4 3 3 2 nd C → L - L - L L c. Return true if consists of a single node without a 3 4 3 2 C ← L ----- self-loop, and false otherwise. 4 If no edge is found, Silverchain finishes the extension item → I0 ---- I1 of DFAs. Silverchain examines edges in the order that ← I1 -- I2 -- an edge further from the node for the start nonterminal ← I2 ----- is examined earlier. Here, the start nonterminal is a nonterminal that is specified on the right-hand side of (b) The table representation of the constructed RDPDAs at this step the supplementary rule with = in given grammar. In the case of G in Figure6, Silverchain selects (list, item) begin end list item text through the subprocesses shown in Figure7. ′ idoc ⇌ D0 − L D0 - D0 -- (3) Apply inline expansion to the production rule for ns . Z 1 All the occurrences of nd in the rule are expanded. For list → L0 L1 ---- example, when ns is list and nd is item, then the rule → L1 --- L3 L2 list ′ for will be expanded into: → L2 Z− L1L3 L4 L3 L3 L2 list -> " b e g i n " (item|(text list?))+ " end " ; → L3 - L4 - L3 L2 ← as we have seen before. L4 ----- (4) Update G to reflect the inline expansion above. The item → I0 ---- I1 ′ selected edge (ns , nd ) is removed. New edges are added ← I1 Z− L1I2 - I2 -- to G to represent reference relations in the expanded ← I2 ----- production rule. Then, go back to (2). G is updated to ′ ′ list’ → L0 L1 ---- the graph in Figure8 when ( list, item) is selected for ′ ′ ′ → L1 --- L3 L2 expansion. → ′ − ′ ′ ′ ′ ′ ′ L2 ZL1L3 LSS4 ϵ L3 L3 L2 → ′ ′ ′ ′ Decomposition into SCC at (2) is required to avoid an infinite L3 - LSS4 ϵ - L3 L2 ′ regression of inline expansion caused by mutual recursion in ← L4 ----- grammar. Suppose that the edge (idoc, list) in Figure6 is selected for expansion. The dotted edge is removed and the ′ ′ red edge is added as shown in Figure 9a. Then the edge (idoc, begin L text L end D 1 2 D item) in Figure 9a is selected for expansion and the graph 0 ′ ′ 0 pop D0; D0 pop L ; D0 pop L ; is updated to Figure 9b, where the dotted edge is removed 1 2 push L′ D ; push L′ ; and the red edge is added. Figure 9b is equivalent to Figure 1 0 2 6. Further inline expansion will cause infinite regression. By the decomposition, Silverchain selects an edge composing Figure 10. An example of state transitions of the RDPDA a mutual-recursion cycle earlier than other edges. Such an for idoc edge is expanded and the cycle is transformed into non-cyclic edges and self-loops. Which edge is first selected among cyclic edges does not matter to resolve infinite regression. To support method chaining above, Silverchain extends DFAs to RDPDAs (real-time deterministic pushdown automata, 3.3 Constructing RDPDAs from DFAs deterministic pushdown automata without ϵ-transitions). Since an RDPDA has a stack, it can recognize languages that With the fluent API obtained at the previous step, the users may contain recursively defined nonterminals in their gram- still cannot compose an itemized document as follows: mar. Real-timeness, the absence of ϵ-transitions, is important // Fully non-subchaining style idoc().begin().text( " Item " ).end().eval(); since the encoding scheme mentioned in Section 3.1 cannot generate a method for an ϵ-transition.
204 Silverchain: A Fluent API Generator GPCE’17, October 23–24, 2017, Vancouver, Canada
To explain the extension at this step, we introduce table Table 2. The RDPDAs constructed without checking the representation of a set of automata. In the table representa- emptiness of rows or cells tion, all states of an automaton and all input symbols for the automaton are enumerated across the rows and the columns (a) Duplicating the rows for T at (ii) of the table, respectively. Table 1a is the table representation begin end extra T of the three DFAs obtained after the second step (the DFAs ′ in Figure 2a, 2c, and4). The rows in the table are grouped S → S0 Z− T1S1 -- S1 for each DFA and separated by horizontal lines. A row with ← S1 ---- → represents the initial state of the DFA corresponding to . the group that the row belongs to, and a row with ← rep- . resents an accepting state of the DFA. Each cell represents ′ ′ T’ → T0 T1 --- the destination of a state after consuming the symbol in the → ′ − ′ ′ ′ ′ T1 ZT1T2 TSS3 ϵ? - T2 column of the cell. → ′ ′ T2 - TSS3 ϵ? -- Table 1b is the table representation of three RDPDAs that → ′ ′ T3 -- TSS4 ϵ? - Silverchain constructs from three DFAs in Table 1a at this → T ′ ---- step. In the table representation of RDPDAs, a row with 4 → represents the state stacked at the initialization of the ′ RDPDA for the nonterminal corresponding to the group that (b) Overwriting U0 with V1U0 at (v) the row belongs to. For example, the stack of the RDPDA V for idoc is initialized with D0 as specified in the first rowin begin end Table 1b. Each cell in the table for RDPDAs represents the ′ U → U0 @U@0 V1 U0? - U1 states pushed into the stack when consuming the symbol ← U1 --- in the column of the cell. Figure 10 shows an example of . state transitions of the RDPDA for idoc when “begin text . end” is input. Every time an RDPDA consumes an input V’ → V ′ V ′ -- symbol, it pops the top element in its stack and pushes zero 0 1 → V ′ V ′ ϵ - V ′ or more elements to its stack according to the table. An 1 @@3 2 → V ′ - V ′ ϵ - RDPDA recognizes a symbol sequence if it has the state for 2 @@3 → V ′ --- a row with ← on the top of the stack after consuming the 3 final symbol. The extension by Silverchain from DFAs to RDPDAs is explained as modifying the table for the DFAs. During the (iv) Find a non-empty cell for the initial state of the DFA modification, Silverchain duplicates rows in the table. In in the duplicated row. Let the value in that cell be α list ′ Table 1b, the lowest five rows are such duplicated rows. The and its column be γ . When ν is , α is L1 and γ is duplication represents recursion in grammar. The cells in begin. red are modified to connect the existing rows (i.e. automata) (v) For each row ρ that has a non-empty value β in column and duplication. The table is modified as follows: ν, put the series αβ in row ρ of column γ if the cell in row ρ of column γ is empty. The four changes in column begin in Table 1b are applied at this step. (i) Find a nonterminal that is not expanded at the previous In Table 1b, the transition L × begin → L′ L added at (v) extension step in Section 3.2 and let the nonterminal 2 1 3 specifies that, when begin is read, a nested nonterminal list’ be ν. If no nonterminal is found, Silverchain finishes starts and the automaton moves to L′ . The nested nontermi- the extension from DFAs to RDPDAs. In Table 1a, list 1 nal list’ ends at the transitions added at (iii), for example, can be ν. which is L′ × end → ϵ. L′ is popped and moved to L , which (ii) Duplicate the rows for the DFA for ν if the accepting 2 2 3 has been pushed at (v). states of the DFA has no transitions. If any of the ac- cepting states has a transition, go back to (i). The five 3.3.1 Limitation rows at the bottom of Table 1b are such duplications At (ii), Silverchain does not duplicate rows if any of the rows since an accepting state L has no transition. 4 for accepting states is not empty. Hence, the non-subchaining (iii) Replace the value in a cell with ϵ if the value is a style will not be available for the corresponding nonterminal. duplicated row for the accepting states of the DFA for At (v), Silverchain does not put the series αβ if the target cell ν. ϵ does not push anything to a stack. The transition is not empty. Since the automaton never moves to duplicated only pops from a stack. The two changes in column rows, the non-subchaining style will not be available either. end are performed at this step. These cases occur in the following grammars:
205 GPCE’17, October 23–24, 2017, Vancouver, Canada T. Nakamaru, K. Ichikawa, T. Yamazaki, and S. Chiba
1 c l a s s Idoc { //(ii) 2 void eval () { } S -> T ; 3 T -> " b e g i n " T "end" "extra" ? 4 c l a s s StartingMethods { | "begin" "end" "extra" ?; 5 s t a t i c State0
206 Silverchain: A Fluent API Generator GPCE’17, October 23–24, 2017, Vancouver, Canada list of method names. The following shows the bodies of the own code for semantic actions when re-generating a type- generated methods in Figure 11: safe interface. However, this scheme requires the users to im- // Line5 in Figure 11 plement a complicated compiler to evaluate an entire method // Invoked at the beginning ofa chain chain in the eval method. This work will be tedious. s t a t i c State0
207 GPCE’17, October 23–24, 2017, Vancouver, Canada T. Nakamaru, K. Ichikawa, T. Yamazaki, and S. Chiba
′ ′ at the state L2 reads text, its state L2 does not change. Hence, provide more enhanced APIs that can be seen in manually Silverchain generates a method with a variable-length argu- written APIs: ′ ′ ments for the transition L1×text→ L2. // With graphviz-java node ( "X" ).with(Shape.RECTANGLE, Style.FILLED);
4 Use Case // With the generatedAPI node ( "X" ).beginAttr() To compare a fluent API generated by Silverchain anda . attr ( "shape=rectangle" ) manually written fluent API, we gave the grammar of DOT1 . attr ( "style=filled" ).endAttr(): to Silverchain. DOT is an external DSL for describing a graph The API provided by graphviz-java uses the domain-specific and used as follows: knowledge of DOT that is not represented in the grammar // Directed graph with two edges of DOT. Therefore, it cannot be generated by Silverchain but digraph G { A -> B it is also difficult to add such an extra API to the generated B -> A API. } As another use case, we gave the grammar of the DSL used The terminals that cannot be converted into method names by Silverchain for describing input grammar. The generated are replaced as appropriate, for example, “{” is replaced with API is a fluent API that defines a fluent API. With the API, beginGraph. With the fluent API generated by Silverchain, grammar of IDoc (Figure1) is written as shown in Figure 12. programmers can describe the graph above in Java as follows: Figure 12a shows a chain that expresses the IDoc’s grammar digraph().id( "G" ) without subchaining. Figure 12b shows a chain that uses .beginGraph() subchaining style to group semantically related subchains. . edge ( nodeId ( "A" ).arrow().nodeId( "B" ), Figure 12c shows a chain that uses subchaining style to nodeId ( "B" ).arrow().nodeId( "A" ) dynamically change a part of the entire chain. The users of ) .endGraph().eval(); the generated API can use any style that suits their usage. With graphviz-java2, a manually written fluent API, the same graph can be described as follows: 5 Related Work graph ( "G" ).directed() “Fluent interface” is coined by Fowler et al. [10] and a library . with ( with its fluent interface is called a fluent API. The ideato node ( "A" ).link(node( "B" )) node ( "B" ).link(node( "A" )) make fluent APIs type-safe is also pointed out by Fowler et ) al. [10] and its origin can be seen in the idea called typestate .toFile(...); [27]. There are several studies on fluent APIs that support As seen from these examples, the generated API can be used only non-subchaining style [11, 30]. Although such APIs are as the graphviz-java API. In addition, programmers can de- not common in practice, they can be regarded as a technique scribe the graph in non-subchaining style as follows: to embed a DSL into a general-purpose language that does digraph().id( "G" ) not have features to extend its own syntax. .beginGraph() . node ( "A" ).arrow().node( "B" ) Gil et al. proposed an algorithm to generate a fluent API . node ( "B" ).arrow().node( "A" ) that is type-safe and supports only non-subchaining style .endGraph().eval(); (hereinafter referred to as no-subchaining API for short) The first drawback of the generated API is that redun- [11, 24]. The algorithm generates such an API by encoding dant method calls are required such as beginGraph(). These a jump-stack real-time deterministic pushdown automaton methods can be omitted by using information obtained from (JRDPDA) into class definitions. Since JRDPDAs can recog- the table representation of RDPDAs. A method can be omit- nize deterministic context-free languages [7] and this class ted if it is the only transition that exists between states and of languages is much larger than the class that RDPDAs no ambiguity arises after omitting that transition. However, can recognize [16], the algorithm of Gil et al. can generate if such automatic omission is applied naively, a method chain no-subchaining APIs for grammars that Silverchain cannot with the generated API might be unreadable for program- generate them for. mers. Some redundant symbols in a DSL are necessary for With a no-subchaining API generated by the algorithm of programmers to read and understand written sentences. To Gil et al., however, the compilation time of a chain grows ex- avoid this problem, Silverchain does not apply that automatic ponentially to the length of the chain in the worst case. This omission. Another drawback of the generated API is that the is caused by the exponential growth of the size of the type API uses a lot of mechanically named classes such as State1 at the end of a chain, as Gil et al. showed in their experiment and State2. Programmers can hardly edit those classes to using Java 8 [11]. Here, the size of a type is defined by the number of type names in the textual representation of the 1http://www.graphviz.org/content/dot-language type. The exponential growth of compilation time occurs not 2https://github.com/nidi3/graphviz-java only in Java 8 but also in C++ and in Java 9 EA, which has
208 Silverchain: A Fluent API Generator GPCE’17, October 23–24, 2017, Vancouver, Canada
String javaSrc = grammar() .START().eq().symbol( "idoc" ).semicolon() . symbol ( "idoc" ).arrow().symbol( "list" ).star().semicolon() . symbol ( "list" ).arrow().literal( "begin" ). symbol ( "item" ).plus().literal( "end" ).semicolon() . symbol ( "item" ).arrow().symbol( "text" ). symbol ( "list" ).question().semicolon() . symbol ( "text" ).colon().STRING().semicolon() . toJava ();
(a) Without subchaining
Rule start = START().eq().symbol( "idoc" ).semicolon(); Rule idoc = symbol( "idoc" ).arrow().symbol( "list" ).star().semicolon(); Rule list = ... ; Rule item = ... ; Rule text = ... ; String javaSrc = grammar().rule(start, idoc, list, item, text).toJava();
(b) Grouping semantically related chains by subchaining
Factor items = allowEmptyList? symbol( "item" ).star() : symbol( "item" ). plus () ; String javaSrc = grammar() .START() ... . symbol ( "list" ).arrow().literal( "begin" ).factor(items). literal ( "end" ).semicolon() ... . toJava ();
(c) Dynamically changing a part of the expression
Figure 12. Code examples using the generated API for the DSL used in Silverchain a new type-checking strategy [8, 25]. (Figure 13 shows the the length of the chain even in the worst case but that API experiment in [11] performed on our machine using C++ and internally uses uncommon features such as implicit classes Java 9 EA.) Considering these facts, checking an extremely and implicit conversions to avoid the exponential growth. large type essentially takes a long time and the improvement Therefore, the technique used in ScaLALR is not portable to of compiler implementation is unlikely to solve this problem. other languages such as Java and C++. On the other hand, On the other hand, with APIs generated by Silverchain, the the technique used in Silverchain is relatively simple and is size of the type at the end of a chain grows linearly even in portable to languages whose type system has generic classes. the worst case. For that reason, the compilation time grows EriLex [30] is a tool to generate a no-subchaining API from at most linearly to the length of a chain. It is also problematic grammar by encoding an RDPDA into class definitions. The from a practical point of view that the algorithm of Gil et al. encoding method in EriLex is similar to the one in Silverchain does not take subchaining style into account. and is portable to other languages. The drawback of EriLex In theory, a no-subchaining API can be built for any gram- is that input grammar needs to be LL(1) in Greibach normal mar (including grammars that are not context-free) if the form [13], which is a form that most of manually written API is implemented in a language with a Turing-complete grammars do not follow. Since the users of EriLex need to type system. For instance, if given grammar is context-free, a rewrite grammar into that form, EriLex cannot generate a no-subchaining API can be built by creating a CYK parser [5, subchaining style API for each nonterminal. Furthermore, 22, 31] implemented by a Turing machine emulated on a type there is no algorithm to rewrite grammar into the form re- system [14]. However, using a Turing machine on the back quired by EriLex as far as we know. Whether given grammar is overly complex for most DSLs and often causes practical can be rewritten into that form is undecidable [26]. There problems. For example, javac throws a StackOverflowError are several tools known as fluent API generators such as even for a small chain of two or three methods with a no- CLARA [2], TS4J [3], and fluflu [29]. These tools, however, subchaining API generated by Grigore’s algorithm [14], one generate only subchaining style APIs. of algorithms that uses a Turing machine emulated on Java The role of type parameters in Silverchain differs from type system. ones in manually written fluent APIs such as AssertJ [6], ScaLALR [18] generates a no-subchaining API in Scala j2html [20], and jOOQ [12]. In those manually written fluent from grammar by encoding an LALR parser into class def- APIs, type parameters are used to eliminate boilerplate code initions. With APIs generated by ScaLALR, the size of the in classes for APIs. For example in AssertJ, type parameters type at the end of a chain does not grow exponentially to
209 GPCE’17, October 23–24, 2017, Vancouver, Canada T. Nakamaru, K. Ichikawa, T. Yamazaki, and S. Chiba
40 3 35 Another future work is to improve the emulation of DSLs 2.5 30 by using the mechanism in the host language. For exam- 2 25 ple, DSL sentences are currently emulated only by method 20 1.5 15 1 chaining, but they could be emulated in a better way by 10 Compilation time (s) Compilation time (s) 0.5 mapping a part of DSL’s syntax to the similar syntax in a 5 host general-purpose language (e.g. if-else syntax in a DSL 0 0 5 10 15 20 25 0 0 5 10 15 20 25 Chain length Chain length to the same syntax in a host language). The properties of (a) The result in Java 9 EA (b) The result in C++ an emulated DSL other than syntactic rules could also be statically checked if the DSL’s type system or name binding system is also mapped to host language mechanism. c l a s s Exp
210 Silverchain: A Fluent API Generator GPCE’17, October 23–24, 2017, Vancouver, Canada
https://doi.org/10.1145/321250.321254 [23] Ansgar Konermann. 2010. Emulating "self types" using Java Generics to [14] Radu Grigore. 2017. Java Generics Are Turing Complete. In Proceedings simplify fluent API implementation. (November 2010). Retrieved Sep- of the 44th ACM SIGPLAN Symposium on Principles of Programming tember 4, 2017 from http://web.archive.org/web/20130721224442/http: Languages (POPL 2017). ACM, New York, NY, USA, 73–85. https: /passion.forco.de/content/emulating-self-types-using-java-generics //doi.org/10.1145/3009837.3009871 -simplify-fluent-api-implementation [15] Hamcrest. 2012. Hamcrest. (August 2012). Retrieved September 4, [24] Tomer Levy. 2017. A Fluent API for Automatic Generation of Fluent 2017 from http://hamcrest.org/ APIs in Java. Ph.D. Dissertation. Israel Institute of Technology. [16] Michael Harrison and Ivan Havel. 1972. Real-Time Strict Deterministic [25] Vicente Romero and Maurizio Cimadamore. 2016. JEP 215: Tiered Languages. SIAM J. Comput. 1, 4 (1972), 333–349. https://doi.org/10. Attribution for javac. (July 2016). Retrieved September 4, 2017 from 1137/0201024 http://openjdk.java.net/jeps/215 [17] Paul Hudak. 1996. Building Domain-specific Embedded Languages. [26] Daniel Rosenkrantz and Richard Stearns. 1969. Properties of Deter- ACM Comput. Surv. 28, 4es, Article 196 (Dec. 1996). https://doi.org/10. ministic Top Down Grammars. In Proceedings of the First Annual ACM 1145/242224.242477 Symposium on Theory of Computing (STOC ’69). ACM, New York, NY, [18] Kazuhiro Ichikawa. 2016. phenan/scalalr: ScaLALR : LALR parser USA, 165–180. https://doi.org/10.1145/800169.805431 generator for embedded DSLs in Scala. (December 2016). Retrieved [27] Robert Strom and Shaula Yemini. 1986. Typestate: A programming September 4, 2017 from https://github.com/phenan/scalalr language concept for enhancing software reliability. IEEE Transactions [19] Kazuhiro Ichikawa and Shigeru Chiba. 2017. User-Defined Opera- on Software Engineering SE-12, 1 (Jan 1986), 157–171. https://doi.org/ tors Including Name Binding for New Language Constructs. The Art, 10.1109/TSE.1986.6312929 Science, and Engineering of Programming 1, 2, Article 15 (2017). [28] Ken Thompson. 1968. Programming Techniques: Regular Expression [20] j2html. 2017. Fast and fluent Java HTML5 builder - Java HTML builder. Search Algorithm. Commun. ACM 11, 6 (June 1968), 419–422. https: (September 2017). Retrieved September 4, 2017 from https://j2html.co //doi.org/10.1145/363347.363387 m/ [29] Peter Verhas. 2014. verhas/fluflu: Fluent Api Creator. (July 2014). [21] jMock. 2012. jMock - An Expressive Mock Object Library for Java. Retrieved September 4, 2017 from https://github.com/verhas/fluflu (December 2012). Retrieved September 4, 2017 from http://www.jmoc [30] Hao Xu. 2010. EriLex: An Embedded Domain Specific Language Gener- k.org/ ator. Springer Berlin Heidelberg, Berlin, Heidelberg. 192–212 pages. [22] Tadao Kasami. 1965. An Efficient Recognition and Syntax-Analysis Algo- https://doi.org/10.1007/978-3-642-13953-6_11 rithm for Context-Free Languages. Technical Report. DTIC Document. [31] Daniel Younger. 1967. Recognition and parsing of context-free lan- guages in time n3. Information and Control 10, 2 (1967), 189 – 208. https://doi.org/10.1016/S0019-9958(67)80007-X
211