Appendix To: Learning Disentangled Representations of Texts With

Home , Bottom of the Bottle

Appendix to: Learning Disentangled Representations of Texts with Application to Biomedical Abstracts Sarthak Jain Edward Banner Jan-Willem van de Meent Northeastern University Northeastern University Northeastern University [email protected] [email protected] [email protected]

Iain J. Marshall Byron C. Wallace King’s College London Northeastern University [email protected] [email protected]

A Example of PICO Triplet generation safe when administered as long-term treatment.

As an illustrative example, we walk through con- For study S, the summaries in the CDSR are as struction of a single triplet for the PICO domain follow. First the P summary (sP ) , in detail. Recall that we first randomly draw two Number of children: 10 Inclusion criteria: ado- reviews, R1 and R2. In this case, R1 consists lescents (puberty stage at least 2, at least 12 of studies involving nocturnal enuresis and review years) Exclusion criteria: treatment in previous R2 concerns asthma. From R1 we randomly select 2 weeks, daytime wetting, UTI, urinary tract two studies S, S0. abnormalities Previous treatment: failed using alarms, desmopressin, other drugs Age range 11- (1) Here we sample abstract A for study S, 21, median 13 years Baseline wetting 4.7 (SD shown below: 1.1) wet nights/week Department of Paediatric Surgery, Sweden In recent years the treatment of primary nocturnal enuresis (PNE) with desmopressin (DDAVP) I Summary (sI ): has been promising. The route of administration until now had been intranasal, but because the A : desmopressin orally (dosage based on titra- tablets were introduced for the treatment of dia- tion period) B : placebo Duration 4 weeks each betes insipidus they have also become available for the treatment of PNE. To find the optimal dosage of desmopressin tablets and to compare The O summary (sO) desmopressin’s efficacy with placebo in a group of adolescents with severe monosymptomatic Wet nights during trial (number, mean, SD): A: enuresis. The long-term safety of desmopressin 10, 1.8 (SD 1.4); B: 10, 4.1 (1.5) Side effects: was also studied in the same group of patients. headache (5); abdominal pain (6); nausea and The effect of oral desmopressin (1-deamino-8-D- vertigo (1) All resolved while treatment contin- arginine-vasopressin) (DDAVP tablets, Minirin) ued was investigated in 25 adolescents (ages 11 to 21 years) with severe monosymptomatic nocturnal enuresis. The first part of the dose-ranging study (2) From study S0, the summaries in the CDSR comprised a single-blind dose titration period, followed by a double-blind, crossover efficacy are reproduced as follows. First, the P Summary 0 period comparing desmopressin with placebo. (sP ): The final part was an open long-term study consisting of two 12-week treatment periods. The Number of children: 135 Dropouts: 23 excluded efficacy of the drug was measured in reductions for non-compliance, and 39 lost to follow up in- of the number of wet nights per week. During the cluding 12 failed with alarms Inclusion criteria: first dose-titration period, the majority of the pa- monosymptomatic nocturnal enuresis, age > 5 tients were given desmopressin 400 micrograms, years Exclusion criteria: previous treatment with and the number of wet nights decreased from a DDAVP or alarm, urological pathology, diurnal mean of 4.9 to 2.8. During the double-blind pe- enuresis, UTI Age, mean years: 11.2 Baseline riod, a significant reduction of wet nights was ob- wetting: A 21% dry nights, B 14% dry nights served (1.8 vs 4.1 for placebo). During the two long-term periods, 48% and 53% of the patients I Summary (s0 ): could be classified as responders (0 to 1 wet night I per week) and 22% and 23.5% as intermediate responders (2 to 3 wet nights per week). No A (62): desmopressin 20 g intranasally increas- weight gain was observed due to water retention. ing to 40 g if response partial B (73): alarm After cessation of the drug, 44% of the patients (pad-and-bell) Duration of treatment 3 months. had a significant decrease in the number of wet If failed at that time, changed to alternative arm nights. Oral desmopressin has a clinically signif- 0 icant effect on patients with PNE, and therapy is The O summary (sO) DRY nights at 3 months: A 85%; B: 90% Num- B.1 PICO Domain ber not achieving 14 dry nights: A 12/39; B: 6/37 Side effects: not mentioned B.1.1 Baselines For the TF-IDF baseline, we use the implemen- 2 00 tation in scikit-learn . The TF-IDF was fit on all (3) From review R2, we sample one study S . CDSR data. The resulting transformation was ap- Matched summaries in the CDSR are as follows. plied to Cohen corpus. We use cosine similarity to P Summary (s00 ): P evaluate TF-IDF model. n = 8 Mean age = 52 Inclusion: intrinsic asthma, For Doc2Vec, we used the gensim implemen- constant reversibility > 20% None had an acute tation3 with an 800d embedding size, and a win- exacerbation at time of study Exclusion: none listed dow size of 10. These parameters were selected via random search over validation data. We oth- 00 I summary (sI ): erwise used the default settings in gensim. The vocabulary used was the same as above. #1: Atenolol 100 mg Metoprolol 100 mg Placebo #2: Terbutaline (IV then inhaled) after Tx or For LDA, we again used the scikit-learn imple- placebo mentation, setting the number of topics to 7; this was selected via line search over the validation set 00 O summary (sO): across the range 1 to 50. FEV1 Symptoms For the NVDM baseline, we used hidden layer comprising 500 dimensions and an output embedding dimension of 200. We used Tanh activa- 0 (4) We note how the summaries for S and S are tion functions. The model was trained using the similar to each other (but not identical) since they Adam optimizer, using a learning rate of 5e-5. We belong in the same review whereas they are quite performed early stopping based on validation per- 00 different from summaries for S which belongs in plexity. a different review. Now we construct the triplet For the ABAE model, we use the same hyper- (s, d, o)P as follows: parameters as reported in the original He et al. (2017) paper. We experimented with 20, 50 and s = s0 : P 100 aspects. Of these, 100 performed best, which Number of children: 135 Dropouts: 23 excluded is what we used to generate the reported results. for non-compliance, and 39 lost to follow up in- For the DSSM model, we used 300d filters with cluding 12 failed with alarms Inclusion criteria: monosymptomatic nocturnal enuresis, age > 5 filter windows of sizes (1, 6) and output embed- years Exclusion criteria: previous treatment with ding dimension of 256. We used tanh activation DDAVP or alarm, urological pathology, diurnal for both the convolution and output layers, and enuresis, UTI Age, mean years: 11.2 Baseline wetting: A 21% dry nights, B 14% dry nights imposed l2 regularization on weights (1e-5). The model was trained using the Adam optimizer with 00 0 0 So we have d : A and o = [sP |sI |sO]: early stopping on validation loss.

n = 8 Mean age = 52 Inclusion: intrinsic asthma, B.2 Beer Advocate Domain constant reversibility > 20% None had an acute exacerbation at time of study Exclusion: none B.2.1 Baselines listed — A (62): desmopressin 20 g intranasally increasing to 40 g if response partial B (73): For the TF-IDF baseline, we used the scikit-learn alarm (pad-and-bell) Duration of treatment 3 implementation. TF-IDF parameters were fit on months. If failed at that time, changed to alternative arm — DRY nights at 3 months: A 85%; the available training set. We used cosine similar- B: 90% Number not achieving 14 dry nights: A ity for evaluation. 12/39; B: 6/37 Side effects: not mentioned For Doc2Vec, we used the same method as B Implementation Details & Baseline above, here setting (after line search) the embed- Hyperparameters ding size to 800d and window size to 7. For the LDA baseline, we followed the same All tokenisation has been done using default strategy as above, arriving at 4 topics. spaCy1 tokenizer. 2http://scikit-learn.org/ 1https://spacy.io/ 3https://radimrehurek.com/gensim/ For NVDM, we set the number of dimensions tions for both convolution and output layer and l2 in the hidden layer to 500, and set the embedding regularization on weights of 1e-6. The model was dimension to 145. As an activation for the hidden trained using the Adam optimizer with early stop- layer we used Tanh. Again the model was trained ping on validation loss. using the Adam optimizer, with a learning rate of 5e-5 and early stopping based on validation per- C Yelp!/TripAdvisor Dataset Word plexity. Activations For the ABAE baseline, we use the same hyperparameters as in He et al. (2017) paper, since Domain celebrity, cypress, free, inadequate, BeerAdvocate was one of the evaluation datasets kitchens, maroma, maya, scratchy, suburban, used in said work. supply For NVDM + Triplet Loss, we used same set- Sentiment awful, disgusting, horrible, tings as reported above for NVDM, with loss mediocre, poor, rude, sells, shame, unaccept- weighting of 0.001. able, worst For the DSSM baseline, we used 200d filters Table 1: Most activated words for hotels/restaurant with filter windows of sizes (2, 4) and output em- data. bedding dimension of 256. We used tanh activations for both convolution and output layer and l2 We did not have sufficient space to include the regularization on weights of 1e-6. The model was most activated words per aspect (inferred via the trained using Adam optimizer with early stopping gating mechanism) for the Yelp!/TripAdvisor cor- on validation loss. pus in the manuscript (as we did for the other do- mains) and so we include them here. B.3 Yelp! and TripAdvisor Domain B.3.1 Baselines D Highlighted Text To induce the baseline TF-IDF vectors, we used To visualize the output of gating mechanism g ∈ the scikit-learn implementation. TF-IDF parame- RN×1, we perform the following transformations: ters were fit on the available training set. Our vo- 1. Since the gating is applied on the top convo- cabulary comprised those words that appeared in lution layer, there is no one-to-one correspon- at least 5 unique reviews, which corresponded to dence between words and values in g. Hence, 10K words. We used cosine similarity for evalua- we convolved the output a 5 length mean fil- tion. ter to smooth the gating output. For Doc2Vec, we set the embedding size to 800d and the window size to 7. For LDA, we again 2. We normalize the range of values in g to [0, 1] used the scikit-learn implementation, setting the using the following equation: number of topics to 10. g − min(g) For NVDM, we set the number of dimensions g = (1) in the hidden layer to 500, and set the embedding max(g) − min(g) dimension to 200. As an activation for the hidden layer we used Tanh. Again the model was trained In the following subsections, we provide 3 ex- using the Adam optimizer, with a learning rate of amples from each of our datasets highlighted with 5e-5 and early stopping based on validation per- corresponding gating output. plexity. D.1 PICO Domain For ABAE, we use the same hyperparameters in He et al. (2017), since Yelp was another one of the Color Legend : Population Intervention Outcome Example 1 evaluation datasets used in their work. determined the efficacy of unk ( unk ) in a clinical For NVDM + Triplet Loss, we used same set- population of aggressive , urban children diagnosed with tings as above for NVDM, and again used loss attention deficit unk disorder ( adhd ) . in previous studies of unk children with adhd , unk has been shown to be effective weighting of 0.001. when compared with placebo . eighteen unk children ( ages For the DSSM baseline, we used 200d filters qqq to qqq years ) , diagnosed with adhd and attending a unk treatment program for youth with unk behavior disorders , with filter windows of sizes (1, 4) and output em- participated in a double-blind placebo trial with assessment bedding dimension of 256. We used tanh activa- data obtained from staff in the program and parents at home . based on staff ratings of the children ’s behavior in the treated with sulphonylurea during pregnancy and a reference program and an academic classroom , the children displayed group of qqq diabetic women treated with insulin during significant improvements in adhd symptoms and aggressive pregnancy . results : the prevalence of pre-eclampsia was behavior with unk and high-dose unk conditions . at home , significantly increased in the group of women treated with parents and unk reported few significant differences between metformin compared to women treated with sulphonylurea placebo and unk on behavior ratings . in both settings , unk or insulin ( qqq vs. qqq vs. qqq % , p ¡ qqq ) . no difference in was well tolerated with few side effects found during active neonatal morbidity was observed between the orally treated drug conditions . and unk group ; no cases of severe hypoglycaemia or jaundice determined the efficacy of unk ( unk ) in a clinical were seen in the orally treated groups . however , in the group population of aggressive , urban children diagnosed with of women treated with metformin in the third trimester , the attention deficit unk disorder ( adhd ) . in previous studies of perinatal mortality was significantly increased compared to unk children with adhd , unk has been shown to be effective women not treated with metformin ( qqq vs. qqq % , p ¡ qqq ) . when compared with placebo . eighteen unk children ( ages conclusion : treatment with metformin during pregnancy was qqq to qqq years ) , diagnosed with adhd and attending a unk associated with increased prevalence of pre-eclampsia and a treatment program for youth with unk behavior disorders , high perinatal mortality . participated in a double-blind placebo trial with assessment aims : to assess maternal and neonatal complications data obtained from staff in the program and parents at home in pregnancies of diabetic women treated with oral . based on staff ratings of the children ’s behavior in the hypoglycaemic agents during pregnancy . methods : a cohort program and an academic classroom , the children displayed study including all unk registered , orally treated pregnant significant improvements in adhd symptoms and aggressive diabetic patients set in a diabetic unk service at a university behavior with unk and high-dose unk conditions . at home , hospital : qqq women treated with metformin , qqq women parents and unk reported few significant differences between treated with sulphonylurea during pregnancy and a reference placebo and unk on behavior ratings . in both settings , unk group of qqq diabetic women treated with insulin during was well tolerated with few side effects found during active pregnancy . results : the prevalence of pre-eclampsia was drug conditions . significantly increased in the group of women treated with determined the efficacy of unk ( unk ) in a clinical metformin compared to women treated with sulphonylurea population of aggressive , urban children diagnosed with or insulin ( qqq vs. qqq vs. qqq % , p ¡ qqq ) . no difference in attention deficit unk disorder ( adhd ) . in previous studies of neonatal morbidity was observed between the orally treated unk children with adhd , unk has been shown to be effective and unk group ; no cases of severe hypoglycaemia or jaundice when compared with placebo . eighteen unk children ( ages were seen in the orally treated groups . however , in the group qqq to qqq years ) , diagnosed with adhd and attending a unk of women treated with metformin in the third trimester , the treatment program for youth with unk behavior disorders , perinatal mortality was significantly increased compared to participated in a double-blind placebo trial with assessment women not treated with metformin ( qqq vs. qqq % , p ¡ qqq ) . data obtained from staff in the program and parents at home conclusion : treatment with metformin during pregnancy was . based on staff ratings of the children ’s behavior in the associated with increased prevalence of pre-eclampsia and a program and an academic classroom , the children displayed high perinatal mortality . significant improvements in adhd symptoms and aggressive Example 3 behavior with unk and high-dose unk conditions . at home , purpose : congestive heart failure is an important cause parents and unk reported few significant differences between of patient morbidity and mortality . although several placebo and unk on behavior ratings . in both settings , unk randomized clinical trials have compared beta-blockers was well tolerated with few side effects found during active with placebo for treatment of congestive heart failure , a drug conditions . meta-analysis unk the effect on mortality and morbidity has Example 2 not been performed recently . data unk : the unk , unk , aims : to assess maternal and neonatal complications and unk of unk electronic unk were unk from qqq to july in pregnancies of diabetic women treated with oral qqq unk were also identified from unk of unk unk . study hypoglycaemic agents during pregnancy . methods : a cohort selection : all randomized clinical trials of beta-blockers study including all unk registered , orally treated pregnant versus placebo in chronic stable congestive heart failure were diabetic patients set in a diabetic unk service at a university included . data extraction : a specified protocol was followed hospital : qqq women treated with metformin , qqq women to extract data on patient characteristics , unk used , overall treated with sulphonylurea during pregnancy and a reference mortality , hospitalizations for congestive heart failure , and group of qqq diabetic women treated with insulin during study quality . data unk : a unk unk model was used to unk pregnancy . results : the prevalence of pre-eclampsia was the results . a total of qqq trials involving qqq qqq patients significantly increased in the group of women treated with were identified . there were qqq deaths among qqq patients metformin compared to women treated with sulphonylurea randomly assigned to placebo and qqq deaths among qqq or insulin ( qqq vs. qqq vs. qqq % , p ¡ qqq ) . no difference in patients assigned to unk therapy . in these groups , qqq neonatal morbidity was observed between the orally treated and qqq patients , respectively , required hospitalization for and unk group ; no cases of severe hypoglycaemia or jaundice congestive heart failure . the probability that unk therapy were seen in the orally treated groups . however , in the group reduced total mortality and hospitalizations for congestive of women treated with metformin in the third trimester , the heart failure was almost qqq % . the best estimates of these perinatal mortality was significantly increased compared to advantages are qqq unk unk and qqq fewer hospitalizations women not treated with metformin ( qqq vs. qqq % , p ¡ qqq ) . per qqq patients treated in the first year after therapy . the conclusion : treatment with metformin during pregnancy was probability that these benefits are clinically significant ( ¿ associated with increased prevalence of pre-eclampsia and a qqq unk unk or ¿ qqq fewer hospitalizations per qqq patients high perinatal mortality . treated ) is qqq % . both selective and unk agents produced aims : to assess maternal and neonatal complications these unk effects . the results are unk to any unk publication in pregnancies of diabetic women treated with oral unk . conclusions : unk therapy is associated with clinically hypoglycaemic agents during pregnancy . methods : a cohort meaningful reductions in mortality and morbidity in patients study including all unk registered , orally treated pregnant with stable congestive heart failure and should be routinely diabetic patients set in a diabetic unk service at a university offered to all patients similar to those included in trials . hospital : qqq women treated with metformin , qqq women purpose : congestive heart failure is an important cause of patient morbidity and mortality . although several apricot , unk smith apple , almost like a unk wine . taste was randomized clinical trials have compared beta-blockers unk . i unk a couple of bottles an tried them so if they were with placebo for treatment of congestive heart failure , a skunked , it was unk unk that were bad . there was no real meta-analysis unk the effect on mortality and morbidity has apricot taste , and no sweetness . it tasted like hay but not in not been performed recently . data unk : the unk , unk , the good wheat beer sense , more like dirty hay thats been and unk of unk electronic unk were unk from qqq to july under the budweiser unk . mouthfeel was thin and carbonated qqq unk were also identified from unk of unk unk . study . drinkability ... well this is the second beer in my long list selection : all randomized clinical trials of beta-blockers that i ’ve actually had to pour out . versus placebo in chronic stable congestive heart failure were appearance was gold , clear , no head , and fizzy . almost included . data extraction : a specified protocol was followed looked like champagne . aroma was actually real good . sweet to extract data on patient characteristics , unk used , overall apricot , unk smith apple , almost like a unk wine . taste was mortality , hospitalizations for congestive heart failure , and unk . i unk a couple of bottles an tried them so if they were study quality . data unk : a unk unk model was used to unk skunked , it was unk unk that were bad . there was no real the results . a total of qqq trials involving qqq qqq patients apricot taste , and no sweetness . it tasted like hay but not in were identified . there were qqq deaths among qqq patients the good wheat beer sense , more like dirty hay thats been randomly assigned to placebo and qqq deaths among qqq under the budweiser unk . mouthfeel was thin and carbonated patients assigned to unk therapy . in these groups , qqq . drinkability ... well this is the second beer in my long list and qqq patients , respectively , required hospitalization for that i ’ve actually had to pour out . congestive heart failure . the probability that unk therapy appearance was gold , clear , no head , and fizzy . almost reduced total mortality and hospitalizations for congestive looked like champagne . aroma was actually real good . sweet heart failure was almost qqq % . the best estimates of these apricot , unk smith apple , almost like a unk wine . taste was advantages are qqq unk unk and qqq fewer hospitalizations unk . i unk a couple of bottles an tried them so if they were per qqq patients treated in the first year after therapy . the skunked , it was unk unk that were bad . there was no real probability that these benefits are clinically significant ( ¿ apricot taste , and no sweetness . it tasted like hay but not in qqq unk unk or ¿ qqq fewer hospitalizations per qqq patients the good wheat beer sense , more like dirty hay thats been treated ) is qqq % . both selective and unk agents produced under the budweiser unk . mouthfeel was thin and carbonated these unk effects . the results are unk to any unk publication . drinkability ... well this is the second beer in my long list unk . conclusions : unk therapy is associated with clinically that i ’ve actually had to pour out . meaningful reductions in mortality and morbidity in patients appearance was gold , clear , no head , and fizzy . almost with stable congestive heart failure and should be routinely looked like champagne . aroma was actually real good . sweet offered to all patients similar to those included in trials . apricot , unk smith apple , almost like a unk wine . taste was purpose : congestive heart failure is an important cause unk . i unk a couple of bottles an tried them so if they were of patient morbidity and mortality . although several skunked , it was unk unk that were bad . there was no real randomized clinical trials have compared beta-blockers apricot taste , and no sweetness . it tasted like hay but not in with placebo for treatment of congestive heart failure , a the good wheat beer sense , more like dirty hay thats been meta-analysis unk the effect on mortality and morbidity has under the budweiser unk . mouthfeel was thin and carbonated not been performed recently . data unk : the unk , unk , . drinkability ... well this is the second beer in my long list and unk of unk electronic unk were unk from qqq to july that i ’ve actually had to pour out . qqq unk were also identified from unk of unk unk . study Example 2 selection : all randomized clinical trials of beta-blockers poured into a nonic pint glass ... jet black , unk no versus placebo in chronic stable congestive heart failure were highlights around the edges at all , dense tan unk head ... included . data extraction : a specified protocol was followed looks great ! a bit of sediment in the bottom of the bottle . to extract data on patient characteristics , unk used , overall bottle read “ live ale , keep unk . ” the store where i bought mortality , hospitalizations for congestive heart failure , and it from had it on the shelf at room temp : ( the smell was study quality . data unk : a unk unk model was used to unk surprising ... an earthy roasted smell , mixed with day old the results . a total of qqq trials involving qqq qqq patients coffee . also some weird “ off ” licorice notes . taste was dry were identified . there were qqq deaths among qqq patients at the start , and very dry on the finish . bitter roast notes with randomly assigned to placebo and qqq deaths among qqq a sort of unk . full bodied with appropriate carbonation . i was patients assigned to unk therapy . in these groups , qqq very excited to try this beer , and i was pretty disappointed . and qqq patients , respectively , required hospitalization for this bottle could be old , or the flavors could be “ off ” from congestive heart failure . the probability that unk therapy the room temp unk . reduced total mortality and hospitalizations for congestive poured into a nonic pint glass ... jet black , unk no heart failure was almost qqq % . the best estimates of these highlights around the edges at all , dense tan unk head ... advantages are qqq unk unk and qqq fewer hospitalizations looks great ! a bit of sediment in the bottom of the bottle . per qqq patients treated in the first year after therapy . the bottle read “ live ale , keep unk . ” the store where i bought probability that these benefits are clinically significant ( ¿ it from had it on the shelf at room temp : ( the smell was qqq unk unk or ¿ qqq fewer hospitalizations per qqq patients surprising ... an earthy roasted smell , mixed with day old treated ) is qqq % . both selective and unk agents produced coffee . also some weird “ off ” licorice notes . taste was dry these unk effects . the results are unk to any unk publication at the start , and very dry on the finish . bitter roast notes with unk . conclusions : unk therapy is associated with clinically a sort of unk . full bodied with appropriate carbonation . i was meaningful reductions in mortality and morbidity in patients very excited to try this beer , and i was pretty disappointed . with stable congestive heart failure and should be routinely this bottle could be old , or the flavors could be “ off ” from offered to all patients similar to those included in trials . the room temp unk . poured into a nonic pint glass ... jet black , unk no D.2 BeerAdvocate highlights around the edges at all , dense tan unk head ... looks great ! a bit of sediment in the bottom of the bottle . Color Legend : Look Aroma Palate Taste bottle read “ live ale , keep unk . ” the store where i bought Example 1 it from had it on the shelf at room temp : ( the smell was appearance was gold , clear , no head , and fizzy . almost surprising ... an earthy roasted smell , mixed with day old looked like champagne . aroma was actually real good . sweet coffee . also some weird “ off ” licorice notes . taste was dry at the start , and very dry on the finish . bitter roast notes with lift up to the rooms ! the location was great and at night we a sort of unk . full bodied with appropriate carbonation . i was left the windows open and listened to the bustle of the street very excited to try this beer , and i was pretty disappointed . below and a unk off somewhere ... we felt so ’ roman ” ! this bottle could be old , or the flavors could be “ off ” from they gave us champagne on our last night ! the gentleman the room temp unk . who works the night shift even gave us a personal escort to poured into a nonic pint glass ... jet black , unk no unk unk , a maze of metro stops , and delivered us right at the highlights around the edges at all , dense tan unk head ... gate ! unk was just so nice ! i highly recommend this small looks great ! a bit of sediment in the bottom of the bottle . hotel and would absolutely stay there again if we are lucky bottle read “ live ale , keep unk . ” the store where i bought enough to return ! they deserve their # qqq tripadvisor rating it from had it on the shelf at room temp : ( the smell was ! thanks you , unk unk , for your part in making this trip such surprising ... an earthy roasted smell , mixed with day old a pleasure ! coffee . also some weird “ off ” licorice notes . taste was dry we have just returned from our first trip to rome and what at the start , and very dry on the finish . bitter roast notes with a wonderful time we had ! our stay at unk unk was everything a sort of unk . full bodied with appropriate carbonation . i was we had hoped for and all we had read about it was true ! the very excited to try this beer , and i was pretty disappointed . room was small , but very clean and comfortable . the staff this bottle could be old , or the flavors could be “ off ” from was fantastic and friendly , very helpful and the breakfast the room temp unk . every day was wonderful ! we loved the lift up to the rooms Example 3 ! the location was great and at night we left the windows this is a real “ nothing ” beer . pours a unk yellow color , open and listened to the bustle of the street below and a looking more like unk water than beer . strange unk smell , unk off somewhere ... we felt so ’ roman ” ! they gave us with minimal unk hop aroma unk away behind whatever it is champagne on our last night ! the gentleman who works the that unk in this brew . taste is equally as unk . nearly unk , the night shift even gave us a personal escort to unk unk , a maze most you get from this beer is a slightly sweet unk flavor and of metro stops , and delivered us right at the gate ! unk was a lot of carbonation in the beginning . watery and thin , but just so nice ! i highly recommend this small hotel and would something that goes down easy so the drinkability is unk up absolutely stay there again if we are lucky enough to return ! slightly . unk . they deserve their # qqq tripadvisor rating ! thanks you , unk this is a real “ nothing ” beer . pours a unk yellow color , unk , for your part in making this trip such a pleasure ! looking more like unk water than beer . strange unk smell , Example 3 with minimal unk hop aroma unk away behind whatever it is oh , unk . true to name , you ’re like a guilty indulgence that unk in this brew . taste is equally as unk . nearly unk , the that i want more of . while visiting cleveland , i went to unk most you get from this beer is a slightly sweet unk flavor and for a late dinner around qqq on a thursday . i sat at the bar , a lot of carbonation in the beginning . watery and thin , but and there was quite a full crowd because they have a second , something that goes down easy so the drinkability is unk up late happy hour from qqq with great deals on drinks and small slightly . unk . plates . i ordered the house red wine ( $ qqq at happy hour this is a real “ nothing ” beer . pours a unk yellow color , ! ) , and two suggestions of the bartender : the roasted dates looking more like unk water than beer . strange unk smell , to start and the steak entree with a side of brussels sprouts with minimal unk hop aroma unk away behind whatever it is . the dates ( $ qqq ) were incredibly decadent : roasted and that unk in this brew . taste is equally as unk . nearly unk , the topped with almonds , bacon , unk and parsley , they are a most you get from this beer is a slightly sweet unk flavor and party in your both of sweet and salty and spicy . the sirloin a lot of carbonation in the beginning . watery and thin , but ( $ qqq ) , which the bartender told me is a new menu item something that goes down easy so the drinkability is unk up , was beautifully cooked and topped with parmesan , truffle slightly . unk . butter , arugula and mushrooms . it was good , although there this is a real “ nothing ” beer . pours a unk yellow color , are other things on the menu that might be more exciting . the looking more like unk water than beer . strange unk smell , stand out dish of the evening was the fried brussels sprouts with minimal unk hop aroma unk away behind whatever it is , which i would very much like the recipe for so i can unk that unk in this brew . taste is equally as unk . nearly unk , the myself on them every night . those babies , topped with crispy most you get from this beer is a slightly sweet unk flavor and crumbled bits of unk , capers and walnuts , were pure bliss a lot of carbonation in the beginning . watery and thin , but . and let ’s be honest , how often do you really have the something that goes down easy so the drinkability is unk up opportunity to call unk sprouts unk ? probably not very often slightly . unk . . i unk off the evening with a glass of unk scotch , neat . perfection . D.3 Yelp!/TripAdvisor Dataset oh , unk . true to name , you ’re like a guilty indulgence that i want more of . while visiting cleveland , i went to unk Color Legend : Sentiment Domain for a late dinner around qqq on a thursday . i sat at the bar , Example 1 i love everything about this place , i find my and there was quite a full crowd because they have a second , self there more often then i want , the deserts are amazing , late happy hour from qqq with great deals on drinks and small so many choices , the deli is awesome , the sushi bar is great plates . i ordered the house red wine ( $ qqq at happy hour , you can not go wrong with this place , if you want a formal ! ) , and two suggestions of the bartender : the roasted dates dining you can sit at the full service restaurant . to start and the steak entree with a side of brussels sprouts i love everything about this place , i find my self there . the dates ( $ qqq ) were incredibly decadent : roasted and more often then i want , the deserts are amazing , so many topped with almonds , bacon , unk and parsley , they are a choices , the deli is awesome , the sushi bar is great , you party in your both of sweet and salty and spicy . the sirloin can not go wrong with this place , if you want a formal dining ( $ qqq ) , which the bartender told me is a new menu item you can sit at the full service restaurant . , was beautifully cooked and topped with parmesan , truffle Example 2 we have just returned from our first trip to butter , arugula and mushrooms . it was good , although there rome and what a wonderful time we had ! our stay at unk are other things on the menu that might be more exciting . the unk was everything we had hoped for and all we had read stand out dish of the evening was the fried brussels sprouts about it was true ! the room was small , but very clean and , which i would very much like the recipe for so i can unk comfortable . the staff was fantastic and friendly , very helpful myself on them every night . those babies , topped with crispy and the breakfast every day was wonderful ! we loved the crumbled bits of unk , capers and walnuts , were pure bliss . and let ’s be honest , how often do you really have the opportunity to call unk sprouts unk ? probably not very often . i unk off the evening with a glass of unk scotch , neat . perfection . PICO Domain listening, respondent, perceived, motivation, attitude, participant, asking, apprehension, helpfulness, antismoking virtual, prototype, smart, tracking, internally, locked, haptic, handle, autism, autistic psychosocial, gluconate, mental, prospective, social, genetic, psychological, environmental, sulphadoxine, reproductive regain, obviously, drive, relieved, promoted, bid, diego, mdi, bocs, rescue workplace, caring, carers, helpfulness, occupational, awareness, spiritual, motivating, personal, educational euro, biochemical, virological, effectivity, yearly, oxidation, dos, reversion, quitter, audiologic semen, infertile, uncircumcised, sperm, fertility, azoospermia, fe, ejaculation, caput, particulate obstetrician, radiologist, cardiologist, technician, physician, midwife, nurse, gynecologist, physiotherapist, anaesthetist nonsignificant, bias, unadjusted, quadratic, constraint, apgar, undue, trend, nonstatistically, ckd dependency, dependence, comorbid, dependent, type, sensitivity, insensitive, manner, specific, comorbidity stem, progenitor, resident, concomitant, recruiting, mobilized, promoted, precursor, malondialdehyde, reduces anemic, cardiorespiratory, normothermic, hemiplegic, thromboelastography, hematological, hemodilution, dyslipidemic premenopausal, locoregional, postmenopausal, menopausal, cancer, nonmetastatic, nsclc, pelvic, tirilazad, operable conscious, intubated, anaesthetized, terlipressin, corticotropin, ventilated, resuscitation, anesthetized, resuscitated, vasopressin quasi, intergroup, midstream, static, voluntary, csa, vienna, proprioceptive, stroop, multinational trachea, malnourished, intestine, diaphragm, semitendinosus, gastrocnemius, undernourished, pancreas, lung, nonanemic pound, mile, customized, pack, per, hundred, liter, oz, thousand, litre lying, correlated, activity, matter, functioning, transported, gut, subscale, phosphodiesterase, impacting count, admission, disparity, spot, parasitaemia, visit, quartile, white, age, wbc Yelp!-TripAdvisor Domain like, hate, grow, love, coming, come, hygienic, desperate, reminds, unsanitary, complainer food, service, restaurant, resturants, steakhouse, restaraunt, tapa, eats, restraunts, ate, eatery bunny, chimney, ooze, sliver, masked, adorned, cow, concrete, coating, triangle, beige cause, get, brain, nt, annoying, guess, time, sometimes, anymore, cuz, hurry week, bosa, month, today, california, visited, newton, bought, yogurtland, exellent, recent caring, outgoing, demeanor, gm, personable, engaging, employee, respectful, interaction, associate, incompetent staff, excellent, personel, good, unseasoned, superb, unremarkable, excellant, tasty, personnel, unfailingly place, sounding, scenery, cuppa, temptation, kick, indulge, confection, coffeehouse, lavender sweetened, watery, raisin, flavoring, doughy, mmmm, crumble, hazelnut, consistency, lemonade, powdered upgrade, premium, room, staff, offer, amenity, perk, suite, whereas, price, pricing walk, lined, directed, nearest, headed, adjacent, gaze, grab, lovely, surrounded, field time, afternoon, night, day, moment, evening, went, rushed, place, boyfriend, came hotel, property, travelodge, radisson, metropolitan, intercontinental, lodging, ibis, sofitel, novotel, doubletree vegetable, dish, herb, premade, delicacy, chopped, masa, lentil, tamale, canned, omlettes stay, staying, return, trip, vacation, honeymoon, visit, outing, revisit, celebrate, stayed great, awesome, amazing, excellent, fantastic, phenomenal, exceptional, nice, outstanding, wonderful, good room, cupboard, bedcover, drapery, bathroom, dusty, carpeting, laminate, ensuite, washroom, bedspread utilized, day, excercise, recycled, tp, liner, borrowed, depot, vanished, restocked argued, blamed, demanded, called, registered, referred, transferred, contacted, claiming, denied, questioned neck, bruise, suspended, souvenir, tragedy, godiva, depot, blazing, peice, investigating BeerAdvocate Dataset shade, ruddy, transluscent, yellowish, translucent, hue, color, foggy, pours, branded recommend, like, disturb, conform, recomend, suggest, consider, imagine, liken, resemble longneck, swingtop, tapered, greeted, squat, ml, stubby, getting, stubbie, oz earthy, toffee, fruit, fruity, molasses, herbal, woody, floral, pineapple, raisin scarily, faster, quicker, obliged, inch, trace, warranted, mere, frighteningly, wouldve dubbel, quad, doppelbock, weizenbock, witbier, barleywine, tripel, dopplebock, brew, dipa lack, chestnut, purple, mahogany, nt, weakness, burgundy, coppery, fraction, garnet offering, product, beer, stuff, brew, gueuze, saison, porter, geuze, news mich, least, bass, pacifico, sumpin, inferior, grolsch, westy, yanjing, everclear foam, disk, head, heading, buts, edging, remains, lacing, froth, liquid starbucks, coffe, pepsi, expresso, kahlua, cofee, liqour, esspresso, cuban, grocery roasty, hop, chocolatey, hoppy, chocolaty, roasted, roast, robust, espresso, chocolately bodied, body, feel, barley, mouthfeel, tone, enjoyably, texture, moutfeel, mothfeel lasted, crema, flute, head, retentive, smell, fing, duper, compact, retention nt, rather, almost, never, kinda, fiddle, manner, easier, memphis, hard wafted, jumped, onset, proceeds, consisted, rife, consists, lotsa, citrisy, consisting hop, hoppy, assertive, inclined, tempted, doubtful, manageable, bearable, importantly, prominant great, good, decent, crazy, fair, killer, ridiculous, serious, insane, tremendous beer, brew, aperitif, nightcap, simply, intro, truly, ale, iipa, aipa wayne, harrisburg, massachusetts, arizona, jamaica, lincoln, kc, oklahoma, adult, odd

Table 2: Top words generated for each topic using ABAE method