Table S1: Sensitivity, specificity, PPV, NPV, and F1 score of NLP vs. ICD for identification of symptoms for (A) BioMe development cohort; (B) BioMe validation cohort; (C) MIMIC-III; (D) 1 year of notes from patients in BioMe calculated using manual chart review. A) Fatigue Nausea and/or vomiting Anxiety Depression NLP (95% ICD (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P CI) 0.99 (0.93- 0.59 (0.43- <0.00 0.25 (0.12- <0.00 <0.00 0.54 (0.33- Sensitivity 0.99 (0.9 – 1) 0.98 (0.88 -1) 0.3 (0.15-0.5) 0.85 (0.65-96) 0.02 1) 0.73) 1 0.42) 1 1 0.73) 0.57 (0.29- 0.9 (0.68- Specificity 0.89 (0.4-1) 0.75 (0.19-1) 0.68 0.97 (0.77-1) 0.03 0.98 (0.83-1) 0.22 0.81 (0.53-0.9) 0.96 (0.79-1) 0.06 0.82) 0.99) 0.99 (0.92- 0.86 (0.71- 0.94 (0.79- 0.79 (0.59- PPV 0.96 (0.82-1) 0.3 0.95 (0.66-1) 0.02 0.95 (0.66-1) 0.16 0.93 (0.68-1) 0.12 1) 0.95) 0.99) 0.92) 0.13 (0.03- <0.00 0.49 (0.33- <0.00 0.66 (0.48- NPV 0.89 (0.4-1) 0.007 0.94 (0.63-1) 0.34 (0.2-0.51) 0.97 (0.81-1) 0.86 (0.6-0.95) 0.04 0.35) 1 0.65) 1 0.81) <0.00 <0.00 <0.00 F1 Score 0.99 0.83 0.88 0.57 0.95 0.63 0.82 0.79 0.002 1 1 1 Itching Cramp Pain NLP (95% ICD (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P CI) 0.98 (0.86- 0.24 (0.09- <0.00 0.09 (0.01- <0.00 0.52 (0.37- <0.00 Sensitivity 0.98 (0.85-1) 0.99 (0.93-1) 1) 0.45) 1 0.29) 1 0.66) 1 0.89 (0.72- 0.5 (0.37- Specificity 0.96 (0.8-1) 0.98 (0.86-1) 0.68 0.98 (0.88-1) 0.18 0.5 (0-1) 1 0.98) 0.66) 0.88 (0.69- PPV 0.96 (0.8-1) 0.8 (0.54-1) 0.32 0.8 (0.16-1) 0.22 0.99 (0.93-1) 0.98 (0.87-1) NA* 0.97) 0.98 (0.85- 0.57 (0.41- <0.00 0.58 (0.43- <0.00 NPV 0.98 (0.86-1) 0.5 (0-1) 0.02 (0-0.08) NA* 1) 0.72) 1 0.72) 1 <0.00 <0.00 <0.00 F1 Score 0.97 0.56 0.91 0.28 0.99 0.68 1 1 1 *Denotes 95% confidence intervals and P values that could not be calculated due to insufficient cells in 2x2 tables. B) Fatigue Nausea and/or vomiting Anxiety Depression NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P 0.71 0.93 (0.78- 0.33 0.14 0.78 (0.52- 0.44 Sensitivity 0.99 (0.92-1) 0.51 0.54 0.91 (0.71-0.99) 1 0.09 (0.56-0.85) 0.99) (0.17-0.51) (0.03-0.32) 0.94) (0.19-0.64) 0.75 (0.19- 0.5 0.81 (0.54- 0.74 1 0.79 (0.59- 1 Specificity 1 1 0.83 (0.63-0.95) 0.4 0.5 0.99) (0.09-0.99) 0.96) (0.68-1) (0.85-1) 0.92) (0.87-1) 0.94 0.9 (0.74- 0.91 1 1 PPV 0.98 (0.88-1) 0.21 0.95 0.83 (0.63-0.95) 0.12 0.7 (0.46-0.88) 0.006 (0.84-1) 0.98) (0.59-1) (0.29-1) (0.63-1) 0.14 0.87 (0.60- 0.43 0.56 0.85 (0.65- 0.74 NPV 0.86 (0.29-1) (0.02-0.43) 0.02 <0.001 0.91 (0.71-0.99) <0.001 0.06 0.98) (0.24-0.58) (0.35-0.67) 0.96) (0.51-0.83) F1 Score 0.98 0.81 0.92 0.48 0.87 0.25 0.74 0.61 Itching Cramp Pain NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P 0.26 0.13 0.39 Sensitivity 0.97 (0.82-1) 0.50 0.98 (0.86-1) 0.3 0.99 (0.91-1) 0.82 (0.06-0.37) (0.02-0.27) (0.24-0.53) 0.67 (0.46- 1 0.77 (0.55- 1 0.8 Specificity 0.73 0.53 0.2 (0.005-0.72) 1 0.83) (0.81-1) 0.92) (0.8-1) (0.03-1) 0.68 (0.48- 1 0.83 (0.64- 1 0.94 PPV 0.02 0.11 0.91 (0.79-0.98) 0.56 0.84) (0.48-1) 0.94) (0.29-1) (0.8-1) 0.66 0.51 0.14 NPV 0.97 (0.81-1) <0.001 0.97 (0.80-1) <0.001 0.67 (0.03-1) 0.29 (0.28-0.6) (0.25-0.56) (0-0.18) F1 Score 0.80 0.41 0.90 0.03 0.95 0.55 C) Fatigue Nausea and/or vomiting Anxiety Depression ICD (95% NLP (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P CI) 0.04 (0- 0.82 (0.6- 0.43 (0.18- Sensitivity 0.98 (0.86-1) <0.001 0.05 (0-0.23) <0.001 0.96 (0.78-1) 0.13 (0.03-0.34) <0.001 0.71 (0.42-0.92) 0.22 0.21) 0.95) 0.71) 0.98 (0.87- 0.86 (0.67- Specificity 0.77 (0.56-0.91) 0.03 0.98 (0.88-1) 0.1 0.93 (0.76-1) 0.98 (0.87-1) 0.34 0.81 (0.64-0.92) 0.99 (0.9-1) 0.02 1) 0.96) 0.67 (0.03- 0.82 (0.6- PPV 0.8 (0.61-0.92) NA* 0.67 (0.03-1) 0.39 0.92 (0.73-0.99) 0.86 (0.29-1) 0.35 0.59 (0.33-0.82) 0.92 (0.54-1) NA* 1) 0.95) 0.53 (0.38- 0.86 (0.67- 0.57 (0.42- 0.82 (0.67- NPV 0.98 (0.83-1) NA* <0.001 0.96 (0.8-1) 0.57 (0.42-0.72) <0.001 0.88 (0.72-0.97) NA* 0.67) 0.96) 0.71) 0.92) F1 Score 0.88 0.08 0.09 0.82 0.09 0.003 0.94 0.23 <0.001 0.65 0.59 Itching Cramp Pain ICD (95% NLP (95% CI) P NLP (95% CI) ICD (95% CI) P NLP (95% CI) ICD (95% CI) P CI) 0.2 (0.16- Sensitivity 0.8 (0.16-1) 0.34 0.5* 0.5* NA* 0.99 (0.91-1) 0.07 (0.02-0.2) <0.001 1) 0.99 (0.93- Specificity 0.98 (0.89-1) 0.68 0.96 (0.86-1) 0.99 (0.93-1) 0.34 0.11 (0-0.48) 0.95 (0.66-1) <0.001 1) 0.2 (0.01- PPV 0.67 (0.09-0.99) 0.5* 0.26 0.5* NA* 0.84 (0.7-0.93) 0.86 (0.29-1) 0.19 0.99) 0.96 (0.86- NPV 0.99 (0.92-1) 0.04 0.99 (0.93-1) 0.99 (0.93-1) NA* 0.67 (0.03-1) 0.19 (0.09-0.33) 0.3 1) F1 Score 0.72 0.29 <0.001 0.29 0.5 1 0.91 0.13 0.24 *Denotes 95% confidence intervals and P values that could not be calculated due to insufficient cells in 2x2 tables. D) Fatigue Nausea and /or vomiting Anxiety Depression NLP (95% CI) ICD (95% CI) NLP (95% CI) ICD (95% CI) NLP (95% CI) ICD (95% CI) NLP (95% CI) ICD (95% CI) Sensitivity 1 (0.79-1) 0.25 (0.07-0.52) 0.96 (0.72-1) 0.09 (0.002 – 0.41) 0.91 (0.5-1) 0.09 (0-0.33) 0.70 (0.35-0.93) 0.20 (0.03-0.56) Specificity 0.79 (0.49-0.95) 0.97 (0.77-1) 0.89 (0.67-0.99) 0.97 (0.82-1) 0.91 (0.74-0.99) 0.98 (0.86-1) 0.98 (0.83-1) 1.00 (0.83-1) PPV 0.84 (0.6-0.97) 0.89 (0.4-1) 0.85 (0.55-0.98) 0.67 (0.03-1) 0.71 (0.29-0.96) 0.5 (0-1) 0.93 (0.59-1) 1.00 (0.16-1) NPV 1 (0.72-1) 0.54 (0.33-0.73) 0.97 (0.8-1) 0.66 (0.42-0.79) 0.98 (0.85-1) 0.83 (0.65-0.94) 0.87 (0.66-0.97) 0.71 (0.51-0.87) F1 Score 0.91 0.39 0.9 0.16 0.8 0.15 0.8 0.32 Itching Cramp Pain NLP (95% CI) ICD (95% CI) NLP (95% CI) ICD (95% CI) NLP (95% CI) ICD (95% CI) Sensitivity 0.93 (0.59 -1) 0.14 (0.004-0.58) 0.91 (0.48-1) 0.2 (0.005-0.72) 0.96 (0.78-1) 0.09 (0.01-0.28) Specificity 0.98 (0.85-1) 1.00 (0.85-1) 0.98 (0.86-1) 0.98 (0.86-1) 0.57 (0.18-0.9) 0.93 (0.59-1) PPV 0.93 (0.59-1) 1.00 (0.03-1) 0.91 (0.48-1) 0.67 (0.03-1) 0.88 (0.69-0.97) 0.8 (0.16-1) NPV 0.98 (0.82-1) 0.79 (0.6-0.92) 0.98 (0.86-1) 0.86 (0.68-0.96) 0.80 (0.28-0.99) 0.25 (0.11-0.45) F1 Score 0.93 0.25 0.91 0.31 0.92 0.16 Table S2: ICD 9 and 10 codes for identification of A) hemodialysis and peritoneal dialysis patients; B) comorbidities; and C) symptoms.