8th European HIV Drug Resistance Workshop

M. Zazzi on behalf of the EuResist Network (www.euresist.org)

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, EuResist status

 Funded by the EU JAN-06 to JUN-08, then set as a European Network (legal entity)

 Data collected from ~40,000 patients (Italy, , , Luxembourg, Belgium, Spain)

 Data modeling by IBM , Max Planck Institute for Informatics, Informa & Rome TRE University

 Several methods investigated, much effort on feature selection and derivation

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist – TCE definition

CD4 Viral load Genotype Treatment switch Viral load

time 0 to 12 weeks Short-term model: 4-12 weeks Pre-therapy HIV RNA Patient demographics (age, gender, race, route of ) Past AIDS diagnosis Past treatments Past genotypes

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist – labeling therapies

Baseline data HIV genotype at 0 to 12 weeks before treatment VL at 0 to 12 weeks before treatment Additional variables when available

Treatment switch

VL at 4 to 12 weeks (8-week outcome)

SUCCESS FAILURE Undetectable or >2 log Detectable and not >2 decrease VL log decrease VL

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist – engines

 Three prediction engines developed independently  Generative-Discriminative (by IBM)  Evolutionary (by Max-Planck Institute)  Mixed effects (by Rome TRE & Informa)

 Then, combined into a unique engine and made freely available on the web

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy The EuResist combined engine

3143 therapies, Short-term outcome (8 weeks)

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy Rosen-Zvi, Bioinformatics 2008 EuResist vs. Expert interpretation (EVE study)

Form the invitation letter:

The requested response include a categorical (C) answer and a quantitative (Q) estimate:

C) Given this HIV genotype and patient information, will the indicated therapy be successful (i. e. will it make HIV RNA decrease

by at least 2 logs or to undetectable levels in 8 weeks) ?

Q) Given this HIV genotype and patient information, what probability of success would you estimate for the indicated therapy?

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist vs. Expert interpretation (EVE study)

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy Distribution of the 25 therapies by year and type

5 4 No. of 3 cases 2 1 1998 1999 2000 2001 2002 2003 2004 2005 2006

Year of therapy

NRTI-only therapy NNRTI-based therapy *PI-based therapy

*17 cases with boosted PI, 2 unboosted ATV, 1 NFV Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy

Patient characteristics

FEATURE MEDIAN (IQR) Baseline log viral load 4.67 (4.38-4.99) Baseline CD4 counts 298 (134-412) Number of previous treatment lines 5 (3-6) Number of NRTI mutations at baseline 3 (3-4) Number of NNRTI mutations at baseline 1 (0-2) Number of PI mutations at baseline 2 (0-3)

Number of available previous viral load data 15 (8-25) Number of available previous CD4 counts 14 (10-30) Number of available previous genotypes 1 (0-3)

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy Distribution of wrong and correct calls

25 HAART cases randomly selected form the EuResist db:

•Obsolete therapies excluded

•Wild type genotype excluded •All clinical and virological information available

12 experts enrolled, response obtained from 10:

•On‐line anonymous rating •Only European (E) vs. non‐ European (N) setting traceable

•Use of any interpretation system allowed (and declared)

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy Sensitivity, specificity and accuracy

76%

Specificity = ability to call failure Sensitivity = ability to call success

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy 100

90

80

70

60 Average Expert Best Expert 50 EuResist Worst Expert Sensitivity 40

30

20

10

0 0 10 20 30 40 50 60 70 80 90 100 100-Specificity

AUC 95% CI Best Expert 0.853 0.655 - 0.961

EuResist 0.787 0.578 - 0.923 P = 0.011 Average Expert 0.777 0.567 - 0.917

Worst Expert 0.653 0.438 - 0.830

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist vs. Expert interpretation (EVE study)

Correlation between EuResist and avg(expert) probability of success Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist vs. Expert interpretation (EVE study)

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist vs. Expert interpretation (EVE study) EuResist & most experts incorrect

EXPERT 1 EXPERT 2 EXPERT 3 EXPERT 4 EXPERT 5 EXPERT 6 EXPERT 7 EXPERT 8 EXPERT 9 EXPERT 10 EuResist

ACTUAL OUTCOME F S F S F F F S S S S S S S S S F S F F S S F F S Incorrect prediction F Treatment failure S Treatment success

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist vs. Expert interpretation (EVE study) Unexpected drug efficacy

EXPERT 1 EXPERT 2 EXPERT 3 EXPERT 4 EXPERT 5 EXPERT 6 EXPERT 7 EXPERT 8 EXPERT 9 EXPERT 10 EuResist

ACTUAL OUTCOME F S F S F F F S S S S S S S S S F S F F S S F F S Incorrect prediction F Treatment failure S Treatment success

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy EuResist vs. Expert interpretation (EVE study) Adherence issues?

EXPERT 1 EXPERT 2 EXPERT 3 EXPERT 4 EXPERT 5 EXPERT 6 EXPERT 7 EXPERT 8 EXPERT 9 EXPERT 10 EuResist

ACTUAL OUTCOME F S F S F F F S S S S S S S S S F S F F S S F F S Incorrect prediction F Treatment failure S Treatment success

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy Why do we err (so much)?

 Limitations in the definition of success can “create” prediction errors  Expected adherence cannot be accounted for based on presently available training data set  Genotype shortcomings  Impact of past mutations  Short-term drug activity on partially resistant strains (e. g. 215 revertants, Y181C with EFV)  Unweighted factors (host genetics)

Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy Perspectives

 EuResist is being updated regularly and expanding  Other European countries  Non-European countries

 Engines being retrained with the expanded dataset (~40,000 patients)  Small increase in accuracy ○ Upper bound of accuracy already reached?  Tipranavir, Darunavir, Etravirine now computed Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy If you have data and data sharing attitude, please consider becoming a EuResist partner!

www.euresist.org Presented at the 8th European HIV Drug Resistance Workshop, March 17-19 2010, Sorrento, Italy