Web Services Programmatic access to EMBL-EBI resources Andrew Cowley [email protected] [email protected] About me • Bioinformatics Trainer • External Services Team • Website infrastructure • Tool framework • User support 29 October 2012 Andrew Cowley - External Services 2 EMBL-EBI has many services 29 October 2012 Andrew Cowley - External Services 3 And the website has nice interfaces 29 October 2012 Andrew Cowley - External Services 4 Web interfaces are great • For testing a hypothesis • For training • For publishing :-) 29 October 2012 Andrew Cowley - External Services 5 But what if.. • I want to examine every sequence in a proteome? • My data needs repetitive analysis? More people? 29 October 2012 Andrew Cowley - External Services 6 What are my options? • Download the programs to my own computer/server • Download the data to my own computer/server • Complete flexibility • Run your own programs/data 29 October 2012 Andrew Cowley - External Services 7 Some databases… • ENA/EMBL-Bank (nucleotide) Not to mention… • 250+ million entries • Array experiments • UniProt (protein sequence) • 26 million entries • Small molecules • PDBe (protein structure) • Pathways • 85 thousand entries • Ontologies • InterPro (protein families and motifs) • 24 thousand entries • Literature 29 October 2012 Andrew Cowley - External Services 8 And they’re growing.. 29 October 2012 Andrew Cowley - External Services 9 And they’re growing.. 29 October 2012 Andrew Cowley - External Services 10 And they’re growing.. 29 October 2012 Andrew Cowley - External Services 11 • Big storage • Big compute • Keeping it all up-to-date 29 October 2012 Andrew Cowley - External Services 12 Web Services • Remotely run your tasks on our servers + data • Programmatic interface • SOAP and REST APIs • Simple • Robust • Standard • Can incorporate into workflows 29 October 2012 Andrew Cowley - External Services 13 Features • Multiple options to analyse the results • Standard raw output • Interactions with external applications • Helpful visual representations 29 October 2012 Andrew Cowley - External Services 14 Web Services – what’s available? • ~100+ Tools • Sequence similarity search (BLAST, FASTA, PSI-BLAST, SSEARCH etc.) • Multiple sequence alignments (Clustal Omega, DbClustal, MUSCLE, Prank etc.) • Protein functional analysis (InterProScan, Phobius) • Pairwise sequence alignments (LALIGN, EMBOSS Needle, Stretcher, Matcher, Water) • Phylogeny • Sequence translation • Data retrieval • DbFetch – access to 47 database categories • Search • EBI Search – pull back information from fields within ~470 million entries 29 October 2012 Andrew Cowley - External Services 15 Popular! Tool job submissions Avg. 1.3m per month Avg. 4m per month 29 October 2012 Andrew Cowley - External Services 16 How? • Well documented • http://www.ebi.ac.uk/Tools/webservices • Pre-compiled clients available for a large variety of languages • SOAP • WSDL: http://www.ebi.ac.uk/Tools/services/soap/{tool}?wsdl • Most programming languages have SOAP client libraries • Generate stubs or dynamically call methods • REST • http://www.ebi.ac.uk/Tools/services/rest/{tool}/{method}/{params} • Basic HTTP requests • Web browser, HTTP client libraries, CURL… 29 October 2012 Andrew Cowley - External Services 17 Only 3 simple steps... • List parameters • Get parameter details Meta- → Name, description, values... Information Input parameters • Run (Email, title, values...) → Job Identifier • Check status Submission → RUNNING, FINISHED, ERROR... Job identifier (e.g. iprscan-S20110708-094729-0726-35857540-pg) • List results available → Name, description, media type... Results • Get result analysis → Output, text, binaries (images)... 29 October 2012 Andrew Cowley - External Services 18 Example uses • Produce orthologous alignments and trees (BLAST → Identifiers → ClustalW/Omega → True phylogenies) • Study protein-protein interactions • Tie in EBI Search and link BLAST/InterProScan results with literature citations • Compatible with YABI, GALAXY, Taverna, Triana, Membrane... • Pre-designed workflows at myExperiment 29 October 2012 Andrew Cowley - External Services 19 Acknowledgements • Funding • European Union (FELICS Research Infrastructure; EMBRACE project; ORIEL Project) • Wellcome Trust • National Institutes of Health (UniProt Project) • European Molecular Biology Laboratory • Special thanks • External Services and other groups at EMBL-EBI • Tool authors and collaborators http://www.ebi.ac.uk/Tools/webservices 29 October 2012 Andrew Cowley - External Services 20 29 October 2012 Andrew Cowley - External Services 21 .