DOCSLIB.ORG

Web Workflows for Data Mining in the Cloud

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Web Workflows for Data Mining in the Cloud WEB WORKFLOWS FOR DATA MINING IN THE CLOUD Janez Kranjc Doctoral Dissertation Jožef Stefan International Postgraduate School Ljubljana, Slovenia Supervisor: Prof. Dr. Nada Lavrač, Jožef Stefan Institute, Ljubljana, Slovenia, and Jožef Stefan International Postgraduate School, Ljubljana, Slovenia Co-Supervisor: Assoc. Prof. Dr. Marko Robnik Šikonja, Faculty of Computer and Infor- mation Science, University of Ljubljana, Slovenia Evaluation Board: Assist. Prof. Dr. Martin Žnidaršič, Chair, Jožef Stefan Institute, Ljubljana, Slovenia, and Jožef Stefan International Postgraduate School, Ljubljana, Slovenia Assoc. Prof. Dr. Igor Mozetič, Member, Jožef Stefan Institute, Ljubljana, Slovenia, and Jožef Stefan International Postgraduate School, Ljubljana, Slovenia Prof. Dr. Hendrik Blockeel, Member, Kaotolieke Universiteit Leuven, Leuven, Belgium Janez Kranjc WEB WORKFLOWS FOR DATA MINING IN THE CLOUD Doctoral Dissertation SPLETNI DELOTOKI ZA RUDARJENJE PODATKOV V OBLAKU Doktorska disertacija Supervisor: Prof. Dr. Nada Lavrač Co-Supervisor: Assoc. Prof. Dr. Marko Robnik Šikonja Ljubljana, Slovenia, March 2017 v Acknowledgments This thesis would have never been finished without the help of a number of people. Their support and help throughout my studies are greatly appreciated. First of all I would like to thank my supervisor Nada Lavrač for her infinite amount of patience, enthusiasm, support and her always helpful and insightful research ideas and comments. I am also very grateful to my co-supervisor Marko Robnik Šikonja for the copious amount of time spent reviewing my work and his greatly appreciated comments and ideas. I would also like to thank the members of my evaluation board Martin Žnidaršič, Igor Mozetič and Hendrik Blockeel for reading my thesis and providing comments for improvement. I also wish to thank the funding bodies that financially supported my research: the Department of Knowledge Technologies at the Jožef Stefan Institute, the Jožef Stefan International Postgraduate School, and the European Commission for funding the research projects e-LICO, BISON, FIRST, ConCreTe, MUSE, and HBP in which I was involved. Special thanks goes to Vid Podpečan for his work that helped spark the ideas that eventually became this dissertation and his contributions to my work and all his helpful insights and ideas. I would also like to thank everyone I had the pleasure of cooperating and co-authoring papers with, especially: Matic Perovšek for the work we did together on TextFlows; Jasmina Smailović for our work on sentiment analysis and active learning; Roman Orač for implementing the DiscoMLL library that contributed to big data process- ing in ClowdFlows; and Anže Vavpetič for his numerous contributions and creating the Relational Data Mining package. I am thankful to all the work colleagues at the Department of Knowledge Technologies for creating a great working environment. I would like to thank Darko Aleksovski for his numerous contributions to the open-source projects developed during the time of my studies. Thanks also to Jan Kralj, Borut Sluban and Miha Grčar for their valuable time for productive and insightful discussions that greatly motivated my research work. Last but not least, I must thank my family and friends for their encouragement and support. Finally, I owe sincere and earnest thankfulness to my wife Nina for her love, understanding, support and patience throughout my study. vii Abstract The thesis addresses the development of an innovative data mining platform ClowdFlows and novel knowledge discovery scenarios implemented therein as executable data mining workflows. The ClowdFlows platform is implemented as a cloud-based web application with a graphical user interface which supports the construction, execution and sharing of data mining workflows. Big data analytics is supported by several algorithms, including novel ensemble techniques implemented using the MapReduce programming model, and a special stream mining module for real-time analysis using continuous parallel workflow execution. The adaptability of ClowdFlows is demonstrated through descriptions of two platforms that expand the usage of the web-based workflow environment to fields other than data mining. The ConCreTeFlows platform is a novel platform for computational creativity, while TextFlows is focused mainly on text mining and natural language processing. The ConCreTeFlows platform is demonstrated with a conceptual blending use case, while fea- tures of TextFlows are demonstrated on three use cases: comparison of document classifiers and different part-of-speech taggers on a text categorization problem, and outlier detection in document corpora. We present novel use cases in Inductive Logic Programming (ILP) and Relational Data Mining (RDM). The main novelty is a propositionalization technique called wordification which can be seen as a transformation of a relational database into a corpus of text docu- ments. The wordification methodology and the evaluation procedure are implemented as executable workflows in ClowdFlows. The implemented workflows include several other ILP and RDM algorithms, as well as the utility components that enable access to these techniques to a wider research audience. The real-time analysis and data stream mining capabilites are demonstrated on a novel active learning scenario for dynamic adaptive sentiment analysis, which is able to handle changes in data streams and adapt its behavior over time. Established stream mining techniques are transferred to the visual programming paradigm and demonstrated through the sentiment analysis use case, using active learning of microblogging data streams with a linear Support Vector Machine. The thesis contributes to open-source scientific software. The ClowdFlows platform, its adaptations ConCreTeFlows and TextFlows, and all the described use cases are publicly available. This enables experiment reproducibility, as well as workflow adaptations and enhancements. ix Povzetek Disertacija obravnava razvoj inovativne platforme za podatkovno rudarjenje, imenovane ClowdFlows, ter nove načine odkrivanja znanja, implementirane v platformi v obliki izvr- šljivih delotokov. Platforma ClowdFlows je izdelana kot spletna aplikacija v oblaku, njen grafični uporabniški vmesnik pa omogoča izdelavo, izvedbo in objavo delotokov za podat- kovno rudarjenje. Analiza velikih podatkov je mogoča s pomočjo več algoritmov, med drugim z inovativnimi ansambelskimi tehnikami, ki uporabljajo paradigmo MapReduce, in z modulom rudarjenja podatkovnih tokov v realnem času, ki uporablja neprekinjeno vzporedno izvajanje delotokov. Prilagodljivost platforme ClowdFlows je prikazana z opisom dveh izpeljanih platform, ki širita uporabnost delotokov spletne platforme na druga področja. Platforma ConCreTe- Flows je namenjena računalniški ustvarjalnosti, medtem ko se platforma TextFlows osredo- toča na tekstovno rudarjenje in procesiranje naravnega jezika. Platforma ConCreTeFlowa je prikazana s primerom uporabe konceptualnega povezovanja, medtem ko so posebnosti platforme TextFlows prikazane na treh primerih uporabe: s primerjavo klasifikatorjev do- kumentov in različnih označevalcev besednih vrst ter z odkrivanjem osamelcev v zbirkah besedil. V disertaciji so predstavljeni novi primeri uporabe na področjih induktivnega logičnega programiranja in relacijskega podatkovnega rudarjenja. Novost je metoda transformacije relacijskih podatkovnih baz v zbirke besedil, imenovana wordification (besedizacija). Eval- vacija razvite metode je vključena v platformo ClowdFlows kot izvršljiv delotok. Poleg opisane evalvacije so kot delotoki dostopni tudi drugi algoritmi za relacijsko podatkovno rudarjenje in induktivno logično programiranje, prav tako pa tudi pomožne komponente, ki omogočajo dostopnost teh tehnologij širši javnosti. Možnosti analiz v realnem času in rudarjenje podatkovnih tokov je prikazano na pri- meru uporabe aktivnega učenja za dinamično prilagajanje analize sentimenta v podat- kovnih tokovih. Uveljavljene tehnike za rudarjenje podatkovnih tokov smo uporabili po načelu vizualnega programiranja in prikazali na primeru uporabe analize sentimenta na podatkovnih tokovih kratkih spletnih sporočil. Disertacija prispeva tudi k odprtokodni programski opremi. Vsi delotoki in izvorna koda opisanih primerov uporabe, platform ClowdFlows, ConCreTeFlows in TextFlows so javno dostopni. S tem omogočimo ponovljivost eksperimentov in prilagoditve ter izboljšave delotokov. xi Contents Abbreviations xiii 1 Introduction1 1.1 Problem Definition................................ 1 1.2 Purpose of the Dissertation ........................... 2 1.3 Goals of the Dissertation............................. 3 1.4 Hypotheses .................................... 4 1.5 Scientific Contributions ............................. 4 1.6 Dissemination of the Developed Software.................... 6 1.7 Organization of the Thesis............................ 7 2 The ClowdFlows Platform9 2.1 Data Mining Platforms.............................. 9 2.2 Big Data Mining ................................. 10 2.2.1 Batch data processing .......................... 11 2.2.2 Data stream mining ........................... 12 2.3 Development of the ClowdFlows Platform................... 12 2.4 Related Publication ............................... 13 3 Adaptations of ClowdFlows 37 3.1 Creating an Adaptation of the ClowdFlows Platform............. 37 3.1.1 Setting up a development version.................... 37 3.1.2 Creating ClowdFlows widgets and packages.............. 38 3.1.3 ClowdFlows Unistra ..........................
Load more

Recommended publications
  • The Design and Realisation of the Myexperiment Virtual Research Environment for Social Sharing of Workflows
    The Design and Realisation of the myExperiment Virtual Research Environment for Social Sharing of Workflows David De Roure a Carole Goble b Robert Stevens b aUniversity of Southampton, UK bThe University of Manchester, UK Abstract In this paper we suggest that the full scientific potential of workflows will be achieved through mechanisms for sharing and collaboration, empowering scientists to spread their experimental protocols and to benefit from those of others. To facilitate this process we have designed and built the myExperiment Virtual Research Environ- ment for collaboration and sharing of workflows and experiments. In contrast to systems which simply make workflows available, myExperiment provides mecha- nisms to support the sharing of workflows within and across multiple communities. It achieves this by adopting a social web approach which is tailored to the par- ticular needs of the scientist. We present the motivation, design and realisation of myExperiment. Key words: Scientific Workflow, Workflow Management, Virtual Research Environment, Collaborative Computing, Taverna Workflow Workbench 1 Introduction Scientific workflows are attracting considerable attention in the community. Increasingly they support scientists in advancing research through in silico ex- perimentation, while the workflow systems themselves are the subject of ongo- ing research and development (1). The National Science Foundation Workshop on the Challenges of Scientific Workflows identified the potential for scientific advance as workflow systems address more sophisticated requirements and as workflows are created through collaborative design processes involving many scientists across disciplines (2). Rather than looking at the application or ma- chinery of workflow systems, it is this dimension of collaboration and sharing that is the focus of this paper.
    [Show full text]
  • Data Curation+Process Curation^Data Integration+Science
    BRIEFINGS IN BIOINFORMATICS. VOL 9. NO 6. 506^517 doi:10.1093/bib/bbn034 Advance Access publication December 6, 2008 Data curation 1 process curation^data integration 1 science Carole Goble, Robert Stevens, Duncan Hull, Katy Wolstencroft and Rodrigo Lopez Submitted: 16th May 2008; Received (in revised form): 25th July 2008 Abstract In bioinformatics, we are familiar with the idea of curated data as a prerequisite for data integration. We neglect, often to our cost, the curation and cataloguing of the processes that we use to integrate and analyse our data. Downloaded from https://academic.oup.com/bib/article/9/6/506/223646 by guest on 27 September 2021 Programmatic access to services, for data and processes, means that compositions of services can be made that represent the in silico experiments or processes that bioinformaticians perform. Data integration through workflows depends on being able to know what services exist and where to find those services. The large number of services and the operations they perform, their arbitrary naming and lack of documentation, however, mean that they can be difficult to use. The workflows themselves are composite processes that could be pooled and reused but only if they too can be found and understood. Thus appropriate curation, including semantic mark-up, would enable processes to be found, maintained and consequently used more easily.This broader view on semantic annotation is vital for full data integration that is necessary for the modern scientific analyses in biology.This article will brief the community on the current state of the art and the current challenges for process curation, both within and without the Life Sciences.
    [Show full text]
  • Data Management in Systems Biology I
    Data management in systems biology I – Overview and bibliography Gerhard Mayer, University of Stuttgart, Institute of Biochemical Engineering (IBVT), Allmandring 31, D-70569 Stuttgart Abstract Large systems biology projects can encompass several workgroups often located in different countries. An overview about existing data standards in systems biology and the management, storage, exchange and integration of the generated data in large distributed research projects is given, the pros and cons of the different approaches are illustrated from a practical point of view, the existing software – open source as well as commercial - and the relevant literature is extensively overviewed, so that the reader should be enabled to decide which data management approach is the best suited for his special needs. An emphasis is laid on the use of workflow systems and of TAB-based formats. The data in this format can be viewed and edited easily using spreadsheet programs which are familiar to the working experimental biologists. The use of workflows for the standardized access to data in either own or publicly available databanks and the standardization of operation procedures is presented. The use of ontologies and semantic web technologies for data management will be discussed in a further paper. Keywords: MIBBI; data standards; data management; data integration; databases; TAB-based formats; workflows; Open Data INTRODUCTION the foundation of a new journal about biological The large amount of data produced by biological databases [24], the foundation of the ISB research projects grows at a fast rate. The 2009 (International Society for Biocuration) and special edition of the annual Nucleic Acids Research conferences like DILS (Data Integration in the Life database issue mentions 1170 databases [1]; alone Sciences) [25].
    [Show full text]
  • Distributed Computing Environments and Workflows for Systems
    Going with the Flow Distributed Computing for Systems Biology Using Taverna Prof Carole Goble The University of Manchester, UK http://www.mygrid.org.uk http://www.omii.ac.uk Data pipelines in bioinformatics Genscan Resources /Services EMBL BLAST Clustal-W Multiple Query Protein sequences Clustal-W sequence alignment 2 Example in silico experiment: Investigate© the evolutionary relationships between proteins [Peter Li] z Manual creation z Semi-automation using bespoke software 12181 acatttctac caacagtgga tgaggttgtt ggtctatgtt ctcaccaaat ttggtgttgt 12241 cagtctttta aattttaacc tttagagaag agtcatacag tcaatagcct tttttagctt z Issues: 12301 gaccatccta atagatacac agtggtgtct cactgtgatt ttaatttgca ttttcctgct 12361 gactaattat gttgagcttg ttaccattta gacaacttca ttagagaagt gtctaatatt 12421 taggtgactt gcctgttttt ttttaattgg gatcttaatt tttttaaatt attgatttgt 12481 aggagctatt tatatattct ggatacaagt tctttatcag atacacagtt tgtgactatt z Volatility of data12541 ttcttataag in tctgtggtttlife ttatattaatsciences gtttttattg atgactgttt tttacaattg 12601 tggttaagta tacatgacat aaaacggatt atcttaacca ttttaaaatg taaaattcga 12661 tggcattaag tacatccaca atattgtgca actatcacca ctatcatact ccaaaagggc 12721 atccaatacc cattaagctg tcactcccca atctcccatt ttcccacccc tgacaatcaa z Data and metadata12781 taacccattt tctgtctcta storage tggatttgcc tgttctggat attcatatta atagaatcaa z Integration of heterogeneous biological data z Visualisation of models z Brittleness 3 © 12181 acatttctac caacagtgga tgaggttgtt ggtctatgtt ctcaccaaat ttggtgttgt 12241 cagtctttta aattttaacc tttagagaag agtcatacag tcaatagcct
    [Show full text]
  • Mygrid: a Collection of Web Pages
    Home The myGrid team produce and use a suite of tools designed to “help e-Scientists get on with science and get on with scientists”. The tools support the creation of e-laboratories and have been used in domains as diverse as systems biology, social science, music, astronomy, multimedia and chemistry. The tools have been adopted by a large number of projects and institutions. The team has developed tools and infrastructure to allow: • the design, editing and execution of workflows in Taverna • the sharing of workflows and related data by myExperiment • the cataloguing and annotation of services in BioCatalogue and Feta • the creation of user-friendly rich clients such as UTOPIA These tools help to form the basis for the team’s work on e-Labs. Taverna Workbench 2.1.2 available for download The myGrid team have released Taverna Workbench 2.1.2. It supports secure access to data on the web, secure web services and secured private workflows. Taverna 2.1.2 includes • Copy/paste, shortcuts, undo/redo, drag and drop • Animated workflow diagram • Remembers added/removed services • Secure Web services support • Secure access to resources on the web • Up-to-date R support • Intermediate values during workflow runs • myExperiment integration • Excel and csv spreadsheet support Hosted at the School of Computer Science, University of Manchester, UK. Copyright 2008 - All Rights Reserved Sponsored by Microsoft, EPSRC, BBSRC, ESRC and JISC From www.mygrid.org.uk/ 1 27 May 2010 Outreach >Outreach The myGrid consortium put a lot of effort into reaching out to users. The team do not just “preach to the converted” but seek new users by offering Tutorials and giving talks, papers and posters in a very wide variety of places.
    [Show full text]
  • Hosts: Monash Eresearch Centre and Messagelab Seminar :The Long Tail Scientist Presenter: Prof Carole Goble, Computer Science
    Hosts: Monash eResearch Centre and MessageLab Seminar :The Long tail Scientist Presenter: Prof Carole Goble, Computer Science, University of Manchester Venue: Seminar Room 135, Building 26 Clayton Time and Date: Wed 3 August 2011, 5-6pm Abstract Big science with big, coordinated and collaborative programmes – the Large Hadron Collider, the Sloan Sky Survey, the Human Genome and its successor the 1000 Genomes project – hogs headlines and fascinates funders. But this big science makes up a small fraction of research being done. Whilst the big journals – Nature, Science - are often the first to publish breakthrough research, work in a vast array of smaller journals still contributes to scientific knowledge. Every day, PhD students and post-docs are slaving away in small labs building up the bulk of scientific data. In disciplines like chemistry, biology and astronomy they are taking advantage of the multitude of public datasets and analytical tools to make their own investigations. Jim Downing at the Unilever Centre for Molecular Informatics was one of the first to coin the term of “Long Tail Science” – that large numbers of small researcher units is an important concept. We are not just standing on the shoulders of a few giants but standing on the shoulders of a multitude of the average sized. Ten years ago I started up the myGrid e- Science project (http://www.mygrid.org.uk ) specifically to help the long tail bioinformatician, and later other long tail scientists from other disciplines. And it turns out that the software, services and methods we develop and deploy (the Taverna workflow system, myExperiment, BioCatalogue, SysMO-SEEK, MethodBox) apply just as well to the big science projects.
    [Show full text]
  • A Web 2.0 Virtual Research Environment
    my Experiment – A Web 2.0 Virtual Research Environment David De Roure Carole Goble Electronics and Computer Science School of Computer Science University of Southampton, UK The University of Manchester, UK [email protected] [email protected] +44 23 8059 2418 +44 161 275 6195 ABSTRACT established as a means of automating the processing of e-Science has given rise to new forms of digital object in scientific data in a scalable and reusable way. the Virtual Research Environment which can usefully be my shared amongst collaborating scientists to assist in The Experiment Virtual Research Environment (VRE) generating new scientific results. In particular, descriptions provides a personalised environment which enables users to of Scientific Workflows capture pieces of scientific share, re-use and repurpose experiments. Our vision is that knowledge which may transcend their immediate scientists should be able to swap workflows and other scientific objects as easily as citizens can share documents, application and can be shared and reused in other my experiments. We are building the my Experiment Virtual photos and videos on the Web. Hence Experiment owes Research Environment to support scientists in sharing and far more to social networking websites such as MySpace collaboration with workflows and other objects. Rather than (www.myspace.com) and YouTube (www.youtube.com) adopting traditional methods prevalent in the e-Science than to the traditional portals of Grid computing, and is developer community, our approach draws upon the social immediately familiar to the new generation of scientists. software techniques characterised as Web 2.0. In this paper Where many e-Science projects have focused on bringing we report on the preliminary design work of my Experiment.
    [Show full text]
  • Taverna Workflow Management System
    http://www.mygrid.org.uk/ http://www.taverna.org.uk/ Taverna Robert Haines, Stian Soiland-Reyes myGrid, University of Manchester [email protected] http://orcid.org/0000-0002-9538-7919 IS-ENES2 workshop on workflows, Hamburg, 2014-06-03 This work is licensed under a Creative Commons Attribution 3.0 Unported License Taverna in Context • Comprehensive Scientific Workflow Management System + auxiliary tools/repositories • Based at Manchester with multiple contributions and collaborations • Releases: Three major; numerous rolling intermediate. First release 2004. • Downloads: 90,000+ cumulative; 1000 in first month per intermediate release; user audit for May 2013 had 900+ unique addresses use Taverna • Users: ~380 sites and institutions have used or use Taverna • Support: mailing list, community list and Jira Taverna workflows • Sophisticated analysis pipelines • A set of services to analyze or manage data (local or remote) • Workflows run through the workbench or via a server • Automation of data flow through services • Control of service invocation • Iteration over data sets • Provenance collection • Extensible and open source Taverna workflows • Dataflow – Graphically connect data between drag-and- dropped services • Service types – REST, SOAP, Command Line, web interactions, scripts (R, Python, Beanshell) – Domain-specific plugins – Your tool? Taverna workflows • Nested workflows • Components – Reusable and inter-compatible workflow fragments – Grouped into families – Semantically annotated – Curated Application RepositoriesRepositories
    [Show full text]
  • Bit Brochure.Pdf
    Register by March 5 and Save up to $200! Final Agenda ENABlING TECHNOlOGy. Leveraging Data. TransformING mEDICINE. Featured Presentations by: CONCURRENT TRACKS: KEYNOTE PreseNTATIONS BY: 1 IT Infrastructure — Hardware Platinum Sponsors: 2 IT Infrastructure — Software John Halamka, m.D., m.S., CIO, Harvard Medical School 3 Bioinformatics and Next-Gen Data 4 Systems and Predictive Biology 5 Cheminformatics and Computer-Aided Modeling Font: Optima Regular Colors: Dark Blue #003399 (CMYK 96, 69, 3, 0) eClinical Trials Technology Light Blue #6699CC (CMYK 62, 22, 3, 0) 6 Gold Sponsors: 7 eHealth Solutions Christoph Westphal, m.D., Ph.D., CEO, Sirtris Pharmaceuticals; Senior Vice President, Centre of Excellence for External Drug Discovery, EVENT FEATURES: GlaxoSmithKline Access All Seven Tracks for One Price Network with 1,500+ Attendees Hear 100+ Technology and Scientific Presentations Attend Bio-IT World’s Best Practices Awards KEYNOTE PANEL: Connect with Attendees Using The Future of Personal Genomics Bronze Sponsors: CHI’s Intro-Net A special plenary panel discussion featuring: Participate in the Poster Competition James Heywood, Co-founder and Chairman, PatientsLikeMe See the Winners of the following 2010 Awards: Dan Vorhaus, J.D., M.A., Attorney, Robinson, Bradshaw & Hinson; Benjamin Franklin Editor, Genomics Law Report Best of Show Dietrich Stephan, Ph.D., President & CEO, Ignite Institute Best Practices View Novel Technologies and Solutions Kevin Davies, Ph.D., Editor-in-Chief, Bio-IT World in the Expansive Exhibit Hall And More... And Much More! Corporate Support Sponsor Bio-ITWorldExpo.com Organized & Managed by: Official Publication: Cambridge Healthtech Institute 250 First Avenue, Suite 300, Needham, MA 02494 Phone: 781-972-5400 • Fax: 781-972-5425 • Toll-free in the U.S.
    [Show full text]
  • A Web 2.0 Virtual Research Environment
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by e-Prints Soton my Experiment – A Web 2.0 Virtual Research Environment David De Roure Carole Goble Electronics and Computer Science School of Computer Science University of Southampton, UK The University of Manchester, UK [email protected] [email protected] +44 23 8059 2418 +44 161 275 6195 ABSTRACT established as a means of automating the processing of e-Science has given rise to new forms of digital object in scientific data in a scalable and reusable way. the Virtual Research Environment which can usefully be my shared amongst collaborating scientists to assist in The Experiment Virtual Research Environment (VRE) generating new scientific results. In particular, descriptions provides a personalised environment which enables users to of Scientific Workflows capture pieces of scientific share, re-use and repurpose experiments. Our vision is that knowledge which may transcend their immediate scientists should be able to swap workflows and other scientific objects as easily as citizens can share documents, application and can be shared and reused in other my experiments. We are building the my Experiment Virtual photos and videos on the Web. Hence Experiment owes Research Environment to support scientists in sharing and far more to social networking websites such as MySpace collaboration with workflows and other objects. Rather than (www.myspace.com) and YouTube (www.youtube.com) adopting traditional methods prevalent in the e-Science than to the traditional portals of Grid computing, and is developer community, our approach draws upon the social immediately familiar to the new generation of scientists.
    [Show full text]
  • Tracking Workflow Execution with Tavernaprov Data Bundle
    Tracking workflow execution with TavernaProv Document id: https://github.com/stain/2016-provweek-tavernaprov/ Stian Soiland-Reyes, Pinar Alper, Carole Goble; eScience Lab, University of Manchester Apache Taverna is a scientific workflow system for combining web services and local tools. Taverna records provenance of workflow runs, intermediate values and user interactions, both as an aid for debugging while designing the workflow, but also as a record for later reproducibility and comparison. Taverna also records provenance of the evolution of the workflow definition (including a chain of wasDerivedFrom relations), attributions and annotations; for brevity we here focus on how Taverna's workflow run provenance extends PROV and is embedded with Research Objects. Data bundle A workflow run can be exported from Taverna as a workflow data bundle; a Research Object bundle in the form of a ZIP archive that contains the workflow definition (itself a Research Object), annotations, inputs, outputs and intermediate values, and a PROV-O trace of the workflow execution, showing every process execution within the workflow run, linking to the produced and consumed values using relative paths. A workflow run can thus be downloaded as a single file from a Taverna Server, shared (myExperiment, SEEK), published (ROHub, Zenodo), imported in another Taverna Workbench, shown in the Databundle Viewer or modified with the DataBundle API. Abstraction levels PROV is a generic model for describing provenance. While this means there are generally many multiple ways to express the same history in PROV, a scientific workflow run with processors and data values naturally match the Activity and Entity relations wasGeneratedBy and used .
    [Show full text]
  • Viewed Or Downloaded, and a Rating Scale That Reports About the Quality of the Resource
    Pérez et al. Journal of Biomedical Semantics 2013, 4:12 http://www.jbiomedsem.com/content/4/1/12 JOURNAL OF BIOMEDICAL SEMANTICS RESEARCH Open Access BioUSeR: a semantic-based tool for retrieving Life Science web resources driven by text-rich user requirements María Pérez1*, Rafael Berlanga2, Ismael Sanz1 and María José Aramburu1 Abstract Background: Open metadata registries are a fundamental tool for researchers in the Life Sciences trying to locate resources. While most current registries assume that resources are annotated with well-structured metadata, evidence shows that most of the resource annotations simply consists of informal free text. This reality must be taken into account in order to develop effective techniques for resource discovery in Life Sciences. Results: BioUSeR is a semantic-based tool aimed at retrieving Life Sciences resources described in free text. The retrieval process is driven by the user requirements, which consist of a target task and a set of facets of interest, both expressed in free text. BioUSeR is able to effectively exploit the available textual descriptions to find relevant resources by using semantic-aware techniques. Conclusions: BioUSeR overcomes the limitations of the current registries thanks to: (i) rich specification of user information needs, (ii) use of semantics to manage textual descriptions, (iii) retrieval and ranking of resources based on user requirements. Keywords: Resources discovery, Semantic annotation, Information retrieval, Life science Background is prodigious, and the sheer amount of available resources In this section we introduce the context of our work and to manage research data is a source of severe difficulties. our motivation. Then, we summarize the related work and In this work, we consider web resources as any appli- we present the rationale of our proposal.
    [Show full text]