DOCSLIB.ORG

Databases and Tools for Structured Data

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Databases and Tools for Structured Data Databases and Tools for Structured Data An increasing number of research projects are now making use of some form of structured data – that is, data which consists of sets of comparable information, in which multiple items or objects share certain common features. Even if you have no intention of producing a website or presenting your data publicly, there are many situations in which databases or database-like tools may provide the best way of organising this sort of information. This article provides some tips on how to select the tools that are most appropriate for a given research project. Contents Research Research When a word processor is not enough ................................................................... 1 Spreadsheets ........................................................................................................... 1 Relational databases .............................................................................................. 2 XML databases ...................................................................................................... 3 TEI XML ............................................................................................................. 4 RDF data ................................................................................................................ 5 Where to go for more information ........................................................................ 6 When a word processor is not enough Some researchers use a word processor for a considerable portion of their work, particularly when writing books, papers, theses, and notes. Because of this familiarity, it can be tempting to use a word processor for everything. While this approach can save time which would otherwise need to be invested in learning new software and methods, it can lead to people generating extremely long documents, from which it is a struggle to retrieve useful information in any meaningful manner. For structured data, one of the approaches described below may be a better bet. Spreadsheets If the information you are working with consists of a number of discrete objects, each of which shares essentially the same limited set of characteristics, a spreadsheet may provide the ideal means for structuring your data. Historical surveys, such as censuses, and bureaucratic records are often most clearly set out in the form of a spreadsheet; a list of the bibliographic details of a set of books might be another example, or a list of financial returns from a publishing house. Spreadsheet software is well adapted to sort and re-sort records alphabetically or numerically, and is ideal if you wish to conduct numerical analysis – to establish means and medians in a particular dataset, for instance, or visualise information in the form of charts and graphs. Figure 1: Example of information in a spreadsheet Useful for: Ordering simple records Numerical analysis Generating charts and graphs Research Research Disadvantages: Not as good at handling complex relationships as relational databases Popular software packages: Microsoft Excel OpenOffice.org Calc Relational databases Spreadsheets are fine for a lot of basic data organisation and analysis, but in some cases a relational database offers significant advantages. If you’re working with information or sources that have relationships with other objects, which in turn have interesting properties or relationships, using a relational database is probably a good idea. When you construct a relational database, you can create separate tables (which individually tend to look much like spreadsheets) and link fields within each table to fields in other tables. So, for instance, you could create a table of bibliographic details about books, including the names of the authors, and link this to a separate table of authors, containing information about when they were born and died, where they were educated, and so forth. If you wished, you could link the information about where they were educated to another table, providing information about the size and location of the school or university they attended. Relational databases cater for one-to-many relationships, or even many-to-many. Relational databases can be designed to enable quite complex cross-searching, for instance, listing all the books published by authors who attended a particular university during a given period. Searches of databases are called queries, and are written in a query-language such as SQL (Structured Query Language) – though many database software packages include a query-building function which will automatically convert your instructions to SQL for you. Learning to construct queries is not difficult, but there are various clever tricks that one can pick up to perform complex but efficient searches. Research Research Figure 2: Example of a relational database structure Useful for: Situations where you are not sure in advance how you (or others) will want to query your data, and wish to keep your options open and flexible Spotting unexpected relationships between things Hosting information on the Web and allowing others to search it Disadvantages: Efficiently structuring large databases can be a challenge Popular software packages: Microsoft Access FileMaker Pro MySQL (particularly for databases hosted on the Web) PostgreSQL (particularly for databases hosted on the Web) XML databases Spreadsheets and relational databases can be very useful if you are working with essentially consistent data – where there are a limited number of shared characteristics common to each record in a given table. If the information you wish to analyse is difficult to characterise in such a way, however, you may wish to take a different approach. XML (eXtensible Markup Language) is a standard for tagging information in order to render it machine-readable. It is primarily used to assist textual analysis, as it can be used to indicate particular characteristics that apply to particular sections in a text. For instance, you may have a number of texts which cover very different subjects, but you want to find all instances where a particular individual or event is mentioned. You could surround each personal name with tags indicating that the part of the text is question is a personal name – <name>Christopher Columbus</name>, for example. You could then search your texts for all the people named in them, or index each occurrence of a specific name. You could also use XML to create a standardised version of a name that occurs in many different variants, in order to render it searchable but without having to alter the original spellings in the document itself. Other tags can be used to indicate how a text should be displayed. Enclosing a piece of text between two emphasis tags, for instance, will indicate to a Web browser or some other XML reader that it should be displayed as bold, or italic. The precise interpretation of how an XML file should be displayed can be customised – the important thing is that XML separates content from its representation, ensuring that the document does not become unreadable just because the technology used to display it has changed. As is the case when working with relational databases, it is possible to create quite complicated queries when working with XML-tagged documents. XQuery is one popular language for searching XML databases. As with SQL and its Research Research equivalents, it is fairly straightforward to learn how to return results from simple searches, but complex queries can also be constructed with a little more knowledge and experience. XML is not only used to indicate textual content, but is also widely used in linguistics to indicate parts of speech or features of spoken language. It is also popular amongst those working with manuscripts or multiple editions, to indicate variations, alternative translations, and so forth. TEI XML Text Encoding Initiative (TEI) XML is a schema established to aid consistency and interoperability between digital humanities projects. Essentially, the TEI has defined a number of labels (about 500) for use when tagging texts, so that people do not end up having to create their own definitions every time they want to make a text machine-readable. The TEI guidelines are available from http://www.tei-c.org/Guidelines/. The University of Oxford is a centre of expertise in TEI XML. OUCS runs an annual summer school, and members of the University can email [email protected] at any time for free advice. Figure 3 (below) shows part of the play The Raigne of King Edvvard the Third marked up into XML. Some of the tags instruct the Web browser how to display the text, whereas others are ‘invisible’, but can assist searching and analysis of the text. For instance, homographs are indicated in the XML, but are not flagged up in the text displayed in the browser. One can see here that the browser has not been instructed to recognise all of the rendering information in the XML original, as it is not displaying the names of the speakers or the stage directions in italic. Research Research Figure 3: Example of a text with TEI XML mark-up, rendered into simple HTML Useful for: Working with texts Providing access to textual databases via the Web Textual and linguistic analysis Disadvantages: Tagging documents is time-consuming You need to ensure that you tag elements consistently Popular software packages: Oxygen XML Editor is useful for editing and tagging XML documents and checking that they meet TEI standards eXist is a free, open source, native XML database management
Load more

Recommended publications
  • Health Sensor Data Management in Cloud
    Special Issue - 2015 International Journal of Engineering Research & Technology (IJERT) ISSN: 2278-0181 NCRTS-2015 Conference Proceedings Health Sensor Data Management in Cloud Rashmi Sahu Department of Computer Science and Engineering BMSIT,Avallahalli,Yelahanka,Bangalore Visveswariya Technological University Abstract--Wearable sensor devices with cloud computing uses its software holds 54% of patient records in US and feature have great impact in our daily lives. This 2.5% of patient records in world wide.[9] technology provides services to acquire, consume and share personal health information. Apart from that we can be How resource wastage can be refused by using cloud connected with smart phones through which we can access technology information through sensor devices equipped with our smart phone. Now smartphones has been resulted in the new ways. It is getting embedded with sensor devices such as Suppose there are 3 Hospitals A,B,C.Each hospital cameras, microphones, accelerometers, proximity sensors, maintains their own network database server,they have GPS etc. through which we can track information and management department and softwares,maintainance significant parameter about physiology. Some of the department and softwares.They organizes their own data wearable tech devices are popular today like Jawbone Up and they maintained by their own.But there is resource and Fitbit Flex, HeartMath Inner Balance Sensor, wastage,means three different health organizations Tinke.This paper is survey in area of medical field that utilizing resources having paid and costs three times of represents why cloud technologies used in medical field and single plus waste of data space also.so why can’t we how health data managed in cloud.
    [Show full text]
  • Requirements for XML Document Database Systems Airi Salminen Frank Wm
    Requirements for XML Document Database Systems Airi Salminen Frank Wm. Tompa Dept. of Computer Science and Information Systems Department of Computer Science University of Jyväskylä University of Waterloo Jyväskylä, Finland Waterloo, ON, Canada +358-14-2603031 +1-519-888-4567 ext. 4675 [email protected] [email protected] ABSTRACT On the other hand, XML will also be used in ways SGML and The shift from SGML to XML has created new demands for HTML were not, most notably as the data exchange format managing structured documents. Many XML documents will be between different applications. As was the situation with transient representations for the purpose of data exchange dynamically created HTML documents, in the new areas there is between different types of applications, but there will also be a not necessarily a need for persistent storage of XML documents. need for effective means to manage persistent XML data as a Often, however, document storage and the capability to present database. In this paper we explore requirements for an XML documents to a human reader as they are or were transmitted is database management system. The purpose of the paper is not to important to preserve the communications among different parties suggest a single type of system covering all necessary features. in the form understood and agreed to by them. Instead the purpose is to initiate discussion of the requirements Effective means for the management of persistent XML data as a arising from document collections, to offer a context in which to database are needed. We define an XML document database (or evaluate current and future solutions, and to encourage the more generally an XML database, since every XML database development of proper models and systems for XML database must manage documents) to be a collection of XML documents management.
    [Show full text]
  • XML Normal Form (XNF)
    Ryan Marcotte www.cs.uregina.ca/~marcottr CS 475 (Advanced Topics in Databases) March 14, 2011 Outline Introduction to XNF and motivation for its creation Analysis of XNF’s link to BCNF Algorithm for converting a DTD to XNF Example March 14, 2011 Ryan Marcotte 2 March 14, 2011 Ryan Marcotte 3 Introduction XML is used for data storage and exchange Data is stored in a hierarchical fashion Duplicates and inconsistencies may exist in the data store March 14, 2011 Ryan Marcotte 4 Introduction Relational databases store data according to some schema XML also stores data according to some schema, such as a Document Type Definition (DTD) Obviously, some schemas are better than others A normal form is needed that reduces the amount of storage needed while ensuring consistency and eliminating redundancy March 14, 2011 Ryan Marcotte 5 Introduction XNF was proposed by Marcelo Arenas and Leonid Libkin (University of Toronto) in a 2004 paper titled “A Normal Form for XML Documents” Recognized a need for good XML data design as “a lot of data is being put on the web” “Once massive web databases are created, it is very hard to change their organization; thus, there is a risk of having large amounts of widely accessible, but at the same time poorly organized legacy data.” March 14, 2011 Ryan Marcotte 6 Introduction XNF provides a set of rules that describe well-formed DTDs Poorly-designed DTDs can be transformed into well- formed ones (through normalization – just like relational databases!) Well-formed DTDs avoid redundancies and update
    [Show full text]
  • Model-Based XML to Relational Database Mapping Choices
    International Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-8 Issue-3S, October 2019 Model-based XML to Relational Database Mapping Choices Emyliana Song, Su-Cheng Haw, Fang-Fang Chua Type Definition file (DTD) or XML schema to define Abstract— Extensible Markup Language (XML) technology structure of XML document. For model-based mapping, is widely used for data exchange and data representation in both DTD and XML schema is not needed. online and offline mode. This structured format language able to be transformed into other formats and share information The rest of the paper is organized as follows. Existing and across platforms. XML is simple; however, it is designed to related approaches on model-based mapping schemes are accommodate changes. For this paper, a study on reviewed in section 2. Section 3 discussed the performance transformation of XML document into relational database is evaluation carried out in the experiment of selected conducted. Crucial part of this process is how to maintain the approaches. Experimental results and analysis of the hierarchy and relationships between data in the document into findings are presented in section 4. And lastly, Section 5 database. Approaches that are discussed in this paper each uses own unique way of data storing technique and database design. conclude the paper. Therefore, each algorithm is assessed with three datasets constitute of small, medium and large size XML file. The II. LITERATURE REVIEW efficiency of the algorithms is being tested on time taken for data storing and query execution process. At the end of the Throughout the years, numerous mapping schemes have evaluation, we discuss factors that affect algorithm performance been proposed to resolve issues on transforming XML to and present suggestions to improve mapping scheme for future relational database structure.
    [Show full text]
  • XML in Oracle 9I
    Geoff Lee Senior Product Manager Oracle Corporation XML in Oracle9i A Technical Overview Agenda ! Survey ! Technical Overview ! Summary ! Q & A Agenda ! Survey ! Technical Overview ! Summary ! Q & A XML in Oracle9i - Overview ! XML and its family of standards are vital to the future of e-Business ! Oracle9i XML Developer’s Kits support the family of XML standards to provide a complete XML-enabled Internet application platform ! Oracle9i Database Native XML support enables fast, flexible, and scalable storage and retrieval of XML data and documents ! XML Messaging and Transformation support in AQ provide a centralized, easy to manage, secure infrastructure for global messaging Agenda ! Survey ! Technical Overview – XML – XDK – Database Native XML – XML Messaging ! Summary ! Q & A Application Requirements 1 Internet Content 2 Internet Application Management Development ConsolidateConsolidate InternetInternet contentcontent MakeMake webweb sitesite transactional,transactional, BuildBuild dynamicdynamic webweb sites/portalssites/portals secure,secure, scalablescalable andand availableavailable 5 3 Enterprise Application Business Intelligence Integration Capture,Capture, analyze,analyze, andand shareshare IntegrateIntegrate webweb sites,sites, ERP,ERP, businessbusiness intelligenceintelligence legacylegacy systems,systems, supplierssuppliers 4 Mobile Information Access MakeMake webweb sitessites accessibleaccessible fromfrom anyany mobilemobile devicedevice Internet N-Tier Architecture Web Browser HTTP Listener Dispatcher Web Server HTML Application
    [Show full text]
  • Transactional Support in Native Xml Databases
    TRANSACTIONAL SUPPORT IN NATIVE XML DATABASES Theo Härder, Sebastian Bächle and Christian Mathis University of Kaiserslautern, Gottlieb-Daimler-Str., 67663 Kaiserslautern, Germany Keywords: XML database management, concurrency control, logging and recovery, elementless XML storage. Abstract: Apparently, everything that can be said about concurrency control and recovery is already said. None the less, the XML model poses new problems for the optimization of transaction processing. In this position paper, we report on our view concerning XML transaction optimization. We explore aspects of fine-grained transaction isolation using tailor-made lock protocols. Furthermore, we outline XML storage techniques where storage representation and logging can be minimized in specific application scenarios. 1 INTRODUCTION set of standards for information exchange and representation. It seems, the more domains are When talking about transaction management, every- conquered by XML (by defining schemas for body implicitly refers to relational technology. It is business cooperation), the more the relational true that the basic concepts of ACID transactions systems approach “legacy”. (Härder and Reuter, 1983) were primarily laid in the Hence, efficient and effective transaction-pro- context of flat table processing and the related query tected collaboration on XML documents (XQuery languages and later adjusted to object orientation. As Update Facility) becomes a pressing issue. a major advance for transaction processing, Weikum Solutions, optimal in the relational world, may fail and Vossen (2002) unified concurrency control and to be appropriate because of the documents’ tree recovery for both the page and object model. Perfor- characteristics and differing processing models. mance concerns led to a refinement of the page Structure variations and workload changes imply model to exploit records as more fine-grained units that transaction-related protocols must exhibit better of concurrency control.
    [Show full text]
  • XML Native Databases and Legislative Documents: a White Paper
    Preserving State Government Digital Information Minnesota Historical Society XML Native Databases and Legislative Documents: A White Paper Abstract XML native databases can improve access to and use of text-based XML encoded information by providing full-text indexing and native storage, and they can be even more powerful when used with other XML standards and tools such as XQuery and XForms. Any comments, corrections, or recommendations may be sent to the project team, care of: Nancy Hoffman Project Analyst Minnesota Historical Society [email protected] / (651) 259-3367 The Problem Electronic legislative records present a classic information management problem – how can text- based information be as easy to access and use as data stored in traditional database systems?1 A common solution has been to take data out of a document and place it into defined fields in a traditional database management system - a process called “shredding.” Removing and storing data elements in this way is not only difficult and time consuming, but more importantly, the database cannot take full advantage of the value of the text because the actual documents are stored in a separate file system or are completely obscure when stored as objects (called binary large object types or BLOBs ) inside the database.2 XML Changes Everything The adoption of XML bill drafting systems for legislation and increasing use of XML to structure related information opens new possibilities afforded by XML-related tools and 1 http://www.cs.ubc.ca/grads/resources/thesis/May04/Fengdong_Du.pdf [accessed 12/18/2009] 2 http://www.ibm.com/developerworks/xml/library/x-xml2008prevw.html [accessed 12/22/2009] Minnesota Historical Society / State Archives Page 1 of 12 NDIIPP XML Native Databases White Paper Version 1, December 2009 http://www.mnhs.org/ndiipp standards that have developed over the ten-plus years since its introduction.
    [Show full text]
  • Examining Database Persistence of ISO/EN 13606 Standardized Electronic Health Record Extracts: Relational Vs
    Sánchez-de-Madariaga et al. BMC Medical Informatics and Decision Making (2017) 17:123 DOI 10.1186/s12911-017-0515-4 RESEARCH ARTICLE Open Access Examining database persistence of ISO/EN 13606 standardized electronic health record extracts: relational vs. NoSQL approaches Ricardo Sánchez-de-Madariaga1* , Adolfo Muñoz1, Raimundo Lozano-Rubí2,3, Pablo Serrano-Balazote4, Antonio L. Castro1, Oscar Moreno1 and Mario Pascual1 Abstract Background: The objective of this research is to compare the relational and non-relational (NoSQL) database systems approaches in order to store, recover, query and persist standardized medical information in the form of ISO/EN 13606 normalized Electronic Health Record XML extracts, both in isolation and concurrently. NoSQL database systems have recently attracted much attention, but few studies in the literature address their direct comparison with relational databases when applied to build the persistence layer of a standardized medical information system. Methods: One relational and two NoSQL databases (one document-based and one native XML database) of three different sizes have been created in order to evaluate and compare the response times (algorithmic complexity) of six different complexity growing queries, which have been performed on them. Similar appropriate results available in the literature have also been considered. Results: Relational and non-relational NoSQL database systems show almost linear algorithmic complexity query execution. However, they show very different linear slopes, the former being much steeper than the two latter. Document-based NoSQL databases perform better in concurrency than in isolation, and also better than relational databases in concurrency. Conclusion: Non-relational NoSQL databases seem to be more appropriate than standard relational SQL databases when database size is extremely high (secondary use, research applications).
    [Show full text]
  • Translating Between XML and Relational Databases Using XML Schema and Automed
    Imperial College of Science, Technology and Medicine (University of London) Department of Computing Translating between XML and Relational Databases using XML Schema and Automed Andrew Charles Smith acs203 Submitted in partial fulfillment of the requirements for the MSc Degree in Advanced Computing of the University of London and for the Diploma of Imperial College of Science, Technology and Medicine. September 2004 Acknowledgements I would like to thank my supervisor, Peter McBrien, for his support during this project and his help with the Automed API. I would also like to thank Lucas Zamboulis for his help with the XML queries and Nicolas Debarnot for his help with LATEX. 1 Contents 1 Introduction 8 2 XML and RDBMS 11 2.1 Motivating Example . 11 2.2 RDBMS . 11 2.3 XML . 12 2.3.1 Constraining XML . 12 2.3.2 XML Schema . 12 2.4 Representing Data Graphically . 17 2.4.1 XPath . 18 2.4.2 XMLSPY . 19 2.4.3 Entity-Relationship Models . 19 2.5 Querying . 19 2.5.1 XQuery . 20 2.5.2 SQL . 20 3 Moving Data between RDBMSs and XML 21 3.1 XML/Relational Schemas . 21 3.1.1 Differences between XML and relational schemas . 21 3.2 The Translation Process . 21 3.3 Main problems . 22 3.4 Existing Approaches . 22 3.4.1 Storing XML in relational databases . 23 3.4.2 Exporting relational data to XML . 24 3.4.3 Generating an XML schema from a Relational schema . 26 3.5 Choosing the most appropriate schema . 26 3.5.1 LegoDB .
    [Show full text]
  • XML Programming with SQL/XML and Xquery
    XML programming with SQL/XML and XQuery by J. E. Funderburk S. Malaika B. Reinwald Most business data are stored in relational a company is as profound as it is between compa- database systems, and SQL (Structured nies. Much of the data being exchanged are oper- Query Language) is used for data retrieval ational: data that enable transactions, determine the and manipulation. With XML (Extensible course of business processes, and in the aggregate, Markup Language) rapidly becoming the de become business intelligence data that affect deci- facto standard for retrieving and exchanging sions of business leaders. data, new functionality is expected from traditional databases. Existing SQL The XML data format provides a way of regularizing applications will evolve to retrieve relational the storage of semi-structured data, historical data, data as XML data using database or SQL and other information requiring content manage- extensions for XML. New XML data will be ment. XML can be used to store the content itself stored, searched, and manipulated in the and data mined from the content. Data mined from database as a “first class” citizen along with the content can be used to form catalogs, similar in concept to card catalogs in libraries, which contain existing relational data. Furthermore, new existence and location information and possibly other applications will emerge that solely operate in interesting summary information. Information mined terms of XML. These new XML applications from content can be stored and used in business or operate on the same database using an XML scientific intelligence queries. query language, XQuery. In this paper, we describe an integrated database architecture Information that is low in quantity and importance that enables SQL applications with XML can be stored using a variety of simple techniques.
    [Show full text]
  • Database Management Systems Ebooks for All Edition (
    Database Management Systems eBooks For All Edition (www.ebooks-for-all.com) PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Sun, 20 Oct 2013 01:48:50 UTC Contents Articles Database 1 Database model 16 Database normalization 23 Database storage structures 31 Distributed database 33 Federated database system 36 Referential integrity 40 Relational algebra 41 Relational calculus 53 Relational database 53 Relational database management system 57 Relational model 59 Object-relational database 69 Transaction processing 72 Concepts 76 ACID 76 Create, read, update and delete 79 Null (SQL) 80 Candidate key 96 Foreign key 98 Unique key 102 Superkey 105 Surrogate key 107 Armstrong's axioms 111 Objects 113 Relation (database) 113 Table (database) 115 Column (database) 116 Row (database) 117 View (SQL) 118 Database transaction 120 Transaction log 123 Database trigger 124 Database index 130 Stored procedure 135 Cursor (databases) 138 Partition (database) 143 Components 145 Concurrency control 145 Data dictionary 152 Java Database Connectivity 154 XQuery API for Java 157 ODBC 163 Query language 169 Query optimization 170 Query plan 173 Functions 175 Database administration and automation 175 Replication (computing) 177 Database Products 183 Comparison of object database management systems 183 Comparison of object-relational database management systems 185 List of relational database management systems 187 Comparison of relational database management systems 190 Document-oriented database 213 Graph database 217 NoSQL 226 NewSQL 232 References Article Sources and Contributors 234 Image Sources, Licenses and Contributors 240 Article Licenses License 241 Database 1 Database A database is an organized collection of data.
    [Show full text]
  • Synopsis Data Structures for XML Databases: Models, Issues, and Research Perspectives
    18th International Workshop on Database and Expert Systems Applications Synopsis Data Structures for XML Databases: Models, Issues, and Research Perspectives Angela Bonifati Alfredo Cuzzocrea Icar CNR, Italian National Research Council Icar CNR, Italian National Research Council & Via P. Bucci 41C, I-87036 Rende, Italy DEIS, University of Calabria [email protected] Via P. Bucci 41C, I-87036 Rende, Italy [email protected] Abstract in nature and not prone to be represented as a set of rela- tions or objects; (ii) quite often, XML documents appear Due to the lack of efficient native XML database man- in a schema-less fashion (e.g., like those of corporate B2B agement systems, XML data manipulation and query eval- and B2C e-commerce Web systems), thus making the rela- uation may be resource-consuming, and represent a bottle- tional translation more difficult; (iii) the inherent “richness” neck for several computationally intensive applications. To of the standard XML query language, which defines a com- overcome the above limitations, a possible solution consists prehensive class of queries with possibly complex syntax in computing synopsis data structures from XML databases, and predicates (e.g., for clause of XQuery queries [6], twig i.e. compressed representations providing a “succinct” de- XML queries [7], partial- and exact-match XPath queries scription of the original databases while ensuring low com- [8] etc.); (iv) the ambiguity of the XML semantics during putational overhead and high accuracy for many XML pro- query evaluation [20]; (v) problematic update management cessing tasks. Specifically, these data structures are very issues posed by processing XML data [2, 3, 31].
    [Show full text]