Getting the Data You Can't

Total Page:16

File Type:pdf, Size:1020Kb

Getting the Data You Can't The Marriage of VFE, SQL, Visual Studio John W. Fitzgerald, MD Gregory P. Sanders, MD CHUG 2014 Fall Conference GETTING THE DATA YOU CAN’T GET: WHEN MEL JUST ISN’T ENOUGH The Problem . Built in data symbols are powerful but may not provide the data you need MEDS_AFTER() PROB_LIST_CHANGES() ALL_NEW() ….etc . Mldefs.txt may be available to address your needs but are undocumented, difficult to locate, and may not survive version changes. There are tables with no MEL connections . Complex relationships are generally not available Our Approach . Create SQL Centricity data access statements in Visual Studio, Visual Scripts as an .exe . Store in Centricity Client . Pass arguments to the .exe from VFE . Return results from .exe call . Parse the results as required. Non-trivial Set of Skills Required . VFE . Data Symbols . Centricity table structure and relationships . Model solution in Crystal, Access . SQL statement construction . Visual Studio or equivalent . Lots of debugging THE CLIENT . Folder on each local machine . EXE sits in Client folder for easy access . Code is run locally . Client calls EXE and passes its authentication to it . Client has access to network (ie, SQL server) . EXE must be pushed out to all Clients Dataflow: Call & Query Centricity SQL Centricity calls EXE EXE queries SQL Database EXE Dataflow: Compile & Return Centricity SQL EXE returns data to Centricity SQL returns data to EXE EXE Inside the EXE . Capture Passed Data . Compile SQL Statement . Connect to Database . Organize Queried Data . Pass Back to Centricity Capture Passed Data . Output Data . File name and path designated with “ /o “ primer . Input Data . Data passed to EXE from Centricity . Desginated with “ /i “ primer . Separated by spaces Compile SQL Statement . Read-only (SELECT statement) . Specify which columns desired . JOIN tables if necessary . Passed data used in WHERE statement . ORDER BY SELECT Birthdate FROM PatientProfile WHERE PatientID = ‘<passed data>’ SQL for ICD9 Search SELECT CODE, DESCRIPTION FROM ICD9codes WHERE DESCRIPTION Like ‘<data>%’ OR CODE Like ‘<data>%’ ORDER BY CODE, DESCRIPTION 11 SQL for RxPillCount SELECT MEDICATE.DESCRIPTION, MEDICATE.GPI, PRESCRIB.QUANTITY, PRESCRIB.REFILLS, PRESCRIB.CLINICALDATE FROM PatientProfile INNER JOIN MEDICATE ON PatientProfile.PId = PRESCRIB.MID INNER JOIN PRESCRIB ON MEDICATE.MID = PRESCRIB.MID WHERE PatientProfile.PatientID = <data> AND MEDICATE.stopdate = ’12/31/4700’’ ORDER BY MEDICATE.DESCRIPTION, PRESCRIB.CLINICALDATE 12 Connect to Database . SQL connection string . Connect using ADO . Need appropriate credentials in database 13 Organize Queried Data . Open a recordset of data based upon SQL statement . If you are looking for one record: grab the data . If you are looking for a recordset: cycle through the recordset . Format data accordingly add labels choose delimiter 14 Pass Back to Centricity . Create a text file with the formatted data . Write the text file: . use file name provided in “ /o “ variable . this includes the path . Saving the file: . Centricity waiting for file . Centricity opens file and returns contents 15 Advantages . Local Client . fast . return path built in to process . multiple users do not compete for same EXE . Scalable . Put multiple EXEs in Client . Build EXE that accepts “FUNCTION” argument 16 Limitations . Updates . recompile EXE . distribute to all Clients . Hardwired Information . SQL statement . Connection . database location . credentials 17 Resources: Basic VFE Coding Reference Resources: MEL Rosetta Stone MEL: RunTextProcess Resources: Data Dictionary . Data Dictionary for Centricity Data schema Basic database structure List of tables Identifiers, time stamp, various ID conventions Designing reports, linking tables Complete table definitions: columns, type, description Linkages to other tables . Database linking diagrams Resources: Data Dictionary Resources: Table Definition-1 Resources: Table Definition-2 Resources: Table Definition-3 Resources: Linkage Diagram ICD9Code Table Structure Using Access to Model Solution SELECT dbo_ICD9Codes.Code, dbo_ICD9Codes.Description FROM dbo_ICD9Codes WHERE (((dbo_ICD9Codes.Description) Like "*" & [Search String] & "*")); Access Modeling: Results VFE Code: RunTextProcess A Simple Example: Search ICD9 Table by Description A Simple Example: Search ICD9 Table by ICD9 Code VFE Code: Adding Problem Table Relationships: Meds Table Relationships: Meds, Output Rx Pill Count: Table Relationships Pill Count: Data VFE Filtering: Narcotic Pill Count GPI left 4 characters: 6510—hydromorphone, levorphanol, meperidine, codeine, fentanyl, methadone 6520—buprenorphine, nalbuphine, pentazocine 6599—oxycodone-acetaminophen, other combinations VFE Filtering: Narcotic Pill Count Conclusion . Using .exe calls via RunTextProcess() essentially any data can be retrieved from Centricity . Multiple skill sets required . Filter at server or client, balance flexibility vs speed, consider ease of coding, server vs client side . Minimize bandwidth issues . In the argument string, multiple subroutines can be called .
Recommended publications
  • SQL Server Column Store Indexes Per-Åke Larson, Cipri Clinciu, Eric N
    SQL Server Column Store Indexes Per-Åke Larson, Cipri Clinciu, Eric N. Hanson, Artem Oks, Susan L. Price, Srikumar Rangarajan, Aleksandras Surna, Qingqing Zhou Microsoft {palarson, ciprianc, ehans, artemoks, susanpr, srikumar, asurna, qizhou}@microsoft.com ABSTRACT SQL Server column store indexes are “pure” column stores, not a The SQL Server 11 release (code named “Denali”) introduces a hybrid, because they store all data for different columns on new data warehouse query acceleration feature based on a new separate pages. This improves I/O scan performance and makes index type called a column store index. The new index type more efficient use of memory. SQL Server is the first major combined with new query operators processing batches of rows database product to support a pure column store index. Others greatly improves data warehouse query performance: in some have claimed that it is impossible to fully incorporate pure column cases by hundreds of times and routinely a tenfold speedup for a store technology into an established database product with a broad broad range of decision support queries. Column store indexes are market. We’re happy to prove them wrong! fully integrated with the rest of the system, including query To improve performance of typical data warehousing queries, all a processing and optimization. This paper gives an overview of the user needs to do is build a column store index on the fact tables in design and implementation of column store indexes including the data warehouse. It may also be beneficial to build column enhancements to query processing and query optimization to take store indexes on extremely large dimension tables (say more than full advantage of the new indexes.
    [Show full text]
  • When Relational-Based Applications Go to Nosql Databases: a Survey
    information Article When Relational-Based Applications Go to NoSQL Databases: A Survey Geomar A. Schreiner 1,* , Denio Duarte 2 and Ronaldo dos Santos Mello 1 1 Departamento de Informática e Estatística, Federal University of Santa Catarina, 88040-900 Florianópolis - SC, Brazil 2 Campus Chapecó, Federal University of Fronteira Sul, 89815-899 Chapecó - SC, Brazil * Correspondence: [email protected] Received: 22 May 2019; Accepted: 12 July 2019; Published: 16 July 2019 Abstract: Several data-centric applications today produce and manipulate a large volume of data, the so-called Big Data. Traditional databases, in particular, relational databases, are not suitable for Big Data management. As a consequence, some approaches that allow the definition and manipulation of large relational data sets stored in NoSQL databases through an SQL interface have been proposed, focusing on scalability and availability. This paper presents a comparative analysis of these approaches based on an architectural classification that organizes them according to their system architectures. Our motivation is that wrapping is a relevant strategy for relational-based applications that intend to move relational data to NoSQL databases (usually maintained in the cloud). We also claim that this research area has some open issues, given that most approaches deal with only a subset of SQL operations or give support to specific target NoSQL databases. Our intention with this survey is, therefore, to contribute to the state-of-art in this research area and also provide a basis for choosing or even designing a relational-to-NoSQL data wrapping solution. Keywords: big data; data interoperability; NoSQL databases; relational-to-NoSQL mapping 1.
    [Show full text]
  • Data Dictionary a Data Dictionary Is a File That Helps to Define The
    Cleveland | v. 216.369.2220 • Columbus | v. 614.291.8456 Data Dictionary A data dictionary is a file that helps to define the organization of a particular database. The data dictionary acts as a description of the data objects or items in a model and is used for the benefit of the programmer or other people who may need to access it. A data dictionary does not contain the actual data from the database; it contains only information for how to describe/manage the data; this is called metadata*. Building a data dictionary provides the ability to know the kind of field, where it is located in a database, what it means, etc. It typically consists of a table with multiple columns that describe relationships as well as labels for data. A data dictionary often contains the following information about fields: • Default values • Constraint information • Definitions (example: functions, sequence, etc.) • The amount of space allocated for the object/field • Auditing information What is the data dictionary used for? 1. It can also be used as a read-only reference in order to obtain information about the database 2. A data dictionary can be of use when developing programs that use a data model 3. The data dictionary acts as a way to describe data in “real-world” terms Why is a data dictionary needed? One of the main reasons a data dictionary is necessary is to provide better accuracy, organization, and reliability in regards to data management and user/administrator understanding and training. Benefits of using a data dictionary: 1.
    [Show full text]
  • USER GUIDE Optum Clinformatics™ Data Mart Database
    USER GUIDE Optum Clinformatics Data Mart Database 1 | P a g e TABLE OF CONTENTS TOPIC PAGE # 1. REQUESTING DATA 3 Eligibility 3 Forms 3 Contact Us 4 2. WHAT YOU WILL NEED 4 SAS Software 4 VPN 5 3. ABSTRACTS, MANUSCRIPTS, THESES, AND DISSERTATIONS 5 Referencing Optum Data 5 Optum Review 5 4. DATA USER SET-UP AND ACCESS INFORMATION 6 Server Log-In After Initial Set-Up 6 Server Access 6 Establishing a Connection to Enterprise Guide 7 Instructions to Add SAS EG to the Cleared Firewall List 8 How to Proceed After Connection 8 5. BEST PRACTICES FOR DATA USE 9 Saving Programs and Back-Up Procedures 9 Recommended Coding Practices 9 6. APPENDIX 11 Version Date: 27-Feb-17 2 | P a g e Optum® ClinformaticsTM Data Mart Database The Optum® ClinformaticsTM Data Mart is an administrative health claims database from a large national insurer made available by the University of Rhode Island College of Pharmacy. The statistically de-identified data includes medical and pharmacy claims, as well as laboratory results, from 2010 through 2013 with over 22 million commercial enrollees. 1. REQUESTING DATA The following is a brief outline of the process for gaining access to the data. Eligibility Must be an employee or student at the University of Rhode Island conducting unfunded or URI internally funded projects. Data will be made available to the following users: 1. Faculty and their research team for projects with IRB approval. 2. Students a. With a thesis/dissertation proposal approved by IRB and the Graduate School (access request form, see link below, must be signed by Major Professor).
    [Show full text]
  • Activant Prophet 21
    Activant Prophet 21 Understanding Prophet 21 Databases This class is designed for… Prophet 21 users that are responsible for report writing System Administrators Operations Managers Helpful to be familiar with SQL Query Analyzer and how to write basic SQL statements Objectives Explain the difference between databases, tables and columns Extract data from different areas of the system Discuss basic SQL statements using Query Analyzer Use Prophet 21 to gather SQL Information Utilize Data Dictionary This course will NOT cover… Basic Prophet 21 functionality Seagate’s Crystal Reports Definitions Columns Contains a single piece of information (fields) Record (rows) One complete set of columns Table A collection of rows View Virtual table created for easier data extraction Database Collection of information organized for easy selection of data SQL Query A graphical user interface used to extract Analyzer data SQL Query Analyzer Accessed through Microsoft SQL Server SQL Server Tools Enterprise Manager Perform administrative functions such as backing up and restoring databases and maintaining SQL Logins Profiler Run traces of activity in your system Basic SQL Commands sp_help sp_help <table_name> select * from <table_name> Select <field_name> from <table_name> Most Common Reporting Areas Address and Contact tables Order Processing Inventory Purchasing Accounts Receivable Accounts Payable Production Orders Address and Contact tables Used in conjunction with other tables These tables hold every address/contact in the
    [Show full text]
  • Uniform Data Access Platform for SQL and Nosql Database Systems
    Information Systems 69 (2017) 93–105 Contents lists available at ScienceDirect Information Systems journal homepage: www.elsevier.com/locate/is Uniform data access platform for SQL and NoSQL database systems ∗ Ágnes Vathy-Fogarassy , Tamás Hugyák University of Pannonia, Department of Computer Science and Systems Technology, P.O.Box 158, Veszprém, H-8201 Hungary a r t i c l e i n f o a b s t r a c t Article history: Integration of data stored in heterogeneous database systems is a very challenging task and it may hide Received 8 August 2016 several difficulties. As NoSQL databases are growing in popularity, integration of different NoSQL systems Revised 1 March 2017 and interoperability of NoSQL systems with SQL databases become an increasingly important issue. In Accepted 18 April 2017 this paper, we propose a novel data integration methodology to query data individually from different Available online 4 May 2017 relational and NoSQL database systems. The suggested solution does not support joins and aggregates Keywords: across data sources; it only collects data from different separated database management systems accord- Uniform data access ing to the filtering options and migrates them. The proposed method is based on a metamodel approach Relational database management systems and it covers the structural, semantic and syntactic heterogeneities of source systems. To introduce the NoSQL database management systems applicability of the proposed methodology, we developed a web-based application, which convincingly MongoDB confirms the usefulness of the novel method. Data integration JSON ©2017 Elsevier Ltd. All rights reserved. 1. Introduction solution to retrieve data from heterogeneous source systems and to deliver them to the user.
    [Show full text]
  • Product Master Data Management Reference Guide
    USAID GLOBAL HEALTH SUPPLY CHAIN PROGRAM Procurement and Supply Management PRODUCT MASTER DATA MANAGEMENT REFERENCE GUIDE Version 1.0 February 2020 The USAID Global Health Supply Chain Program-Procurement and Supply Management (GHSC-PSM) project is funded under USAID Contract No. AID-OAA-I-15-0004. GHSC-PSM connects technical solutions and proven commercial processes to promote efficient and cost-effective health supply chains worldwide. Our goal is to ensure uninterrupted supplies of health commodities to save lives and create a healthier future for all. The project purchases and delivers health commodities, offers comprehensive technical assistance to strengthen national supply chain systems, and provides global supply chain leadership. GHSC-PSM is implemented by Chemonics International, in collaboration with Arbola Inc., Axios International Inc., IDA Foundation, IBM, IntraHealth International, Kuehne + Nagel Inc., McKinsey & Company, Panagora Group, Population Services International, SGS Nederland B.V., and University Research Co., LLC. To learn more, visit ghsupplychain.org DISCLAIMER: The views expressed in this publication do not necessarily reflect the views of the U.S. Agency for International Development or the U.S. government. Contents Acronyms ....................................................................................................................................... 3 Executive Summary ...................................................................................................................... 4 Background
    [Show full text]
  • Master Data Management: Dos & Don’Ts
    Master Data Management: dos & don’ts Keesjan van Unen, Ad de Goeij, Sander Swartjes, and Ard van der Staaij Master Data Management (MDM) is high on the agenda for many organizations. At Board level too, people are now fully aware that master data requires specific attention. This increasing attention is related to the concern for the overall quality of data. By enhancing the quality of master data, disrup- tions to business operations can be limited and therefore efficiency is increased. Also, the availability of reliable management information will increase. However, reaching this goal is not an entirely smooth process. MDM is not rocket science, but it does have specific problems that must be addressed in the right way. This article describes three of these attention areas, including several dos and don’ts. C.J.W.A. van Unen is a senior manager at KPMG Management Consulting, IT Advisory. [email protected] A.S.M de Goeij RE is a manager at KPMG Management Consulting, IT Advisory. [email protected] S. Swartjes is an advisor at KPMG Risk Consulting, IT Advisory. [email protected] A.J. van der Staaij is an advisor at KPMG Risk Consulting, IT Advisory. [email protected] Compact_ 2012 2 International edition 1 Treat master data as an asset Introduction Do not go for gold immediately Every organization has to deal with master data, and seeks Although it may be very tempting, you should not go for ways to manage the quality of this specific group of data in the ‘ideal model’ when (re)designing the MDM organiza- an effective and efficient manner.
    [Show full text]
  • MDM Physical Data Dictionary
    IBM InfoSphere Master Data Management InfoSphere Master Data Management Version 11.3 MDM Physical Data Dictionary GI13-2668-01 IBM InfoSphere Master Data Management InfoSphere Master Data Management Version 11.3 MDM Physical Data Dictionary GI13-2668-01 Note Before using this information and the product that it supports, read the information in “Notices and trademarks” on page 195. Edition Notice This edition applies to version 11.3 of IBM InfoSphere Master Data Management and to all subsequent releases and modifications until otherwise indicated in new editions. © Copyright IBM Corporation 1996, 2014. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Chapter 1. Tables...........1 CDEMEMATCHFUNCTIONTP ........46 ACCESSDATEVAL ............1 CDEMEMATCHWORDTP .........47 ADDACTIONTYPE ............2 CDENDREASONTP ...........48 ADDRESS ...............2 CDENTITYLINKSTTP ...........49 ADDRESSGROUP ............5 CDFREQMODETP ............50 AGREEMENTSPECVAL ..........5 CDGENERATIONTP ...........50 BANKACCOUNT ............6 CDHIGHESTEDUTP ...........51 BILLINGSUMMARY............7 CDHOLDINGTP ............52 CAMPAIGN ..............9 CDIDSTATUSTP.............53 CAMPAIGNASSOCIAT ..........10 CDIDTP ...............53 CATEGORY ..............11 CDINACTREASONTP...........54 CATEGORYNLS .............12 CDINCOMESRCTP............55 CATEQUIV ..............13 CDINDUSTRYTP ............56 CATHIERARCHY ............14 CDLANGTP ..............56 CATHIERARCHYNLS ...........15 CDLINKREASONTP
    [Show full text]
  • The Database/Data Dictionary Interface
    PANEL : THE DATABASE/DATA DICTIONARY INTERFACE Larry KERSCHBERG (Chairperson)-Un iversity of South Carolina, Paul FEHDER-IBM Santa Teresa Lab, Georqe GAGNAC-Cincom Systems , Inc., John MXIIRE and llichael JAKES-Software A.G. of North America, Inc., Shamkant NAVATHE-University of Florida, Edgar SIBLEY-Alpha Omega Group, Inc. The purpose of this panel is to review the state- . Privacy, integrity, and security of-the-art in Data Dictionary systems and to issues. explore future capabilities and architectures for such systems. In particular, the interface . How can the dictionary bottleneck between Data Dictionarv svstems and Database be ameliorated? Flanaaement systems wili be discussed in relation to the following issues: 2. The Role of Data Dictionaries in Information Resources Management 1. Architecture of Data Dictionary Systems As data dictionaries are endowed with Current thinking regarding data dictionaries increased intelligence and enhanced capa- indicates that the dictionary is to be a bilities, they will play a key role in repository of information about an enter- information resource management. We wish prise. Thus, this repository, or knowledge to explore several aspects of this role, base, not only contains information about namely: the "semantics" of data stored in databases, but also knows about enterprise concepts A. Tools for enterprise administration. such as organizational structure and dyna- mics, forms, memos and other documents B. Application development tools. together with their organizational flows, and the description of information resources C. Managing computer hardware and software such as hardware and software systems. resources. In this framework we consider the following D. Communication mechanisms among data issues: dictionaries. A.
    [Show full text]
  • A Dictionary of DBMS Terms
    A Dictionary of DBMS Terms Access Plan Access plans are generated by the optimization component to implement queries submitted by users. ACID Properties ACID properties are transaction properties supported by DBMSs. ACID is an acronym for atomic, consistent, isolated, and durable. Address A location in memory where data are stored and can be retrieved. Aggregation Aggregation is the process of compiling information on an object, thereby abstracting a higher-level object. Aggregate Function A function that produces a single result based on the contents of an entire set of table rows. Alias Alias refers to the process of renaming a record. It is alternative name used for an attribute. 700 A Dictionary of DBMS Terms Anomaly The inconsistency that may result when a user attempts to update a table that contains redundant data. ANSI American National Standards Institute, one of the groups responsible for SQL standards. Application Program Interface (API) A set of functions in a particular programming language is used by a client that interfaces to a software system. ARIES ARIES is a recovery algorithm used by the recovery manager which is invoked after a crash. Armstrong’s Axioms Set of inference rules based on set of axioms that permit the algebraic mani- pulation of dependencies. Armstrong’s axioms enable the discovery of minimal cover of a set of functional dependencies. Associative Entity Type A weak entity type that depends on two or more entity types for its primary key. Attribute The differing data items within a relation. An attribute is a named column of a relation. Authorization The operation that verifies the permissions and access rights granted to a user.
    [Show full text]
  • A Storage Advisor for Hybrid-Store Databases
    A Storage Advisor for Hybrid-Store Databases Philipp Rosch¨ Lars Dannecker Gregor Hackenbroich SAP Research, SAP AG SAP Research, SAP AG SAP Research, SAP AG Chemnitzer Str. 48 Chemnitzer Str. 48 Chemnitzer Str. 48 01187 Dresden, Germany 01187 Dresden, Germany 01187 Dresden, Germany [email protected] [email protected] [email protected] Franz Farber¨ SAP AG Dietmar­Hopp­Allee 16 69190 Walldorf, Germany [email protected] ABSTRACT ented row-wise instead. Row-store technology is widely used With the SAP HANA database, SAP offers a high-perfor- and highly advanced. Especially, row stores are (much) bet- mance in-memory hybrid-store database. Hybrid-store data- ter suited in transactional scenarios (OLTP - Online Trans- bases—that is, databases supporting row- and column-orien- actional Processing) with many updates and inserts of data ted data management—are getting more and more promi- as well as point queries. nent. While the columnar management offers high-perfor- A current trend in databases is to leverage the advan- mance capabilities for analyzing large quantities of data, the tages of both techniques by comprising both a row and a row-oriented store can handle transactional point queries as column store (e.g., see [13]). Such hybrid-store databases of- well as inserts and updates more efficiently. To effectively fer high-performance transactions and analyses at the same take advantage of both stores at the same time the novel time. They provide the capability to increase the efficiency question whether to store the given data row- or column- of operational BI for enterprises, where in most cases mixed oriented arises.
    [Show full text]