Stable/Src/Hdmf
Total Page:16
File Type:pdf, Size:1020Kb
HD HierarchicalM DataF Modeling Framework HDMF Release 3.1.1 Aug 13, 2021 Getting Started 1 Installing HDMF 3 1.1 Option 1: Using pip...........................................3 1.2 Option 2: Using conda..........................................3 2 Tutorials 5 2.1 DynamicTable Tutorial..........................................5 2.2 MultiContainerInterface.........................................9 2.3 AlignedDynamicTable.......................................... 13 2.4 ExternalResources............................................ 19 2.5 DynamicTable How-To Guide...................................... 24 3 Introduction 39 4 Software Architecture 41 4.1 Main Concepts.............................................. 44 4.2 Additional Concepts........................................... 46 5 Citing HDMF 49 5.1 BibTeX entry............................................... 49 5.2 Using RRID............................................... 49 5.3 Using duecredit.............................................. 49 6 API Documentation 51 6.1 hdmf.common package.......................................... 51 6.2 hdmf.container module.......................................... 73 6.3 hdmf.build package........................................... 76 6.4 hdmf.spec package............................................ 91 6.5 hdmf.backends package......................................... 108 6.6 hdmf.data_utils module......................................... 120 6.7 hdmf.utils module............................................ 124 6.8 hdmf.validate package.......................................... 128 6.9 hdmf.testing package........................................... 133 6.10 hdmf package............................................... 134 7 Extending Standards 139 7.1 Creating new Extensions......................................... 139 7.2 Saving Extensions............................................ 141 i 7.3 Incorporating extensions......................................... 142 7.4 Documenting Extensions......................................... 144 7.5 Further Reading............................................. 144 8 Building API Classes 145 8.1 The register_class function/decorator.................................. 145 9 Export 147 9.1 FAQ.................................................... 147 10 Validating HDMF Data 149 11 Installing HDMF for Developers 151 11.1 Set up a virtual environment....................................... 151 11.2 Install from GitHub........................................... 152 11.3 Run tests................................................. 152 11.4 Install latest pre-release......................................... 153 12 Contributing Guide 155 12.1 Code of Conduct............................................. 155 12.2 Types of Contributions.......................................... 155 12.3 Contributing Patches and Changes.................................... 156 12.4 Styleguides................................................ 157 12.5 Endorsement............................................... 157 12.6 License and Copyright.......................................... 157 13 How to Make a Roundtrip Test 159 13.1 H5RoundTripMixin ......................................... 159 14 Software Process 161 14.1 Continuous Integration.......................................... 161 14.2 Coverage................................................. 161 14.3 Requirement Specifications....................................... 161 14.4 Versioning and Releasing........................................ 162 15 How to Make a Release 163 15.1 Prerequisites............................................... 163 15.2 Documentation conventions....................................... 164 15.3 Setting up environment.......................................... 164 15.4 PyPI: Step-by-step............................................ 164 15.5 Conda: Step-by-step........................................... 165 16 How to Update Requirements Files 167 16.1 requirements.txt............................................. 167 16.2 requirements-(dev|doc).txt........................................ 167 16.3 requirements-min.txt........................................... 168 17 Copyright 169 18 License 171 Python Module Index 173 Index 175 ii HDMF, Release 3.1.1 HDMF is a Python package for working with standardizing, reading, and writing hierarchical object data. HDMF is a by-product of the Neurodata Without Borders: Neurophysiology (NWB) project. The goal of NWB was to enable collaborative science within the neurophysiology and systems neuroscience communities through data standardization. The team of neuroscientists and software developers involved with NWB recognize that adoption of a unified data format is an important step toward breaking down the barriers to data sharing in neuroscience. HDMF was central to the NWB development efforts, and has since been split off with the intention of providing it as an open-source tool for other scientific communities. If you use HDMF in your research, please use the following citation: A. J. Tritt et al., “HDMF: Hierarchical Data Modeling Framework for Modern Science Data Standards,” 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, 2019, pp. 165-179, doi: 10.1109/BigData47090.2019.9005648. Getting Started 1 HDMF, Release 3.1.1 2 Getting Started CHAPTER 1 Installing HDMF HDMF requires having Python 3.6, 3.7, or 3.8 installed. If you don’t have Python installed and want the simplest way to get started, we recommend you install and use the Anaconda Distribution. It includes Python, NumPy, and many other commonly used packages for scientific computing and data science. HDMF can be installed with pip, conda, or from source. HDMF works on Windows, macOS, and Linux. 1.1 Option 1: Using pip If you are a beginner to programming in Python and using Python tools, we recommend that you install HDMF by running the following command in a terminal or command prompt: pip install hdmf 1.2 Option 2: Using conda You can also install HDMF using conda by running the following command in a terminal or command prompt: conda install-c conda-forge hdmf 3 HDMF, Release 3.1.1 4 Chapter 1. Installing HDMF CHAPTER 2 Tutorials 2.1 DynamicTable Tutorial This is a tutorial for interacting with DynamicTable objects. This tutorial is written for beginners and does not de- scribe the full capabilities and nuances of DynamicTable functionality. Please see the DynamicTable How-To Guide for more complete documentation. This tutorial is designed to give you basic familiarity with how DynamicTable works and help you get started with creating a DynamicTable, adding columns and rows to a DynamicTable, and accessing data in a DynamicTable. 2.1.1 Introduction The DynamicTable class represents a column-based table to which you can add custom columns. It consists of a name, a description, a list of row IDs, and a list of columns. 2.1.2 Constructing a table To create a DynamicTable, call the constructor for DynamicTable with a string name and string description. from hdmf.common import DynamicTable users_table= DynamicTable( name='users', description='a table containing data/metadata about users, one user per row', ) 2.1.3 Adding columns You can add columns to a DynamicTable using DynamicTable.add_column. 5 HDMF, Release 3.1.1 users_table.add_column( name='first_name', description='the first name of the user', ) users_table.add_column( name='last_name', description='the last name of the user', ) 2.1.4 Adding ragged array columns You may want to add columns to your table that have a different number of entries per row. This is called a “ragged array column”. To do this, pass index=True to DynamicTable.add_column. users_table.add_column( name='phone_number', description='the phone number of the user', index=True, ) 2.1.5 Adding rows You can add rows to a DynamicTable using DynamicTable.add_row. You must pass in a keyword argument for every column in the table. Ragged array column arguments should be passed in as lists or numpy arrays. The ID of the row will automatically be set and incremented for every row, starting at 0. # id will be set to 0 automatically users_table.add_row( first_name='Grace', last_name='Hopper', phone_number=['123-456-7890'], ) # id will be set to 1 automatically users_table.add_row( first_name='Alan', last_name='Turing', phone_number=['555-666-7777','888-111-2222'], ) 2.1.6 Displaying the table contents as a pandas DataFrame pandas is a popular data analysis tool for working with tabular data. Convert your DynamicTable to a pandas DataFrame using DynamicTable.to_dataframe. users_df= users_table.to_dataframe() users_df Accessing the table as a DataFrame provides you with powerful methods for indexing, selecting, and querying tabular data from pandas. Get the “last_name” column as a pandas Series: 6 Chapter 2. Tutorials HDMF, Release 3.1.1 users_df['last_name'] Out: id 0 Hopper 1 Turing Name: last_name, dtype: object The index of the DataFrame is automatically set to the table IDs. Get the row with ID = 0 as a pandas Series: users_df.loc[0] Out: first_name Grace last_name Hopper phone_number [123-456-7890] Name: 0, dtype: object Get single cells of the table by indexing with both ID and column name: print('My first user:', users_df.loc[0,'first_name'], users_df.loc[0,'last_name']) Out: My first user: Grace Hopper 2.1.7 Adding columns that reference rows of other DynamicTable objects You can create a column that references rows of another DynamicTable. This is analogous to a foreign key in a relational database. To do this, use the table