ANU Data Uploader User Manual

ANU Data Commons ANU Data Uploader User Manual

of 11 This work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 3.0 Australia License.

of 11 Contents

of 11 Overview This document lists and explains the steps required to install and use ANU Data Uploader, a component of the ANU Data Commons software.

License Use of ANU Data Uploader is governed by the GNU GPL3 license.

Acknowledgement This project is supported by the Australian National Data Service (ANDS). ANDS is supported by the Australian Government through the National Collaborative Research Infrastructure Strategy Program and the Education Investment Fund (EIF) Super Science Initiative.

Description ANU Data Uploader is a client application that enables creating collections and uploading datasets to them in a scripted, non-interactive manner. It also provides a user-friendly graphical interface to manage files uploaded to collections. It is distributed as a single Zip file that must be expanded into a directory from where the program can be executed.

System Requirements As ANU Data Uploader is a Java application, it can run on any platform that supports Java. This requires Java installed and configured on the system that will run this program. Java can be downloaded from http://www.oracle.com/technetwork/java/javase/downloads/index.html .

A text editor will be required to make configuration changes, although most operating systems have a basic text editor built-in. Windows has Notepad and Linux has vi or vim.

Files in Redistributable Package  DcClient-0.0.1-SNAPSHOT.jar The JAR file containing java classes and dependencies.

 DcClient.bat File to execute the application in Windows. Contains configuration values such as Data Commons' URL, path of local directory where bags are stored and other JVM options.

 DcClient.sh File to execute the application in Linux/Unix/Mac OS. Contains configuration values such as Data Commons' URL, path of local directory where bags are stored and other JVM options.

 Readme.txt Contains syntax information about how to use the program.

of 11  Sample CollInfo.txt Template for a collection information parameter file.

 cacerts File containing public keys of self-signed certificates used by non-production Web Application servers. If required, the public certificates can be extracted from this file and imported into your own CA certs list.

Installation To install the program in Windows simply extract the contents of the Zip file into an empty directory to which the user logged in has read and write access.

The application is provided its configuration settings by the bat or shell script that executes that Jar file. If using the program on Windows, open the file DcClient.bat else open DcClient.sh for editing.

Modify these variables:

 AppServer: refers to the base URL of the Data Commons web application.

 BagsDir: refers to the location where collection files will be stored on the local drive. When changing this value, ensure that the user profile under which the program is executed has read and write access to that directory. On Windows, the script defaults this value to a subdirectory called 'Bags' within the program directory. On *nix, it defaults to the subdirectory 'Bags' in the user's home directory. The directory will be created on program start if it does not exist.

On Linux, the permissions of the shell script may require changing to include the execute bit. Run the following command in the program directory:

chmod a+x DcClient.sh

Graphical User Interface To start the application in GUI mode, run the .bat or .sh file without any command line parameters:

Windows:

DcClient.bat

Linux:

./DcClient.sh

That will bring up a window similar to the one below:

of 11 For the program to be able to communicate the server, you must login with your credentials. To do so, click on the Login button in the top right corner of the window, type in your username and password in the Login dialog box and click OK.

of 11 On successfully logging in, the Login button's text changes to display the username and display name of the user. Simply close the window when done using the program – you do not need to logout from the server.

Once logged in you can switch users by clicking the button and entering the credentials of the new user.

Create/Download a Bag All files uploaded to a collection are combined into a container format called a BagIt Archive, or Bag for short. ANU Data Uploader downloads an entire bag at a time and uploads a bag by packaging files on the client machine.

of 11 To download the bag of a collection, enter that collection's Identifier in the Pid text box and click Get.

If the collection record specified does not have any files uploaded to it, you will be asked to create a bag. Files can then be added to this bag, saved, and uploaded to Data Commons.

While the bag is being downloaded, a progress window similar to the one below is displayed:

Once the download is finished, the files in the collection are displayed in the Local Bag tab.

Making Changes to a Downloaded Bag The files in the collection can then be copied to any location on the local machine by dragging a file or directory from the ANU Data Uploader Window and dropping into a local directory. Similarly, files can be added into the collection by dragging files from a local directory into the data directory in the Local Bag pane. Files in a collection can also be deleted and renamed by right clicking on them that pops up a context menu similar to the one below:

of 11 Saving a Bag After making changes to the files in the collection in the ANU Data Uploader application, the changed files must be packaged into a Bag before it can be uploaded to the Data Commons. To do so, click the Save button.

This will save the files and create or update supplementary files as required for a BagIt Archive making the archive ready for upload.

Uploading a Bag Before a bag can be uploaded, ensure that it has been saved as described in the section Saving a Bag. Click on the Upload button to begin the uploading process during which a progress dialog will be displayed.

of 11 Uploading a bag replaces the bag already in Data Commons with the one being uploaded. The existing bag is archived and its files not available for download.

Command Line Interface

Creating Collection and Uploading Files Creating a collection and uploading files to it through the command line requires a collection parameter file for each record. A sample collection parameter file is provided in the redistributable package that can be copied and modified for each collection to be created. A collection parameter file stores data in a simple Key=Value format. Refer to the comments in the template file and make changes as necessary.

Once the template file is ready, execute the following command:

DcClient –c COLLECTION_FILENAME –u USERNAME –p PASSWORD Download a Single Collection's Files To download a single collection's files to your local drive execute the following command:

DcClient –d PID –u USERNAME –p PASSWORD

of 11 Save Files of a Collection To save the files after making changes to the files locally stored execute the following command:

DcClient –s PID –u USERNAME –p PASSWORD Upload Files to a Collection To upload a collection's files after saving them execute the following command:

DcClient –l PID –u USERNAME –p PASSWORD

Storing Login Credentials in a Text File To eliminate the need to include a user's login credentials in the command line it is recommended that they be stored in a file in the following format:

username=[USERNAME]

password=[PASSWORD]

Then modify the permissions of the file so only the user has read and write permissions to it.

Then execute any of the commands mentioned above with the '-p' switch removed and the argument for '-u' as the path to the password file. For example, if the file containing login credentials is stored in C:\Users\Abc\Password.txt:

DcClient –d PID –u "C:\Users\Abc\Password.txt"