Andromeda Documentation Release 1.0.0-Beta

Andromeda Documentation Release 1.0.0-beta

Mike Lloyd

August 05, 2015

Contents

1 Contents 3 1.1 Quickstart...... 3 1.2 Commands...... 3 1.3 Andromeda Manifest Format...... 9 1.4 Local Manifest...... 12 1.5 OpenStack Manifest...... 12

i ii Andromeda Documentation, Release 1.0.0-beta

Andromeda is a Python-based repository management tool for the OpenStack repositories. This is a refactor of the repo project for Android, but with some differences. While it is mostly identical for most intents and purposes, there are some differences: • Works in Python 2.6-2.7, migration to Python 3.x will be complete soon. • More PEP 8 compliant. • It is configured for the OpenStack Gerrit instance, not Google’s. • Andromeda does not assume Gerrit user name is the first part of your email address...it must be set in the Git global configuration. • This is an actual package instead of running curl http://... > file. • Better documentation.

Contents 1 Andromeda Documentation, Release 1.0.0-beta

2 Contents CHAPTER 1

Contents

1.1 Quickstart

For the impatient or those that want to ﬁgure it out independently, here is the quick start variant with the OpenStack Sandbox repository: git config --global gitreview.username yourgerritusername andromeda init -u https://github.com/mynameismevin/andromeda-sandbox --config-name andromeda sync -j8 andromeda start test-branch-01 sandbox cd sandbox && touch "test-file-01" && echo "Andromeda is awesome!" > test-file-01 && git add . && git commit -m "Andromeda still rocks!" && cd .." andromeda upload

1.1.1 Quickstart Explained

Here is a quick and dirty explanation of what just happened. 1. Your Gerrit user name was configured globally for OpenStack. 2. Andromeda was initialised in the current directory and verified your information. 3. Andromeda pulled the remote repositories locally with 8 threads. 4. Andromeda initialised a new branch, test-branch-01, in the sandbox repository. 5. Added a new file to the sandbox and committed it. 6. Created a review in the OpenStack Review board. For more information on specific commands, please read the rest of the documentation. If you’ve ever used repo by Android, then you’ll be fine, just replace repo with andromeda.

1.2 Commands

Here is a list of the current commands that Andromeda supports: abandon, branch, branches, checkout, diff, download, grep, init, prune, rebase, smartsync, stage, start, status, sync, upload

3 Andromeda Documentation, Release 1.0.0-beta

1.2.1 Abandon andromeda abandon permanently abandons a development branch by deleting it (and all its history) from your local repository. It is equivalent to git branch -D .

1.2.2 Branch/Branches

Summarizes the currently available topic branches.

Branch Display

The branch display output by this command is organized into four columns of information; for example:

*P nocolor | in repo repo2 |

The ﬁrst column contains a * if the branch is the currently checked out branch in any of the speciﬁed projects, or a blank if no project has the branch checked out. The second column contains either blank, p or P, depending upon the upload status of the branch: (blank): branch not yet published by andromeda upload

P: all commits were published by andromeda upload p: only some commits were published by andromeda upload

The third column contains the branch name. The fourth column (after the | separator) lists the projects that the branch appears in, or does not appear in. If no project list is shown, then the branch appears in all projects.

1.2.3 Checkout andromeda checkout checks out an existing branch that was previously created by ‘andromeda start’. The command is equivalent to: andromeda forall [...] -c git checkout

1.2.4 Diff andromeda diff shows the differences between the latest commit and your working tree.

1.2.5 Download andromeda download downloads a change from the OpenStack Review system and makes it available in your project’s local working directory.

4 Chapter 1. Contents Andromeda Documentation, Release 1.0.0-beta

1.2.6 Forall

Executes the same shell command in each project.

Output Formatting

The -p option causes Andromeda to bind pipes to the command’s stdin, stdout and stderr streams, and pipe all output into a continuous stream that is displayed in a single pager session. Project headings are inserted before the output of each command is displayed. If the command produces no output in a project, no heading is displayed. The formatting convention used by -p is very suitable for some types of searching, e.g. andromeda forall -p -c git log -SFoo will print all commits that add or remove references to Foo. The -v option causes Andromeda to display stderr messages if a command produces output only on stderr. Normally the -p option causes command output to be suppressed until the command produces at least one byte of output on stdout.

Environment

The pwd variable is the project’s working directory. If the current client is a mirror client, then pwd is the Git repository. ANDROMEDA_PROJECT is set to the unique name of the project. ANDROMEDA_PATH is the path relative the the root of the client. ANDROMEDA_REMOTE is the name of the remote system from the manifest. ANDROMEDA_LREV is the name of the revision from the manifest, translated to a local tracking branch. If you need to pass the manifest revision to a locally executed git command, use ANDROMEDA_LREV. ANDROMEDA_RREV is the name of the revision from the manifest, exactly as written in the manifest. Shell positional arguments ($1 $2 .. $#) are set to any arguments following . Unless -p is used, stdin, stdout, stderr are inherited from the terminal and are not redirected.

1.2.7 Grep

Search for the speciﬁed patterns in all project ﬁles.

Boolean Options

The following options can appear as often as necessary to express the pattern to locate: -e PATTERN --and, --or, --not, -(, -)

Further, the -r/--revision option may be specified multiple times in order to scan multiple trees. If the same file matches in more than one tree, only the first result is reported, prefixed by the revision name it was found under.

Examples

Look for a line that has ’#define’ and either ’MAX_PATH or ’PATH_MAX’:

1.2. Commands 5 Andromeda Documentation, Release 1.0.0-beta

andromeda grep -e '#define' --and -\$ -e MAX_PATH -e PATH_MAX \$

Look for a line that has ’NODE’ or ’Unexpected’ in ﬁles that contain a line that matches both expressions: andromeda grep --all-match -e NODE -e Unexpected

1.2.8 Init

Initialize andromeda in the current directory. The andromeda init command is run once to install and initialize andromeda. The latest andromeda source code and manifest collection is downloaded from the server and is installed in the .andromeda/ directory. The optional -b argument can be used to select the manifest branch to checkout and use. If no branch is specified, master is assumed. The optional -m argument can be used to specify an alternate manifest to be used. If no manifest is specified, the manifest default.xml will be used. The --reference option can be used to point to a directory that has the content of a --mirror sync. This will make the working directory use as much data as possible from the local reference directory when fetching from the server. This will make the sync go a lot faster by reducing data traffic on the network.

Switching Manifest Branches

To switch to another manifest branch, andromeda init -b otherbranch may be used in an existing client. However, as this only updates the manifest, a subsequent andromeda sync (or andromeda sync -d) is necessary to update the working directory ﬁles.

1.2.9 Prune

Prune (delete) already merged topics.

1.2.10 Rebase

Rebase local branches on upstream branch. andromeda rebase uses git rebase to move local changes in the current topic branch to the HEAD of the upstream history, useful when you have made commits in a topic branch but need to incorporate new upstream changes “underneath” them.

1.2.11 Smartsync andromeda smartsync is a shortcut for sync -s.

1.2.12 Stage andromeda stage stages ﬁles to prepare the next commit.

6 Chapter 1. Contents Andromeda Documentation, Release 1.0.0-beta

1.2.13 Start

Start a new branch for development. andromeda start begins a new branch of development, starting from the revision speciﬁed in the manifest.

1.2.14 Status

Show the working tree status. andromeda status compares the working tree to the staging area (aka the index), and the most recent commit on this branch (HEAD), in each project specified. A summary is displayed, one line per file where there is a difference between these three states. The -j/--jobs option can be used to run multiple status queries in parallel. The -o/--orphans option can be used to show objects that are in the working directory, but not associated with an andromeda project. This includes unmanaged top-level files and directories, but also includes deepe items. For example, if dir/subdir/proj1 and dir/subdir/proj2 are andromeda projects, dir/subdir/proj3 will be shown if it is not known to andromeda.

1.2.15 Status Display

The status display is organized into three columns of information, for example if the ﬁle ‘subcmds/status.py’ is modi- ﬁed in the project ‘devstack’ on branch ‘devwork’: project devstack/ branch devwork -m subcmds/status.py

The ﬁrst column explains how the staging area (index) differs from the last commit (HEAD). Its values are always displayed in upper case and have the following meanings: -: no difference A: added (not in HEAD, in index ) M: modified ( in HEAD, in index, different content ) D: deleted ( in HEAD, not in index ) R: renamed (not in HEAD, in index, path changed ) C: copied (not in HEAD, in index, copied from another) T: mode changed ( in HEAD, in index, same content ) U: unmerged; conflict resolution required

The second column explains how the working directory differs from the index. Its values are always displayed in lower case and have the following meanings: -: new / unknown (not in index, in work tree ) m: modified ( in index, in work tree, modified ) d: deleted ( in index, not in work tree )

1.2.16 Sync

Updates the working tree to the latest revision. The andromeda sync command synchronizes local project directories with the remote repositories speciﬁed in the manifest. If a local project does not yet exist, it will clone a new local directory from the remote repository and set up tracking branches as speciﬁed in the manifest. If the local project already exists, ‘andromeda sync’ will update the remote branches and rebase any new local changes on top of the new remote changes.

1.2. Commands 7 Andromeda Documentation, Release 1.0.0-beta andromeda sync will synchronize all projects listed at the command line. Projects can be specified either by name, or by a relative or absolute path to the project’s local directory. If no projects are specified, andromeda sync will synchronize all projects listed in the manifest. The -d/--detach option can be used to switch specified projects back to the manifest revision. This option is especially helpful if the project is currently on a topic branch, but the manifest revision is temporarily needed. The -s/--smart-sync option can be used to sync to a known good build as specified by the manifest-server element in the current manifest. The -t/--smart-tag option is similar and allows you to specify a custom tag/label. The -u/--manifest-server-username and -p/--manifest-server-password options can be used to specify a username and password to authenticate with the manifest server when using the -s or -t option. If -u and -p are not specified when using the -s or -t option, andromeda sync will attempt to read authentication credentials for the manifest server from the user’s .netrc file. andromeda sync will not use authentication credentials from -u/-p or .netrc if the manifest server specified in the manifest file already includes credentials. The -f/--force-broken option can be used to proceed with syncing other projects if a project sync fails. The --no-clone-bundle option disables any attempt to use $URL/clone.bundle to bootstrap a new Git repository from a resumeable bundle file on a content delivery network. This may be necessary if there are problems with the local Python HTTP client or proxy configuration, but the Git binary works. The --fetch-submodules option enables fetching Git submodules of a project from server. The -c/--current-branch option can be used to only fetch objects that are on the branch specified by a project’s revision. The --optimized-fetch option can be used to only fetch projects that are fixed to a sha1 revision if the sha1 revision does not already exist locally.

SSH Connections

If at least one project remote URL uses an SSH connection (ssh://, git+ssh://, or user@host:path syn- tax) andromeda will automatically enable the SSH ControlMaster option when connecting to that host. This feature permits other projects in the same andromeda sync session to reuse the same SSH tunnel, saving connection setup overheads. To disable this behavior on UNIX platforms, set the GIT_SSH environment variable to ssh. For example: export GIT_SSH=ssh andromeda sync

Compatibility

This feature is automatically disabled on Windows, due to the lack of UNIX domain socket support. This feature is not compatible with url.insteadof rewrites in the user’s ~/.gitconfig. andromeda sync is currently not able to perform the rewrite early enough to establish the ControlMaster tunnel. If the remote SSH daemon is Gerrit Code Review, version 2.0.10 or later is required to ﬁx a server side protocol bug.

1.2.17 Upload

Upload changes for code review.

8 Chapter 1. Contents Andromeda Documentation, Release 1.0.0-beta

The andromeda upload command is used to send changes to the OpenStack Review system. It searches for topic branches in local projects that have not yet been published for review. If multiple topic branches are found, andromeda upload opens an editor to allow you to select which branches to upload. andromeda upload searches for uploadable changes in all projects listed at the command line. Projects can be speciﬁed either by name, or by a relative or absolute path to the project’s local directory. If no projects are speciﬁed, andromeda upload will search for uploadable changes in all projects listed in the manifest. If the --reviewers or --cc options are passed, those emails are added to the respective list of users, and emails are sent to any new users. Users passed as --reviewers must already be registered with the OpenStack Review system, or the upload will fail.

OpenStack Review URL

https://review.openstack.org/

Conﬁguration

These are read from each .git/config file the OpenStack repositories by default, but can be changed globally. To change this globally, review.URL.autoupload: To disable the Upload ... (y/N)? prompt, you can set a per-project or global Git configuration option. If review.URL.autoupload is set to true then repo will assume you always answer y at the prompt, and will not prompt you further. If it is set to false then andromeda will assume you always answer n, and will abort. review.URL.autoreviewer: To automatically append a user or mailing list to reviews, you can set a per-project or global Git option to do so. review.URL.autocopy: To automatically copy a user or mailing list to all uploaded reviews, you can set a per-project or global Git option to do so. Specifically, review.URL.autocopy can be set to a comma separated list of reviewers who you always want copied on all uploads with a non-empty --re argument. review.URL.uploadtopic: To add a topic branch whenever uploading a commit, you can set a per-project or global Git option to do so. If review.URL.uploadtopic is set to true then andromeda will assume you always want the equivalent of the -t option to the andromeda command. If unset or set to false then andromeda will make use of only the command line option.

1.3 Andromeda Manifest Format

An andromeda manifest describes the structure of the andromeda client; that is the directories that are visible and where they should be obtained from with git. The basic structure of a manifest is a bare Git repository holding a single default.xml XML ﬁle in the top level directory. Manifests are inherently version controlled, since they are kept within a Git repository. Updates to manifests are automatically obtained by clients during andromeda sync.

1.3. Andromeda Manifest Format 9 Andromeda Documentation, Release 1.0.0-beta

1.3.1 XML File Format

A manifest XML ﬁle (e.g. default.xml) roughly conforms to the following DTD:

A description of the elements and their attributes follows.

1.3.2 Element manifest

The root element of the ﬁle.

1.3.3 Element remote

One or more remote elements may be specified. Each remote element specifies a Git URL shared by one or more projects and (optionally) the OpenStack Review server those projects upload changes through. name: A short name unique to this manifest file. The name specified here is used as the remote name in each project’s .git/config, and is therefore automatically available to commands like git fetch, git remote, git pull and git push. fetch: The Git URL prefix for all projects which use this remote. Each project’s name is appended to this prefix to form the actual URL used to clone the project. review: Hostname of the Gerrit server where reviews are uploaded to by andromeda upload. This attribute is optional; if not specified then andromeda upload will not function.

10 Chapter 1. Contents Andromeda Documentation, Release 1.0.0-beta

1.3.4 Element default

At most one default element may be speciﬁed. Its remote and revision attributes are used when a project element does not specify its own remote or revision attribute. remote: Name of a previously deﬁned remote element. Project elements lacking a remote attribute of their own will use this remote. revision: Name of a Git branch (e.g. master or refs/heads/master). Project elements lacking their own revision attribute will use this revision.

1.3.5 Element manifest-server

At most one manifest-server may be speciﬁed. The url attribute is used to specify the URL of a manifest server, which is an XML RPC service that will return a manifest in which each project is pegged to a known good revision for the current branch and target. The manifest server should implement: GetApprovedManifest(branch, target)

The target to use is deﬁned by environment variables TARGET_PRODUCT and TARGET_BUILD_VARIANT. These variables are used to create a string of the form $TARGET_PRODUCT-$TARGET_BUILD_VARIANT, e.g. nova-userdebug. If one of those variables or both are not present, the program will call GetApprovedManifest without the target paramater and the manifest server should choose a reasonable default target.

1.3.6 Element project

One or more project elements may be specified. Each element describes a single Git repository to be cloned into the repo client workspace. name: A unique name for this project. The project’s name is appended onto its remote’s fetch URL to generate the actual URL to configure the Git remote with. The URL gets formed as: ${remote_fetch}/${project_name}.git where ${remote_fetch} is the remote’s fetch attribute and ${project_name} is the project’s name attribute. The suffix .git is always appended as repo assumes the upstream is a forrest of bare Git repositories. The project name must match the name Gerrit knows, as Gerrit is being used for code reviews. path: An optional path relative to the top directory of the andromeda client where the Git working directory for this project should be placed. If not supplied the project name is used. remote: Name of a previously defined remote element. If not supplied the remote given by the default element is used. revision: Name of the Git branch the manifest wants to track for this project. Names can be relative to refs/heads (e.g. just master) or absolute (e.g. refs/heads/master). Tags and/or explicit SHA-1s should work in theory, but have not been extensively tested. If not supplied the revision given by the default element is used.

1.3.7 Element remove-project

Deletes the named project from the internal manifest table, possibly allowing a subsequent project element in the same manifest ﬁle to replace the project with a different source.

1.3. Andromeda Manifest Format 11 Andromeda Documentation, Release 1.0.0-beta

This element is mostly useful in the local_manifest.xml, where the user can remove a project, and possibly replace it with their own deﬁnition.

1.4 Local Manifest

Additional remotes and projects may be added through a local manifest, stored in $TOP_DIR/.repo/local_manifest.xml. For example: $ cat .andromeda/local_manifest.xml

Users may add projects to the local manifest prior to a andromeda sync invocation, instructing repo to automatically download and manage these extra projects.

1.5 OpenStack Manifest

OpenStack Manifest ﬁle:

12 Chapter 1. Contents