coala Documentation Release 0.4.1.dev0
The coala Developers
February 16, 2016
Home
i ii CHAPTER 1
coala Installation
This document contains information on how to install coala. Supported platforms are Linux and Windows. coala is known to work on OS X as well. coala is tested against CPython 3.3, 3.4 and 3.5. In order to run coala you need to install Python. It is recommended, that you install Python3 >= 3.3 from http://www.python.org. The easiest way to install coala is using pip (Pip Installs Packages). If you don’t already have pip, you can install it like described on https://pip.pypa.io/en/stable/installing.html. Note that pip is shipped with recent python versions by default.
1.1 System wide installation
The simplest way to install coa ais to do it system-wide. But, This is generally discouraged in favor or using a virtualenv. To install the latest most stable version of coala system-wide, use: $ pip3 install coala
To install the nightly build from our master branch, you can do:
Note: For this and all future steps, some steps require root access (also known as administrative privileges in Win- dows). Unix based (Max, Linux) - This can be achieved by using sudo in front of the command sudo command_name instead of command_name Windows - The easiest way on windows is to start a command prompt as an administrator and start setup.py.
$ pip3 install coala --pre
1.2 Installing inside a virtualenv
Virtualenv is probably what you want to use during development, you’ll probably want to use it there, too. You can read more about it at their documentation - http://virtualenv.readthedocs.org First, we need to install virtualenv to the system. This can be done with pip3 easily:
1 coala Documentation, Release 0.4.1.dev0
pip3 install virtualenv
Once you have virtualenv installed, just fire up a shell and create your own environment. I usually create a project folder and a venv folder: $ virtualenv venv
Now, whenever you want to work on the project, you only have to activate the corresponding environment. On Unix based systems (OSX and Linux) this can be done with: $ source venv/bin/activate
And on Windows this is done with: $ venv\scripts\activate
Finally, you install coala inside the activated vertualenv with: $ pip3 install coala
1.3 Installing coala from source
In order to install coala from source, it is recommended to install git. See http://git-scm.com/ for further information and a downloadable installer or use your package manager on linux to get it.
Note: Again, here it is recommended to install coala inside a virtualenv. This can be done by creating a virtualenv and running the installation commands after the virtualenv has been activated.
After having git installed, you can download the source code of coala with the following command: git clone https://github.com/coala-analyzer/coala/ cd coala
You can now install coala with a simple: python3 setup.py install
For the above to work, you may also need to install setuptools which can be installed by running pip3 install setuptools
You will have coala installed into your python scripts directory. On an unixoid system it is probably already available on your command line globally.
1.3.1 Alternate installation
If you want to install coala to an alternate location you can e.g. call python3 setup.py install --prefix=/your/prefix/location. Other options are documented on https://docs.python.org/3.3/install/#alternate-installation.
2 Chapter 1. coala Installation coala Documentation, Release 0.4.1.dev0
1.4 Dependencies
This section lists dependencies of coala that are not automatically installed. On Windows, you can get many with nuget (https://www.nuget.org/), on Mac Homebrew will help you installing dependencies (http://brew.sh/).
1.4.1 JS Dependencies coala features a lot of bears that use linters written in JavaScript. In order for them to be usable, you need to install them via npm (http://nodejs.org/): npm install -g jshint alex remark dockerfile_lint csslint coffeelint
If a bear still doesn’t work for you, please make sure that you have a decent version of npm installed. Many linux distributions ship a very old one.
1.4.2 Binary Dependencies
Some bears need some dependencies available: • PHPLintBear: Install php • IndentBear: Install indent (be sure to use GNU Indent, Mac ships a non-GNU version that lacks some func- tionality.) • CSharpLintBear: Install mono-mcs
1.4.3 Clang coala features some bears that make use of Clang. In order for them to work, you need to install libclang: • Ubuntu: apt-get install libclang1 • Fedora: dnf install clang-libs (Use yum instead of dnf on Fedora 21 or lower.) • ArchLinux: pacman -Sy clang • Windows: nuget install ClangSharp • OSX: brew install llvm --with-clang If those do not help you, search for a package that contains libclang.so. On windows, you need to execute this command to add the libclang path to the PATH variable permanently (you need to be an administrator): setx PATH "%PATH%;%cd%\ClangSharp.XXX\content\x86" \M For x86 python or for x64 python: setx PATH "%PATH%;%cd%\ClangSharp.XXX\content\x64" \M Replace “XXX” with the ClangSharp version you received from nuget.
1.4. Dependencies 3 coala Documentation, Release 0.4.1.dev0
1.4.4 Generating Documentation coala documentation can be generated by fetching the documentation requirements. This can be achieved by pip3 install -r docs-requirements.txt
To generate the documentation coala uses sphinx. Documentation can be generated by running the following command from the docs\ directory. make html
You can then open docs\_build\html\index.html in your favourite browser. See Writing Documentation for more information.
4 Chapter 1. coala Installation CHAPTER 2
coala Tutorial
Welcome to this little tutorial. It is meant to be a gentle introduction to the usage of coala.
2.1 Prerequisites
In order to complete this tutorial you will need coala installed. (Installation is actually not required but it’s more convenient and recommended.)
2.2 Get Some Code
In order to perform a static code analysis on your code you will need some code to check. If you do not have own code you want to check, you can retrieve our tutorial samples: git clone https://github.com/coala-analyzer/coala-tutorial.git
Please note that the commands given in this tutorial are intended for use with this sample code and may need minor adjustments.
2.3 Let’s Start!
There are two options how to let coala know what kind of analysis it should perform on which code.
2.3.1 Command Line Interface
In order to specify the files to analyze, you can use the --files argument of coala like demonstrated below. For all file paths, you can specify (recursive) globs. Because analysis routines can do many various things we named them Bears. Thus you can specify the kind of analysis with the --bears argument. Please type the following commands into the console: cd coala-tutorial coala --files=src/\*.c --bears=SpaceConsistencyBear --save
5 coala Documentation, Release 0.4.1.dev0
coala will now ask you for missing values that are needed to perform the analysis, which in this case is only the use_spaces setting. We recommend setting it to true. coala will now check the code and, in case you use the tutorial code, yield two results. In the case of the SpaceCon- sistencyBear you will probably not be able to see those trailing whitespace errors on your console (yet, we’re working on that). Feel free to experiment a bit. You’ve successfully analysed some code! But don’t stop reading - you don’t have to enter all those values again!
2.3.2 Configuration Files
coala supports a very simple configuration file. If you’ve executed the instructions from the CLI section above, coala will already have such a file readily prepared for you. Go, take a look at it: cat.coafile
This should yield something like this: [Default] bears = SpaceConsistencyBear files = src/*.c use_spaces = yeah
If you now invoke coala it will parse this .coafile from your current directory. This makes it easy to specify once for your project what is checked with which bears and make it available to all contributors. Feel free to play around with this file. You can either edit it manually or add/edit settings via coala --save ... invocations. If you want coala to save settings every time, you can add save = True manually into your .coafile.
2.4 Sections
Thats all nice and well but we also have a Makefile for our project we want to check. So let us introduce another feature of our configuration syntax: sections. The line [Default] specifies that everything below will belong to the Default section. If nothing is specified, a setting will implicitly belong to this section. Let’s check the line lengths of our Makefile: coala -S Makefiles.bears=LineLengthBear Makefiles.files=Makefile --save
As you can see, the -S (or --settings) option allows to specify arbitrary settings. Settings can be directly stored into a section with the section.setting syntax. By default the LineLengthBear checks whether each line contains 80 chars or less in a line. To change this value, use the max_line_length inside the .coafile. coala will now yield any result you didn’t correct last time plus a new one for the Makefile. This time coala (or better, the LineLengthBear) doesn’t know how to fix the issue but still tries to provide as much helpful information as possible and provides you the option to directly open the file in an editor of your choice.
Note: If your editor is already open this may not work, because the other process will shortly communicate with the existent process and return immediately. coala handles this for some editors automatically, if your’s does not work yet - please file a bug so we can include it!
6 Chapter 2. coala Tutorial coala Documentation, Release 0.4.1.dev0
If you changed one file in multiple results, coala will merge the changes if this is possible. coala should have appended something like this to your .coafile: [Makefiles] bears= LineLengthBear files= Makefile
As you see, sections provide a way to have different configurations for possibly different languages in one file. They are executed sequentially.
2.5 Auto-applying results
Often you don’t want to look at trivial results like spacing issues. For that purpose coala includes a special setting called default_actions that allows you to set the action for a bear that shall be automatically applied on run. Let’s automatically fix python code. Take a look at our sample python code: $ cat src/add.py
""" This is a simple library that provide a function that can add numbers.
Cheers! """
def add(a,b): return a+b; import sys
That looks horrible, doesn’t it? Let’s fix it!
$ coala -S python.bears=PEP8Bear python.files=\*\*/\*.py \ python.default_actions=PEP8Bear:ApplyPatchAction --save # other output ... Executing section python... [INFO][11:03:37] Applied 'ApplyPatchAction' for 'PEP8Bear'. [INFO][11:03:37] Applied 'ApplyPatchAction' for 'PEP8Bear'. coala would now fix all spacing issues and without bothering you again. Currently following actions are available: • ApplyPatchAction: Applies a given patch (if existent). • ShowPatchAction: Just displays a given patch (if existent) without doing something. For debugging purposes: • PrintDebugMessageAction: Prints a debug message for the appearing result.
2.6 Setting Inheritance
All settings in the default section are implicitly inherited to all other sections (if they do not override their values). We can use that to save a few lines!
2.5. Auto-applying results 7 coala Documentation, Release 0.4.1.dev0
Lets add the following section to our .coafile: [TODOS] bears= KeywordBear
And execute coala with the -s argument which is the same as --save. I recommend setting case insensitive keywords to TODO, FIXME and case sensitive keywords empty. After the results we’ve already seen, we’ll see a new informational one which informs us that we have a TODO in our code. Did you note that we didn’t specify which files to check this time? This is because all settings, including files = src/*.c, from the Default section are already available in every other section implicitly. Thus the default section is a good point to set things like logging and output settings or specifying a default set of files to check.
2.7 Ignoring Issues
There are several ways to ignore certain issues, so you aren’t lost if any routines yield false positives.
2.7.1 Ignoring Files coala lets you ignore whole files through the ignore setting: files = **/*.h ignore = **/resources.h
This configuration would include all header (.h) files but leaves out resource headers.
2.7.2 Ignoring code Inside Files
Sometimes you need finer-graded ignores. Imagine you have a LineLengthBear that shall not run on some code segments, because you can’t wrap them: code="that's checked normally"
# Ignore LineLengthBear unwrappable_string="some string that is super-long and would exceed the limit"
You can also skip an area: # Start ignoring LineLengthBear unwrappable_string_2= unwrappable_string+"yeah it goes even further..." another_unwrappable_string= unwrappable_string+ unwrappable_string_2 # Stop ignoring
You can also conditionally combine ignore rules! Bear names will be split by comma and spaces, invalid bear names like and will be ignored. # Ignore LineLengthBear and SpaceConsistencyBear variable = "Why the heck are spaces used instead of tabs..." + "so_long"
If you put an all instead of the bear names directly after the ignore/ignoring keyword, the results of all bears affecting those lines will be ignored.
8 Chapter 2. coala Tutorial coala Documentation, Release 0.4.1.dev0
2.8 Enabling/Disabling Sections
Now that we have sections we need some way to control, which sections are executed. coala provides two ways to do that:
2.8.1 Manual Enabling/Disabling
If you add the line TODOS.enabled=False to some arbitrary place to your .coafile or just enabled=False into the TODOS section, coala will not show the TODOs on every run. Especially for those bears yielding informational messages which you might want to see from time to time this is a good way to silence them.
2.8.2 Specifying Targets
If you provide positional arguments, like coala Makefiles, coala will execute exclusively those sections that are specified. This will not get stored in your .coafile and will take precedence over all enabled settings. You can specify several targets seperated by a space. What was that TODO again?
2.9 Show bears’ information
To get help on using a bear or to get a description of the bear, use the --show-bears argument: coala--bears=SpaceConsistencyBear--show-bears
This will display a large amount of information regarding the bears that have been specified (in the .coafile of in the CLI arguments). It shows: • A description of what the bear does • The sections which uses it • The settings it uses (optional and required)
2.10 Integrating coala into Your Project
It’s easy to add coala to your project in a way that does not force your developers even to install coala using git submodules. This also has the advantage that all your developers are using exactly the same version of coala. You can try it out in the coala-tutorial repository: git submodule add https://github.com/coala-analyzer/coala.git git commit -m 'Add coala submodule' git add .coafile git commit -m 'Add .coafile'
You can now use coala/coala as if it were the installed binary. Here’s the instructions for your developers: git submodule init git submodule update coala/coala
2.8. Enabling/Disabling Sections 9 coala Documentation, Release 0.4.1.dev0
2.11 Continuing the Journey
If you want to know about more options, take a look at our help with coala -h. If you liked or disliked this tutorial, feel free to drop us a note at our bug tracker (github) or mailing list (https://groups.google.com/forum/#!forum/coala- devel). If you need more flexibility, know that coala is extensible in many ways due to its modular design: • If you want to write your own bears, take a look at sources lying in bears and coalib/bearlib. • If you want to add custom actions for results, take a look at the code in coalib/results/results_actions. • If you want to have some custom outputs (e.g. HTML pages, a GUI or voice interaction) take a look at modules lying in coalib/output. Happy coding!
10 Chapter 2. coala Tutorial CHAPTER 3
Guide to Write a Bear
Welcome. This document presents information on how to write a bear for coala. It assumes you know how to use coala. If not please read our main tutorial! The sample sources for this tutorial lie at our coala-tutorial repository, go clone it with: git clone https://github.com/coala-analyzer/coala-tutorial.git
All paths and commands given here are meant to be executed from the root directory of the coala-tutorial repository.
3.1 What is a bear?
A bear is meant to do some analysis on source code. The source code will be provided by coala so the bear doesn’t have to care where it comes from or where it goes. A bear can communicate with the user via two ways: • Via log messages • Via results Log messages will be logged according to the users settings and are usually used if something goes wrong. However you can use debug for providing development related debug information since it will not be shown to the user by default. If error/failure messages are used, the bear is expected not to continue analysis.
3.2 A Hello World Bear
Below is the code given for a simple bear that sends a debug message for each file: from coalib.bears.LocalBear import LocalBear class HelloWorldBear(LocalBear): def run(self, filename, file): self.debug("Hello World! Checking file", filename,".")
This bear is stored at ./bears/HelloWorldBear In order to let coala execute this bear you need to let coala know where to find it. We can do that with the -d (--bear-dirs) argument:
11 coala Documentation, Release 0.4.1.dev0
coala -f src/*.c -d bears -b HelloWorldBear -L DEBUG You should now see the debug message for our sample file. The Bear class also supports warn and err.
3.3 Communicating with the User
Now we can send messages through the queue, we can do the real work. Lets say: • We want some information from the user (e.g. the tab width if we rely on indentation) • We’ve got some useful information for the user and want to show it to him. This might be some issue with his code or just an information like the number of lines. So let’s extend our HelloWorldBear a bit, I’ve named the new bear with the creative name CommunicationBear: from coalib.bears.LocalBear import LocalBear from coalib.results.Result import Result class CommunicationBear(LocalBear): def run(self, filename, file, user_input: str): """ Communicates with the user.
:param user_input: Arbitrary user input. """ self.debug("Got'{ui}' as user input of type {type}.".format( ui=user_input, type=type(user_input)))
yield Result(message="A hello world result.", origin=self, file=filename)
Try executing it:
coala -f=src/\*.c -d=bears -b=CommunicationBear -L=DEBUG
Hey, we’ll get asked for the user_input! Wasn’t that easy? Go ahead, enter something and observe the output. So, what did coala do here? First, coala looked at the parameters of the run method and found that we need some value named user_input. Then it parsed our documentation comment and found a description for the parameter which was shown to us to help us choose the right value. After the needed values are provided, coala converts us the value into a string because we’ve provided the str annotation for this parameter. If no annotation is given or the value isn’t convertible into the desired data type, you will get a coalib.settings.Setting.Setting. Your docstring can also be used to tell the user what exactly your bear does. Try executing coala -d bears -b CommunicationBear --show-bears
This will show the user a bunch of information related to the bear like: - A description of what the bear does - The sections which uses it - The settings it uses (optional and required)
12 Chapter 3. Guide to Write a Bear coala Documentation, Release 0.4.1.dev0
3.3.1 What Data Types are Supported?
The Setting does support some very basic types: • String (str) • Float (float) • Int (int) • Boolean (bool, will accept values like true, yes, yeah, no, nope, false) • List of strings (list, values will be split by comma) • Dict of strings (dict, values will be split by comma and colon) If you need another type, you can write the conversion function yourself and use this function as the annotation. We’ve provided a few advanced conversions for you: • coalib.settings.Setting.path, converts to an absolute file path relative to the file/command where the setting was set • coalib.settings.Setting.path_list, converts to a list of absolute file paths relative to the file/command where the setting was set • coalib.settings.Setting.typed_list(typ), converts to a list and applies the given conversion (typ) to each element. • coalib.settings.Setting.typed_ordered_dict(key_type, value_type, default), converts to a dict while applying the key_type conversion to all keys, the value_type conversion to all values and uses the default value for all unset keys. Use typed_dict if the order is irrelevant for you.
3.4 Results
In the end we’ve got a result. If a file is provided, coala will show the file, if a line is provided, coala will also show a few lines before the affecting line. There are a few parameters to the Result constructor, so you can e.g. create a result that proposes a code change to the user. If the user likes it, coala will apply it automatically - you don’t need to care. Your function needs to return an iterable of Result objects: that means you can either return a list of Result objects or simply yield them and write the method as a generator.
Note: We are currently planning to simplify Bears for bear writers and us. In order to make your Bear future proof, we recommend writing your method in generator style. Don’t worry: in order to migrate your Bears to our new API, you will likely only need to change two lines of code. For more information about how bears will look in the future, please read up on https://github.com/coala- analyzer/coala/issues/725 or ask us on https://gitter.im/coala-analyzer/coala.
3.4. Results 13 coala Documentation, Release 0.4.1.dev0
14 Chapter 3. Guide to Write a Bear CHAPTER 4
The .coafile Specification
This document gives a short introduction into the specification of a coala configuration file. It is meant to be rather factual, if you wish to learn by example, please take a look at the coala Tutorial.
4.1 Naming, Scope and Location
You can use up to three coafiles to configure your project. 1. A project-wide coafile. 2. A user-wide coafile. 3. A system-wide coafile.
4.1.1 Project-Wide coafile
It is a convention that the project-wide coafile is named .coafile and lies in the project root directory. If you follow this convention, simply executing coala from the project root will execute the configuration specified in that file. Settings given in the project-wide coafile override all settings given by other files and can only be overridden by settings given via the command line interface.
4.1.2 User-Wide and System-Wide coafile
You can place a .coarc file in your home directory to set certain settings user wide. Those settings will automatically be taken for all projects executed with that user. All settings specified here override only settings given by the system wide coafile which has the lowest priority. The default_coafile must lie in the coala installation directory and is valid for everyone using this coala installation
4.2 Setting Inheritance
Every coafile consists out of one or more sections. Section names are case insensitive. The default section is implicitly available and all settings which have no section specified belong to it. The default section is special because all settings residing in it are automatically inherited to all other sections specified in the same coafile. This is an example coafile:
15 coala Documentation, Release 0.4.1.dev0
enabled= True overridable=2
[section-1] overridable=3 other=4
[section-2] overridable=5 other=2
This coafile would be interpreted the very same as this one, written a bit more explicitly: [default] enabled= True overridable=2
[section-1] enabled= True overridable=3 other=4
[section-2] enabled= True overridable=5 other=2
4.2.1 Comments, Escaping and Multiline Values and Keys
Comments are simply done with a preceding #. If you want to use a # within a value, you can simply escape it: a_key = a\#value # And a comment at the end!
Any line not containing an unescaped = is simply appended to the value of the last key: a_key = a value # this is not part of the value that /= is very long!
Similarly, you can also set a value to multiple keys: key_1, key_2 = value is equivalent to key_1 = value and key_2 = value in separate lines. As the backslash is the escape character it is recommended to use forward slashes as path seperator even on windows (to keep relative paths platform independent), use double-backslashes if you really mean a backslash in all places.
16 Chapter 4. The .coafile Specification CHAPTER 5
Glob - Extended unix style pathname expansion
In coala, files and directories are specified by file name. To allow input of multiple files without requiring a large number of filenames, coala supports a number of wildcards. These are based on the unix-style glob syntax and they are not the same as regular expressions.
5.1 Syntax
The special characters used in shell-style wildcards are: PATTERN MEANING ‘[seq]’ Matches any character in seq. Cannot be empty. Any special character looses its special meaning in a set. ‘[!seq]’ Matches any character not in seq. Cannot be empty. Any special character looses its special meaning in a set. ‘(seq_a|seq_b)’ Matches either sequence_a or sequence_b as a whole. More than two or just one sequence can be given. ‘?’ Matches any single character. ‘*’ Matches everything but the directory separator ‘**’ Matches everything.
5.2 Examples
5.2.1 ‘[seq]’
Matches any character in seq. Cannot be empty. Any special character looses its special meaning in a set. Opening and closing brackets can be part of a set, although closing brackets have to be placed at the first position. >>> from coalib.parsing.Globbing import fnmatch >>> fnmatch("aaa","a[abc]a") True >>> fnmatch("aaa","a[bcd]a") False >>> fnmatch("aaa","a[a]]a") False >>> fnmatch("aa]a","a[a]]a") True >>> fnmatch("aaa","a[]abc]a") True
17 coala Documentation, Release 0.4.1.dev0
>>> fnmatch("aaa","a[[a]a") True >>> fnmatch("a[a","a[[a]a") True >>> fnmatch("a]a","a[]]a") True >>> fnmatch("aa","a[]a") False >>> fnmatch("a[]a","a[]a") True
5.2.2 ‘[!seq]’
Matches any character not in seq. Cannot be empty. Any special character looses its special meaning in a set. >>> fnmatch("aaa","a[!a]a") False >>> fnmatch("aaa","a[!b]a") True >>> fnmatch("aaa","a[b!b]a") False >>> fnmatch("a!a","a[b!b]a") True >>> fnmatch("a!a","a[!]a") False >>> fnmatch("aa","a[!]a") False >>> fnmatch("a[!]a","a[!]a") True
5.2.3 ‘(seq_a|seq_b)’
Matches either sequence_a or sequence_b as a whole. More than two or just one sequence can be given. Parentheses cannot be part of an alternative, unless they are escaped by brackets. Parentheses that have no match are ignored as well as ‘|’-separators that are not inside matching parentheses
>>> fnmatch("aXb","a(X|Y)b") True >>> fnmatch("aYb","a(X|Y)b") True >>> fnmatch("aZb","a(X|Y)b") False >>> fnmatch("aXb","(a(X|Y)b|c)") True >>> fnmatch("c","(a(X|Y)b|c)") True >>> fnmatch("a","a|b") False >>> fnmatch("a|b","a|b") True >>> fnmatch("(a|b","(a|b") True >>> fnmatch("(aa","(a(a|b)") True
18 Chapter 5. Glob - Extended unix style pathname expansion coala Documentation, Release 0.4.1.dev0
>>> fnmatch("a(a","(a(a|b)") False >>> fnmatch("a(a","(a[(]a|b)") True >>> fnmatch("aa","a()a") True >>> fnmatch("","(abc|)") True
5.2.4 ‘?’
Matches any single character. >>> fnmatch("abc","a?c") True >>> fnmatch("abbc","a?c") False >>> fnmatch("a/c","a?c") True >>> fnmatch("a\\c","a?c") True >>> fnmatch("a?c","a?c") True >>> fnmatch("ac","a?c") False
5.2.5 ‘*’
Matches everything but the directory separator
Note: The directory separator is platform specific. ‘/’ is never matched by ‘*’. ‘\’ is matched on Linux, but not on Windows.
>>> fnmatch("abc","a *c") True >>> fnmatch("abbc","a *c") True >>> fnmatch("a/c","a *c") False >>> fnmatch("a?c","a *c") True >>> fnmatch("ac","a *c") True
5.2.6 ‘**’
Matches everything.
>>> fnmatch("abc","a **c") True >>> fnmatch("abbc","a **c") True
5.2. Examples 19 coala Documentation, Release 0.4.1.dev0
>>> fnmatch("a/c","a **c") True >>> fnmatch("a?c","a **c") True >>> fnmatch("ac","a **c") True
20 Chapter 5. Glob - Extended unix style pathname expansion CHAPTER 6
Exit Codes
The following is a list of coalas exit codes and their meanings: • 0 - coala executed succesfully but yielded no results. • 1 - coala executed succesfully but yielded results. • 2 - Invalid arguments were passed to coala in the command line. • 3 - The file collector exits with this code if an invalid pattern is passed to it. • 4 - coala was executed with an unsupported version of python • 5 - coala executed successfully. Results were found but patches to the results were applied successfully • 130 - A KeyboardInterrupt (Ctrl+C) was pressed during the execution of coala. • 255 - Any other general errors.
21 coala Documentation, Release 0.4.1.dev0
22 Chapter 6. Exit Codes CHAPTER 7
External APIs
Use coala to make your application better.
7.1 DBus
The coala DBus API essentially creates a server which can have multiple clients connected to it. The clients commu- nicate to the coala server using DBus. To start the server, first install coala and then simple run coala-dbus. This spawns a bus with the name org.coala_analyzer.v1 as a SessionBus and can be verified and tested using a DBus debugger like DFeet. The bus has 1 object-path by default - /org/coala_analyzer/v1. This is the first point of contact with coala for any client. The object-path /org/coala_analyzer/v1 has the interface org.coala_analyzer.v1 which contains the two methods - CreateDocument and DisposeDocument. These are used to tell coala about the documents you wish to analyze.
7.1.1 CreateDocument
Args: the path of the document Returns: object-path for the document A document is defined by it’s path. The path should be a absolute path, not a relative one. This method returns an object-path which will be hence forth used to interact with that document.
7.1.2 DisposeDocument
Args: the path to the document Returns: none It disposes the object-path corresponding to the given path. Now, the object-path returned by the CreateDocument method also has the interface org.coala_analyzer.v1. This interface is used to handle which config file coala will use to analyze the document, and the function to get analysis results. It contains 4 functions:
23 coala Documentation, Release 0.4.1.dev0
7.1.3 GetConfigFile
Args: none Returns: the config file path
7.1.4 SetConfigFile
Args: the config file path Returns: the config path which is set after it executes.
7.1.5 FindConfigFile
Args: the config file path Returns: the config path which is set after it executes. It attempts to find the config file related to the file by searching in parent directories till it finds a .coafile.
7.1.6 Analyze
Args: none Returns: an array of DBus structures containing: • The name of the section • Boolean which is true if all bears in the section executed successfully • List of results where each result is a list which contains: (str)origin, (str)message, (str)file, (str)line_nr, (str)severity
24 Chapter 7. External APIs CHAPTER 8
Linter Bears
Welcome. This tutorial aims to show you how to use the Lint class in order to integrate linters in your bears.
8.1 Why is This Useful?
A lot of programming languages already have linters implemented, so if your project uses a language that does not already have a linter Bear you might need to implement it on your own. Don’t worry, it’s easy!
8.2 What do we Need?
First of all, we need the linter that we are going to use. In this tutorial we will build the HTMLTutorialBear so we need an html linter. This one will do. Since it is a python package we can go ahead and install it with $ pip install html-linter
8.3 Writing the Bear
Since we are going to use the Lint class we should go ahead and import it together with LocalBear (all bears that handle only one file inherit from LocalBear). Also we will go ahead and write the class head. It should inherit both LocalBear and Lint. from coalib.bearlib.abstractions.Lint import Lint from coalib.bears.LocalBear import LocalBear
class HTMLTutorialBear(LocalBear, Lint):
To make our bear use a linter we will have to overwrite some of the predefined values of the Lint class. Some of the most important are executable and output_regex. We use executable to specify the linter executable. In our case it would be executable='html_lint.py'
The output_regex is used to group parts of the output (such as lines, columns, severity and message) so it can be used by the Lint class to yield Results. (more on communicating with the user Writing Bears) In order to figure out the output_regex we have to first see how the linter output looks. I will use this file as sample.html
25 coala Documentation, Release 0.4.1.dev0
Hello, world!
Now we can run $ html_lint.py sample.html 1:1: Info: Optional Tags: Omit optional tags (optional): You may remove the opening "html" tag. 2:3: Info: Optional Tags: Omit optional tags (optional): You may remove the opening "body" tag. 4:3: Info: Optional Tags: Omit optional tags (optional): You may remove the closing "body" tag. 5:1: Info: Optional Tags: Omit optional tags (optional): You may remove the closing "html" tag.
Our output_regex should look like this (line):(column): (severity): (message)
Or in python regex
(?P
Now that we found out our output_regex (we might want to test it just to be sure, I used https://regex101.com/#python). We can now compile and add our regex using the re python library like so (I split the regex into 2 lines to focus on readability) import re from coalib.bearlib.abstractions.Lint import Lint from coalib.bears.LocalBear import LocalBear class HTMLTutorialBear(LocalBear, Lint): executable='html_lint.py' output_regex= re.compile( r'(?P
26 Chapter 8. Linter Bears coala Documentation, Release 0.4.1.dev0
class HTMLTutorialBear(LocalBear, Lint): executable='html_lint.py' output_regex= re.compile( r'(?P
As with every other bear (see Writing Bears) we have to define our run method. def run(self, filename, file):
return self.lint(filename)
And that should be enough. The lint() method of the Lint class will do the rest for us. We can test our bear like this $ coala --bear-dirs=. --bears=HTMLTutorialBear --files=sample.html
Note: In order for the above command to work we should have 2 files in our current dir: HTMLTutorialBear.py and our sample.html. Naming is very important in coala. coala will look for bears by their filename and display them based on their classname.
8.4 Adding Settings to our Bear
If we run $ html_lint.py -h
We can see that there is a --disable option which lets us disable some checks. Let’s add that functionality to our bear. First of all we have to import the setting that we are going to use from coalib. Since --disable needs a comma separated list we can use a list to keep our options. For that we will import typed_list like so from coalib.settings.Setting import typed_list
typed_list(item_type) is a function that converts the given input (which the user will pass to the Bear as a setting) into a list of items and afterwards will apply a conversion to type item_type to each item in the list (you can also use basic types like int, bool, etc. see Writing Bears) Next, we have to add our setting as a parameter for the run() method of our bear. We will give the param a sugestive name like htmllint_ignore. def run(self, filename, file, htmllint_ignore: typed_list(str)=[]): ''' Checks the code with `html_lint.py` on each file separately.
8.4. Adding Settings to our Bear 27 coala Documentation, Release 0.4.1.dev0
:param htmllint_ignore: List of checkers to ignore. '''
Note: The documentation of the param is parsed by coala and it will be used as help to the user for that specific setting.
The last thing we need to do is join the strings in the html_ignore, append them to --disable= and add it as an argument. There are alot of ways of doing that. ignore=','.join(part.strip() for part in htmllint_ignore) self.arguments='--disable='+ ignore return self.lint(filename)
8.5 Finished Bear
Well done, you made it this far! Now you should have built a fully functional HTML Lint Bear. If you followed the code from this tutorial it should look something like this import re from coalib.bearlib.abstractions.Lint import Lint from coalib.bears.LocalBear import LocalBear from coalib.settings.Setting import typed_list from coalib.results.RESULT_SEVERITY import RESULT_SEVERITY class HTMLTutorialBear(LocalBear, Lint): executable='html_lint.py' output_regex= re.compile( r'(?P
def run(self, filename, file, htmllint_ignore: typed_list(str)=[]): ''' Checks the code with `html_lint.py` on each file separately.
:param htmllint_ignore: List of checkers to ignore. ''' ignore=','.join(part.strip() for part in htmllint_ignore) self.arguments='--disable='+ ignore return self.lint(filename)
28 Chapter 8. Linter Bears coala Documentation, Release 0.4.1.dev0
8.6 Running and Testing our Bear
By running $ coala --bear-dirs=. --bears=HTMLTutorialBear -B
We can see that our Bear setting is documented properly. To use coala with our Bear on sample.html we run $ coala --bear-dirs=. --bears=HTMLTutorialBear --files=sample.html
To use our htmllint_ignore setting we can do $ coala --bear-dirs=. --bears=HTMLTutorialBear -S htmllint_ignore=optional_tag --files=sample.html
This will not output anything because all the messages had the optional_tag. You now know how to write a linter Bear and also how to use it in your project. Congratulations!
8.6. Running and Testing our Bear 29 coala Documentation, Release 0.4.1.dev0
30 Chapter 8. Linter Bears CHAPTER 9
Getting Involved
We are working hard to make coala reality. If you want to help us you can do the following things: • Join us on our mailing list. • coala developer list: https://groups.google.com/forum/#!forum/coala-devel – [email protected] • coala announcement list: https://groups.google.com/forum/#!forum/coala-announcements – [email protected] • Join us on our developer chatroom: https://gitter.im/coala-analyzer/coala • Run our tests. Just follow the instructions on the Testing page. Report back to us if anything goes wrong unexpectedly. • Give us feedback. If you think we’re doing something useless or useful with some specific things, please write us! We are interested in your needs and favors so you get the highest possible quality. • Drive bug discussions. Just state your opinion, everywhere! • Review code. Get a glance at what we’re doing and say something if you don’t understand something or think we’re doing something wrong. • Write documentation. If you need some additional information for documenting feel free to ask us! • Write code. Usually code we need is indicated through bug reports. Please get in contact with us so we can coordinate coding together. Please take a look at the other files in this directory for getting started with specific parts of coala. We appreciate any help. (Partially with words, partially with chocolate if you live near hamburg.) Modularity, clean good code as well as a high usability for both users and developers of analyse routines (called bears) stand in the foreground of the development. We will not speed up our development if it needs sacrificing any aspect of quality.
31 coala Documentation, Release 0.4.1.dev0
32 Chapter 9. Getting Involved CHAPTER 10
Reviewing
This document is a guide to coalas review process. It will be extended over time.
10.1 Review Process
The review process for coala is as follows: 1. Anyone can submit commits for review. This usually happens on WIP branches, submitting goes via GitHub Pull Requests. 2. A reviewer reviews every commit by its own and validates that every commit is a good change and does not destroy anything. 3. If a commit does not fulfill the expectations of the reviewer, go to step one. 4. If the commits are not linearly mergeable into master, rebase and go to step one. 5. All commits are acknowledged and fit linearly onto master. All continuous integration services (as described below) pass. Anyone with collaborator permission may leave the @rultor merge command to get the PR merged automatically.
10.2 Continous Integration
It is only allowed to merge a pull request into master if all of the following apply: • The build/tests pass on all services. (circleci, appveyor) • Scrutinizer shows passed. (That is: no new issues, no new classes with rating D or worse, project quality metric may only get better.) • All statements and branches are covered by your tests. (codecov.io) The coverage values may go down by a commit, however this is to be avoided. Tests must work for every commit. Continuous integration is always done for the last commit on a pull request.
10.3 Reviewing Commits
• Generated code is not intended to be reviewed. Instead rather try to verify that the generation was done right. The commit message should expose that.
33 coala Documentation, Release 0.4.1.dev0
• Every commit is reviewed independently from the other commits. • Tests should pass for each commit. If you suspect that tests might not pass and a commit is not checked by continuous integration, try running the tests locally. • Check the surroundings. In many cases people forget to remove the import when removing the use of something or similar things. It is usually good to take a look at the whole file to see if it’s still consistent. • Check the commit message. • Take a look at continuous integration results in the end even if they pass: • Coverage shall not fall. • Scrutinizer oftentimes yields helpful results.
34 Chapter 10. Reviewing CHAPTER 11
Codestyle for coala
coala follows the PEP8 codestyle with a maximum line length of 80 characters including newline. Invoke coala to let it correct your code automatically.
11.1 Additional Style Guidelines
11.1.1 Documentation Comments
A documentation comment consists of 2 parts - the description of what the function/class/module does followed by the parameters it takes in, the return value it gives out and the exceptions it can raise. Nothing should be written on the first and last line where the docstring begins and ends, and a newline should sep- arate the description and the rest. Each message in the documentation comment must end with a full-stop. Also the description of all arguments and the return value should begin at the same column. Example: def area(length, breadth): """ Finds the area of a rectangle of the given length and breadth.
:param length: The length of the rectangle. :param breadth: The breadth of the rectangle. :return: The area of the rectangle. :raises ValueError: Raises ValueError if the arguments are not of type float or int. """
If the description for a param or other keywords exceeds 1 line, continue it in the next. Make sure that the second line is aligned Below the first line. Example : :param something: A very long line describing the variable something in great detail. :return: This message also started in the same column and it starts again at the same column as the rest of the messages.
35 coala Documentation, Release 0.4.1.dev0
11.1.2 Type Checking
If you want to assure that parameters have a certain type, you can use the enforce_signature decorator and simply annotate your function with the allowed types: @enforce_signature def concatenate_strings(a: str, b: str, c: (str, None)=None): if c is None: c="" return a+b+c
This will raise a TypeError if a, b or c are no strings and c is not None.
36 Chapter 11. Codestyle for coala CHAPTER 12
Testing
You can help us testing coala in several ways.
12.1 Executing our Tests coala has a big test suite. It is meant to work on every platform on every PC. If you just execute our tests you are doing us a favor. To run tests, you first need to install some dependencies. This can be done by executing: $ pip3 install -r test-requirements.txt
You can then execute our tests with $ py.test and report any errors you get! To run our tests, you can also use python3 setup.py test
Note: If you need to customize test running, you can get more options about allowing skipped tests, getting code coverage displayed or omitting/selecting tests using py.test directly. $ py.test --help
Note: You will not get a test coverage of 100% - the coverage on the website is merged for several python versions.
12.2 Using test coverage
To get coverage information, you can run: $ py.test --cov
You can view the coverage report as html by running: $ py.test --cov --cov-report html
The html report will be saved .htmlreport inside the coala repository.
37 coala Documentation, Release 0.4.1.dev0
38 Chapter 12. Testing CHAPTER 13
Introduction
Tests are an essential element to check if your written components in coala really do work like they should. Even when you think “I really looked over my code, no need for tests” you are wrong! Bugs introduced when not writing tests are often the most horrible ones, they have the characteristic to be undiscoverable (or only discoverable after dozens of hours of searching). Try to test as much as possible! The more tests you write the more you can be sure you did everything correct. Especially if someone else modifies your component, he can be sure with your tests that he doesn’t introduce a bug.
13.1 Actually Writing a Test
So how do you implement a test in coala? First up, tests are placed into the bears/tests (if you want to write a test for a bear) or coalib/tests (if you test a component written for the coalib) directory. They are also written in python (version 3) and get automatically executed by running: $ py.test
There’s only one constraint: The name of the test file has to end with Test.py (for example MyCustomTest.py, but not MyCustomTestSuite.py).
Note: Often you don’t want to run all available tests. To run your specific one, type (in the coala root folder): $ py.test -k
You can even give partial names or queries like “not MyCustomTest” to not run a specific test. More information can be got with py.test -h
Coming to the test file structure. Every test script starts with your system imports, i.e.: # Imports the python 're' package for regex processing. import re import unittest # ...
As said before, in the next line your own imports follow. So just import what you need from coala: import coalib.your_component.component1 import coalib.your_component.component2 # ...
39 coala Documentation, Release 0.4.1.dev0
Note: You can use system imports here also, but the coala codestyle suggests to place them above the three setup lines, like before.
Then the actual test suite class follows, that contains the tests. Each test suite is made up of test cases, where the test suite checks the overall functionality of your component by invoking each test case. The basic declaration for a test suite class is as follows: class YourComponentTest(unittest.TestCase): # Your test cases. pass
You should derive your test suite from unittest.TestCase to have access to the setUp() and tearDown() functions (covered in section below: ‘‘setUp()‘‘ and ‘‘tearDown()‘‘) and also to the assertion functions. Now to the test cases: To implement a test case, just declare a class member function without parameters, starting with test_. Easy, isn’t it? class YourComponentTest(unittest.TestCase): # Tests somethin'. def test_case1(self): pass
# Doesn't test, this is just a member function, since the function name # does not start with 'test_'. def not_testing(self): pass
But how do you actually test if your component is correct? For that purpose you have asserts. Asserts check whether a condition is fulfilled and pass the result to the overall test-suite-invoking-instance, that manages all tests in coala. The result is processed and you get a message if something went wrong in your test. Available assert functions are listed in the section Assertions below. So an example test that succeeds would be: # The sys import and setup is not needed here because this example doesn't use # coala components. import unittest
class YourComponentTest(unittest.TestCase): # Tests somethin'. def test_case1(self): # Does '1' equal '1'? Interestingly it does... mysterious... self.assertEqual(1,1) # Hm yeah, True is True. self.assertTrue(True)
Note: Tests in coala are evaluated against their coverage, means how many statements will be executed from your component when invoking your test cases. A branch coverage of 100% is needed for any commit in order to be pushed to master - please ask us on gitter if you need help raising your coverage! The branch coverage can be measured locally with the py.test --cov command. See also: Module Executing Tests Documentation of running Tests with coverage
40 Chapter 13. Introduction coala Documentation, Release 0.4.1.dev0
As our coverage is measured across builds against several python versions (we need version specific branches here and there) you will not get the full coverage locally! Simply make a pull request to get the coverage measured automatically. If some code is untestable, you need to mark your component code with # pragma: no cover. Important: Provide a reason why your code is untestable. Code coverage is measured using python 3.3 and 3.4 on linux. # Reason why this function is untestable. def untestable_func(): # pragma: no cover # Untestable code. pass
13.2 setUp() and tearDown()
Often you reuse components or need to make an inital setup for your tests. For that purpose the function setUp() exists. Just declare it inside your test suite and it is invoked automatically once at test suite startup: class YourComponentTest(unittest.TestCase): def setUp(self): # Your initialization of constants, operating system API calls etc. pass
The opposite from this is the tearDown() function. It gets invoked when the test suite finished running all test cases. Declare it like setUp() before: class YourComponentTest(unittest.TestCase): def tearDown(self): # Deinitialization, release calls etc. pass
13.3 Assertions
Here follows a list of all available assertion functions supported when inheriting from unittest.TestCase: • assertEqual(a, b) Checks whether expression a equals expression b. • assertNotEqual(a, b) Checks whether expression a not equals expression b. • assertTrue(a) Checks whether expression a is True. • assertFalse(a) Checks whether expression a is False. • assertIs(a, b) Checks whether expression a is b. • assertIsNot(a, b) Checks whether expression a is not b.
13.2. setUp() and tearDown() 41 coala Documentation, Release 0.4.1.dev0
• assertIsNone(a) Checks whether expression a is None. • assertIsNotNone(a) Checks whether expression a is not None. • assertIn(a, list) Checks whether expression a is an element inside list. • assertNotIn(a, list) Checks whether expression a is not an element inside list. • assertIsInstance(a, type) Checks whether expression a is of type type. • assertNotIsInstance(a, type) Checks whether expression a is not of type type. • assertRaises(error, function, *args, **kwargs) Checks whether function throws the specific error. When calling this assert it invokes the function with the specified *args and **kwargs. If you want more information about the python unittest-module, refer to the official documentation and for asserts the subsection [assert-methods] (https://docs.python.org/3/library/unittest.html#assert-methods).
13.4 Kickstart
This section contains a concluding and simple example that you can use as a kickstart for test-writing. Put the code under the desired folder inside coalib/tests or bears/tests, modify it to let it test your stuff and run the test from the coala root folder py.test. # Import here your needed system components. import sys import unittest
sys.path.insert(0,".") # Import here your needed coala components.
# Your test unit. The name of this class is displayed in the test evaluation. class YourTest(unittest.TestCase): def setUp(self): # Here you can set up your stuff. For example constant values, # initializations etc. pass
def tearDown(self): # Here you clean up your stuff initialized in setUp(). For example # deleting arrays, call operating system API etc. pass
def test_case1(self): # A test method. Put your test code here. pass
42 Chapter 13. Introduction CHAPTER 14
Writing Documentation
This document gives a short introduction on how to write documentation for the coala project. Documentation is written in reStructuredText and rendered by readthedocs.org to our lovely users. You can view the current documentation on http://coala.rtfd.org. After getting the coala source code (see Installation Instructions), you can start hacking on existent documentation files. They reside in the docs directory and are written in reStructuredText. If you want to add new pages, you need to alter the index.rst file in the coala docs directory. Please read http://www.sphinx-doc.org/en/stable/markup/toctree.html#toctree-directive for an explanation of the syntax. You can test the documentation locally through simply running make html in the coala docs directory. This gen- erates docs\_build\html\index.html that you can view on your browser. (Use pip install sphinx if you do not have sphinx yet.)
43 coala Documentation, Release 0.4.1.dev0
44 Chapter 14. Writing Documentation CHAPTER 15
Introduction
At coala we are looking heavily at the maintainability of the code. Code is more often read than written! It is obvious that we need good code. In order to do that we are verifying that every change to our code (i.e. the commits) is making it better.
15.1 What Makes a Good Commit
A good commit is atomic. It describes but only one logical change and not more. Why do we do that? Because we find more bugs if we do! Also it features a good commit message.
15.2 Why Do We Need Good Commits?
• An atomic commit is way easier to review. The reviewer thus will be able to review faster and find more bugs due to the lower complexity of the change. • Atomic commits are like good objects in object oriented programming - you can split up a bigger thing into many small objects. Reducing complexity is the key to developing good software and finding its bug before they occur. • Good commit messages make it easy to check at a glance what happened in a time range. • It is way easier to revert single changes without side effects. Reverting multiple commits at a time is easy, reverting a part of a commit is not. • git blame will be much more effective. It is the best documentation you can get. The older your code is, the more documentation it has. The better the commit messages are, the better is your hidden documentation. Your commit messages document the reason for every single change you did to any line. • git bisect will be much more effective. If you bisect through atomic commits to find the commit which caused a bug, you should be able to identify the real cause of the bug fastly. Good commit messages and atomicity of commits are key to that ability.
15.2.1 How to Write Good Commit Messages
There are a few things to consider when writing a commit message, namely: • The first line may hold up to 50 chars excluding newline and is called shortlog.
45 coala Documentation, Release 0.4.1.dev0
• The shortlog should have a tag and must have a short description: tag: Short description. • The tag is usually the affected class or package and not mandatory. It gives context to the commit. • The short description starts with a big letter and is written in imperative present tense (i.e. Add something, not Adding something or Added something). It has no trailing period. • The second line must be empty. • All following lines may hold up to 72 chars excluding newline. • These lines are the long description. The long description is not mandatory but may help expressing what you’re doing. • The shortlog shall describe the change as exactly as possible. If it is a bugfix, don’t describe the bug but the change. • In the long description you can add reasoning for your changes. This is especially important for complex changes that are not self explanatory. This is also the right place to explain related bugs. • If the commit fixes a bug, add the following line at the end: Fixes https://github.com/coala-analyzer/coala/issues/###, this way the commit will ap- pear at the bug and several revisions can be tracked this way. • Be sure to use the full URL, if we move from github, the links should still work. • This will automatically close the according bug when pushed to master if you have the permissions on GitHub. Also see: https://wiki.gnome.org/Git/CommitMessages
15.2.2 Example setup: Install .coafile via package_data
When installing the .coafile to distutils.sysconfig.get_python_lib, we ignore that this is not the installation directory in every case. Thus it is easier, more reliable and platform independent to let distutils install it by itself.
Fixes https://github.com/coala-analyzer/coala/issues/269
46 Chapter 15. Introduction CHAPTER 16
Coverage Installation Hints for OSX Users:
16.1 1. Make sure you have installed Xcode and Homebrew.
16.2 2. Install Python3. brew search python # This should display python3 brew install python3 python3 --version # To check the installed version
16.3 3. Create Virtual environments with pyvenv
# Create Virtual Env named myenv pyvenv myenv
# This will create a folder named myenv in the # current directory. To activate this environment just type source myenv/bin/activate
# You can start Python3 by typing: python
16.4 4. Virtualenvwrapper with Python 3:
# Installation pip install virtualenv pip install virtualenvwrapper
# Folder to contain Virtual Environments mkdir ~/.virtualenvs
# Add the following in ~/.bash_profile export WORKON_HOME=~/.virtualenvs source /usr/local/bin/virtualenvwrapper.sh
# Activate Changes source ~/.bash_profile
47 coala Documentation, Release 0.4.1.dev0
# Get Python3 path (python3_pth) where python3
# Create a new virtual environment with Python3 mkvirtualenv --python=python3_path myenv
16.5 Finally!
# Install python-coverage3 by easy_install coverage
48 Chapter 16. Coverage Installation Hints for OSX Users: CHAPTER 17
A Hitchhiker’s Guide to Git(1): Genesis
17.1 Introduction
17.1.1 What is This?
This post tries to provide a gentle introduction to git. While it is aimed at newcomers it is also meant to give a rather good overview and understanding about what one can do with git and thus is extensive. It follows the premise: If you know how a thing works, you can make it do what you want it to. Please don’t be afraid as I’m trying to use light language and many examples so you can read fast through it and understand anyway (if I succeed). I’m also trying to highlight important things at least for long texts to ease reading a bit more. However, at some points this tutorial may require you to think for yourselves a bit - it’s a feature, not a bug.
17.1.2 For Whom is This?
This tutorial is meant for everyone willing to learn git - it does not matter if you are a developer who never really got the hang of it or a student wanting to learn. Even if you are doing non coding tasks like design or documentation git can be a gift from heaven. This tutorial is for everyone - don’t be scared by its length and believe me, it pays off! The only requirement is that you have git installed, know cd, ls and mkdir and have something to commit - and who doesn’t have any digital data!?
17.1.3 What’s in There?
This tutorial is the first out of three tutorials which are meant to free your mind from a traditional view on filesys- tems.
Genesis
This tutorial, Genesis (i.e. “Creation” or “Beginning”), will cover some very basic things: • Configuring git so you can use it how you want. (Basic, aliases) • Creating your git repository (play god once!) • Creating your first git commits. • Learning what the repository is and where commits get stored.
49 coala Documentation, Release 0.4.1.dev0
• Browsing through history. • Ignoring files.
Exodus
Exodus (i.e. “going out”) will cover: • Store temporary changes without committing. • Navigating commits in git. • Sharing commits to a remote place. • Locally (huh, that’s sharing?) • Via email • To a server • To a client • Working with others (i.e. non-linear). • Join things. • Linearize things. • Writing good commits. • Editing commits. (Actually not editing but it feels like that.)
Apocalypse
Apocalypse (i.e. “uncovering”) will try to uncover some more advanced features of git, finally freeing your mind from your non-versioned filesystem: • Finding more information about code. • Finding causes of bugs in git. • Reverting commits. • Reviewing commits. • Travelling though time and changing history (you want me to believe you’ve never wanted to do that?) • Getting back lost things. • Let git do things automatically.
17.1.4 Some Warnings
A short warning: If you ever really got the hang of git you will not be able to use something else without symptoms of frustration and disappointment - you’ll end up writing every document versioned as an excuse to use git. A warning for windows users: you may need to use equivalent commands to some basic UNIX utilities or just install them with git. (Installer provides an option for that.) In general it’s a bit like travelling with Vogons - avoid when possible. A warning for GUI users: Don’t use your GUI. Be it the GitHub App or SourceTree or something else - they usually try to make things more abstract for us, thus they hinder us from understanding git and we can then not make git do what we want. Being able to communicate directly with git is a great thing and really bumps productivity!
50 Chapter 17. A Hitchhiker’s Guide to Git(1): Genesis coala Documentation, Release 0.4.1.dev0
I wrote this tutorial to the best of my knowledge and experience, if you spot an error or find something important is missing, be sure to drop me a message!
17.1.5 Preparation...
So go now, grab a cup of coffee (or any other drink), a towel, take your best keyboard and open a terminal beneath this window!
17.2 What’s Git for Anyway?
Before we really get started it is important to know what git roughly does: git is a program that allows you to manage files. To be more specific git allows you to define changes on files. In the end your repository is just a bunch of changes that may be related to each other.
17.3 Setting Up Git
Before we can continue we’ll have to set up a few tiny things for git. For this we will use the git config --global command which simply stores a key value pair into your user-global git configuration file (usually stored at ~/.gitconfig).
17.3.1 WHOAMI
Let’s tell git who we are! This is pretty straightforward: $ git config --global user.name "Ford Prefect" $ git config --global user.email [email protected]
This makes git store values for “name” and “email” within the “user” section of the gitconfig.
17.3.2 Editor
For some operations git will give you an editor so you can enter needed data. This editor is vim by default. Some people think vim is great (vim is great!), some do not. If you belong to the latter group or don’t know what vim is and how to operate it, let’s change the editor: $ # Take an editor of your choice instead of nano $ git config --global core.editor nano
Please make sure that the command you give to git always starts as an own process and ends only when you finished editing the file. (Some editors might detect running processes, pass the filename to them and exit immediately. Use -s argument for gedit, --wait argument for sublime.) Please don’t use notepad on windows, this program is a perfect example of a text editor which is too dumb to show text unless the text is written by itself.
17.4 Create a Repository
So, lets get started - with nothing. Let’s make an empty directory. You can do that from your usual terminal:
17.2. What’s Git for Anyway? 51 coala Documentation, Release 0.4.1.dev0
$ mkdir git-tutorial $ cd git-tutorial $ ls -a ./ ../
So, lets do the first git command here: $ git init Initialized empty Git repository in /home/lasse/prog/git-tutorial/.git/ $ ls -a ./ ../ .git/
So now we’ve got the .git folder. Since we just created a repository with git init, so we can deduce, that this .git directory must in fact be the repository!
17.5 Creating a God Commit
So, let’s create some content we can manage with git: $ echo 'Hello World!' >> README $ cat README Hello World!
Since we know, that the .git directory is our repository, we also know that we did not add this file to our repository yet. So how do we do that? As I’ve hinted before, our git repository does not contain files but only changes - so how do we make a change out of our file? The answer lies in (1) git add and (2) git commit which allow us to (1) specify what files/file changes we want to add to the change and (2) that we want to pack those file changes into a so called commit. Git also offers a helper command so we can see what will be added to our commit: git status. Let’s try it out: $ git status On branch master
Initial commit
Untracked files: (use "git add
README
nothing added to commit but untracked files present (use "git add" to track) $ git add README $ git status On branch master
Initial commit
Changes to be committed: (use "git rm --cached
new file: README
So obviously with git add we can stage files. What does that mean?
52 Chapter 17. A Hitchhiker’s Guide to Git(1): Genesis coala Documentation, Release 0.4.1.dev0
As we know, when we’re working in our directory any actions on files won’t affect our repository. So in order to add a file to the repository, we’ll have to put it into a commit. In order to do that, we need to specify, what files/changes should go into our commit, i.e. stage them. When we did git add README, we staged the file README, thus every change we did until now to it will be included in our next commit. (You can also partially stage files so if you edit README now the change won’t be committed.) Now we’ll do something very special in git - creating the first commit! (We’ll pass the -v argument to get a bit more info from git on what we’re doing.) $ git commit -v
You should now get your editor with contents similar to this: # Please enter the commit message for your changes. Lines starting # with '#' will be ignored, and an empty message aborts the commit. # On branch master # # Initial commit # # Changes to be committed: # new file: README #ref: refs/heads/master
# ------>8 ------# Do not touch the line above. # Everything below will be removed. diff --git a/README b/README new file mode 100644 index 0000000..c57eff5 --- /dev/null +++ b/README @@ -0,0 +1 @@ +Hello World!
Since we’re about to create a change, git asks us for a description.
Note: Git actually allows to create commits without a description with a special argument. This is not recommended for productive collaborative work!)
Since we passed the -v parameter, git also shows us below what will be included in our change. We’ll look at this later. Commit messages are usually written in imperative present tense and should follow certain guidelines. We’ll come to this later. So, let’s enter: Add README as our commit message, save and exit the editor. Now, let’s take a look at what we’ve created, git show is the command that shows us the most recent commit: $ git show commit ec6c903a0a18960cd73df18897e56738c4c6bb51 Author: Lasse Schuirmann
Add README
diff --git a/README b/README new file mode 100644 index 0000000..980a0d5
17.5. Creating a God Commit 53 coala Documentation, Release 0.4.1.dev0
--- /dev/null +++ b/README @@ -0,0 +1 @@ +Hello World!
So what do we see here: • It seems that commits have an ID, in this case ec6c903a0a18960cd73df18897e56738c4c6bb51. • Commits also have an author and a creation date. • Of course they hold the message we wrote and changes to some files. What we see below the diff ... line is obviously the change. Let’s take a look at it: since git can only describe changes, it takes /dev/null (which is a bit special, kind of an empty file, not important here), renames it to README and fills it with our contents. So, this commit is pretty godish: It exists purely on it’s own, has no relations to any other commit (yet, it’s based on an empty repository, right?) and creates a file out of nothing (/dev/null is somehow all and nothing, kind of a unix black hole).
17.6 Inspecting What Happened
So, let’s look in our repository! $ ls -la .git total 52 drwxrwxr-x. 8 lasse lasse 4096 Feb 27 16:05 ./ drwxrwxr-x. 3 lasse lasse 4096 Feb 27 14:11 ../ drwxrwxr-x. 2 lasse lasse 4096 Feb 27 14:11 branches/ -rw-rw-r--. 1 lasse lasse 486 Feb 27 14:12 COMMIT_EDITMSG -rwxrw-r--. 1 lasse lasse 92 Feb 27 14:11 config* -rw-rw-r--. 1 lasse lasse 73 Feb 27 14:11 description -rw-rw-r--. 1 lasse lasse 23 Feb 27 14:11 HEAD drwxrwxr-x. 2 lasse lasse 4096 Feb 27 14:11 hooks/ -rw-rw-r--. 1 lasse lasse 104 Feb 27 14:11 index drwxrwxr-x. 2 lasse lasse 4096 Feb 27 14:11 info/ drwxrwxr-x. 3 lasse lasse 4096 Feb 27 14:12 logs/ drwxrwxr-x. 7 lasse lasse 4096 Feb 27 14:12 objects/ drwxrwxr-x. 4 lasse lasse 4096 Feb 27 14:11 refs/ $
Now let’s look into it further to get to know what it is a bit more. I will try to cover only important parts here, if you’re interested even deeper, you can try DuckDuckGo or take a look at this: http://git-scm.com/docs/gitrepository-layout
17.6.1 The config file
The config file is a similar file to the one where our settings in the beginning got stored. (User and editor configuration, remember?) You can use it to store settings per repository.
17.6.2 The objects directory
The objects directory is an important one: It contains our commits. One could do a full tutorial on those things but that’s not covered here. If you want that, check out: http://git- scm.com/book/en/v2/Git-Internals-Git-Objects
54 Chapter 17. A Hitchhiker’s Guide to Git(1): Genesis coala Documentation, Release 0.4.1.dev0
We just saw the ID of the commit we made: ec6c903a0a18960cd73df18897e56738c4c6bb51 Now let’s see if we find it in the objects directory: $ ls .git/objects 98/ b4/ ec/ info/ pack/ $ ls .git/objects/ec 6c903a0a18960cd73df18897e56738c4c6bb51
So, when we create a commit, the contents (including metadata) are hashed and git stores it finely into the objects directory. That isn’t so complicated at all, is it?
Task: Objects git show accepts a commit ID as an argument. So you could e.g. do git show ec6c903a0a18960cd73df18897e56738c4c6bb51 instead of git show if this hash is the current commit. Investigate what the other two objects are, which are stored in the objects directory. (Ignore the info and pack subdi- rectory.) Do git show again and take a look at the line beginning with “index”. I’m sure you can make sense out of it!
17.6.3 The HEAD File
The HEAD file is here so git knows what the current commit is, i.e. with which objects it has to compare the files in the file system to e.g. generate a diff. Let’s look into it: $ cat .git/HEAD ref: refs/heads/master
So it actually only references to something else. So let’s take a look into refs/heads/master - what ever this is: $ cat .git/refs/heads/master ec6c903a0a18960cd73df18897e56738c4c6bb51
So this HEAD file refers to this master file which refers to our current commit. We’ll see how that makes sense later.
17.7 Creating a Child Commit
Now, let’s go on and create another commit. Let’s add something to our README. You can do that by yourself, I’m sure! Let’s see what we’ve done: $ git diff diff --git a/README b/README index 980a0d5..c9b319e 100644 --- a/README +++ b/README @@ -1 +1,2 @@ Hello World! +Don't panic!
17.7. Creating a Child Commit 55 coala Documentation, Release 0.4.1.dev0
Let’s commit it. However, since we’re a bit lazy we don’t want to add the README manually again; the commit command has an argument that allows you to auto-stage all changes to all files that are in our repository. (So if you added another file which is not in the repository yet it won’t be staged!) git commit -a -v
Well, you know the game. Can you come up with a good message on your own? $ git show commit 7b4977cdfb3f304feffa6fc22de1007dd2bebf26 Author: Lasse Schuirmann
README: Add usage instructions
diff --git a/README b/README index 980a0d5..c9b319e 100644 --- a/README +++ b/README @@ -1 +1,2 @@ Hello World! +Don't panic!
So this commit obviously represents the change from a file named README which contents are stored in object 980a0d5 to a file also named README which contents are stored in object c9b319e.
17.8 A Glance At Our History
Let’s see a timeline of what we’ve done: $ git log commit 7b4977cdfb3f304feffa6fc22de1007dd2bebf26 Author: Lasse Schuirmann
README: Add usage instructions
commit ec6c903a0a18960cd73df18897e56738c4c6bb51 Author: Lasse Schuirmann
Add README
That looks fairly easy. However I cannot withstand to point out that despite commits look so fine, linearly arranged here, they are actually nothing more than commit objects, floating around in the .git/objects/ directory. So git log just looks where HEAD points to and recursively asks each commit what it’s parent is (if it has one). Since every good hitchhiker does know how to travel through time and change events, we’ll learn to do that in the next chapter ;)
56 Chapter 17. A Hitchhiker’s Guide to Git(1): Genesis coala Documentation, Release 0.4.1.dev0
17.9 Configuring Git Even Better
17.9.1 Better staging
It is worth to mention that git add also accepts directories as an argument. I.e. git add . recursively adds all files from the current directory. In order to generally ignore certain patterns of files (e.g. it’s bad practice to commit any generated stuff), one can write a .gitignore file. This file can look as follows: README~ # Ignore gedit temporary files *.o # Ignore compiled object files
The exact pattern is defined here: http://git-scm.com/docs/gitignore Files matching this pattern will: • Not be added with git add unless forced with -f • Not be shown in git status as unstaged It is usually a good idea to commit the .gitignore to the repository so all developers don’t need to care about those files.
17.9.2 Aliases
So, we’ve learned quite some stuff. However git command’s aren’t as intuitive as they could be sometimes. They could be shorter too. So let’s define us some aliases of the commands we know. The ones given here are only suggestions, you should choose the aliases in a way that suits best for you!
Aliasing Git Itself
If you’re using git much, you might want to add alias g=git to your .bashrc or .zshrc or whatever. (On windows you’re a bit screwed. But what did you expect? Really?)
Aliasing Git Commands
Let’s let git give us our editor since we don’t want to edit just one value: git config --global --edit
You can add aliases through the [alias] section, here are the aliases I suggest: [alias] a= add c= commit-v ca= commit-v-a d= diff dif= diff # Thats a typo :) i= init l= log st= status stat= status
17.9. Configuring Git Even Better 57 coala Documentation, Release 0.4.1.dev0
17.10 Conclusion
So what did we learn? We did some basic git commands: • git config: accessing git configuration • git init: creating a repository • git status: getting current status of files, staging and so on • git add: staging files for the commit • git diff: showing the difference between the current commit and what we have on our file system • git commit: writing staged changes to a commit • git log: browsing history We also learned how git organizes commits, how it stores files and how we can make git ignore files explicitly. I hope this helped you understanding a bit what git does and what it is. The next tutorial will hopefully cover all the basics. (Some were already hinted here.) .o88Oo._ d8P .ooOO8bo._ 88 '*Y8bo. __ YA '*Y8b __ ,dPYb, YA 68o68**8Oo. IP'`Yb "8D *"' "Y8o I8 8I Y8 'YB .8D I8 8P '8 d8' 8D I8 8' 8 d8888b d AY ,gggo, ,ggggo, ,gggo,gg I8 dP ,gggo,gg Y, d888888 d' _.oP" dP" "Yb dP" "Y8go*8P" "Y8I I8dP dP" "Y8I q. Y8888P' d8 i8' i8' ,8P i8' ,8I I8P i8' ,8I "q. `Y88P' d8" ,d8,_ _,d8, ,d8' d8, ,d8b,,d8b,_ ,d8, ,d8b, Y ,o8P ooP""Y8888PP*"Y8888P" "Y8888P"`Y88P'"Y88P"Y8888P"`Y8 oooo888P"
58 Chapter 17. A Hitchhiker’s Guide to Git(1): Genesis CHAPTER 18
What is coala?
coala is a simple COde AnaLysis Application. Its goal is to make static code analysis easy while remaining completely modular and therefore extendable and language independent. coala is written with a lower case “c”. Code analysis happens in python scripts while coala manages these, tries to provide helpful libraries and provides multiple user interfaces. (Currently we support only Console output but others will follow.) One could say coala unites all language independent parts of a linter and is arbitrarily extensible. To get started, take a look at our Installation Instructions and the Tutorial.
59 coala Documentation, Release 0.4.1.dev0
60 Chapter 18. What is coala? CHAPTER 19
What do I get?
19.1 As a User coala allows you to simply check your code against certain quality requirements. The checking routines are named Bears in coala. You can easily define a simple project file to check your project with all bears either shipped with coala or ones you found in the internet and trust.
19.2 As a Developer
If you are not satisfied with the functionality given by the bears we provide, you can easily write own bears. coala is written with easiness of extension in mind. That means: no big boilerplate, just write one small object with one routine, add the parameters you like and see how coala automates the organization of settings, user interaction and execution parallelization. You shouldn’t need to care about anything else than just writing your algorithm! See Writing Bears for more information on this. Also, coala provides an external API using the dbus message protocol. This allows other applications to easily use the code analysis functionalities coala provides in their applications. See External APIs for more information.
61 coala Documentation, Release 0.4.1.dev0
62 Chapter 19. What do I get? CHAPTER 20
Status of the Project
We are currently working hard to make this project reality. coala is currently usable, in an alpha stage and already provides many features. If you want to see how the development progresses, check out https://github.com/coala-analyzer/coala
63 coala Documentation, Release 0.4.1.dev0
64 Chapter 20. Status of the Project CHAPTER 21
Indices and tables
• genindex • modindex • search
65