Mataara Documentation Release 0.1
Total Page:16
File Type:pdf, Size:1020Kb
Mataara Documentation Release 0.1 Catalyst IT Sep 08, 2020 Architecture 1 Name 3 1.1 Features..................................................3 1.2 Client-Server Model Design.......................................4 1.3 Front-End Architecture..........................................5 1.4 Getting Started..............................................8 1.5 Installing the Client...........................................8 1.6 Configuration............................................... 10 1.7 Drush Commands............................................ 12 1.8 Getting Started.............................................. 13 1.9 Installing the Server........................................... 14 1.10 Generating an OpenSSL key-pair.................................... 18 1.11 Configuration............................................... 19 1.12 Configuring SAML authentication.................................... 22 1.13 Getting Started.............................................. 25 1.14 Quick Start guide............................................. 26 1.15 Environment setup............................................ 26 1.16 Fabric tasks................................................ 29 1.17 Useful Commands............................................ 31 1.18 Database Backup and Restore...................................... 33 1.19 Setting up your development environment for ReactJS......................... 34 1.20 Report Processing............................................ 34 1.21 Mataara report format (JSON)...................................... 34 1.22 Archimedes report format (legacy XML)................................ 39 1.23 Key Development Technologies..................................... 42 1.24 Source code repository structure..................................... 42 i ii Mataara Documentation, Release 0.1 Mataara is a security reporting tool that collects information on websites and their dependencies, uploading reports to a server that monitors module vulnerabilities and site configuration changes from a central location. The primary intent of the Mataara project is to provide a system that helps site hosting and development organisations to manage and secure their software configurations across large numbers of sites. The documentation for Mataara is organised into a number of sections: • System Architecture • Drupal Client • Server • Development • Development Quickstart Guide The Mataara project was created at Catalyst IT. Architecture 1 Mataara Documentation, Release 0.1 2 Architecture CHAPTER 1 Name An earlier version of this project was known as Archimedes, and a significant proportion of the code still refers to itself as such. Because naming things is hard, and there were already several other projects called Archimedes out there, it was decided to change the name to something more distinctive. Mataara (pronounced by syllable: ma-ta-a-ra) is a word from the Maori¯ language that means vigilance, alertness, watchfulness. (It is unrelated to the Kenyan tea factory of the same name.) 1.1 Features The following is a description of the Mataara system and some of its features. For a better and more complete understanding of the software, it might be best to download the source code of the client and server modules, and experiment with the system in a test environment. Mataara currently consists of a server built in Django and ReactJS, communicating via a REST API, and a Drupal CMS client module. It is intended to be an extensible system, and development of a client for the SilverStripe CMS is underway. 1.1.1 Drupal Client Reporting Mataara’s Drupal client is an installable module that periodically sends reports to the Mataara server (daily by default, but configurable within Drupal’s cron system). The primary payload of the report is a list of the modules and themes installed on the host website, that can be checked against drupal.org’s APIs to determine whether they have known security vulnerabilities. Reports also contain details pertinent to the security posture of the site, including: • A generated site key • Hostname • Core platform version • Databases connected to the site 3 Mataara Documentation, Release 0.1 • The number of users, administrator roles and content nodes Report data is encoded in JSON and public-key encrypted during transport. Reports can be sent to the server via either HTTP/S POST requests or by email attachment. The report format is designed to be customisable, and new items can be added to the report by extending a base ReportItem PHP class in the client module. For use in the system console, Drush commands are available to send reports or to generate a report in various for- mats. Settings can also be configured through the standard Drupal administration interface, including the transport mechanism (HTTP or email) and the server’s public key for encryption. The Drupal client module has legacy support for Drupal 6, is compatible with Drupal 7 and 8, and will support Drupal 9. In order to ensure it can be used in as many hosting environments as possible, has no dependencies on any other Drupal modules. 1.1.2 Django Server Structure The Django server receives reports via either an HTTP/S endpoint or via email, where they are decrypted and placed in a task processing queue. Queue workers ingest the reports and match them to Site nodes within the system, updating them with any new information contained in the reports. Reports are also archived in their raw JSON format for reference, and potentially for further processing in other tools. Reports are matched against sites using their generated site key, and can also be grouped into a “site group”. This is intended for multiple versions of the same site across different environments (e.g. development, staging, production), to make them easier to keep track of and to store common information. Note: At the time of writing this site group subsystem is quite new, and does not have front-end functionality yet, but it is envisioned that this will allow for integrations with other systems for issue management, bug tracking, customer relationship management, etc. The ReactJS frontend is primarily for viewing and querying site data, including modules and security advisories. At present, server management commands are primarily handled through the Django admin web interface, or by console management commands. 1.1.3 Drupal Security Advisories and Project Release History Earlier versions of Mataara used Drupal.org advisories as the primary source of truth for matching against insecure module versions. The data within the advisories is sometimes hard to automatically parse. As a result, the system has moved to using the release history API as its primary means of making Drupal modules as insecure. The Mataara server periodically checks the Drupal.org Release History API for every project it has encountered, checking their release types for insecure versions. Drupal security advisories are still checked via the Drupal.org D7 APIs, and saved for display to Mataara users. The legacy advisory parser will also automatically correct errors in some older advisories. Advisory errors and warnings will be logged to a Sentry server if you have one available. In the frontend, sites can be checked for vulnerable module releases (including Drupal core releases), and modules with vulnerable releases can list the sites potentially affected by those vulnerabilities. 1.2 Client-Server Model Design Mataara uses a client-server model that practises inversion of control; the server passively receives encrypted reports from client modules installed in the other systems. 4 Chapter 1. Name Mataara Documentation, Release 0.1 This allows Mataara to be a write-only repository of information about sites over time, decoupled from any “dial in” command-and-control systems, so that it can capture an understanding of the security posture of websites without needing to have ongoing privileged access to each host. This reduces the inbound attack surface on individual sites, and provides a means for managing the release update cycle on sites hosted in managed secure environments such as intranets, where it may not be desirable for them to call out directly to public update servers for information. The Mataara server will independently check for version updates and security advisories, allowing them to be matched against incoming reports. A planned future enhancement is to provide a subscription-notification system for users, so that users who are subscribed to a site or group of sites can be notified when a new insecure version is discovered. 1.3 Front-End Architecture 1.3.1 Introduction Mataara uses the ReactJS framework to provide a front-end user interface, built on top of Django REST Framework APIs. The React frontend also uses Redux to store the state of the app in a single state tree. What does Redux do, and how does it work? Redux is a predictable state container for JavaScript apps. It helps you write applications that behave consistently, run in different environments (client, server, and native), and are easy to test. Redux only has a single store that holds no logic by itself. Actions are dispatched and handled directly by the store, eliminating the need for a standalone dispatcher. In turn, the store passes the actions to state-changing functions called reducers. API or client-side middleware may also be included to add authentication, or to stop, modify or add more actions. Redux Components Redux features four major system components: Store: Controls the state. Actions: Describe the changes