Kuma Documentation Release latest Mozilla October 30, 2020 Contents 1 Development 3 2 Getting started 5 2.1 Installation................................................5 2.2 Development............................................... 10 2.3 Troubleshooting............................................. 24 2.4 The Kuma test suite........................................... 26 2.5 Client-side testing............................................ 28 2.6 Feature toggles.............................................. 30 2.7 Celery and async tasks.......................................... 33 2.8 Search.................................................. 35 2.9 Localization............................................... 36 2.10 CKEditor................................................. 39 2.11 Getting Data............................................... 41 2.12 Documentation.............................................. 45 2.13 Docker.................................................. 45 2.14 Deploying MDN............................................. 46 2.15 Rendering Documents.......................................... 50 2.16 Building, collecting, and serving assets................................. 63 i ii Kuma Documentation, Release latest Kuma is the platform that powers MDN (developer.mozilla.org) Contents 1 Kuma Documentation, Release latest 2 Contents CHAPTER 1 Development Code https://github.com/mdn/kuma Issues P1 Bugs (to be fixed in current or next sprint) P2 Bugs (to be fixed in 180 days) All developer.mozilla.org bugs Pull Request Queues Dev Docs https://kuma.readthedocs.io/en/latest/installation.html CI Server https://travis-ci.com/mdn/kuma Forum https://discourse.mozilla.org/c/mdn Matrix #mdn room Servers What’s Deployed on MDN? https://developer.allizom.org/ (stage) https://developer.mozilla.org/ (prod) 3 Kuma Documentation, Release latest 4 Chapter 1. Development CHAPTER 2 Getting started Want to help make MDN great? Our contribution guide lists some good first projects and offers direction on submitting code. Contents: 2.1 Installation Kuma uses Docker for local development and integration testing, and we are transitioning to Docker containers for deployment as well. Current Status of Dockerization: • Kuma developers are using Docker for daily development and maintenance tasks. Staff developers primarily use Docker for Mac. Other staff members and contributors use Docker’s Ubuntu packages. • The development environment can use a lot of resources. On Docker for Mac, the environment runs well with 6 CPUs and 10 GB of memory dedicated to Docker. It can be run successfully on 2 CPUs and 2 GB of memory. • The Docker development environment is evolving rapidly, to address issues found during development and to move toward a containerized design. You may need to regularly reset your environment to get the current changes. • The Docker development environment doesn’t fully support a ‘production-like’ environment. For example, we don’t have a documented configuration for running with an SSL connection. • When the master branch is updated, the kuma_base image is refreshed and published to DockerHub. This image contains system packages and third-party libraries. • Our TravisCI builds include a target that build Docker containers and runs the tests inside. • Our Jenkins server builds and publishes Docker images, and runs integration tests using Docker. • We are documenting tips and tricks on the Troubleshooting page. • Feel free to ask for help on Matrix at #mdn or on discourse. 2.1.1 Docker setup 1. Install the Docker platform, following Docker’s instructions for your operating system, such as Docker for Mac for MacOS, or for your Linux distribution. Non-Linux users should increase Docker’s memory limits (Windows, macOS) to at least 4 GB, as the default of 2 GB is insufficient. 5 Kuma Documentation, Release latest Linux users will also want to install Docker Compose and follow post-install instructions to confirm that the development user can run Docker commmands. To confirm that Docker is installed correctly, run: docker run hello-world If you find any error using docker commands without sudo visit using docker as non-root user. 2. Clone the kuma Git repository, if you haven’t already: git clone --recursive https://github.com/mdn/kuma.git If you think you might be submitting pull requests, consider forking the repository first, and then cloning your fork of it. 3. Ensure you are in the existing or newly cloned kuma working copy: cd kuma 4. Initialize and customize .env: cp.env-dist.dev.env vim.env # Or your favorite editor Linux users should set the UID parameter in .env (i.e. change #UID=1000 to UID=1000) to avoid file permission issues when mixing docker-compose and docker commands. MacOS users do not need to change any of the defaults to get started. Note that there are settings in this file that can be useful when debug- ging, however. 5. Pull the Docker images and build the containers: docker-compose pull docker-compose build (The build command is effectively a no-op at this point because the pull command just downloaded pre-built docker images.) 6. Start the containers in the background: docker-compose up -d 2.1.2 Load the sample database Download the sample database with either of the following wget or curl (installed by default on macOS) commands: wget -N https://mdn-downloads.s3-us-west-2.amazonaws.com/mdn_sample_db.sql.gz curl -O https://mdn-downloads.s3-us-west-2.amazonaws.com/mdn_sample_db.sql.gz Next, upload that sample database into the Kuma web container with: docker-compose exec web bash -c "zcat mdn_sample_db.sql.gz | ./manage.py dbshell" (This command can be adjusted to restore from an uncompressed database, or directly from a mysqldump command.) Then run the following command: docker-compose exec web ./manage.py migrate This will ensure the sample database is in sync with your version of Kuma. 6 Chapter 2. Getting started Kuma Documentation, Release latest 2.1.3 Compile locales Localized string databases are included in their source form, and need to be compiled to their binary form: docker-compose exec web make localecompile Dozens of lines of warnings will be printed: cd locale; ./compile-mo.sh . ./af/LC_MESSAGES/django.po:2: warning: header field 'PO-Revision-Date' still has the initial default value ./af/LC_MESSAGES/django.po:2: warning: header field 'Last-Translator' still has the initial default value ... ./zu/LC_MESSAGES/javascript.po:2: warning: header field 'PO-Revision-Date' still has the initial default value ./zu/LC_MESSAGES/javascript.po:2: warning: header field 'Last-Translator' still has the initial default value Warnings are OK, and will be fixed as translators update the strings on Pontoon. If there is an error, the output will end with the error, such as: ./az/LC_MESSAGES/django.po:263: 'msgid' and 'msgstr' entries do not both end with '\n' msgfmt: found 1 fatal error These need to be fixed by a Kuma developer. Notify them in the #mdn Matrix room or open a bug. You can continue with installation, but non-English locales will not be localized. 2.1.4 Generate static assets Static assets such as CSS and JS are included in source form, and need to be compiled to their final form: docker-compose exec web make build-static A few thousand lines will be printed, like: ## Generating JavaScript translation catalogs ## processing language en_US processing language af processing language ar ... ## Compiling (Sass), collecting, and building static files ## Copying '/app/kuma/static/img/embed/promos/survey.svg' Copying '/app/kuma/static/styles/components/home/column-callout.scss' Copying '/app/build/locale/jsi18n/fy-NL/javascript.js' ... Post-processed 'build/styles/editor-locale-ar.css' as 'build/styles/editor-locale-ar.css' Post-processed 'build/styles/locale-ln.css' as 'build/styles/locale-ln.css' Post-processed 'build/styles/editor-locale-pt-BR.css' as 'build/styles/editor-locale-pt-BR.css' .... 1870 static files copied to '/app/static', 125 post-processed. 2.1.5 Visit the homepage Open the homepage at http://localhost.org:8000 . You’ve installed Kuma! 2.1.6 Create an admin user Many Kuma settings require access to the Django admin, including configuring social login. It is useful to create an admin account with password access for local development. 2.1. Installation 7 Kuma Documentation, Release latest If you want to create a new admin account, use createsuperuser: docker-compose exec web ./manage.py createsuperuser This will prompt you for a username, email address (a fake address like [email protected] will work), and a password. If your database has an existing account that you want to use, run the management command. Replace YOUR_USERNAME with your username and YOUR_PASSWORD with your password: docker-compose run --rm web ./manage.py ihavepower YOUR_USERNAME \ --password YOUR_PASSWORD With a password-enabled admin account, you can log into Django admin at http://localhost.org:8000/admin/login 2.1.7 Update the Sites section 1. After logging in to the Django admin (an alternative is using the login test-super with password test-password), scroll down to the Sites section. 2. Click on “Change”. 3. Click on the entry that says localhost:8000. 4. Change both the domain and display name from localhost:8000 to localhost.org:8000. 5. Click “Save”. 2.1.8 Enable GitHub/Google authentication (optional) Since Google’s OAuth requires us to use a valid top-level-domain, we’re going to use http://localhost.org:8000 as an example for every URL here. To automate setting Django up for social auth you can run docker-compose exec web