Mozilla Services Documentation Release Tarek Ziade Sep 11, 2017 Contents 1 How To... 3 2 Services 21 3 Server-side development guide 121 4 Client Development 151 5 Miscellaneous 185 i ii Mozilla Services Documentation, Release Welcome to the Mozilla Services Documentation front page. This site contains technical information about the various services and products provided by the Mozilla Services Team. The documentation on this site is meant to be more declarative and static in nature. Other information surrounding the products and services detailed on this site can be found at: • Mozilla Services Team Wiki • Mozilla Developer Network (MDN) The wiki contains team info and updates. MDN contains content specific for helping developers. To contribute to this site, see About this Website. Contents 1 Mozilla Services Documentation, Release 2 Contents CHAPTER 1 How To... This section contains simple how-tos. If you want to see a new one, let us know in the mailing list. Run your own Sync-1.1 Server The Firefox Sync Server is deployed on our systems using RPM packaging, and we don’t provide any other packaging or publish official RPMs yet. The easiest way to install a Sync Server is to checkout our repository and run a build in-place. Once this is done, Sync can be run behind any Web Server that supports the WSGI protocol. Note: These instructions are for the sync server protocol used by Firefox 28 and earlier. Firefox 29 and later include a new sync service that is incompatible with this server. For a server compatible with Firefox 29 and later, see Run your own Sync-1.5 Server. Prerequisites The various parts are using Python 2.6 and Virtualenv. Make sure your system has them. Or install them: • Python 2.6 downloads: http://python.org/download/releases/2.6.6 • Virtualenv: http://pypi.python.org/pypi/virtualenv To run the server, you will also need to have these packages installed: • python-dev • make • mercurial • sqlite3 • openssl-dev 3 Mozilla Services Documentation, Release For example, under a fresh Ubuntu, you can run this command to meet all requirements: $ sudo apt-get install python-dev mercurial sqlite3 python-virtualenv libssl-dev Building the server Get the latest version at https://hg.mozilla.org/services/server-full and run the build command: $ hg clone https://hg.mozilla.org/services/server-full $ cd server-full $ make build This command will create an isolated Python environment and pull all the required dependencies in it. A bin directory is created and contains a paster command that can be used to run the server, using the built-in web server. Note: Occasionally the build may fail due to network issues that make PyPI inaccessible. If you receive an error about “Could not find suitable distribution”, try waiting a little while and then running the build again. If you like, you can run the testsuite to make sure everything is working properly: $ make test If this gives you an error about “pysqlite2”, you may need to install the “pysqlite” package like so: $ ./bin/pip install pysqlite Basic Configuration The server is configured using an ini-like file to specify various runtime settings. The file “etc/sync.conf” will provide a useful starting point”. There is one setting that you must specify before running the server: the client-visible URL for the storage service node. To ensure that the Registration and Node-Assignment flow works correctly, this should be set to the URL at which you will be running the server. Open “etc/sync.conf”, locate and uncomment the following lines: [nodes] fallback_node= http://localhost:5000/ By default the server is configured to use a SQLite database for the storage and the user APIs, with the database file stored at “/tmp/test.db”. You will almost certainly want to change this to a more permanent location: [storage] sqluri= sqlite:////path/to/database/file.db [auth] sqluri= sqlite:////path/to/database/file.db Alternatively, consider using a different database backend as described in Using MYSQL or LDAP or .... 4 Chapter 1. How To... Mozilla Services Documentation, Release Running the Server Now you can run the server using paster and the provided “development.ini” file: $ bin/paster serve development.ini Starting server in PID 29951. serving on 0.0.0.0:5000 view at http://127.0.0.1:5000 Once the server is launched, you can run the Firefox Sync Wizard and choose http://localhost:5000 as your Firefox Custom Sync Server. You should then see a lot of output in the stdout, which are the calls made by the browser for the initial sync. Note: As of October 2015, the Mozilla-hosted sync-1.1 service has been decommissioned, so you must perform some additional configuration to let Firefox use a self-hosted key-exchange server; see below. Configuring Firefox While the sync setup dialog allows you to specify a custom storage server, it does not provide any UI for specifying a custom key-exchange server. In order to pair with a second device, you will need to go to “about:config”, search for “jpake.serverURL” and change its value to the URL of your server with a “/jpake” suffix: • services.sync.jpake.serverURL: http://localhost:5000/jpake Updating the server You should periodically update your code to make sure you’ve got the latest fixes. The following commands will update server-full in place: $ cd /path/to/server-full $ hg pull $ hg update $ make build By default, the build command will checkout the latest released tags for each server product. If you need access to a fix that has not yet been released (or if you just want to live on the bleeding edge) then you can build the development channel like so: $ make build CHANNEL=dev Note: Due to a change in how authentication is handled, users upgrading from a build made prior to January 2012 may need to migrate user accounts into a new database table. To do so: 1. Check that the [auth] section in your config file is using the “services.user.sql.SQLUser” backend. 2. Check if your database contains a “users” table. 3. If so, use the following migration script to move data into the “user” table: deps/server-core/migrations/auth.sql_to_user.sql_migration.txt 1.1. Run your own Sync-1.1 Server 5 Mozilla Services Documentation, Release Security Notes File Permissions The default configuration of the server uses a file-based sqlite database, so you should carefully check that the permis- sions on this file are appropriate for your setup. The file and its containing directory should be writable by the user under which the server is running, and inaccessible to other users on the system. You may like to set the umask of the server process to ensure that any files it creates are readable only by the appropriate user. For example: $ umask 007 $ bin/paster serve development.ini Disabling New Users The default configuration of the server allows new users to create an account through Firefox’s builtin setup screen. This is useful during initial setup, but it means that anybody could sync against your server if they know its URL. You can disable creation of new accounts by setting auth.allow_new_users to false in the config file: [auth] allow_new_users= false Using MYSQL or LDAP or ... Instead of SQLite, you can use alternative backends: • Open-LDAP to store the users • A SQLAlchemy-compatible database, to store the sync data and/or the users Sync has been tested on MySQL and Postgres. In order to use a specific Database, you need to install the required headers, and the required Python library in the local Python environment. See http://www.sqlalchemy.org/docs/core/engines.html#supported-dbapis For example, to run everything in MySQL: 1. install libmysqlclient-dev and mysql-server 2. install Mysql-Python by running bin/easy_install Mysql-Python 3. change the configuration file located at etc/sync.conf For #3, see Configuring the application. For SQL databases, the code will create three tables: • user: contains the user accounts, mapping email to numeric id. • collections: contains collection names for each user, by numeric id. • wbo: contains individual sync records for each user, by numeric id. 6 Chapter 1. How To... Mozilla Services Documentation, Release Running behind a Web Server The built-in server should not be used in production, as it does not really support a lot of load. If you want to set up a production server, you can use different web servers that are compatible with the WSGI protocol. For example: • Apache combined with mod_wsgi • NGinx with Gunicorn or uWSGI • lighttpd with flup, using the fcgi or scgi protocol Note: Remember, you must set the nodes.fallback_node option to the client-visible URL of your sync server. For example, if your server will be located at http://example.com/ff-sync/, the fallback node should be set to this value in your config file: [nodes] fallback_node= http://example.com/ff-sync/ Apache + mod_wsgi Here’s an example of an Apache 2.2 setup that uses mod_wsgi: <Directory/path/to/sync> Order deny,allow Allow from all </Directory> <VirtualHost \*:80> ServerName example.com DocumentRoot/path/to/sync WSGIProcessGroup sync WSGIDaemonProcess sync user=sync group=sync processes=2 threads=25 WSGIPassAuthorization On WSGIScriptAlias//path/to/sync/sync.wsgi CustomLog/var/log/apache2/example.com-access.log combined ErrorLog/var/log/apache2/example.com-error.log </VirtualHost> Here’s the equivalent setup for Apache 2.4, which uses a different syntax for acess control: <Directory/path/to/sync> Require all granted </Directory> <VirtualHost \*:80> ServerName example.com DocumentRoot/path/to/sync WSGIProcessGroup sync WSGIDaemonProcess sync user=sync group=sync processes=2 threads=25 WSGIPassAuthorization On WSGIScriptAlias//path/to/sync/sync.wsgi CustomLog/var/log/apache2/example.com-access.log combined ErrorLog/var/log/apache2/example.com-error.log </VirtualHost> 1.1.