Seedbox Release
Total Page:16
File Type:pdf, Size:1020Kb
seedbox Release June 20, 2015 Contents 1 Features 3 1.1 Getting Started..............................................3 1.2 Design..................................................7 1.3 Changelog................................................8 1.4 Source Code............................................... 19 i ii seedbox, Release SeedboxManager is an automated task manager for synchronizing files from a seedbox to your home library. Contents 1 seedbox, Release 2 Contents CHAPTER 1 Features • Copy multiple files simultaneously from remote server using secure protocol. • Maintains cache of files seen and state of processing. • Admin UI and REST APIs for managing process and cached data. Contents: 1.1 Getting Started Create virtualenv virtualenv ~/seedbox/ Start virtualenv cd ~/seedbox source bin/activate Install SeedboxManager in the virtualenv: mkdir etc pip install SeedboxManager Running SeedboxManager: seedmgr Running SeedboxManager from crontab: crontab -e @hourly /home/USER/seedbox/bin/seedmgr >> /home/USER/seedbox/etc/seedbox/cron-sync.log 2>&1 Note: As part of installing in virtualenv the sample configuration files will be installed into the ~/seedbox/etc/seedbox folder. Starting Admin UI and REST API: dbadmin passwd --password <your_password> dbadmin run sqlite:////home/USER/.seedbox/torrent.db >> /home/USER/seedbox/etc/seedbox/admin.log 2>&1 3 seedbox, Release 1.1.1 Available Tasks by Phase Phases and Built-in Tasks Phase Task Description prepare filecopy copy supported media files related to torrents from download directory to sync directory fileunrar decompress rar media files related to torrents from download directory to sync directory activate filesync rsync files in sync directory to remote server location complete filedelete delete media files from sync directory after successful sync to remote server location Congiguration Possible configuration file locations (General to specific): /etc /etc/seedbox # if virtualenv used ~/seedbox/etc ~/seedbox/etc/seedbox ~ ~/.seedbox <current working directory> Note: configuration filename: seedbox.conf virtualenv approach is the recommended approach. Multiple configuration files are supported such that each supported folder is checked for a configuration file and loaded from most general to more specific. Each successive file will override values from the previous. The folder of the most specific configuration file found will be considered the resource folder where all log files are stored by default. Command line interface: usage: seedmgr [-h] [--config-dir DIR] [--config-file PATH] [--logconfig LOG_CONFIG] [--logfile LOG_FILE] [--loglevel LOG_LEVEL] [--version] optional arguments: -h, --help show this help message and exit --config-dir DIR Path to a config directory to pull *.conf files from. This file set is sorted, so as to provide a predictable parse order if individual options are over-ridden. The set is parsed after the file(s) specified via previous --config-file, arguments hence over-ridden options in the directory take precedence. --config-file PATH Path to a config file to use. Multiple config files can be specified, with values in later files taking precedence. The default files used are: None --logconfig LOG_CONFIG specific path and filename of logging configuration (override defaults) --logfile LOG_FILE specify name of log file (location is resource path) --loglevel LOG_LEVEL specify logging level to log messages at --version show program's version number and exit Sample Configuration A generated configuration file that contains each option a designation for required, a help message, default value, and associated type. 4 Chapter 1. Features seedbox, Release Sample Configuration [DEFAULT] # # Options defined in seedbox.options # # **REQUIRED** Base path (string value) #base_path=<None> # Location torrent client stores data files (string value) #base_client_path=<None> [database] # # Options defined in seedbox.db # # The connection string used to connect to the database (string value) #connection=sqlite:///$config_dir/torrent.db # Timeout before idle sql connections are reaped (integer value) #idle_timeout=3600 # Verbosity of SQL debugging information. 0=None, 100=All (integer value) #connection_debug=0 [process] # # Options defined in seedbox.process.flow # # name of tasks associated with prepare phase (list value) #prepare= # name of tasks associated with activate phase (list value) #activate= # name of tasks associated with complete phase (list value) #complete= # # Options defined in seedbox.process.manager # # max processes to use for performing sync of torrents (integer value) #max_processes=4 [tasks] # 1.1. Getting Started 5 seedbox, Release # Options defined in seedbox.tasks.base # # Location to temp media copies for syncing to library (string value) #sync_path=/tmp/sync [tasks_filesync] # # Options defined in seedbox.tasks.filesync # # rsync dryrun option (boolean value) #dryrun=false # rsync verbose option (boolean value) #verbose=false # rsync progress option (boolean value) #progress=false # rsync perms option (boolean value) #perms=true # rsync delayupdates option (boolean value) #delayupdates=true # rsync recursive option (boolean value) #recursive=true # rsync chmod option (string value) #chmod=ugo+rwx # rsync-ssh identity option (ssh key) (string value) #identity=<None> # rsync-ssh port (string value) #port=22 # User name on remote system (ssh) (string value) #remote_user=<None> # Host name/IP Address of remote system (string value) #remote_host=<None> # rsync destination path (string value) #remote_path=<None> [tasks_synclog] # # Options defined in seedbox.tasks.subprocessext # # Output directory for stdout files (string value) #stdout_dir=$config_dir/sync_out 6 Chapter 1. Features seedbox, Release # Output directory for stderr files (string value) #stderr_dir=$config_dir/sync_err # Write output to stdout (boolean value) #stdout_verbose=false # Output verbose details about exceptions (boolean value) #stderr_verbose=true [torrent] # # Options defined in seedbox.torrent # # **REQUIRED** Location of the .torrent files (string value) #torrent_path=<None> # **REQUIRED** Location(s) of the media files (list value) #media_paths=<None> # **REQUIRED** Location of the downloading torrents (string value) #incomplete_path=<None> # List of video filetypes to support. (ignore others) (list value) #video_filetypes=.avi,.mp4,.mkv,.mpg # List of compressed filetypes to support. (ignore others) (list value) #compressed_filetypes=.rar # Minimum file size of a media file (integer value) #minimum_file_size=75000000 1.2 Design 1.2.1 Key Design Aspects Leverage plugin pattern to provide implementations for Tasks and Database. All plugins are registered as entry points within python setup for the project. Tasks seedbox.tasks.base.BaseTask provides the base implementation for a Task. All Tasks must implement the following methods • is_actionable if the result is True then the Task will be executed else skip executing the Task. • execute provides the Task specific implementation; all exceptions are handled by the BaseTask to guarantee consistent error handling. The BaseTask provides the ability to include additional media files for processing, and captures telemetry (execution time) for each Task execution. 1.2. Design 7 seedbox, Release Database seedbox.db.base.Connection provides the abstract definition for the interacting with a specific database implementation. The base capabilities include • connection / session creation • create • update • delete • read • wipe data • backup data • upgrade model/schema See also: • Dynamic Code Patterns Extending Your Applications with Plugins • SQLAlchemy 1.3 Changelog 1.3.1 2.3.15 • [RELEASE] Update to version v2.3.15 • [TASK] Leverage invoke for managing releases • [TASK] Simplified custom torrent parser • [TASK] Update to latest pbr 1.3.2 2.3.14 • [RELEASE] Update to version v2.3.14 • [TASK] Database API access clean up • [TASK] Drop support for Python 2.6 and 3.3 • [TASK] Cleanup options define and loading • updated how source code documentation handled • updated landscape config • added landscape configuration • [TASK] Basic cleanup of code base • update doc formatting • updated alpine version • added .dockerignore • added Dockerfile 8 Chapter 1. Features seedbox, Release • fix file permissions 1.3.3 2.3.13 • [RELEASE] Update to version v2.3.13 • [FEATURE] Update requirements • updated version from 3 to 4 • [BUGFIX] Fix total_time column type 1.3.4 2.3.12 • [RELEASE] Update to version v2.3.12 • [TASK] Switch to oslo_config • [FEATURE] Added Admin UI and REST APIs • [TASK] Fixed error message formatting • [TASK] Standards alignment with hacking • [TASK] Requirements updates • [TASK] Fixed error message formatting • [TASK] Standards alignment with hacking • [TASK] Requirements updates • [TASK] Document updates • [TASK] Remove intersphinx dependency • [TASK] Upgrade version of oslo.config • [TASK] Update README to include Waffle • [TASK] Reduce output from update_version • [TASK] General cleanup activities 1.3.5 2.3.11 • Update to version v2.3.11 • [TASK] General cleanup activities • [TASK] Apply shell lint changes to update_version • [TASK] Fix up spelling errors • [BUGFIX] Fix capture of filename when coming from compressed file • remove patch for what was oslo.config a bug; replaced original way of setting config_dir with set_default. Consistent response regardless of version of oslo.config • minor change to handle the fact we do not publish the release branch to the remote, and git flow raises an exception when attempting to remove the remote branch. now catching that exception