ofxtools Documentation Release 0.8.22 Christopher Singley Nov 20, 2020 Contents: 1 Installing ofxtools 3 1.1 Installation dependencies.........................................3 1.2 Standard installation...........................................3 1.3 Bleeding edge installation........................................4 1.4 Developer’s installation.........................................4 1.5 Extra goodies...............................................4 2 Downloading OFX Data With ofxget5 2.1 Locating ofxget..............................................5 2.2 Using ofxget - TL;DR..........................................5 2.3 Storing ofxget passwords in the system keyring.............................6 2.4 Using ofxget - in depth..........................................7 2.5 Scanning for OFX connection formats.................................. 11 3 Using OFXClient in Another Program 15 4 Parsing OFX Data 17 4.1 Deviations from the OFX specification................................. 19 5 Generating OFX 21 6 Using ofxtools with SQL 23 7 Contributing to ofxtools 27 8 Adding New OFX Messages 29 8.1 Request and Response.......................................... 29 8.2 Recurring Requests............................................ 36 8.3 Synchronization............................................. 40 8.4 Extending the Message Set........................................ 42 9 Additional Resources 45 9.1 More open-source OFX code...................................... 45 10 What is it? 47 11 Where is it? 49 i 12 Installation Dependencies 51 ii ofxtools Documentation, Release 0.8.22 ofxtools is a Python library for working with Open Financial Exchange (OFX) data - the standard format for downloading financial information from banks and stockbrokers. OFX data is widely provided by financial institutions so that their customers can import transactions into financial management software such as Quicken, Microsoft Money, or GnuCash. If you want to download your transaction data outside of one of these programs - if you wish to develop a Python application to use this data - if you need to generate your own OFX-formatted data. ofxtools is for you! Contents: 1 ofxtools Documentation, Release 0.8.22 2 Contents: CHAPTER 1 Installing ofxtools You have a few options to install ofxtools. If you like, you can install it in a virtual environment, but since ofxtools has no external dependencies, that doesn’t really gain you much. A simpler option for keeping clutter out of your system Python site is the user install option, which is recommended if only one system user needs the package (the normal situation). 1.1 Installation dependencies You need to install Python 3 (at least version 3.6) in order to use ofxtools. It won’t work at all under Python 2. In order to use the OFX client to download OFX files, your Python 3 installation needs to be able to validate SSL certificates. Users of Mac OS X should heed the following note from the ReadMe.rtf included with the Python installer as of version 3.6: This variant of Python 3.6 now includes its own private copy of OpenSSL 1.0.2. Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used. This also means that the trust certificates in system and user keychains managed by the Keychain Access application and the security command line utility are no longer used as defaults by the Python ssl module. For 3.6.0, a sample com- mand script is included in /Applications/Python 3.6 to install a curated bundle of default root certificates from the third-party certifi package. To facilitate keeping this important security package up to date, it’s advisable for Mac users to instead employ pip: $ pip install certifi 1.2 Standard installation If you just want to use the ofxtools library, and you don’t have any special needs, you should probably install the most recent release on PyPI: 3 ofxtools Documentation, Release 0.8.22 $ pip install --user ofxtools Or if you want to install it systemwide, as root just run: $ pip install ofxtools 1.3 Bleeding edge installation To install the most recent prerelease (which is where the magic happens, and also the bugs), you can download the current master, unzip it, and install via the included setup file: $ pip install --user . 1.4 Developer’s installation If you want to hack on ofxtools, you should clone the source and install is in development mode: $ git clone https://github.com/csingley/ofxtools.git $ cd ofxtools $ pip install -e . $ pip install -r ofxtools/requirements-development.txt 1.5 Extra goodies In addition to the Python package, these methods will also install the ofxget script - a basic command line interface for downloading files from OFX servers. pip uninstall ofxtools will remove this script along with the package. Some financial institutions make you use their web application to generate OFX (or QFX) files that you can download via your browser. If they give you a choice, prefer “OFX” or “Microsoft Money” format over “QFX” or “Quicken”. Other financial institutions are good enough to offer you a server socket, to which ofxtools can connect and download OFX data for you. 4 Chapter 1. Installing ofxtools CHAPTER 2 Downloading OFX Data With ofxget 2.1 Locating ofxget The ofxget shell script should have been installed by pip along with the ofxtools library. If the install location isn’t already in your $PATH, you’ll likely want to add it. User installation • Mac: ~/Library/PythonX.Y/bin/ofxget • Windows: AppData\Roaming\Python\PythonXY\Scripts\ofxget • Linux/BSD/etc.: ~/.local/bin/ofxget Site installation • Mac: /Library/Frameworks/Python.framework/Versions/X.Y/bin/ofxget • Windows: Good question; anybody know? • Linux/BSD/etc.: /usr/local/bin/ofxget Virtual environment installation • </path/to/venv/root>/bin/ofxget If all else fails, you can execute python -m ofxtools.scripts.ofxget, or directly run python </path/ to/ofxtools>/scripts/ofxget.py. You can check where exactly that is by opening a Python interpreter and saying: >>> from ofxtools.scripts import ofxget >>> print(ofxget.__file__) 2.2 Using ofxget - TL;DR Find your financial institution’s nickname: 5 ofxtools Documentation, Release 0.8.22 $ ofxget list If your financial institution is listed, then the quickest way to get your hands on some OFX data is to say: $ ofxget stmt <server_nickname> -u <your_username> --all Enter your password when prompted. However, you really won’t want to set the --all option every time you download a statement; it’s very inefficient. Slightly more verbosely, you might say : $ ofxget acctinfo <server_nickname> -u <your_username> --write $ ofxget stmt <server_nickname> The first command requests a list of accounts and saves it to your config file along with your user name. This is in the nature of a first-time setup chore. The second command is the kind of thing you’d run on a regular basis. It requests statements for each account listed in your config file for a given server nickname. 2.3 Storing ofxget passwords in the system keyring Note: this feature is experimental. Expect bugs; kindly report them. Rather than typing them in each time, you can securely store your passwords in the system keyring (if one is available) and have ofxget retrieve them for you. Examples of such keyring software include: • Windows Credential Locker • Mac Keychain • Freedesktop Secret Service (used by GNOME et al.) • KWallet (used by KDE) To use these services, you will need to clutter up your nice clean ofxtools by installing the python-keyring package. $ pip install --user keyring Additionally, KDE users will need to install dbus-python. Note the recommendation in the python-keyring docs to install it systemwide via your package manager. Once these dependencies have been satisfied, you can pass the --savepass option to ofxget anywhere it wants a password, e.g. $ ofxget acctinfo <server_nickname> -u <your_username> --write --savepass That should set you up to download statements easily. To overwrite an existing password, simply add the --savepass option again and you will be prompted for a new password. To delete a password entirely, you’ll need to use your OS facilities for managing these passwords (they are stored under “ofxtools”, with an entry for each server nickname). 6 Chapter 2. Downloading OFX Data With ofxget ofxtools Documentation, Release 0.8.22 2.4 Using ofxget - in depth ofxget takes two positional arguments - request type (mandatory) and server nickname (optional) - along with a bunch of optional keyword arguments. See the --help for explanation of the script options. Available request types (as indicated in the --help) are list, scan, prof, acctinfo, stmt, stmtend and tax1099. We’ll work through most of these in an example of bootstrapping a full configuration for American Express. 2.4.1 Basic connectivity: requesting an OFX profile We must know the OFX server URL in order to connect at all. ofxtools contains a database of all US financial institutions listed on the OFX Home website that I could get to speak OFX with me. If you can’t find your bank in ofxget (or if you’re having a hard time configuring a connection), OFX Home should be your first stop. If you prefer, the OFX Blog also makes the same data available in a different format. Be sure to review user-posted comments on either site. You can also try the fine folks at GnuCash, who share the struggle. OFX Home has a listing for AmEx, giving a URL plus the ORG/FID pair (i.e. <FI><ORG> and <FI><FID> in the signon request.) This aggregate is optional per the OFX spec, and if your FI is running its own OFX server it is optional - many major providers don’t need it to connect. However, Quicken always sends <FI>, so your bank may require it anyway. AmEx appears to be one of these; its OFX server throws HTTP error 503 if you omit ORG/FID. Using the connection information from OFX Home, first we will try to establish basic connectivity by requesting an OFX profile, which does not require authenticating a login. $ ofxget prof --org AMEX --fid 3101 --url https://online.americanexpress.com/myca/ ,!ofxdl/desktop/desktopDownload.do\?request_type\=nl_ofxdownload This hairy beast of a command can be used for any arbitrary OFX server.