Welcome to config_resolver’s documentation!

User Manual

Rationale

Configuration values are usually found on well defined locations. On Linux systems this is usually /etc or the home folder. The code for finding these config files is always the same. But finding config files can be more interesting than that:

  • If config files contain passwords, the application should issue appropriate warnings if it encounters an insecure file and refuse to load it.
  • The expected structure in the config file can be versioned (think: schema). If an application is upgraded and expects new values to exist in an old version file, it should notify the user.
  • It should be possible to override the configuration on a per installed instance, even per-execution.

config_resolver tackles all these challenges in a simple-to-use drop-in module. The module uses no additional external modules (no additional dependencies, pure Python).

Additionally, the existing “default values” mechanisms in Python is broken, in that cannot have two different default values for the same variable name in two different sections.

Description / Usage

The module provides two main classes:

The simple usage for both is identical. The only difference is the above mentioned decision to load files or not:

from config_resolver imoprt Config
cfg = Config('acmecorp', 'bird_feeder')

This will look for config files in (in that order):

  • /etc/acmecorp/bird_feeder/app.ini
  • ~/.acmecorp/bird_feeder/app.ini
  • ./.acmecorp/bird_feeder/app.ini

If all files exist, one which is loaded later, will override the values of an earlier file. No values will be removed, this means you can put system-wide defaults in /etc and specialise from there.

Files are parsed using the default Python configparser.ConfigParser (i.e. ini files).

Advanced Usage

Versioning

It is pretty much always useful to keep track of the expected “schema” of a config file. If in a later version of your application, you decide to change a configuration value’s name, remove a variable, or require a new one the end-user needs to be notified.

For this use-case, you can create versioned config_resolver.Config instances in your application:

cfg = Config('group', 'app', version='2.1')

Config file example:

[meta]
version=2.1

[database]
dsn=foobar

If you don’t specify a version number in the construcor, an unversioned file is assumed.

Only “major” and “minor” numbers are supported. If the application encounters a file with a different “major” value, it will raise a config_resolver.IncompatibleVersion exception. Differences in minor numbers are only logged with a “warning” level.

Rule of thumb: If your application accepts a new config value, but can function just fine with default values, increment the minor number. If on the other hand, something has changed, and the user needs to change the config file, increment the major number.

Requiring files (bail out if no config is found)

Since version 3.3.0, you have a bit more control about how files are loaded. The config_resolver.Config class takes a new argument: require_load. If this is set to True, an OSError is raised if no config file was loaded. Alternatively, and, purely a matter of taste, you can leave this on it’s default False value and inspect the loaded_files attribute on the config_resolver.Config instance. If it’s empty, nothing has been loaded.

Overriding internal defaults

Both the search path and the basename of the file (app.ini) can be overridden by the application developer via the API and by the end-user via environment variables.

By the application developer

Apart from the “group name” and “application name”, the config_resolver.Config class accepts search_path and filename as arguments. search_path controls to what folders are searched for config files, filename controls the basename of the config file. filename is especially useful if you want to separate different concepts into different files:

app_cfg = Config('acmecorp', 'bird_feeder')
db_cfg = Config('acmecorp', 'bird_feeder', filename='db.ini')

By the end-user

The end-user has access to two environment variables:

  • <GROUP_NAME>_<APP_NAME>_PATH overrides the default search path.
  • <GROUP_NAME>_<APP_NAME>_FILENAME overrides the default basename.

Note

If an application uses more than one config instance, the environment variable will override all of them! In that case, it is recommended to only override the “_PATH” variable. It should prove sufficient.

Logging

All operations are logged using the default logging package. The log messages include the absolute names of the loaded files. If a file is not loadable, a WARNING message is emitted. It also contains a couple of DEBUG messages. If you want to see those messages on-screen you could do the following:

import logging
from config_resolver import Config
logging.basicConfig(level=logging.DEBUG)
conf = Config('mycompany', 'myapplication')

Environment Variables

The resolver can also be manipulated using environment variables to allow different values for different running instances. The variable names are all upper-case and are prefixed with both group- and application-name.

<group_name>_<app_name>_PATH

The search path for config files. You can specify multiple paths by separating it by the system’s path separator default (: on Linux).

If the path is prefixed with +, then the path elements are appended to the default search path.

<group_name>_<app_name>_FILENAME
The file name of the config file. Note that this should not be given with leading path elements. It should simply be a file basename (f.ex.: my_config.ini)

Difference to ConfigParser

There is one major difference to the default Python configparser.ConfigParser: the .get method accepts a “default” parameter. If specified, that value is returned if configparser.ConfigParser does not have a value. I find the support for default values in the core library’s configparser.ConfigParser lacking, you cannot have two options with the same name in two sections with different values. Imagine the following:

[database1]
dsn=sqlite:///tmp/db.sqlite3

[database2]
dsn=sqlite:///tmp/db2.sqlite3

In the core configparser.ConfigParser you could not specify two different default values!

Note

The core configparser.ConfigParser default mechanism still takes precedence!

Debugging

Creating the config object will not raise an error (except if asked to do so). Instead it will always return a valid, (but possibly empty) config_resolver.Config instance. So errors can be hard to see sometimes.

Your first stop should be to configure logging and look at the emitted messages.

In order to determine whether any config file was loaded, you can look into the loaded_files instance variable. It contains a list of all the loaded files, in the order of loading. If that list is empty, no config has been found. Also remember that the order is important. Later elements will override values from earlier elements.

Additionally, another instance variable named active_path represents the search path after processing of environment variables and runtime parameters. This may also be useful to display informtation to the end-user.

Examples

A simple config instance (with logging):

import logging
from config_resolver import Config

logging.basicConfig(level=logging.DEBUG)
cfg = Config("acmecorp", "bird_feeder")
print cfg.get('section', 'var')

An instance which will not load unsecured files:

import logging
from config_resolver import SecuredConfig

logging.basicConfig(level=logging.DEBUG)
cfg = SecuredConfig("acmecorp", "bird_feeder")
print cfg.get('section', 'var')

Loading a versioned config file:

import logging
from config_resolver import Config

logging.basicConfig(level=logging.DEBUG)
cfg = Config("acmecorp", "bird_feeder", version="1.0")
print cfg.get('section', 'var')

Default values:

import logging
from config_resolver import Config

logging.basicConfig(level=logging.DEBUG)
cfg = Config("acmecorp", "bird_feeder", version="1.0")

# This will not raise an error (but emit a DEBUG log entry).
print cfg.get('section', 'example_non_existing_option_name', default=10)

# this may raise a "NoOptionError"
print cfg.get('section', 'example_non_existing_option_name')

# this may raise a "NoSectionError"
print cfg.get('example_non_existing_section_name', 'varname')

Indices and tables