patroni.config module¶
Facilities related to Patroni configuration.
-
class
patroni.config.
Config
(configfile: str, validator: Optional[Callable[[Dict[str, Any]], List[str]]] = <function default_validator>)¶ Bases:
object
Handle Patroni configuration.
This class is responsible for:
Building and giving access to
effective_configuration
from:Config.__DEFAULT_CONFIG
– some sane default values;dynamic_configuration
– configuration stored in DCS;local_configuration
– configuration from config.yml or environment.
Saving and loading
dynamic_configuration
into ‘patroni.dynamic.json’ file located in local_configuration[‘postgresql’][‘data_dir’] directory. This is necessary to be able to restoredynamic_configuration
if DCS was accidentally wiped.Loading of configuration file in the old format and converting it into new format.
Mimicking some
dict
interfaces to make it possible to work with it as with the oldconfig
object.
- Variables
PATRONI_CONFIG_VARIABLE – name of the environment variable that can be used to load Patroni configuration from.
__CACHE_FILENAME – name of the file used to cache dynamic configuration under Postgres data directory.
__DEFAULT_CONFIG – default configuration values for some Patroni settings.
-
PATRONI_CONFIG_VARIABLE
= 'PATRONI_CONFIGURATION'¶
-
__CACHE_FILENAME
= 'patroni.dynamic.json'¶
-
__DEFAULT_CONFIG
: Dict[str, Any] = {'loop_wait': 10, 'postgresql': {'parameters': <CaseInsensitiveDict{'wal_level': 'hot_standby', 'hot_standby': 'on', 'max_connections': 100, 'max_wal_senders': 10, 'max_prepared_transactions': 0, 'max_locks_per_transaction': 64, 'track_commit_timestamp': 'off', 'max_replication_slots': 10, 'max_worker_processes': 8, 'wal_log_hints': 'on'} at 7f2b89f38040>, 'use_slots': True}, 'retry_timeout': 10, 'standby_cluster': {'archive_cleanup_command': '', 'create_replica_methods': '', 'host': '', 'port': '', 'primary_slot_name': '', 'recovery_min_apply_delay': '', 'restore_command': ''}, 'ttl': 30}¶
-
__get_and_maybe_adjust_int_value
(config: Dict[str, Any], param: str, min_value: int) → int¶ Get, validate and maybe adjust a param integer value from the config
dict
.- Parameters
config –
dict
object with new global configuration.param – name of the configuration parameter we want to read/validate/adjust.
min_value – the minimum possible value that a given param could have.
- Returns
an integer value which corresponds to a provided param.
-
__init__
(configfile: str, validator: Optional[Callable[[Dict[str, Any]], List[str]]] = <function default_validator>) → None¶ Create a new instance of
Config
and validate the loaded configuration using validator.Note
Patroni will read configuration from these locations in this order:
file or directory path passed as command-line argument (configfile), if it exists and the file or files found in the directory can be parsed (see
_load_config_path()
), otherwiseYAML file passed via the environment variable (see
PATRONI_CONFIG_VARIABLE
), if the referenced file exists and can be parsed, otherwisefrom configuration values defined as environment variables, see
_build_environment_configuration()
.
- Parameters
configfile – path to Patroni configuration file.
validator – function used to validate Patroni configuration. It should receive a dictionary which represents Patroni configuration, and return a list of zero or more error messages based on validation.
- Raises
ConfigParseError
: if any issue is reported by validator.
-
_build_effective_configuration
(dynamic_configuration: Dict[str, Any], local_configuration: Dict[str, Union[Dict[str, Any], Any]]) → Dict[str, Any]¶ Build effective configuration by merging dynamic_configuration and local_configuration.
Note
local_configuration takes precedence over dynamic_configuration if a setting is defined in both.
- Parameters
dynamic_configuration – Patroni dynamic configuration.
local_configuration – Patroni local configuration.
- Returns
_description_
-
static
_build_environment_configuration
() → Dict[str, Any]¶ Get local configuration settings that were specified through environment variables.
- Returns
dictionary containing the found environment variables and their values, respecting the expected structure of Patroni configuration.
-
_load_config_file
() → Dict[str, Any]¶ Load configuration file(s) from filesystem and apply values which were set via environment variables.
- Returns
final configuration after merging configuration file(s) and environment variables.
-
_load_config_path
(path: str) → Dict[str, Any]¶ Load Patroni configuration file(s) from path.
If path is a file, load the yml file pointed to by path. If path is a directory, load all yml files in that directory in alphabetical order.
- Parameters
path – path to either an YAML configuration file, or to a folder containing YAML configuration files.
- Returns
configuration after reading the configuration file(s) from path.
- Raises
ConfigParseError
: if path is invalid.
-
static
_process_postgresql_parameters
(parameters: Dict[str, Any], is_local: bool = False) → Dict[str, Any]¶ Process Postgres parameters.
Note
If is_local configuration discard any setting from parameters that is listed under
CMDLINE_OPTIONS
as those are supposed to be set only through dynamic configuration.When setting parameters from
CMDLINE_OPTIONS
through dynamic configuration their value will be validated as per the validator defined in that very same attribute entry. If the given value cannot be validated, a warning will be logged and the default value of the GUC will be used instead.Some parameters from
CMDLINE_OPTIONS
cannot be set even if not is_local configuration:listen_addresses
: inferred frompostgresql.listen
local configuration or fromPATRONI_POSTGRESQL_LISTEN
environment variable;
port
: inferred frompostgresql.listen
local configuration or fromPATRONI_POSTGRESQL_LISTEN
environment variable;
cluster_name
: set throughscope
local configuration or throughPATRONI_SCOPE
environmentvariable;
hot_standby
: always enabled;wal_log_hints
: always enabled.
- Parameters
parameters – Postgres parameters to be processed. Should be the parsed YAML value of
postgresql.parameters
configuration, either from local or from dynamic configuration.is_local – should be
True
if parameters refers to local configuration, orFalse
if parameters refers to dynamic configuration.
- Returns
new value for
postgresql.parameters
after processing and validating parameters.
-
_safe_copy_dynamic_configuration
(dynamic_configuration: Dict[str, Any]) → Dict[str, Any]¶ Create a copy of dynamic_configuration.
Merge dynamic_configuration with
__DEFAULT_CONFIG
(dynamic_configuration takes precedence), and processpostgresql.parameters
from dynamic_configuration through_process_postgresql_parameters()
, if present.Note
The following settings are not allowed in
postgresql
section as they are intended to be local configuration, and are removed if present:connect_address
;proxy_address
;listen
;config_dir
;data_dir
;pgpass
;authentication
;
Besides that any setting present in dynamic_configuration but absent from
__DEFAULT_CONFIG
is discarded.- Parameters
dynamic_configuration – Patroni dynamic configuration.
- Returns
copy of dynamic_configuration, merged with default dynamic configuration and with some sanity checks performed over it.
-
_validate_and_adjust_timeouts
(config: Dict[str, Any]) → None¶ Validate and adjust
loop_wait
,retry_timeout
, andttl
values if necessary.Minimum values:
loop_wait
: 1 second;retry_timeout
: 3 seconds.ttl
: 20 seconds;
Maximum values: In case if values don’t fulfill the following rule,
retry_timeout
andloop_wait
are reduced so that the rule is fulfilled:loop_wait + 2 * retry_timeout <= ttl
- Parameters
config –
dict
object with new global configuration.
Check
nofailover
/failover_priority
config and warn user if it’s contradictory.Note
To preserve sanity (and backwards compatibility) the
nofailover
tag will still exist. A contradictory configuration is one wherenofailover
isTrue
butfailover_priority > 0
, or wherenofailover
isFalse
, butfailover_priority <= 0
. Essentially,nofailover
andfailover_priority
are communicating different things. This checks for this edge case (which is a misconfiguration on the part of the user) and warns them. The behaviour is as iffailover_priority
were not provided (i.enofailover
is the bedrock source of truth)
-
property
config_file
¶ Path to Patroni configuration file, if any, else
None
.
-
copy
() → Dict[str, Any]¶ Get a deep copy of effective Patroni configuration.
- Returns
a deep copy of the Patroni configuration.
-
property
dynamic_configuration
¶ Deep copy of cached Patroni dynamic configuration.
-
get
(key: str, default: Optional[Any] = None) → Any¶ Get effective value of
key
setting from Patroni configuration root.Designed to work the same way as
dict.get()
.- Parameters
key – name of the setting.
default – default value if key is not present in the effective configuration.
- Returns
value of key, if present in the effective configuration, otherwise default.
-
classmethod
get_default_config
() → Dict[str, Any]¶ Deep copy default configuration.
- Returns
copy of
__DEFAULT_CONFIG
-
get_global_config
(cluster: Optional[patroni.dcs.Cluster]) → patroni.config.GlobalConfig¶ Instantiate
GlobalConfig
based on input.Use the configuration from provided cluster (the most up-to-date) or from the local cache if cluster.config is not initialized or doesn’t have a valid config.
- Parameters
cluster – the currently known cluster state from DCS.
- Returns
GlobalConfig
object.
-
property
local_configuration
¶ Deep copy of cached Patroni local configuration.
- Returns
copy of
_local_configuration
-
reload_local_configuration
() → Optional[bool]¶ Reload configuration values from the configuration file(s).
Note
Designed to be used when user applies changes to configuration file(s), so Patroni can use the new values with a reload instead of a restart.
- Returns
True
if changes have been detected between current local configuration
-
save_cache
() → None¶ Save dynamic configuration to
patroni.dynamic.json
under Postgres data directory.Note
patroni.dynamic.jsonXXXXXX
is created as a temporary file and than renamed topatroni.dynamic.json
, whereXXXXXX
is a random suffix.
-
set_dynamic_configuration
(configuration: Union[patroni.dcs.ClusterConfig, Dict[str, Any]]) → bool¶ Set dynamic configuration values with given configuration.
- Parameters
configuration – new dynamic configuration values. Supports
dict
for backward compatibility.- Returns
True
if changes have been detected between current dynamic configuration and the new dynamic configuration,False
otherwise.
-
class
patroni.config.
GlobalConfig
(config: Dict[str, Any])¶ Bases:
object
A class that wraps global configuration and provides convenient methods to access/check values.
It is instantiated either by calling
get_global_config()
orConfig.get_global_config()
, which picks either a configuration from providedCluster
object (the most up-to-date) or from the local cache ifClusterConfig
is not initialized or doesn’t have a valid config.-
__init__
(config: Dict[str, Any]) → None¶ Initialize
GlobalConfig
object with given config.- Parameters
config – current configuration either from
ClusterConfig
or fromConfig.dynamic_configuration()
.
-
check_mode
(mode: str) → bool¶ Checks whether the certain parameter is enabled.
- Parameters
mode – parameter name, e.g.
synchronous_mode
,failsafe_mode
,pause
,check_timeline
, and so on.- Returns
True
if parameter mode is enabled in the global configuration.
-
get
(name: str) → Any¶ Gets global configuration value by name.
- Parameters
name – parameter name.
- Returns
configuration value or
None
if it is missing.
-
get_int
(name: str, default: int = 0) → int¶ Gets current value of name from the global configuration and try to return it as
int
.- Parameters
name – name of the parameter.
default – default value if name is not in the configuration or invalid.
- Returns
currently configured value of name from the global configuration or default if it is not set or invalid.
-
get_standby_cluster_config
() → Union[Dict[str, Any], Any]¶ Get
standby_cluster
configuration.- Returns
a copy of
standby_cluster
configuration.
-
property
is_paused
¶ True
if cluster is in maintenance mode.
-
property
is_standby_cluster
¶ True
if global configuration has a validstandby_cluster
section.
-
property
is_synchronous_mode
¶ True
if synchronous replication is requested and it is not a standby cluster config.
-
property
is_synchronous_mode_strict
¶ True
if at least one synchronous node is required.
-
property
maximum_lag_on_failover
¶ Currently configured value of
maximum_lag_on_failover
from the global configuration.Assume
1048576
if it is not set or invalid.
-
property
maximum_lag_on_syncnode
¶ Currently configured value of
maximum_lag_on_syncnode
from the global configuration.Assume
-1
if it is not set or invalid.
-
property
min_synchronous_nodes
¶ The minimal number of synchronous nodes based on whether
synchronous_mode_strict
is enabled or not.
-
property
primary_start_timeout
¶ Currently configured value of
primary_start_timeout
from the global configuration.Assume
300
if it is not set or invalid.Note
master_start_timeout
is still supported to keep backward compatibility.
-
property
primary_stop_timeout
¶ Currently configured value of
primary_stop_timeout
from the global configuration.Assume
0
if it is not set or invalid.Note
master_stop_timeout
is still supported to keep backward compatibility.
-
property
synchronous_node_count
¶ Currently configured value of
synchronous_node_count
from the global configuration.Assume
1
if it is not set or invalid.
-
-
patroni.config.
default_validator
(conf: Dict[str, Any]) → List[str]¶ Ensure conf is not empty.
Designed to be used as default validator for
Config
objects, if no specific validator is provided.- Parameters
conf – configuration to be validated.
- Returns
an empty list –
Config
expects the validator to return a list of 0 or more issues found while validating the configuration.- Raises
ConfigParseError
: if conf is empty.
-
patroni.config.
get_global_config
(cluster: Optional[patroni.dcs.Cluster], default: Optional[Dict[str, Any]] = None) → patroni.config.GlobalConfig¶ Instantiates
GlobalConfig
based on the input.- Parameters
cluster – the currently known cluster state from DCS.
default – default configuration, which will be used if there is no valid cluster.config.
- Returns
GlobalConfig
object.