patroni.postgresql package¶
Submodules¶
- patroni.postgresql.bootstrap module
- patroni.postgresql.callback_executor module
- patroni.postgresql.cancellable module
- patroni.postgresql.citus module
- patroni.postgresql.config module
- patroni.postgresql.connection module
- patroni.postgresql.misc module
- patroni.postgresql.postmaster module
- patroni.postgresql.rewind module
- patroni.postgresql.slots module
- patroni.postgresql.sync module
- patroni.postgresql.validator module
Module contents¶
-
class
patroni.postgresql.
Postgresql
(config: Dict[str, Any])¶ Bases:
object
-
POSTMASTER_START_TIME
= 'pg_catalog.pg_postmaster_start_time()'¶
-
TL_LSN
= "CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_{0}file_name(pg_catalog.pg_current_{0}_{1}()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_current_{0}{2}_{1}(), '0/0')::bigint END, pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_last_{0}_replay_{1}(), '0/0')::bigint, pg_catalog.pg_{0}_{1}_diff(COALESCE(pg_catalog.pg_last_{0}_receive_{1}(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_{0}_replay_paused()"¶
-
__init__
(config: Dict[str, Any]) → None¶ Initialize self. See help(type(self)) for accurate signature.
-
_do_stop
(mode: str, block_callbacks: bool, checkpoint: bool, on_safepoint: Optional[Callable[[…], Any]], on_shutdown: Optional[Callable[[…], Any]], before_shutdown: Optional[Callable[[…], Any]], stop_timeout: Optional[int]) → Tuple[bool, bool]¶
-
_get_gucs
() → patroni.collections.CaseInsensitiveSet¶ Get all available GUCs based on
postgres --describe-config
output.- Returns
all available GUCs in the local Postgres server.
-
_pre_promote
() → bool¶ Runs a fencing script after the leader lock is acquired but before the replica is promoted. If the script exits with a non-zero code, promotion does not happen and the leader key is removed from DCS.
-
_query
(sql: str, *params: Any) → List[Tuple[Any, …]]¶ Execute sql query with params and optionally return results.
- Parameters
sql – SQL statement to execute.
params – parameters to pass.
- Returns
a query response as a list of tuples if there is any.
- Raises
Error
if had issues while executing sql.PostgresConnectionException
: if had issues while connecting to the database.RetryFailedError
: if it was detected that connection/query failed due to PostgreSQL restart.
-
_wait_for_connection_close
(postmaster: patroni.postgresql.postmaster.PostmasterProcess) → None¶
-
static
_wal_position
(is_primary: bool, wal_position: int, received_location: Optional[int], replayed_location: Optional[int]) → int¶
-
property
available_gucs
¶ GUCs available in this Postgres server.
-
call_nowait
(cb_type: patroni.postgresql.callback_executor.CallbackAction) → None¶ pick a callback command and call it without waiting for it to finish
-
property
callback
¶
-
property
can_advance_slots
¶ True
if :attr:major_version
is greater than 110000.
-
can_create_replica_without_replication_connection
(replica_methods: Optional[List[str]]) → bool¶ go through the replication methods to see if there are ones that does not require a working replication connection.
-
property
cb_called
¶
-
check_for_startup
() → bool¶ Checks PostgreSQL status and returns if PostgreSQL is in the middle of startup.
-
check_startup_state_changed
() → bool¶ Checks if PostgreSQL has completed starting up or failed or still starting.
Should only be called when state == ‘starting’
- Returns
True if state was changed from ‘starting’
-
checkpoint
(connect_kwargs: Optional[Dict[str, Any]] = None, timeout: Optional[float] = None) → Optional[str]¶
-
property
cluster_info_query
¶ Returns the monitoring query with a fixed number of fields.
The query text is constructed based on current state in DCS and PostgreSQL version:
function names depend on version. wal/lsn for v10+ and xlog/location for pre v10.
for primary we query timeline_id (extracted from pg_walfile_name()) and pg_current_wal_lsn()
for replicas we query pg_last_wal_receive_lsn(), pg_last_wal_replay_lsn(), and pg_is_wal_replay_paused()
for v9.6+ we query primary_slot_name and primary_conninfo from pg_stat_get_wal_receiver()
for v11+ with permanent logical slots we query from pg_replication_slots and aggregate the result
for standby_leader node running v9.6+ we also query pg_control_checkpoint to fetch timeline_id
if sync replication is enabled we query pg_stat_replication and aggregate the result. In addition to that we get current values of synchronous_commit and synchronous_standby_names GUCs.
If some conditions are not satisfied we simply put static values instead. E.g., NULL, 0, ‘’, and so on.
-
connection
() → Union[connection3, Connection3[Any]]¶
-
controldata
() → Dict[str, str]¶ return the contents of pg_controldata, or non-True value if pg_controldata call failed
-
property
create_replica_methods
¶
-
property
data_dir
¶
-
property
database
¶
-
property
enforce_hot_standby_feedback
¶
-
ensure_major_version_is_known
() → bool¶ Calls configure_server_parameters() if _major_version is not known
- Returns
True if _major_version is set, otherwise False
-
follow
(member: Optional[Union[patroni.dcs.Leader, patroni.dcs.Member]], role: str = 'replica', timeout: Optional[float] = None, do_reload: bool = False) → Optional[bool]¶ Reconfigure postgres to follow a new member or use different recovery parameters.
Method may call on_role_change callback if role is changing.
- Parameters
member – The member to follow
role – The desired role, normally ‘replica’, but could also be a ‘standby_leader’
timeout – start timeout, how long should the start() method wait for postgres accepting connections
do_reload – indicates that after updating postgresql.conf we just need to do a reload instead of restart
- Returns
True - if restart/reload were successfully performed, False - if restart/reload failed None - if nothing was done or if Postgres is still in starting state after timeout seconds.
-
get_major_version
() → int¶ Reads major version from PG_VERSION file
- Returns
major PostgreSQL version in integer format or 0 in case of missing file or errors
-
get_replication_connection_cursor
(host: Optional[str] = None, port: Union[int, str] = 5432, **kwargs: Any) → Iterator[Union[cursor, Cursor[Any]]]¶
-
property
global_config
¶
-
initdb
(*args: str, **kwargs: Any) → bool¶ Builds and executes the initdb command.
- Parameters
args – List of arguments to be joined into the initdb command.
kwargs – Keyword arguments to pass to
subprocess.call
.
- Returns
True
if the result ofsubprocess.call`, the exit code, is ``0
.
-
is_running
() → Optional[patroni.postgresql.postmaster.PostmasterProcess]¶ Returns PostmasterProcess if one is running on the data directory or None. If most recently seen process is running updates the cached process based on pid file.
-
latest_checkpoint_location
() → Optional[int]¶ Returns checkpoint location for the cleanly shut down primary. But, if we know that the checkpoint was written to the new WAL due to the archive_mode=on, we will return the LSN of prev wal record (SWITCH).
-
property
lsn_name
¶
-
property
major_version
¶
-
parse_wal_record
(timeline: str, lsn: str) → Union[Tuple[str, str, str, str], Tuple[None, None, None, None]]¶
-
property
pending_restart
¶
-
pg_ctl
(cmd: str, *args: str, **kwargs: Any) → bool¶ Builds and executes pg_ctl command
- Returns
!True when return_code == 0, otherwise !False
-
pg_isready
() → str¶ Runs pg_isready to see if PostgreSQL is accepting connections.
- Returns
‘ok’ if PostgreSQL is up, ‘reject’ if starting up, ‘no_resopnse’ if not up.
-
pg_stat_replication
() → List[Dict[str, Any]]¶ - Returns
a result set of ‘SELECT * FROM pg_stat_replication’.
-
pg_tblspc_realpaths
() → Dict[str, str]¶ Returns a dict containing the symlink (key) and target (values) for the tablespaces
-
pg_wal_realpath
() → Dict[str, str]¶ Returns a dict containing the symlink (key) and target (value) for the wal directory
-
pgcommand
(cmd: str) → str¶ Return path to the specified PostgreSQL command.
Note
If
postgresql.bin_name.*cmd*
was configured by the user then that binary name is used, otherwise the default binary name cmd is used.- Parameters
cmd – the Postgres binary name to get path to.
- Returns
path to Postgres binary named cmd.
-
promote
(wait_seconds: int, task: patroni.async_executor.CriticalTask, before_promote: Optional[Callable[[…], Any]] = None) → Optional[bool]¶
-
query
(sql: str, *params: Any, retry: bool = True) → List[Tuple[Any, …]]¶ Execute sql query with params and optionally return results.
- Parameters
sql – SQL statement to execute.
params – parameters to pass.
retry – whether the query should be retried upon failure or given up immediately.
- Returns
a query response as a list of tuples if there is any.
- Raises
Error
if had issues while executing sql.PostgresConnectionException
: if had issues while connecting to the database.RetryFailedError
: if it was detected that connection/query failed due to PostgreSQL restart or if retry deadline was exceeded.
-
replication_state
() → Optional[str]¶ Checks replication state from pg_stat_get_wal_receiver().
Note
Available only since 9.6
- Returns
streaming
,in archive recovery
, orNone
-
replication_state_from_parameters
(is_primary: bool, receiver_state: Optional[str], restore_command: Optional[str]) → Optional[str]¶ Figure out the replication state from input parameters.
Note
This method could be only called when Postgres is up, running and queries are successfuly executed.
- Is_primary
True is postgres is not running in recovery
- Receiver_state
value from pg_stat_get_wal_receiver.state or None if Postgres is older than 9.6
- Restore_command
value of
restore_command
GUC for PostgreSQL 12+ or postgresql.recovery_conf.restore_command if it is set in Patroni configuration- Returns
None for the primary and for Postgres older than 9.6;
’streaming’ if replica is streaming according to the pg_stat_wal_receiver view;
’in archive recovery’ if replica isn’t streaming and there is a restore_command
-
reset_cluster_info_state
(cluster: Optional[patroni.dcs.Cluster], nofailover: bool = False, global_config: Optional[GlobalConfig] = None) → None¶ Reset monitoring query cache.
It happens in the beginning of heart-beat loop and on change of synchronous_standby_names.
- Parameters
cluster – currently known cluster state from DCS
nofailover – whether this node could become a new primary. Important when there are logical permanent replication slots because “nofailover” node could do cascading replication and should enable hot_standby_feedback
global_config – last known
GlobalConfig
object
-
restart
(timeout: Optional[float] = None, task: Optional[patroni.async_executor.CriticalTask] = None, block_callbacks: bool = False, role: Optional[str] = None, before_shutdown: Optional[Callable[[…], Any]] = None, after_start: Optional[Callable[[…], Any]] = None) → Optional[bool]¶ Restarts PostgreSQL.
When timeout parameter is set the call will block either until PostgreSQL has started, failed to start or timeout arrives.
- Returns
True when restart was successful and timeout did not expire when waiting.
-
property
role
¶
-
schedule_sanity_checks_after_pause
() → None¶ After coming out of pause we have to: 1. configure server parameters if necessary 2. sync replication slots, because it might happen that slots were removed 3. get new ‘Database system identifier’ to make sure that it wasn’t changed
-
property
server_version
¶
-
start
(timeout: Optional[float] = None, task: Optional[patroni.async_executor.CriticalTask] = None, block_callbacks: bool = False, role: Optional[str] = None, after_start: Optional[Callable[[…], Any]] = None) → Optional[bool]¶ Start PostgreSQL
Waits for postmaster to open ports or terminate so pg_isready can be used to check startup completion or failure.
- Returns
True if start was initiated and postmaster ports are open, False if start failed, and None if postgres is still starting up
-
property
state
¶
-
stop
(mode: str = 'fast', block_callbacks: bool = False, checkpoint: Optional[bool] = None, on_safepoint: Optional[Callable[[…], Any]] = None, on_shutdown: Optional[Callable[[int], Any]] = None, before_shutdown: Optional[Callable[[…], Any]] = None, stop_timeout: Optional[int] = None) → bool¶ Stop PostgreSQL
Supports a callback when a safepoint is reached. A safepoint is when no user backend can return a successful commit to users. Currently this means we wait for user backends to close. But in the future alternate mechanisms could be added.
- Parameters
on_safepoint – This callback is called when no user backends are running.
on_shutdown – is called when pg_controldata starts reporting Database cluster state: shut down
before_shutdown – is called after running optional CHECKPOINT and before running pg_ctl stop
-
property
supports_multiple_sync
¶ True if Postgres version supports more than one synchronous node.
- Type
returns
-
property
sysid
¶
-
terminate_postmaster
(postmaster: patroni.postgresql.postmaster.PostmasterProcess, mode: str, stop_timeout: Optional[int]) → Optional[bool]¶
-
terminate_starting_postmaster
(postmaster: patroni.postgresql.postmaster.PostmasterProcess) → None¶ Terminates a postmaster that has not yet opened ports or possibly even written a pid file. Blocks until the process goes away.
-
wait_for_port_open
(postmaster: patroni.postgresql.postmaster.PostmasterProcess, timeout: float) → bool¶ Waits until PostgreSQL opens ports.
-
wait_for_startup
(timeout: float = 0) → Optional[bool]¶ Waits for PostgreSQL startup to complete or fail.
- Returns
True if start was successful, False otherwise
-
property
wal_dir
¶
-
property
wal_flush
¶ For PostgreSQL 9.6 onwards we want to use pg_current_wal_flush_lsn()/pg_current_xlog_flush_location().
-
property
wal_name
¶
-
-
patroni.postgresql.
null_context
()¶