Connection Pooling¶
The PoolManager
class provides top-level
access to the queries pooling mechanism, managing pools of connections by DSN in
instances of the Pool
class. The connections are
represented by instances of the Connection
class. Connection
holds the psycopg2
connection handle as well as lock information that lets the Pool and PoolManager
know when connections are busy.
These classes are managed automatically by the Session
and should rarely be interacted with directly.
If you would like to use the PoolManager
to shutdown all connections to PostgreSQL, either reference it by class or using
the PoolManager.instance
method.
-
class
queries.pool.
PoolManager
[source]¶ The connection pool object implements behavior around connections and their use in queries.Session objects.
We carry a pool id instead of the connection URI so that we will not be carrying the URI in memory, creating a possible security issue.
-
classmethod
add
(pid, connection)[source]¶ Add a new connection and session to a pool.
Parameters: - pid (str) – The pool id
- connection (psycopg2.extensions.connection) – The connection to add to the pool
-
classmethod
clean
(pid)[source]¶ Clean the specified pool, removing any closed connections or stale locks.
Parameters: pid (str) – The pool id to clean
-
classmethod
create
(pid, idle_ttl=60, max_size=1, time_method=None)[source]¶ Create a new pool, with the ability to pass in values to override the default idle TTL and the default maximum size.
A pool’s idle TTL defines the amount of time that a pool can be open without any sessions before it is removed.
A pool’s max size defines the maximum number of connections that can be added to the pool to prevent unbounded open connections.
Parameters: - pid (str) – The pool ID
- idle_ttl (int) – Time in seconds for the idle TTL
- max_size (int) – The maximum pool size
- time_method (callable) – Override the use of
time.time()
method for time values.
Raises: KeyError
-
classmethod
free
(pid, connection)[source]¶ Free a connection that was locked by a session
Parameters: - pid (str) – The pool ID
- connection (psycopg2.extensions.connection) – The connection to remove
-
classmethod
get
(pid, session)[source]¶ Get an idle, unused connection from the pool. Once a connection has been retrieved, it will be marked as in-use until it is freed.
Parameters: - pid (str) – The pool ID
- session (queries.Session) – The session to assign to the connection
Return type:
-
classmethod
get_connection
(pid, connection)[source]¶ Return the specified
Connection
from the pool.Parameters: - pid (str) – The pool ID
- connection (psycopg2.extensions.connection) – The connection to return for
Return type:
-
classmethod
has_connection
(pid, connection)[source]¶ Check to see if a pool has the specified connection
Parameters: - pid (str) – The pool ID
- connection (psycopg2.extensions.connection) – The connection to check for
Return type: bool
-
classmethod
has_idle_connection
(pid)[source]¶ Check to see if a pool has an idle connection
Parameters: pid (str) – The pool ID Return type: bool
-
classmethod
instance
()[source]¶ Only allow a single PoolManager instance to exist, returning the handle for it.
Return type: PoolManager
-
classmethod
is_full
(pid)[source]¶ Return a bool indicating if the specified pool is full
Parameters: pid (str) – The pool id Return type: bool
-
classmethod
lock
(pid, connection, session)[source]¶ Explicitly lock the specified connection in the pool
Parameters: - pid (str) – The pool id
- connection (psycopg2.extensions.connection) – The connection to add to the pool
- session (queries.Session) – The session to hold the lock
-
classmethod
remove
(pid)[source]¶ Remove a pool, closing all connections
Parameters: pid (str) – The pool ID
-
classmethod
remove_connection
(pid, connection)[source]¶ Remove a connection from the pool, closing it if is open.
Parameters: - pid (str) – The pool ID
- connection (psycopg2.extensions.connection) – The connection to remove
Raises: ConnectionNotFoundError
-
classmethod
set_idle_ttl
(pid, ttl)[source]¶ Set the idle TTL for a pool, after which it will be destroyed.
Parameters: - pid (str) – The pool id
- ttl (int) – The TTL for an idle pool
-
classmethod
-
class
queries.pool.
Pool
(pool_id, idle_ttl=60, max_size=1, time_method=None)[source]¶ A connection pool for gaining access to and managing connections
-
add
(connection)[source]¶ Add a new connection to the pool
Parameters: connection (psycopg2.extensions.connection) – The connection to add to the pool Raises: PoolFullError
-
busy_connections
¶ Return a list of active/busy connections
Return type: list
-
clean
()[source]¶ Clean the pool by removing any closed connections and if the pool’s idle has exceeded its idle TTL, remove all connections.
-
closed_connections
¶ Return a list of closed connections
Return type: list
-
connection_handle
(connection)[source]¶ Return a connection object for the given psycopg2 connection
Parameters: connection (psycopg2.extensions.connection) – The connection to return a parent for Return type: Connection
-
executing_connections
¶ Return a list of connections actively executing queries
Return type: list
-
free
(connection)[source]¶ Free the connection from use by the session that was using it.
Parameters: connection (psycopg2.extensions.connection) – The connection to free Raises: ConnectionNotFoundError
-
get
(session)[source]¶ Return an idle connection and assign the session to the connection
Parameters: session (queries.Session) – The session to assign Return type: psycopg2.extensions.connection Raises: NoIdleConnectionsError
-
id
¶ Return the ID for this pool
Return type: str
-
idle_connections
¶ Return a list of idle connections
Return type: list
-
idle_duration
¶ Return the number of seconds that the pool has had no active connections.
Return type: float
-
is_full
¶ Return True if there are no more open slots for connections.
Return type: bool
-
lock
(connection, session)[source]¶ Explicitly lock the specified connection
Parameters: - connection (psycopg2.extensions.connection) – The connection to lock
- session (queries.Session) – The session to hold the lock
-
locked_connections
¶ Return a list of all locked connections
Return type: list
-
remove
(connection)[source]¶ Remove the connection from the pool
Parameters: connection (psycopg2.extensions.connection) – The connection to remove Raises: ConnectionNotFoundError Raises: ConnectionBusyError
-
-
class
queries.pool.
Connection
(handle)[source]¶ Contains the handle to the connection, the current state of the connection and methods for manipulating the state of the connection.
-
busy
¶ Return if the connection is currently executing a query or is locked by a session that still exists.
Return type: bool
-
closed
¶ Return if the psycopg2 connection is closed.
Return type: bool
-
executing
¶ Return if the connection is currently executing a query
Return type: bool
-
free
()[source]¶ Remove the lock on the connection if the connection is not active
Raises: ConnectionBusyError
-
id
¶ Return id of the psycopg2 connection object
Return type: int
-
lock
(session)[source]¶ Lock the connection, ensuring that it is not busy and storing a weakref for the session.
Parameters: session (queries.Session) – The session to lock the connection with Raises: ConnectionBusyError
-
locked
¶ Return if the connection is currently exclusively locked
Return type: bool
-