Connection Pooling

The PoolManager class provides top-level access to the queries pooling mechanism, managing pools of connections by DSN in instances of the Pool class. The connections are represented by instances of the Connection class. Connection holds the psycopg2 connection handle as well as lock information that lets the Pool and PoolManager know when connections are busy.

These classes are managed automatically by the Session and should rarely be interacted with directly.

If you would like to use the PoolManager to shutdown all connections to PostgreSQL, either reference it by class or using the PoolManager.instance method.

class queries.pool.PoolManager[source]

The connection pool object implements behavior around connections and their use in queries.Session objects.

We carry a pool id instead of the connection URI so that we will not be carrying the URI in memory, creating a possible security issue.

classmethod add(pid, connection)[source]

Add a new connection and session to a pool.

Parameters:
classmethod clean(pid)[source]

Clean the specified pool, removing any closed connections or stale locks.

Parameters:pid (str) – The pool id to clean
classmethod create(pid, idle_ttl=60, max_size=1, time_method=None)[source]

Create a new pool, with the ability to pass in values to override the default idle TTL and the default maximum size.

A pool’s idle TTL defines the amount of time that a pool can be open without any sessions before it is removed.

A pool’s max size defines the maximum number of connections that can be added to the pool to prevent unbounded open connections.

Parameters:
  • pid (str) – The pool ID
  • idle_ttl (int) – Time in seconds for the idle TTL
  • max_size (int) – The maximum pool size
  • time_method (callable) – Override the use of time.time() method for time values.
Raises:

KeyError

classmethod free(pid, connection)[source]

Free a connection that was locked by a session

Parameters:
classmethod get(pid, session)[source]

Get an idle, unused connection from the pool. Once a connection has been retrieved, it will be marked as in-use until it is freed.

Parameters:
  • pid (str) – The pool ID
  • session (queries.Session) – The session to assign to the connection
Return type:

psycopg2.extensions.connection

classmethod get_connection(pid, connection)[source]

Return the specified Connection from the pool.

Parameters:
Return type:

queries.pool.Connection

classmethod has_connection(pid, connection)[source]

Check to see if a pool has the specified connection

Parameters:
Return type:

bool

classmethod has_idle_connection(pid)[source]

Check to see if a pool has an idle connection

Parameters:pid (str) – The pool ID
Return type:bool
classmethod instance()[source]

Only allow a single PoolManager instance to exist, returning the handle for it.

Return type:PoolManager
classmethod is_full(pid)[source]

Return a bool indicating if the specified pool is full

Parameters:pid (str) – The pool id
Return type:bool
classmethod lock(pid, connection, session)[source]

Explicitly lock the specified connection in the pool

Parameters:
classmethod remove(pid)[source]

Remove a pool, closing all connections

Parameters:pid (str) – The pool ID
classmethod remove_connection(pid, connection)[source]

Remove a connection from the pool, closing it if is open.

Parameters:
Raises:

ConnectionNotFoundError

classmethod report()[source]

Return the state of the all of the registered pools.

Return type:dict
classmethod set_idle_ttl(pid, ttl)[source]

Set the idle TTL for a pool, after which it will be destroyed.

Parameters:
  • pid (str) – The pool id
  • ttl (int) – The TTL for an idle pool
classmethod set_max_size(pid, size)[source]

Set the maximum number of connections for the specified pool

Parameters:
  • pid (str) – The pool to set the size for
  • size (int) – The maximum number of connections
classmethod shutdown()[source]

Close all connections on in all pools

classmethod size(pid)[source]

Return the number of connections in the pool

Parameters:pid (str) – The pool id

:rtype int

class queries.pool.Pool(pool_id, idle_ttl=60, max_size=1, time_method=None)[source]

A connection pool for gaining access to and managing connections

add(connection)[source]

Add a new connection to the pool

Parameters:connection (psycopg2.extensions.connection) – The connection to add to the pool
Raises:PoolFullError
busy_connections

Return a list of active/busy connections

Return type:list
clean()[source]

Clean the pool by removing any closed connections and if the pool’s idle has exceeded its idle TTL, remove all connections.

close()[source]

Close the pool by closing and removing all of the connections

closed_connections

Return a list of closed connections

Return type:list
connection_handle(connection)[source]

Return a connection object for the given psycopg2 connection

Parameters:connection (psycopg2.extensions.connection) – The connection to return a parent for
Return type:Connection
executing_connections

Return a list of connections actively executing queries

Return type:list
free(connection)[source]

Free the connection from use by the session that was using it.

Parameters:connection (psycopg2.extensions.connection) – The connection to free
Raises:ConnectionNotFoundError
get(session)[source]

Return an idle connection and assign the session to the connection

Parameters:session (queries.Session) – The session to assign
Return type:psycopg2.extensions.connection
Raises:NoIdleConnectionsError
id

Return the ID for this pool

Return type:str
idle_connections

Return a list of idle connections

Return type:list
idle_duration

Return the number of seconds that the pool has had no active connections.

Return type:float
is_full

Return True if there are no more open slots for connections.

Return type:bool
lock(connection, session)[source]

Explicitly lock the specified connection

Parameters:
locked_connections

Return a list of all locked connections

Return type:list
remove(connection)[source]

Remove the connection from the pool

Parameters:connection (psycopg2.extensions.connection) – The connection to remove
Raises:ConnectionNotFoundError
Raises:ConnectionBusyError
report()[source]

Return a report about the pool state and configuration.

Return type:dict
set_idle_ttl(ttl)[source]

Set the idle ttl

Parameters:ttl (int) – The TTL when idle
set_max_size(size)[source]

Set the maximum number of connections

Parameters:size (int) – The maximum number of connections
shutdown()[source]

Forcefully shutdown the entire pool, closing all non-executing connections.

Raises:ConnectionBusyError
class queries.pool.Connection(handle)[source]

Contains the handle to the connection, the current state of the connection and methods for manipulating the state of the connection.

busy

Return if the connection is currently executing a query or is locked by a session that still exists.

Return type:bool
close()[source]

Close the connection

Raises:ConnectionBusyError
closed

Return if the psycopg2 connection is closed.

Return type:bool
executing

Return if the connection is currently executing a query

Return type:bool
free()[source]

Remove the lock on the connection if the connection is not active

Raises:ConnectionBusyError
id

Return id of the psycopg2 connection object

Return type:int
lock(session)[source]

Lock the connection, ensuring that it is not busy and storing a weakref for the session.

Parameters:session (queries.Session) – The session to lock the connection with
Raises:ConnectionBusyError
locked

Return if the connection is currently exclusively locked

Return type:bool