tokio.connectors.cachingdb module¶
This module provides generic infrastructure for retrieving data from a relational database that contains immutable data. It can use a local caching database (sqlite3) to allow for reanalysis on platforms that cannot access the original remote database or to reduce the load on remote databases.
-
class
tokio.connectors.cachingdb.
CachingDb
(dbhost=None, dbuser=None, dbpassword=None, dbname=None, cache_file=None)[source]¶ Bases:
object
Connect relational database with an optional caching layer interposed.
-
__init__
(dbhost=None, dbuser=None, dbpassword=None, dbname=None, cache_file=None)[source]¶ Connect to a relational database.
If instantiated with a cache_file argument, all queries will go to that SQLite-based cache database. If this class is not instantiated with a cache_file argument, all queries will go out to the remote database.
If none of the connection arguments (
db*
) are specified, do not connect to a remote database and instead rely entirely on the caching database or a separate call to theconnect()
method.Parameters: - dbhost (str, optional) – hostname for the remote database
- dbuser (str, optional) – username to use when connecting to database
- dbpassword (str, optional) – password for authenticating to database
- dbname (str, optional) – name of database to use when connecting
- cache_file (str, optional) – Path to an SQLite3 database to use as a caching layer.
Variables: - saved_results (dict) – in-memory data cache, keyed by table names
and whose values are dictionaries with keys
rows
andschema
.rows
are a list of row tuples returned from earlier queries, andschema
is the SQL statement required to create the table corresponding torows
. - last_hit (int) – a flag to indicate whether the last query was found in the caching database or the remote database
- cache_file (str) – path to the caching database’s file
- cache_db (sqlite3.Connection) – caching database connection handle
- cache_db_ps (str) – paramstyle of the caching database as defined by PEP-0249
- remote_db – remote database connection handle
- remote_db_ps (str) – paramstyle of the remote database as defined by PEP-0249
-
_query_mysql
(query_str, query_variables)[source]¶ Run a query against the MySQL database and return the full output.
Parameters:
-
_query_sqlite3
(query_str, query_variables)[source]¶ Run a query against the cache database and return the full output.
Parameters:
-
close
()[source]¶ Destroy connection objects.
Close the remote database connection handler and reset state of remote connection attributes.
-
connect
(dbhost, dbuser, dbpassword, dbname)[source]¶ Establish remote db connection.
Connects to a remote MySQL database and defines the connection handler and paramstyle attributes.
Parameters:
-
connect_cache
(cache_file)[source]¶ Open the cache database file and set the handler attribute.
Parameters: cache_file (str) – Path to the SQLite3 caching database file to be used.
-
drop_cache
(tables=None)[source]¶ Flush saved results from memory.
If tables are specified, only drop those tables’ results. If no tables are provided, flush everything.
Parameters: tables (list, optional) – List of table names (str) to flush. If omitted, flush all tables in cache.
-
query
(query_str, query_variables=(), table=None, table_schema=None)[source]¶ Pass a query through all layers of cache and return on the first hit.
If a table is specified, the results of this query can be saved to the cache db into a table of that name.
Parameters: - query_str (str) – SQL query expressed as a string
- query_variables (tuple) – parameters to be substituted into query_str if query_str is a parameterized query
- table (str, optional) – name of table in the cache database to save the results of the query
- table_schema (str, optional) – when table is specified, the SQL line to initialize the table in which the query results will be cached.
Returns: Tuple of tuples corresponding to rows of fields as returned by the SQL query.
Return type:
-
save_cache
(cache_file)[source]¶ Commit the in-memory cache to a cache database.
This method is currently very memory-inefficient and not good for caching giant pieces of a database without something wrapping it to feed it smaller pieces.
Note
This manipulates the
cache_db*
attributes in a dirty way to prevent closing and re-opening the original cache db. If theself.open_cache()
is ever changed to include tracking more state, this function must also be updated to retain that state while the old cache db state is being temporarily shuffled out.Parameters: cache_file (str) – Path to the cache file to be used to write out the cache contents. This file will temporarily pre-empt the cache_file attribute and should be a different file.
-
-
tokio.connectors.cachingdb.
get_paramstyle_symbol
(paramstyle)[source]¶ Infer the correct paramstyle for a database.paramstyle
Provides a generic way to determine the paramstyle of a database connection handle. See PEP-0249 for more information.
Parameters: paramstyle (str) – Result of a generic database handler’s paramstyle attribute Returns: The string corresponding to the paramstyle of the given database connection. Return type: str