e15190.runlog.downloader.MySqlDownloader¶
- class e15190.runlog.downloader.MySqlDownloader(auto_connect=False, verbose=True)¶
This class downloads the MySQL database from WMU.
Due to some in-house security measures, the MySQL database is stored at Western Michigan University. To analyze the data, we download the database from WMU and store it locally at NSCL/FRIB’s server. All tables are downloaded as pandas dataframes, and stored in an HDF files. This allows quicker access to the data and more complicated analysis.
It is encouraged to use the
withstatement when interacting with this class. Here is an example:>>> from e15190.runlog import downloader >>> with downloader.MySqlDownloader(auto_connect=True) as dl: >>> df = dl.get_table('runtarget')
Functions
__init__([auto_connect, verbose])Constructor for
MySqlDownloader.connect([verbose])Establish connection to the MySQL database.
decorate(func_that_returns_tuples_of_tuples)Converts a function that returns tuples of tuples into a function that returns tuples of lists.
disconnect([verbose])Disconnect from the MySQL database.
download([download_path, table_names, ...])Convert tables into pandas dataframes and save into an SQLite3 file
Returns all table names in the MySQL database.
get_table(table_name)Returns the table as a pandas dataframe.
Properties
CREDENTIAL_PATHDOWNLOAD_PATH- __init__(auto_connect=False, verbose=True)¶
Constructor for
MySqlDownloader.- Parameters
auto_connect (bool, default False) – Whether to automatically connect to the MySQL database. Login credentials are needed to connect to the database. If you are using context manager, this parameter is irrelevant — connection is always being established.
verbose (bool, default True) – Whether to print the progress of connecting to the MySQL database. This setting has less priority than individual class functions, i.e. if other functions have explicitly set a verbose value, then this global setting will be ignored.
- static decorate(func_that_returns_tuples_of_tuples)¶
Converts a function that returns tuples of tuples into a function that returns tuples of lists.
This is primarily used to decorate
MySqlDownloader.cursor.fetchall().- Parameters
func_that_returns_tuples_of_tuples (function) – Function that returns tuples of tuples, e.g.
cursor.fetchall().- Returns
deco_func – Function that returns a list of lists.
- Return type
function
- connect(verbose=None)¶
Establish connection to the MySQL database.
Upon successful connection,
self.connectionis set to the connection object andself.cursoris set to the cursor object. Fetch functions ofself.cursorare decorated to return lists of lists, includingfetchall,fetchoneandfetchmany.- Parameters
verbose (bool, default None) – Whether to print the progress of connecting to the MySQL database. If
None, the global setting is used.- Raises
FileNotFoundError – If the key file is not found.
- get_all_table_names()¶
Returns all table names in the MySQL database.
- Returns
table_names – List of all table names in the MySQL database.
- Return type
list
- get_table(table_name)¶
Returns the table as a pandas dataframe.
- Parameters
table_name (str) – Name of the table to be downloaded.
- Returns
table – Table as a pandas dataframe.
- Return type
pandas.DataFrame
- Raises
ValueError – If the table is not found.
- download(download_path=None, table_names=None, auto_disconnect=False, verbose=True)¶
Convert tables into pandas dataframes and save into an SQLite3 file
- Parameters
download_path (str, default None) – File path to the HDF file. If
None, the file is saved at$DATABASE_DIR/runlog/downloads/mysql_database.db.auto_disconnect (bool, default False) – Whether to automatically disconnect from the MySQL database after all tables have been downloaded.
table_names (list of str, default None) – List of table names to download. If
None, all tables are downloaded.verbose (bool, default True) – Whether to print the progress of downloading. If
None, the global setting is used.
- Returns
download_path – File path to the SQLite3 file.
- Return type
pathlib.Path
- disconnect(verbose=None)¶
Disconnect from the MySQL database.
- Parameters
verbose (bool, default None) – Whether to print the progress of disconnecting. If
None, the global setting is used.