e15190.runlog.downloader.MySqlDownloader

class e15190.runlog.downloader.MySqlDownloader(auto_connect=False, verbose=True)

This class downloads the MySQL database from WMU.

Due to some in-house security measures, the MySQL database is stored at Western Michigan University. To analyze the data, we download the database from WMU and store it locally at NSCL/FRIB’s server. All tables are downloaded as pandas dataframes, and stored in an HDF files. This allows quicker access to the data and more complicated analysis.

It is encouraged to use the with statement when interacting with this class. Here is an example:

>>> from e15190.runlog import downloader
>>> with downloader.MySqlDownloader(auto_connect=True) as dl:
>>>     df = dl.get_table('runtarget')

Functions

__init__([auto_connect, verbose])

Constructor for MySqlDownloader.

connect([verbose])

Establish connection to the MySQL database.

decorate(func_that_returns_tuples_of_tuples)

Converts a function that returns tuples of tuples into a function that returns tuples of lists.

disconnect([verbose])

Disconnect from the MySQL database.

download([download_path, table_names, ...])

Convert tables into pandas dataframes and save into an SQLite3 file

get_all_table_names()

Returns all table names in the MySQL database.

get_table(table_name)

Returns the table as a pandas dataframe.

Properties

CREDENTIAL_PATH

DOWNLOAD_PATH

__init__(auto_connect=False, verbose=True)

Constructor for MySqlDownloader.

Parameters
  • auto_connect (bool, default False) – Whether to automatically connect to the MySQL database. Login credentials are needed to connect to the database. If you are using context manager, this parameter is irrelevant — connection is always being established.

  • verbose (bool, default True) – Whether to print the progress of connecting to the MySQL database. This setting has less priority than individual class functions, i.e. if other functions have explicitly set a verbose value, then this global setting will be ignored.

static decorate(func_that_returns_tuples_of_tuples)

Converts a function that returns tuples of tuples into a function that returns tuples of lists.

This is primarily used to decorate MySqlDownloader.cursor.fetchall().

Parameters

func_that_returns_tuples_of_tuples (function) – Function that returns tuples of tuples, e.g. cursor.fetchall().

Returns

deco_func – Function that returns a list of lists.

Return type

function

connect(verbose=None)

Establish connection to the MySQL database.

Upon successful connection, self.connection is set to the connection object and self.cursor is set to the cursor object. Fetch functions of self.cursor are decorated to return lists of lists, including fetchall, fetchone and fetchmany.

Parameters

verbose (bool, default None) – Whether to print the progress of connecting to the MySQL database. If None, the global setting is used.

Raises

FileNotFoundError – If the key file is not found.

get_all_table_names()

Returns all table names in the MySQL database.

Returns

table_names – List of all table names in the MySQL database.

Return type

list

get_table(table_name)

Returns the table as a pandas dataframe.

Parameters

table_name (str) – Name of the table to be downloaded.

Returns

table – Table as a pandas dataframe.

Return type

pandas.DataFrame

Raises

ValueError – If the table is not found.

download(download_path=None, table_names=None, auto_disconnect=False, verbose=True)

Convert tables into pandas dataframes and save into an SQLite3 file

Parameters
  • download_path (str, default None) – File path to the HDF file. If None, the file is saved at $DATABASE_DIR/runlog/downloads/mysql_database.db.

  • auto_disconnect (bool, default False) – Whether to automatically disconnect from the MySQL database after all tables have been downloaded.

  • table_names (list of str, default None) – List of table names to download. If None, all tables are downloaded.

  • verbose (bool, default True) – Whether to print the progress of downloading. If None, the global setting is used.

Returns

download_path – File path to the SQLite3 file.

Return type

pathlib.Path

disconnect(verbose=None)

Disconnect from the MySQL database.

Parameters

verbose (bool, default None) – Whether to print the progress of disconnecting. If None, the global setting is used.