e15190.runlog.data_cleansing.MySqlCleanser

class e15190.runlog.data_cleansing.MySqlCleanser(verbose=True)

This is a class of methods to perform data cleansing on the tables freshly downloaded from MySQL at WMU.

Functions

__init__([verbose])

cleanse([verbose])

cleanse_save_runscalers([runs, runs_per_chunk])

save([force, verbose])

Properties

INPUT_PATH

OUTPUT_PATH

__init__(verbose=True)
_cleanse_runlog()

[INCOMPLETE] Perform data cleansing on runlog.

The runlog table from the WMU MySQL database contains many entries. A valid run in the runlog is composed of a pair of entries that satisfy all the following conditions: - Run number either between 2000 - 3000 or 4000 - 5000. - Exactly one entry has a state of begin, and another entry has a state of end. - The end state must happen after the begin state (compare date). - Both titles must be identical.

Any invalid runs will be discarded after data cleansing, except for entries that have event as their host. All event entries will be kept.

**Remark: Runs that are invalid are not all trash. But for our purpose of data analysis, we do not care about them. On the other hand, not all valid runs contain useful physics. There is no check on whether the detectors are working properly, the runs are electronics are set correctly, etc. All these are not being taken care in this stage of data cleansing. We leave the responsibility to analyses after this.