Data Quality

The GMOS network produces data coming in near real-time from a large number of sources. In order to ensure data reliability and comparability, a significant effort has been made to implement a system designed to quality assure and quality control of incoming datasets. This system, GMOS-Data Quality Management (G-DQM), uses a web based approach with real-time adaptive monitoring procedures aimed at preventing the production of poor-quality data. G-DQM is plugged on the GMOS Cyberinfrastructure, in order to exploit the features of this framework.

The data process for quality is described in the figure: after data acquisition, datasets are processed for quality assurance using a workflow where each step is responsabile of a data filtering phase.

In step one (1), G-DQM runs an automated process that filters the raw data stored in GMOS databases. The system compares the dataset against some flags corresponding to quality criteria. The flags are grouped in three sets: valid, warning and invalid. 

The second step (2) consists in a instrument reports compiled by site operators during their visits to stations. Field notes, anomalies, routine controls and part changes are reported in the station e-logbook, which is provided as a different service by means of a web application integrated into the GMOS Cyberinfrastructure. GMOS SOPs are fully integrated into the e-logbook, which also serves as a reminder for routine maintenance.

The third step (3) requires the site operator's approval of the intermediate flagged dataset. Site operators are allowed to clarify data records prior to their full approval. At the end of the above processes the system outputs are fully QAed/QCed.

A further process (4) computes GOM and PBM concentrations for those sites that are performing speciation. After step (4), measurements tagged as invalid are tossed and only the valid data will be considered available for dissemination purposes.

Step (5) thus stores the final valid datasets that will be accessible from the GMOS web portal for Dissemination purposes.


Up