Latest Developments in Data Quality Assessment for Risk Assessment
Graham Merrington, wca Consulting; Adam Ryan, International Zinc Association; Carolina Di Paolo, Dow; Lisa Nowell, US Government
Members of the SETAC community came together at a recent workshop in Copenhagen, Denmark, to define Criteria for Reporting and Evaluation of Exposure Data (CREED) as best practices for assessing the quality of environmental exposure data (measured concentrations in media) for diverse purposes. Once the CREED methodology is fully developed, it will be a key improvement in risk assessments and the management decisions supported by them.Participants in person and online discuss best practices for assessing environmental exposure data quality.
Individuals interested in participating in the CREED beta test should contact Irene Bramke for details. Beta testing is scheduled to begin on 30 June and end on 31 August.Environmental risk assessment is generally undertaken using two types of numerical values: One generated from exposure data and one generated from ecotoxicity hazard data. The quality of risk assessments is therefore strongly dependent on the quality of the datasets used to estimate exposure. Yet, while scientists and practitioners have developed frameworks to assess the reliability and relevance of ecotoxicity data for limit value derivation, there are no comparable comprehensive schemes or processes for exposure data evaluation. Recognizing that the evaluation of exposure datasets is not immune to the potential inconsistencies of expert judgment, workshop participants postulated that exposure datasets should be evaluated for both reliability and relevance, much like ecotoxicity data. Therefore, they developed a scheme for data users to assess exposure data in relation to both reliability and relevancy, and to communicate data usability criteria supporting expert judgment in a transparent manner. Assessment of reliability pertains to the inherent quality of the dataset and includes evaluation of the methods for sample collection, chemical analysis, and data processing steps. An understanding of the sampling design by which data were originally collected can help inform data processing decisions. As risk assessment practitioners, we recognize that chemical exposure data requirements will be different for different end uses or purposes. Chemical exposure data relevancy assessment clearly and unambiguously evaluates whether the dataset in question is appropriate for a specific end use or application. In other words, will the exposure data provide useful information for answering the questions posed for that application? Therefore, this relevancy assessment should consider various aspects of the study design by which the data were originally collected and processed to establish what landscape and conditions (e.g., type of hydrologic system, land uses, spatial and temporal distribution of samples, season of year, flow conditions) are represented by the dataset and whether the dataset is appropriate for the evaluator’s end use or purpose. Finally, the assessor should weigh the findings of both reliability and relevance assessments versus the original intended purpose and make the final decision on whether or not to proceed with data use accordingly. A preliminary generalized schematic of the CREED process is given below. The evaluation will be appropriate for assessing existing datasets for a specific purpose, but also for those planning chemical monitoring programs that have specific purposes or uses of the data in mind. It is intended that a report card will be produced at the end of the evaluation, appropriate to a specific analyte and purpose. While the CREED approach is currently a work in progress, we are eager to recruit volunteers to apply the CREED approach using defined example dataset(s). It is anticipated that feedback from such a beta testing effort will allow us to make refinements to the CREED approach prior to its finalization. Currently, the workshop organizers are actively recruiting beta testers. Individuals interested in participating in the CREED beta test should contact Irene Bramke for details. Beta testing is scheduled to begin on 30 June and end on 15 August. Beta testing participants will be expected to review up to three data use scenarios and all scores and feedback will be collected via an online survey form. It is anticipated that the time required to apply the CREED approach to these data use scenarios and complete the survey form will be less than three hours. The CREED developers are planning to hold a webinar describing the CREED approach at 8:30 p.m. CET on 21 June. We encourage you to get involved. This is an opportunity for you to contribute to refinement of the CREED approach before it becomes a widely used data usability evaluation method.
Thank you to the participants for their time and energy: Brad Clark, Michelle Hladik, Heinz Rudel, Arjan Markus, Stefano Polesello, Wei Shi, Iain Wilson, Adam Peters, Claus Svendsen, Leonard Oste, Andrew Harford, Mick Hamer, Mike Beking, Steve Lofts, Rochelle Bohaty, Thomas Backhaus, Derek Wallace, Jeanne Vuaille, Adam Ryan, Carolina Di Paolo, Caroline Whalley, Irene Bramke, Jenny Stauber, Ryan Otter, Yves Verhaegen, Zhanyun Wang, Drew Sazbo, Lisa Nowell, Chuck Peck, Graham Merrington.
Financial support of the workshop was provided by Concawe, GSK, Metals Environmental Research Association (MERA), SETAC Journals (ET&C and IEAM), Syngenta and Unilever.
Author's Contact: [email protected]