In the era of big data, one may often need to quantitatively measure the agreement between two multi-dimensional datasets, which could either be a vector field or a combination of multiple scalar and vector fields. To meet this need, a Multivariable Integrated Evaluation (MVIE) method was proposed and applied to climate model evaluation. With this method, one can quantitatively evaluate the overall performance of a climate model in simulating multiple variables, e.g., air temperature, precipitation, and vector wind, against observed ones.
Recently, an article, published in
Geoscientific Model Development, improved the MVIE method and developed a simple-to-use Multivariable Integrated Evaluation Tool (MVIETool) coded with Python/NCL to facilitate climate model evaluation and models inter-comparison.
"The improved MVIE method can provide a more comprehensive and precise evaluation of climate model performance. With the support of the MVIETool, one can easily evaluate model performance in terms of each individual variable and/or multiple variables." says Ms. ZHANG Meng-Zhuo, the first author of the paper from the School of Atmospheric Sciences, Nanjing University, China.
According to the article, the key improvements to the MVIE method include the following three aspects: (1) Area-weighting is taken into the definition of statistics in MVIE, which makes the evaluation results of spatial fields more accurate. (2) The improved MVIE allows a mixed evaluation of scalar and vector fields. (3) A multivariable integrated skill score is proposed as a flexible and normalized index to quantitatively measure a model's ability to simulate multiple fields.
A schematic chart illustrates climate model evaluation with the MVIETooL. (Image by ZHANG Meng-Zhuo)
"In addition to climate model evaluation, the improved MVIE method may also be applied to other areas, e.g., machine learning. One may use the MVIE method to measure the overall accuracy of multiple variables generated by a machine learning model relative to the target values," says Prof. XU Zhongfeng from the Institute of Atmospheric Physics, Chinese Academy of Sciences. He is also the corresponding author of the paper.
References
Zhang Meng-Zhuo, Zhongfeng Xu*, Ying Han, and Weidong Guo, 2021: An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison, Geosci. Model Dev., 14, 3079–3094, https://doi.org/10.5194/gmd-14-3079-2021
