Performance of models

Climate models provide projections of future climate changes derived from fundamental principles of the climate system. One advantage of this approach is that future changes in all climate variables are derived together in a physically consistent way. One disadvantage is that models cannot perfectly represent the Earth’s climate – some regions of the modelled climate may be slightly hotter or drier, for example, than the climate we experience in reality. These differences are known as error or bias.

Despite climate modelling representing the best available information on plausible future climates, no climate model provides a perfect representation of the real climate system. For this reason there are differences between the output of the NARCliM simulations and observations of the climate1.

The performance of the global climate models (GCMs) and regional climate models (RCMs) selected for NARCliM are discussed below, and preliminary findings on the performance of the NARCliM modelling presented.

Performance of RCM simulations driven by NCEP re-analysis

NCEP re-analysis driven RCM simulations for the 1950-2009 period were validated based on gridded 5km resolution minimum and maximum temperature and mean rainfall data developed under the Australian Water Availability Project (AWAP)2. Results show that regional climate downscaling improves the simulation of precipitation and minimum and maximum temperature. Moreover, preliminary results indicate that the RCMs improve model performance in accounting for the influence of major climate drivers such as El Nino–Southern Oscillation.

Performance of GCM-driven RCM simulations compared to GCMs only

Despite selection of GCMs based on performance, it is notable that a significant spatial scale problem exists between the scale of the processes that GCMs can represent (larger than about 300km) and the scales of interest for impacts and adaptation studies which are often only tens of kilometres or less.

Dynamic downscaling with RCMs was found to capture climate phenomena not resolved by GCMs including the influence of mountains, coastlines and local land-atmosphere feedbacks. Results indicate that the NARCliM simulations more adequately resolve spatial and temporal variations in the NSW climate, and particularly in the climate of the eastern seaboard.

Why is bias correction done?

Climate models are not perfect and often display errors or biases in their modelled climates. For some applications these biases may present significant difficulties when attempting to use the projected climates, particularly if the impact or application is sensitive to non-linearities in the system such as threshold effects (e.g. the number of days with temperatures above 35°C).

The chosen RCMs were found to act independently but do have a tendency towards overestimating precipitation and underestimating maximum temperature. Therefore, in addition to making available the actual NARCliM model output data set, statistical methods were used to correct NARCliM model bias and produce a bias-corrected data set. Only rainfall and temperature (minimum and maximum) were bias corrected as they are the only variables with long, reliable observational time series on which to base the bias correction.

The method used to produce bias-corrected data is outlined in Tech Note 33. Given that bias correction removes the physical consistency between climate variables, this technical noted also present guidelines to consider when deciding whether to use actual model output or bias-corrected data.

How are the quality of NARCliM projections being evaluated?

Initially a technical quality assurance process was undertaken to identify any issues. This involved checks on the size, number, inner structure and format of model output files, metadata quality assurance, and checks on geographical coordinates and time steps.

Basic scientific quality assurance was subsequently conducted for post-processed climate variables to check that these variables fall within a reasonable range, with the range selected based on the observation record4.

Intermediate scientific quality assurance has been undertaken for key variables such as rainfall, temperature and wind with model projections compared to observational estimates and errors quantified using various statistics such as mean bias.

Advanced scientific quality assurance is also being conducted for some special climate systems, with advanced diagnostics being applied to test how well the NARCliM simulations account for such systems (e.g. East Coast Lows)5. The degree to which simulations reproduce the main drivers of NSW climate is also under investigation.

Results from the scientific quality assurance assessments will be published by project researchers in the coming year.

Footnotes

  1. Olson, R., J. P. Evans, D. Arg¨ueso and A. Di Luca, 2014: NARCliM Climatological Atlas. NARCliM Technical Note 4, 417 pp, NARCliM Consortium, Sydney, Australia. Technical Report 4 NARCliM Climatological Atlas [PDF 53.1MB]
  2. Jones, D. A., Wang, W. and Fawcett, R. 2009. High-quality spatial climate data-sets for Australia. Australian Meteorological and Oceanographic Journal, 58, 233-248.
  3. Evans, J.P. and D. Argüeso, 2014: Guidance on the use of bias corrected data. NARCliM Technical Note 3, 7pp., NARCliM Consortium, Sydney, Australia. http://www.ccrc.unsw.edu.au/sites/default/files/NARCliM/publications/TechNote3.pdf
  4. Ji, F., Evans, J.P, Argueso D. and Di Luca A., NARCliM Data Quality Assurance Report, Report compiled by the Office of Environment and Heritage and the University of NSW.
  5. Ji F., Evans J.P., Argueso D., Fita L., and Di Luca A. “Using Large-scale Diagnostic Quantities to Investigate Change in East Coast Lows”, submitted to Climate Dynamics.