Most numerical models are run and compared to data in a subjective
manner. This paper demonstrates how model performance statistics can be
used to calibrate and/or validate hydrodynamic models in a more
objective way. Statistics were also used to compare model runs that used
different amounts of field data in order to inform the debate about the
optimum mix of modelling and measurement.
The hydrodynamics around the mouth of the Teign estuary (UK) have been
simulated using two coastal area numerical modelling systems. Model
performance statistics were calculated to assess the accuracy of the
predictions of the measured currents at a number of locations around the
estuary mouth. The relative mean absolute error was used as it is
applicable to vectors as well as scalars and measures all types of
errors. An adjusted relative mean absolute error was also used to reduce
the effect of measurement error. A classification table was adopted that
categorises the results according to the size of the error. In addition,
time series and scatter plots were used to judge the performance of the
modelling systems.
Calm conditions during a spring tide were simulated, as was a relatively
large storm. The two modelling systems gave more or less equal
performances when run in engineering mode (where default values were
used for most of the system settings). In each case, one modelling
system performed better than the other at some locations and worse than
it at other locations. One model was also run using a scientific
approach, where different amounts of information were used to alter the
model settings and sensitivity tests were performed. The model
performance statistics showed that using more data does not necessarily
lead to better model predictions. New methods for incorporating data
into the operation of a model need to be evaluated thoroughly before
they can be used without site-specific calibration.@en