The fractions skill score for ensemble forecast verification

The Fractions Skill Score (FSS) is a neighbourhood verification method originally designed to verify deterministic forecasts of binary events. Previous studies employed different approaches to computing an ensemble-based FSS for probabilistic forecast verification. We show that the formulation of the ensemble-based FSS substantially affects verification results. Comparing four different approaches, we determine how the ensemble-based FSS depends on ensemble size, neighbourhood size, and frequency of occurrence of the forecast event. Furthermore, we derive an empirical formula for the expected dependence of the FSS on ensemble size. Our results are based on high-resolution 1000-member ensemble precipitation forecasts over Germany for a high-impact weather period. The large ensemble enables us to study the influence of ensemble size on forecast skill in terms of a probabilistic skilful spatial scale. We demonstrate that only one form of ensemble-based FSS, which we refer to as probabilistic FSS, is well-behaved and exhibits a reasonable dependence on ensemble size.