In evidence-based horticultural evaluation, significance carries authority. It shapes which rootstocks advance, which cultivars are commercialised and which cultivation practices are adopted across orchards and regions. Yet in practice, we quickly learn something important: not all significant results matter equally — and some results that matter a great deal never quite reach statistical significance at all. This is not a failure of statistics. It is a misunderstanding of what statistics tell us.
Kramer and colleagues explain in their guidance for horticultural research that statistical tests are used to determine whether an observed difference is unlikely to be due to random variation alone. A P-value below a chosen threshold indicates rarity under a null hypothesis, not importance. Crucially, P-values cannot tell us whether a difference is large enough to matter horticulturally or economically. Statistical significance answers a question about certainty, not relevance. This distinction is often blurred because the word significant has a different meaning outside statistics. As Martínez-Abraín points out, in everyday language, the term implies importance or magnitude. In statistics, it does not. Confusing the two leads directly to misinterpretation — and, in industry-facing horticultural systems, to poor decisions.
Horticultural relevance is a different question
Across disciplines, the literature converges on the same conclusion. Lovell argues that statistical analysis and biological interpretation are complementary but distinct, and that horticultural importance should not be inferred from only statistical significance. Instead, researchers should focus on effect size and define, in advance, the minimum difference that would matter in practice.
Across the literature, the same distinction appears repeatedly. Lovell separates statistical certainty from biological meaning. Martínez-Abraín formalises this divide, while Saxon offers a clear illustration: statistically significant differences in soil microbial communities that correspond to only a few hundred cells are unlikely to influence crop performance. The statistics are significant; biologically, it is not. Across these studies, the message is consistent and explicit: statistical significance is not a proxy for horticultural importance.
Why this matters in perennial, multi-environment systems
Perennial horticulture amplifies this issue. Long-term orchard trials are expensive, spatially constrained, and biologically noisy. Replication is often limited by design rather than by poor planning. Environmental heterogeneity, year effects, and genotype × environment interaction (G×E) are inherent features of the system. In multi-environment trials, real horticultural responses may express differently across sites or seasons. This increases variability while still pointing to a consistent underlying pattern. In this context, Kramer and colleagues caution strongly against treating non-significant results as “no effect”. Effects of potential horticultural or economic importance should still be reported, provided the associated uncertainty is acknowledged. In other words, failure to reach statistical significance does not imply the absence of an effect. It implies uncertainty.
Keeping research relevant
Consider a familiar scenario in rootstock evaluation. A candidate rootstock shows a 10–15% increase in cumulative yield efficiency over five years relative to the industry standard. The response is consistent across seasons and environments, but variability between blocks and sites is high, and replication is limited — a common reality in long-term orchard trials. The overall analysis returns a P-value just above 0.05. A strict threshold-based interpretation would label this result “not statistically significant”. However, that conclusion goes beyond what the statistics actually report.
Kramer et al. state explicitly that P-values cannot determine whether a difference is large enough to matter, and that results with potential horticultural or economic importance should not be dismissed solely because they fail to reach statistical significance. Lovell reinforces this by arguing that the more meaningful question is whether the observed effect exceeds a relevant horticultural threshold. This does not mean such a result should be treated as proof. Rather, a stable and horticulturally plausible response of this magnitude can legitimately direct future research. As the literature makes clear, lack of statistical significance often reflects limited sample size or high variability in a particular dataset — not the absence of an underlying effect. With improved trial design, greater replication, or targeted follow-up studies, the same response may be tested more conclusively in future datasets. Handled correctly, such signals help ensure that research effort remains focused on questions that matter in orchards and packhouses, rather than on differences that are merely easy to detect statistically.
How we think about significance at Provar
As researchers at Provar, we distinguish explicitly between three kinds of significance, because each answers a different question:
- Statistical significance. Is the observed difference unlikely to be due to chance? This addresses certainty. It does not tell us whether the difference matters in practice.
- Horticultural relevance. Is the size of the effect meaningful in a production system? This depends on effect size relative to a relevant threshold — for example, a change in growth habit, yield efficiency, colour development, or stress response that would influence orchard or packhouse outcomes.
- Technical relevance. Does the difference influence decisions or risk in the industry? This is where yield efficiency, pack-out class, colour compliance, storage behaviour, management flexibility, and economic risk come into play.
These forms of significance are related, but they are not interchangeable.
When statistics matter more than horticulture
The opposite situation is equally important. As Saxon demonstrates, statistically significant differences can be horticulturally trivial when absolute magnitudes are small. This challenge is increasingly common in modern horticulture, where large datasets, automated measurements, and fine-scale sensors dramatically increase statistical power. A small but statistically significant shift in colour metrics may not move fruit into a different pack-out class. A measurable and statistically significant difference in firmness may still have no impact on storage behaviour or consumer experience. In such cases, statistical significance risks overstating practical importance. As Martínez-Abraín warns, equating significant with important can actively hinder knowledge accumulation and decision-making.
So, is some significance more significant than others?
In evidence-based horticultural evaluation, yes. Statistical significance tells us whether an effect is likely to be certain. Horticultural relevance indicates whether it matters in the production system. Technical relevance tells us whether it matters to growers, packhouses, and the industry. Sound and meaningful evaluation respects all three. As Kramer and colleagues conclude, the goal is not to report significant P-values, but to explain why observed differences are — or are not — horticulturally and economically consequential. This allows research at Provar to sit confidently at the interface of data, science and decision-making — and to deliver insightful, statistically defensible, but useful to the industry.
From the CEO: When uncertainty becomes a starting point
In applied horticultural research, how we interpret statistical significance has real consequences. It shapes which cultivars advance, which systems are explored further, and how confidently evidence is translated into practice. From my perspective, the greatest risk is not uncertainty itself, but how easily statistical non-significance is interpreted as “no effect.” Statistics are powerful tools — but they are tools. They help us assess whether an observed difference is unlikely to be random. They do not, on their own, define importance or relevance.
This matters particularly in perennial orchard systems. Biological variability, environmental heterogeneity, and constraints on replication are not design flaws — they are realities. In this context, consistent and horticulturally plausible responses may remain statistically uncertain. That uncertainty does not imply irrelevance; it signals where confidence is still developing.
At Provar, we see uncertainty not as something to eliminate, but something to carry forward transparently. Researchers have a responsibility to present effect sizes, variability and limitations clearly. But interpretation does not sit with researchers alone. Growers, packhouses, marketers and industry decision-makers all engage with evidence from different vantage points. When uncertainty is made explicit, evidence becomes a shared platform for discussion rather than a binary filter. That shift — from certainty-seeking to risk-aware interpretation — is where collaborative, industry-relevant research begins.
— Tristan Dorfling, CEO, Provar
Sources:
Agathokleous E. 2022. Rethinking biological significance in plant science: When statistical significance is not enough. Plant, Cell & Environment. 45(2):305–308.
Grabowski B. 2021. How statistical significance became the gold standard for scientific claims. Philosophy of Science. 88(4):579–601.
Kramer M, et al. 2019. Statistical considerations for horticultural field experiments: Beyond P-values. HortScience. 54(10):1681–1686.
Saxon E. 2015. Statistical or biological significance? BMC Biology. 13:91.
Rae is the Scientific Publications and Research Conceptualisation Lead at Provar. She is a qualified horticulturist, scientific writer and editor and holds a doctorate in plant breeding. In her role, she works at the intersection of scientific research and applied evaluation, developing research concepts, trial designs, and strategies for implementation. Rae also plays a key role in the scientific development of Provar’s Culteva™ platform, creating literature-based concepts to ensure scientific rigour. Additionally, she supports Provar’s scientific publications, technical reports, conference materials, project planning and research funding applications, collaborating across various disciplines and organisations.
