Emerging risks in food and feed safety often manifest through weak, ambiguous signals long before they reach regulatory attention. A new study published by #HOLiFOODproject researchers presents the Weak Signal Miner (WSM), an AI-based, end-to-end pipeline for detecting such early-stage signals within large text corpora.
Abstract
Emerging risks in food and feed safety often manifest through weak, ambiguous signals long before they reach regulatory attention. This paper presents the Weak Signal Miner (WSM), an AI-based, end-to-end pipeline for detecting such early-stage signals within large text corpora. The method combines transformer-based micro-topic modelling, adaptive temporal aggregation, and a recency-and-magnitude scoring scheme to identify rare topics that show disproportionate recent growth. An optional interpretation layer using large language models (LLMs) provides intuitive labels, novelty and severity scores to support expert evaluation without replacing human judgement. Retrospective analyses demonstrate the tool’s effectiveness: the WSM surfaces signals related to Perfluorooctanesulfonic acid (PFOS) contamination (2004–2007) and citrinin in red yeast rice supplements (2010–2017) well before their formal recognition in an EFSA (European Food Safety Authority) opinion. Weak signal mining provides a data-driven complement to expert-based emerging risk identification, by systematically surfacing potential issues that may precede, escape, or only later enter traditional expert information channels and formal institutional processes.
Read more here: Enhancing regulatory foresight through weak signal mining: An AI-based micro-topic emergence scoring approach – ScienceDirect