Summary of round I of the experimentation Workshops

LL#1 Identification and Monitoring of Food Safety Risks 

On 17 June 2024, the first phase of HOLiFOOD experimentation Living Lab, associated to WP1 (Big Data technologies and Artificial Intelligence (AI) for food safety detection and prevention), took place online via a workshop.

The first experimentation workshop, following on from last year’s previous exploration workshop, aimed to understand the specific needs of the real-life use of the tools from the stakeholders. The experimentation workshop was coordinated by Wageningen Food Safety Research (WFSR), conducted with Bundesinstitut Für Risikobewertung (BfR), the University of Veterinary Medicine Budapest (UVMB) and Newcastle University (UNEW), and organized in collaboration with Agency for the Promotion of European Research (APRE). It was entitled “Identification and Monitoring of Food Safety Risks.” This Living Lab aimed to identify the needs of stakeholders concerning models and tools for Emerging Risk Identification (ERI) and to gather their feedback on specific AI-based tools.  

During the workshop, a brief overview of AI in emerging risks identification and its role in the food safety risk analysis framework and decision-making processes was provided.

Interactive sessions and specific co-creation exercises allowed participants to provide feedback on (mock-up) outputs from prediction- and text mining models (as under development in WP1), engaging in discussions on how these tools can facilitate better decision-making and risk management.  

27 Participants from multiple sectors were present (Figure 1), with 37% of the total participants coming from the public sector, 32% from academia, 17% from civil society and 14% from industry, representing the entire quadruple helix. The origin of the participants covered a total of 12 different European countries and one representative from Africa.

The co-creation activity emphasized the need for more data to make the ERI tools more complete and highlighted that much information in this field is not openly shared, making it a delicate topic within food safety. Overall, participants acknowledged the potential of the ERI tools but noted the implementation difficulties for structured organizations, suggesting the interest and need to participate in workshops on the topic to gain more knowledge in the field and for related applications.

The feedback collected by the HOLiFOOD LL1 team will be incorporated into the further development of the tools presented – as part of ongoing WP1 work. Then a second experimentation Living Lab, which will be held in 2025 , will test and fine-tune the ERI tools with the users.

 

LL#2 Holistic risk assessment and acceptance

For LL2 a Delphi Study, a well-established systematic user co-creation approach, was adopted. The Delphi method is useful to collect and catalyse group judgments while avoiding the negative effects related to interpersonal biases, strong personalities, defensive attitudes and unproductive disagreements. Throughout this process, panel experts can consider feedback from other panel members and change their opinions without the risk of embarrassment.

Within HOLiFOOD LL2 we asked experts for their input using a traditional Delphi consisting of the following steps:

1. Scoping. Asking experts and stakeholders using anonymous online surveys to scope the baselines, metrics, factors and other elements that they view as important in the estimation of food risks in general and specific to the HOLiFOOD supply chains of Legumes, Maize and Chicken.
2. Explore Consensus. Elaborate the information collected in the first step through the Delphi method to understand the consensus and dissent about current and future state risks/regulations and policies.

3. Workshops were run to validate, expand, nuance and critique the outputs from anonymous survey work.

After the first phase of the Delphi (exploration LL2) launched in June 2023 and ended with the analysis in October 2023, the 2nd phase of the Delphi survey (experimentation LL2) was launched following the three steps described above.

On Wednesday, 28 February 2024 a workshop was held, in which LL2 contributors were all surprised by the collective anonymous views of the peers. The participants noted that the results had made them reconsider the priorities and choices they had planned for their work. It was agreed that this was of added value, but also that researchers are experts and whilst this Delphi process may inform their research planning, in the end they should adopt approaches that are defensible, practical and in line with contractual obligation.

This second round continued from mid-April to end May 2024 as the work proceeded to move through further inductive and deductive phases. This round had a lower response rate of 68.4%; this is normal in Delphi research.

The panel was composed of 19 experts and made up of 90% academics or full-time researchers, and 10% from food safety authorities or agencies. Nationally the panel was split evenly between Denmark, France, Germany, Hungary, The Netherlands and the United Kingdom.

 

LL#3 Platform co-design

On June 18th, Agroknow, as the leader of WP6 focusing on the Novel digital infrastructure for food safety, organized a virtual Living Lab event in collaboration with the EFRA project. For HOLiFOOD, this event – focusing on AI application for food protection and food security – was organised as the first experimentation phase workshop of LL3.

The event aimed to foster collaboration and gather insights from experts in food science and data science. A diverse group of 15 participants attended the online event, including researchers, risk assessors, data scientists, and regulatory affairs experts.

The first hour featured presentations by Agroknow, detailing their work within the HOLiFOOD and EFRA projects, along with presentations from industrial partners in the poultry and animal feed sectors. For HOLiFOOD, during the presentations, the developed dashboard for emerging food safety hazards was presented.

The event thus provided a comprehensive showcase of the mycotoxin prediction model, developed in HOLiFOOD, offering an in-depth look at its functionality and potential applications. The presentations were followed by a series of breakout sessions, held during the second hour, where participants were divided into smaller groups to facilitate detailed and in-depth discussions. They explored the nuances of data analysis and the specific methodologies employed in HOLiFOOD WP6. During these sessions, attendees provided critical feedback on various aspects of the model, including the data sources it utilizes, the factors that should be considered in its analysis, and the overall approach used.

To capture detailed feedback, forms were distributed designed to gather participants’ insights on several key areas. This feedback focused on how well the current data sources align with the stakeholders‘ needs, additional data points or factors that might enhance the model’s accuracy, and suggestions for refining the analytical approach. The insights gained from these forms will play a crucial role in shaping our future work, ensuring that our model evolves in response to real-world requirements and challenges. The discussions yielded valuable recommendations on improving data inclusion strategies and optimizing the model’s practical utility.

In general, the participants of the event provided varying perspectives that enriched the dialogue, leading to a robust exchange of ideas. This collaborative effort is vital for enhancing our digital infrastructure, as it ensures that our models are not only scientifically sound but also practically applicable in routine assessments.

 

Figure 2: A screenshot of the virtual workshop taken during the LL2 with the participants.