Project summary
Funding - £1.5 million via the UKRI DRI Phase 2 call
Timeline - March 2024 to November 2025
Aim - to enhance the Environmental Data Service (EDS) by developing standardised, trusted, and interoperable infrastructure for managing sensor data across environmental domains.
Core project team - Sam Pepler (project lead), Emily Anderson, Louise Darroch, Andrew Kingdon, Alexandra Kokkinaki, Alexander Tate, Poppy Townsend, Philip Trembath, Carl Watson
Previous research - the project builds on the outcomes of the ENHANCE-EDS project and aligns with the NERC Digital Strategy and UKRI’s vision for Digital Research Infrastructure.
Main goals of project
- To improve accessibility of sensor data by a wide range of users, from AI to non-specialists
 - To enable sensor data to be used to tackle multi-disciplinary environmental challenges
 - To provide a proof-of-concept service combining air temperature data from multiple data sources (e.g. ship, aircraft, ground-based stations) in near-real time
 
Why it matters
For society
- Easily accessible sensor data will enable better decisions to be made to mitigate environmental issues and address Government mission key challenges
 
For users of the data (e.g. researchers, developers)
- Enhanced ability to tackle complex multi-domain environmental challenges (such as coastal erosion or flooding) that require data comparison, integration and analysis across disciplines
 - Facilitates real-time data sharing and broader accessibility of data
 - Use of standardised terminology to prevent confusion between different domains and sectors
 - Improves user experience and supports their needs
 
For the EDS
- Enhances capability to support science community through provision of simplified integrated co-designed services
 - Enables integration with data from other disciplines and sectors
 - Reduces duplication of effort and supports automation
 - Reduces technical debt and enhances maintainability of data services
 
What work was undertaken?
Exploratory work
- Creation of a live demonstrator service (using standardised API based on OGC sensorThingsAPI) that publishes real time sensor streams and their metadata, from multiple environmental domains.
 - Successful integration of near-real time temperature data from the research ships (RRS Sir David Attenborough, RRS Discovery, RRS James Cook), ground-based stations (Cape Verde Atmospheric Observatory) into a live demonstrator service
 
- An interactive online (Jupyter) notebook to show how end users could integrate multiple streams of temperature data using the demonstrator service
 - Research to understand the types of users this work will benefit
 - Recommendations for how the demonstrator service should be launched and operationalised have been based on findings from the user research
 - Standardised metadata for temperature sensor data across domains, which forms the basis for the demonstrator service
 - AI-based data quality assurance and validation tools to demonstrate how these data could be used by models
 
Operational improvements and decisions
- Documented data governance recommendations
 - A list of lessons learned from the project
 - An updated stakeholder map and dissemination plan for the environmental data service – specifically looking at sensor data stakeholders
 - Template guidance for how to disseminate project work
 - Improved cross-data centre relationships and collaborations focussed on tackling specific complex data challenges
 
What are the (immediate) benefits?
Predominantly the initial benefits have been felt by experts working across our organisations. For example; data managers (who ensure data are Findable, Accessible, Interoperable and Reuseable) and software engineers (who build tools to share/access/analyse data). The project has enabled enhanced foundational infrastructure and relationships to form with impacts such as:
- Shared support for our services, including integrated and efficient workflows
 - The project leveraged existing solutions across centres to avoid reinventing the wheel (e.g. sharing knowledge about semantics, vocabularies and standards)
 - Using common and standardised terminology to minimise misunderstandings between researchers across different disciplines
 - Improved system resilience and extensibility
 - Shared learning, knowledge, solutions and collaboration across our network of experts
 - This project facilitated connections which led to new cross domain projects (e.g. API4AI)
 
Who will benefit from the projects work in the future? And how?
Anyone wishing to access environmental data from across multiple different domains will benefit from the foundational work undertaken by this project. It will also benefit discipline focussed research by simplifying access and lowering barriers for entry. We anticipate it being particularly useful for:
- Researchers/scientists/data analysts/modellers - who integrate sensor data across multiple domains
- Improving reliability of data analysis: reduction in disciplinary silos and bridging gaps between domains to ensure domain knowledge is transparent and traceable
 - More accessible, easier to use data for non-specialists
 
 - AI systems – to automate and build tools to support data ingest/quality check/access/analysis
- More accessible, AI-ready data: by tackling inconsistent formats, standards, and metadata across datasets
 
 - Sensor deployers/instrument operators/remote operators – who are responsible for ensuring the data is high quality
- A lowering of barriers for data providers: by giving them a plug and play system that can evolve for new and broader communities
 
 - Managers of Digital Research Infrastructure – who can learn and build upon the project outputs for their research domains
- International alignment: our work has expanded upon global standards and open-source tools that others can benefit from
 - Act as expert consultants: digital research infrastructures in different domains will have similar challenges and can adapt project outputs for their use cases
 
 - Decision makers/policymakers (e.g. government, local councils) - who use environmental data from multiple domains to make decisions that impact wider society
- More informed decision making: supporting stakeholders with robust data to make informed infrastructure and environmental choices
 - Preventative value: avoiding disasters (e.g. floods) through improved modelling and planning
 - Future real-time decision-making and improved policy decisions
 
 
Ultimately, citizens – the individuals who are affected by environmental change in their local area – will benefit due to the foundational data infrastructure enabling informed robust decisions to be made in response to our changing climate.
Reflections and insights from project team
We asked the project team what they personally found interesting about this work:
- Enjoyed the dynamic, user-responsive nature of the project compared to more conservative operational approaches
 - Found the work challenging but rewarding
 - Appreciated early-stage problem-solving
 - Felt passionate about standards and semantics
 - Took pride in empowering junior staff to solve real problems
 - Valued opportunities for networking and collaboration
 - Enjoyed explored new ways of working
 
Here are some surprises they identified:
- Encountered misaligned terminology
 - Faced a lack of accessible raw data
 - Discovered the complexity of quality vocabularies
 - Recognised the power and flexibility of open-source tools
 - Struggled to align time and priorities
 - Observed high competence across teams
 - Appreciated the value of informal knowledge exchange
 
And finally, some key challenges:
- Had trouble coordinating across organisations with fragmented funding
 - Faced challenges when stepping into leadership roles unexpectedly
 - Found that project scope and objectives were unclear at the outset
 - Encountered technical goals that were unachievable due to resource constraints
 - Noted that immediate success metrics were hard to define; suggested that success might be better measured through long-term collaboration and reuse of solutions.
 
Diversity of project roles
Tackling cross-domain challenges requires many different skillsets – some are often overlooked. Here we describe some of the ones we’ve found useful for this project:
- Connector and translator - bridging teams, identifying shared challenges, and fostering collaboration. It was an unexpected role that emerged organically through the project rather than being predefined.
 - Planner, monitorer and reporter – an essential role for managing and delivering successful project outcomes
 - Interrogator and engager – asking all the difficult to answer questions to ensure we are all on the same page, understanding project direction for different beneficiaries, piecing together the complex details to create a clearer big picture
 - Clarifier, storyteller and disseminator – translating dull, dry or jargon heavy information into an engaging and coherent story, adapting messaging to the intended audience, asking ‘why’ a lot!