Data Pipelining and How it can benefit the NHS

In this blog we sit down with Berwyn Jones, the public sector lead at Cribl, to talk about telemetry data pipelining and how it can help NHS Trusts. Telemetry data includes logs, metrics, and traces from IT systems and applications, which are crucial for maintaining operational efficiency and security in healthcare environments. Data pipelining involves collecting, processing, and routing data from various sources to different destinations. This process can significantly improve how organisations like NHS Trusts manage their IT and security information.

How long have you been involved in public sector?

I’ve been involved in technology into the public sector for over 20 years. During that time, I’ve worked with all the big central governments, departments and agencies e.g. HMRC, DWP, Home Office, DVLA, DHSC etc; but I’ve also worked across universities and NHS Trusts as well.

What trends have you seen over the last few years in particular, that persist today?

In those 20+ years there have been plenty of trends, but I’d say the biggest I’ve seen happening in the last few years are around the digitisation of services and providing more online services to the citizen. More organisations have also been moving data and systems into the cloud, rather than managing them on premise within their own data centers. And more recently, we’re seeing a rise in the use of generative AI which creates even more data for IT and security teams to manage.

As you’ve alluded to above there has been a lot of change in the public sector, along with adoption of technology. What ongoing challenges do you see for public sector organisations as they look to continue to deliver on these trends?

When it comes to delivering on these trends, one major challenge that I’m seeing is the massive growth in data. According to IDC, there’s a 28% compound annual growth rate in terms of the rise in telemetry data. The second challenge faced is vendor lock-in. Once they’ve got all this data and they place this data within their systems, that data then becomes proprietary to those systems, so there’s an element of vendor lock-in there. Lastly, organisations are struggling with the skills gap. It’s pretty well documented across the UK that there is an IT skill shortage, and that those problems even worse across the public sector. As a result there aren’t enough technical resources to drive through the required transformation at the pace that people want to drive it. It’s holding people back.

And focusing on the NHS specifically, what are you seeing across NHS centrally, and at the local level, within those general themes of data growth, vendor lock in and skills shortages? How are those specifically impacting the NHS?

If you think about the systems that the NHS are using and the criticality of them, those systems need to be available at all times, 24/7. These systems help to save patients’ lives. Secondly, the attack surface has widened. As we’ve seen, there’s not only this movement of data from on premise to the cloud, but we’re also seeing now SaaS applications/solutions being used. So, the attack surface where cyber security attackers and adversaries can get in has widened significantly. Last but not least, as the volume of telemetry data has increased, the budgets for managing that data haven’t really kept up pace with that. As mentioned, data is growing at up to 28% per annum, but the budgets for managing that data aren’t growing at the same rate. There’s a clear tension existing between data growth and the budget for managing that data.

However, for NHS Trusts the opportunity is that with the availability of more data, there is the opportunity to drive better outcomes for both patients and the NHS. If the right data can get to the right people and place at the right time, we can deliver better outcomes. On the other hand, in terms of the Cribl space of things, it’s the ability to do this within their existing budgets and make better use of their existing tools. Rather than constantly having to go and ask for more budgets to cope with the data growth, there’s the opportunity to do this while making the best possible use of existing budgets.

So in your most recent role, you are leading UK public sector at Cribl. Clearly you’ve seen something where you have identified that Cribl can help with some of the ongoing challenges faced by the NHS, and indeed public sector. What did you see at Cribl that excited you to join that organisation and how it can help?

I’m massively excited about the opportunity at Cribl and how we’re going to be able to help the UK public sector across both its IT and security data. At Cribl, we’re currently working with hundreds of customers globally and that spans across all industries, including some of the UK’s key public sector organisations. Our software and services are used in some of the most demanding environments worldwide.

I think what excites me most about Cribl, and what’s the real differentiator, is that it’s completely different to any organisation I’ve worked with in my career to date. Cribl’s truly vendor-agnostic. There is no requirement that customers rip and replace technology platforms that they’ve already invested in. It’s about working with your existing technology choices and working to make them work better for you; to give the NHS greater choice, flexibility and control over its data.

You’ve got to take control over your data if you want to take control over your budgets and drive improved outcomes.

Telemetry data is a double-edged sword, it’s both an asset and a liability. As mentioned, it can be a liability due to data volume growth, rising costs, vendor lock-in and a shortage of folks with the skills to be able to work with that data.

Data management is certainly a hot topic at the moment, for many industries. Digging into the NHS in particular again, how do you see Cribl help the NHS Trusts?

As mentioned, telemetry data can be an asset and a liability, but with Cribl we can help to remove those liabilities and help the NHS realise the true potential of its telemetry data, so that data becomes a real asset. Overall, NHS can get better quality data into their systems to help drive better outcomes, help improve the availability of its IT operations, to improve its overall cybersecurity posture. And doing all of this whilst keeping the budgets for managing the data in check!

Imagine the value of being able to remove vendor lock-in, improve the mean time to investigation and the mean time to resolution, while also removing operational blind spots and being able to onboard data that they can’t today. Organisations have got operational blind spots because it’s either too expensive or there was a lack of engineering resources to get that data into their systems. As the old adage says, “You cannot act upon what you cannot see”.  So Cribl helps organisations gain greater observability across all of its telemetry data.

Based on your experience working with a lot of different customers who adopted the platform and looking at their data, if an NHS Trust wanted to combat some of those problems you’ve talked around where might you suggest they start? What will be your top three tips for them if they wanted to start using Cribl or looking at Cribl to manage their data better?

What got organisations to where they were yesterday isn’t suitable for the next decade. They need to really think about having a data modernisation strategy, I think it’s absolutely key!

3 Tips that I would recommend to underpin a data modernisation strategy are:

1. Looking to adopt a data pipeline management solution for IT and security data. There was a recent blog by Forrester and it basically says, if you’re not using a data pipeline management solution for your security and IT data, then you need to.
2. Separating telemetry data, they need to separate the analysis from the retention. There’s a tendency now, with some organisations just to put all of their data into their SIEM or into the observability platforms. As we know, that’s really expensive, so there’s an opportunity to separate what needs to be retained for compliance purposes and then what data can then be used within the system of analysis to help drive the better cyber security posture, or the better IT operations.
3. Streamlining the collection of data. Today organisations are collecting the same data multiple times and then rooting it to multiple destinations. There’s the ability to think about collecting that data ONCE and then routing it to where you need to.

Conclusion

In conclusion, the NHS faces significant challenges in managing its growing volume of telemetry data while dealing with budget constraints and skills shortages. Cribl offers a unique, vendor-agnostic solution that can help NHS Trusts overcome these obstacles. By implementing a data pipeline management solution, separating data analysis from retention, and streamlining data collection, NHS organisations can improve their operational efficiency, enhance cybersecurity, and drive better patient outcomes.

As the healthcare sector continues to evolve, taking control of data management will be crucial for the NHS to meet its future challenges and deliver high-quality care to patients across the UK.