All about Licence management

Many customers approach us with the desire to have more control over their Splunk growth and ensure they’re gaining value from the product.   There are many aspects involved, but it mainly starts with licence management (meaning cost).  Whilst not the most recent, but still an interesting metric is from SiliconAngle citing customer spending data and the year-over-year downward trend.  We will let you read the data, but simply, the number of customers planning to decrease or eliminate their Splunk spend had increased by over 50%. This is backed by the nature of the conversations we are having too.

What is licence management?

Licence management is taking control over the type of licence you buy and how it’s consumed. In the case of Splunk, a customer has either an ingest or an SVC licence and these will be sized based on how much data you’re storing and ingesting into the platform, and/or the compute performed on the searches and alerts. As a note, Cribl also did a good piece here (albeit a while back) but we like the diagram. With different pricing models available, licence management is all about how to control and have knowledge over all the data in the platform, and how it will be used in order to achieve a cost-effective solution.

Why is licence management a growing problem and what can be done about it?

It’s thought that by 2025, 85% of data would have migrated to cloud; meaning a lot more ephemeral and microservices orientated than before. This elasticity means more logs period.

There are more logs than ever and data volumes are continuously growing, causing the cost to go up for customers using SIEM, along with the importance of licence management. This calls for a redressing in the market which can be completed in many ways. One way is to avoid putting all your log data in the SIEM, use data streaming products (such as Cribl, Tenzir, apache beam, Arroyo etc). However, that can only go so far before the security team starts to panic about the value given the pruning of data, so a balance is needed. A balance of discovery and methodology to approach what data you need, to what coverage in the Splunk system and for what purpose.

Typically most customers in their journey of Splunk fall into the habit of “let’s put everything in there and then figure out how to use it”. This is plain wrong; you should think about what data you’ve got, your security posture, your frameworks, security standards, what risk registers, what governance risk compliance you wish to achieve (the same thinking should be applied to Production and Service monitoring). A methodological approach of discover & design should help you figure out what use cases, alerts and searches you want in your Splunk system. As well as what data you will need to achieve this.

Beware, the longer you wait to improve your licence management the worse it’s going to get, as the velocity of the data is not slowing.

How else can you improve or achieve better licence management?

Another aspect is considering where you are going to long term persist this data that’s increased with the exponential explosion of logs, also how your retention period affects your cost. In summary, you likely need a new strategy for data persistence. For example, if you’re storing data for 90 days but only searching 30 days, you’re paying to store 60 days of storage that you might not need.  Furthermore, as with most customers, for compliance or audit purposes (this is always cited to us), older data (>90 days) is often moved into archive storage, which is often an additional cost with Splunk. In almost all cases we see an additional expense that are excessive, there are often better methods to employ.  Please note the federated data search approach which also incurs additional costs!

Obviously, there are other data persistence platforms that are far cheaper, that can be compressed if you need to just keep the data for risk and compliance purposes.  But, selecting ‘cheap’ storage, is not the answer either, remember the data strategy!

In summary there is a clear data strategy needed for storage and what search/reporting is required on it, beyond its SIEM/Monitoring lifecycle.

But do I need to worry about licence management and sizing?

Yes, definitely! It’s your duty as a customer to take care of your own licence management, not for the vendor to tell you the sizing of the licence. It’s for you to monitor against your data strategy and model, compare any deltas or surprises and surges in activity/data sources. It is for you come up with a strategy for persistence, streaming and so on.

Many clients don’t have a Splunk test environment, which can make testing theories around the licence and figuring out your retention and sizing policies difficult. Without a test environment, if you’re going to change anything you must change it on your production server. However if something goes wrong or something doesn’t perform the changes as anticipated or expected, you can’t reverse it. You’ve already made that change and deleted that data! That’s where we’ve helped tackle licence growth by using a different approach: we created a model so the client can implement different changes to understand how to reduce things, without having to have the high risk of doing it within their Splunk production environment.

Are there any other important things that people usually don’t consider, when thinking about licence management?

Perhaps the most important thing a lot of people don’t realise is how big their data can get, either naturally or through acquisitions & mergers, particularly if a large user community grows without a robust operating model to manage the use of the platform. It can be all to easy for users to add data sources, change retention policies and create expensive searches if not done correctly.

Another habit is people tend to fill up their licence because they can, which isn’t a good policy because you should only be putting things into specific platforms (like Splunk) because you need that data in there.

We see it equally in production monitoring where it might be platform monitoring, especially with the adoption of cloud; there’s an exponential increase in AWS CloudWatch data for example. With everything turned on ingestion can become very verbose, with a lot of the data never used!

Conclusion

Monitoring a license is fairly straight forward, but that just gives you a metric. Hopefully this blog hints at the approaches and strategies you can consider to manage your licence properly. We would advocate starting at first principles from a perspective of what data you have and why would you log ingest it? Effective licence management can help to avoid over-licencing, which can lead to unnecessary costs, but also under-licencing, which could result in compliance issues or insufficient system performance. Effective licence management ensures that the organisation is maximising the value of its SIEM investment while maintaining legal and operational integrity. Most of all it’s bringing back control to YOU!