AI and FinOps Predicted to Lead Observability Innovation in 2024

In recently published articles, three large observability companies have made predictions for the trends we will see in the observability area in 2024 and beyond. These contributions suggest that the fieldsd of AI Integration, FinOps, OpenTelemetry and Security and Governance will impact observability significantly in the year ahead. Grafana Labs, Dynatrace and Logz.io have each published their observability predictions for 2024, and there are four themes that run through all of the articles. AI Integration in Observability: Marc Chipouras at Grafana Labs emphasizes a shift towards more thoughtful utilization of AI in observability. 2024 will see a focus on leveraging AI for time and cost savings, augmenting human expertise rather than replacing it. Asaf Yigal at Logz.io notes that incidents taking more than an hour to resolve (MTTR) have increased from 64% in 2022 to 73% in 2023, and posits that growing complexity of cloud-native systems is causing this. He goes on to suggest using AI to help sift through the large amount of observability data collected by many organizations to prioritise and save money, and that AI can help with the prioritisation of alerts. Dynatrace’s engineering team also posits using AI on observability data for resource utilization, enabling organizations to minimise emissions and costs. “We won’t see AI take jobs from SREs, DevOps specialists, or engineers. Rather, AI will become more of a trusted tool to understand systems quickly through signal correlation, anomaly detection, root cause analysis, and performance optimization.” – Marc Chipouras (Grafana Labs) Financial Observability (FinOps): Richard Hartmann at Grafana Labs predicts a data-driven approach in FinOps, enabling companies to correlate costs with profit centers. This will inform strategic decision-making, avoiding excessive cost-cutting in revenue-generating areas. JuanJo Ciarlante at Grafana Labs also highlights the importance of budgeting amidst economic uncertainly. Dynatrace add to this, suggesting that the combined pressure of adopting sustainable business practices and managing escalating cloud costs will focus IT more heavily on FinOps. Asaf Yigal at Logz.io expands on this by suggesting that smart data collection and data hygiene in observability products will help businesses to further reduce their costs. “One problem with running cloud-native applications on multi-cloud, multi-region architectures is that they generate enormous amounts of observability data-and increasingly so as organizations scale. With the volume of observability data exploding, most organizations aren’t equipped to handle the budget-breaking costs that ensue.” – Asaf Yigal (Logz.io) OpenTelemetry: staff at Grafana Labs predict a big year for OpenTelemetry, with it providing an important link between application and infrastructure observability thanks to high affinity with Prometheus, with OpenTelemetry adopting semantics shared with Prometheus. Continuous profiling is expected to become integral to OpenTelemetry’s capabilities, combining with traces, metrics and logs to offer deeper insights into application performance. And OpenTelemetry standards will help improve CI/CD observability, helping developers to maintain reliability and performance of CI/CD systems. “I n 2024, more organizations will leverage OpenTelemetry standards to bring observability to CI/CD pipelines, making the software delivery process fully observable.” – Dimitris Sotirakis (Grafana Labs) Security and Governance: in a recent Dynatrace survey, 57% of DevOps respondents suggested that the lack of observability data made it hard to improve automation in a compliant way. This suggests that organizations may start to mandate data observability, so that the business can depend on high-quality reliable data streams driving decision-making. In addition, Dynatrace believe that SIEM systems will become obsolete, as the rise of AI using generative, causal, and predictive techniques surpasses more traditional capabilities. Asaf Yigal at Logz.io also emphasizes the convergence of security and observability, and that observability solutions need to cater for both. Technologies like Kubernetes require a shared interest in performance and security across operational and security teams, and observability solutions that bridge these areas will come to the fore. To summarise, observability leaders predict AI integration, FinOps focus, OpenTelemetry growth, and Security-Governance convergence as key trends shaping observability in 2024 and beyond.