CSPs must adopt an observability platform to unlock the key benefits of cloud-native environments

The topic of observability has quickly gained traction and has grabbed the attention of every cloud-native (and would-be cloud-native) organisation. The highly distributed nature of cloud-native environments has led enterprises to adopt technologies that support correlations between domains within the cloud stack to enable real-time visibility into these complex environments. Communication service providers (CSPs) must also embrace such technologies; their networks are more complex than most cloud-native environments, but the foundational principles of observability remain the same.

In this article, we summarise what observability is, how it is different from monitoring and what the most effective method of implementing observability is from a CSP perspective. Our report, The role of observability in telecoms and how vendors can facilitate its implementation, provides further details, as well as insights into the challenges associated with implementing observability and the current vendor landscape.

Observability is a critical enabler of CSPs’ cloud-native operations

Observability is defined as the ability to measure the state of a system based on the data produced by that system. It is a cloud-native concept that is used in distributed IT systems to provide insights into the internal state of a cloud-native environment (including applications, cloud platforms and cloud physical infrastructure). CSPs can use these insights to unlock the benefits of cloud-native environments (such as increased agility when managing the network) and move away from monolithic monitoring techniques that do not give a holistic view of the network.

However, observability is far more than just an extension of monitoring. The key difference between monitoring and observability is that the latter offers the ability to unify multiple types of telemetry data (metrics, traces and logs) from the various components within a cloud-native stack, thereby allowing for much deeper visibility. Monitoring focuses on tracking known outcomes (primarily based on metrics) for specific services and systems that run in siloes, while observability takes a more holistic and de-siloed approach, which allows for the consolidated collection of data from all systems in a cloud-native environment. This makes it possible to for CSPs to correlate observability data from their cloud-native systems across the business through a single pane-of-glass, and allows for the identification of previously unknown issues.

An observability platform is the most effective implementation of an observability framework

An observability framework is a collection of tools that work together in a set mechanism that is designed by either CSPs or vendors/systems integrators. An observability framework can be implemented via an observability platform. This allows for all telemetry data in a cloud-native environment to be collected, processed, stored, analysed and exposed in a standardised manner to address multiple use cases simultaneously. CSPs can implement a full-stack observability framework (across the application, cloud platform and physical infrastructure layers) by adopting such an observability platform, as shown in Figure 1.

Figure 1: Observability platform architecture for telecom environments

There are three essential capabilities that an observability platform should provide.

• Data collection and processing. Observability platforms collect telemetry data in a standardised way using open-source tools, then correlate and filter data to enable visibility and reduce storage needs. OpenTelemetry is an example of a framework that is used to achieve this via automated instrumentation and encoding.

• Data storage. Metrics, traces and logs are stored in separate stores or in a combined solution. This storage needs to scale well. The architecture that supports data storage will differ depending on CSPs’ data strategies.

• Analytics. Observability platforms also analyse and correlate observability data to provide insights that are then exposed to third-party applications. AI/ML-based approaches to analytics can handle complex analytics that occur within very large and varied data sets.

The combination of these elements enables the value of observability to be maximised.

What are the benefits and challenges of implementing an observability platform?

Observability platforms can help CSPs that are migrating to cloud-native systems by providing actionable insights into the state of their networks. These insights provide internal teams with several benefits, including the faster and more-accurate resolution of issues (due to the direct and standardised access to more-granular and relevant datasets). The constant availability of granular data across the cloud stack when using an observability platform will help CSPs to generate better data sets that can be used to train and update AI/ML models for AIOps and other AI-related use cases. Other use cases for observability include edge workload management, application optimisation and new service creation. Observability will also enable CSPs to achieve top-level business objectives by relating operational KPIs to organisational targets and the end user.

However, the scope of the use cases currently addressed by telecoms-specific observability solutions is limited. Many solutions focus on a single use case, typically related to service assurance, which can lead to complexity when trying to integrate multiple use cases from other vendors. Adopting observability on a per-use-case basis without a single platform approach can take away from a critical difference between monitoring and observability: correlations across domains and identifying unknown unknowns. A solution to this problem is to adopt open observability solutions that use open-source tools and allow interoperability between solutions from multiple vendors. This and other associated challenges are described in our report The role of observability in telecoms and how vendors can facilitate its implementation.