How a Need for Full Tech Stack Visibility Is Powering Observability Suites

As IT infrastructure becomes increasingly complex, the ability to see across the full IT stack, from end to end, is becoming increasingly important. That’s because the independent nature of its various components makes it exceedingly difficult to monitor and identify where, exactly, a degradation in performance is taking place that would allow someone to provide a suitable fix.

To gain visibility across the full IT stack ― not just into the network but also into the applications, database, containers, cloud services, and compute and storage infrastructure ― organizations need an end-to-end perspective that observability vendors cannot yet provide. To meet the observability needs of those organizations, vendors are expanding their capabilities across the stack accordingly, an expansion that could ultimately lead to single observability platforms that address multiple observability solutions and stakeholder needs.

Increased complexity leads to greater observability needs 

Observability needs have traditionally differed depending on the stakeholder, the purchasing influence of which differs depending on the solution (see Figure 1).

While members of IT ops and network ops typically monitor and maintain the performance of their organizations’ technology (compute and storage) infrastructure and networks, respectively, line of business owners are concerned with service delivery and performance trends. IT security and security ops, meanwhile, keep tabs on various parts of the system in order to detect threats and/or anomalies. 

But observability use cases are converging, driven by several trends. One is that as organizations undergo digital transformations, app performance is being directly linked to revenue and competitive differentiation. In the meantime, with the growth of hybrid workloads, there is an increased push to build products that enhance enterprise-wide vulnerability detection and resolution capabilities.

There is also a “shift left” taking place, whereby applications are finding a wider audience among stakeholders with other responsibilities, including the network manager and developer building observable code. The architectures of those applications are also becoming increasingly complex — think the adoption of microservices/containers, or the use of edge computing — making a robust observability tool necessary. And the more visibility developers have into how end users interact with applications, the better they can optimize the user experience. Plus, having to switch between multiple interfaces to arrive at an aggregated view of system performance across the full stack is a recipe for observability fatigue.

So as more organizations move to the cloud and there is an uptick in the number of potential network vulnerabilities — all while the various pieces of underlying infrastructure, from apps to networking, continue to fragment — IT departments need a more holistic view of their organizations’ infrastructure so they can better manage and ultimately optimize it. Single observability platforms would give them that needed visibility across the entire tech stack. But while vendors see the value in a single pane of glass view, the engineering isn’t there yet.

Vendors are moving to provide a full stack view

Observability suites provide full IT stack visibility through a range of tools. Cloud network performance management (NPM), for example, analyzes network interactions with public/hybrid cloud networks or SaaS-hosted apps, while enterprise NPM focuses on packet loss rates, bandwidth utilization and other variables associated with network performance within local networks and/or on-premises data centers.

If an IT department could take a holistic view of its stack, from end to end, it would be able to understand how the various pieces interact with one another, which would enable the department to make informed decisions that would optimize those interactions. Besides, switching between multiple interfaces to arrive at an aggregated view of system performance across the full stack is deeply inefficient.

Vendors have begun addressing this complexity by expanding beyond their original areas of focus (see Figure 2). The landscape of observability solutions started as a mix of disparate products focused on different segments, with limited overlap across functionality, stakeholders and users. Among the solutions, some featured active and passive monitoring while others featured just passive. Now the greater complexity of applications, which itself is driving a need for more simplified, consolidated reporting, is causing those observability platforms’ use cases to converge.

Vendors, having started out with individual tools designed to deliver observability into a single area, have begun expanding their toolboxes through acquisitions to meet this need. Over the next five to 10 years and beyond, organizations are going to expect vendors to develop a single pane of glass to allow for a single observability platform that addresses multiple observability solutions and that various stakeholders can leverage at the same time (see Figure 3).

The convergence of this highly dynamic yet traditionally fragmented vendor landscape is already making it more concentrated. In the process, it’s becoming a source of increasing opportunity, whether you are a vendor trying to decide where to expand your capabilities, an investor seeking the best growth pathway and avenues to scale, or an IT decision-maker who is looking to take advantage of the visibility that such convergence affords and the increased return on investment it will, by extension, enable.

But this won’t be a winner-take-all situation. The observability suite market is huge and powered by innovation. There are solutions — and opportunities — for everyone.