OpenTelemetry Is Too Sophisticated, VictoriaMetrics Says

(NicoElNino/Shutterstock)

To say the oldsters at VictoriaMetrics aren’t large followers of OpenTelemetry after implementing its metrics library could be placing it mildly. “OpenTelemetery is so sophisticated and bloated,” stated Aliaksandr Valialkin, the CTO and co-founder of the observability software program firm.

OpenTelemetry is a Cloud Native Computing Basis (CNCF) undertaking that was created in 2019 to supply an open supply resolution to a thorny pc drawback: to create a unified customary for the three foremost varieties of observability information, together with traces, metrics, and logs.

With a single customary, the unique OTel considering goes, utility builders will achieve entry to raised instruments to correlate various kinds of observability information with particular servers or digital machines, thereby making it simpler to trace down and repair software program issues. As a bonus, prospects would now not be locked into proprietary IT monitoring options.

OTel was shaped by the merger of OpenCensus and OpenTracing, so it shouldn’t be shocking that tracing is probably the most mature of the three-pronged undertaking, with the library tracing delivered in 2021. That was adopted in Could 2022 with the preliminary launch of the metrics functionality, whereas the overall availability of OTel logging was lastly delivered in November 2023.

Many of the IT monitoring business, in addition to the three cloud giants, have gotten on board the OTel practice. From established distributors like Splunk and Dynatrace to newer outfits like Elastic and Sumo Logic, supporting OTel has been standard throughout the business.

When a few of VictoriaMetrics’s prospects requested that the corporate add assist for OTel metrics, the corporate obliged. Below the steering of CTO Valialkin, VictoriaMetric builders added the OTel metrics library to its providing, which permits customers to gather, shops, and analyze metrics coming from a variety of various techniques.

Nevertheless, the combination with OTel metrics didn’t go as deliberate. Based on Valialkin, the addition of the metrics library brought about the VictoriaMetrics binary to extend in dimension by 50%, or a number of hundred megabytes.

“It’s an enormous worth for including assist for extra ingestion format, like OpenTelemetry,” Valialkin advised Datanami. “Ultimately, we re-wrote this library from scratch and carried out it mainly solely for VictoriaMetrics. And we’re proud of the end-result as a result of it elevated the binary dimension by lower than 1%, and that’s significantly better than 50% with the official library for OpenTelemetry.”

The OTel library for metrics is so bloated and inefficient as a result of it tries to do an excessive amount of, in keeping with Valialkin. It was designed by a committee to handle the assorted metrics wants of a number of distributors. Nevertheless, the top product ended up together with a wide variety of “edge circumstances” that VictoriaMetrics prospects will hardly ever, if ever encounter.

Many of the performance included within the OpenTelemtery customary for metrics isn’t really utilized by any distributors or prospects, Valialkin stated. “I believe that most individuals don’t cope with the low-level particulars of the OpenTelemetry protocol,” he stated. “That’s why they don’t seem to be conscious of this subject which we confronted when including assist for OpenTelemetry in VictoriaMetrics.”

(Shutterstock/AI-generated)

In brief, OpenTelemetery for metrics is a “automobile designed by a monkey,” the corporate stated.

“Our ache level was that our try to undertake this customary for metric congestion brought about numerous hassle,” stated Roman Khavronenko, a VictoriaMetrics engineer. “It was actually painful to do. It was costly by way of engineering hours. It was costly by way of high quality of code.”

Others have filed related compliants. A consumer on Hacker Information final 12 months stated working with OTel was not simple.

“I discover myself having to suppose orthogonally to frequent sense each time I attempt to use certainly one of its SDKs,” wrote consumer iofiiiiiiiii. “Nothing works the way in which you count on it to, every thing has 3 layers of pointless abstraction and must be approached through the again door. Many options have caveats about when it really works, the place it really works, how a lot it really works, throughout what part of the moon it really works and the way lengthy your strings could be when Jupiter is seen within the sky.”

Nevertheless, regardless of the ache, OTel customers nonetheless see the worth of the method.  Hacker Information consumer iofiiiiiiiii said that  the open supply undertaking is “a partial win.” “Whereas the developer expertise is fairly disagreeable and I’m additionally disillusioned with the precise each day utilization, from an architectural perspective it opens up new alternatives that didn’t exist earlier than.”

Although Valialkin and the VictoriaMetrics workforce have soured on OTel as an entire, the core concept–to standardize observability information–stays a great one.

“I really like the concept of OpenTelemtery to unify observability right into a single place so as to have the ability to acquire and remodel and course of observability information in a single place,” Valialkin stated. “The concept is nice. However the implementation is just not superb. We’re not very proud of it.”

A better option for metrics already exists, Valialkin stated. Prometheus has a a lot smaller library, and its textual content exposition protocol is environment friendly, easy, and extensively used. It’s already a de-facto customary, and would make a greater official customary than what the OTel workforce got here up with, he stated.

“It’s extensively utilized by many software program corporations,” Valialkin stated. “It has good protection for metrics. It’s simple to debug. It’s simple to implement. It’s simple to parse in comparison with OpenTelemetry. And it isn’t too complicated. It isn’t bloated in comparison with OpenTelemetry.”

Alas, Prometheus is barely used for metrics, and doesn’t assist OTel requirements for labeling, which is essential for correlating with different observability information sorts. “It could be onerous to change between logs and traces and metrics when utilizing Prometheus textual content exposition format for metrics assortment,” Valialkin stated.

VictoriaMetrics is transferring ahead with its assist for the OTel metrics information customary, albeit by its personal customized library. Even with the slimmed down customized library, ingesting OTel metrics information is slower and consumes extra sources than any of the opposite metric information sorts supported by VictoriaMetrics.

“We’ve assist of 10 protocols, or one thing like that, for ingestion,” Khavronenko stated. “None of them is so complicated.”

In the meantime, VictoriaMetrics is transferring ahead with a brand new product: VictoriaLogs. The corporate can also be adopting OTel’s log protocol for the product, and is girding for extra of the identical.

“I believe that it shares the identical complexity as the usual for metrics,” Valialkin stated,” so which may be the principle stumbling level.”

Associated Objects:

OpenTelemetry Good points Momentum as Observability Customary

Why Roblox Picked VictoriaMetrics for Observability Information Overhaul

Open Supply Occasions Collection Database VictoriaMetrics Sees Vital Development