With GPUs, you don’t have that visibility or the flexibleness to request, “I would like 4 gigabytes of that GPU, and I solely need one gigahertz of that GPU to go together with it.” As an alternative, the most typical setup at the moment is all or nothing—you request the entire GPU or none of it. The transparency problem is that GPUs require an strategy to monitoring and understanding utilization that’s all their very own, as a result of GPUs are specialised and mix features of CPU and reminiscence. That problem is compounded by the truth that a node can have a number of bodily GPUs in a system (generally as much as eight). It’s additionally attainable so as to add or take away GPUs from methods. That’s one thing sometimes seen in on-premises environments, and one thing you’d not sometimes see with CPUs. These dynamics illustrate why gaining GPU visibility requires a recent strategy.
How Kubecost permits GPU monitoring and optimization
Kubecost meets the GPU visibility problem by understanding which nodes have GPUs and whether or not these nodes are on a public cloud supplier or in an on-premises setting. Kubecost additionally understands what these nodes price, and subsequently understands proportionally what the GPU prices. That’s true whether or not a enterprise makes use of one of many “massive three” cloud suppliers, or self-provides node prices primarily based by itself personal cloud configuration.
With these GPU prices in hand, the subsequent step is to have a look at GPU utilization. Kubecost identifies price allocation primarily based not solely on GPUs requested, but in addition on GPU utilization, with a purpose to acknowledge idle capability. Kubecost additionally scrapes commonplace metrics, together with utilization info, offered by Nvidia software program. (We plan to broaden to AMD and extra GPU manufacturers.) By combining price and utilization info, Kubecost can decide GPU effectivity, which is among the largest questions in enterprise leaders’ minds as GPUs develop ever extra highly effective and dearer.