GPU monitoring differs significantly from the monitoring of general CPU and memory resources, and requires a different approach. Here’s how Kubecost meets the challenge.
Credit: Realstock/Shutterstock
Many enterprise engineering teams—perhaps most at this point—have dived headlong into GPUs to build out proofs-of-concept and operationalize new products. While competitive pressures dictated the direction, GPU cost concerns have begun moving from the back burner to the front. In conversations with leaders of these businesses, we at Kubecost find that one of the biggest concerns is the feasibility of running GPUs at scale. The pain point is generally expressed something like this: “Our new AI product requires us to spend two million dollars a month on GPUs, but we have no insight into that spending. Are we using them efficiently? How much are we wasting?”
What Kubecost has now begun doing for GPUs is very similar to what we started doing six years ago for …