As artificial intelligence drives faster insights and real-time decision-making across the enterprise, the Cloudera AI Inference service, designed to operationalize machine learning at scale, is gaining traction.
To boost large language model performance and the private deployment of models, the Cloudera AI Inference service uses Nvidia NIM microservices and accelerated computing, according to Priyank Patel (pictured), vice president of artificial intelligence and machine learning at Cloudera Inc.
“What we are integrating is the software stack that the Nvidia team has built out, something called NIM — NIM microservices,” Patel stated. “It’s an integrated hardware-software layer that sits above their [graphics processing units]. We learned more of what goes into the NIM, and that really formed the basis of the Cloudera AI Inference service. It’s the model serving offering from Cloudera that works anywhere on public clouds as well as on-premises and fundamentally enables our customers and enterprises to have private endpoints for AI to be able to build and run …