Instant GPUs, Infinite AI: GMI Cloud Launches On-Demand GPU Cloud Product

As AI adoption accelerates across industries, companies are encountering unprecedented barriers in accessing the GPU resources necessary for innovation. High down payments, long contracts, and multi-month lead times have placed AI innovation just out of reach for many. But today, GMI Cloud is changing that landscape with the launch of its On-Demand GPU Cloud Product, providing instant, scalable, and affordable access to top-tier NVIDIA GPUs.

Versatile Optionality to Meet Global Demand for Compute:

The current surge in global demand for AI compute power requires companies to be strategic in their approach for accessing GPUs. In a fast-evolving landscape, organizations are being asked to pay a 25–50% down payment and sign up for a 3-year contract with the promise of gaining access to reserved GPU infrastructure in 6–12 months.

While certainly valuable for large-scale AI initiatives and projects such as foundation model training or ongoing inferencing, reserved bare-metal/private cloud solutions are not fit for all use cases. Certain businesses, especially startups, don’t always have the budget or long-term forecasting capabilities to commit to large GPU installations. They need flexibility to scale up or down based on application requirements. Similarly, enterprise data science teams often require agility to experiment, prototype, and evaluate AI applications quickly.

GMI Cloud On-Demand GPUs

GMI Cloud is dedicated to driving innovation by providing increased accessibility to top-tier GPU compute. Today we are launching an On-Demand GPU Cloud Product that offers a needed solution, allowing organizations to bypass long lead-times and access GPU resources without the need for long-term contracts. We’ve seen the frustration that companies feel from not being able to access GPUs in an effective manner. Accessibility is currently the primary roadblock to innovation for many companies – we built GMI Cloud On-Demand to eliminate this problem. The on-demand model is perfect for users who need instant short-term access to one or two instances to take on projects that demand high computational power like rapid prototyping or model fine-tuning. GMI Cloud On-Demand offers almost instantaneous access to NVIDIA H100 computing resources and gives additional optionality next to our reserved private cloud GPUs.

Benefits of GMI Cloud’s On-Demand Model

• Added Flexibility: Scale GPU resources up or down almost instantaneously without long-term commitments or down payments.
• Hassle-Free Deployment: Deploy AI models effortlessly with one-click container launches using our expertly pre-built docker image library. We reduce time and complexity in setting up environments, allowing your teams to focus on innovation rather than infrastructure.
• Cloud-Native Orchestration: Manage and scale AI workloads seamlessly with NVIDIA software and Kubernetes integration, from control plane to management APIs. We provide scalability and flexibility, enabling your business to adapt quickly to changing demands without compromising on performance.

Technical Features and Benefits:

NVIDIA Software Stack Integration:

GMI Cloud’s On-Demand GPU Cloud Product includes a comprehensive NVIDIA software stack for seamless deployment and inference:

• TensorRT: High-performance deep learning inference library optimized for NVIDIA GPUs. TensorRT accelerates the inference of models across different frameworks, significantly reducing latency for real-time applications.
• NVIDIA Triton Inference Server: An open-source inference serving software that supports multiple frameworks, including TensorFlow, PyTorch, ONNX, and OpenVINO. Triton allows deployment of ensembles, dynamic batching, and model optimization for efficient inferencing.
• NVIDIA NGC Containers: Access prebuilt NVIDIA GPU-optimized containers from the NGC catalog. Includes models and containers for vision, NLP, speech, and recommendation systems.

Kubernetes Orchestration:

GMI Cloud’s Kubernetes-managed platform offers scalable orchestration for ML workloads

• Multi-Tenancy and Isolation: Kubernetes namespaces and resource quotas ensure secure isolation and efficient resource allocation.
• Automatic Scaling: Horizontal Pod Autoscaling (HPA) dynamically adjusts the number of pod replicas based on workload demands.
• GPU Resource Scheduling: Native support for NVIDIA GPUs via Kubernetes Device Plugins, ensuring optimal GPU utilization and scheduling.

Inference Model Deployment:

GMI Cloud’s On-Demand GPU Cloud Product simplifies the deployment and inferencing of various models:

• LLaMA 3: Fine-tune and infer across different LLaMA 3 model sizes, ranging from 8B to 70B parameters.
• Mixtral 8x7B: Deploy Mixtral, a multi-LLM ensemble designed for parallel inferencing.
• Stable Diffusion: Efficiently generate high-quality images using Stable Diffusion’s state-of-the-art diffusion models.
• Gemma 8x16B: Inference support for Google’s large-scale Gemma models, optimized for parallel inference serving.

On-Demand GPU Use Cases

Startups and Researchers:

• Early-Stage Startups: Quickly prototype AI projects and scale GPU resources based on traction without the need for long-term contracts or large capital investments.
• ML Researchers: Experiment with new models, algorithms, and techniques using flexible pay-as-you-go pricing, perfect for short-term or unpredictable workloads.
• Fine-Tuning Specialists: Optimize and fine-tune models like LLaMA 3, Mixtral, and Gemma without the overhead of setting up private infrastructure.

Enterprise Data Science Teams:

• Data Scientists and Analysts: Prototype, evaluate, and scale AI applications with almost instantaneous GPU access, enabling agile experimentation and testing.
• AI Teams with Tight Deadlines: Accelerate model training and inference while avoiding delays from multi-month lead times and long-term commitments.
• Private Cloud Complement: Use On-Demand instances to supplement existing private cloud infrastructure, offering overflow capacity for burst workloads.

ML Practitioners and DevOps Engineers:

• ML Engineers: Efficiently deploy and infer models like Stable Diffusion, Mixtral, and Triton with preconfigured NVIDIA software stack environments.
• DevOps Teams: Leverage Kubernetes orchestration with GPU scheduling, namespace isolation, and automatic scaling to streamline ML workflows.
• Model Deployment Specialists: Seamless integration with NVIDIA Triton, TensorRT, and NGC containers ensures hassle-free inferencing across various AI models.

Getting Started:

GMI Cloud offers competitive pricing at $4.39/hour for on-demand access to NVIDIA H100 GPUs for 14-days. Visit gmicloud.ai to access our On-Demand GPU Cloud and unlock unlimited AI potential.

Visit GMI Cloud’s booth at Computex in Taiwan in June for hands-on demonstrations of our On-Demand GPU Cloud Product and other innovative AI solutions.