RunPod: AI Cloud Infrastructure and GPU Compute

Key Features

• On-Demand GPUs

  Dedicated GPU instances known as Pods that provide control over the VM, drivers, and environment.

• Serverless GPU Compute

  Compute workers that scale from zero to thousands, with pay-per-second billing and no idle costs.

• Multi-Node GPU Clusters

  Supports the deployment of GPU clusters for processing compute-heavy tasks.

• Low-Latency Inference

  Infrastructure designed to support inference serving with sub-100ms latency.

• Persistent Network Storage

  S3-compatible storage for AI pipelines with no ingress or egress fees.

• RunPod Assistant

  A natural language interface for managing cloud GPU resources.

Use Cases

• AI Model Inference

  Serving machine learning models in real time with low-latency GPUs.

• Model Fine-Tuning

  Using scalable compute to train and refine AI models.

• AI Agent Deployment

  Deploying AI agents that can run and scale based on demand.

• Compute-Heavy Workloads

  Processing large data workloads using multi-node GPU clusters.

Integrations

• GitHub
• CI/CD
• Docker Hub
• Amazon ECR

FAQ

What is the difference between GPU Pods and Serverless on RunPod?

GPU Pods are dedicated instances providing control over the VM and environment, while Serverless provides autoscaling workers for workloads without manual infrastructure setup.

How does RunPod handle billing?

RunPod uses a pay-per-second billing model for compute, so users pay for the exact time an instance is running.

Can I use my own environment or Docker image?

Yes, GPU Pods support custom Docker images from registries like Docker Hub or ECR.