Ray & KubeRay, with Richard Liaw and Kai-Hsun Chen

In this episode, guest host and AI correspondent Mofi Rahman interviews Richard Liaw and Kai-Hsun Chen from Anyscale about Ray and KubeRay. Ray is an open-source unified compute framework that makes it easy to scale AI and Python workloads, while KubeRay integrates Ray’s capabilities into Kubernetes clusters.

Do you have something cool to share? Some questions? Let us know:

- web: kubernetespodcast.com

- mail: [email protected]

- twitter: @kubernetespod

News of the week

• CNCF Blog - LitmusChaos audit complete!

• Kubernetes Podcast from Google episode 234 - LitmusChaos, with Karthik Satchitanand

• Google Cloud Blog - Run your AI inference applications on Cloud Run with NVIDIA GPUs

• Diginomica article - KubeCon China - at 33-and-a-third, Linux is a long player. So, why does Linus Torvalds hate AI?

• CNCF-Hosted Co-Located Event Schedule for KubeCon NA 2024 

• Google Kubernetes Engine Release Notes - August 20, 2024 (1.31 available in Rapid Channel)

• Kubernetes Podcast from Google - Kubernetes v1.31: "Elli", with Angelos Kolaitis

• Red Hat Press Release - Red Hat OpenStack Services on OpenShift is Now Generally Available

• Red Hat Enables OpenStack to Run Natively on OpenShift Platform

• Broadcom Revamps Tanzu to Simplify Cloud-Native App Development and Deployment

• Tanzu Platform 10 Offers Cloud Foundry Users Deep Visibility and Productivity Enhancements

• VMware Explore Conference Website

• CNCF Blog - Announcing 500 Kubestronauts

• CNCF - Kubestronaut FAQ

• Dapr Day 2024 Virtual Event Website

Links from the interview

• Kai-Hsun Chen on LinkedIn

• Richard Liaw on LinkedIn

• Ray from the RISE Lab at UC Berkeley

• Ray: A Distributed System for AI by Robert Nishihara and Philipp Moritz - Jan 9, 2018

• KubeRay Docs

• KubeRay on GitHub

• PyTorch

• Apache Airflow

• Apache Spark

• Kubeflow

• Apache Submarine (retired)

• Jupyter Notebooks

• VS Code

• Examples of schedulers for Batch/AI workloads in Kubernetes

• Kueue

• Volcano

• Apache Yunikorn

• Examples of observability tools for Batch/AI workloads in Kubernetes

• Prometheus

• Grafana

• Fluentbit

• Examples of loadbalancers

• Nginx

• Istio

• Ray Data: Scalable Datasets for ML

• Dask Python - Parallel Python

• Ray Serve: Scalable and Programmable Serving

• HPA - Horizontal Pod Autoscaling in Kubernetes

• Karpenter - “Just-in-time nodes for any Kubernetes cluster”

• Lazy Computation Graphs with the Ray DAG API

• Types of hardware accelerators

• Google Cloud Tensor Processing Units (TPUs)

• AMD Instinct

• AMD Radeon

• AWS Trainium

• AWS Inferentia

• Pandas

• Numpy

• KubeCon EU 2024 - Accelerators(FPGA/GPU) Chaining to Efficiently Handle Large AI/ML Workloads in K8s - Sampath Priyankara, Nippon Telegraph and Telephone Corporation & Masataka Sonoda, Fujitsu Limited

• NVidia Megatron

Links from the post-interview chat

• DRA - Dynamic Resource Allocation in Kubernetes

• Different ways of Running RayJob on Kubernetes

• Ray framework diagram in the docs