Przejdź do trybu offline z Player FM !
Generative AI on Kubernetes
Manage episode 406140511 series 3332465
In this episode of the Kubernetes Bytes podcast, Ryan and Bhavin sit down with Janakiram MSV - an advisor, analyst and architect to talk about how users can run Generative AI models on Kubernetes. The discussion revolves around Jani's home lab and his experimentation with different LLM models and how to get them running on NVIDIA GPUs. Jani has spent the past year becoming a subject matter expert in GenAI, and this discussion highlights all the different challenges he faced and what lessons he learnt from them.
Check out our website at https://kubernetesbytes.com/
Episode Sponsor: Elotl
- https://elotl.co/luna
- https://www.elotl.co/luna-free-trial
Timestamps:
- 02:02 Cloud Native News
- 15:31 Interview with Jani
- 01:11:00 Key takeaways
Cloud Native News:
- https://www.techerati.com/press-release/octopus-deploy-acquires-codefresh-to-boost-kubernetes-and-cloud-native-delivery/
- https://www.civo.com/blog/kubefirst-joins-civo
- https://cast.ai/kubernetes-cost-benchmark
- https://www.techradar.com/pro/vmware-customers-are-jumping-ship-as-broadcom-sales-continue-heres-where-theyre-moving-to
- https://cloudonair.withgoogle.com/events/techbyte-making-ai-ml-scalable-cost-effective-gke
- https://dok.community/dok-events/dok-day-kubecon-paris/
- https://training.linuxfoundation.org/certification/certified-argo-project-associate-capa
Show Links:
- https://www.youtube.com/janakirammsv
- https://www.linkedin.com/in/janakiramm/
- - NVIDIA Container Toolkit - https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html
- NVIDIA Device Plugin - https://github.com/NVIDIA/k8s-device-plugin
- NVIDIA Feature Discovery - https://github.com/NVIDIA/gpu-feature-discovery
- Hugging Face Text Gen Inference - https://huggingface.co/docs/text-generation-inference/index
- Hugging Face Text Embeddings Inference - https://huggingface.co/docs/text-embeddings-inference/index
- ChromaDB - https://www.trychroma.com/
74 odcinków
Manage episode 406140511 series 3332465
In this episode of the Kubernetes Bytes podcast, Ryan and Bhavin sit down with Janakiram MSV - an advisor, analyst and architect to talk about how users can run Generative AI models on Kubernetes. The discussion revolves around Jani's home lab and his experimentation with different LLM models and how to get them running on NVIDIA GPUs. Jani has spent the past year becoming a subject matter expert in GenAI, and this discussion highlights all the different challenges he faced and what lessons he learnt from them.
Check out our website at https://kubernetesbytes.com/
Episode Sponsor: Elotl
- https://elotl.co/luna
- https://www.elotl.co/luna-free-trial
Timestamps:
- 02:02 Cloud Native News
- 15:31 Interview with Jani
- 01:11:00 Key takeaways
Cloud Native News:
- https://www.techerati.com/press-release/octopus-deploy-acquires-codefresh-to-boost-kubernetes-and-cloud-native-delivery/
- https://www.civo.com/blog/kubefirst-joins-civo
- https://cast.ai/kubernetes-cost-benchmark
- https://www.techradar.com/pro/vmware-customers-are-jumping-ship-as-broadcom-sales-continue-heres-where-theyre-moving-to
- https://cloudonair.withgoogle.com/events/techbyte-making-ai-ml-scalable-cost-effective-gke
- https://dok.community/dok-events/dok-day-kubecon-paris/
- https://training.linuxfoundation.org/certification/certified-argo-project-associate-capa
Show Links:
- https://www.youtube.com/janakirammsv
- https://www.linkedin.com/in/janakiramm/
- - NVIDIA Container Toolkit - https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/index.html
- NVIDIA Device Plugin - https://github.com/NVIDIA/k8s-device-plugin
- NVIDIA Feature Discovery - https://github.com/NVIDIA/gpu-feature-discovery
- Hugging Face Text Gen Inference - https://huggingface.co/docs/text-generation-inference/index
- Hugging Face Text Embeddings Inference - https://huggingface.co/docs/text-embeddings-inference/index
- ChromaDB - https://www.trychroma.com/
74 odcinków
Wszystkie odcinki
×Zapraszamy w Player FM
Odtwarzacz FM skanuje sieć w poszukiwaniu wysokiej jakości podcastów, abyś mógł się nią cieszyć już teraz. To najlepsza aplikacja do podcastów, działająca na Androidzie, iPhonie i Internecie. Zarejestruj się, aby zsynchronizować subskrypcje na różnych urządzeniach.