In this episode, Dean speaks with Yuval Gabay, MLOps Engineer at WSC Sports. They talk about MLOps methodologies, standardizing deployment in the organization, and closing the loop back from production into training.
In this episode, I had the pleasure of speaking with Yuval Gabay, MLOps Engineer at WSC Sports. Yuval builds better infrastructure and automation for developing, training, and deploying machine learning models at scale, with a focus on video data. We talk about MLOps methodologies, standardizing deployment in the organization, and closing the loop back from production into training.
Watch the Video
Listen to the Audio
Read the Highlights & Summary
Let’s dive into the world of MLOps (Machine Learning Operations) and explore how WSC Sports, a leading sports analytics company, has implemented MLOps in their infrastructure to develop, train, and deploy models at a large scale. We will focus on the insights shared by Yuval Gabay, a DevOps turned MLOps engineer at WSC Sports, during his interview on the MLOps Podcast by DagsHub.
Overview of WSC Sports and its Challenges
WSC Sports specializes in analyzing videos for sports broadcasters, including the NBA and premier leagues worldwide. Their primary goal is to create highlights by analyzing videos and identifying key plays. This requires processing large volumes of video, audio, and data and providing accurate and timely results to their customers. WSC Sports faced challenges in building infrastructure and automating the development, training, and deployment of models at such a massive scale.
Getting into MLOps
Yuval Gabay joined WSC Sports as a DevOps engineer and discovered his passion for machine learning during an extra course in data science. WSC Sports recognized the need for an MLOps engineer, and Yuval took on the role of building and scaling the MLOps infrastructure. His main focus was to bring more knowledge of infrastructure to the data science team and implement best practices in MLOps.
The First MLOps Challenge: Scalability and Infrastructure
When Yuval started as an MLOps engineer, WSC Sports already had multiple services running in production. Yuval’s first challenge was to maintain and enhance the existing infrastructure. He had to rethink the implementation and engineering behind the MLOps infrastructure, taking into account the unique requirements of machine learning models. Some of the specific challenges included scaling the system, managing GPU resources for models that required them, and defining clear boundaries for each application running on Kubernetes.
Implementing GitOps for MLOps
As WSC Sports continued to scale its MLOps infrastructure, the need for centralized management and version control became apparent. To address this challenge, Yuval and his team adopted GitOps, a concept that involves using Git as a single source of truth to manage all resources. They leveraged tools like Argo CD, Helm, and Kustomize to standardize and automate the deployment and configuration management of services. GitOps allowed them to have full traceability, enable collaboration, and implement testing, ensuring the reliability and robustness of their infrastructure.
The Impact of Large Language Models
Yuval acknowledged the growing significance of large language models (LLMs) in the field of machine learning. These models, such as ChatGPT and MidJourney, offer exciting opportunities in understanding images, videos, and audio. Yuval envisions improved capabilities for tasks like action detection and image generation based on text prompts. However, adopting LLMs also presents challenges in terms of infrastructure scalability, data management, and model training. WSC Sports is actively exploring how they can leverage LLMs to enhance their ML capabilities.
The Role of the MLOps Engineer
Yuval emphasized the importance of MLOps engineers in setting the tone for deploying and managing machine learning models. They provide guidance, implement best practices, and empower the data science team to utilize the tools and frameworks that suit their needs. MLOps engineers play a crucial role in ensuring the robustness, scalability, and performance of ML applications in production. They also act as the first responders in case of issues or model failures.
Conclusion
WSC Sports’ journey in implementing MLOps provides valuable insights into the challenges and opportunities in this rapidly evolving field. From scaling infrastructure to adopting GitOps practices, WSC Sports has built a robust and efficient MLOps system that supports their data science team in delivering high-quality services. As the field of machine learning continues to advance, embracing large language models and exploring their potential will be a key focus for organizations like WSC Sports.
