Union.ai’s Union Cloud management enables new ML and data products
As machine learning and data products have grown more powerful and complex, a new generation of software tools has arisen to orchestrate the tasks behind them.
Union.ai today made the power of orchestration accessible with the release of Union Cloud, a managed version of the popular Flyte orchestrator that includes data, models and infrastructure to start building quickly at scale. By handling infrastructure tasks, the Union Cloud platform cuts engineering overhead so data and ML teams can focus on building amazing products no matter the size of the company.
The company announced Union Cloud’s availability at KubeCon + CloudNativeCon North America 2022, the Cloud Native Computing Foundation’s annual conference. Union.ai also announced the availability of Union Sandbox, a cloud-hosted Flyte sandbox that will let new Flyte users get started immediately without having to use a local sandbox to develop workflows and pipelines.
“Most ML-powered products never make it to production. Until now, only organizations with significant resources and skills can operate ML and Data products efficiently and reliably at scale,” said Union.ai CEO Ketan Umare. “This is what led me to start Union to help any team deliver reliable ML products at scale.” Before founding Union.ai, Umare was on the team at Lyft that created Flyte, which the transportation company open-sourced under Union.ai’s management in 2020.
Both Flyte and Union Cloud run native on Kubernetes, the open-source system for automating deployment, scaling and management of containerized applications. “Modern application development technologies help developers reduce operational overhead and create value faster,” said Matt Klein, a recent CNCF board member and creator of Envoy, Lyft’s open-source edge and service proxy.
“Flyte has been built from the ground up as cloud-native platform for ML and data orchestration. Leveraging Kubernetes as its execution engine enables Flyte to deliver scalable machine learning and data workflows with less complexity.”How orchestration changes the world
ML and data products handle data-intensive, real-time workloads such as sorting billions of biomarker patterns within a routine blood sample to detect cancers early or processing satellite images to track methane emissions globally at concentrations as small as two parts per billion within a 100-meter area. Creators of these products must be able to update them as rapidly as new data becomes available — far more quickly than traditional software development workflows can handle.
Orchestration automates a process or workflow across multiple environments, and it coordinates and manages a complex web of systems and services.
“Flyte actually is used in every single one of these [bioinformatics] components,” said Jeev Balakrishnan, staff software engineer with Freenome, which is using Flyte to orchestrate its cancer-detection biotech. “It’s across our entire data engineering and data-science stacks. It’s not an overstatement to say that Flyte is really a workhorse at Freenome.”
There’s a lot of things that we like about Flyte,” said Nicholas LoFaso, lead backend engineer for MethaneSAT’s data-processing pipeline. “Being Kubernetes native and highly scalable is obviously a big deal. The fact that it’s originally out of Lyft and has the backing of all these other partners like Spotify — where it’s running 5 million workflows — gives us confidence that our use case is handled. We’re not running as many workflows, but each one requires a ton of processing.”
KubeCon 2022 runs Oct. 24 to 28. Meet Union.ai at KubeCon Booth SU 91 or visit the virtual booth at https://bit.ly/3Cy6iz5. To request a free trial of Union Cloud or try Flyte for free in the hosted sandbox, visit the company’s website.
Visit AITechPark for cutting-edge Tech Trends around AI, ML, Cybersecurity, along with AITech News, and timely updates from industry professionals!
