Sign in to confirm you’re not a bot
This helps protect our community. Learn more

CI/CD for Data - Building Dev/Test Data Environments with Open Source Stacks - Vinodhini Duraisamy

Continuous Delivery Foundation
2.83K subscribers
<__slot-el>
<__slot-el>
59 views 1 year ago
CI/CD for Data - Building Dev/Test Data Environments with Open Source Stacks - Vinodhini Duraisamy, Treeverse Building a high quality software product is a breeze compared to building a high quality data product. Because software development being an evolved field has defined development best practices and the tools to implement these practices throughout the software lifecycle. On the contrary, most of the data teams have difficulty incorporating best practices like testing changes in isolation, reproducing a data error or rolling back in case of quality issues in production. With an ever-increasing number of organizations relying on data to make business decisions, the cost of data error in production is significantly higher too. Thus it becomes imperative for data teams to ensure the accuracy, resiliency and observability of data in production. So how does one go about extending software development best practices to data products? In this talk, you will learn about industry best practices for data lifecycle management and the open source tools such as Jenkins and lakeFS that help implement them. For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/ ...more
...more
CI/CD for Data - Building Dev/Test Data Environments with Open Source Stacks - Vinodhini Duraisamy
0Likes
59Views
2023May 26
CI/CD for Data - Building Dev/Test Data Environments with Open Source Stacks - Vinodhini Duraisamy, Treeverse Building a high quality software product is a breeze compared to building a high quality data product. Because software development being an evolved field has defined development best practices and the tools to implement these practices throughout the software lifecycle. On the contrary, most of the data teams have difficulty incorporating best practices like testing changes in isolation, reproducing a data error or rolling back in case of quality issues in production. With an ever-increasing number of organizations relying on data to make business decisions, the cost of data error in production is significantly higher too. Thus it becomes imperative for data teams to ensure the accuracy, resiliency and observability of data in production. So how does one go about extending software development best practices to data products? In this talk, you will learn about industry best practices for data lifecycle management and the open source tools such as Jenkins and lakeFS that help implement them. For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

Follow along using the transcript.

Continuous Delivery Foundation

2.83K subscribers