Cost-Effective Data Pipelines

eBook Details:

• Paperback: 286 pages
• Publisher: WOW! eBook (August 29, 2023)
• Language: English
• ISBN-10: 1492098647
• ISBN-13: 978-1492098645

eBook Description:

Cost-Effective Data Pipelines: Balancing Trade-Offs When Developing Pipelines in the Cloud

The low cost of getting started with cloud services can easily evolve into a significant expense down the road. That’s challenging for teams developing data pipelines, particularly when rapid changes in technology and workload require a constant cycle of redesign. How do you deliver scalable, highly available products while keeping costs in check?

By focusing on the entire design process, you’ll be able to deliver cost-effective, high-quality products. This Cost-Effective Data Pipelines book helps you:

• Reduce cloud spend with lower cost cloud service offerings and smart design strategies
• Minimize waste without sacrificing performance by rightsizing compute resources
• Drive pipeline evolution, head off performance issues, and quickly debug with effective monitoring
• Set up development and test environments that minimize cloud service dependencies
• Create data pipeline code bases that are testable and extensible, fostering rapid development and evolution
• Improve data quality and pipeline operation through validation and testing

With this practical guide, author Sev Leonard provides a holistic approach to designing scalable data pipelines in the cloud. Intermediate data engineers, software developers, and architects will learn how to navigate cost/performance trade-offs and how to choose and configure compute and storage. You’ll also pick up best practices for code development, testing, and monitoring.