First Data Engineering Newsletter
In this issue, we'll start with understanding in detail DataOps, then move on to explain Data Platform architecture at Hurb.com.
Data Engineers of Netflix — Interview with Kevin Wylie#
interviewKevin Wylie, a Data engineer in Netflix, shares his story of how he got into Data engineering. He also shared why he chose to join Netflix,and how the company's culture was different from others.
Delta Lake for Dummies?#
tutorialAnant Mishra, shares the challenge with updating history data, and how Delta Lake comes to rescue. He describes how to use Delta lake through code snippets to easily manage updates and bring data reliability in a scalable way.
How do you keep your pipelines from data lake to the DWH idempotent?#
discussionThis discussion on Reddit is around bringing idempotency to a data pipeline which involoves migrating data from S3 to Datawarehouse.
Growing the Delta Ecosystem to Rust and Python with Delta-RS#
Delta-RS is a project with an objective to bring delta lake outside Spark ecosystem. Written in Rust programming language, it allows to build native bindings for Golang, python, NodeJS and many other programming languages.
video