First Data Engineering Newsletter

Welcome to the first data engineering newsletter. In this issue, we'll start with the company culture of Netflix, as explained by a Data engineer, and then we'll quickly look at Delta Lake.

Data Engineers of Netflix — Interview with Kevin Wylie#

interview

Kevin Wylie, a Data engineer in Netflix, shares his story of how he got into Data engineering. He also shared why he chose to join Netflix,and how the company's culture was different from others.

Read More

Delta Lake for Dummies?#

tutorial

Anant Mishra, shares the challenge with updating history data, and how Delta Lake comes to rescue. He describes how to use Delta lake through code snippets to easily manage updates and bring data reliability in a scalable way.

Learn More

How do you keep your pipelines from data lake to the DWH idempotent?#

discussion

This discussion on Reddit is around bringing idempotency to a data pipeline which involoves migrating data from S3 to Datawarehouse.

Read More

Growing the Delta Ecosystem to Rust and Python with Delta-RS#

Delta-RS is a project with an objective to bring delta lake outside Spark ecosystem. Written in Rust programming language, it allows to build native bindings for Golang, python, NodeJS and many other programming languages.

video