Learn about DataOps, Data Platform architecture and more
In this issue, we'll start with understanding in detail DataOps, then move on to explain Data Platform architecture at Hurb.com.
Getting to Know DataOps#
articleIn this article, Loh Meng Xin describes how to excel in DataOps by indentifying patterns in the type of data problems, document the datasets for better discovery, handle changes in data sources and finally providing better data governance.
Data Platform Architecture at Hurb.com#
articleIn this article, Lucas Rolim describes how Data Engineering team at Hurb.com establishes single source of truth in order to make data trustable.
Nested Data Types in Spark 3.1#
tutorialThis tutorial by David Vrba describes the data type, Struct and the different transforming functions.
Data Optimization for Compacted Partitions#
In this article, David Smith explains how storage and query performance can be optimized using compacted partitions.
Data Mesh at CMC Markets: Past, Present and Future#
Learn how the team at CMC Markets approached the data mesh and the different architectural and technical challenges they faced.
video