Netflix recently hosted their Data Engineering Summit, bringing engineers from different teams together to share many use cases and best practices. Having the chance to watch all the series, It provides valuable insights on various topics, especially in designing and executing products and services at scale.
A big shout-out to Netflix team 👏
Here is the series topics:
The Netflix Data Stack
Netflix building blocks of their Data stack. Learn more about how batch and streaming data pipelines built at Netflix.
📺 https://lnkd.in/epjZJHYt
Data Processing Patterns
Apply different processing strategies for batch pipelines by implementing generic abstractions to help scale, be more efficient, handle late-arriving data, and be more fault tolerant.
📺 https://lnkd.in/eRYnbQ8Y
Streaming SQL on Data Mesh using Apache Flink
Talks about how a managed Streaming SQL using Apache Flink can help unlock new Stream Processing use cases at Netflix.
📺 https://lnkd.in/etW9zfZS
Building Reliable Data Pipelines
Talks about the importance of reliable data pipelines and how to build them covering tools from testing to validation and auditing. The talk uses Apache Spark as an example, but the concepts generalize regardless of your specific tools.
📺 https://lnkd.in/ehQ-DC7i
Knowledge Management — Leveraging Institutional Data
Shares experiences about the Knowledge Management project at Netflix, which seeks to leverage language modeling techniques and metadata from internal systems to improve the impact of the >100K memos that circulate within the company.
📺 https://lnkd.in/e_mT7PyS
Psyberg, An Incremental ETL Framework Using Iceberg
Introduce Psyberg, an incremental ETL framework. Learn about how Psyberg leverages Iceberg metadata to handle late-arriving data, and improves data pipelines.
📺 https://lnkd.in/eCdjx4-C
Start/Stop/Continue for optimizing complex ETL jobs
Shares a case study to demonstrate an effective approach for optimizing complex ETL jobs.
📺 https://lnkd.in/eTNFuBwd
Media Data for ML Studio Creative Production
In the last 2 decades, Netflix has revolutionized the way video content is consumed, however, there is significant work to be done in revolutionizing how movies and tv shows are made. Netflix team showcase how data and insights are being utilized to accomplish such a vision.
📺 https://lnkd.in/ejc8sB3h
#dataarchitecture #dataengineering #data #machinelearning #ai #netflix #datacommunity #iceberg #llm
A big shout-out to Netflix team 👏
Here is the series topics:
The Netflix Data Stack
Netflix building blocks of their Data stack. Learn more about how batch and streaming data pipelines built at Netflix.
📺 https://lnkd.in/epjZJHYt
Data Processing Patterns
Apply different processing strategies for batch pipelines by implementing generic abstractions to help scale, be more efficient, handle late-arriving data, and be more fault tolerant.
📺 https://lnkd.in/eRYnbQ8Y
Streaming SQL on Data Mesh using Apache Flink
Talks about how a managed Streaming SQL using Apache Flink can help unlock new Stream Processing use cases at Netflix.
📺 https://lnkd.in/etW9zfZS
Building Reliable Data Pipelines
Talks about the importance of reliable data pipelines and how to build them covering tools from testing to validation and auditing. The talk uses Apache Spark as an example, but the concepts generalize regardless of your specific tools.
📺 https://lnkd.in/ehQ-DC7i
Knowledge Management — Leveraging Institutional Data
Shares experiences about the Knowledge Management project at Netflix, which seeks to leverage language modeling techniques and metadata from internal systems to improve the impact of the >100K memos that circulate within the company.
📺 https://lnkd.in/e_mT7PyS
Psyberg, An Incremental ETL Framework Using Iceberg
Introduce Psyberg, an incremental ETL framework. Learn about how Psyberg leverages Iceberg metadata to handle late-arriving data, and improves data pipelines.
📺 https://lnkd.in/eCdjx4-C
Start/Stop/Continue for optimizing complex ETL jobs
Shares a case study to demonstrate an effective approach for optimizing complex ETL jobs.
📺 https://lnkd.in/eTNFuBwd
Media Data for ML Studio Creative Production
In the last 2 decades, Netflix has revolutionized the way video content is consumed, however, there is significant work to be done in revolutionizing how movies and tv shows are made. Netflix team showcase how data and insights are being utilized to accomplish such a vision.
📺 https://lnkd.in/ejc8sB3h
#dataarchitecture #dataengineering #data #machinelearning #ai #netflix #datacommunity #iceberg #llm
