Jeff Meyerson talks with Frances Perry about Apache Beam, a unified batch and stream processing model. Topics include a history of batch and stream processing, from MapReduce to the Lambda Architecture to the more recent Dataflow model, originally defined in a Google paper. Dataflow overcomes the problem of event time skew by using watermarks and other methods discussed between Jeff and Frances. Apache Beam defines a way for users to define their pipelines in a way that is agnostic of the underlying execution engine, similar to how SQL provides a unified language for databases. This seeks to solve the churn and repeated work that has occurred in the rapidly evolving stream processing ecosystem.
Frances Perry on Apache Beam
Find Your Interest
- Pete Koomen on A/B Testing
- Everyone Should Use Generics Except You
- 차예련, '학폭의혹' 속 SNS 관리ing..프로필 이어 '공구링크' 삭제·팔로잉 정리[Oh!쎈 이슈]
- My life before & after automated testing
- Llewelyn Falco on Approval Testing
- PODS & Red Rover: Pete Warhurst
- Learning Scala at SoundCloud
- Kevin Hu on Data Observability
- Testing SQL for BigQuery
- Features You Don't Want






