Scalable Architectures - Taming the Twitter Firehose
Handling lots of real-time streams of information, when Twitter alone is producing 330+ million tweets a day and 35 million links to news and media, can be a daunting task.
This talk will cover how to develop a platform that can deal with billions of items per day, perform complex analysis and serve thousands of customers in real-time, by dissecting a large scale architecture into small components and explaining what processes to put in place, what pitfalls to avoid and how to keep the system running. We will walk through several scalability patterns, clever techniques and cutting-edge technologies, high-throughput message queues, nosql databases, monitoring tools, each accompanied by one or more concrete examples.
has been working with large enterprise UK companies
for the past years and is now CTO at DataSift
He's an international conference speaker
and a long-time contributor
to many open source projects
, Book Review
, Cheat Sheet
, Data structures
, Firebird SQL
, Message Queues
, SQL Server
, Zend Framework