Every day digital advertising companies generate large amounts of data. These are Big Data companies, and the massive data they take in every day is one of the most important assets for their business.
At a recent DataWeek San Francisco panel, moderated by Russell Sullivan of Aerospike, adMarketplace CTO, Mike Yudin, joined BlueKai CSO, Alex Hooshmand, and LiveRail CTO, Andrei Dunca, to speak on the topic Instant Ad Response via Real-Time Transactional Analysis.
How much data is Big Data? Yudin, defined it very simply:
In the case of real-time response, more data than can feed into a servers’ physical RAM and more data than traditional databases can deal with, and querying should happen very fast in both cases; 1 millisecond or less in real time, and 1 second or less for the user interface.”
Hooshmand tells his description more literally:
We do that about a trillion times a month, and last month we actually hit run rates of about 2 trillion times a month. So, all of that information has to get captured because we're also the platform that accounts for what's happening with this data. We have to understand where the source of the data is, where it's going, and in some cases we have to charge people for it. So, there's a class of problems around taking that data and aggregating it in a way that can then show up in our reporting. There, the big problem is just the sheer volume of data that you have to deal with.”