What is the relationship between Beer and Diapers? What role did Analytics play in finding the co-relation between beer and diapers? How did business benefit after finding the co-relation? Could a Business Head find the relationship without the use of Analytics?
The story goes like this, some time ago there was a report about Wal-Mart when the company decided to combine the data from its 'loyalty card systems' to from its 'point-of-sale systems'. The former provided Wal-Mart with demographic data about its customers, the latter told it where, when and what those customers bought. The resultant data was mined to find correlations. Some of these were obvious; people who buy gin are also likely to buy tonic water. They often also buy lemons. However, one correlation stood out. Beer and Diapers!
On Friday afternoons, young American males who buy nappies also bought beer. No one had predicted that result, so no one would ever have even asked the question in the first place. Hence, this is an excellent example of the difference between data mining and querying.
After this finding was established, it was easy to back extrapolate from the effect to the cause. Following where the findings:
- Young American males frequently indulged in ritualized carousing behavior with friends of Friday nights.
- Friday night party usually involved the consumption of beer.
- However, if the American male is a father to a baby then he tended to miss his Friday night parties. The father had to take care of the baby on the weekends. So young American males buy diapers after they become fathers of babies!
Now imagine, the proud new father is walking around the retail store on Friday afternoon. The new father knows that there is no way that he is going to get out of the house to join his mates at the bar for the Friday night party. However, there is nothing to stop him from drinking beer at home on a Friday night or all weekend, while he also takes care of the baby at home.
After seeing the results of the data mining, Wal-Mart moved the beer racks next to the diapers racks. The beer and diaper sales went up. All Wal-Mart did was remind the father of that fact that when he buys beer, he also needs to buy diapers for the baby.
Advanced and Pervasive Analytics is the Future
Analytics is not just about data, small data, big data or technology. It is more about finding new insights, new answers and new ideas in the data you already have. For small amounts of data, an excel sheet can provide you with answers. Even today Microsoft Excel is the world’s most widely used Business Intelligence Tool. However, when the data multiples tenfold, we need something more than excel. We need RDBMS data warehouses. When the data grows even further, from gigabytes and to terabytes and petabytes of data, we need more robust and cost-effective tools. We need technology that is more scalable than RDBMS, we need NoSQL databases and Big Data tools.
A Gartner 2015 technology trends report says that Analytics will take center stage as the volume of data generated by embedded systems increases and vast pools of structured and unstructured data within and outside the enterprise are analyzed.
In fact, every app needs to be analytic in nature. With IoT, wearable devices and social media, there is a huge amount of data coming in for businesses. Organizations need to filter this data and deliver it at the right time.
Big data is an important enabler for this trend, but the focus needs to shift to thinking about big questions and answers before the Big Data. The value lies in the answers, not in the data.
Some interesting Facts about Data and how it will impact business
- Every 2 days we create as much information as we did from the beginning of time until 2003.
- Over 90% of all the data globally was created in the past 2 years.
- It is expected that the amount of digital information in existence will have grown from 3.2 zettabytes today to 40 zettabytes by 2020.
- The total amount of data being captured and stored by industry doubles every 1.2 years.
- Every minute we send 204 million emails, generate 1.8 million Facebook likes, send 278,000 Tweets, and up-load 200,000 photos to Facebook
- Google alone processes over 40 thousand search queries per second, making it over 3.5 billion in a single day.
- Around 100 hours of video are uploaded to YouTube every minute and it would take you around 15 years to watch every video uploaded by users in one day.
- Facebook users share 30 billion pieces of content between them every day.
- If you took all the data created in just one day and made DVDs, you could stack them on top of each other and reach the moon – twice.
- AT&T is thought to hold the world’s largest volume of data in one unique database – its phone records database is 312 terabytes in size, and contains almost 2 trillion rows.
- 570 new websites spring into existence every minute of every day.
- Today’s data centres occupy an area of land equal in size to almost 6,000 football fields.
- Between them, companies monitoring Twitter to measure “sentiment” analyze 12 terabytes of tweets every day.
- The amount of data transferred over mobile networks increased by 81% to 1.5 exabytes per month between 2012 and 2014. Video accounts for 53% of that total.
- The NSA is thought to analyze 1.6% of all global internet traffic – around 30 petabytes (30 million gigabytes) every day.
- The value of the Hadoop market is expected to soar from $2 billion in 2013 to $50 billion by 2020, according to market research firm Allied Market Research.
- The number of Bits of information stored in the digital universe is thought to have exceeded the number of stars in the physical universe in 2007.
- This year, there will be over 1.2 billion smart phones in the world (which are stuffed with sensors and data collection features), and the growth will continue.
- The boom of the Internet of Things will mean that the amount of devices that connect to the Internet will rise from about 13 billion today to 50 billion by 2020.
- 12 million RFID tags – used to capture data and track movement of objects in the physical world – had been sold in by 2011. By 2021, it is estimated that number will have risen to 209 billion as the Internet of Things takes off.
- Big data has been used to predict crimes before they happen – a “predictive policing” trial in California was able to identify areas where crime will occur three times more accurately than existing methods of forecasting.
- By better integrating big data analytics into healthcare, the industry could save $300bn a year. That’s the equivalent of reducing the healthcare costs of every man, woman and child by $1,000 a year.
- Retailers could increase their profit margins by more than 60% through the full exploitation of big data analytics.
- The big data industry is expected to grow from US$10.2 billion in 2013 to about US$54.3 billion by 2017.
Arunkumar Nair
Reference links
http://www.theregister.co.uk/2006/08/15/beer_diapers/
https://www.linkedin.com/pulse/20140925030713-64875646-big-data-the-eye-opening-facts-everyone-should-know
http://www.information-management.com/gallery/gartners-top-10-strategic-tech-trends-for-2015-10026168-1.html