Cricket can explain almost anything in life. For close to three decades the intricacies of reverse swing remained a mystery and a Pakistani dark art. Phlegmatic fast bowler Sarfaraz Nawaz is credited for bringing it on to the world stage. He passed on his secrets to Imran Khan who then gave it to his protégés Wasim Akram and Waqar Younis. The mystery of reverse swing reached a fractious peak in 1992 when England having found no answers to Burewala Bomber’s prodigious banana in-swingers and toe-crushers indulged in their version of fox and sour grapes. They called Pakistanis cheaters. The sardonic Waqar smirked through his now famous retort: give me an orange and I will still get them out. Shortly after the end of the acrimonious series British universities jumped in to the mystery. After all, it was an oddity that defied the existing laws of physics and fluid dynamics.

In less than a year Cardiff University unraveled the mystery of the reverse swing by employing ultra slow motion cameras. Within five years of that Australian cricket coaches were using statistical packages, modeling and plotting software and vector graphics and countless hours of television footage to understand the nuances of every fast bowler. Today the mystery of reverse  swing has eroded so much that a 20-year-old debutant number 11 Aussie Ashton Agar can almost score a century against a strong English side boasting, ironically, of the best reverse swing exponent in recent times. It’s a massive slice of irony for an England that had cried wolf when faced with reverse swing several years back.

Ironies apart, there’s something that’s happened in recent years. It has taken the bite off the once unplayable turn of Ajanta Mendis and unveiled the closely guarded fortress of the forward defensive stroke of Sachin Tendulkar. That something is the meshing together of ultra slow motion videography, statistical tools and packages, hotspots and snickometers and environmental factors like pitch conditions, cloud cover and humidity percentage that generates real-time actionable intelligence. Welcome to Big Data.

Human beings are predictable. We may like to think otherwise and see ourselves as unpredictable ergo unique. But our genetic and biological conditioning makes us look for patterns to create meaning out of the world. The core of human existence is an inherent search for pattern. Pattern is the primary constituent of aesthetics, art and science. In fact the holy grail of all scientific endeavours is to understand and unravel the fundamental principles of the universe; yet another meta-attempt at recognising pattern.

It has helped us to develop foresight, strategic thinking and secure our existence by building a knowledge base. But it also makes us resistant to change, reluctant to accept new ideas and slow in recognising shifts. For long predicting patterns have been a human activity aided and abetted by technology. But in the last couple of years an important threshold has been breached that’s both exciting and chilling. Today predicting patterns is a technological activity with human abetment and involvement. Patterns are the eco-system of Big Data.

Take this for instance. Google and Microsoft do not usually come together. But when they do it means that they have are on to a tectonic shift. Adam Sadilek of Google and John Krumm of Microsoft have developed a software system that can predict your future. What these two did was to attach GPS devices to 300 residents of Seattle and track their lives for three months. That’s 32,000 days and 150 million data points. When these varied data points were meshed together both realised – an alarming one for big brother theorists – that people were boringly predictable. In short their algorithm could predict where you will have your cup of coffee on the morning of August 16, 2015. The researchers sign off their report thus: “Focusing on one individual at a time, we can provide better reminders, search results, and advertisements by considering all the locations the person is likely to be close to in the future.” Definitely exciting. And chilling.

The foundation of Big Data consists of four inter-related blocks: storage, processing power, algorithms and information points. Remove the four and you are left with plain vanilla data. Data becomes big when multiple information points are meshed together and sieved through algorithms to create an emergent intelligence aka patterns. With IPv6 the era of bountiful internet protocol (IP) addresses is making a comeback. Electronic devices will be embedded with chips that will help them connect to the internet and communicate with other devices.

In fact, it’s already happening. Such electronic gateways will create their own digital footprint and they will be captured, stored, sieved and imbued with intelligence. Big data is here to stay, and get bigger. So there are two questions: what can it do for us and what can it do to us? First, Big Data can drastically improve governance and plug loopholes.  For instance, its systems and algorithms can make our roads, highways and signals smart by assessing in real-time traffic flows, jam patterns, violators and even the health of a car and its occupant. But the same algorithm can be used to hyper-focus on an individual car to track each and every one its movements. That’s unsettling.

Second, Big Data can bring about, and already is bringing about, startling efficiencies in business models. With the ability to mesh together varied data points, not previously possible, and derive intelligence out it, Big Data is the new foundation of business analytics. It’s the new competitive advantage and the biggest impact is being seen among retailers.

Amazon and Carrefour have seen up to 60% operating efficiencies after they implemented Big Data solutions. The interesting part is these efficiencies have not come exclusively from smoother logistical and distribution network. A lot of it has come from offering customised and personalised solutions to customers based on tracking and predicting their behaviour. Again, nothing prevents the same algorithm from expanding its hold on you, from the store and into your life. Unsettling, again.

Third, Big Data can provide the always-on blanket network that digital evangelists and futurists have been talking about for decades. Such a network can connect government, big and small businesses, civil society organisations, informal networks and the people. Knowledge transfer, best practices, issues of governance and policy feedback can happen simultaneously and in real time. In creating such an ecosystem the potential for marginalisation and elimination of real life social and community practices, like the highly successful Angadia money transfer system in Gujarat-Mumbai, is high. This can impose a monochromatic way of life on everyone. That’s unsettling too.

For those wanting to escape Big Data, is there a way out? Cricket has a story that might give a clue. In the 1992 World Cup Imran Khan blooded a rotund batsman called Inzamam ul Haq. The pugnacious Aussies, in particular, scented blood and were all over him, sledging him repeatedly. But Haq kept scoring and during the final made a telling contribution helping Pakistan win the World Cup. During a recent interview for a programme on batting greats, Rameez Raja asked him about his incredible calmness. His reply had Raja rolling with laughter: “Rameez Bhai, mujhe kuch samajh hee nahin ata tha (I never understood what they were saying).” The only way out, if one were to take the essence of what Inzamam was saying, is to keep out of the grid. No Twitter, no Facebook, no credit cards, no electronic transfers. In short no digital life. For many that would mean no life at all.