"Big data is the new business black" ... enterprises
will invest more than $US120 billion by 2015 in data capture and
analysis across hardware, software and services.
Photo: istock
Customer information is suddenly worth billions.
There's another mining boom you may have missed. It too
involves paying young people six-figure salaries in their first jobs,
and exploring deeper for resources which may have been previously
overlooked. But it's not about driving trucks or digging holes. It's
about building algorithms and crunching facts and numbers. It's mining
for data.
Big data is the new business black. It's a catch-all
phrase for the billions of transactions and other bits of information
about their customers, suppliers and operations logged by businesses and
governments the world over every day. Yesterday's storage problem has
become today's strategic asset. Turns out there's gold in them thar
files.
''This is the biggest industry that people are only now
starting to talk about,'' says Anthony Goldbloom, a 28-year-old former
Reserve Bank of Australia statistician who has moved his start-up data
analytics business, Kaggle, to Silicon Valley where NASA is among its
clients.
''The whole place is big data mad,'' he says.
''Industries like banking, insurance, and increasingly pharmaceuticals
are competing on the back of predictive models that get built [by mining
data].''
Enterprises are using data analysis not just to improve
their everyday business processes, but also to build predictive models
of consumer behaviour. Retailers, telcos, airlines, hotels, health care
and credit card companies are among those with information-rich customer
data. In Australia, ''only really leading companies have realised this
as an opportunity'', Goldbloom says. To his knowledge, Telstra, Myer,
the University of Melbourne and the NSW Roads and Maritime Services are
among those known to have applied large-scale data analysis to their
operations.
A 2011 report by McKinsey and Company said using big data
to drive efficiency and quality in the US health care industry could
create $US300 billion value a year, including cost savings. Using
personal location data to sell services to individuals could ''capture
$US600 billion in consumer surplus'', it said.
Global market intelligence firm IDC has estimated
enterprises will invest more than $US120 billion by 2015 in data capture
and analysis across hardware, software and services. IBM is investing
heavily, now boasting almost 9000 ''business analytics and optimisation
consultants'', 400 researchers and a global network of ''analytics
solutions centres'', aiming for $16 billion in business analytics
revenue by 2015.
''There is no end in sight. I think it is an endless
opportunity,'' the company's chief data analytics guru, Jeff Jonas,
says. Among other things, he is working on what might best be described
as a digital concierge service. Customers who opt to give the service
access to their personal information - such as one or more of their
social media accounts and their digital personal organiser - would
receive timely and, according to Jonas, helpful prompts to guide their
actions and purchases.
Imagine, Jonas says. ''It says, 'I can see from your
Facebook that you like Metallica, and I can see you will be in New York
next week, and Metallica is there on the same night. Would you like to
get a ticket?'
''You could subscribe to a service that would be so
precise in its prescriptions that you could feel like it saves you time
and money,'' says Jonas, IBM Fellow and chief scientist, IBM Entity
Analytics. He acknowledges people's possible discomfort at the idea of
unsolicited advertisements, but insists users of the service would come
to see it as ''friendly'' and ''genius''.
Another company capitalising on the growing expanse of
data, and one of IBM's rivals, is the Silicon Valley data-sorting firm
Tibco.
The company's chief technology officer, Matt Quinn, says
Tibco's products aim to give clients the right information, at the right
time, in the right place and in the right context.
One of its more unusual client groups, casino operators,
use the company's software to gauge gambler satisfaction. The system
collects data each time a gambler uses their casino loyalty card - be it
for gambling, purchasing food in the restaurant or paying for snacks
from their room minibar - to create a pattern of their behaviour.
The system can see, for example, that the last few times
Gambler X visited the casino, they stayed for two days and lost between
$200 and $300, then promptly left the gaming floor, spent no more money
and went home the next day.
''As a casino operator you don't want that, you want to make people happy and spend more money,'' Quinn says.
Enter Tibco's event processing software. When the system
detects the customer is falling into a particular pattern, such as the
consistent losing streak that caused them to leave during their last
visit, it sends an automatic note to a gaming floor attendant to offer
the person a free meal, or ticket to a show.
The idea is to distract the gambler long enough that
they'll come back later and continue to play and lose money, albeit in
more palatable amounts. While no Australian casinos use Tibco's
products, they are beginning to show interest, Quinn says.
Department stores, whose loyalty programs amass gigabytes
of data on their shoppers' every purchase, have also begun using
Tibco's event processing software to model their customer's spending
patterns and predict their next buy.
Quinn says the software would pick up that every year
Customer A spent a fair bit of money around the beginning of October,
which could suggest they had a few birthdays around that time.
This would be a perfect time for the store to offer this
customer a discount or voucher, a more effective and targeted marketing
strategy than TV advertising and catalogues, he says.
''The systems we're building are helping [companies] better understand who their customers are.''
Tibco products also analyse data to keep customers loyal.
In late 2010, Tibco bought the technology startup Loyalty
Lab, whose software platform specifies the best time for a company to
engage customers in their loyalty program based on data it has collected
on previous customer interactions.
''It's technology and psychology rolled into one,'' Quinn says.
Trained loyalty scientists build systems that trawl
through a company's customer behaviour data to predict certain events,
such as what might prompt them to leave.
Loyalty Lab's director of global solutions strategy,
Michael Greenberg, says for telcos, for example, there are several
events, such as phone calls dropping out, the end of a contract, or more
than two complaints, that suggest a customer may be getting ready to
take their business elsewhere.
That is when Tibco advises companies make contact with these customers with an offer, he says.
While big businesses make up the bulk of Tibco's clients,
Nasdaq, the US Major League Baseball, the Department of Homeland
Security and major hospitals also use the company's data-sorting
software.
In 2010, scientists at the Walter and Eliza Hall
Institute of Medical Research in Melbourne began using Tibco's
visualisation software, Spotfire, to help find new drug therapies for
diseases like cancer and malaria.
To locate potential drug candidates, the researchers run
tests on tens of thousands of compounds each day, generating hundreds of
thousands of data points for each potential therapy.
The co-ordinator of the the institute's high-throughput
laboratory, biochemist Kurt Lackovic, says the unit is capable of
running 80,000 reactions a day.
Spotfire takes the results of each reaction and presents
them in diagrams, making it easier for researchers to spot trends,
outliers and errors rather than searching for a needle in a rapidly
expanding digital haystack.
Hype notwithstanding, big data will turn out to be a
useful value adding tool rather than a ''panacea to solve everything'',
says Robert Hillard, Deloitte Consulting technology lead partner.
The privacy issues raised by its use have yet to be
resolved, he says. It is crucial people are given the choice between
opting in and opting out of the use of their identifiable personal data,
he says. Companies approaching customers need to be ''utterly
transparent'' about how they know what they know about a customer.
IBM's Jonas, whose personal motto on privacy issues is
''don't surprise the customer'', says it's important for people to
realise how commercially powerful the data which locates them in time
and space is, such as that communicated by GPS devices in cars and
mobile phones. People are putting more data online about themselves than
ever before, yet when signing up for the latest ''irresistible''
service, few bother to read the terms of use, he says.
