Big Data project involves Engineering practice, Science and Arts, it's an inter-disciplinary pursuit.
Big Data has been defined as: 'high volume, velocity and or
variety of information assets that demand cost-effective, innovative forms of
information processing that enable enhanced insight, decision-making, and process
automation.”. And there are many definitions you can find for Big Data and it
is all about using technologies that were not existed 10 years ago that allow
you to process great amount of data in various forms and create Value/Insight
out of it.
Analytics is part of
the equation, but isn't "Big" data concerned more with the volume and
complexity of data available. The statistical and analytic tools available
haven't changed dramatically over the last ten years but the different types of
data and behavioral interactions that are now captured and stored have
multiplied exponentially. The customers know the value of their Big Data; but
they have difficulty to just store the data while this is probably less than 1%
of the challenge. "Big" data is more about how captured data is
described/codified/structured in a way that enables data to be
"remixed" to solve problems, provide new insights or create new
services.
There are many challenging
pieces in Big Data. Analytics is effectively one piece; but it is only part
of the equation. Departments were receptive to the improvements. The
technology in these departments swings between two extremes: advances that
enable better, faster performance, and equipment that becomes easily inundated
by data demand. So there are many other challenging pieces like
1): Sensors technologies
2): Technologies to transport this tsunami of data
3): Technologies to store
4): Technologies to compute and extract the value from the noise
5): Need billions of brains to find new ideas, "some brains will be engineered"
Clearly there’s an endless field of opportunities to solve important problems
1): Sensors technologies
2): Technologies to transport this tsunami of data
3): Technologies to store
4): Technologies to compute and extract the value from the noise
5): Need billions of brains to find new ideas, "some brains will be engineered"
Clearly there’s an endless field of opportunities to solve important problems
6) Avoid Big Data pitfall, it is called "dark
data" or ‘WONR’-Write Once
Never Read’
Big Data project
involves Engineering practice, Science and Arts (This is the most
interesting part when you need to be creative and find the question you want to
ask). And it’s also about
imagination, and having an understanding of the working principles of an
industry, vertical, or a problem. Then figuring out a way to improve it or
change the mechanism. Analytics is the analysis of historical data done by BI
tool. Big Data is the challenge faced by cross-organizational boundary in
storing, processing, & reporting of nearly 80% of the unstructured data collected
in this world.
Key Big Data issue is
not storage or processing of data. BUT processing "any data
type" from source, and able to send to target "requested"
data type. It can be any data type too. So the potential issues include:1). "Open Data Processing" Frameworks/ Architectures
2). Data Quality, a must if you have legacy data to be moved into big data
3)
4). Self Service – So make sure that all above components work together seamlessly. Also make use of monitoring & automation languages or tools.
Big Data is not just
the storage and data analysis; it's the complete ecosystem from capture with
new instruments or sensors to the emerging neuro-morphic analytics; this is
the complete redesign of industry around the focus on extreme data. The drive
for more data and the technological advances to support that demand aligned at
just the right moment, Now with Big data, you can close this loop of analytics
and action as continuous and iterative process, but it's also created an
insatiable hunger -- a hype factor -- that has taken the industry by storm,
more precisely, it reflects the fundamental shift to converge machine
intelligence with human wisdom.
There's risk in managing Big Data, but there is a much higher risk and failure not making any move
in the Big Data direction in particular in this ultra competitive world. Big
Data is a reality, more than analytics.
0 comments:
Post a Comment