Sunday, July 28, 2013

Five Key Factors in Driving Big Data Project Success


Big Data is hot trend, but Big Data project has very low success rate, so the real question is how organizations deal with the situation. Do they continue to pour resource into a failing project, or do they shut it down early to learn what they can get from the experience and move forward? What are the key factors to be considered in order to drive Big Data project success?


1.  Have a Business Goal in Mind 

It needs to be clear what the outcomes of the project will be, who can/will benefit and how they can/will benefit. It is critical that at the end of the day you have the right team in place to use the insights that come from the projects, what better decision will you be able to make as a result of the analysis and how will you drive and track those better decisions,

It is understandable that many BDA projects are started but in a competitive internal environment, so most of these projects have no defined business driver or requirement. As long as Big Data projects continue to be treated as pure IT projects, the failures will continue to pile up. Getting the machinery running is generally not the issue, it is the failure to deliver value because the end-users and core decision makers are ill equipped to "get it" and can not extract and use the inherent value in the mass of data at their fingertips. Converting big data, into understandable, digestible, useful and actionable information is the business purpose from Big Data project.

2 Assume Data Imperfection & Immature Technology 

Data will be incomplete and less than pristine (duplicates, incompleteness etc.) in the Big Data analytics, and the analytics must accommodate the real state of the data. By the time a project gets going, there's probably around 30% more data (both in volume and diversity) available. The analytics will need to be refined and adapted because you don't know everything when you start. Your framework for developing and deploying the analytics should support an iterative development approach

Lack of resources, high data volume, new technology that is not production ready, variety of data sources and high data volume is plaguing the industry. This is also encouraging innovation at a very fast rate, but it will take time for industry to mature and stabilize and then the success rate will increase and will come more in line with industry standard success rate of projects or even higher. 

3. Big Data Talent  

The pitfalls of most of these projects is because Big Data seems like a disruptive technology and the old users of the systems are not equipped with these new concepts and will go a long way to see it fails.

 There are generally two distinct skill-sets, technical on the architecture side and the analytical skill set to create action based on the information. You need a solid team for both to have a shot. If not you wasted time, money, effort and potentially damaged  prospects for the next projects coming down the pike. For example if your new project can tie your CRM and other support systems all up in a nice wrapper and you can see how purchases, the loyalty program and your customer service activities can help someone put their customers into groups, it doesn't do any good if there isn't someone in marketing or in a specialized analytical role/group that can DO something with that information, like recognizing triggers that a customer is more/less receptive to a certain kind of marketing/up-sell/cross-sell messaging or that there seems to be a risk of the customer leaving. 

"The statistician is no longer an alchemist expected to produce gold from any worthless material offered him. He is more like a chemist capable of assaying exactly how much of value it contains, and capable also of extracting this amount, and no more. In these circumstances, it would be foolish to commend a statistician because his results are precise or to reprove because they are not. If he is competent in his craft, the value of the result follows solely from the value of the material given him. It contains so much information and no more. His job is only to produce what it contains." Ronald A. Fisher

4. Collaborate Effective Processes with Right Talent 

Poor planning and coordination are organization and project management related, however, the nature of most big data projects requires the ability to read and rapidly process variety of data, high volume data sources, making coordination critical. Things get complicated very quickly as the structure of these data sources and the business needs continue to evolve. If organizations are unable to staff the right kind of data scientist(s) who have all the skills to take the project forward, and understand the business needs, they end up staffing a lot of individuals to fulfill the skill set needs. This creates a communication and coordination issues...communication touch points increase with the square of number of individuals involved in the project .. and we know how things go from there.

That has remained constant, or at least linearly increased, is the number of prospective users of data who seek magic over math, confuse computational speed with statistical precision, and generally would prefer the answers they seek over the answers that past analysis has given them regardless of scale.

On the business side, Businesses haven’t yet fully understand how to redesign business processes so that they can take advantage of what big data can potentially tell.  Poor cooperation, poor planning and lack of skills should be categorized under organizational failures rather than Big Data centered project failures though.

5. The Big Data Life Cycle Management 

The life cycle of big data is to evaluate capacity planning for Big data services through timely intervals and expected growth cycles. The velocity and a metric for relevant time sensitivity is important in the collection processes of what you are analyzing and for what purpose pursuant to the predictive analytics needed to support the model coupled with levels of complexity.

 For a couple of decades, IT has been pushing towards a "single source of truth" in data management. A lot of money and technology is aimed at getting everything to line up and agree. There is a whole "Master Data Management" industry evolved out of the desire for consistency and correctness - never mind the legal requirements to get the data "right". Big data often threatens all this and makes a lot of IT shops nervous, so projects get sidetracked, slowed or abandoned because it takes the new angle and multi-dimensional lens in capturing insight upon Big Data.

Until organizations get an operational context that can respond to new capabilities, the project can be technically successful, but never deliver enough business value to gain support. In that situation you probably won't get to do another one.... Big Data has all tough characteristics, how to build the Big Data modeling framework as a systematic approach to master volume, velocity and complexity needs to be thought through.

Mastering Big Data is more as journey, not a one time project,. It is the nature of the beast that "noise over signal" dominates every epistemological effort. A certain amount of project failure ought to be tolerated if innovation is being sought. With considering all key factors listed above, organizations can make progress in driving Big Data success.

4 comments:

Hi every one, here every person is sharing these kinds of experience, so it's nice to read this website, and I used to
go to see this blog daily.

My homepage annonce libertine


Tag: PM201A52. Let me share all of you about #5 Tips for Project Management Success,, I hope you enjoy it

1. Plan your day using time management techniques

As a project manager, time management skills are essential because you are dealing with a wide range of tasks that demand a quick turnaround time. Planning your day will go a long way in keeping you organized and increasing your productivity. Assist your task planning by using project management software which helps you track the work of you and your team.

If you are not very tech savvy, a simple to-do list can also be a great organizational tool. Prioritize your most important tasks by putting them at the top of the list and less important ones at the bottom. Having a visual plan of your daily tasks helps to keep you on track and aware of time.

Related post: Free ebook 104 secrets to become a great project manager

2. Include stakeholders in important project conversations

While you will have plenty of responsibilities regarding the project, don’t neglect your clients.

Good communication is essential is keeping both parties informed of project progression, curtailing scope creep, and apprised of changing requirements. Some clients may have different expectations when it comes to communication, so make sure to establish the frequency and type of communication (like emails, phone calls, and face-to-face conversations) at the beginning of your project.

Establishing communication expectations early helps alleviate stakeholder uncertainty about communication frequency and delivery.

3. Regularly communicate with your team

Daily team communication helps keep misunderstandings and unclear requirements under control. Keeping your team informed in every step of the project is essential to project management success.

For example, a study published by Procedia Technology found that good communication skills were the cornerstone of project management. The study examined over 300 “construction project managers, architects, construction managers, engineers and quantity surveyors” and their successes and failures on various construction projects.

4. Anticipate project setbacks

Even the best-laid plans often go awry.

Remember that even with a high amount of planning and attention to detail, your project may still encounter some challenges. Pay attention to complaints from stakeholders or colleagues, and other warning signs, like a missed deadline or cost overrun, that there may be a problem.

Preventing a crisis will keep your project running smoothly, save you a lot of time, and keep you, your team, and your stakeholders confident in progressing with the project.

Unfortunately not every complication can be avoided. Crisis management skills are essential for dealing with the unexpected. Project managers need to be flexible and pragmatic. Improvise and make sharp decisions when needed.

Related post: 92 free project management templates

5. Stay focused on the details

A common problem project managers encounter is having the project aims not aligned with the organization’s objectives. A great project manager will strategize a plan for the project to lead back to the overall success of the business.

Know your project’s scope by heart and avoid wandering outside of the project’s requirements. It’s too easy to get lost in minor details and forget what your focus is, so a well-planned project scope is essential for success.

And final, you should use KPI to measure effectiveness of the project, here are full list: 76 project management KPIs



IEEE Final Year projects Project Center in Chennai are consistently sought after. Final Year Students Projects take a shot at them to improve their aptitudes, while specialists like the enjoyment in interfering with innovation. For experts, it's an alternate ball game through and through. Smaller than expected IEEE Final Year project centers ground for all fragments of CSE & IT engineers hoping to assemble. <Final Year Projects for CSE It gives you tips and rules that is progressively critical to consider while choosing any final year project point.

JavaScript Training in Chennai

JavaScript Training in Chennai

The Angular Training covers a wide range of topics including Components, project projects for cseAngular Directives, Angular Services, Pipes, security fundamentals, Routing, and Angular programmability. The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

Google brain is working in the data warehouse solution to make it a huge success for the world. We hope that society will soon use AI devices at a reasonable cost.

Post a Comment