“Big data” also is not equal to “great wisdom”

cloud network hunting note: technology developers and media early painted a picture for us the upcoming “era of big data.” “Big data” omniscient omnipotent; With the support of the “big data”, the company operating efficiency by leaps and bounds; “Big data” can also help you make the most informed decision, make you invincible. Is not too great! But hunting cloud network editing jun to remind everybody, just like all the high-tech propaganda, “big data” is inevitable hype. So, do you still believe in the future?

in recent years, the “big data” has spread. Technology developers and the media frenzy of propaganda, how could you not know that the “big data”? Even if you don’t know there is always heard of it. Let’s take a look at how they trumpeted the so-called “big data” : “big data” omniscient omnipotent; With the support of the “big data”, the company operating efficiency by leaps and bounds; “Big data” can also help you understand the data, make the smartest decisions, make your company all the time full of competitive advantage.

how tempting propaganda! Of course, we can’t say one hundred percent reported against the facts. People just for high-tech publicity is always too optimistic ahead. In fact, many companies are found at the current conditions to realize the “big data” difficult, ideal is very plentiful, reality is very skinny. Indeed, in the aspect of data collection and processing, may have a considerable advantage. But the real use of these data, and with the help of the better decisions it is another thing. So where is the problem? Said most companies in the “big data” and “understanding” between of big data, the lack of certain important link. If this problem is not solved, so people just is a pile of seemingly useful data, but hard to dig out the useful value.

as a senior in the industry of silicon valley recently revealed that although the recent startup activities and financing conditions, large data collection and data processing seems to be widely followed, but a huge gap between reality and expectations are still unable to turn a blind eye. He said, “big data haven’t really into big knowledge, insights and wisdom.” With their predictions, we from real era of “big data” has a long way to go.

 big data

hype and the reality, cannot confuse STH with STH else

we want to capture value from big data method is as simple as possible, such as import data, run the program, the visionary conclusions finally. Do you think it possible? If wisdom so readily available, that everyone can be a jobs. In fact, gain valuable information from large data than “import, run, output trilogy” is much more complex. “Data prediction: the strategy of Big Data (Data Divination: Big Data Strategies),” the author of Pam Baker (Pam) Baker, said the Data directly answer instance does exist, but is only found in certain cases, rarely happen. We don’t hope that the exception, we need is a universal law.

“perhaps, some would argue that we can cite many examples, in these cases, data often can give a very clear answer. Such as prediction analysis can accurately predict the plane or a dead time of the parts and components of water supply system can also tell us the best time for replacement parts, in order to maximize the use of before in the old part scrapped its residual value.” Baker explains.

“but,” she went stressed that “more cases, we can’t get to the answer directly. You can choose from a number of possible actions one or do nothing, particular case is particular analysis, this is the reality we face.”

baker fits. Some decisions it is based on the data. Data is not “cold number”, they are “elves” sentimentality, as Bruce springsteen sings in a song, they need to be “a little milk of human kindness”. People can by developing good indicators and powerful algorithm to data mining. But this is far from enough, people must through their own understanding and insight to truly understand the “inner world” of data, to make full use of the value behind the data. Some data is very “obvious”, some is “euphemism”, we cannot treat as the same.

 big data

algorithm of the limitations of

a step further, we hope that big data can make enterprise users instant access to the data directly, so that they can, charmed life anytime and anywhere to make every one of the best decisions. Desire is a good thing, just by our current technical conditions, we also can not reach such a complex magical level.

to do this, first of all, we need enough data expert to help us to analysis the data, extract useful information from large amount of information. Together with Kholsa Ventures invested several big data technology companies (such as Parstream) investors Keith said, the company is very need an expert to guide data processing complex data analysis, but the majority of business users is difficult to do this.

the Persian said, you will need these experts to develop the application data and algorithms, to undertake a large amount of data research task. But in already have these data expert company, the data is not experts have been engaged in these advanced complex data work, perhaps in part because they need to take the time to deal with some simple data analysis. Data of experts can greatly be buried here.

in the ideal case, pull the Persian continued, experts developed a set of tools, data when there is a party need answers can quickly find out the answer to analysis throughout the organization. In this day and age, speed is everything. The last thing we want to see what happened is that when we urgently need answers, we can only hope that experts, data and passively waiting for.

the starting point is good, but the problem is that even the smartest people developed the most complex algorithm, for complex problem is still not the most direct answer. No matter how complex algorithm, also can’t do overall consideration, the difficult to measure the specific factors of more helpless. If a method can do all of these, it is a human brain, the more trouble.

 big data 3

I need a good “midfielder hand”

a baseball game that can help us better understand the limitation of the algorithm – level two players, they can. Data geeks will tell you that after years of research and development, they created Sabermetrics algorithm, can provide you with choosing the best players of all decision-making information you need. They also developed a whole series of data statistical algorithms, such as “replace winning percentage (Wins Above Replacement)”. FanGraphs explanation to replacement “odds” are as follows: “if a player can’t play wounded, their team had to find a minor league baseball player or team” lesser “doing the substitute bench, loss how many?” To this, they adopted a series of standards to measure the winning percentage difference between the two.

if the complexity of this algorithm only used to accurately measure the value of the player, that’s no big problem. But there are some problems, such as a player performance under pressure? He practiced hard? He’s the captain of the which kind of type? Or with his players how to get along with? All of these problems how to use the algorithm to calculate? Don’t these problems is important? If you want to into the consideration of the algorithm, and how to quantify these factors?

pure data analysis of the following will tell you everything can be quantified, maybe they said yes. But I also did see a lot of the same players, under the condition of nearly the same, they have the gap, though their performance on the data analysis of it should be very close.

in the enterprise, human resources experts in recruiting free programmers will encounter similar baseball player. You may have two professional skills level of applicants to apply for the position, but one of the better interpersonal skills, good cooperation with colleagues, and the other candidates are hard to get along with and cooperation, apparently only from your resume, it is hard to see how these “soft power”. Even has the support of a large amount of data, it is difficult to take into consideration all aspects to the possible results, especially and involving people.

 big data 1

aware that one false step will make a great difference

any a responsible doctor rigorous to tell you, even if both the patient’s symptoms are very similar, the treatment is not the same, still need to be in strict accordance with the individual to decide, the difference of the age, weight and other health problems and special factors and so on, will affect the final treatment effect.

take the medical procedures used in the intelligent analysis platform for IBM’s Watson. When I asked a friend about some doctor recently started using Watson auxiliary diagnosis and treatment, he immediately fry. He insisted his own health problems and treatment does not need a machine to decide. His worry about completely reasonable, but in Watson’s case, the machine did not directly give the doctor can follow blindly answer, only signs, patient information, according to the existing conditions and combined with the current conditions of scientific research results, reference scheme of the treatment is given.

as I described before data experts, doctors are also very busy, they could not give patients a doctor while also be familiar with all the latest developments in the field of themselves. There are so many related research (of course this is a good thing). So they need Watson auxiliary. Watson, able to quickly filter the current study, but I still need a doctor according to the actual situation to determine the final direction of treatment. I prefer to put this process is called the art of science. Brought us infinite possibilities of knowledge, but in the end it’s still is a doctor and not a machine.

enterprise will also face similar uncertainty, this requires human intervention, using their knowledge, through the power of the data, make a decision to uncertainty.

 big data 2

in the future we can walk far?

a lot of the time machine can give people need to take years to be the answer and vision. Baker pointed out, such as big data has been in helping us more profound understanding of the disease, especially cancer, there are many aspects are human researchers has never been involved in. “There is no big data to provide us with enough information, we will never find the best treatment plan (at least in recent years, there is no hope. Here, I want to say is, ‘big data’ can be very accurate.”

and she also believes the machine learning ability in the near future will reach a enough mature stage. When the machine may do more for our decision, after all, because the human brain capacity is limited, can’t deal with the use of all available information.

I can’t say that her thought is wrong, for now, however, the ability of collecting and processing data has gone far beyond the comprehension of these data. Baker says, forecast analysis has been developing in advance, sometimes data can answer directly, but in more cases, it is still a complex interactive process. nullnullnullnullnull