Need To Know What Big Data Is?

Just like cloud computing, Big Data has become a hot topic of 2012. What lies behind the hype?

In this hyper-competitive world, forcing the rival company to continuously reduce margins, business sees big data as an opportunity to get the ultimate weapon in the fight for survival. As predicted by the experts, by the end of 2012, over 90% of the Fortune 500 will actively prepare for at least a few initiatives in the region of big data. What is big data and why they have to worry about?

What Is Big Data?

The Simplest Definition

The term “Big Data” refers simply to the management and analysis of large amounts of data. According to the report, Big Data: The next Frontier for Innovation, Productivity and competition, the term “Big Data” refers to data sets whose size is beyond the capabilities of typical databases (DB) Named by, storage, management and analysis of information. In addition, the world’s data repository is definitely growing. As presented in mid-2011, the report analyst firm IDC “Digital Universe Study” predicted that the total global volume of data created and replicated in 2011 could be around 1.8 zettabyte (1.8 trillion. gigabytes) – about 9 times more than what was established in 2006.

More Complex Definition

However, “Big Data” suggests something more than just an analysis of huge amounts of information. The problem is not that the organization creates huge amounts of data, but the fact that most of them are presented in a format that poorly match the traditional format of a structured database – web-logs, videos, text documents, or machine code, for example, geospatial data. All of this is stored in a variety of different stores, or even outside the organization. As a result, corporations are able to have access to the huge volume of the data and do not have the necessary tools to establish the relationship between these data and make them the basis for meaningful conclusions. Adding the fact that the data is updated more often, and you have a situation in which traditional methods of analysis of information cannot keep up with the huge volume of constantly updated data, which ultimately paves the way to big data technologies.

Best Definition

In fact, the concept of big data involves working with a huge amount of information and a variety of very frequently updated data and located in different sources in order to increase efficiency, create new products and improve competitiveness. Big data has a combined engineering and technology that extract meaning from the data at the extreme limit of practicality.

Real Trend or Just a Hoax?

Doubters

Not everyone in the IT industry believes that big data has the same “high” value, as the myth created around it. Some experts say that the access to the heap of facts and the ability to analyze does not mean that you do it right.

Some experts are arguing that this is a dubious competitive advantage – to spend hours pondering the data that everyone has, and that the idea of big data is the use of new information and draw conclusions, which no one did. Even in this situation, it is important to quickly understand the meaning and context to data, and in some cases, it can be difficult.

When Will The Time Come For Big Data?

Experts do not think that companies should dive into the topic of big data, if they do not believe that it will bring answers to their questions.

The leaders of the industry should be able to describe the problem they want to solve with the help of big data, whether the acceleration of existing processes (for example, to detect fraud) or the introduction of new, previously considered impractical or too expensive (for example, streaming data from” smart “sensors and assessment of the impact peak of meteorological information to fluctuations in demand). If you cannot articulate the purpose of their efforts in the field of big data, do not begin to deal with them.

This process requires an understanding of what information is needed to make better decisions. If the best way of obtaining such information is the analysis of large data, the more likely it is time to start moving in that direction. If such information can be obtained using conventional technology business analysis, it may be time to use big data, which is not yet come.

How Big Is The Difference Between Business Intelligence And Big Data?

The business process analysis is a descriptive analysis of the results achieved by the business during a certain period, while the speed of processing large data leads to predictive analysis that can offer business advice for the future. Technology allows large data to analyze more data types in comparison with business intelligence tools, which allows focusing on structured repositories.

Working with large data is not like the normal process of business intelligence, where a simple addition of the known values yields the result: for example, the result of the addition of data on paid bills become sales for the year. When working with large data result is obtained in the course of their treatment by the successive modeling: first, a hypothesis, based statistical, visual or semantic model, based on its fidelity to the hypothesis is checked, and then makes the following. This process requires the researcher or interpreting visual values or making interactive queries based on the knowledge or the development of adaptive algorithms, “Machine Learning“, the ability to obtain the desired result. The lifetime of this algorithm can be quite short.

Pitfalls

Do You Know Where Your Data Is?

It makes no sense to implement a solution for working with large data-only to realize that critical information scattered throughout the organization cannot reach unknown places. Most companies already do not possess all the information within their own organizations, and just die in the attempt to analyze the additional information obtained from processing of large data.

Lack of Skills

Even if a company decides to implement technologies to handle large data, it may encounter difficulties in attracting qualified employees. From specialist to work with the data (as well as their intellectual analysis) requires a unique combination of skills, including a strong background in mathematics and statistics, a deep knowledge of statistical tools such as SAS, SPSS, or based on open source statistical package; ability to find patterns in the data. All of this must be supported by a good knowledge of the subject area and excellent communication skills to understand the problems of intelligence and how to resolve them.

Finding specialists that meet this combination of requirements is not easy, as it takes about half a million managers and analysts to work on the analysis of big data and make decisions based on the results.

For staff, it is important to fully understand what they are doing. Big Data form the relationship, and then you are the only solution, whether they are reliable in terms of statistics or not. The number of permutations and possibilities that you can make means that many people can affect the result.

Personal Data

Tracking of personal customer data in order to stimulate demand seems an attractive idea for the seller, but does not seem necessary for the purchaser of this product. Not everyone wants to make their life become the subject of analysis and depending on how you will develop rules for the use of personal data in a variety of different countries, companies will be cautious in their plans to work with big data, including methods of data collection. These rules may result in fines in the case of very aggressive policy in this area, but even greater risk may result in the loss of trust.

Safety

Customers trust companies to ensure the security of their personal data. However, since large data represents a completely new area for these products, which were developed without adequate attention to safety issues, despite the fact that the vast amounts of stored information make the task of ensuring the safety of their storage, is more important than ever before.

Over the last year or two, there have been several well-publicized cases of leakage of confidential data, including the leakage of information about hundreds of thousands of customers. The government promises to review the laws on notification of cases of leakage of confidential information from the time of the 2008 analysis of the security of personal data. The government advises companies to be prepared for a situation where they will be required to inform customers about the cases of loss and theft of personal data. In addition, government said that they would take tough measures against organizations that are having an irresponsible storage of sensitive information.

Steps to Large Data.

If you decide to move in the direction of big data, it is important to be fully prepared and to approach the project in an organized way, to answer a number of questions.

What would you like to know? Here we have to decide what you want to find out with this big data, which we cannot get from the current system. If the answer is – nothing, then maybe you should wait to start this project.

What are your information assets? Can you build in the asset system of cross-references to certain laws and formulate lessons? Is it possible to create new products for data on these assets? If not, then how can you make this possible?

Once you figure it out, time to prioritize. Select the most potentially valuable area for the application of the techniques and technology of big data, prepare a business case to run “pilot” (proof of concept), drawing attention to a set of skills that you will need in the implementation. You will need to speak with the owners of the data to get a complete picture.

Run a pilot project and make sure you have well-formulated test completion, to assess the results. This may be a good time to offer the owner to take responsibility of information resources for the project.

For the conclusion of “pilot project”, estimate how it works, are you getting real conclusions and recommendations? Whether the work is paid off? Can this project be replicated in other parts of the organization? Is there any other information that can be included in it? This will help to answer the question – whether to launch a full project made by “pilot”, or something must be correct?

So what are you waiting for? Time to think about the Big Data.