Big data is a term defined for data sets that are large or complex that traditional data processing applications are inadequate. Big Data basically consists of analysis zing, capturing the data, data creation, searching, sharing, storage capacity, transfer, visualization, and querying and information privacy.
While the term “big data” is the new in this era, as it is the act of gathering and storing huge amounts of information for eventual analysis is ages old. The concept came into existence in the early 2000s when Industry analyst Doug Laney the definition of big data as the three categories as follows:
Volume: Organizations collects the data from relative sources, which includes business transactions, social media and information from sensor or machine-to-machine data. Before, storage was a big issue but now the advancement of new technologies (such as Hadoop) has reduced the burden.
Velocity: Data streams unparalleled speed of velocity and have improved in timely manner. RFID tags, sensors and smart metering are driving the need to deal with torrents of data in real time operations.
Variety: Data comes in all varieties in form of structured, numeric data in traditional databases to unstructured text documents, email, video, audio, stock ticker data and financial transactions.
In SAS, we consider two additional dimensions with respect to big data:
Big data works on the data produced by various devices and their applications. Below are some of the fields that are involved in the umbrella of Big Data.
Black Box Data: It is an incorporated by flight crafts, which stores a large sum of information, which includes the conversation between crew members and any other communications (alert messages or any order passed)by the technical grounds duty staff.
Social Media Data: Social networking sites such as Face book and Twitter contains the information and the views posted by millions of people across the globe.
Stock Exchange Data: It holds information (complete details of in and out of business transactions) about the ‘buyer’ and ‘seller’ decisions in terms of share between different companies made by the customers.
Power Grid Data: The power grid data mainly holds the information consumed by a particular node in terms of base station.
Transport Data: It includes the data’s from various transport sectors such as model, capacity, distance and availability of a vehicle.
Search Engine Data: Search engines retrieve a large amount of data from different sources of database.
The importance of big data is how you utilize the data which you own. Data can be fetched from any source and analyze it to solve that enable us in terms of
1) Cost reductions
2) Time reductions,
3) New product development and optimized offerings, and
4) Smart decision making.
Combination of big data with high-powered analytics, you can have great impact on your business strategy such as:
Large amounts of data streaming in from countless sources, banks have to find out unique and innovative ways to manage big data. It’s important to analyze customers needs and provide them service as per their requirements, and minimize risk and fraud while maintaining regulatory compliance. Big data have to deal with financial institutions to do one step from the advanced analytics.
When government agencies are harnessing and applying analytics to their big data, they have improvised a lot in terms of managing utilities, running agencies, dealing with traffic congestion or preventing the affects crime. But apart from its advantages in Big Data, governments also address issues of transparency and privacy.
Educator regarding Big Data provides a significant impact on school systems, students and curriculums. By analyzing big data, they can identify at-risk students, ensuring student’s progress, and can implement an improvised system for evaluation and support of teachers and principals in their teachings.
When it comes to health care in terms of Patient records. Treatment plans. Prescription information etc., everything needs to be done quickly and accurately and some aspects enough transparency to satisfy stringent industry regulations. Effective management results in good health care to uncover hidden insights that improve patient care.
Manufacturers can improve their quality and output while minimizing waste where processes are known as the main key factors in today’s highly competitive market. Several manufacturers are working on analytics where they can solve problems faster and make more agile business decisions.
Customer relationship maintains is the biggest challenge in the retail industry and the best way to manage will be to manage big data. Retailers must have unique marketing ideas to sell their products to customers, the most effective way to handle transactions, and applying improvised tactics of using innovative ideas using BigData to improve their business.
Big Data is being converted into nuggets of information and then it becomes very straightforward for most business enterprises as we now know what their customers want, what are the products are rapidly fast moving, what are the expectations of the end users from the customer service, speed up the time sequence for marketing, methods on cost reduction, and methods to build economies of scale in a highly efficient manner. Hence Big Data leads to big time benefits for organizations and hence there exists a demand about it in the IT world.
Barriers that are imposed on big data are as follows:
Enterprise servers are using the above measures to overcome the barriers mentioned above.
|Latency||1 ms to 100 ms||1 min to 100 min|
|Concurrency||1000 to100,000||1 to 10|
|Access Pattern||Writes and Reads||Reads|
|End User||Customer||Data Scientist|
|Technology||NoSQL Database||MapReduce, MPP Database|
Learn SQL in 16 hrs from experts