Brief History of Big Data

Brief History of Big Data © Shutterstock | McIekThis article ventures into the world of big data, the history of big data all the way from its early stages in the 20th Century to big data as we know it now. The article also discusses some of the most prominent applications of big data in the world we live in.WHAT IS BIG DATA?In the business landscape of today, data management can be a major determinant of whether you succeed or fail. Most businesses have begun to realize the importance of incorporating strategies that can transform them through the application of big data. In this endeavor, businesses are realizing that big data is not simply a single technology or technique. Rather, big data is a trend that stretches across numerous fields in business and technology.Big Data is the term used to refer to initiatives and technologies that comprise of data that is too diverse, fast evolving, and vast for ordinary technologies, infra- structure, and skills to address exhaustively. That is; the volume, velocity and variety of the data is far too great. Despite the complexity of this data, advances in technology are allowing businesses to draw value from big data.For example, in your businesses can be positioned to track consumer web clicks in order to identify consumers’ behavioral trends and modify the business’s campaigns, advertisements, and pricing to fit the consumers’ persona.An additional example would be where energy service providers assess household consumption levels in order to predict impending outages and promote more efficient energy consumption.Additionally, health provision bodies may be able to monitor the spread as well as the emergence of illnesses by analyzing social media data. There are numerous applications of big data, the most noteworthy of which will be discussed a little later in the article. Big Data involves the creation of large amounts of complex data, its storage, its retrieval, and finally its analysis.The following are the three Vs of big data.Volum e. Two decades ago, typical computers may have had about ten gigabytes of memory. Today, however, social media platforms such as Facebook will take in over half a billion terabytes of data on a daily basis. Similarly, Boeing airplanes generate hundreds of terabytes in flight data in a single flight. The wide spread use of smartphones and tablets results in the generation of billions of terabytes of consistently updated data feeds that are of infinitely diverse genres.Velocity. Clickstreams capture user behavior at millions of events each second. For example, stock trading market changes are reflected within microseconds. Computer processes exchange data between billions of gadgets, infrastructure, and sensors in order to generate accurate and applicable data in real-time. For example, on-line gaming systems support millions of users operating concurrently and with each producing multiple inputs every second.Variety. Big data does not just refer to numbers and dates, big data is all that inclusive of audio, video, unstructured text, social media information, and so much more. Database systems of about two decades ago had been designed to address a smaller volume of structured data, slower, and fewer updates. They were designed to process structured and predictable forms of data. These traditional databases were also designed to operate on single servers, which would make an increase in capacity an expensive endeavor. Programs and applications have evolved to serve large volumes of users and the use of the olden databases has become a liability for most businesses as opposed to an asset. Big Data databases, for example MongoDB, solve these issues and avail businesses great value.HISTORY OF BIG DATAThe term Big Data was coined by Roger Mougalas back in 2005. However, the application of big data and the quest to understand the available data is something that has been in existence for a long time. As a matter of fact, some of the earliest records of the applicatio n of data to analyze and control business activities date as far back as7,000 years.This was with the introduction of accounting in Mesopotamia for the recording of crop growth and herding. The principles continued to grow and improve and John Graunt in 1663 recorded and analyzed information on the rate of mortality in London. John Graunt did this in an effort to raise awareness on the effects of the bubonic plague that was ongoing at the time.In his book ‘Natural and Political Observations Made upon the Bills of Mortality’, John Graunt provided the world with the first statistical analysis of data ever recorded. The book was an insight into the causes of death in seventeenth century England. Due to his work, John Graunt is widely regarded as the pioneer of the field of statistics.After the works of Graunt, accounting principles continued to improve and develop but nothing extra ordinary quite took place until recently in the 20th Century when the information era began. The star ting point of modern data begins in 1889 when a computing system was invented by Herman Hollerith in an attempt to organize census data.After Herman Hollerith’s input, the next noteworthy data development leap happened in 1937 under Franklin D. Roosevelt’s presidential administration in the United States. After the United States congress passed the Social Security Act, the government was required to keep track of millions of Americans. The government contracted IBM to develop a punch card-reading system that would be applied in this extensive data project.However, the very first data-processing machine was named ‘Colossus’ and was developed by the British in order to decipher Nazi codes in World War II, 1943. This machine worked by searching for any patterns that would appear regularly in the intercepted messages. The machine worked at a record rate of five thousand characters per second, which reduced work that would take weeks to just a few hours.From this development, the National Security Agency (NSA) was created in the United States in 1952. Employees of the NSA were tasked with decrypting the obtained messages during the course of the Cold War. Machine development at this stage had advanced to a level where machines could independently and automatically collect and process information.The first data centre was built by the United States government in 1965 for the purpose of storing millions of tax returns and fingerprint sets. This was achieved by transferring every record onto magnetic tapes that were to be stored systematically in a central location. This project, however, did not persist due to fear of sabotage or acquisition. However, it is a widely accepted that this initiative was the starting point of electronic big storage.Tim Berners-Lee a British computer scientist invented the World Wide Web in 1989. Berners-Lee’s intention was to enable the sharing of information through a hypertext system. He had no idea what kind of impact his inv ention would have on the world. As we entered the 1990’s, the creation of data grew at an extremely high rate as more devices gained capacity to access the internet.The first super-computer was built in 1995. This computer had the capacity to handle work that would take a single person thousands of years in a matter of seconds.And so came the 21st Century This is when the world was first introduced to the term Big Data by Roger Mougalas. In the same year (2005), Yahoo created the now open-source Hadoop with the intention of indexing the entire World Wide Web. Today, Hadoop is used by millions of businesses to go through colossal amounts of data.During this period, social networks were rapidly increasing and large amounts of data were being created on a daily basis. Businesses and governments alike began to establish big data projects. For example, in 2009 in the largest biometric database ever created, the Indian government stored fingerprint and iris scans of all of its citizens. Eric Schmidt gave a speech at the Techonomy conference in Lake Tahoe, California in 2010. In his speech he presented that there were 5 exabytes of data stored since the beginning of time up to the year 2003. Eric Schmidt possibly could not imagine that by the year 2016, the same amount of data would normally be created every two days. The rate at which big data is growing does not seem to be slowing down either. Over the past number of years, there have been various organizations that have come up in an attempt to deal with big data, for example, HCL. These organizations’ business is aiding other businesses to understand big data. Everyday more and more businesses are moving towards the acceptance and exploitation ofbig data.Although it seems like big data has been around for a long time now and that we are getting closer to the pinnacle, big data may just be at its formidable stages. Big data in the near future may end up making big data now seem like a poultry amount.What does t he Future HoldThere you go; that was a brief history of Big Data. Looking into history can give us a minor insight into the future. Two decades ago, businesses that possessed information were the most successful whilst today, the most successful businesses are the ones that interpret and use the information in the best way.It may be fair to assume that in the future, the success of businesses will not only lie in those who analyze and implement big data the best, but also those who use big data to their greatest advantage and make strategic decisions for the future.Learn why most big data projects will fail.[slideshare id=55470639doc=5signsyourbigdataprojectisdoomedtofail11-151124163153-lva1-app6891w=640h=330]APPLICATION OF BIG DATA IN THE WORLD TODAYIndustry influencers agree that big data has become a game changer in just about all modern industries in the last couple of years. As big data continues to influence our daily lives, there has been a shift of interest in the subject. T he focus has changed from simply trying to grasp the concept of this phenomenon to finding tangible value in its application.For most of us, I am sure the term Big Data inspires the image of rows of humming servers and a sequence of flickering lights. Big data, however, persists beyond the storage of information. There are several areas in which big data is being applied; the following section presents you with the most prominent areas in which big data applies today.In Understanding and Targeting ConsumersThis is among the most popular and publicized areas in which big data is being used. In business, big data helps your business to analyze data and better understand the consumers’ behaviors and interests. Your business ought to expand beyond its traditional data sets. By incorporating the use of data obtained from social media, browser logs, and sensor data, you will be able to get a clearer picture of what your consumers need. Once you understand this, your business will be bet ter positioned to create predictive models and position itself to meet consumer needs.Big data can, therefore, apply in analyzing and understanding your audience’s interests. For example, some people are even of the opinion that President Obama’s second election win was due to his team’s superior ability to use big data analytics to understand the audience’s interests and appeal to them. In theory, this is plausible, and big data can be used to predict and influence even events as big and important as government elections. How much more so for your business?Self OptimizationBig data does not only apply to your business, but can also apply to you as an individual. You can now benefit from data generated by devices such as smart watches. These devices have the capacity, for example, to monitor the amount of calories you intake in a day, your activity level, as well as your sleep patterns. While the real-time information may be exciting to observe, for example, your calorie int ake at the end of the day, the real value lies in analyzing your collective data.With the analysis of data collected from you over a certain period of time, you will be able to make adjustments in your personal life in order to be more productive, to eat healthy, to acquire sufficient amounts of sleep, and so forth.Improvement of HealthcareThis is another area where big data has played and continues to play a major role. For example, computing big data enables health providers to analyze and decode DNA issues in a matter of minutes. Big data will also allow us to discover diseases faster than would be possible without it. On top of this, big data allows healthcare providers to predict the patterns of diseases and, therefore, measures can be set up to prevent further spread of the diseases.Apple launched a health app known as ResearchKit. Through such an application, researchers can collect data from individual phones to be compiled for various health studies. For example, as a patie nt your device may prompt you to indicate how you feel about treatment services.This data along with that gathered from thousands, if not millions, of other participants will reveal information that compels medical practitioners step up the quality of their services. Data gathered from this application and similar ones, can be used to gather information on specific diseases. For example, information on patients of terminal illnesses can be compiled to be used in the furtherance of research.In addition, big data is already in use in the monitoring of babies who are premature or sick. Through the recording of each heartbeat as well as the breathing pattern of the babies brought to the unit, infections are detectable way before the onset of physical symptoms. This way, treatment is administered early because every hour counts with such fragile babies. This prompt administration of treatment, therefore, increases the babies’ chances of survival.Security and Law Enforcement Improvement Another sector where big data is heavily applied is in the enhancement and enablement of law enforcement. Governmental institutions, for example, the NSA in the United States use big data to detect and deter potential terrorist activities. In business, on the other hand, big data analytics can be used for the prevention of cyber attacks and unauthorized access. For the police department, big data tools enable the officers to predict and deter criminal activities.In 2014, the Chicago Police Department in Illinois, United States, sent out officers to pay a visit to persons that had been identified as most likely to commit crimes. This group of people was generated by a computer through the analysis of big data. The officers visited the individuals on their list, not to interrogate or detain them but to offer them information about jobs and training programs.The officers also educated these individuals on the consequences of certain crimes and their dynamics. As much as the intentions of the police department were sincere, the exercise was quickly shut down when the public opined that the exercise was ‘profiling’. I recognize the importance of security but I have to agree with that opinion.IN CONCLUSIONAlthough the era of big data has only recently begun, businesses and governments alike are already taking advantage of it. However, big data can be misused, for example, the Chicago Police Departments initiative in following up on people who were identified as potential criminals through big data analysis may have been done with the best of intentions.Nonetheless, the initiative can still be considered profiling and a tool through which people can be stigmatized for who they are or their past mistakes.   However, big data may very well be a double edged sword because through monitoring of social media activity and analyzing people’s likes and interests in big data, terrorist attacks can be averted. Yet this is also an invasion of privacy. Despite this downsid e, the benefits of big data carry much more weight and its applications in business, health, governance, and beyond should be encouraged.Big data has been slowly developing over the last few centuries and in the course of the past decade, big data has quickly evolved to become what we know it to be today. One vital point to note is that big data is not only about accumulating and storing massive amounts of information but, more importantly, utilizing that information to solve issues in business as well as in our society.Big data seems to evolve simultaneously with advancement in technology. Therefore, as we advance in technology, big data will continue to grow as a field and in volume, possibly to levels we cannot even fathom right now.