A Simple Definition of Big Data.
Big Data is the analysis of huge set of information to identify trends, patterns, and correlations between outcomes of different sources, in order to form a conclusion of what will happen next in near real time. This huge set of information is highly disseminated, unstructured, and streaming in real time (this is not information that can be analysed after the fact).
5 Best Big Data Quotes
“You can have data without information, but you cannot have information without data.”
“Without big data, you are blind and deaf in the middle of a freeway.”
“Big data is at the foundation of all the megatrends that are happening today, from social to mobile to cloud to gaming.”
“There were 5 exabytes of information created between the dawn of civilization through 2003, but that much information is now created every 2 days.”
“Hiding within those mounds of data is knowledge that could change the life of a patient, or change the world.”
An Equally True Statement About Big Data
"The amount of nonsense that can be packed into the term 'Big Data' doubles approximately every two years."
Interesting Big Data Facts and Figures
Check out the Infographics for more Big Data facts and figures.
How Data Has Changed Over Time.
Data used to be perceived as static information, whose usefulness was finished once the purpose for which it was collected was achieved. Today, data had become a raw material of business, a vital economic input, used to create new form of economic value. In fact, data can be cleverly reused to enable innovation and new services. Data can reveal secrets to those with the right mindset and tools to listen.
When Location Becomes Data
Think for a moment when someone said "I was in Paris on 30th September 1930". In olden days, this statement was said to have 2 information, with only one of them could be quantified - date. Today, this statement has 2 quantifiable information - location and date. With the introduction of the Universal Transverse Mercator (UTM) coordinate system in the 1940s, location could then be recorded in a standardised numerical format. When a GPS receiver is attached to an object - human or otherwise, the object can be tracked to produce information about location, date, and time of the object, making it a very useful piece of data to be exploited. With wireless sensors installed in so many devices, equipment, machinery, vehicles and the like, location information relating to an object is increasing at exponential rate. The analysis of Big Data now includes location information and other data such as temperature, humidity, wind speed / direction, and UV level relating to a location.
Is There Small Data Before Big Data?
Everything grows from small to big, including data. Therefore, there is Small Data that is usually used to describe data whose volume and format can be easily used for self-service analytics (to be explained below). A commonly quoted axiom is that "Big Data is for machines. Small Data is for people". Small Data is often organised and packaged, readily accessible, understandable, and actionable for everyday tasks. Examples of Small Data include sales data, weather forecasts, fuel consumption history, sports results, and telephone directory.
Self-service analytics is a process where business users perform advanced analytics by manipulating data to identify
business opportunities, without requiring them to have a background in statistics or technology.
Software vendors are introducing products that allow business users to work with data aggregated from a range of sources. The software provides users with a dashboard that allows the users to query and manipulate large amounts of data. In the past, such data analysis was solely the domain of trained data analysts, who are now often referred to as data scientists.
The 3 Significant Shifts in Dealing with Data
Due to the emergence of Big Data, there has been some significant shifts away from how we deal with data (Big Data vs Small Data):
In the past, we relied on using sample to form a conclusion about the population. Today, we can analyse far more data than before.
When we analyse ALL the data, we can see details that could not be observed by sampling.
Big Data gives us the chance to see data within data.
Analysing more data permits us to relax on exactitude.
In a small data environment, we analyse and quantify as precisely as possible.
With big data, we are satisfied with a sense of general direction rather than knowing the exact element of a phenomenon.
What we lose in accuracy at the micro level is what we gain in insight at the macro level.
When dealing with Big Data, we move away from age-old seach for causality - a human nature to search for causes.
Instead of looking for causes, we are satisfied with patterns and correlations in the data that provide us with unprecedented insights.
The correlations may not tell us precisely why something is happening, but they affirm that it is happening.
What is Open Data?
The term “open data” refers to non-proprietary and machine-readable data that anyone is free to use, reuse, manipulate, and disseminate without legal or technical restrictions. The first open government data policy was launched in Washington, D.C. in 2008, when the Chief Technology Officer of the District of Columbia released on the Internet more than 400 datasets on the District’s budget, contracts, crime statistics, and more. Developers were encouraged to manipulate, repurpose, and integrate these datasets, creating new applications free for all to use. When President Barack Obama entered office in 2009, he endorsed a similar initiative at the federal level, requiring federal agencies and departments to release open datasets to a designated website, Data.gov.
See our compilation of resources on Open Data here and there.
BIG DATA SCANDAL
Data is the new gold. It is a good gold when data is used in a ethical way to improve efficiencies, drive business performances, enhance better lifestyles, improve safety, reduce wastage and the list goes on and on. It is a bad gold when it is used in an unethical way to achieve certain motives.
And the best unethical application of big data is in a dirty business known as politic. This is best illustrated by the BIG DATA SCANDAL that took place on 18.03.2018 involving Cambridge Analytica that misused the Facebook profiles of 50 million people in order to target them with political ads to swing their voting decisions.
There are many news and articles on this topic over the last few days. However, I quite like this article which is said to be originally published on 28.01.2017 titled:
The Data That Turned the World Upside Down
How Cambridge Analytica used your Facebook data to help the Donald Trump campaign in the 2016 election.
Please click here to read full article, but I will append some of the highlights here:
On November 9 at around 8.30 AM., Michal Kosinski woke up in the Hotel Sunnehus in Zurich. The 34-year-old researcher had come to give a lecture at the Swiss Federal Institute of Technology (ETH) about the dangers of Big Data and the digital revolution. Kosinski gives regular lectures on this topic all over the world. He is a leading expert in psychometrics, a data-driven sub-branch of psychology. When he turned on the TV that morning, he saw that the bombshell had exploded: contrary to forecasts by all leading statisticians, Donald J. Trump had been elected president of the United States.
For a long time, Kosinski watched the Trump victory celebrations and the results coming in from each state. He had a hunch that the outcome of the election might have something to do with his research. Finally, he took a deep breath and turned off the TV.
On the same day, a then little-known British company based in London sent out a press release: "We are thrilled that our revolutionary approach to data-driven communication has played such an integral part in President-elect Trump's extraordinary win," Alexander James Ashburner Nix was quoted as saying. Nix is British, 41 years old, and CEO of Cambridge Analytica. He is always immaculately turned out in tailor-made suits and designer glasses, with his wavy blonde hair combed back from his forehead. His company wasn't just integral to Trump's online campaign, but to the UK's Brexit campaign as well.
How dangerous is big data?
Anyone who has not spent the last five years living on another planet will be familiar with the term Big Data. Big Data means, in essence, that everything we do, both on and offline, leaves digital traces. Every purchase we make with our cards, every search we type into Google, every movement we make when our mobile phone is in our pocket, every "like" is stored. Especially every "like." For a long time, it was not entirely clear what use this data could have—except, perhaps, that we might find ads for high blood pressure remedies just after we've Googled "reduce blood pressure."
On November 9, it became clear that maybe much more is possible. The company behind Trump's online campaign—the same company that had worked for Leave.EU in the very early stages of its "Brexit" campaign—was a Big Data company: Cambridge Analytica.
Kosinski and his team tirelessly refined their models. In 2012, Kosinski proved that on the basis of an average of 68 Facebook "likes" by a user, it was possible to predict their skin color (with 95 percent accuracy), their sexual orientation (88 percent accuracy), and their affiliation to the Democratic or Republican party (85 percent). But it didn't stop there. Intelligence, religious affiliation, as well as alcohol, cigarette and drug use, could all be determined. From the data it was even possible to deduce whether someone's parents were divorced.
The strength of their modeling was illustrated by how well it could predict a subject's answers. Kosinski continued to work on the models incessantly: before long, he was able to evaluate a person better than the average work colleague, merely on the basis of ten Facebook "likes." Seventy "likes" were enough to outdo what a person's friends knew, 150 what their parents knew, and 300 "likes" what their partner knew. More "likes" could even surpass what a person thought they knew about themselves. On the day that Kosinski published these findings, he received two phone calls. The threat of a lawsuit and a job offer. Both from Facebook.
"It is my privilege to speak to you today about the power of Big Data and psychographics in the electoral process." The logo of Cambridge Analytica— a brain composed of network nodes, like a map, appears behind Alexander Nix. "Only 18 months ago, Senator Cruz was one of the less popular candidates," explains the blonde man in a cut-glass British accent, which puts Americans on edge the same way that a standard German accent can unsettle Swiss people. "Less than 40 percent of the population had heard of him," another slide says. Cambridge Analytica had become involved in the US election campaign almost two years earlier, initially as a consultant for Republicans Ben Carson and Ted Cruz. Cruz—and later Trump—was funded primarily by the secretive US software billionaire Robert Mercer who, along with his daughter Rebekah, is reported to be the largest investor in Cambridge Analytica.
"So how did he do this?" Up to now, explains Nix, election campaigns have been organized based on demographic concepts. "A really ridiculous idea. The idea that all women should receive the same message because of their gender—or all African Americans because of their race." What Nix meant is that while other campaigners so far have relied on demographics, Cambridge Analytica was using psychometrics.
Nix clicks to the next slide: five different faces, each face corresponding to a personality profile. It is the Big Five or OCEAN Model. "At Cambridge," he said, "we were able to form a model to predict the personality of every single adult in the United States of America." The hall is captivated. According to Nix, the success of Cambridge Analytica's marketing is based on a combination of three elements: behavioral science using the OCEAN Model, Big Data analysis, and ad targeting. Ad targeting is personalized advertising, aligned as accurately as possible to the personality of an individual consumer.
Nix candidly explains how his company does this. First, Cambridge Analytica buys personal data from a range of different sources, like land registries, automotive data, shopping data, bonus cards, club memberships, what magazines you read, what churches you attend. Nix displays the logos of globally active data brokers like Acxiom and Experian—in the US, almost all personal data is for sale. For example, if you want to know where Jewish women live, you can simply buy this information, phone numbers included. Now Cambridge Analytica aggregates this data with the electoral rolls of the Republican party and online data and calculates a Big Five personality profile. Digital footprints suddenly become real people with fears, needs, interests, and residential addresses.
The methodology looks quite similar to the one that Michal Kosinski once developed. Cambridge Analytica also uses, Nix told us, "surveys on social media" and Facebook data. And the company does exactly what Kosinski warned of: "We have profiled the personality of every adult in the United States of America—220 million people," Nix boasts.
FEATURED BIG DATA EVENTS
Use the promotion code MP10 for discount here.
Click here to see more events.
We are reseller of 5 products and services. Find out more here .
FEATURED ONLINE COURSES
FEEL LIKE FINDING A BOOK ON BIG DATA?
TOP 10 PICKS BY OUR TEAM
Big Data by Country
We share some good websites in each country that show their commitment in Big Data initiatives, whether they are by governments, organisations, or even individuals.
> South Korea
Are you from a country that has ambitious Big Data initiatives that you would like to add? Please contact us.
IF YOU LIKE THIS WEBSITE AND ITS CONTENT, PLEASE MAKE A DONATION OF ANY AMOUNT AT >>>