TECH BYTES | Big Data: The essential things you need to know
Big data is a general term used to refer to large data sets or vast amounts of data that are collected frequently in sizes that database management and processing tools find difficult to capture and analyze within a reasonable period of time.
Smart city and communication network concept. GUI (Graphical User Interface). IoT (Internet of Things).
Data has come along way since it was informally defined as unprocessed information. It has gone on to something useful that must be mined and even hijacked for profit. Presently, there’s a lot of noisy chatter everywhere about Big Data.
You have to ask, “Yo, what’s so Big about a mere hodge-podge of ones and zeroes?” That’s the techie in you who’s puzzled and the rest of humanity would go on with their usual business unmindful that Big Data has a big role in shaping our individual lives in the here and now.
Wait, that may be our loudmouth making brazen claims but here’s the thing. Big data is all over the inner workings of today’s digital revolution so everyone should know a thing or two about its big intents and purposes. Let TSB hand hold you in appreciating what this big deal called Big Data is all about.
What is Big Data?
Big data is a general term used to refer to large data sets or vast amounts of data that are collected frequently in sizes that database management and processing tools find difficult to capture and analyze within a reasonable period of time.
To separate it from ordinary data, Big data is characterized by volume, velocity and variety.
Volume is the amount of data being generated every second, velocity means the rate at which the data is generated and variety is the form of data whether structured data like numeric data, unstructured data like text, images, videos, and financial transactions, or semi-structured data like JSON or XML.
Where does Big data come from?
Big data is generated from social media, e-commerce sites and large organizations in the airlines, manufacturing and retail and commerce industry. Data from social media could provide significant insights on consumer behavior and preferences that companies could integrate with other analytical tools. For example, the billions of Likes and comments posted on Facebook daily constitute a gold mine of data that could be transformed to useful information for top-level decision makers.
Transactional big data from big retailers could be used to improve delivery of services to all types of customers leading to increased sales and revenues. On the other hand, machine data from industrial equipment, sensors and web logs tracks user behavior online that could help in improving the performance of critical machinery and equipment.
What’s in it for you?
Big data is changing society and is affecting our lives in more ways than we could possibly fathom. It’s easy to see how the massive amounts of data being transformed to intelligence could be a boon to businesses. But that’s not to say, you as a consumer would merely be at the receiving end of all the developments. Here are four ways Big Data benefits could trickle down to ordinary folks:
• Improved customer service resulting from feedback on online platforms
• Better products arising from constructive responses sent by users as well as from the sensors in the products themselves
• Reduced incidence of crime and lives at risk since big data fed into predictive models could quickly send alerts to the police or to health care professionals
• Privacy protection because Big Data tools enable service providers to detect fraud and illegal activity in seconds
Are there risks and disadvantages?
• Data Quality – The initial data could be very messy, inconsistent and incomplete and it costs more to turn it into intelligence
• Discovery Analyzing petabytes of data using extremely powerful algorithms to find patterns and insights are very difficult.
• Storage – The more data an organization has, the more complex the problems of managing it could become so the need a storage system which could easily scale up or down on-demand.
• Security – Since the data is huge in size, keeping it secure is another challenge. It includes user authentication, restricting access based on a user, recording data access histories, proper use of data encryption etc.
• Analytics – Given a deluge of data of varying quality, only state-of-the art, expensive Big data analytical tools could turn ‘dirty’ data into useful timely information.
What is Big Data Analytics?
Big Data analytics is the process of collecting, organizing and analyzing a large amount of data to uncover hidden pattern, correlation and other meaningful insights that could help seize new opportunities that leads to more efficient operations, higher profits and happier customers.
To apply Big Data Analytics, data scientists, predictive modelers, statisticians, and other dedicated analytical staff are employed using specialized software tools and applications.
Big Data analytic tools and software enable an organization to process a large amount of data and provide meaningful insights that provide better business decisions in the future to improve their business, which in turn lead to higher sales and revenues.
What are processes involved in Big Data Analytics?
Analytics comprises various technologies that help you get the most valued information from the data.
• Hadoop. The open-source framework that is widely used to store a large amount of data and run various applications on a cluster of commodity hardware. It has become a key technology to be used in big data because of the constant increase in the variety and volume of data and its distributed computing model provides faster access to data.
• Data Mining. Once the data is stored in the data management system, you could use data mining techniques to discover the patterns which are used for further analysis and answer complex business questions. With data mining, all the repetitive and noisy data could be removed and point out only the relevant information that is used to accelerate the pace of making informed decisions.
• Text Mining. With text mining, we could analyze the text data from the web like the comments, likes from social media and other text-based sources like email we could identify if the mail is spam. Text Mining uses technologies like machine learning or natural language processing to analyze a large amount of data and discover the various patterns.
• Predictive Analytics
Predictive analytics uses data, statistical algorithms and machine learning techniques to identify future outcomes based on historical data. It’s all about providing the best future outcomes so that organizations could feel confident in their current business decisions.
Where is The Philippines in global Big Data map?
A few local developments should situate the Philippines in the Big Data revolution that’s sweeping the globe.
• The Philippine Statistical Development Program (PSDP) 2018-2023 is the national strategy to strengthen the Philippine Statistical System through the use of administrative data and the exploration of “Big Data,” or large data sets. One of its objectives is to enhance local and national statistical capacity development to also include citizen-generated data as possible sources of official statistics.
• Past NEDA secretary Pernia had articulated that the Philippine Statistical System must keep pace with the rapid change in technology, emerging demands for various indicators, compliance to the country’s international commitments and the dynamics in the international statistics community. He must be referring to Big Data collaboration when he further explained that each sector should understand its roles in the production, dissemination and use of statistics.
• As early as 2016, Ayala Corp. expressed the need to elevate the country’s Big Data talent pool to harness the business and governance potential of mining information. The company cited that the local firms are sitting on a wealth of data that could help them deliver better services to customers.
• Recently, a former Globe Telecom officer projected that 10 million jobs could be created in the big data space in the next 5 years. It’s big business as well as a social enterprise given its capacity to alleviate poverty by generating millions of jobs.
8. How is Big Data Helping in the Fight Against the Covid-19 Pandemic?
Big Data fits right in to the challenge of making sense of massive data from around the world in order for authorities to respond to a pandemic, such Covid-19, in the following manner:
• Providing immense computational power to analyze a deluge of data of varying shades and sizes in a time of lockdowns and quarantines.
• Subjecting the findings to rigorous analysis in the search for a universal cure
• Even greater use of the massive computational power to ensure that the cure could be safely administered to billions of people of all ages and races.
