In terms of big data, it is a collection of data sets that are very large and complex. The data is so large that traditional data processing tools cannot analyze it. Likewise, the challenges behind big data analytics are enormous. Among them, we can mention how to collect data, how to store data, how to analyze data, visualization, transmission, issues related to data sensitivity and data validity.
The concept of big data itself was born three V, or “three letters V” means:
Cover where big data is a collection of very high volume and sometimes unstructured data. Some of these data are e.g feeds , the user’s click flows through a web page or application. The amount of data volume can reach TB or even PB (petabytes).
Speed or speed, where a data stream must be checked at high speed. The speed of receiving or processing big data must be high. Even some current technologies are able to perform data processing and data evaluation On Time.
Variety Alias changes are talking about traditional data, generally aggregated data Structured and appropriate so that it can be easily analyzed. While in big data, the obtained data are generally unstructured and come from different sources so that they need to be pre-processed to be able to analyze the data.
The history of big data
Big data itself, though, has only become known in the last few years. The history of big data begins in the early 60s and 70s. It was in those years that “citizens of the world” began to become literate in data and analyze it through the science of statistics. Meanwhile, the rise of Facebook and other social media in the 2000s made people realize how important the user data these social media platforms hold.
However, problems arose because the inputs to social media platforms were too large to store and process at the time. This problem started slowly with the advent of Apache Hadoop (now with Apache Spark) and NoSQL. The spark itself is one Open source framework which was specifically created to store and analyze big data. Big data problems can be solved with Spark. Since then, the volume of big data has grown rapidly.
Moreover, the involvement of technology in every aspect of human life magnifies the input of big data. The more information a machine can acquire about its users.
How does big data work?
As an entrepreneur, you may use big data to run your company. With this big data, you can get a lot Insight, insight, insight To do enterprise planning therefore that is why you also need to know how to deal with big data!
Data integration
Compared to traditional data that can be handled by “ETL” aka Extract convert and loadBig data processing is much more complex than that.
This is because big data consists of a collection of different inputs and they need to be processed before the actual processing. For example, you can collect data in the form of consumer comments in public spaces, data about consumer photos of your products on social media, or even the traffic On your website and social media pages
After processing and creating a uniform format, you can then send the data to your business analyst.
manage
In order to be able to manage data properly, the most important factor that you cannot forget is how to store data. There are actually plenty of choices, but for those of you on a tight budget, of course Cloud storage space It is a solution
data analysis
Big investment in collecting and Management Data can only give results if you can do big data analysis.
Unfortunately, as we explained before, traditional data processing software is not able to process large and complex big data. You should be able to model the data Learning the machine And Artificial intelligence so that it can be analyzed correctly. When the data is analyzed, believe me you will find new inputs, insights and discoveries that you can use to improve your company.
Big data applications
As the size of big data increases and the capabilities of big data processing also become more complex. The current application of big data is no longer an idea. Big data applications that were initially in companies for marketing and maintenance purposes are now used in many fields.
Government
It can be said that the use of big data in the public sector makes the government process much easier and more efficient. However, by mastering big data, it means that the government has stronger control and control over its people.
One of the applications of big data in the government that is very popular today is the CRVS system.Civil registration and vital statistics) which was introduced by WHO as a civil registry that includes data related to births, deaths, in detail including causes of death and history of marriage and divorce.
This system has been popular for a long time, in Indonesia it is administered by Disdukcapil (Department of Population and Civil Registration). The problem is that in this traditional method, the accuracy of the data is questioned. Big data is the answer!
Several countries that have launched CVRS based on big data include Rwanda, Oman and New Zealand. WHO is also there with it Critical Event Monitoring (MOVE-IT) system and African countries with their iCivil Africa.
Health services
Big data analytics allows healthcare providers to improve overall services. This includes personalized medicine, prescribing analytics, clinical treatment risks, predictive health analytics and more.
Furthermore, data collection is nothing new in the world of health. Many health services have used electronic medical record technology that contains digital health information.
While electronic medical records have been around for a long time, today’s big data management and analytics technology can help healthcare providers analyze existing collections of electronic medical records.
The only problems related to big data analysis in health care are ethical and Privacy. Because the big data collected in the health world is generally patient health data, which must be protected by law in any country.
Well, I hope this last problem can be solved, because if big data analytics is actually implemented 100% in the health sector, the benefits could be enormous!
education
This is not an objective implementation of big data, but the discovery of big data today has made many educational institutions realize that the previous data processing science is not enough to deal with this big data problem.
In addition, there are several short programs in the format Private boot camps Such as The Data Incubator or General Assembly, which specifically aim to produce experts in the field of big data processing and analysis.
In addition, this topic is also discussed in business schools around the world. Current business schools must realize that business school graduates must be able to manage big data, which is no longer the case One size Like traditional data processing
So that eventually they can produce business consultants and marketing experts who are able to process big data as a whole.
media
If we do not discuss the use of big data in the world of media, there is little talk about the use of big data. Because today’s social media is actually a collection of big data that is ready to be collected and analyzed by media practitioners.
In the past, these practitioners separately collected information from newspapers, magazines, and television programs. Therefore, now the existence of social media and the Internet makes doctors have to collect millions of data related to consumer-to-consumer information. Behavior – ethics at that