{"id":3236,"date":"2019-03-22T06:59:00","date_gmt":"2019-03-22T06:59:00","guid":{"rendered":"https:\/\/www.digitalogy.co\/blog\/?p=3236"},"modified":"2025-09-29T15:14:08","modified_gmt":"2025-09-29T09:44:08","slug":"big-data-a-revolution-or-a-dud","status":"publish","type":"post","link":"https:\/\/www.digitalogy.co\/blog\/big-data-a-revolution-or-a-dud\/","title":{"rendered":"What is Big Data – A revolution or a dud?"},"content":{"rendered":"\n

Big data<\/strong> is surely the hottest technology in this decade. Not many people know it \u201cin and out\u201d but surely they like to Flaunt it. The use of the term \u201cbig data\u201d has become as prevalent as the phenomenon itself. <\/span><\/p>\n\n\n\n

The discussion of \u201cbig data\u201d has generated colossal insights into business management and led companies to rethink their strategies, implementing perceptive and meaningful methods of applying the wealth of information available in the 21st century. It was mentioned at Oracle Open World multiple times; companies are readying themselves to handle so-called Big Data and devices are being developed to handle Big Data. For those who might not have glued themselves to their devices reading news related to technology constantly, the term Big Data could be foreign to them.<\/span><\/p>\n\n\n\n

What is Big Data?<\/span><\/h2>\n\n\n\n

The symbols, quantities,<\/span> or characters on which operations are performed by a computer, may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media. Nothing is new about the discernment of big data, which has been around since at least 2001. In a nutshell, Big Data is your data. It’s the information owned by your company, obtained and processed through new techniques to produce value in the best way possible. Companies have sought for decades to make the best use of information to improve their business capabilities. However, it’s the structure (or lack thereof) and size of Big Data that makes it so unique. Big Data is also unique because it represents both important information – which can give way for new opportunities – and the way this information is analyzed to help open those doors. The analysis goes hand-in-hand with the information, so in this sense “Big Data” represents a noun – “the data” – and a verb – “combing the data to find value.”<\/span><\/p>\n\n\n\n

An exact definition of “big data” is difficult to cover because projects, vendors, practitioners, and business professionals use it quite differently. With that in mind, generally speaking, big data is:<\/span><\/p>\n\n\n\n

    \n
  1. Large database <\/span><\/li>\n\n\n\n
  2. Category of computing strategies and technologies that are used to handle large databases.<\/span><\/li>\n<\/ol>\n\n\n\n

    In this context, “Large database” means a database too large to realistically process or store with traditional tooling or on a single computer. These Datasets are constantly shifting and becoming unmanageable and may vary significantly from organization to organization.<\/span><\/p>\n\n\n\n

    Some of The Big Data Examples<\/strong> in 2023<\/span><\/strong><\/p>\n\n\n\n

    *10 TB of data is generated by Jet engines in 30 minutes of <\/span>flight time. With many thousand flights per day, the generation of data reaches up to many Petabytes.<\/span><\/p>\n\n\n\n

    *The Stock Exchange of New York generates about one TB of new trade data per day.<\/span><\/p>\n\n\n\n

    * Facebook generates 4 million likes every minute and more than 250 billion photos have been uploaded on Facebook to date.<\/span><\/p>\n\n\n\n

    Types of Big Data in 202<\/span>4<\/h2>\n\n\n\n
      \n
    1. Structured<\/li>\n\n\n\n
    2. Unstructured<\/li>\n\n\n\n
    3. Semi-Structured<\/li>\n<\/ol>\n\n\n\n

      Structured<\/span><\/strong><\/p>\n\n\n\n

      The term structured data generally refers to data that has a defined length and format for big data. Over the period of time, <\/span>advancements in computer science have achieved greater success in developing techniques for working with such kinds of data (where the format is well known in advance) and also deriving value from it. However, data scientists nowadays, have forecasted issues when the size of such data grows to a huge extent, typical sizes are being in the range of multiple zettabytes.<\/span><\/p>\n\n\n\n

      Unstructured<\/span><\/strong><\/p>\n\n\n\n

      Unstructured data is data that does not have <\/span>a specified format for big data being stored. If 20 percent of the data available to enterprises is structured data, the other 80 percent is unstructured. The major challenge in addition to the huge size that unstructured data poses is in terms of its processing for deriving value out of it. An example of an unstructured dataset is a data source containing a combination of simple files, photos, videos, etc. Organizations have a large amount of valuable data available to them but unfortunately, they don’t know how to derive value from it since this data is in its raw form or unstructured format.<\/span><\/p>\n\n\n\n

      Semi-Structured<\/span><\/strong><\/p>\n\n\n\n

      Semi-structured datasets contain both forms of data. We can see a <\/span>semi-structured dataset as structured data in form but it is actually not defined e.g. a table definition in relational DBMS.<\/span><\/p>\n\n\n\n

      \"Semi-Structured\"<\/figure>\n\n\n\n

      What characteristics of Big Data make it different?<\/span><\/h2>\n\n\n\n

      Requirements for working with big data systems are the same as the requirements for working with databases of different sizes. On the other hand, the massive scale, the speed of ingesting and processing, and the characteristics of the data that must be dealt with at each stage of the process present significant new challenges when designing solutions. The main goal of most big data systems is to exterior insights and connections from large volumes of data that would not be possible using usual methods. And other characteristics of big data that make big data different from other data processing:<\/span><\/p>\n\n\n\n

      Degree<\/span><\/strong><\/p>\n\n\n\n

      The utter piece of information processed basically defines big data systems. The database can be larger than traditional databases, which hassle more thought at each stage of the processing and storage life cycle. <\/span><\/p>\n\n\n\n

      Repeatedly, because the work necessities exceed the capabilities of a single computer, this becomes a challenge of analyzing, allocating, and coordinating resources from groups of computers. Algorithms capable of breaking tasks into smaller pieces become drastically important.<\/span><\/p>\n\n\n\n

      Rapidity<\/span><\/strong><\/p>\n\n\n\n

      Big data differs extensively from other data systems due to the speed at<\/span> which information moves through the system. Data is regularly flowing into the system from different sources and is often likely to be processed in real-time to gain insights and update the current system.<\/span><\/p>\n\n\n\n

      Feedback has taken many big data practitioners away from a batch-oriented approach and closer to a real-time system. Data is constantly being processed, added, and analyzed in order to keep up with the flow of new information and to surface valuable information early when it is most appropriate. Ideas like this require vigorous systems with highly available components that protect against failures in the data pipeline.<\/span><\/p>\n\n\n\n

      Diversity<\/span><\/strong><\/p>\n\n\n\n

      Problems that occur in big data are often exclusive because of the wide range of both the sources being processed and their comparative quality.<\/span><\/p>\n\n\n\n

      A database can be taken from internal systems like application and server logs, from social media and other external APIs, from device sensors, and from other users\/providers. Big data seeks to handle the information potentially regardless of where it’s coming from by combining all information into a single system.<\/span><\/p>\n\n\n\n

      The content and types of media can differ significantly as well. Media like Photos, video, and audio recordings are used alongside files, structured logs, etc. While conventional data processing systems might expect data to enter the pipeline that is already formatted, and pre-arranged, labeled big data systems usually accept and store data closer to its original state. Preferably, any changes to the original data will be in the memory at the time of processing.<\/span><\/p>\n\n\n\n

      Other distinctiveness<\/span><\/strong><\/p>\n\n\n\n

      Different folks and organizations have suggested expanding the three features, though these proposals have tended to describe challenges rather than qualities of big data. Some common characteristics are:<\/span><\/p>\n\n\n\n