Evolution of Log Analysis Architectures
Open Source For You|August 2024
Going down memory lane, it's easy to see how changing needs have helped evolve log analysers since the mid-2000s. What worked then, will not work today. But the journey is to be cherished.
Krishna Mohan Koyya
Evolution of Log Analysis Architectures

It was the late nineties, and I had about 3-4 years of development experience by then. I had just transitioned from the GSM field to the world of network management systems (NMS) after joining a multinational company in Bengaluru. Although my master’s degree was in computer networks, NMS was entirely new to me. I was familiar with hubs, routers, switches, IP addresses, RFCs, and protocols, but only in theory. This was the first time I had hands-on experience with NMS.

From a technological standpoint, I was quite proficient. I coded in C++ and worked on HP-UX, a variant of UNIX. I had experience with large-scale, mission-critical systems. In a way, I was filled with youthful confidence when I began working with NMS.

In my new role, I was assigned the task of enhancing a log analysis tool as part of a larger NMS. NMS are structured around the FCAPS model, which stands for fault management, configuration management, accounting, performance, and security. This model is defined by ISO. The log analysis tool our team was developing fell under the fault management category. Figure 1 captures the arrangement.

The requirement was straightforward: the log analysis tool needed to gather system logs from all devices across the network, store them in a database, analyse them, and take necessary actions.

This NMS was intended for use by large service providers. In the US market, the adoption of GSM mobile phones was slower compared to VoIP (Voice over IP). While mobile telephony was just beginning in India, offices were transitioning from PSTN technology to VoIP, and VoIP phones were becoming commonplace on every employee’s desk. Each time a call was made or received, or a VoIP phone was powered on or off, a system log was generated. Our log analysis tool needed to collect and analyse all these logs.

هذه القصة مأخوذة من طبعة August 2024 من Open Source For You.

ابدأ النسخة التجريبية المجانية من Magzter GOLD لمدة 7 أيام للوصول إلى آلاف القصص المتميزة المنسقة وأكثر من 9,000 مجلة وصحيفة.

هذه القصة مأخوذة من طبعة August 2024 من Open Source For You.

ابدأ النسخة التجريبية المجانية من Magzter GOLD لمدة 7 أيام للوصول إلى آلاف القصص المتميزة المنسقة وأكثر من 9,000 مجلة وصحيفة.

المزيد من القصص من OPEN SOURCE FOR YOU مشاهدة الكل
Quantum Machine Learning: An Overview
Open Source For You

Quantum Machine Learning: An Overview

Quantum machine learning (QML) is a burgeoning field at the intersection of quantum computing and artificial intelligence. In recent years, the integration of quantum mechanics with machine learning algorithms has sparked substantial interest among researchers and technologists alike. Here's a quick look at the essentials of creating quantum algorithms for AI models, their practical use cases on open source platforms, and best practices for implementing these advanced algorithms.

time-read
7 mins  |
December 2024
Blockchain: Transforming Automotive Traceability, Payments, Sustainability
Open Source For You

Blockchain: Transforming Automotive Traceability, Payments, Sustainability

From advanced battery passports to carbon credits and fortified payments, blockchain is reshaping EV data handling and connectivity.

time-read
6 mins  |
December 2024
"Developers are wasting their time with Kubernetes alone!"
Open Source For You

"Developers are wasting their time with Kubernetes alone!"

At Open Source India 2024, Ram Iyengar, Chief Evangelist, Cloud Foundry Foundation, raised a few eyebrows with his cogent arguments about ways to manage and deploy infrastructure. OSFY's Yashasvini Razdan got some exclusive takes on alternatives for scalable application deployment and management, and the role of Cloud Foundry Foundation in this business....

time-read
10 mins  |
December 2024
Helgrind: Detecting Synchronisation Issues in Multithreaded Programs
Open Source For You

Helgrind: Detecting Synchronisation Issues in Multithreaded Programs

Let's explore how Helgrind can be used to detect and debug multithreading issues with the help of a multithreaded C program.

time-read
3 mins  |
November 2024
The Perfect Process of Booting a PC
Open Source For You

The Perfect Process of Booting a PC

Booting a PC seems as simple as eating a cake. But are you aware of all that goes on behind-the-scenes to bake a delicious cake or seamlessly boot a PC?

time-read
3 mins  |
November 2024
Exploring eBPF and its Integration with Kubernetes
Open Source For You

Exploring eBPF and its Integration with Kubernetes

eBPF, a game-changing technology that extends the capabilities of the Linux kernel, offers significant advantages for Kubernetes networking. It also greatly improves Kubernetes observability by capturing detailed telemetry data directly from the kernel. Read on to find out how its integration with Kubernetes has immense benefits.

time-read
5 mins  |
November 2024
Deploying Generative AI LLMs on Docker
Open Source For You

Deploying Generative AI LLMs on Docker

Built on massive datasets, large language models or LLMS are closely associated with generative Al. Integrating these models with Docker has quite a few advantages.

time-read
8 mins  |
November 2024
Containerisation: The Cornerstone of Multi-Cloud and Hybrid Cloud Success
Open Source For You

Containerisation: The Cornerstone of Multi-Cloud and Hybrid Cloud Success

Open source containerisation software provides the flexibility, cost-effectiveness, and community support needed to build and manage complex multi-cloud and hybrid cloud environments. By leveraging this software, businesses can unlock the full potential of multicloud and hybrid cloud architectures while minimising vendor lock-in risks.

time-read
3 mins  |
November 2024
From Virtual Machines to Docker Containers: The Evolution of Software Development
Open Source For You

From Virtual Machines to Docker Containers: The Evolution of Software Development

Containerisation and Kubernetes have eased software development, making it faster and better. Let's see where these are headed, looking at trends that are making life easier for developers.

time-read
10+ mins  |
November 2024
India's Leap in Supercomputing: Innovating for Tomorrow
Open Source For You

India's Leap in Supercomputing: Innovating for Tomorrow

As India strides towards self-sufficiency in supercomputing, embracing this evolution isn't just an option-it is pivotal for global competitiveness and technological leadership.

time-read
5 mins  |
November 2024