Evolution of Log Analysis Architectures
Open Source For You|August 2024
Going down memory lane, it's easy to see how changing needs have helped evolve log analysers since the mid-2000s. What worked then, will not work today. But the journey is to be cherished.
Krishna Mohan Koyya
Evolution of Log Analysis Architectures

It was the late nineties, and I had about 3-4 years of development experience by then. I had just transitioned from the GSM field to the world of network management systems (NMS) after joining a multinational company in Bengaluru. Although my master’s degree was in computer networks, NMS was entirely new to me. I was familiar with hubs, routers, switches, IP addresses, RFCs, and protocols, but only in theory. This was the first time I had hands-on experience with NMS.

From a technological standpoint, I was quite proficient. I coded in C++ and worked on HP-UX, a variant of UNIX. I had experience with large-scale, mission-critical systems. In a way, I was filled with youthful confidence when I began working with NMS.

In my new role, I was assigned the task of enhancing a log analysis tool as part of a larger NMS. NMS are structured around the FCAPS model, which stands for fault management, configuration management, accounting, performance, and security. This model is defined by ISO. The log analysis tool our team was developing fell under the fault management category. Figure 1 captures the arrangement.

The requirement was straightforward: the log analysis tool needed to gather system logs from all devices across the network, store them in a database, analyse them, and take necessary actions.

This NMS was intended for use by large service providers. In the US market, the adoption of GSM mobile phones was slower compared to VoIP (Voice over IP). While mobile telephony was just beginning in India, offices were transitioning from PSTN technology to VoIP, and VoIP phones were becoming commonplace on every employee’s desk. Each time a call was made or received, or a VoIP phone was powered on or off, a system log was generated. Our log analysis tool needed to collect and analyse all these logs.

This story is from the August 2024 edition of Open Source For You.

Start your 7-day Magzter GOLD free trial to access thousands of curated premium stories, and 9,000+ magazines and newspapers.

This story is from the August 2024 edition of Open Source For You.

Start your 7-day Magzter GOLD free trial to access thousands of curated premium stories, and 9,000+ magazines and newspapers.

MORE STORIES FROM OPEN SOURCE FOR YOUView All
Amazon Bedrock: A Boon for the Financial Services Industry
Open Source For You

Amazon Bedrock: A Boon for the Financial Services Industry

Amazon Bedrock is a fully managed service that provides access to foundation models from top AI providers, enabling organisations to build and scale generative AI applications. It is specifically designed to bring AI solutions to the financial sector. Let's explore all that it can do...

time-read
9 mins  |
December 2024
Quantum-Safe VPNs: The Future of Secure Communication
Open Source For You

Quantum-Safe VPNs: The Future of Secure Communication

As quantum computing continues to advance, it poses a significant threat to traditional cryptographic algorithms that secure our digital communications. Virtual private networks (VPNs), which rely heavily on encryption, are particularly vulnerable. Quantum-safe VPNs utilise post-quantum cryptographic algorithms to protect against quantum attacks.

time-read
3 mins  |
December 2024
Popular Open Source Toolkits for Quantum Machine Learning
Open Source For You

Popular Open Source Toolkits for Quantum Machine Learning

Quantum machine learning is becoming increasingly popular due to its ability to solve the complex problems of the AI age. Here are a few open source libraries and frameworks that help with quantum computations.

time-read
5 mins  |
December 2024
Quantum Computing: Harnessing Open Source for Innovation and Accessibility
Open Source For You

Quantum Computing: Harnessing Open Source for Innovation and Accessibility

We explore how open source initiatives are shaping the future of quantum computing, making it more accessible and driving innovation through collaboration.

time-read
9 mins  |
December 2024
How Quantum Computing Differs from Classical Computing
Open Source For You

How Quantum Computing Differs from Classical Computing

Despite being in its infancy, quantum computing has numerous potential applications in modelling, cybersecurity, AI/ML, and other fields. But how do quantum and classical computing compare with each other? Let's find out...

time-read
4 mins  |
December 2024
From Bits to Qubits: The Growth Story of Quantum Computing
Open Source For You

From Bits to Qubits: The Growth Story of Quantum Computing

Quantum computing may still be in the early stages of evolution, but its potential impact on everyday life is significant. We delve into the key concepts behind it, the reasons for its rapid growth, and how global advancements are shaping its future.

time-read
10+ mins  |
December 2024
Pytket: A Comprehensive Guide to Quantum Circuit Design
Open Source For You

Pytket: A Comprehensive Guide to Quantum Circuit Design

Pytket stands out as a powerful toolkit in the realm of quantum computing, offering a suite of features that cater to both researchers and industry practitioners. Its key strengths include optimisation, platform-agnostic support, flexible quantum circuit design and hybrid algorithm support. These features make Pytket a versatile tool for various quantum computing applications, from machine learning and cryptography to optimisation problems in industrial settings.

time-read
10+ mins  |
December 2024
Cirq: The Open Source Framework for Programming Quantum Computers
Open Source For You

Cirq: The Open Source Framework for Programming Quantum Computers

Explore the key features, capabilities, and impact of Cirq, an open source quantum computing framework developed by Google, on the quantum programming landscape.

time-read
4 mins  |
December 2024
The Role of Open Source in Accelerating Quantum AI
Open Source For You

The Role of Open Source in Accelerating Quantum AI

Here's an overview of how open source frameworks are being utilised to build quantum machine learning models, including quantum neural networks and quantum kernel methods. The challenges and future directions in the quantum AI landscape are also discussed.

time-read
7 mins  |
December 2024
Quantum Machine Learning: An Overview
Open Source For You

Quantum Machine Learning: An Overview

Quantum machine learning (QML) is a burgeoning field at the intersection of quantum computing and artificial intelligence. In recent years, the integration of quantum mechanics with machine learning algorithms has sparked substantial interest among researchers and technologists alike. Here's a quick look at the essentials of creating quantum algorithms for AI models, their practical use cases on open source platforms, and best practices for implementing these advanced algorithms.

time-read
7 mins  |
December 2024