Data Lake empowers key steps in Customer Journey

Bragadeesh Sundararajan, Chief Data Science Officer, Dvara KGFS shares the data lake journey of his organization:

Ravi Lalwani: What is the data lake journey of Dvara KGFS and what were the key milestones?

Bragadeesh Sundararajan: A data lake journey is a process of storing, transforming, and analyzing large amounts of data in a central repository known as a data lake. Our journey involved the following steps:

Data Ingestion: This is the process of collecting and storing data from various transactional databases. Data is ingested in real-time or batch mode, depending on the requirements of the organization.

Data Storage: Once the data has been ingested, it needs to be stored in a central repository. A data lake is a type of repository that can store structured and unstructured data in its raw form, allowing it to be accessed and analyzed later. We are using MySQL as the central data repository for all data ingestions.

Data Transformation: After the data has been stored in the data lake, it typically needs to be transformed and cleaned before it can be analyzed. This may involve processes such as filtering, aggregating, and joining data from different sources. Python scripts are used to transform the data and store it in cleaned data.

Data Analysis: Once the data has been transformed, it can be analyzed using a variety of tools and techniques, such as SQL queries, machine learning algorithms, or visualization tools. The results of this analysis can be used to gain insights, make decisions, and drive business value. Python scripts are used for ML, AI, and data analytics.