AI-DRIVEN DATA LIFECYCLE OPTIMIZATION SYSTEM

Abstract: Today, most organizations store huge amounts of data without knowing what is still useful and what is no longer needed. They often keep everything, which wastes storage space and increases unnecessary costs. The main issue is the lack of a proper system to automatically decide when data should be retained or removed, as existing methods rely on fixed rules like deleting data after a certain number of years, which is not effective for all types of data. To address this, this paper presents an AI-Driven Data Lifecycle Optimization System that scans data, evaluates its usefulness, and decides whether it should be kept or deleted. Instead of fixed rules, it uses machine learning to classify data based on importance and survival analysis to predict when data is no longer needed. The system is built using Python for data processing, R for lifecycle prediction, SQL for managing retention schedules, and Power BI for visualization. The results show that this approach reduces storage costs, saves time, and improves data management compared to traditional rule- based methods, providing a smarter and more efficient solution for managing data lifecycle.

Keywords: Data Lifecycle, AI-Driven System, Machine Learning, Survival Analysis, Data Retention, Power BI, Data Optimization.