Lakehouse Intelligence for Enterprise Analytics
Keywords:
Governed Lakehouse Intelligence, Lakehouse Architecture, Data Governance, MLOps, Scalable Data Pipelines, Feature Store, Data Domain Integration, Model Lifecycle Management, Enterprise Analytics, Data Engineering, Machine Learning Operations, Data Lakehouse.Abstract
Organizations adopting Lakehouse architectures seek a single system for data ingestion, controlled and monitored data domains, and robust support for data consumers. At the same time, enterprises must mature their MLOps practices—moving models from data science experimentation to governed, production-grade analytics. This maturity depends on reproducibility, adherence to corporate governance policies, integrated scheduling and orchestration, and a centralized Feature Store.
This paper introduces the Governed Lakehouse Intelligence Framework, a conceptual model built from modular, interoperable building blocks that establishes standards across three dimensions: Lakehouse Integration, Governance, and Modelling. By adhering to these standards, enterprises can incrementally build scalable data pipelines and MLOps-ready analytics capabilities.
The Framework is evaluated through a multi-faceted approach. First, the Principal Evaluation Approach establishes the need for such a framework among organizations operating Lakehouse architectures. A panel of domain experts then assesses the Framework’s contribution across four dimensions: scaled data domain integration, data governance compliance, enterprise MLOps readiness, and organizational agility and resiliency.
References
1. Armbrust, M., Das, T., Sun, L., Yavuz, B., Zhu, S., Murthy, M., Torres, J., van Hovell, H., Ionescu, A., Łuszczak, M., Şwitakowski, M., Szafrański, M., Li, X., Ueshin, T., Mokhtar, M., Boncz, P., Ghodsi, A., Paranjpye, S., Senster, P., ... Zaharia, M. (2020). Delta Lake: High-performance ACID table storage over cloud object stores. Proceedings of the VLDB Endowment, 13(12), 3411–3424.
2. Armbrust, M., Ghodsi, A., Xin, R., & Zaharia, M. (2021). Lakehouse: A new generation of open platforms that unify data warehousing and advanced analytics. In Proceedings of the Conference on Innovative Data Systems Research (CIDR).
3. Sekhar, P. C., Karri, M., Vidhya, S. S., Gadi, A. L., Lingineni, S. L., & Perada, A. (2025, September). Intrusion Detection in Smart Grids Using Deep Belief Network and K-Means Clustering Model for Electric Vehicle Charging Systems. In 2025 5th International Conference on Emerging Research in Electronics, Computer Science and Technology (ICERECT) (pp. 1-6). IEEE.
4. Harby, A. A., & Zulkernine, F. (2022). From data warehouse to lakehouse: A comparative review. In 2022 IEEE International Conference on Big Data (Big Data) (pp. 389–395).
5. Jain, P., Kraft, P., Power, C., Das, T., Stoica, I., & Zaharia, M. (2023). Analyzing and comparing lakehouse storage systems. In Proceedings of the Conference on Innovative Data Systems Research (CIDR).
6. Schneider, J., Gröger, C., Lutsch, A., Schwarz, H., & Mitschang, B. (2024). The lakehouse: State of the art on concepts and technologies. SN Computer Science, 5, Article 449.
7. Selvamuthu, C. M., Yerasuri, S. S., Manukonda, K. R. R., SnvRamanamurthy, B., Kwatra, C. V., & Hemalatha, P. K. (2025, November). IoT Detection and Authentication Using Machine Learning: A Cybersecurity Perspective. In 2025 International Conference on Emerging Engineering Technologies and Applications (IC-EETA) (pp. 417-422). IEEE.
8. Harby, A. A., & Zulkernine, F. (2025). Data lakehouse: A survey and experimental study. Information Systems, 127, Article 102460.
9. Janssen, N., Ilayperuma, T., Jayasinghe, J., Bukhsh, F., & Daneva, M. (2024). The evolution of data storage architectures: Examining the secure value of the data lakehouse. Journal of Data, Information and Management, 6, 309–334.
10. Ait Errami, S., Hajji, H., Ait El Kadi, K., & Badir, H. (2023). Spatial big data architecture: From data warehouses and data lakes to the LakeHouse. Journal of Parallel and Distributed Computing, 176, 70–79.
11. Movva, V. R. (2025). GENERATIVE AI-DRIVEN DEVOPS: LEVERAGING LARGE LANGUAGE MODELS FOR INTELLIGENT SOFTWARE DELIVERY AND ENGINEERING PRODUCTIVITY IN RETAIL ENTERPRISES. International Journal of Applied Mathematics, 38(2s), 1621-1631.
12. Sawadogo, P., & Darmont, J. (2021). On data lake architectures and metadata management. Journal of Intelligent Information Systems.
13. Azeroual, O., Schöpfel, J., Ivanovic, D., & Nikiforova, A. (2022). Combining data lake and data wrangling for ensuring data quality in CRIS. Procedia Computer Science, 211, 3–16.
14. Begoli, E., Bhattacharya, A., & Kusnezov, D. (2021). A lakehouse architecture for biomedical research and mega-biobanks. In 2021 IEEE International Conference on Big Data (Big Data).
15. Kolla , S. K. (2025). Generative AI-Enabled Precision Medicine Framework For Longitudinal EHR Intelligence and Personalized Clinical Decision Support. INTERNATIONAL JOURNAL OF ADVANCES IN SIGNAL AND IMAGE SCIENCES, 41-50. https://doi.org/10.29284/kcgr6k25
16. Orescanin, D., & Hlupic, T. (2021). Data lakehouse: A novel step in analytics architecture. In Proceedings of the 44th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).
17. Park, S., et al. (2023). Design of vessel data lakehouse with big data and AI analysis technology. Electronics.
18. Levandoski, J., et al. (2024). BigLake: BigQuery’s evolution toward a multi-cloud lakehouse. In Proceedings of the ACM SIGMOD International Conference on Management of Data.
19. Inmon, W. H., Levins, M., & Srivastava, R. (2021). Building the data lakehouse. Technics Publications.
20. Mahadevan, S. (2025). DRIVING SUSTAINABILITY IN AUTO & MANUFACTURING SECTORS: THE ROLE OF CLOUD-BASED SERVERLESS AUTOMATION FOR ERP SYSTEMS. International Journal of Applied Mathematics, 38(7s), 1813-1825.
21. Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2012). Spark: Cluster computing with working sets. In Proceedings of the 2nd USENIX Workshop on Hot Topics in Cloud Computing.
22. Armbrust, M., Das, T., Torres, J., Yavuz, B., Zhu, S., Xin, R., Ghodsi, A., Stoica, I., & Zaharia, M. (2018). Structured streaming: A declarative API for real-time applications in Apache Spark. In Proceedings of the 2018 ACM International Conference on Management of Data (pp. 601–613).
23. Armbrust, M., Xin, R., Lian, C., Huai, Y., Liu, D., Bradley, J. K., Meng, X., Kaftan, T., Franklin, M. J., Ghodsi, A., & Zaharia, M. (2015). Spark SQL: Relational data processing in Spark. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (pp. 1383–1394).
24. Melnik, S., Gubarev, A., Long, J. J., Romer, G., Shivakumar, S., Tolton, M., & Vassilakis, T. (2010). Dremel: Interactive analysis of web-scale datasets. Proceedings of the VLDB Endowment, 3(1–2), 330–339.
25. Reddy Manukonda, K. R. (2023). Investigating the role of exploratory testing in agile software development: a case study analysis. Journal of Artificial Intelligence & Cloud Computing, 2(4), 1-5.
26. Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1), 107–113.
27. Stonebraker, M., Abadi, D. J., Batkin, A., Chen, X., Cherniack, M., Ferreira, M., Lau, E., Lin, A., Madden, S., O’Neil, E., O’Neil, P., Rasin, A., Tran, N., & Zdonik, S. (2005). C-Store: A column-oriented DBMS. In Proceedings of the 31st International Conference on Very Large Data Bases (pp. 553–564).
28. Abadi, D. J., Boncz, P. A., & Harizopoulos, S. (2009). Column-oriented database systems. Proceedings of the VLDB Endowment, 2(2), 1664–1665.
29. Sadalage, P. J., & Fowler, M. (2012). NoSQL distilled: A brief guide to the emerging world of polyglot persistence. Addison-Wesley.
30. Kleppmann, M. (2017). Designing data-intensive applications: The big ideas behind reliable, scalable, and maintainable systems. O’Reilly Media.
31. Nargesian, F., Zhu, E., Miller, R. J., Pu, K. Q., & Arocena, P. C. (2019). Data lake management: Challenges and opportunities. Proceedings of the VLDB Endowment, 12(12), 1986–1989.
32. Bandi, V. D. V. K. AI-Based Anomaly Detection Frameworks in Distributed Enterprise Data Systems.
33. Hai, R., Geisler, S., & Quix, C. (2016). Constance: An intelligent data lake system. In Proceedings of the 2016 International Conference on Management of Data (pp. 2097–2100).
34. Fang, H. (2015). Managing data lakes in big data era: What’s a data lake and why has it become popular in data management ecosystem. In 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (pp. 820–824).
35. Sawadogo, P., & Darmont, J. (2021). Data lakes: A survey of concepts and architectures. International Journal of Data Warehousing and Mining.
36. Vassiliadis, P. (2009). A survey of extract–transform–load technology. International Journal of Data Warehousing and Mining, 5(3), 1–27.
37. Golfarelli, M., Rizzi, S., & Cella, I. (2004). Beyond data warehousing: What’s next in business intelligence? In Proceedings of the 7th ACM International Workshop on Data Warehousing and OLAP (pp. 1–6).
