IJBTT

Enhancing Cybersecurity Through the Unification of Data Analytics, Artificial Intelligence, and Machine Learning in Big Data Cloud Environments: A Databricks Lakehouse Approach
	© 2023 by IJCTT Journal
	Volume-71 Issue-6
	Year of Publication : 2023
	Authors : Mayur Katariya, Digan Parikh
	DOI :  10.14445/22312803/IJCTT-V71I6P104

How to Cite?

Mayur Katariya, Digan Parikh, "Enhancing Cybersecurity Through the Unification of Data Analytics, Artificial Intelligence, and Machine Learning in Big Data Cloud Environments: A Databricks Lakehouse Approach," International Journal of Computer Trends and Technology, vol. 71, no. 6, pp. 25-29, 2023. Crossref, https://doi.org/10.14445/22312803/IJCTT-V71I6P104

Abstract
“Data without analytics is dormant, the cloud without security is vulnerable, and AI without data is blind. Together, they form an unstoppable force in the realm of cybersecurity.” The cybersecurity industry faces numerous challenges in combating the ever-evolving threat landscape. This scholarly article aims to shed light on these challenges and explores how Databricks' lakehouse cloud architecture offers potential solutions. The article provides a comprehensive detail of the current state of the cybersecurity industry and maturity curve and highlights the pressing issues that organizations encounter. The article focuses on the utilization of Databricks' lakehouse architecture as a means to address these challenges. Furthermore, the paper delves into the key features and capabilities of Databricks' lakehouse architecture that enable organizations to efficiently analyze and govern vast amounts of security data, detect anomalies, and identify potential threats in real time. The paper concludes by drawing on real-world examples and industry case studies. This paper offers a detailed overview of the real-world cybersecurity architecture examples, data flow and how different companies are embracing the lakehouse architecture to power their use cases. By unifying the combined power of data analytics, artificial intelligence (AI), and machine learning (ML), the cloud lakehouse architecture helps organizations to break silos, derive actionable insights, offer a promising approach for bolstering cybersecurity defenses, and enhance their proactive cybersecurity strategies at scale on cloud.

Keywords
Databricks lakehouse, Cloud, Bigdata, Analytics, Artificial Intelligence(AI), Machine Learning(ML), Cybersecurity.

Reference

[1] Aaron Drapkin, Data Breaches That Have Happened in 2022 and 2023 so far, 2023. [Online]. Available: https://tech.co/news/data-breaches-updated-list
[2] Jade Artry, Average Data Breach Cost for US Businesses Almost $10 Million, 2023. [Online]. Available: https://tech.co/news/average-us-company-data-breach-cost-10-million
[3] Databricks Website. [Online]. Available: https://www.databricks.com/blog/announcing-general-availability-databricks-sql-serverless
[4] Databricks Website, what is Delta Lake?, 2023. [Online]. Available: https://docs.databricks.com/delta/index.html
[5] Databricks Website, GDPR & CCPA. [Online]. Available: https://docs.databricks.com/security/privacy/gdpr-delta.html
[6] Databricks Website, Compliance Security and Trust Center. [Online]. Available: https://www.databricks.com/trust#compliance
[7] Databricks Website, MLOps. [Online]. Available: https://www.databricks.com/resources/ebook/the-big-book-of-mlops
[8] Databricks Website, Unity Catalog. [Online]. Available: https://docs.databricks.com/data-governance/unity-catalog/index.html
[9] Databricks Website, Medallion-Architecture. [Online]. Available: https://www.databricks.com/glossary/medallion-architecture
[10] Spark + AI Summit, 2018 Vimeo Website. [Online]. Available: https://vimeo.com/274267634
[11] Data and AI Summit, Accidentally Building a Petabyte-Scale Cybersecurity Data Mesh in Azure With Delta Lake at HSBC, YouTube Website, 2022. [Online]. Available: https://www.youtube.com/watch?v=G9x-1s-1TJI
[12] Data and AI Summit, Akamai Interactive Analytics on a Massive Scale Using Delta Lake, YouTube Website, 2022. [Online]. Available: https://www.youtube.com/watch?v=_K9gIOsP4Os
[13] Databricks Website Customers – Akamai. [Online]. Available: https://www.databricks.com/customers/akamai
[14] Databricks Website Customers - Abnormal Security. [Online]. Available: https://www.databricks.com/customers/abnorma