Amazon Redshift: Performance Tuning and Optimization |
||
 |
 |
|
| © 2023 by IJCTT Journal | ||
| Volume-71 Issue-2 |
||
| Year of Publication : 2023 | ||
| Authors : Amol Kulkarni | ||
| DOI : 10.14445/22312803/IJCTT-V71I2P107 | ||
How to Cite?
Amol Kulkarni, "Amazon Redshift: Performance Tuning and Optimization," International Journal of Computer Trends and Technology, vol. 71, no. 2, pp. 40-44, 2023. Crossref, https://doi.org/10.14445/22312803/IJCTT-V71I2P107
Abstract
Data is the lifeline for any enterprise in today's world. Exabytes of data are generated daily due to billions of transactions worldwide, social media, smartphones, and gaming apps use. It is most crucial for an enterprise not only to collect exabytes of data in the OLTP ERP System but also to analyze it for Decision making, predictive analysis, trend analysis and gain insights into business using the Data Warehouse OLAP System. With the advent of Cloud technology, it is possible to handle exabytes of data in cloud-based OLTP ERP systems and analyze it in cloud-based Data Warehouse OLAP Systems. There are various cloud-based data warehouse systems available in the market, but an enterprise has to choose the right system based on cost, integration with the OLTP ERP System, integration with existing business intelligence tools, security, scalability, data loading and query performance. This paper demonstrates how the Amazon Redshift performance optimization options made it the first choice for any enterprise to load huge amounts of data and provide it quickly for decision-making and analytic reporting purposes.
Keywords
Amazon S3, AWS, Business Intelligence (BI), Cloud, Data warehouse, ETL, Machine Learning (ML), SQL.
Reference
[1] Amazon Redshift website: Amazon Redshift System Overview : [Online]. Available: https://docs.aws.amazon.com/redshift/latest/dg/c_redshift_system_overview.html
[2] Amazon Redshift website: Amazon Redshift High Performance Queries:[Online].Available:
https://docs.aws.amazon.com/redshift/latest/dg/c_challenges_achieving_high_performance_queries.html
[3] Amazon Redshift website: Amazon Redshift Automatic Table Optimization: [Online]. Available: https://aws.amazon.com/blogs/big-data/optimizing-tables-in-amazon-redshift-using-automatic-table-optimization/
[4] Amazon Redshift website: Amazon Redshift Automatic Workload Management WLM: [Online]. Available:https://docs.aws.amazon.com/redshift/latest/dg/automatic-wlm.html
[5] Amazon Redshift website: Amazon Redshift Concurrency Scaling: [Online]. Available: https://docs.aws.amazon.com/redshift/latest/dg/concurrency-scaling.html
[6] Amazon Redshift website: Amazon Redshift Implementing Workload Management: [Online]. Available: https://docs.aws.amazon.com/redshift/latest/dg/cm-c-implementing-workload-management.html
[7] Amazon Redshift website: Amazon Redshift Materialized Views: [Online]. Available: https://docs.aws.amazon.com/redshift/latest/dg/materialized-view-overview.html
[8] Amazon Redshift website: Amazon Redshift Database: [Online]. Available: https://docs.aws.amazon.com/redshift/latest/dg/r_Database_objects.html
[9] Amazon Redshift website: Amazon Redshift DynamoDB: [Online]. Available: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/RedshiftforDynamoDB.html
[10] Anurag Gupta et al., “Amazon Redshift and the Case for Simpler Data Warehouses,” ACM SIGMOD International Conference on Management of Data, pp. 1917–1923, 2015. Crossref, https://doi.org/10.1145/2723372.2742795
[11] Forbes Council Post : Five Critical Reasons to Move Your Legacy Data Warehouse to the Cloud [Online]. Available: https://www.forbes.com/sites/forbestechcouncil/2020/02/26/five-critical-reasons-to-move-your-legacy-data-warehouse-to-the-cloud/?sh=5c866fb3205a
[12] Daniel Abadi, Peter Boncz, and Stavros Harizopoulos, “The Design and Implementation of Modern Column-Oriented Database Systems,” Foundations and Trends in Databases, vol. 5, no. 3, pp. 197–280 2012. Crossref, https://doi.org/10.1561/1900000024
[13] Caius Brindescu et al., “Integrate Etleap with Amazon Redshift Streaming Ingestion (preview) to make data available in seconds,” AWS Big Data Blog, 2021. [Online]. Available: https://blog.etleap.com/integrate-etleap-with-amazon-redshift-streaming-ingestion-preview-to-make-data-available-in-seconds
[14] Gerardus Blokdyk, Amazon Redshift Complete Self-Assessment Guide, Createspace Independent Publishing, 2017.
[15] Stefan Bauer, Getting Started with Amazon Redshift, Packt Publishing, 2013.
[16] Shruti Worlikar et al., Amazon Redshift Cookbook: Recipes for building modern data warehousing solutions, Packt Publishing, 2021.
[17] Simplilearn, Cloud Computing Architecture, 2019. [Online]. Available: https://www.simplilearn.com/cloud-computing-architecture-article
[18] Nandula Rohan Kausik, Nandula Nikhil Kartik, and G. Soma Sekhar, "Personal Expense Tracker utilizing Amazon Web Services," International Journal of Computer Trends and Technology, vol. 70, no. 10, pp. 8-14, 2022. Crossref, https://doi.org/10.14445/22312803/IJCTT-V70I11P102
[19] Chandan Patra, “Amazon DynamoDB: What It Is and 10 Things You Should Know,” Cloud Academy Blog, 2022. [Online]. Available: https://cloudacademy.com/blog/amazon-dynamodb-ten-things/
[20] AWS Redshift Architecture: 7 Important Components, HEVO, 2017. [Online]. Available: https://hevodata.com/blog/redshift-architecture/
[21] Nay Lin Soe, “Concurrency Scaling in AWS Redshift,” Servian, 2019. [Online]. Available: https://servian.dev/concurrency-scaling-in-amazon-redshift-f8a3a06cc087
[22] Mark Smallcombe, “We Tested Amazon Redshift Concurrency Scaling: Here are our Results,” Integrate, 2021. [Online]. Available: https://www.integrate.io/blog/we-tested-amazon-redshift-concurrency-scaling-here-are-our-results/
[23] AWS Redshift Spectrum? [Online]. Available: https://ahana.io/answers/what-is-aws-redshift-spectrum/
[24] A Brief Summary of Compound Sort Key and Interleaved Sort Key with Performance Tests | Redshift, [Online]. Available: https://dev.classmethod.jp/articles/redshift-sortkey-usecase-en/
[25] Adam Gatt, Automate Your Amazon Redshift Performance Tuning With Automatic Table Optimization, 2021. [Online]. Available: https://noise.getoto.net/2021/10/06/automate-your-amazon-redshift-performance-tuning-with-automatic-table-optimization/
[26] AWS Snowflake vs. Redshift: Choosing a Modern Data Warehouse, [Online]. Available: https://www.stitchdata.com/resources/snowflake-vs-redshift/
[27] Akshay kumar, Dr.Divakar H R, and Dr.B R Prakash, “Elastic Search Usage of Analysis for Resource Scalability in AWS,” International Journal of Computer Trends and Technology, vol. 69, no. 8, pp. 11-14, 2021. Crossref, https://doi.org/10.14445/22312803/IJCTT-V69I8P10