Literature Survey for the Comparative Study of Various High Performance Computing Techniques

International Journal of Computer Trends and Technology (IJCTT)          
© 2015 by IJCTT Journal
Volume-27 Number-2
Year of Publication : 2015
Authors : Zahid Ansari, Asif Afzal, Moomin Muhiuddeen, Sudarshan Nayak


Zahid Ansari, Asif Afzal, Moomin Muhiuddeen, Sudarshan Nayak "Literature Survey for the Comparative Study of Various High Performance Computing Techniques". International Journal of Computer Trends and Technology (IJCTT) V27(2):80-86, September 2015. ISSN:2231-2803. Published by Seventh Sense Research Group.

Abstract -
The advent of high performance computing (HPC) and graphics processing units (GPU), present an enormous computation resource for large data transactions (big data) that require parallel processing for robust and prompt data analysis. In this paper, we take an overview of four parallel programming models, OpenMP, CUDA, MapReduce, and MPI. The goal is to explore literature on the subject and provide a high level view of the features presented in the programming models to assist high performance users with a concise understanding of parallel programming concepts.

[1] OpenMP Architecture Review Board, ―OpenMP Application Program Interface, 2008, spec30.pdf.
[2] B. Barney, Introduction to Parallel Computing, Lawrence Livermore National Laboratory, 2007,
[3] J. Diaz, C.Munoz-Caro, and A. Nino, ―A survey of parallel programming models and tools in the multi and many-core era, IEEE Transactions on Parallel and Distributed Systems, vol. 23, no.8, pp.1369–1386, 2012.
[4] W. Gropp, S. Huss-Lederman, A. Lumsdaine et al., MPI: The Complete Reference, the MPI-2 Extensions, vol. 2, The MIT Press, 1998.
[5] G. Jost, H. Jin, D. Mey, and F. Hatay, ―Comparing the OpenMP, MPI, and hybrid programming paradigm on an SMP cluster, in Proceedings of the 5th European workshop on OpenMP (EWOMP’03), 2003.
[6] J.Dean and S.Ghemawat, ―MapReduce: simplified data processing on large clusters, Communications of the ACM, vol. 51, no. 1, pp. 107–113, 2008.
[7] S. Ghemawat, H. Gobioff, and S.-T. Leung, ―The Google file system, in Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP ’03), pp. 29–43, October 2003.
[8] C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis,―Evaluating MapReduce for multi-core and multiprocessor systems, in Proceedings of the 13th IEEE International Symposium on High Performance Computer Architecture (HPCA’07), pp. 13–24, Scottsdale, AZ, USA, February 2007.
[9] S. J. Plimpton and K. D. Devine,―MapReduce in MPI for largescale graph algorithms, Parallel Computing, vol. 37, no. 9, pp.610–632, 2011.
[10]. NVIDIA, ―CUDA C Programming Guide, no. July. NVIDIA Corporation, 2013.
[11]. Wikipedia, ―General-purpose computing on graphics processing units, 2013.
[12]. Y.Charlie Hu, Honghui Lu, Alan L Cox and Willy Zwaenepoel, ―OpenMp for Networks of SMPs, Parallel Processing , 13th International and 10th Symposium on Parallel and Distributed Processing, pp. 302-310, 1999.
[13]. John Bircsak, Peter Craig, RaeLyn Crowell, Zarka Cvetanovic, Jonathan Harris, C. Alexander Nelson and Carl D. Offner, ―Extending OpenMP For NUMA Machines, SC `00 Proceedings of the 2000 ACM/IEEE conference on Supercomputing, Article no. 48, 2000.
[14]. Zaid Abdi Alkareem Alyasseri , Kadhim Al-Attar, Mazin Nasser and Ismail, ―Parallelize Bubble and Merge Sort Algorithms Using Message Passing Interface (MPI), Publication eprint arXiv:1411.5283, 2014.
[15]. Pavan Balaji, Darius Buntinas, David Goodell, William Gropp, Torsten Hoefler, Sameer Kumar, Ewing Lusk, Rajeev Thakur and Jesper Larsson Traff, ―MPI on Millions of Core, Parallel Proceesing Letter, vol. 21, issue 01, 2011.
[16]. Colby Ranger, Ramanan Raghuraman, Arun Penmetsa, Gary Bradski and Christos Kozyrakis, ―Evaluating MapReduce for Multicore and Multiprocessor Systems, in Proceedings of the 13th IEEE International Symposium on High Performance Computer Architecture (HPCA `07), pp. 13-24, Scottsdale, AZ, USA, February 2007.
[17]. Hung-chih Yang, Ali Dasdan, Ruey-Lung Hsiao and D. Stott Parker,―Map-reduce-merge: Simplified Relational Data Processing on Large Clusters, in Proceedings ACM SIGMOD international Conference on Management of Data, pp. 1029-1040, 2007.
[18]. Wladimir J. van der Laan, Andrei C. Jalba, and Jos B.T.M. Roerdink, ―Accelerating Wavelet Lifting on Graphics Hardware Using CUDA, IEEE transactions on Parallel and Distributed, vol. 22, issue 01, pp. 132-146, 2010.
[19]. Jedrzej Kowalczuk, Eric T. Psota, Lance C. Pérez, ―Real-time Stereo Matching on CUDA using an Iterative Refinement Method for Adaptive Support-Weight Correspondences, IEEE transactions on Circuits and System for Video Technologies, vol. 23,isuue 01, pp. 94-104,2012.
[20]. The OpenMp Forum. OpenMp Fortran Application Program Interface, Version 1.0, http:/, Oct 1997.
[21]. The OpenMp Forum. OpenMp C and C++ Application Program Interface, Version 1.0, http:/, Oct 1998.
[22]. CUDA Zone,, Oct. 2011.
[23]. Nvidia Developer Zone,, Oct. 2011.
[24]. D. Kirk and W. Hwu, Programming Massively Parallel Processors: A Hands-on Approach. Morgan Kaufmann, 2010.
[25]. Nvidia Company. Nvidia CUDA Programming Guide, v3.0, 2010.
[26]. Nvidia Company. Nvidia CUDA C Programming Best Practices Guide, Version 3.0, 2010.
[27]. T.G. Mattson, B.A. Sanders, and B. Massingill, Patterns for Parallel Programming. Addison-Wesley Professional, 2005.
[28]. Sol Ji Kang, Sang Yeon Lee, and KeonMyung Lee, ―Performance Comparison of OpenMP, MPI, and MapReduce in Practical Problems, Advances in Multimedia, Research Article, 2014.
[29]. Shuai Che_, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W. Sheaffer, Kevin Skadron, ―A performance study of general-purpose applications on graphics processors using CUDA, Published in J. Parallel Distrib. Comput.,vol. 68, pp. 1370-1380, 2008.
[30]. Cleverson Lopes Ledur, Carlos M. D. Zeve, Julio C. S. dos Anjos, ―Comparative Analysis of OpenACC, OpenMP and CUDA using Sequential and Parallel Algorithms, 11th Workshop on Parallel and Distributed Processing (WSPPD), Universidade Luterana do Brasil, Information Systems, BR 116, n. 5.724, Moradas da Colina – Guaba/RS.

OpenMP, MPI, CUDA, MapReduce, GPU.