Survey on Big Data: Management and Challenges

International Journal of Computer Trends and Technology (IJCTT)          
© 2015 by IJCTT Journal
Volume-20 Number-1
Year of Publication : 2015
Authors : K.Indira Gandhi , Sri.C.Sreedhar


K.Indira Gandhi , Sri.C.Sreedhar "Survey on Big Data: Management and Challenges". International Journal of Computer Trends and Technology (IJCTT) V20(1):33-36, Feb 2015. ISSN:2231-2803. Published by Seventh Sense Research Group.

Abstract -
Today we live in the digital world. With increased digitization the amount of structured and unstructured data being created and stored is exploding. The data is being generated from various sources - transactions, social media, sensors, digital images, videos, audios and click streams for domains including healthcare, retail, energy and utilities. Big Data is therefore an all-encompassing term for any collection of large data sets that were difficult to process. However, the fast growth rate of such large data generates numerous challenges, such as the rapid growth of data, transfer speed, diverse data, and security. The current volume of big data sets is too complicated to be managed and processed by conventional relational databases & data warehousing technologies. This development calls for new system architectures for data acquisition, transmission, storage, and large-scale data processing mechanisms. This paper presents various big data definitions, challenges, approaches and the system architecture.

[1] Worldometers, “Real time world statistics,” 2014,
[2] James Manyika, et al. Big data: The next frontier for innovation, competition, and productivity. [Online] Available from:
[3] Edd Dumbill. What is big data? [Online] Availablefrom: [Accessed 9th July 2012].
[4] IDC. Worldwide Big Data Technology and Services 2012-2015 Forecast. [Online] Availablefrom: [Accessed 9th July 2012].
[6] S. Hendrickson, Getting Started with Hadoop with Amazon’s Elastic MapReduce, EMR, 2010.
[7] M. Hilbert and P. López, “The world's technological capacity to store, communicate, and compute information,” Science, vol. 332, no. 6025, pp. 60–65, 2011. View at Publisher • View at Google Scholar • View at Scopus
[8] J. Manyika, C. Michael, B. Brown et al., “Big data: The next frontier for innovation, competition, and productivity,” Tech. Rep., Mc Kinsey, May 2011S.
[9] A. Wahab, M. Helmy, H. Mohd, M. Norzali, H. F. Hanafi, and M. F. M. Mohsin, “Data pre-processing on web server logs for generalized association rules mining algorithm,” Proceedings of World Academy of Science: Engineering & Technology, pp. 48–53.
[10] V. Chandramohan and K. Christensen, “A first look at wired sensor networks for video surveillance systems,” in Proceedings of the 27th Annual IEEE Conference on Local Computer Networks (LCN '02), pp. 728–729, November 2002. View at Publisher • View at Google Scholar
[11] SAND CDBMS 6 at dunnhumby - big data, big users, security, and analytics. YouTube. [Online] 15 September 2010.
[12] M. Lenzerini, ``Data integration: A theoretical perspective''
[13] studies/277256/getting-relevant.thtml.
[14] Y. Zhang, J. Callan, and T. Minka, ``Novelty and redundancy detection in adaptive filtering,''
[15] A. H. Eschenfelder, Data Mining and Knowledge Discovery Handbook, vol. 14.
[16] statistics/
[17] Stephen Kaisler, Frank Armour, J. Alberto Espinosa, William Money, “Big Data: Issues and Challenges Moving Forward”, IEEE, 46th Hawaii International Conference on System Sciences, 2013.
[18] E. B. S. D. D. Agrawal et al., ``Challenges and opportunities with big” _A community white paper developed by leading researchers across the united states,'' The Computing Research Association, CRA White Paper, Feb. 2012
[20] Sam Madden, “From Databases to Big Data”, IEEE, Internet Computing, May-June 2012.
[21] Martin Courtney, “The Larging-up of Big Data”, IEEE, Engineering & Technology, September 2012.
[22] D. Fisher, R. DeLine, M. Czerwinski, and S. Drucker, ``Interactions with big data analytics,'' Interactions, vol. 19, no. 3, pp. 50_59, May 2012.

Big data, Datasets, generation, acquisition, storage, analysis, visualization, challenges.