A Web Crawler Design for Data Warehousing
Prof. Leena H. Patil, AnkitKhobragade, Priyakant Satpudke, Nikhil Sangani "A Web Crawler Design for Data Warehousing". International Journal of Computer Trends and Technology (IJCTT) V50(3):151-154, August 2017. ISSN:2231-2803. www.ijcttjournal.org. Published by Seventh Sense Research Group.
The size of the web is becoming a focus for research activities. The internet is the largest collection of data today. For this computer programs need to conduct any large scale processing of web pages. So we need the use of web crawler at some stage in order to fetch the pages that should be analysed. A web crawler is a program which browses the internet in a methodical, automated manner. This process is called web crawling. A search engine uses web crawler to collect web pages from internet and the web crawler collects it by web crawling. In this paper we have reviewed a web crawler design for data warehousing which allows to search in offline mode also.
 Mini Singh Ahuja Dr Jatinder Singh BalVarnica,“Web Crawler: Extracting the Web Data”, International Journal of Computer Trends and Technology (IJCTT) – volume 13 number 3 – July 2014  Pavalam, S. M., SV Kashmir Raja, Felix K. Akorli, and M. Jawahar, “A Survey of Web Crawler Algorithms,” International Journal of Computer Science, vol. 8, iss. 6, no 1, Nov. 2011.  Component of web search system figure, Accessed July 25,2017. https://www.google.co.in/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&ved=0ahUKEwionMns_aTVAhXLpo8KHaB0BPQQjRwIBw&url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_crawler&psig=AFQjCNGV10hs3Dm9EBk_yJgFmXskFXXq6g&ust=1501090566478463  http://www.mathcs.emory.edu/~cheung/Courses/171/Syllabus/11-Graph/bfs.html  https://www.researchgate.net/figure/315347498_fig1_Fig2-Pseudocode-for-Best-First-Search-algorithm  http://db.cs.duke.edu/courses/fall11/cps149s/notes/a_star.pdf  Shalini Sharma,“Web Crawler”, International Journal of Advanced Research in Computer Science and Software Engineering- Volume 4, Issue 4, April 2014  Aviral Nigam, “Web Crawling Algorithms”, International Journal of Computer Science and Artificial Intelligence Sept. 2014, Vol. 4 Iss. 3, PP. 63-67
Web crawler; Search engine; Offline mode.