INTERNATIONAL JOURNAL OF SCIENTIFIC DEVELOPMENT AND RESEARCH International Peer Reviewed & Refereed Journals, Open Access Journal ISSN Approved Journal No: 2455-2631 | Impact factor: 8.15 | ESTD Year: 2016
open access , Peer-reviewed, and Refereed Journals, Impact factor 8.15
There are various applications which have a huge database. All databases maintain log files that keep records of database changes. This can include tracking various user events. Apache Hadoop can be used for log processing at scale. Log files have become a standard part of large applications and are essential in operating systems, computer networks and distributed systems. Log files are often the only way to identify and locate an error in software, because log file analysis is not affected by any time based issues known as probe effect. This is opposite to analysis of a running program, when the analytical process can interfere with time-critical or resource critical conditions within the analyzed program. Log files are often very large and can have complex structure. Although the process of generating log files is quite simple and straightforward, log file analysis could be a tremendous task that requires enormous computational resources, long time and sophisticated procedures. This often leads to a common situation, when log files are continuously generated and occupy valuable space on storage devices, but nobody uses them and utilizes enclosed information. The overall goal of this project is to design a generic log analyzer using hadoop map-reduce framework. This generic log analyzer can analyze different kinds of log files such as- Email logs, Web logs, Firewall logs Server logs, Call data logs. Today each and every day a lot of data is generated in increasing order. This is because of today’s ecommerce and easy to use technologies. Also, there is increasing number of vulnerabilities in this large data. There are counter measures for these vulnerabilities like antiviruses or anti-malwares. But, for scanning a large data in less time its difficult. So using Hadoop and MapReduce technology we can scan it parallely in less time. In this project we are scanning malware using Hadoop and MapReduce.
"Big Data Analysis for Log file and Malware", International Journal of Science & Engineering Development Research (www.ijsdr.org), ISSN:2455-2631, Vol.1, Issue 7, page no.140 - 145, July-2016, Available :http://www.ijsdr.org/papers/IJSDR1607023.pdf
Downloads:
000336257
Publication Details:
Published Paper ID: IJSDR1607023
Registration ID:160383
Published In: Volume 1 Issue 7, July-2016
DOI (Digital Object Identifier):
Page No: 140 - 145
Publisher: IJSDR | www.ijsdr.org
ISSN Number: 2455-2631
Facebook Twitter Instagram LinkedIn