Big Data Analysis for Log file and Malware
Malware, Hadoop, MapReduce, Log files, log analyzer, Heterogeneous database
There are various applications which have a huge database. All databases maintain log files that keep records of database changes. This can include tracking various user events. Apache Hadoop can be used for log processing at scale. Log files have become a standard part of large applications and are essential in operating systems, computer networks and distributed systems. Log files are often the only way to identify and locate an error in software, because log file analysis is not affected by any time based issues known as probe effect. This is opposite to analysis of a running program, when the analytical process can interfere with time-critical or resource critical conditions within the analyzed program. Log files are often very large and can have complex structure. Although the process of generating log files is quite simple and straightforward, log file analysis could be a tremendous task that requires enormous computational resources, long time and sophisticated procedures. This often leads to a common situation, when log files are continuously generated and occupy valuable space on storage devices, but nobody uses them and utilizes enclosed information. The overall goal of this project is to design a generic log analyzer using hadoop map-reduce framework. This generic log analyzer can analyze different kinds of log files such as- Email logs, Web logs, Firewall logs Server logs, Call data logs. Today each and every day a lot of data is generated in increasing order. This is because of today’s ecommerce and easy to use technologies. Also, there is increasing number of vulnerabilities in this large data. There are counter measures for these vulnerabilities like antiviruses or anti-malwares. But, for scanning a large data in less time its difficult. So using Hadoop and MapReduce technology we can scan it parallely in less time. In this project we are scanning malware using Hadoop and MapReduce.
"Big Data Analysis for Log file and Malware", IJSDR - International Journal of Scientific Development and Research (www.IJSDR.org), ISSN:2455-2631, Vol.1, Issue 7, page no.140 - 145, July-2016, Available :https://ijsdr.org/papers/IJSDR1607023.pdf
Volume 1
Issue 7,
July-2016
Pages : 140 - 145
Paper Reg. ID: IJSDR_160383
Published Paper Id: IJSDR1607023
Downloads: 000346998
Research Area: Engineering
Country: Ahmednagar, Maharashtra, India
ISSN: 2455-2631 | IMPACT FACTOR: 9.15 Calculated By Google Scholar | ESTD YEAR: 2016
An International Scholarly Open Access Journal, Peer-Reviewed, Refereed Journal Impact Factor 9.15 Calculate by Google Scholar and Semantic Scholar | AI-Powered Research Tool, Multidisciplinary, Monthly, Multilanguage Journal Indexing in All Major Database & Metadata, Citation Generator
Publisher: IJSDR(IJ Publication) Janvi Wave
