The popularity, cost-effectiveness and ease of information exchange that electronic mails offer to electronic device users has been plagued with the rising number of unsolicited or spam emails. Driven by the need to protect email users from this growing menace, research in spam email filtering/detection systems has being increasingly active in the last decade. However, the adaptive nature of spam emails has often rendered most of these systems ineffective. While several spam detection models have been reported in literature, the reported performance on an out of sample test data shows the room for more improvement. Presented in this research is an improved spam detection model based on Extreme Gradient Boosting (XGBoost) which to the best of our knowledge has received little attention spam email detection problems. Experimental results show that the proposed model outperforms earlier approaches across a wide range of evaluation metrics. A thorough analysis of the model results in comparison to the results of earlier works is also presented.
National Institute of Technology Silchar（锡尔恰尔国立技术学院）
Machine unlearning for security is studied in this context. Several spam email detection methods exist, each of which employs a different algorithm to detect undesired spam emails. But these models are vulnerable to attacks. Many attackers exploit the model by polluting the data, which are trained to the model in various ways. So to act deftly in such situations model needs to readily unlearn the polluted data without the need for retraining. Retraining is impractical in most cases as there is already a massive amount of data trained to the model in the past, which needs to be trained again ju…
Unsolicited email (spam) is still a problem for users of the email service. Even though current email anti-spam solutions filter most spam emails, some spam emails still are delivered to the inbox of users. A special class of spam emails advertises websites, e.g., online dating sites or online pharmacies. The success rate of this kind of advertisement is rather low, however, as sending an email does only involve minimal costs, even a very low success rate results in enough revenue such that this kind of advertisement pays off. The anti-spam approach presented in this paper aims on increasing t…
Spam messages muddle up users inbox, consume network resources, and build up DDoS attacks, spread malware. Our goal is to present a definite figure about the characteristics of spam and spam vulnerable email accounts. These evaluations help us to enhance the existing technology to combat spam effectively. We collected 400 thousand spam mails from a spam trap set up in a corporate mail server for a period of 14 months form January 2006 to February 2007. Spammers use common techniques to spam end users regardless of corporate server and public mail server. So we believe that our spam collection …
Traditional spam classification requires the end-user to reveal the content of its received email to the spam classifier which violates the privacy. Spam classification over encrypted emails enables the classifier to classify spam email without accessing the email, hence protects the privacy of email content. In this paper, we construct a spam classification framework that enables the classification of encrypted emails. Our classification model is based on a neural network with a quadratic network part and a multi-layer perception network part. The quadratic network architecture is compatible …
Canadian Institute for Cybersecurity（加拿大网络安全研究所）；University of New Brunswick（新不伦瑞克大学）
Anomalies in emails such as phishing and spam present major security risks such as the loss of privacy, money, and brand reputation to both individuals and organizations. Previous studies on email anomaly detection relied on a single type of anomaly and the analysis of the email body and subject content. A drawback of this approach is that it takes into account the written language of the email content. To overcome this deficit, this study conducted feature extraction and selection on email header datasets and leveraged both multi and one-class anomaly detection approaches. Experimental analys…
Deutsche-Telekom Laboratories（德国电信实验室）；Ben-Gurion-University of the Negev（内盖夫本-古里安大学）
The battle between email service providers and senders of mass unsolicited emails (Spam) continues to gain traction. Vast numbers of Spam emails are sent mainly from automatic botnets distributed over the world. One method for mitigating Spam in a computationally efficient manner is fast and accurate blacklisting of the senders. In this work we propose a new sender reputation mechanism that is based on an aggregated historical data-set which encodes the behavior of mail transfer agents over time. A historical data-set is created from labeled logs of received emails. We use machine learning alg…
Using multi group asymmetric public and private keys, this paper proposes a encryption email communication system, which makes email communication more secure, lowers the service providerś network and storage consumption, and completely eliminates any spam. Prior to this, PGP (Pretty Good Privacy) partially solved the encryption problem, but it only encrypts the content of the mail. We would also expect a mail system to provide safe and reliable services, completely eliminate the spam emails, reduce the consumption of service providerś traffic and storage resources, and specially focus on prot…
Email is a private medium of communication, and the inherent privacy constraints form a major obstacle in developing effective spam filtering methods which require access to a large amount of email data belonging to multiple users. To mitigate this problem, we envision a privacy preserving spam filtering system, where the server is able to train and evaluate a logistic regression based spam classifier on the combined email data of all users without being able to observe any emails using primitives such as homomorphic encryption and randomization. We analyze the protocols for correctness and se…
EvoMail: Self-Evolving Cognitive Agents for Adaptive Spam and Phishing Email Defense
People’s Liberation Army 77606 Unit（中国人民解放军77606单位）；Field Engineering College, Army Engineering University of PLA（陆军工程大学工程学院）；College Of Software, ZhengZhou University of Light Industry（郑州轻工业大学软件学院）
Evaluating and Combating the Impact of Concept Drift on the Performance of Machine Learning-Based Phishing Detection Systems
Department of Computer Science, School of Mathematics, Computer Science and Engineering, City St George’s, University of London, UK（伦敦大学城市学院计算机科学系，数学、计算机科学与工程学院，伦敦大学）
Unsolicited bulk email (aka. spam) is a major problem on the Internet. To counter spam, several techniques, ranging from spam filters to mail protocol extensions like hashcash, have been proposed. In this paper we investigate the effectiveness of several spam filtering techniques and technologies. Our analysis was performed by simulating email traffic under different conditions. We show that genetic algorithm based spam filters perform best at server level and naive Bayesian filters are the most appropriate for filtering at user level.
Proof-of-work (POW) schemes have been proposed in the past. One prominent system is HASHCASH (Back, 2002) which uses cryptographic puzzles . However, work by Laurie and Clayton (2004) has shown that for a uniform proof-of-work scheme on email to have an impact on SPAM, it would also be onerous enough to impact on senders of "legitimate" email. I suggest that a non-uniform proof-of-work scheme on email may be a solution to this problem, and describe a framework that has the potential to limit SPAM, without unduly penalising legitimate senders, and is constructed using only current SPAM filter t…
Electronic mail services have become an important source of communication for millions of people all over the world. Due to this tremendous growth, there has been a significant increase in spam traffic. Spam messes up user's inbox, consumes network resources and spread worms and viruses. In this paper we study the characteristics of spam and the technology used by spammers. In order to counter anti spam technology, spammers change their mode of operation, therefore continues evaluation of the characteristics of spam and spammers technology has become mandatory. These evaluations help us to enh…
PwC（普华永道）；University of New South Wales (UNSW)（新南威尔士大学（UNSW））
Email threat is a serious issue for enterprise security. The threat can be in various malicious forms, such as phishing, fraud, blackmail and malvertisement. The traditional anti-spam gateway often maintains a greylist to filter out unexpected emails based on suspicious vocabularies present in the email's subject and contents. However, this type of signature-based approach cannot effectively discover novel and unknown suspicious emails that utilize various evolving malicious payloads. To address the problem, in this paper, we present Holmes, an efficient and lightweight semantic based engine f…
In our modern society telephony has developed to an omnipresent service. People are available at anytime and anywhere. Furthermore the Internet has emerged to an important communication medium. These facts and the raising availability of broadband internet access has led to the fusion of these two services. Voice over IP or short VoIP is the keyword, that describes this combination. The advantages of VoIP in comparison to classic telephony are location independence, simplification of transport networks, ability to establish multimedia communications and the low costs. Nevertheless one can easi…
In this paper, we introduce DeepQuarantine (DQ), a cloud technology to detect and quarantine potential spam messages. Spam attacks are becoming more diverse and can potentially be harmful to email users. Despite the high quality and performance of spam filtering systems, detection of a spam campaign can take some time. Unfortunately, in this case some unwanted messages get delivered to users. To solve this problem, we created DQ, which detects potential spam and keeps it in a special Quarantine folder for a while. The time gained allows us to double-check the messages to improve the reliabilit…
Spam can be defined as unsolicited bulk email. In an effort to evade text-based filters, spammers sometimes embed spam text in an image, which is referred to as image spam. In this research, we consider the problem of image spam detection, based on image analysis. We apply convolutional neural networks (CNN) to this problem, we compare the results obtained using CNNs to other machine learning techniques, and we compare our results to previous related work. We consider both real-world image spam and challenging image spam-like datasets. Our results improve on previous work by employing CNNs bas…
An improved email classification method based on Artificial Immune System is proposed in this paper to develop an immune based system by using the immune learning, immune memory in solving complex problems in spam detection. An optimized technique for e-mail classification is accomplished by distinguishing the characteristics of spam and non-spam that is been acquired from trained data set. These extracted features of spam and non-spam are then combined to make a single detector, therefore reducing the false rate. (Non-spam that were wrongly classified as spam). Effectiveness of our technique …
The security evaluation for Mail Distribution Systems focuses on certification and reliability of sensitive data between mail servers. The need to certify the information conveyed is a result of known weaknesses in the simple mail transfer protocol (SMTP). The most important consequence of these weaknesses is the possibility to mislead the recipient, which is achieved via spam (especially email spoofing). Email spoofing refers to alterations in the headers and/or the content of the message. Therefore, the authenticity of the message is compromised. Unfortunately, the broken link between certif…
With the generalization of mobile communication systems, solicitations of all kinds in the form of messages and emails are received by users with increasing proportion of malicious ones. They are customized to pass anti-spam filters and ask the person to click or to open the joined dangerous attachment. Current filters are very inefficient against spear phishing emails. It is proposed to improve the existing filters by taking advantage of user own analysis and feedback to detect all kinds of phishing emails. The method rests upon an interface displaying different warnings to receivers. Analysi…
We propose a new detection algorithm that uses structural relationships between senders and recipients of email as the basis for the identification of spam messages. Users and receivers are represented as vectors in their reciprocal spaces. A measure of similarity between vectors is constructed and used to group users into clusters. Knowledge of their classification as past senders/receivers of spam or legitimate mail, comming from an auxiliary detection algorithm, is then used to label these clusters probabilistically. This knowledge comes from an auxiliary algorithm. The measure of similarit…
Spam messes up user's inbox, consumes network resources and spread worms and viruses. Spam is flooding of unsolicited, unwanted e mail. Spam in blogs is called blog spam or comment spam.It is done by posting comments or flooding spams to the services such as blogs, forums,news,email archives and guestbooks. Blog spams generally appears on guestbooks or comment pages where spammers fill a comment box with spam words. In addition to wasting user's time with unwanted comments, spam also consumes a lot of bandwidth. In this paper, we propose a software tool to prevent such blog spams by using Baye…
Image spam threat detection has continually been a popular area of research with the internet's phenomenal expansion. This research presents an explainable framework for detecting spam images using Convolutional Neural Network(CNN) algorithms and Explainable Artificial Intelligence (XAI) algorithms. In this work, we use CNN model to classify image spam respectively whereas the post-hoc XAI methods including Local Interpretable Model Agnostic Explanation (LIME) and Shapley Additive Explanations (SHAP) were deployed to provide explanations for the decisions that the black-box CNN models made abo…
Once an email problem, spam has nowadays branched into new territories with disruptive effects. In particular, spam has established itself over the recent years as a ubiquitous, annoying, and sometimes threatening aspect of online social networks. Due to its prevalent existence, many works have tackled spam on Twitter from different angles. Spam is, however, a moving target. The new generation of spammers on Twitter has evolved into online creatures that are not easily recognizable by old detection systems. With the strong tangled spamming community, automatic tweeting scripts, and the ability…
In this research, we aim to explore the potential of natural language models (NLMs) such as GPT-3 and GPT-2 to generate effective phishing emails. Phishing emails are fraudulent messages that aim to trick individuals into revealing sensitive information or taking actions that benefit the attackers. We propose a framework for evaluating the performance of NLMs in generating these types of emails based on various criteria, including the quality of the generated text, the ability to bypass spam filters, and the success rate of tricking individuals. Our evaluations show that NLMs are capable of ge…
Spam domains are sources of unsolicited mails and one of the primary vehicles for fraud and malicious activities such as phishing campaigns or malware distribution. Spam domain detection is a race: as soon as the spam mails are sent, taking down the domain or blacklisting it is of relative use, as spammers have to register a new domain for their next campaign. To prevent malicious actors from sending mails, we need to detect them as fast as possible and, ideally, even before the campaign is launched. In this paper, using near-real-time passive DNS data from Farsight Security, we monitor the DN…
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Machine learning models have been widely used in security applications such as intrusion detection, spam filtering, and virus or malware detection. However, it is well-known that adversaries are always trying to adapt their attacks to evade detection. For example, an email spammer may guess what features spam detection models use and modify or remove those features to avoid detection. There has been some work on making machine learning models more robust to such attacks. However, one simple but promising approach called {\em randomization} is underexplored. This paper proposes a novel randomiz…
One of the biggest problems with the Internet technology is the unwanted spam emails. The well disguised phishing email comes in as part of the spam and makes its entry into the inbox quite frequently nowadays. While phishing is normally considered a consumer issue, the fraudulent tactics the phishers use are now intimidating the corporate sector as well. In this paper, we analyze the various aspects of phishing attacks and draw on some possible defenses as countermeasures. We initially address the different forms of phishing attacks in theory, and then look at some examples of attacks in prac…
Existence of spam URLs over emails and Online Social Media (OSM) has become a massive e-crime. To counter the dissemination of long complex URLs in emails and character limit imposed on various OSM (like Twitter), the concept of URL shortening has gained a lot of traction. URL shorteners take as input a long URL and output a short URL with the same landing page (as in the long URL) in return. With their immense popularity over time, URL shorteners have become a prime target for the attackers giving them an advantage to conceal malicious content. Bitly, a leading service among all shortening se…
In the era of large-scale internet scanning, misconfigured websites are a frequent cause of data leaks and security incidents. Previous research has investigated sending automated email notifications to operators of insecure or compromised websites, but has often met with limited success due to challenges in address data quality, spam filtering, and operator distrust and disinterest. While several studies have investigated the design and phrasing of notification emails in a bid to increase their effectiveness, the use of other contact channels has remained almost completely unexplored due to t…
The malaise of electronic spam mail that solicit illicit partnership using bogus business proposals (popularly called 419 mails) remained unabated on the internet despite concerted efforts. In addition to these are the emergence and prevalence of phishing scams that use social engineering tactics to obtain online access codes such as credit card number, ATM pin numbers, bank account details, social security number and other personal information (22). In an age where dependence on electronic transaction is on the increase, the web security community will have to devise more pragmatic measures t…
Spam messes up users inbox, consumes resources and spread attacks like DDoS, MiM, Phishing etc., Phishing is a byproduct of email and causes financial loss to users and loss of reputation to financial institutions. In this paper we study the characteristics of phishing and technology used by phishers. In order to counter anti phishing technology, phishers change their mode of operation; therefore continuous evaluation of phishing helps us to combat phishers effectively. We have collected seven hundred thousand spam from a corporate server for a period of 13 months from February 2008 to Februar…
Spam messes up users inbox, consumes resources and spread attacks like DDoS, MiM, phishing etc. Phishing is a byproduct of email and causes financial loss to users and loss of reputation to financial institutions. In this paper we examine the characteristics of phishing and technology used by Phishers. In order to counter anti-phishing technology, phishers change their mode of operation; therefore a continuous evaluation of phishing only helps us combat phisher effectiveness. In our study, we collected seven hundred thousand spam from a corporate server for a period of 13 months from February …
