Intrusion Detection Artificial Intelligence
Code: DAP/24-05
Active
Funding: Defence Funded Research
Start: January 2024
End: December 2027
Duration: 49 months
This project is the followup of DAP/20-03.
Contrary to currently available intrusion detection system (IDS) solutions, the MASFAD framework focuses on the use of domain knowledge of the structure of attacks and their behavior, with regards to emergent threats. The system implements analysis algorithms, which analyze the data collected from various sources in the network and look for specific Advanced Persistent Threat (APT) characteristics, producing evidence which is aggregated together, producing a “suspiciousness” score, to be reviewed by a domain expert. The analyst in turn can use their domain knowledge as well as context information, injected by the platform into the data, to decide what can be considered a threat and what is not. The analysis algorithms are encapsulated in different detection modules, or "agents", each of which is designed to automate the detection of specific APT characteristic, related to abnormal behavior. The MASFAD architecture allows for the integration of new agents in a plug-and-play fashion. All agents act as a black box, ingesting raw data and producing relevant evidence through appropriate analysis. Since each agent depends on a list of parameters to define how the analysis will be handled, fine-tuning them is relatively easy. Indeed, this offers powerful capabilities to deploy the same type of agent, with focus on different indicators of abnormal activity as prescribed by their parameter values. So far we have incorporated machine-learning algorithms for regulating the aggregation of the evidences by observing the results produced and adapting the parameters of the aggregation accordingly.
Currently, the MASFAD framework has shown during testing high true detection rate of malicious threats. Our goal is to continue the development of the system by focusing on three major aspects:
Intrusion Detection Visual Analytics APT Detection
The constant stream of data produced daily, the complicated environment and the need for quick reaction to malicious attacks make the life of cyber defense analyst a living nightmare. Many wonder how are we supposed to be able to review the gigabytes of logs produced daily, how can we manage to analyze them all and extract valuable insight into what is happening in the network?
ReadTools Offensive Security APT Detection
Defining cyber attacks is a difficult task. They vary in origins, goals and, at first glance, the techniques used might seem very different. Luckily a popular model was defined by Lockheed Martin, still used to this day, which illustrates very well the lifecycle of a typical cyber attack. The Cyber Kill Chain, popular but controversial, defines the 7 principal steps of an attack. There have been many advances, since its original conception, one of which is the wildly acclaimed ATT&CK Matrix for Enterprise.
ReadDetecting suspicious or malicious activity in a network is not a trivial task. In recent years the attacks perpetrated have grown in sophistication and frequency. For this reason a new detection tool was developed, in the form of the Multi Agent Ranking framework (MARk). MARk sets the groundwork for the implementation of large scale detection and ranking systems through the implementation of a distributed storage in conjuncture with highly specialized, stand-alone detector agents. The detector agents are responsible for analyzing specific predefined characteristics and producing a report of any suspicious activity encountered.
ReadRecently I have encountered an error I wasn't too familiar with how to resolve, working with the ELK Stack. This specific error is the "[circuit_breaking_exception] [parent] Data too large, data for [<http_request>]". It is not directly visible where the error originates from, but with some sleuthing I discovered that it is caused by Elasticsearch preventing some requests from executing to avoid possible out of memory errors, as detailed in Elasticsearch Circuit Breaker documentation.
Read