It goes without saying that the Internet is far from secure. As the number of Internet-connected devices increases, so do the number of cyberattacks we have to deal with. Numerous industry reports reveal significant upswings in software vulnerabilities year after year. These are issues plaguing enterprises of all sizes, within the public and private sector. In light of these findings, it becomes imperative for businesses, regardless of size, to prioritize cybersecurity and re-evaluate their current defense mechanisms against this evolving threat landscape. The evolving cyber threat landscape has emphasized the importance of adopting proactive approaches to manage and mitigate cybersecurity risks. Organizations can take a great many steps to achieve this, but mainly choose security measures that revolve around compliance requirements and standardized methodologies and frameworks to improve overall security posture. However, it remains unclear to which extent such investments have their desired effect. This is mainly because security is a latent property that cannot be measured directly. Alternative approaches have recently emerged that aim to measure security in a more direct manner. Instead of relying on self-reported data, internal or otherwise, firms gather externally accessible data and subsequently train a classifier using this data, enabling it to predict, with a certain level of accuracy, which organizations are likely to experience (large-scale) breaches. Still, it is not clear how metrics derived from purely external measurements compare to the security level derived from internal measurements of an organization's network. This reveals the necessity of taking into account the internal state of networks when observing external security signals, instead of exclusively relying on externally observable or publicly reported data breaches. This dissertation studies the feasibility of security incident prediction and risk estimation. It examines how external network scan data can be leveraged to infer information about the internal state of security of an organization's network. Thus, we aim to answer the following research question: How can internal security incidents be predicted through the leverage of external network signals?@en

Signature-based network intrusion detection systems (NIDSs) and network intrusion prevention systems (NIPSs) remain at the heart of network defense, along with the rules that enable them to detect threats. These rules allow Security Operation Centers (SOCs) to properly defend a network, yet we know almost nothing about how rules are created, evaluated and managed from an organizational standpoint. In this work, we analyze the processes surrounding the creation, management, and acquisition of rules for network intrusion detection. To understand these processes, we conducted interviews with 17 professionals who work at Managed Security Service Providers (MSSPs) or other organizations that provide network monitoring as a service or conduct their own network monitoring internally. We discovered numerous critical factors, such as rule specificity and total number of alerts and false positives, that guide SOCs in their rule management processes. These lower-level aspects of network monitoring processes have generally been regarded as immutable by prior work, which has mainly focused on designing systems that handle the resulting alert flows by dynamically reducing the number of noisy alerts SOC analysts need to sift through. Instead, we present several recommendations that address these lower-level aspects to help improve alert quality and allow SOCs to better optimize workflows and use of available resources. These recommendations include increasing the specificity of rules, explicitly defining feedback loops from detection to rule development, and setting up organizational processes to improve the transfer of tacit knowledge.@en

Notwithstanding the predicted demise of signature-based network monitoring, it is still part of the bedrock of security operations. Rulesets are fundamental to the efficacy of Network Intrusion Detection Systems (NIDS). Yet, they have rarely been studied in production environments. We partner with a Managed Security Service Provider (MSSP) to gain more insight into the evolution of rulesets, the alerts that they trigger and the incidents that get investigated. We analyze a combined ruleset - including both commercial and proprietary rules - that consists of 130 thousand rules and was used to monitor hundreds of networks. We find that these rulesets keep growing over time but there is almost no overlap among them in terms of detection options or what indicators of compromise they contain. The combined ruleset triggered more than 62 million alerts and led to 150 thousand incident investigations by SOC analysts, though the vast majority of rules never triggered a single alert. We find that just 0.5% of all rules are responsible for more than 80% of the alerts and incidents and only 1.2% of all alerts were deemed to merit closer investigation. Of all incidents, 16% were labeled as false positives and 9% carried significant risk to the client organization. Independently of the type of rule, updating rules is a minor activity. Most rules are never modified and only a fraction is deleted, except for periodic purges in some sets. Seven in-depth interviews with rule developers corroborate the patterns we found in our analysis. Finally, we identify several rule management practices that influence rule and ruleset efficacy, such as supplementing commercial rules with your own and making rules as specific as possible.@en

Asset discovery is fundamental to any organization's cybersecurity efforts. Indeed, one must accurately know which assets belong to an IT infrastructure before the infrastructure can be secured. While practitioners typically rely on a relatively small set of well-known techniques, the academic literature on the subject is voluminous. In particular, the Internet measurement research community has devised a number of asset discovery techniques to support many measurement studies over the past five years. In this paper, we systematize asset discovery techniques by constructing a framework that comprehensively captures how network identifiers and services are found. We extract asset discovery techniques from recent academic literature in security and networking and place them into the systematized framework. We then demonstrate how to apply the framework to several case studies of asset discovery workflows, which could aid research reproducibility. These case studies further suggest opportunities for researchers and practitioners to uncover and identify more assets than might be possible with traditional techniques. @en