One of the issues that I run into constantly with threat modeling is the noise level.
That is to say, many false positives.
One way to look at this, I suppose, is that these false positives are an indication that you’ve cast a wide enough net.
But this leads to hours tracking down the justification to write off these NA threats.
(These false positives may also tend to overwhelm developers who are new to threat modeling - it is difficult to convince them that threat modeling works when they have to spend so much time weeding out the junk.)
I tend to think that the answer to this is “better data”. However, given that threat modeling is ideally done as early in the SDLC as possible, when quality/rich/complete design data may not yet be available, how can one mitigate these false positives, and tune threat modeling to achieve higher quality, less noisy results?
If you are willing to share more on your process or tooling then you might get more specific advice on dealing with this issue.
One way to view “hours tracking down the justification to write off these NA threats” is an investment of time in identifying the common controls across your systems, and I’ve always found that to be useful information. Of course it’s only useful if it’s made available to your teams.
This speaks to what has worked for me in the past regarding noise (or any laborious part of the process), which is to define your own process on top of whatever tooling you are using, and document the conventions that your teams can adopt in order to make threat modelling an easier activity. For instance if you are using STRIDE and having to give the same answer for every repudiation threat, then you can establish a convention that the default control of logging always applies, unless stated otherwise (you could define some criteria). Documenting default responses and conventions for your teams for the most common noise producing threats can enable them to deal with those efficiently, or preferably not at all.
Longer term it may be worth exploring other threat modelling approaches that have minimal noise levels, but undoubtedly there’ll be other trade-offs to make, although they might be ones more suited to your situation.
resonating on the request from Dave to you, to share more details about where this noise comes from.
As a general guidance, I can recomend to apply Threat Modeling with the following mantra in mind:
“If there was an reliable automated way to identify an issue, then it should not be part of TM”.
Thereby, your Threat Modeling approach may vary depending on whether you are dealing with a Bespoken or COTS/Binary Freeware application. Bespoken applications allow you to proof “low-level” security concern by the means of scan tools (code or configurations). The Threat Modeling expert may therefore guide on general concepts (e.g. appliance of appropriate input sanitization libs) but otherwise recommend the appliance of security tools which often times also come with the according guidance.
In contrast to that, COTS would not provide you this option in most of the cases. Thereby, much more low-level aspects may need to be stressed. A way to still ensure a reduction of noise for these, may be to evaluate first with the 3rd party vendor on either contractual/certification assertation like ISO-27034-1, or evaluate possibilities to gain insight into its development process and scan tool results.
Bill, as I work with my customers this question often comes up. How do I limit the “Noise”? I personally do not like the term “Noise” because all identified Threats have some level of value. I think instead of how do I filter and prioritize my efforts. As you identified, to mark a Threat as NA requires as much effort sometimes as it would be to implement a correlating control.
There are solutions available that will help automate the process (hint, hint) and will also allow you to filter and prioritize Threats and Countermeasures (Controls). These solutions will automatically identify your highest level threats and or Countermeasures that are required if you have a compliance standard that you must adhere to.
Leveraging such a solution will most certainly help you with the “Noise”
If it is a False Positive or if it is a Noise then we need to re-iterate the process of making it more reasonable but also real. Sometimes we get overwhelmed with the threats and just jot down a lot of them without giving a direction towards the context wrt the entity. If you think that the false positives or noise is more in the threatmodels you are creating, you need to attach the context and narrow down to the issues which are more relevant rather than just listing it. The more you move away from theoretical threats to more concise and context aware threats the more mature your model will turn out to be but remember it is a repetitive as well iterative process so dont loose hope. All the best
All the best