Threat modeling and risk management

I have a new, longish post on the topic, Shostack + Associates > Shostack + Friends Blog > Risk Management and Threat Modeling. Eager for your feedback.

2 Likes

Really great points, and I like the bridge building analogy. I actively try to avoid as much “risk assessment” as possible when it comes to threat modelling, as it’s a rabbit hole that is too easy to go down and often doesn’t actual provide much value (in my experience).

I tend to focus more on has this thing (that is being built and threat modelled) been “security engineered” well (much like a safety engineer would review a bridge). If there are basic security engineering things missing then they need to be fixed. I tend to think about it more like the thing should achieve a certain bar of security - call it a standard, requirements, whatever you want. Often the “bar” is not explicitly defined, but that can be defined in time, and security best practices are available for most things anyway.

Of course technical and business constraints exist that often mean the system can’t follow a best practice, and this is where you would like a risk assessment to help, but it rarely does (in my experience). Better to focus on applying good engineering to solve the problem as best as possible, and if needs must, capture the threats in sufficient detail to explain them to someone who owns the risk so they can make a judgement call (and no, I don’t think doing that ‘capturing’ in most risk assessment frameworks helps).

3 Likes

I like how you express it in such simple terms that anyone can understand. It takes me back to the time I was reading “ Waltzing with Bears: Managing Risk on Software Projects” by T. DeMarco and T. Lister.
The message in your post echoes a very similar point made in the book, emphasizing the importance of proactive risk management in software development, a threat model isn’t explicitly named, but advocating for a proactive approach to risk management means that you effectively are doing threat modeling, in my view.

thanks! I think we need to call attention to the fact that a great many of us are noticing that risk assessment ‘doesn’t actual provide much value’ and stop wasting energy on it.

Thanks, @adamshostack , for posting this. I didn’t understand some of your previous talk about “not mixing TM and risk management”. That one clarifies your point a lot. :+1:

If what you fix is not guided by risk assessment

Won’t that create All-critical bias?

Sure, we could fix 1000 “easily addressed” things, but that will also be expensive in sum.

:de: Kleinvieh macht auch Mist.
:gb: Even small animals produce manure.

What are your thoughts?

Also loved (and agree) how @steve_gibbons commented at LinkedIn:

I’d argue that the “fix it” activity is often “evaluate priority and add to the backlog queue.” (Yes that’s easy to get wrong.)

So doesn’t priority bring risk assessment back into the game anyway?

1 Like

I don’t think the risk assessments are the issue. I use risk assessments in the same way as I use threat modeling, I focus on what can go wrong and address the potential risk and what we can do about them and we do it as the team is starting the work, not after, and the whole team is always there. That means that we effectively are doing the threat modeling and risk assessment together. In this way, our risk assessments process is very much the same as our threat modeling process. The big difference is how the document looks. The rest is effectively a threat modeling session with risk management jargons. We also do pure threat modeling as well. It all depends on whether we need a drawing or not.
What I do see is that I have to drive the risk assessments much more then I have to drive the threat modeling session. Risk management wouldn’t happen if I wasn’t there. The software team asks me to do the session together with them though. The risk management happen when the team comes back to me and tells me that there is something they can’t do because of some technical difficulties. That’s when we get to have the risk management discussion.

2 Likes

Glad that this helps clarify. I believe we have the problem you point out today, with risk management being applied, and so first (respectfully) assert that your comparison is unfair.

Second, I think risk management makes it worse because its effort that doesn’t meet the needs of those doing the prioritization, and maybe we should free up that energy to be invested in more helpful ways.

I explicitly create a model in which we are not “effectively are doing the threat modeling and risk assessment together.” Like any model, we can judge its usefulness. I’m fine with you not seeing it as useful, but I do ask that you engage with “what if we treat risk quantification as separate.” If that’s useless to you, I apologize for wasting your time. I think its useful which is why I shared it.

1 Like

I do see the usefulness. It’s just that I have never really spent much time on risk quantification or arguing with others over whether threats or risks needs to be mitigated or accepted, but that is probably due to my fortunate circumstances or lack of experience. I have no problem in seeing that it is quite useful under other circumstances and will keep it in the back of my mind.

I think that is an unnecessarily strong statement and, respectfully, an oversimplification that only leads to organizational silo-building.

I do agree with the article on the point that linking technical threats to business risks can be hard, but I also think this should not hold us back from trying—and getting better at it with practice.

If you manage to establish the connection, you benefit from showing how your threat modelling process contributes to reaching business priorities and potentially gain far more management buy-in. Depending on the organizational culture you operate in, this can be the decisive factor in the success of your threat modeling efforts.

Another point, if I understood correctly from the article, was that many of the identified threats do not require much further thought in terms of risk assessment—as they are either trivial to implement or so grave in their impact or likelihood that they demand immediate attention anyway.

Fair point. But still, wouldn’t the threat modelling process benefit if we had the means to show how often it helps identify high-impact, low-hanging fruits with low-cost implementations? If anything, that would be a great argument to support the process, so I see it as an opportunity lost not to communicate these results in a structured way.

The Cyber Resilience Act is becoming more and more prominent in discussions around the importance of threat modelling. CRA talks about risk management, and we claim that threat modelling is the natural way to do it—or rather, we don’t see an alternative. I think the argument has its merits, and I actually support it. However, CRA still refers to risk management as a required activity, so in order to argue that threat modelling is our means to do it, we should do our homework and establish a clear connection between the two processes. One way to do this is to do the hard work of linking (at least a subset of) technical threats to business risks, properly assess them, and feed them into the risk management process. Also, I don’t currently see any other obvious way to actually demonstrate to a non-SME auditor that threat modelling fulfills the CRA requirements.

I agree with @hewerlin that prioritizing something in your backlog is also a form of risk rating—just informal and non-transparent. It still takes effort, however, so I’m somewhat unconvinced that a lot of time and energy can be saved by not documenting the thinking behind the priorities. I believe the real savings would actually come from becoming better (as in more efficient) at doing the assesment with quantititive methods and navigating away from qualitative assesments (low-medium-high) that indeed, have marginal utility.

Thanks Daniel. I do think the silos are there, between ‘engineering’ and ‘risk’ and more, that’s appropriate, because if you let people set risk numbers, there’s a chance they’ll manipulate them, and so many orgs want it silo’d.

To your metrics point, we agree, and I don’t see how a risk/threat breakdown changes that. Maybe the addition of the words “high impact”, but i think you can get there with “OMG this would have stank” without quantification.

Lastly, you make a great point about CRA. “we should do our homework and establish a clear connection between the two processes.” My thinking, and I’m speaking with folks about this, is to push threat modeling within the standards. That’s complicated because you need people who want to spend time in standards bodies, :grin: but I don’t see a way around that.

I have observed some of the following consequences, simplified. I have written about them in more detail in the TM of TM - ID numbers from that model:

No risk assessment 3.1.1 → All critical bias 3.1.7

All critical bias → Unfeasible / high effort mitigations 3.2.1

Unfeasible high effort mitigations → Undone mitigations 3.3.3

Undone mitigations → Insecurity → Actual damage

All critical bias → (Insane and) untrusted rating scheme 3.1.8

I believe we need something sane that tells us when not to waste our energy (because unlikely / not so bad / protection suffices / 
).

How do you consider this an unfair comparison?

Off topic, but still, I don’t think the motive behind neglecting prioritizing security is inaccurate or lacking risk quantification. In my experience, the main challenge is a lacking understanding for the value of threat modeling and risk management. You can’t fix that with better or more information because the inherent issue is the perception that it takes a lot of resources, time and doesn’t add value. Using more time and resources won’t help you. I would instead go the other way. Why? The threats and mitigation are usually well understood and easily applied. The challenge is convincing the organization that it should be done on a daily basis and that the inconvenience is worth it. You do that by making it trivial to do it, not by bringing more complexity to the table.

2 Likes

“Sure, we could fix 1000 “easily addressed” things, but that will also be expensive in sum.” < We have this problem today, adding expensive risk management work doesn’t get those easily addressed things fixed.

2 Likes

Ah. How about

Less busywork + cost of effective risk assessment that doesn’t suffer from all critical
<
A lot of work
<
A lot of work + cost of ineffective risk assessment

? :smiley:

1 Like

I don’t know I even believe this: “Less busywork + cost of effective risk assessment that doesn’t suffer from all critical”

Maybe it would help to know: Which threats are you going to accept as “not important?” (And would you put the list on your website? :slight_smile: )

My experience has been that risk quantification is usually something that comes up when someone don’t want to do the extra work and usually after the thing has been built. I try to avoid it altogether. It’s better for me to use critical, high, medium and low because that means I can push the team to increase the priority without having to go to much into the details as to why the priority needs to increase. This is why, where I currently work, it doesn’t matter that we mix risk management and threat modeling. Nobody asks me about how critical it is before it’s so critical that they need to do a hot fix.

Like you hint at Publish your TM. :trade_mark: :wink:

It’s usually one of the three causes: unlikely (strong pre conditions) / low impact (weak post conditions) / already defended.

When I teach “what would ruin a day at the beach” toy example, I sometimes get :shark: shark attack or :ocean: Tsunami. Yes, that would really suck. But we will most likely have a great day at the beach without Tsunami when we just accept. Same with technical risks, some of which are ridiculous.

Threats with strong preconditions are another case. Let’s give an example: Apps usually have high impact threats after account takeover or server takeover. We may mitigate for additional harm reduction. But those takeovers’ protect / detect / respond should have been considered anyway and hopefully been made unlikely. So account / server takeover defense may already be sufficient defense for those follow-up threats.

Are threats with strong pre-conditions any different from threats with low likelihood?