nprssfetimg-57.png

NIST and No-notice: Finding the Goldilocks zone for phishing simulation difficulty

Earlier this year, the National Institute for Standards and Technology (NIST) published updated recommendations for phishing simulations in security awareness training programs. We discussed it on our Community page soon after the updated standards were released, but the substance of the change bears repeating.

Practical exercises include no-notice social engineering attempts to collect information, gain unauthorized access, or simulate the adverse impact of opening malicious email attachments or invoking, via spear-phishing attacks, malicious web links.” – NIST SP 800-53, Rev. 5, Section 5.3 (pg. 60)

This update includes a recommendation forno-noticephishing simulations to be delivered at the beginning of security awareness training programs to more accurately gauge the readiness of a set of users to recognize a phishing attempt.

The thinking obviously being that letting users in on the phishing simulation game will heighten suspicion of their inbox and skew baseline results. This concern can be thought as a spin-off of the well-studiedObserver Effectknown in many scientific fields; observing the behavior of something necessarily changes that behavior.

While it might be tempting for a Chief Information Security Officer (CISO) or other IT professional to take high grades on a phishing simulation a sign of a job well done, that can be a dangerous conclusion to draw. Phishing tests that are too easy do little to address a problem thats become one of the most common methods of entry for ransomware attacks.1 If IT professionals grade on a curve here, theyre doing very little to improve their organizations overall cyber resilience.

Combatting this false sense of confidence about usersability to spot phishing attacks requires making sure simulations arent too easy to spot.

What makes a phishing simulation too easy?

After putting some thought into that question, NIST researchers published a paper last year in the Journal of Cybersecurity citing three key criteria for determining if a phishing simulation makes for good training.

According to the authors, “low click rates do not necessarily indicate training effectiveness and may instead mean the phishing emailswere:

  1. Too obviousEither errors were too overt or these templates were running something akin to the Nigerian Prince scam. Either way, they wont help an employee overcome todays more sophisticated phishing attempts
  2. Not relevant to staffWere all busy at work. So deleting an email offering 25% off at Eds Golf Cart Repair Shop doesnt mean a user is an expert at spotting scams. It just means there was nothing in the simulation that enticed anyone to click.
  3. The phish was repeated or similar to one that was Phish me once, shame on mebut seriously, this drives home the importance of having a wide range of phishing templates. These programs work best when theyre ongoing, so its important to switch it up.

On the other hand, a phishing simulation is convincing if it does the following to some degree:

  • Mimics a workplace process or practice
  • Has workplace relevance
  • Aligns with other situations or events, including those external to the workplace
  • Presents consequences for NOT clicking (e.g., buy gift cards or we lose the client)
  • References targeted training, specific warnings or other exposure

Tip: NIST has devised a weighted version of this scale, “the phish scale,” you can use to determine the difficulty of your simulations. A phishing simulation that has all of the above characteristics would be considered extremely difficult. Thats good, right?

Too much difficulty can be dangerous, too

Any security awareness training program thats too difficult is liable to leave learners feeling put off, resigned to failure, or worse, coming away without any practical security learnings. This is especially true if users are punished too harshly for failing to spot a difficult phishing simulation.

Any program thats both difficult and relying on a stick rather than a carrot for motivation runs the risk of:

  • Reinforcing negative stereotypes of security training programs
  • Encouraging employees togamethe system by sharing information about tests
  • Fostering animosity towards the organizations overall security posture
  • Inviting legal trouble from dissatisfied employees

For security awareness training to be successful, it has to be collaborative. Learners should feel like theyre part of something constructive, rather than just subjected to another type of performance review.

Hitting the sweet spot

Finding the appropriate difficulty level for phishing simulations is one of the reasons the initial, no-notice NIST recommendation is so important. It helps administrators establish baseline results that most accurately reflect usersreal understanding of phishing attacks. But we dont recommend a training program be hidden from employees forever.

Instead, after initial results have been established, its better to announce the program publicly along with its goals, evaluation criteria and a point of contact for those interested in learning more. Once users are in the know, subsequent phishing simulations can focus on incremental improvements over the baseline results. As scores rise across the board, the difficulty can be gradually increased over time.

One essential recommendation: Always report publicly on positive results. Let users know theyre managing to catch more and more difficult simulations. Be as specific as possible, as in, “click-through rates dropped from A to B in this exercise.” This will help establish a sense of shared responsibility for organizational security andgamifythe experience.

Calibrating your security awareness training is an ongoing experience. Dont be afraid to adjust your simulations based on results. Happy learning.

Ready to establish your own successful security awareness training? Try us out free for 30 days

1. Hiscox. “Cyber Readiness Report 2021.” (April 2021)

Kyle Fiehler

About the Author

Kyle Fiehler

Copywriter

Kyle Fiehler is a writer and brand journalist for Webroot. For over 5 years hes written and published custom content for the tech, industrial, and service sectors. He now focuses on articulating the Webroot brand story through collaboration with customers, partners, and internal subject matter experts..

Tags: , , ,
Previous Post
nprssfetimg-58.png
Solutions Cyber Security Cybercrime How-to Resources Social Media Work From Home

Survey: How well do IT pros know AI and machine learning?

Next Post
nprssfetimg-56.png
Solutions Cyber Security Cybercrime Resources Social Media Tips Work From Home

3 reasons even Chromebook™ devices benefit from added security

Cyber Security, Stress, Money News 2021