The
reality is that pen-testing provides no guarantees of security, and does
not address the weaknesses in an organisation's ability to detect and
respond to a sophisticated attack; or its ability to manage a cyber
crisis and take the timely decisions to enact cyber defence or system
continuity plans.
Consequently,
this is driving the need for more sophisticated and technically-based
crisis exercises to identify causes of failure.
To most firms, a real-world attack
simulation is as much a 'game changer' as actually being targeted. In
both cases, firms can expect to learn hard lessons but the war game
process ensures that the organisation is ready to absorb the lessons,
and identify the benefits without the pain or damage of an actual
breach. This point cannot be underestimated. In a real event there is
invariably a catalogue of human and management failures consistent with
the inability to think clearly under pressure.
In reality, most lessons are only learnt
after a real event, even when the overriding climate is negative or less
orientated towards learning. A war-game, which simulates a prolonged
attack, aims to provide lessons before a real event, and enables
learning during an attack. In short, it can develop a firm's ability to
interpret and apply experience into real-time learning.
Cyber war games derive significant
learning across multiple levels of decision-makers, and can be
structured specifically to bring together the CISO, security leadership
team, security operations centre, incident response, as well as the
forensics, risk, and crisis management teams.
War gaming is an excellent and effective
way for large organisations to identify the weaknesses in communications
and coordination between these groups. In times of crisis, the
cascading effects of an attack and the impacts are often exacerbated by
the decisions taken, and the process of decision-making by these groups.
Learning how these groups take certain decisions when faced with
uncertainties, or adapt and enact response plans when tackling
‘unknowns' is vital to successful response.
Cyber war games are new and are slowly
being adopted because there are currently few bodies providing the scope
of capabilities required to conduct such an exercise. Elements to be
considered include the set-up of both internal and external
directorates, the preparation of the 'red team', and any required custom
tools are necessary to move away from a one-dimensional desktop
approach. To prepare an organisation's ‘blue team' with the appropriate
preparation may even require a pre-exercise review of all of the
following, depending on the objectives set: policies and procedures –
the gap is measured against best practices, employed methodologies,
deployed technologies, and past lessons learned.
A well-crafted war game incorporates both a
‘fundamental surprise' that the organisation had not anticipated and a
number of ‘situational surprises' which were known cyber risks for which
the organisation has little or no advanced warning.
Much of the pre-exercise planning should
aim at developing appropriate knowledge and intelligence in order to
define the exercise in a manner that can be controlled and developed
over time, and tests the different capabilities.
The ‘storyline' can commence with a
technical event to kick off the assessment of initial implications, and
the event would then be developed through situational feeds from the
directorate.
The initial objectives should be to test
detection: by the systems; by the incident response team; and the
analysis of the forensic team. More can then be provided by the
directorate including intelligence, such as analysis of the threat
community, IP information, and pieces of a malware. The exercise can
then examine the fundamentals of communication and decision-making,
specifically who is taking decisions and on what basis; and what is the
process of taking alerts/indications and deriving useful information
from then: and then transforming that information into knowledge
throughout this first technical phase.
At this point, a major new technical event
may be introduced, or the original event may be taken in a new
direction to trigger a new cycle of detection and decision-making.
Evaluation may focus more on how the new event affects the decisions
previously taken, the need for additional resources, and whether a new
risk assessment should take place. With a second phase escalation of the
attack, the evaluation can examine who is assessing the risk throughout
the event, who is involved in the process, what indicators are in
place, and how they conduct a timely assessment of the possible
implications from the new event.
Using this approach will allow escalation
towards the involvement of the crisis management team, and an
examination of their team, what stage they were involved and how they
receive the relevant information. The exercise can also test the team's
communication effectiveness, who precisely was evolved and how they
supported the whole process.
The more significant element in the
learning process is the incorporation of observation, decision-logging,
and mentoring as part of the war game process, while a full debrief and
post exercise workshop should establish lessons learnt, capability gaps
and the modifications required in technology and processes.
It is advised that a full day is then
allocated to analyse all events, and outcomes of the exercise, reviewing
performance of the different groups, and the effectiveness of deployed
the technology. The teams involved should be encouraged to appraise the
effectiveness of work process, and develop lessons to be learned with
the observers and mentors.
The 'learning by doing' opportunity that
war games provide identifies failures in breach incident response as
well as failures in security.
This should ensure a balance between
security and implementing the appropriate response, but also offer a
list of immediate tactical priorities for remediation, as well as short
term changes. It can also pick up previously peripheral issues that had
not been addressed or prioritised specifically because they may have
been proven to be more critical to the overall security apparatus than
previously recognised. Often these are 'human' aspects known to be
weaknesses, though not recognised and addressed at an organisational
level.
By establishing the right war game
framework, particularly because a technical attack process in central,
the learning objectives are set at the top of the agenda if the
organisation is astute enough to accept that a breach will occur, and
the success is measured by how it deals with this.
To support this shift in perspective,
end-of-exercise workshops can be used to help participants understand
what was previously lacking, and provide the opportunity to build
consensus around priorities from board level down through the risk,
business continuity, and security teams.
The iterative process of this type of
workshop can offer a forum for planning that integrates investment, and
priorities between prevention, defence, and a shared understanding of
the converged nature of cyber risk. This pre-emptive approach to develop
effective cyber defence and identifying causes of future failure
identifies priorities for response training, and the development of a
response doctrine that can provide agility and options.