American Board of Information Security and Computer Forensics

Results and Discussion

Speed and accuracy of detection. For this experiment, with its three devices which were either simple, moderate, or difficult to detect, two factors were important. The first, as in Experiment 1, was detection time. The second was whether detection was accomplished at all. The effect of training on detection was significant. Participants in the training condition were significantly more likely to detect all three devices than were those in the control condition, χ2 (1) = 4.57, p = .032. Twelve of twenty-three training respondents (52%) successfully detected all three devices; only five of twenty-three control respondents (22%) did so.

As in Experiment 1, the effect of participant gender was not significant. However, in terms of simple-device detection time, male performance was 40.10% superior to that of females. The fact that this relatively large difference was non-significant again reflects very large standard deviations. This large disparity in the performance of females and males may reflect the influence of social expectations. We have previously provided evidence that gender-based expectations can reduce women’s visual/spatial cognitive performance relative to that of men in task contexts in which no sex difference is observed when those expectations are removed (Sharps, Welton, & Price, 1993; Sharps, Price, & Williams, 1994). Regardless, this result underscores the importance of understanding individual differences in this area.

Again as in Experiment 1, trained respondents (mean detection time 18.28 seconds, SD = 13.21 seconds) found the “simple” device significantly faster than did control respondents (mean 34.14 seconds, SD = 22.13 seconds), F (1, 44) = 8.72, p = .005. This reflected a 47% advantage in time to detection as a result of training, similar to the 53% observed in Experiment 1.

Training did not result in enhanced detection speed for the moderately difficult device, nor, interestingly, did it enhance the likelihood of finding this device significantly. This will be further discussed below in the “Error Analysis” section with reference to errors of movement.

Perhaps the most important results involve the detection of the difficult device. Real devices are most likely to be concealed and the best-concealed of the mock IEDs used here was the stick-timer grenade, which was concealed in the shadows beneath an office chair and placed parallel to one of the chair’s horizontal supports. Training did not enhance the speed of detection of this device, but it did dramatically enhance the ability of trainees to find it at all. Only five of the control-group respondents found the difficult device. However, twelve of the respondents in the training group did so. The difference was significant, χ2 (1) = 4.57, p = .032. Not only were respondents in the training condition faster at detecting the simple device, they were more likely to detect all three devices, including the complex device—the IED which was most difficult to find.

Error analysis. These results are presented in Table 1. The training provided in this experiment significantly reduced the average number of errors across three of the five error types. Errors of SEARCH were significantly reduced, F (1,44) = 6.77, p = .013, as were errors of OBSERVATION, F (1, 44) = 19.48, p < .001, and failures to KEEP SEARCHING, F (1, 44) = 31.26, p < .001.

Trainees, as opposed to respondents in the control group, were more likely to engage in a comprehensive search of the room, examining all quadrants and searching above, below, and at their line of sight for IEDs. They were also more likely to identify and report an IED when they saw one. Finally, trainees were more likely to continue their search for additional IEDs even when they had already identified one. Thus, the training provided in Experiment 2 resulted in a significant advantage in IED search and detection beyond the levels observed with the control group.

Errors of Evaluation were not reduced by the training. Those who received training were just as likely to identify an innocuous object as a potential IED as those who did not. This result may in fact be construed as salutary. In a potentially IED-rich environment, it is far more dangerous to mistake an IED for an innocuous object than it is to mistake an innocuous object for an IED. The degree to which this may prove important in the field should be the subject of further research.

Errors of Movement also were not significantly reduced by the training provided. This speaks directly to the finding cited above with reference to the “moderate” device—the fact that our training did nothing to improve the detection of this device. Although the device, a single-pipe bomb, was situated in plain sight, it was only in plain sight from a specific perspective within the field test chamber. The moderate IED was placed between a computer and its printer. It was not directly visible unless the participant moved to a position in which there was a direct line of sight to its location.

The training provided was presented in a PowerPoint presentation. Trainees did not move during the training; they were seated before a screen on which the presentation was made. Therefore, although they were informed of the need to move about any given environment to enhance their perspective, they received no direct motor training in this crucial aspect of search.

Was this the reason for the absence of a training effect for the moderate device? An additional analysis was conducted to address this question. We correlated the presence of errors of Movement with the amount of time needed to detect each device. Errors of Movement were significantly correlated with the amount of time needed to detect the moderate device, r (22) = 0.59, p < .001. No such correlation was observed  with the simple or complex device. Thus, errors of Movement, which were not reduced by SMOKE training in isolation, were the most probable explanation for the anomalous results observed with the moderate device.

This result strongly underscores a point made above: SMOKE training, the front-loaded, explicit, feature-intensive training system presented here, cannot replace field training. Rather, it is presented as a powerful adjunct to field training. The present findings indicate that the SMOKE system must be accompanied by rigorous field training, in realistic environments, for the detection of IEDs. Trainees in possession of the cognitive skills engendered by SMOKE must search realistic mock environments as well, in order to foster the movement skills needed for successful searches and also to fine-tune the skills acquired in the SMOKE training for real-world environments. From the perspective of cognitive science, the SMOKE system should be presented, followed by realistic training provided by field training experts, and then presented again; this will optimize both the primacy and recency effects (see Baddeley, 1990; Crowder, 1976; also see Sharps et al., 1996) for optimal memory and retention of the training, as discussed above.

Cognitively based training such as SMOKE cannot, and is not intended to, replace solid field training. However, the present experiments have demonstrated that such training provides a powerful cognitive framework for successful field training, and for ultimate success at IED search and detection.

Individual differences. Interestingly, neither of the scales of the Shipley instrument, which deals with general cognitive functioning, was related to successful searches across conditions of detection difficulty. Neither were the scales of the Digit Span, which deals with attentional and short-term memory capabilities, nor our adaptation of the Rey-Ostereith figure, which deals with the ability to perceive and remember a complex visual pattern.

However, a more interesting pattern was observed with reference to the “difficult” IED condition alone, which presented the most realistic of the three mock IEDs presented in the field test evolution. In the control condition, none of the individual-difference measures manifested significance. However, across the control and training conditions, performance on the Rey-Ostereith figure was in fact significantly related to the ability to detect the difficult device, F (1, 30) = 4.39, p = .045. In the training condition, this significance was maintained, F (1, 12) = 11.70, p = .005. The Shipley Abstractions scale also proved to be related to performance in the training condition, F (1, 12) = 4.90, p = .047.

The most parsimonious explanation of this pattern of results follows. Neither intellect, short-term memory, attention, nor even the ability to remember a complex visual pattern predicted the ability to search for and detect IEDs in the absence of cognitively based training. However, what was predicted by our adaptation of the Rey-Ostereith figure and by the Shipley Abstractions elements, was the ability to benefit from the training provided within the SMOKE framework, at least with reference to more difficult and therefore more realistic IEDs. Those with better abstraction powers benefited more from the training than did those without. Perhaps more importantly, the ability to remember and process a complex visual figure (the Rey-Ostereith) was also directly related to the ability to benefit from SMOKE training. These are preliminary results, of course, and further research will be absolutely necessary to verify and, if appropriate, to extend them; but these findings strongly indicate the potential for an empirically-based pathway to predict those who are more likely to benefit from cognitively based IED search and detection training.

General Discussion

Taken together, these experiments have demonstrated the following:

Five specific types of errors characterize unsuccessful search and detection of IEDs. These are described in detail above.

Cognitively based, front-loaded, explicit, feature-intensive training can be used to reduce many of these errors significantly, and to provide superior performance in the detection of IEDs, as suggested by earlier work (e.g, Sharps, 2003, 2010).

The effects of such training are strong in isolation; however, it is highly probable that far stronger and more salutary effects will be achieved in combination with a comprehensive program of realistic field training.

It may be possible to predict the probability that a given trainee will benefit substantially from this type of training, based on his or her capabilities with reference to abstraction and to the recall and processing of complex visual stimuli. However, further research is absolutely essential on this issue.

Effective search and detection of IEDs is of substantial importance now, and is likely to assume even greater importance in the future (e.g., Cameron, 2008). The present results have demonstrated the importance of front-loaded, explicit, feature-intensive training (see Sharps, 2003, 2010) as a powerful adjunct to field training methods already in use.

It is of paramount importance to conduct additional research to determine the most important components of this training and the degree to which individual differences may be used to predict trainee success. It is also important to determine the generality of these results across populations, especially with reference to age and experience. Additional research should also address the degree to which the SMOKE method is used consciously by trainees in their approach to field evolutions. Ultimately, however, the present findings serve to underscore the importance of a cognitive approach to IED search and detection and the degree to which this approach has immediate practical benefits for the training of law enforcement, military, and appropriate civilian constituencies in an increasingly risky world. It is hoped that the inclusion of the type of training elucidated in the present experiments will reap substantial benefits, especially for law enforcement and for those charged with homeland security, in terms of enhanced officer survivability, professional effectiveness, and enhanced abilities to protect the civilian population.

References

Baddeley, A.D. (1990). Human memory: Theory and practice. Boston: Allyn & Bacon.

Baddeley, A.D., & Hitch, G.J. (1993). The recency effect: Implicit learning with explicit retrieval? Memory and Cognition, 21, 146-155.

Bransford, J.D., & Johnson, M.K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11, 717-726.

Bransford, J. D., & Johnson, M. K. (1973). Considerations of some problems of comprehension. In W. G. Chase (Ed.), Visual information processing. Orlando, FL: Academic Press.

Cameron, S. (2008). Securing the cities: Agencies working together to detect dangerous radiological materials. The Police Chief, reprinted offprint of article appearing in the October, 2008 edition, pgs. 1-5

Craik, F.I.M., & Lockhart, R.S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684.

Crowder, R.G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum.

Force Science News (2009, February 13). I. New study: When civilians would shoot…and when they think you should. Force Science Research Center, Transmission #117, www.ForceScienceNews.com

Grossman, D., & Christensen, L.W. (2004). On combat. PPCT Research Publications.

Haviland, S.E., & Clark, H.H. (1974). What’s new? Acquiring new information as a process of comprehension. Journal of Verbal Learning and Verbal Behavior, 13, 512-521.

Hess, A.B., & Sharps, M.J. (2006, April). Identification and interpretation of peripheral sources of hazard in complex crime situations. Western Psychological Association, Palm Springs, CA..

Mandler, G. (1967). Organization and memory. IN K.W. Spence & J.T. Spence (Eds.), The psychology of learning and motivation (Vol. 1). Orlando, FL: Academic Press.

Meyers, J.E., & Meyers, K.R. (1995). Rey complex figure test and recognition trial. Lutz, FL: Psychological Assessment Resources, Inc.

Miller, G.A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81-97.

Sharps, M.J. (2003). Aging, representation, and thought: Gestalt and feature-intensive processing. New Brunswick, NJ: Transaction.

Sharps, M.J. (2010). Processing under pressure: Stress, memory, and decision-making in law enforcement. Flushing, NY: Looseleaf Law Publications, Inc.

Sharps, M.J., Barber, T.L., Stahl, H., & Villegas, A.B. (2003). Eyewitness memory for weapons. Forensic Examiner, 12, 34-37.

Sharps, M.J., & Hess, A.B. (2008). To shoot or not to shoot: Response and interpretation of response to armed assailants. Forensic Examiner,17, 53-64.

Sharps, M.J., Hess, A.B., Casner, H., Ranes, B., & Jones, J. (2007). Eyewitness memory in context: Toward a systematic understanding of eyewitness evidence. Forensic Examiner, 16, 20-27.

Sharps, M.J., Janigian, J., Hess, A.B., & Hayward, B. (2009). Eyewitness Memory in Context: Toward a Taxonomy of Eyewitness Error. Journal of Police and Criminal Psychology,24, 36-44.

Sharps, M.J., Price, J.L., & Bence, V.M. (1996). Visual and auditory information as determinants of primacy effects. Journal of General Psychology, 123, 123-136.

Sharps, M.J., Price, J.L., & Williams, J. (1994). Spatial cognition and gender: Instructional influences on mental image rotation performance. Psychology of Women Quarterly, 18, 413-425.

Sharps, M.J., Welton, A., & Price, J.L. (1993). Gender and task in the determination of spatial cognitive performance. Psychology of Women Quarterly, 17, 71-83.

Shipley, W.C. (1940). A self-administering scale for measuring intellectual impairment and deterioration. Journal of Psychology, 9, 371-377.

Shipley, W.C. (1953). The Shipley Institute of Living Scale for measuring intellectual impairment. In A. Weider (Ed.), Contributions toward medical psychology (Vol..2): Theory and diagnostic methods (pp. 751-756). New York: Ronald.

Shipley, W.C., Gruber, C.P., Martin, T.A., & Klein, A.M. (2009). Shipley-2 manual. Los Angeles: Western Psychological Services.

Steblay, N.M. (1992). A meta-analytic review of the weapon focus effect. Law and Human Behavior, 16, 413-424.

Villegas, A.B., Sharps, M.J., Satterthwaite, B., & Chisholm, S. (2005). Eyewitness memory for vehicles. Forensic Examiner, 14, 24-28.

Wechsler, D. (2008). WAIS-IV. New York: Pearson/Psychological Corporation.

Acknowledgements

The authors wish to thank Chief of Police Jerry Dyer and the staff and officers of the Fresno Police Department for their unstinting expert advice, generosity, and support of this research. Thanks also to Sergeant Michael Manfredi of the Fresno Police Department, and to Lisa Giuliani, Amy Balmanno, and Morgan Goodwin for their help in the preparation of the materials for this research. The research, views, and opinions presented in this paper are entirely and solely those of the authors, and do not necessarily reflect the views or opinions of the Fresno Police Department or of its Chief of Police, staff, officers, or employees.