In September of this year, I successfully defended my doctoral dissertation, earning the title of Doctor of Education from Baylor University. In this post, I’m sharing the entirety of that dissertation freely to benefit the information security community. I’ll also provide recommendations on relevant sections of the work based on your role. Finally, I’ll talk a bit about the past, present, and future of my research.

I believe this dissertation represents a significant step forward in understanding the cognitive skills that high-performing analysts rely on when conducting security investigations. The findings here help establish analytic doctrine and should yield significant improvements in how analysts are trained when considered thoughtfully by educators. Similarly, analysts who better understand their own cognitive skills stand to increase their metacognitive awareness. This knowledge has the potential to improve analyst performance as well as their ability to communicate with peers and mentor less experienced practitioners. This paper also further establishes the field of human-centric investigation theory research.

Let’s get straight to the point. You can download my complete dissertation paper for free at the link below.

DOWNLOAD: The Analyst Mindset: A Cognitive Task Assessment of Digital Forensic Analysts

Abstract

Despite significant investment in cyber security, the industry is unable to stem the tide of damaging attacks against computer networks. This unfortunate situation is, in part, because cyber security exists in a state of cognitive crisis defined by tacit knowledge and poorly understood processes. At the heart of the crisis are digital forensic analysts that identify and investigate intrusions. Unfortunately, even skilled analysts in these roles are often unable to explain how they go about the process of finding intruders and assessing their foothold on a network. Without this knowledge, professional and academic educators are unable to build a standardized industry-accepted curriculum for the identification and training of new analysts. While there have been some attempts to inventory the skills, processes, and knowledge required to serve in the digital forensic analyst role, no current efforts provide a thorough, research-backed accounting of the profession with consideration for cognitive skill elements.

This problem of practice study details a cognitive skills assessment of the digital forensic analyst profession by leveraging two Cognitive Task Analysis (CTA) research methods. The Simplified Precursor, Action, Result, Interpretation (PARI) method provided a framework for eliciting procedural skills, and the Critical Decision Method (CDM) supported the discovery of decision-making skills. Using these techniques, interviews conducted with expert analyst practitioners revealed four unique procedural skill categories, characteristics of two significant facets of analyst decision making, and numerous subcategory elements that describe additional dimensions of expert analyst performance. The results converged on a model of diagnostic inquiry that represents the relationships between how analysts formed investigative questions, interpreted evidence, assessed the disposition of events, and chose their next investigative actions. These findings establish explicit knowledge that provides a foundational understanding of how skilled analysts perform investigations. They also lay new groundwork for cyber security’s emergence from its cognitive crisis, with implications for educators and practitioners alike.

Reading Guide

If you’re reading this as an information security practitioner, I recommend reading Chapter 1 (Introduction) for an overview, Chapter 2 (Literature Review) for background information, and then focusing on Chapter 4 (Findings) and the appendices referenced in it. Keep in mind that it is a research document, not a teaching document. It describes the process and results of my research on analyst cognitive processes and is narrowly scoped to the findings that I uncovered. These findings have significant value to analysts and those who support them but are not necessarily meant to be handed to an entry-level analyst on their own. If you want to learn to be an analyst, I recommend my Investigation Theory class, where much of my research (including this dissertation) manifests with learning in mind. It is here that these concepts are scaffolded by other relevant knowledge, paired with examples and demonstrations, and tied to specific learning objectives so that analysts can wield them properly.

If you’re reading this as an educator, then I recommend reading Chapter 1 (Introduction), Chapter 4 (Findings), and Chapter 5 (Distribution of Findings). My primary goal with this research was to identify analyst cognitive skills so that we may better teach those skills to others. I expect that this work will find a home in many community college and university courses that have investigative components. If you do end up building curriculum components around these concepts, I’d love to hear about your approach.

If you’re reading this as an academic researcher, then I recommend reading the entire document so that you may understand my methods as well as the results I uncovered. I put extra effort into describing my cognitive task analysis strategy. My experience as a 15+ year practitioner before moving into the academic research space is atypical but allowed me to conduct this research through a unique lens that would not be possible by researchers lacking professional experience. I tried my best to elaborate on my research methods to highlight how I deployed my expertise to design the study and conduct data collection and analysis. I hope this work will help bring the academic and practitioner communities closer together.

My Research and How I Got Here

I struggled tremendously when I first began my analyst career. I could not understand how investigators took inputs and used them to pivot between various data sources and find evidence of compromise. I distinctly remember sitting in a state of paralysis, starting at a blank search bar, not knowing what to do next. I didn’t have access to many highly skilled people, and those who were could not effectively explain how they connected the dots. I was told to watch how they did their work, play around in the data, and I would eventually figure it out. I eventually did figure it out, but that path was much longer and more frustrating than it should have been. Worse yet, I was continually told that good analysts were born with a particular set of traits, and without them, someone’s chance of doing the job well was limited.

Over time, I recognized that information security is in a state of cognitive crisis. So much of the knowledge we rely on is tacit and unavailable to those seeking the practice this craft. That negatively affects everyone attempting to enter the field, but it affects those who are already marginalized even more. As my interests turned from computers to the humans using them, I felt a desire to make these tacit processes explicit, which led me on a long and challenging journey that included researching and writing the dissertation you see attached to this post. Along this path, I learned that digital forensic investigations are not art, although there is room for creativity to guide an analyst’s path. I also learned that digital forensic investigations are not science, but we can use scientific processes to study how humans can better bridge the gap between perception and reality. Digital forensic analysis is engineering. With the right people in the room asking the right questions, nothing a computer does cannot be explained. That means that the only thing standing between me and knowing what happened is my own ability to understand evidence and behavior. These realizations empowered me, dramatically changed my career trajectory, and are why you’re reading this.

Why an Education Doctorate?

I once read that everything exciting happens on the fringes of where two things meet; the middle is boring because everything is the same. While I’m not sure I agree with that idea completely, I know that most of my professional curiosity is stimulated at the borders shared by cyber security, cognitive psychology, and education. Every investigation involves a human sitting at a console looking at data. Ultimately, those humans have the most to say about whether a compromise is fully discovered and contained.

Paul L. Kirk was a biochemist, criminologist, and early pioneer of forensic science. He was also a successor of Edmund Locard, who is considered by many to be the father of modern forensic science. In 1953, Kirk invoked Locard’s Exchange Principle when he wrote a now-famous quote describing the relationship between an investigator, a criminal, and the evidence they leave behind. I’ve taken the liberty of updating his quote to make it more relevant to modern digital forensics.

Wherever they pivot, whatever they access, whatever they leave behind, even unconsciously, will serve as a silent witness against them.

Not only their authentications or their executions, but the packets they transmit, the files they change, the tool marks they leave, and the data they upload or download.

All of these and more bear mute witness against them. This is evidence that does not forget. It is not confused by the excitement of the moment. It is not absent because human witnesses are. It is factual evidence. Digital evidence cannot perjure itself and it cannot be wholly absent. Only human failure to find, study, and understand evidence can diminish its value.

Chris Sanders, Revised from Paul L. Kirk (1953)

I find the last line of that quote (unchanged from its original version) the most impactful. Ultimately, many of the issues cyber security faces in its cognitive crisis are education problems. We must better understand how and why experts do the things they do to teach them to other people and refine them; something that we’re currently failing at.

Therefore, cyber security is the medium for my work, cognitive psychology provides the framework for understanding how analysts perform, and my findings are expressed through education. My choice to pursue a doctorate in education was primarily focused on the outputs I hope to achieve from my research; a more clear establishment of the human-centric investigation theory research field, a more formal digital forensics analytic doctrine, and the methods to help people learn that doctrine.

What’s next?

While a doctorate is a terminal degree, the document encapsulating it is only the beginning. I plan to continue my research focused on different facets of how analysts perform investigations and strategies for teaching investigation concepts. As a matter of fact, I have ongoing research projects as you read this post.

As part of this continued work, I’m seeking research partners who might want to work or collaborate. This includes:

• Universities who may be interested in providing graduate students to assist in research projects. These students will be compensated for their time and can come from a variety of subject areas like psychology, sociology, or education. They do not have to posses prior cyber security experience, but should be interested in the field and have some exposure to quantitative and qualitative research techniques. These opportunities are remote/online under the Applied Network Defense research umbrella.
• Businesses that wish to provide access to analysts for research subjects. A unique challenge of conducting analyst-centric research is finding enough analysts to serve as research subjects. This is particularly the case when I need to conduct research focused on several analysts within a single security operations center. If your organization is interested in providing analysts for these studies, please reach out. Depending on the scope of the research, there may be some costs associated with participation. However, as part of participating, you’ll receive free and discounted training courses along with priority access to research findings, as well as consulting from me on how to leverage the results meaningfully in your security team.

Please contact me directly if you are interested in either of these opportunities.

My Acknowledgments

While my acknowledgments are included in the dissertation document itself, I thought it important to also include them here just like I do with in all the blog posts that have accompanied the release of my books.

I would like to thank the people who helped make this document possible and contributed to the positive step forward it represents. First and foremost, thank you to my wife Ellen, who I kept awake countless nights by storming into the bedroom rambling on and on about the ideas running through my head following late classes.

Nobody becomes a scholar alone, and I was fortunate to have several amazing people on this journey with me. I want to extend my gratitude to my doctoral colleagues who made this whole experience more enjoyable. I also want to thank my instructors at Baylor who shepherded me along this scholarly experience, with special thanks to my advisor, Dr. Sandi Cooper. I don’t fit the mold of a typical education student, and I appreciate all of you opening up your mind to learn from me as I did from you.

I want to pay special tribute to my students, whose success helps motivate me, including anyone who has ever taken one of my classes, read one of my books, or sat in on one of my conference presentations. Additionally, I want to thank my colleagues that served as sounding boards and provided feedback on my ideas.

This whole project started over a decade ago when I was a struggling young analyst trying to learn the craft. Someone told me that you are either born with the skills needed to do this job, or you are not. I thought that was nonsense, and I have spent the rest of my career gathering the knowledge and data to prove it. The document you are about to read is a step along that path. I don’t remember the name of the person who told me that, but I want to thank them too.

Download and Citation

You can download my complete dissertation paper for free at this link.

The paper will be available in the Baylor and ProQuest databases on December 19th, 2021/

You may cite this work as:

Sanders, C. (2021). The analyst mindset: A cognitive task assessment of digital forensic analysts [Doctoral dissertation, Baylor University]. https://chrissanders.org

The post A Cognitive Skills Assessment of Digital Forensic Analysts – My Doctoral Dissertation first appeared on Chris Sanders.