Testing a supervisor-led extension of self-reflection resilience training: A controlled trial randomized by platoon at the Royal Military College

Abstract

The study tested an extension of a promising adaptive self-reflective approach to resilience training. The extension integrated resilience training content in routine workplace activities via supervisors. Participants were military cadets (N = 168), randomized by platoon into two conditions. The control condition received the original self-reflective resilience training only (n = 85), and the intervention condition additionally received a supervisor-led extension of this programme (n = 83). Participants completed assessments of depression and anxiety symptoms and perceived stress at four time points over five months. Cadet performance scores were also obtained. Findings indicated that participants receiving the supervisor-led extension demonstrated better psychological outcomes earlier than cadets in the control condition. However, at Time 4 both interventions had equivalent levels of mental health outcomes and perceived stress. The supervisor-led condition demonstrated better average performance than the control condition for the performance measure congruent to the workplace activity in which the extension was applied. Mechanisms for the effectiveness of the supervisor-led extension were explored. Analysis suggested that perceived supervisor support for the individual mediated the intervention–psychological outcome relationship. This research demonstrates the effectiveness of a scalable resilience intervention and speaks to a role of supervisors in facilitating resilience via supportive interactions.

Practitioner points

• Supervisors can feasibly extend Self-Reflection Resilience Training by leading adapted After Activity Reviews that focus on the psychological demands of a performance scenario and permit reflection on how individuals cope with work demands.
• Structured supervisor-led conversations that facilitate reflection on the coping process may meaningfully improve upon original Self-Reflection Resilience Training outcomes, including mental health and performance.
• Supervisors leading conversations that take aim at reflecting on coping with work demands can support resilience in mental health by enhancing perceived supervisor support.

INTRODUCTION

Resilience is characterized by the observation of sustained functioning during or quick recovery following psychological risk (Kalisch et al., 2017). Functioning includes individually or organizationally valued and contextually relevant outcomes, such as mental health (Kalisch et al., 2017) and performance (Gucciardi et al., 2018). Resilience training aims to increase an individual's capacity to achieve a resilient outcome despite psychological risk (Robertson et al., 2015; Vanhove et al., 2016). Whilst group-based resilience training programmes have been devised for organizations to prevent the onset of mental ill-health or performance deficits when anticipating employee exposure to psychological threats (Vanhove et al., 2016), they rarely use supervisors as a vector to facilitate the practice of resilience skills at work (Crane et al., 2021) and/or provide opportunities to demonstrate supportive supervision. Equipping supervisors to guide the application of resilience skills at work is an opportunity to use in situ events to practice training skills (Salas & Cannon-Bowers, 2001). Previous work demonstrates the role of leaders in fostering employee resilience (e.g., Adler et al., 2014; Crane et al., 2022). Thus, supervisor involvement in the workplace application of resilience training is proposed to increase the efficacy and/or expeditiousness of resilience training outcomes. To exploit the available levers that have the potential to support organizational resilience training outcomes, the question for this research is can supervisors be used to improve resilience training outcomes and what does their involvement teach us about how employee resilience can be achieved?

The systematic self-reflection approach to resilience training is based on the Systematic Self-reflection Model of Resilience (Crane, Searle, et al., 2019) that proposes that challenging, but manageable stressors, are an opportunity for the development of capacities for resilience. Training based on the Systematic Self-reflection Model of Resilience (Crane, Searle, et al., 2019) teaches a method for reflecting on daily stressors. This method of self-reflection is proposed to promote coping self-insights (see: Falon, Kangas, & Crane, 2021 for review) that propel refinements to resilient capacities (e.g., coping strategies, resources; Crane, Boga, et al., 2019). Randomized controlled trials have demonstrated the promise of adaptive forms of self-reflection as a resilience-strengthening strategy when coupled with moderate stressors in a number of workplace contexts, including military officer training (e.g., Crane, Boga, et al., 2019; Crane, Kangas, et al., 2020; Crane, Kho, et al., 2020; Falon, Karin, et al., 2021). The nature of the systematic self-reflection protocol used in previous trials permits delivery by non-mental health professionals and may provide an avenue for engaging supervisors in practical strategies for supporting the resilience of personnel. In this superiority trial, we tested the added benefit of a supervisor-led extension to self-reflection training on mental health and workplace performance, wherein supervisors guided trainees' application of adaptive self-reflection on daily work and officer training stressors.

To contribute to our theoretical knowledge of how a supervisor-led extension may promote improved trajectories of resilience on mental health and enhance performance outcomes, we explored two potential mechanisms. First, a supervisor-led extension is proposed to enhance perceived supervisor support for the resilience training. Perceptions of supervisor support for the application of resilience training skills may motivate engagement and timely application by personnel (Govaerts & Dochy, 2014; Sims & Adler, 2017). Second, supervisor involvement may enhance perceived supervisor support for the individual (i.e., perceptions that one's supervisor cares about and provides assistance to them; Kottke & Sharafinski, 1988; Nahum-Shani et al., 2014) and in turn support mental health outcomes. Organizational training approaches have emphasized the role of the former mechanism whereby supervisor support for training predicts engagement and application (Holton et al., 2003; Lancaster et al., 2013). However, for mental health outcomes the latter mechanism may be at play given that supervisor support is a known predictor of employee mental health (Rhoades & Eisenberger, 2002; Somers et al., 2018). Delineating between these mechanisms to outcomes will provide practical guidance regarding the behaviours supervisor training protocols should emphasize when engaging in resilience training.

Self-reflection resilience training

Most organizational resilience training interventions have focused on training specific coping strategies (e.g., Cohn & Pakenham, 2008; Millear et al., 2008; Sood et al., 2011). Self-reflection resilience training (SRT) seeks to promote adaptive forms of self-reflection on one's coping process with moderate stressor experiences to enhance coping self-insights. These coping self-insights in turn drive actions to extend and refine one's repertoire of coping strategies (e.g., cognitive and behavioural coping options) and coping enabling resources (e.g., access to professional support networks). Foundational to this approach was previous work in education (e.g., adult transformational learning; Mezirow, 1998), self-reflection for promoting workplace performance (e.g., Anseel et al., 2015; Ellis et al., 2014) and the Metatheory of Resilience and Resiliency (Richardson, 2002) that proposes the involvement of introspective processes in resilient re-integration post-adversity.

The Systematic Self-Reflection Model of Resilience (Crane, Searle, et al., 2019) and the Self-Reflection and Coping Insight Framework (Falon, Kangas, & Crane, 2021) were the first attempts to explicate the self-reflective practices and coping insights necessary for the translation of daily stressor experiences into the development of one's resilient capacities (i.e., the modifiable factors that increase the likelihood of an individual's future resilient outcomes). These models addressed an existing gap in our understanding of how individual capacities are strengthened following daily stressors and the psychological mechanisms involved in this process. The five proposed self-reflective practices identified and taught by SRT via journaling sessions were (1) one's emotional, physical, behavioural and cognitive responses to triggering stressor events, (2) context relevant values and goals, (3) capacities applied to address the event and stressor response, (4) evaluation of applied strategies in relation to values and goals and (5) future plans for adapting capacities for improving one's coping response (Crane, Searle, et al., 2019). These self-reflective practices are thought to result in the broadening of resilient capacities and refinement in strategy or resource application promoting a ‘good fit’ to external conditions. The aim of self-reflection on the coping process is to drive the iterative development of one's resilient capacities over time, as the daily demands of life are encountered, thereby increasing the likelihood of downstream resilience (Crane, Searle, et al., 2019).

Integration of self-reflection resilience training into the workplace via supervisors

Evaluating resilience training across a stressor experience is essential to assess its utility as a preventive tool (Chmitorz et al., 2018; Vanhove et al., 2016). Resilience interventions are often conducted in mentally well populations and are considered successful if participants demonstrate a stable trajectory of mental health across a stressor period, visually depicted in Figure 1. Robust resilience captures those who resist the deleterious effects of stressor exposure on functioning and minimal impact resilience captures those who experience an initial impairment but readily return to baseline functioning when stressors cease (Bonanno & Diminich, 2013). Whilst military training does not uniquely potentiate risk, it bears the potential to effect functioning (e.g., depression and anxiety symptoms have been observed to initially elevate during basic combat training; Martin et al., 2006; Britt et al., 2016) as adjustment to training conditions occur.

FIGURE 1

Graphical depiction of trajectories of minimal impact and robust resilience in the context of a stressor period (indicated by shadowed area) across axes of time and functioning.

Two trials have demonstrated the promise of SRT within the stressor context of military officer training (Crane, Boga, et al., 2019; Falon, Karin, et al., 2021). However, the benefits of SRT on anxiety and depression symptoms followed a trajectory of minimal impact resilience. Both the control and intervention condition demonstrated initial symptom increases with increasing officer training stressors (Crane, Boga, et al., 2019; Falon, Karin, et al., 2021), but only the intervention condition demonstrated a reduction in symptom severity during a period of stressor cessation. Given that the process of resilient capacity refinement is iterative and person-driven, improvements to individuals' resilient capacities and resilient outcomes take time to be observed (Crane, Boga, et al., 2019). Although the attainment of minimal impact resilience was favourable, it is ideal to prevent even mild decreases in functioning during a stressor period. Even mild symptoms of mental ill-health within the workplace can be costly to the organization given that a larger proportion of people are likely to experience mild symptoms, compared with more severe symptoms (McTernan et al., 2013).

To promote the observation of earlier mental health benefits than those observed in previous research and produce a trajectory of robust resilience, methods to expedite or enhance SRT were considered. Timely integration of training into the workplace can expedite and enhance training outcomes (Salas & Cannon-Bowers, 2001). This approach ensures that participants are provided guided opportunities for in situ skill application promptly following training (Salas & Cannon-Bowers, 2001). Given that the SRT does not directly train coping skills, it was considered feasible for non-psychologist supervisors to coach the application of coping self-reflection on daily work-related stressors (e.g., performance demands).

Based on related empirical and theoretical work on organizational training and resilience, we propose two respective mechanisms by which supervisors guiding the application of resilience training skills in situ may influence training efficacy: perceived supervisor support for resilience training and perceived supervisor support for the individual.

Perceived supervisor support for resilience training

By guiding the application of training skills supervisors demonstrate their support for training (Govaerts & Dochy, 2014) and may foster subordinate motivation and self-efficacy to transfer training skills into the workplace (Holton et al., 2003; Lancaster et al., 2013). Previous work has demonstrated an association between leader encouragement of resilience training skill use and communications of training importance with soldier's perceived leader support for training and reports of training transfer (Sims & Adler, 2017). This is aligned with organizational training literature that contains models (e.g., Baldwin & Ford, 1988; Burke & Hutchins, 2007) that denote the contributing role of supervisor support for training, on training transfer. However, the role of supervisors is yet to be explored in the context of resilience training. The influence of supervisors' may be explained via subordinates' belief that supervisors will value efforts to apply skills and a socially learnt appreciation for the training motivating application (Gegenfurtner et al., 2009; Salamon et al., 2021).

Theoretical work speaks to the potential of supervisor support for the resilience training to enhance application and therefore outcomes. According to self-determination theory (Ryan & Deci, 2000), behaviour can be driven by controlled motivators that are characterized by a sense of external or internal pressure. As a controlled motivator, supervisor support for training may externally regulate transfer via explicit instruction to apply training skills. It may also motivate transfer via integrated regulation, a form of autonomous motivation, as individuals come to value the training based on supervisor endorsement. Social cognitive perspectives (Bandura, 1998) propose that skill acquisition requires self-efficacy, that is the belief the one can master the new skill. Supervisors guiding the use of self-reflection at work may therefore enhance subordinates' self-efficacy in the skill and in turn promotes its application at work (Al-Eisa et al., 2008). As the enhanced application of self-reflection on stressors and coping is anticipated to develop individuals' resilient capacities, the current study is designed to determine whether supervisor support for resilience training improves trainees' resilience outcomes.

Perceived supervisor support for the individual

In addition to amplifying perceived supervisor support for training, the engagement of supervisors in integrating resilience training at work may increase perceived supervisor support for the individual. In part, supportive supervision is characterized by the provision of guidance and assistance to subordinates (Kottke & Sharafinski, 1988; Nahum-Shani et al., 2014). This individualized perceived supervisor support is positively associated with the psychological health and performance of subordinates (e.g., Rhoades & Eisenberger, 2002; Somers et al., 2018) and can buffer the effect of work stressors (Britt et al., 2004). Research examining the role of health-oriented leadership (i.e., leadership behaviours that communicate and model: health behaviour, value of health and health awareness; Franke & Felfe, 2011) has demonstrated that high health-oriented leadership is related to a greater likelihood of resilient physical and mental health as opposed to a deteriorating trajectory during stressful work transitions (Arnold & Rigotti, 2021). These findings are aligned with the conservation of resource theory (COR) that asserts that resource possession is both protective to resource loss and facilitates resource gain (Hobfoll, 2011). In an organizational setting, resource possession is proposed to prevent resource loss cycles, that is when individuals or groups lose resources required to overcome challenges during a period of stress, placing them at greater vulnerability for further loss (Hobfoll, 2011). Supervisor-led reflection provides routine opportunities for supervisors to support employee development and well-being potentially uplifting perceptions of supportive supervision. As a resource during high demand, supportive supervision for the individual may reduce future resource loss and/or promote greater capacities for resilience, thereby increasing the likelihood of robust resilient outcomes.

Performance outcomes

The capacity for resilience may also produce performance outcomes by supporting the consistent application of knowledge and skills in contexts of psychological pressure. Resilient capacities such as one's coping strategies and resources may be instrumental in managing contextual demands that may otherwise cause distress and negatively affect performance. Past work demonstrates that intra-personal resources (i.e., general self-efficacy and perceived resilience) enhanced the speed of skill acquisition in a high-pressure context (Crane et al., 2017), but not a low-pressure context. Attention control theory (Eysenck et al., 2007) suggests that high stress takes cognitive attention from the task at hand, potentially impairing performance. In the current study, we test the potential of resilience training to enhance performance outcomes during stressful training scenarios. As for mental health outcomes, the benefits of supervisor involvement in resilience training to performance outcomes may occur via the supervisor support for resilience training that improves the development of resilient capacities and/or more directly via supervisor support for the individual as a resource that is associated with subordinate performance (Talukder & Galang, 2021).

The present study

The current superiority trial tested whether the addition of the supervisor-led extension enhanced resilience training outcomes by supporting a robust resilience trajectory. This methodology to training evaluation captures the added benefit of discrete training elements (Howard & Jacobs, 2016). The supervisor-led extension, integrated self-reflection training (iSRT) was compared with the original SRT via a clustered-randomized controlled trial by platoon, with a sample of cadets at the Australian Royal Military College (RMC) undergoing a demanding period of military training. The iSRT was implemented during two field operations, in which training stressors intensified and cadets had constant and close contact with cadet supervisors (hereon in referred to as instructors).

Figure 2 illustrates the gaps in current research addressed in this superiority trial. In addressing these gaps, the current study contributes a practical supervisor-led extension to Self-Reflection Resilience Training and extends our knowledge of the mechanisms by which supervisor involvement in resilience training influences subordinate resilience. Data collection was designed to capture both the onset and offset of the stressor period, to ascertain cadets' trajectories of resilience (Chmitorz et al., 2018) and pre- and post-field operations to assess the effect of iSRT. Accordingly, cadets completed assessments of depression and anxiety symptoms and perceived stress at Time 1 (T1) the onset of stressors/pre-first field operation, Time 2 (T2) continuing stressors/post-first field operation, Time 3 (T3) continuing stressors/pre-second field operation and Time 4 (T4) the offset of stressors/post-second field operation (Figure 1). Performance scores were obtained at the end of the period of military training.

FIGURE 2

Graphical depiction of the knowledge gaps addressed by the current study. Dotted lines represent gaps in literature and solid lines represent established findings. Pathways b and c reflect the test of two mechanisms through which a supervisor-led extension to SRT may have its effects.

Hypotheses for mental health outcome trajectories

Previous trials of SRT have demonstrated a minimal impact trajectory compared to a control condition that demonstrated progressively increasing symptoms across the onset and offset of the stressor period (Crane, Boga, et al., 2019). Engaging instructors in the supervisor-led extension (iSRT) during field operations was anticipated to facilitate a trajectory of robust resilience; whereby we would prevent or reduce the increase in symptoms during continuing stressors. By contrast, the SRT (control) condition was predicted to follow a pattern of minimal impact resilience as observed in previous trials. Therefore, we anticipated condition differences in trajectories at T2 and T3 as time points of continuing stressor exposure and intervention implementation.

H1.There will be (i) significantly lower rates of change in depression symptoms (a: T2→T4 y; b: T3→T4) in the iSRT condition, compared with the SRT condition. These differences in rates of change will be underpinned by (ii) significantly lower average depression symptoms in the iSRT condition, compared with the SRT condition, at a: T2 and b: T3.

H2.There will be (i) significantly lower rates of change in anxiety symptoms (a: T2→T4; b: T3→T4) in the iSRT condition, compared with the SRT condition. These differences in rates of change will be underpinned by (ii) significantly lower average anxiety symptoms in the iSRT condition, compared with the SRT condition, at a: T2 and b: T3.

H3.There will be (i) significantly lower rates of change in perceived stress (a: T2→T4; b: T3→T4) in the iSRT condition, compared with the SRT condition. These differences in rates of change will be underpinned by (ii) significantly lower average perceived stress in the iSRT condition, compared with the SRT condition, at a: T2 and b: T3.

Hypotheses for condition effects on performance outcomes

H4.The iSRT condition would demonstrate higher average performance scores on (a) merit points, (b) command, (c) foundation warfighting and (d) leadership and character than the SRT condition.

Hypotheses for mediation effects on outcomes

H5.Perceived instructor support for cadets would mediate the association between iSRT condition and the three outcomes: (H5) depression symptoms, (H6) anxiety symptoms and (H7) perceived stress levels at (a) T2 and (b) T3.

H8.Perceived instructor support for resilience training would mediate the association between iSRT condition and the three outcomes: (H8) depression symptoms, (H9) anxiety symptoms and (H10) perceived stress levels at (a) T2 and (b) T3.

H11.Perceived instructor support for cadets would mediate the association between iSRT condition and the four performance outcomes: (a) merit points, (b) command, (c) foundation warfighting and (d) leadership and character.

H12.Perceived instructor support for resilience training would mediate the association between iSRT condition and the four performance outcomes: (a) merit points, (b) command, (c) foundation warfighting and (d) leadership and character.

METHOD

Research context

The RMC in Australia is an Army officer training facility that prepares soldiers to become officers over a course consisting of three-six-month phases. This research was undertaken during the second phase of officer training, referred to as second-class. Second-class is a formative period whereby cadets are evaluated on their military skills and develop their leadership capacity (see Crane, Boga, et al., 2019 for details). Our intention was to capture psychological health across the onset and offset of training demands.

Cadets were divided into platoons of 35 or 36 cadets with four to five instructors assigned to each platoon. Second-class involved two major field operations conducted in bushland: a 17-day offensive operation and a 14-day defensive operation. During operations, platoons independently conducted three simulated warfighting activities (called serials) per day. Each serial had a mission goal and were designed to assess cadets' ability to lead a platoon of their peers. Following each serial instructors conducted an After Action Review with cadets. After Action Reviews are commonly used in the military context to reflect on the execution of military skills and drills (Ellis & Davidi, 2005). Instructors' consistent contact with cadets made field operations an ideal setting for a supervisor-led resilience training extension.

Participants

All cadets attending second-class at the RMC (N = 212) in January 2019 received the SRT introductory brief. The final sample consisted of 168 participants. There were no exclusion criteria. Demographic information and T1 mean scores on main outcomes are detailed in Table 1.

TABLE 1. Sample demographics by training condition

Variable	iSRT		SRT		t or X2	αues
	n	% or mean (SD)	n	% or mean (SD)
Cadet demographics and T1 variables
Gender
Male	67	80.72	67	78.82	1.25	.536
Female	15	18.07	18	21.18
Other	1	1.21
Age (in years)	83	22.77 (3.77)	85	23.51 (4.16)	1.20	.232
ADFA Graduate	42	50.60	39	45.88	.38	.540
Education level
Completed year 12 (high school)	7	8.75	11	13.25	2.89	.717
Some trade qualification completed	0	.00	1	1.21
Some university completed	13	16.25	10	12.05
Currently enrolled in university	4	5.00	6	7.23
Completed university degree	47	58.75	48	57.83
Completed trade qualification	9	11.25	7	8.43
Years of service	78	2.63 (2.42)	78	2.36 (1.77)	−.08	.429
T1 Depression (PHQ-2)	83	1.58 (2.01)	84	2.05 (1.95)	1.73	.085
T1 Anxiety (GAD-2)	83	2.06 (1.69)	85	2.11 (2.01)	.16	.874
T1 Perceived Stress	83	21.81 (7.31)	84	23.47 (7.41)	1.46	.148
Instructor demographics and T1 variables
Years of service	8	11.63 (5.07)	10	13.40 (6.82)	.61	.549
Years of instructor experience	8	1.13 (.35)	10	.67 (.64)	−1.93	.074
T2 Transformational Leadership	7	5.40 (.49)	6	5.88 (.34)	2.03	.067

• Abbreviations: ADFA, Australian Defence Force Academy; GAD-2, Generalized Anxiety Disorder 2-item questionnaire; iSRT, integrated Self-reflection Resilience Training; PHQ-2, Patient Health Questionnaire 2-item; SD, standard deviation; SRT, Self-reflection Resilience Training.

Prior to data collection, cadets were assigned to one of six platoons. Half the platoons were randomly assigned to the intervention (n = 83) or control condition (n = 85) by author MC. Assignment of condition by platoons is necessary in the military context to avoid cross-contamination of the intervention content (e.g., Adler, Bliese, et al., 2015; Adler, Williams, et al., 2015). Twenty-five cadet instructors (n_intervention = 13, n_control = 12) were responsible for the instruction and assessment of their respective platoon.

Design and procedure

The study was reviewed and approved by the Department of Defence and Veterans' Affairs Human Research Ethics Committee (protocol number: 086–18) and pre-registered with the Australian-New Zealand Clinical Trials Registry (registration number: ACTRN12618001702202p). The study was a clustered-randomized controlled trial whereby randomization occurred by platoons. Cadet surveys were conducted at four previously described stressor phases that coincided with particular training events. The T1 survey was completed after the SRT introductory briefing, at the second self-reflection journaling session received by all cadets, due to a delay with ethics approvals. This timing was not considered detrimental to the trial because both conditions received the same treatment up to this point. The T3 survey was completed after the fifth self-reflection journaling session received by cadets, marking the end of the SRT. To capture the effect of iSRT, surveys were conducted pre and post the two field operations that were the context for iSRT implementation. There were 6.5 weeks between T1 and T2; 3.5 weeks between T2 and T3 and 6.5 weeks between T3 and T4. Cadets' instructors were administered three surveys concurrent with cadet T1, T2 and T4 surveys. Cadet performance data were obtained at the completion of second-class.

As part of iSRT, intervention cadet instructors were taught the extension prior to the commencement of second-class in a briefing lead by author MC. The cadet instructor training covered: (1) an overview of the cadet SRT programme; (2) training in a protocol for guiding self-reflection with cadets following a stressor experience, referred to as a Mental Fitness After Action Review (Mental Fitness AAR); and (3) practical coaching skills (e.g., active listening, open-ended questions) with examples. Instructors were advised to conduct a Mental Fitness AAR after conducting the traditional skills and drills AAR that followed every serial and at other opportunities (e.g., performance management). Cadet instructors assigned to control condition platoons were not trained to conduct Mental Fitness AARs and only conducted the skills and drills AAR.

During field exercises, members of the research team and psychology masters students, observed the implementation of the iSRT and provided support to instructors in the initial phases of implementation. The control condition serials and skills and drills AARs were also observed to ensure that components of the intervention (e.g., reflecting on stress and coping experiences) were not spontaneously occurring.

Experimental conditions

Self-reflection training (SRT)

Cadets in the control condition received the SRT programme designed to teach adaptive self-reflection on experienced stressors and applied coping strategies (Crane, Boga, et al., 2019; Falon, Karin, et al., 2021). The SRT was conducted by an RMC psychologist experienced in training large groups of soldiers and who was involved in the previous trial of SRT (Falon, Karin, et al., 2021). The programme reinforced the five reflective practices described previously. As part of the SRT programme, cadets undertook a 40-minute lecture-style session followed by five 15-minute journaling sessions, spaced a week apart at minimum. The initial session introduced the idea of developing resilience via training stressors and how to use the self-reflective journals. Cadets were provided with a list of 32 values (e.g., integrity) and asked to rate these values in importance to ascertain their personally significant leadership values. Important values were referred to during the journaling sessions, as a benchmark for evaluating the effectiveness of coping strategies and resource utilization. The journaling sessions contained questions that guided cadets through the five self-reflective practices with reference to a stressor they experienced in the past week. During field operations cadets completed skills and drills AARs following serials, per usual RMC practices.

Instructor integrated self-reflection training (iSRT)

In the intervention condition, cadets received the SRT programme as described above and the iSRT programme. The iSRT provided cadets with instructor-guided opportunities to apply SRT content following serials. Similar to skills and drills AARs, Mental Fitness AARs consisted of a set of questions asked by instructors designed to encourage cadets to constructively reflect on their task and performance. The questions, however, modelled SRT practices and focused on identifying psychological stressors, reactions (e.g., cognitive, emotional) to these stressors, resilient capacities applied to manage psychological stress, an evaluation of resilient capacities, and what resilient capacities might be applied or changed in the future. The Mental Fitness AARs originally included eight questions and was revised to four questions in response to instructor feedback and observations made after the first field operation. As is necessary in applied work and given the dynamic nature of field experiments, instructors had the flexibility to decide the most appropriate format (i.e., one-on-one or group-based) and timing to conduct Mental Fitness AARs for their cadets. Mental Fitness AARs followed skills and drills AARs in the training schedule and therefore could be conducted up to three times a day for the duration of field operations (a total of 31 days). Mental Fitness AARS typically took between 10 minutes to 20 minutes depending on time available and the extent of discussion. Thus, the Mental Fitness AARs encouraged a regular dialogue between supervisors and subordinates over the course of their leader–follower relationship. However, as instructors were continuously in the company of cadets during field operations, the iSRT altered the interaction but did not extend the contact time between instructors and cadets.

Primary Outcomes1

Mental ill-health symptoms

The Patient Health Questionnaire 2-items (PHQ-2; Kroenke et al., 2003) and the Generalized Anxiety Disorder 2-items (GAD-2; Kroenke et al., 2007) measured depression and anxiety symptoms across all time points, respectively. Respondents reported the frequency of being bothered by particular problems over the last two weeks, using a 4-point scale from 0 (not at all) to 3 (nearly every day). The PHQ-2 demonstrated internal reliability at time points T1-T4 (r_{spearman-brown} = .70–81). The GAD-2 was internally reliable at T1-T3 (r_{spearman-brown} = .80–85) excluding T4 (r_{spearman-brown} = .64).

Perceived stress

The Perceived Stress Scale 10-items (PSS; Cohen et al., 1983) measured perceived stress across all time points. Respondents were asked to indicate the frequency of particular thoughts or feelings over the past two-weeks on a 5-point scale from 0 (never) to 4 (very often). Internal reliability was consistently achieved (α = .80–85).

Performance outcomes

All graduating second-class cadets received a performance score on four dimensions: (1) merit points, (2) command, (3) foundation warfighting and (4) leadership and character. Higher scores indicated greater performance. The merit point score ranged from 0 to 100 and included a selected set of weighted assessments that measured skills and knowledge in academic work, marksmanship and fitness. The remaining performance dimensions were aggregated scores of individually weighted behavioural assessments, initially evaluated by an instructor using a rubric that described the required compliance levels to achieve a set standard and moderated by other staff (e.g., the senior instructor) to ensure consistency in assessments across cadets. The command score ranged from 0 to 24 and assessed cadets' ability to lead their platoon during serials. The foundation warfighting score ranged from 0 to 4 and assessed cadets' ability to apply skills, knowledge and attitudes to effectively plan a tactical response during a serial. The leadership and character score ranged from 0 to 12 and assessed cadets' ability to demonstrate valued skills, knowledge and attitudes to effectively influence a team's performance.

Stressor frequency

Stressor frequency was assessed using a checklist designed for use in the RMC setting (Crane, Boga, et al., 2019; Falon, Karin, et al., 2021). Participants indicated how often 23 stressors occurred in the past week (e.g., a lot of academic work) on a 6-point scale from 0 (no days) to 5 (every day).

Mediator variables

Perceived instructor support for individual cadets was measured at each time point using an adapted version of the Perceived Organizational Support scale (Rhoades et al., 2001; see Supplementary A for modified scale). Three modifications were made to the scale: (1) ‘the organization’ was replaced with ‘my RMC instructor’ (2) two items were removed because of lack of relevance to the context, and (3) items were modified to fit the context (e.g., ‘My RMC instructors are trying to help me [to complete the course]’). Responses to 6 items were provided on a 6-point scale from 1 (strongly disagree) to 6 (strongly agree). Internal reliability was consistently demonstrated (α = .78–.88).

Perceived instructor support for resilience training was measured between T2-T4 using an 8-item scale developed for the study (see Supplementary A for scale). Participants reported their agreement with statements of instructor behaviour (e.g., ‘My instructor motivates me to think about new ways to cope with stress’) on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). Internal reliability was consistently demonstrated (α = .94–.96).

Transformational leadership as a control variable

We tested whether instructors differed in transformational leadership between conditions, given that transformational leadership has a demonstrated positive association with affective well-being and a negative association with burnout in subordinates (Skakon et al., 2010). Instructors completed the transformational leadership inventory at T2 (Podsakoff et al., 1996). The ‘Articulating a Vision’ subscale was excluded because feedback from the RMC leadership team indicated low relevance to the context. Transformational leadership behaviours were rated on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). Internal reliability was demonstrated (α = .72).

Observational ratings of instructor encouragement of self-reflection

Two authors (MK and MC) and two psychology masters students observed and rated (not blinded) the delivery of skills and drills AARs in the control condition and Mental Fitness AARs in the intervention condition (see Supplementary B for description of the ratings). A total of 22 observations were made evenly across conditions.

Analysis strategy

Statistical power

According to the DELTA2 guidelines (Cook et al., 2018), previously reported estimates from similar contexts and designs should be used to determine statistical parameters. The post-hoc power calculation reported in Crane, Boga, et al. (2019) was used as the basis of estimates regarding observed change over time, symptom score variance and within-subject correlation. This power analysis using the Longpower package in R (Donohue & Edland, 2016) indicated that the projected sample of 100 participants within each group was adequately powered to refute minimally detectable symptom difference between the groups that were as small as 20% of the total effect.

Addressing missing cases

Total missing data across all time points were 25.35%. Non-ignorable patterns of missing data on outcome variables (participants with missing data on at least one outcome variable; T2 = 16.07%. T3 = 22.62%, T4 = 36.90%) were examined by testing whether any variables at T1, including condition allocation, predicted missingness at later time points (Little & Rubin, 2014). Backward logistic regression models determined whether any T1 variables were predictors of missing data at later time points (Little, 1995; Karin et al., 2018). Consistent with previous studies (Falon, Karin, et al., 2021), entering the programme as an Australian Defence Force Academy (ADFA) enrollee significantly predicted missing cases at all assessment points, forming a conditional missing at random assumption. Adjusted model-based estimates were generated for each outcome based on a General Estimation Equation (GEE) including ADFA enrolment, platoon (to account for clustered data), condition, time, condition*time, ADFA*time, platoon*time and the T1 equivalent of the outcome to account for the variance in intercepts (Karin et al., 2018). To determine whether missing data replacement changed the pattern of findings, we re-ran our change analyses using un-replaced data (Supplementary C, Tables S3 and S4). The pattern of findings was consistent with the models using replaced data.

Main analyses

GEE models tested the longitudinal changes between conditions whilst accounting for within-subject variance over time (Liang & Zeger, 1986). Other studies have found that symptom outcomes can increase and decrease with the onset and offset of a stressor between sampling times. Consistent with previous work (Falon, Karin, et al., 2021), GEE methodology was employed with a categorical Time variable and an unstructured working correlation matrix to account for the within-subject correlation. This method is also able to account for additional level nesting (clustering) in the data (e.g., platoon) as a source of non-independence of data (given that the intervention condition allocation was randomized by platoon) through fixed covariates that stratify the main estimates of change. Each GEE model included the effect of condition, time, platoon membership, baseline symptoms, ADFA enrolment and the two-way interaction with time for condition, platoon membership, and ADFA enrolment. The inclusion of these additional covariates and time*covariate interactions aimed to stratify the mean rate of change over time, above any influence associated with the random sampling of individuals from ADFA or platoons. Effect sizes were reported using Hedge's g for between-condition and within-condition effects (Hedges, 1981). Similarly, a GEE model that included ADFA enrolment and platoon membership was used to explore the effect of condition on the four performance outcomes.

Testing for mechanisms of symptom change

Perceived instructor support for cadets and instructor support of the resilience training were used as mediators in the association between condition and outcomes via Hayes' bootstrapping approach using the PROCESS macro for SPSS (model 4; Hayes, 2013). Tests of indirect effects were conducted using 1000 bootstrap samples and 95% bias-corrected confidence intervals. Given that the intervention was hypothesized to produce time-dependent between-condition effects and therefore did not meet the assumption of sequential ignorability required for longitudinal mediations (McArdle, 2009; Preacher, 2015), we conducted analyses within specific time points of interest. Models, however, controlled for outcome and mediator variables at prior time points, excluding T1 and ADFA enrolment. Platoon membership was not included as its inclusion produced model overfit and intra-class correlations indicated that the random intercept of platoon explained little variance in outcomes (Table 2).

TABLE 2. Intra-class correlation of platoon membership across outcome variables

Outcome variables	Intra-class correlation
Patient Health Questionnaire 2	.67%
Generalized Anxiety Disorder Scale 2	.36%
Perceived Stress	1.05%
Merit Points	<.01%
Foundation Warfighting	2.03%
Command	<.01%
Leadership and Character	<.01%

Protocol deviations

We utilized the ultra-brief versions of the Patient Health Questionnaire and Generalized Anxiety Disorder scale, the PHQ-2 (Kroenke et al., 2003) and GAD-2 (Kroenke et al., 2007), rather than the PHQ-9 and GAD-7 as specified in the protocol. After the current trials measures were selected, the findings of the previous trial at the RMC (Falon, Karin, et al., 2021) challenged the utility of the PHQ-9 and GAD-7 in this context. Items assessing changes in sleeping, recreational and dietary habits, may be attributed to the officer training context, rather than symptoms of ill-mental health. These measures assess the possible presence of anxiety or depression as opposed to severity (Kroenke et al., 2010).

All main analyses were conducted using SPSS Version 25 and all results were interpreted with the thresholds of α = .05 and power .80. The results were also re-examined through three alternative scenarios that reflect trade-offs between Type I and Type II error: balanced, conservative and liberal. Calculations of alpha for balanced Type I and II error were made using algorithms specified by Laken and Caldwell (2021). Conservative values were based on the application of a Bonferroni family-wise correction to adjust for multiple models of outcomes that may share measurement error. The liberal p-value was based on previous work applying a more liberal p-value, when comparing a resilience intervention to an active control in a military context (Adler et al., 2008; Adler, Bliese, et al., 2015; Adler, Williams, et al., 2015). These viewpoints are offered considering the current study provides an ecological test of superiority effects against an active control, with demonstrated effectiveness, in a limited sample size (N < 200).

RESULTS

Preliminary analyses

Figure 3 presents the flow of participants through the trial. There were no significant differences between conditions on T1 variables or T2 average instructor self-rated transformational leadership (see Table 1). Compared with control condition instructors, the intervention condition instructors were observed to engage in greater usage of practical coaching skills, M_c = .67, SD = .49, M_I = 2.10, SD = .70, t(21) = −5.68, p < .001; encouragement of self-reflection on stressors, M_c = .17, SD = .39, M_I = 1.71, SD = .70, t(21) = −6.61, p < .001; and adherence to iSRT objectives, M_c = 1.77, SD = .94, M_I = 5.39, SD = 1.60, t(20) = −6.47, p < .001.

FIGURE 3

CONSORT flow diagram for SRT only and SRT + iSRT programmes and data collection. SRT = Self-reflection Resilience Training; iSRT = integrated Self-reflection Resilience Training.

There were no differences between conditions on perceived stressor frequency at T1. Average perceived stressor frequency was observed to change across time (F = 141.19, p < .001). Unexpectedly, participants reported the greatest frequency of stressors at T1 (M_T1 = 61.43, SE_T1 = 1.43) with significant declines observed between T1 and all other time points (M_T2 = 55.93, SE_T2 = 1.78; p = .001; M_T3 = 42.95, SE_T3 = 1.76; p < .001; M_T4 = 29.62, SE_T4 = 1.71; p < .001).

Analysis of primary outcomes

A summary of all supported and unsupported hypotheses are presented in Tables 4 and 5. Supplementary D provides a summary of a sensitivity analysis undertaken to re-examine the condition by time effects through a balanced, conservative and liberal viewpoints of Type I and Type II error assumptions. The pattern of test significance and insignificance of the main analyses were mostly consistent under the different scenarios (viewpoints) that prioritize effect sensitivity (reduction of Type II error) and specificity (reduction of Type I error), suggesting a robust pattern of results. Notably, T2 between group differences for the different outcomes remained consistent across most viewpoints (e.g., consistent pattern of significant and non-significant findings), except for the conservative alpha correction, that is overly restrictive in the context of a moderately sampled (<200) superiority trial with a non-clinical sample. Descriptive and effect size statistics for psychological health outcomes are included in Table 3.

TABLE 3. Adjusted means, standard errors and average within-condition and between-condition change in depression symptoms, anxiety symptoms and perceived stress for SRT (control) and iSRT (intervention) conditions

Primary outcomes	Means (standard error) for each time point				Percentage change [95% CI]			Within-condition effect sizes (95% confidence interval)			Between-condition effect sizes (95% confidence interval)
	T1 pre-first field op.	T2 post-first field op.	T3 pre-second field op.	T4 post-second field op.	T1 to T2	T2 to T3	T3 to T4	T1 to T2	T2 to T3	T3 to T4	T1	T2	T3	T4
PHQ-2
SRT	2.24 (.28)	2.57 (.29)	1.74 (.25)	1.08 (.20)	15% [−21%, 50%]	−32% [−62%, −3%]	−38% [−75%, −1%]	.13 [.12, .13]	−.33 [−.34, −.32]	−.31 [−.32, −.30]	−.33 [−.33, −.32]	−.52 [−.52, −.51]	−.14 [−.15, −.13]	.22 [.22, .23]
iSRT	1.43 (.26)	1.33 (.23)	1.42 (.23)	1.52 (.22)	−7% [−55%, 41%]	7% [−41%, 56%]	7% [−38%, 51%]	−.04 [−.05, −.04]	.04 [.04, .05]	.05 [.04, .05]
GAD-2
SRT	2.28 (.30)	1.92 (.29)	2.20 (.29)	.94 (.17)	−16% [−52%, 21%]	14% [−28%, 56%]	−57% [−87%, −27%]	−.13 [−.14, −.12]	.10 [.09, .11]	−.57 [−.57, −.56]	−.13 [.15, −.14]	−.08 [−.09, −.07]	−.34 [−.34, −.33]	.09 [.08–.09]
iSRT	1.94 (.25)	1.71 (.27)	1.39 (.23)	1.08 (.16)	−12% [−50%, 26%]	−19% [−59%, 22%]	−22% [−62%, 17%]	−.10 [−.10, −.09]	−.14 [−.14, −.13]	−.17 [−.18, −.16]
Perceived stress
SRT	24.11 (1.11)	24.16 (1.30)	23.77 (1.15)	19.37 (1.13)	<1% [−9%, 9%]	−2% [−11%, 7%]	−19% [−28%, −9%]	<.01 [> −.01, .01]	−.03 [−.04, −.03]	−.41 [−.42, −.41]	−.28 [−.29, −.28]	−.41 [−.42, −.40]	−.25 [−.26, −.24]	−.09 [−.10, −.09]
iSRT	21.15 (1.10)	19.24 (1.29)	21.07 (1.18)	18.39 (1.15)	−9% [−20%, 2%]	10% [−5%, 24%]	−13% [−24%, −1%]	−.17 [−.18, −.17]	.16 [.15, .17]	−.25 [−.26, −.24]

• Note: Means, standard errors and confidence intervals adjusted for fixed effects of time, condition, ADFA membership and platoon membership. Bolded text indicates significant within-condition or between-condition differences.
• Abbreviations: ADFA, Australian Defence Force Graduate; GAD-2, Generalized Anxiety Disorder 2-item questionnaire; iSRT, integrated Self-reflection Resilience Training; PHQ-2, Patient Health Questionnaire 2-item; SRT, Self-Reflection Resilience Training.
• GEE models for quantifying change over time specified Yij = β₀ + β₁ Time_j + β₂ Platoon_k + β₂ Platoon_k* β₁ Time_j + β₃ADFA + β₃ ADFA*β₁ Time_j + β₄ Baseline symptoms + εij; with m_i × m_i working correlation matrix for each Yij, Var(Yij) = Φv(μij) y; where Φ is a scale parameter and v(·) is a normal distribution; εij ~ N (0, σ2); i is the individual cadet, clustered within j time points (T1, T2, T3.T4), k platoons (six in total); Australian defence force academy (ADFA) enrolment (Yes/No).

TABLE 4. Summary of supported and unsupported hypotheses in relation to mental health outcomes

	Condition effect
	(a) Time 2		(b) Time 3
	i. Between-condition trajectory differences (T2-T4)	ii. Between-condition mean differences	i. Between-condition trajectory differences (T3-T4)	ii. Between-condition mean differences
Hypothesis 1 PHQ-2	✓	✓	✗	✗
Hypothesis 2 GAD-2	✓	✗	✓	✗
Hypothesis 3 Perceived Stress	✓	✓	✗	✗

	Mediation of Instructor Support for Cadets
	(a) Time 2	(b) Time 3
Hypothesis 5 PHQ-2	✓	✗
Hypothesis 6 GAD-2	✓	✗
Hypothesis 7 Perceived Stress	✓	✗

	Mediation of Instructor Support for Resilience Training
	(a) Time 2	(b) Time 3
Hypothesis 8 PHQ-2	✗	✗
Hypothesis 9 GAD-2	✗	✗
Hypothesis 10 Perceived Stress	✗	✗

• Note: ✓ = supported hypothesis, ✗ = unsupported hypothesis.
• Abbreviations: GAD-2, Generalized Anxiety Disorder 2-item questionnaire; PHQ-2, Patient Health Questionnaire 2-item.

TABLE 5. Summary of supported and unsupported hypotheses in relation to performance metrics

Hypothesis	Performance metric
	(a) Merit points	(b) Command	(c) Foundation warfighting	(d) Leadership and character
	Condition Effect
Hypothesis 4	✗	✗	✓	✗
	Mediation of Instructor Support for Cadets
Hypothesis 11	✗	✗	✗	✗
	Mediation of Instructor Support for Resilience Training
Hypothesis 12	✗	✗	✗	✗

• Note: ✓ = supported hypothesis, ✗ = unsupported hypothesis.

TABLE 6. Adjusted means, standard errors and confidence intervals of performance metrics scores by training condition

Outcome	iSRT		SRT		Wald χ^2	p
	M (SE)	CI 95%	M (SE)	CI 95%
Merit Points	65.40 (1.78)	[61.92, 68.88]	60.02 (1.49)	[57.10, 62.94]	3.71	.054
Command	5.73 (.20)	[5.35, 6.12]	5.23 (.19)	[4.86, 5.61]	2.44	.118
Foundation Warfighting	1.65 (.03)	[1.60, 1.70]	1.54 (.03)	[1.48, 1.59]	5.48	.019*
Leadership and Character	2.24 (.13)	[1.88, 2.45]	2.18 (.14)	[1.91, 2.45]	.08	.785

• Abbreviations: iSRT, integrated Self-Reflection Resilience Training; SRT, Self-reflection Resilience Training.
• * p < .05.

TABLE 7. Multiple mediator models with instructor support for cadet as the process variable and depression, anxiety and perceived stress as the outcome

Outcome	Time 2 mediator and outcome		Time 3 mediator and outcome
	b	95% CI [LI:HI]	b	95% CI [LI:HI]
Depression (PHQ-2)
Total effects (c)	−.79**	−1.31: −.27	−.14	−.55: .28
Direct effects (c’)	−.64*	−1.15: −.13	−.18	−.60: .23
Total indirect effects	−.16	−.36: −.01	.05	−.02: .14
X → M	.31*	.02: .59	−.12	−.29: .06
M → Y	−.51**	−.78: −.24	−.39*	−.76: −.03
Anxiety (GAD-2)
Total effects (c)	−.39	−.93: .15	−.35	−.72: .03
Direct effects (c’)	−.27	−.82: .26	−.39*	−.76: −.01
Total indirect effects	−.11	−.28: > −.01	.04	−.04: .13
X → M	.31*	.02: .59	−.09	−.26: .08
M → Y	−.36*	−.65: −.08	−.44*	−.77: −.10
Perceived stress
Total effects (c)	−2.49*	−4.92: −.06	−.60	−2.25: 1.05
Direct effects (c’)	−1.52	−3.81: .77	−.73	−2.37: .92
Total indirect effects	−.97	−1.94: −.08	.13	−.09: .47
X → M	.31*	.02: .59	−.11	−.27: .06
M → Y	−3.18**	−4.40: −1.96	−1.22	−2.72: .28

• Abbreviations: GAD-2, Generalized Anxiety Disorder 2-item questionnaire; PHQ-2, Patient Health Questionnaire 2-item.
• *p < .05, **p < .01.

TABLE 8. Multiple mediator models with perceived instructor support the resilience training as the process variable and depression, anxiety and perceived stress as the outcome

Outcome	Time 2 mediator and outcome		Time 3 mediator and outcome
	b	95% CI [LI:HI]	b	95% CI [LI:HI]
Depression (PHQ-2)
Total effects (c)	−.67*	−1.23: −.11	−.15	−.60: .31
Direct effects (c’)	−.55*	−1.09: −.01	−.08	−.52: .36
Total indirect effects	−.12	−.35: .04	−.07	−.24: .03
X → M	.30	−.13: .73	.18	−.10: .47
M → Y	−.40**	−.61: −.20	−.38**	−.63: −.12
Anxiety (GAD-2)
Total effects (c)	−.29	−.88: .30	−.40	−.81: .01
Direct effects (c’)	−.20	−.79: .38	−.33	−.74: .07
Total indirect effects	−.08	−.26: .03	−.07	−.23: .01
X → M	.30	−.13: .73	.23	−.05: .52
M → Y	−.28*	−.50: −.06	−.28*	−.52: −.05
Perceived stress
Total effects (c)	−1.96	−4.58: .67	−.37	−2.19: 1.46
Direct effects (c)	−1.33	−3.82: 1.16	−.32	−2.16: 1.52
Total indirect effects	−.63	−1.65: .23	−.05	−.37: .21
X → M	.30	−.13: .73	.21	−.07: .50
M → Y	−2.09**	−3.05: −1.14	−.21	−1.29: .87

• Abbreviations: GAD-2, Generalized Anxiety Disorder 2-item questionnaire; PHQ-2, Patient Health Questionnaire 2-item; SRT, Self-reflection Resilience Training.
• *p < .05, **p < .01.

TABLE 9. Multiple mediator models with perceived instructor support for resilience training or perceived instructor support for cadet as the process variable and performance metrics (merit points, command, foundation warfighting and leadership and character) as outcomes

Mediator	Merit points		Command		Foundation warfighting		Leadership and character
	b	95% CI [LI:HI]	b	95% CI [LI:HI]	b	95% CI [LI:HI]	b	95% CI [LI:HI]
Instructor support for resilience training model
Total effects (c)	3.72*	.49: 6.95	.42	−.01: .85	.04	−.02: .10	−.03	−.29: .24
Direct effects (c’)	3.46*	.23: 6.69	.40	−.03: .83	.04	−.02: .10	−.06	−.32: .21
Total indirect effects	.26	−.19: 1.08	.02	−.02: .10	>−.01	−.01: .01	.03	−.02: .12
X → M	.19	−.16: .54	.19	−.15: .54	.19	−.16: .54	.19	−.16: .54
M → Y	1.36	−.41: 3.12	.09	−.14: .32	.02	−.02: .05	.16*	.02: .31
Instructor support for cadet model
Total effects (c)	3.94*	.71: 7.17	.42	−.01: .86	.04	−.02: .10	−.08	−.35: .19
Direct effects (c’)	3.93*	.68: 7.18	.41	−.02: .85	.04	−.02: .10	−.09	−.36: .18
Total indirect effects	.01	−.30: .42	.01	−.03: .11	<.01	−.01: .01	.01	−.04: .07
X → M	.13	−.37: .62	.13	−.37: .62	.13	−.37: .62	.13	−.37: .62
M → Y	.04	−1.27: 1.34	.10	−.08: .27	> −.01	−.02: .10	.08	−.03: .19

• * p < .05.

Depression (PHQ-2)

Aligned with H1a.ii, the intervention condition had significantly lower average depression symptoms compared with the control condition at T2 (p = .006, g = −.52). Contrary to H1.b.ii, there were no between-condition differences at T3. Aligned with H1.a.i and H1.b.i, parameter estimates revealed that the intervention condition demonstrated greater stability in symptoms than the control condition. Despite stressor onset, all within-subject comparisons between conditions indicated that the intervention condition displayed significantly less change over time than the control condition (T1-T4: b = 1.25, Wald χ2 = 12.07; p = .001; T2-T4: b = 1.69, Wald χ2 = 17.81; p < .001; T3-T4: b = .76, Wald χ2 = 4.55; p = .033). There were no changes across time in average depression symptoms in the intervention condition. In contrast, the control condition demonstrated an initial non-significant average increase in depression symptoms between T1 and T2 followed by significant average declines between T2 and T3 (p = .001, g = −.33) and T3 and T4 (p = .001, g = −.31). These declines accumulated such that T4 average depression symptoms were lower than T1 for the control condition (p < .001, g = −.51). However, both conditions demonstrated comparable average depression symptoms at T4. In summary (Figure 4), as cadets responded to stressors the intervention condition appeared to resist an increase in depression symptoms, resulting in significantly lower depression symptoms than the control condition at T2. Thus, the trajectories of the intervention and control conditions were akin to robust and minimal impact resilience, respectively.

FIGURE 4

Estimated marginal means pre-first field operation, post-first field operation, pre-first field operation and post-field operation for depression (PHQ-2) for SRT only (control) and iSRT and SRT (intervention). Brackets represent standard errors and where overlapping indicate a non-significant difference between conditions' estimated marginal means. SRT = Self-reflection Resilience Training; iSRT = integrated Self-reflection Resilience Training; PHQ-2 = Patient Health Questionnaire 2-item.

Anxiety (GAD-2)

Inconsistent with H2.a.ii and H2.b.ii, between-condition comparisons demonstrated no average difference in anxiety symptoms at T2 or T3. In testing H2.a.i and H2.b.i, parameter estimates indicated that the rate of change in anxiety symptoms between T3 and T4, and not T2 and T4, was statistically different between conditions. This was such that the intervention condition was more stable than the control condition (b = .94, Wald χ2 = 5.61; p = .018). Observing within-condition changes, both the intervention and control conditions appear to decrease in symptoms over the stressor period. Within the intervention condition, T4 anxiety symptoms were significantly lower than at T1 (p < .001, g = −.44) and T2 (p = .002, g = −.30). The control condition also demonstrated average lower T4 symptoms compared to T1 (p < .001, g = −.58), T2 (p < .001, g = −.43) and T3 (p < .001, g = −.57). Figure 5 demonstrates that both conditions appear to improve over time on average anxiety symptoms, regardless of the stressor period, with the intervention condition experiencing more stable decline over time. Therefore, there was no clear support for differences in trajectories that demonstrated an added benefit of iSRT.

FIGURE 5

Estimated marginal means pre-first field operation, post-first field operation, pre-first field operation and post-field operation for anxiety (GAD-2) for SRT only (control) and iSRT and SRT (intervention). Brackets represent standard errors and where overlapping indicate a non-significant difference between conditions' estimated marginal means. SRT = Self-reflection Resilience Training; iSRT = integrated Self-reflection Resilience Training; GAD-2 = Generalized Anxiety Disorder 2-item questionnaire.

Perceived stress

Aligned with H3.a.ii, a between-condition difference in perceived stress levels emerged at T2 with the intervention condition demonstrating lower scores than the control condition at T2 (p = .031, g = −.41). However, inconsistent with H3.b.ii there was no between-condition difference in average perceived stress levels at T3. As anticipated (H3.a.i), parameter estimates indicated a difference in slopes between conditions between T2 and T4 (b = 3.94, Wald χ² = 3.99, p = .046) such that the intervention condition experienced more stability over time than the control condition. Having noted this, inconsistent with prediction (H3.b.i) the difference in slopes was not present between T3 and T4. Within-subjects pairwise comparisons indicated that the intervention condition demonstrated significant declines in perceived stress at points: T1 to T2 (p = .038, g = −.17), T3 to T4 (p = .006, g = −.25) and overall between T1 and T4 (p = .002, g = −.27), with a non-significant increase between T2 and T3. A decline in average stress was also observed across time for the control condition for T3 to T4 (p < .001, g = −.41), which underpinned a statistically significant decrease from T1 to T4 (p < .001, g = −.45) and resulted in no significant differences between-condition at T4. In summary (Figure 6), the trajectory of the control condition perceived stress levels was generally higher with a steep decline at the offset of stressors, consistent with minimal impact resilience. In contrast, the intervention condition demonstrated perceived stress levels that were generally lower than the control condition and significantly lower at the onset of stressors.

FIGURE 6

Estimated marginal means pre-first field operation, post-first field operation, pre-first field operation and post-field operation for perceived stress for SRT only (control) and iSRT and SRT (intervention). Brackets represent standard errors and where overlapping indicate a non-significant difference between conditions' estimated marginal means. SRT = Self-reflection Resilience Training; iSRT = integrated Self-reflection Resilience Training.

Performance

Descriptive and inferential statistics for performance scores are presented in Table 6. Aligned with H4.c, GEEs revealed a significant condition effect for foundation warfighting (Wald χ2 = 5.48; p = .019) indicating that the intervention condition outperformed the control condition (M_I = 1.65, SD = .03, M_C = 1.54, SD = .03, g = .53). There were no significant differences between conditions on the other three performance metrics, refuting H4.a, H4.b, and H4.d.

Mediation analysis

Instructor support for the cadet

Results from mediation models are presented in Table 7. Indirect effects were explored between T2 perceived instructor support for the cadet and T2 mental health outcomes (Figure 7a) where condition was the predictor variable. Condition was positively associated with T2 instructor support for the cadet, and T2 instructor support for the cadet was negatively related to all three T2 outcomes. Consistent with H5(a), H6(a) and H7(a), significant indirect effects were observed for all outcomes (depression b = −.16, 95% CI [−.36: −.01]; anxiety b = −.11, 95% CI [−.28: > − .01]; perceived stress b = −.97, 95% CI [−1.94: −.08]). The significant direct effect of condition on T2 depression symptoms, indicated that the T2 instructor support for the cadet partially mediated the intervention effect. Similar mediation paths were also explored for T3 instructor support for the cadet outcomes (Figure 7b). These models supported the negative association between instructor support for the cadet on all three outcomes at that corresponding time point, yet refuted H5(b), H6(b) and H7(b) as the effect of condition on T3 outcomes or indirect effects were inconsequential. Inconsistent with H11, perceived instructor support for the cadet did not mediate condition effects on any performance score (Table 9).

FIGURE 7

Exemplar mediation models exploring the change in outcome variables as a result of change in mediator variables derived at the same time point across two time points (a) time 2 and (b) time 3. ADFA = Australia Defence Force Academy.

Instructor support of the resilience training

Inconsistent with H8, H9 and H10, condition did not predict perceived instructor support of the resilience training at T2 or T3 (Table 8). T2 perceived instructor support of the resilience training was negatively related to all three T2 outcomes. Furthermore, against H12, perceived instructor support of the resilience training did not mediate condition effects on any performance score (Table 9).

DISCUSSION

We extended Self-Reflection Resilience training by innovatively modifying traditional After Action Reviews, to facilitate reflection on coping with psychological work demands. In doing so, instructors were enabled to integrate resilience skill development activities into the training of job-related skills. This superiority trial tested the added benefit of instructors leading coping focused After Action Reviews in situ. It was anticipated that instructors leading this adapted After Action Review would support a robust resilience trajectory with the potential for improving performance.

Summary of findings and theoretical implications

Evidence of distinct trajectories of resilience

Condition comparisons of mean scores and rates of change over time identified distinct trajectories for depression symptoms and perceived stress levels across the stressor period. Cadets in the SRT condition demonstrated a trajectory consistent with minimal impact resilience, aligned with previous research (Crane, Boga, et al., 2019; Falon, Karin, et al., 2021) and cadets in the iSRT condition appeared to demonstrate a robust resilience trajectory. The distinction in trajectories on these outcomes appeared localized to T2. It was predicted that differences in average scores would be observed as long as officer training stressors persisted. However, the SRT condition demonstrated a decline in symptoms at T3, prior to the cessation of stressors.

Inconsistent with predictions, trajectories of anxiety symptoms appeared more alike than disparate between conditions. The distinctiveness of anxiety trajectories has been observed in previous work, when comparing SRT against an active control (i.e., coping skills training, Falon, Karin, et al., 2021). Falon, Karin, et al. (2021) suggested that cognitive symptoms of anxiety may respond in a unique way to the self-reflection process. For example, the reflective process may draw attention to discrepancies between actions and expectations in this high-performance setting, triggering greater anxious thoughts. Conversely, those who perceive less discrepancy may feel affirmed in their capabilities and in turn less anxiety. Thus, individual or cohort differences in these discrepancies may account for differences in the way anxiety symptoms respond to self-reflection. Anxiety symptoms declined across the trial period in both conditions, potentially reflecting a perceived correspondence between actions and expectations highlighted by the self-reflection process for this cohort.

Effect on performance outcomes

The iSRT condition cadets achieved significantly higher foundation warfighting performance scores, compared with the control condition. A condition effect was not observed for any of the other three performance outcomes. A potential explanation for condition effects on this specific outcome is that the Mental Fitness AARs and foundation warfighting shared the same application and assessment context. Distinct from the self-reflective journals in SRT where cadets are encouraged to reflect on different personal stressors, Mental Fitness AARs occurred following post-warfighting scenarios where they were required to reflect on the psychological demands and coping in that scenario. The congruency between the context of Mental Fitness AARs and assessment for foundation warfighting may have led to stronger training transfer in this specific environment (Salas & Cannon-Bowers, 2001). This is supported by the theory of identical elements that asserts that the degree of transfer is dependent on the extent the training environment reflects the performance environment (Thorndike & Woodworth, 1901). Expanding supervisor-led application of self-reflection on coping to alternative performance contexts (e.g., in an academic environment) may be required to produce performance effects in other domains (e.g., merit points). Additionally, the single performance effect may relate to the high demands imposed by the war fighting scenarios. In these scenarios, cadets are responsible for leading their peers and are being directly assessed on their leadership performance both contributing to enhanced performance pressure. The resilience training application may therefore have been more impactful under such high-stress conditions. Based on Attention Control Theory (Eysenck et al., 2007), the observed effects for foundation warfighting performance may be interpreted as indicating that the iSRT enhanced the coping strategies and resources available to cadets in this specific performance setting allowing them to cope better with demands that may otherwise distract cognitive attention from task completion. Previous work has demonstrated that the appropriate application of coping strategies to work stressors is associated with improved performance in a military setting (de Souza & Feitosa, 2015).

Mediating role of perceived instructor support for cadets

Organizational training and resilience scholarship both imply that engaging supervisory leadership in guiding the practice of resilience training skills in situ would improve outcomes, however, suggest different mechanisms of effect. Investigating and distinguishing between these mechanisms informs our understanding of how organizational training and resilience scholarship intersect, in an organizational resilience training context.

Consistent with predictions (H5a, H6a and H7a) based on COR (Hobfoll, 2011), there was evidence that the supervisor-led extension, in part, had its effects on the mental health outcomes via perceived instructor support for the individual. This corresponds with work demonstrating the importance of supervisor support in promoting well-being in military personnel (e.g., Britt et al., 2004; Pflanz & Ogle, 2006). Inconsistent with H5b, H6b and H7b, these indirect effects only emerged at T2 where greatest between-condition differences were observed and not at T3, reflecting the early effects of instructor support driving the condition differences in mental health outcome trajectories.

The between-condition effects confined to T2, may provide insight into how robust versus minimal impact resilience are promoted through resourcing and training, respectively. The trajectory of minimal impact resilience in control condition cadets may be underpinned by the need to learn and apply skills taught and subsequently develop ones' capacities for resilience. Trajectories of development often do not follow linear growth (Day & Sin, 2011) and take time to solidify newly acquired resilience-supportive skills. This is perhaps more so the case in demanding application contexts (Flinn et al., 2016). In the intervention (iSRT) condition, instructor support for cadets may provide an immediate resource, independent of skill acquisition rate, thereby supporting resilience to the initial deleterious effects of early stressors at T2.

A direct condition effect remained for depression symptoms, despite the significant indirect effect via perceived instructor support of the individual, suggesting that other mechanisms may be involved in the relationship between condition and depression. A possible mediator is the emergence of social support amongst cadets, as proposed by the theoretical model for psychological gains from adversity (Mancini, 2019). Whilst the iSRT was instructor-led, instructors most often conducted Mental Fitness AARs in small groups (e.g., platoons). According to the model for psychological gains from adversity, when adverse events afflict groups of people their automatic response is to seek connection with others. An environment that supports reciprocation of affiliative behaviours is one that is more likely to result in pro-social environments that supports positive psychological outcomes. The demands of officer training may have inclined cadets to be more receptive to meaningful social interactions. The instructors who led group conversations about stress and coping may have provided timely reinforcement of cadets' affiliative behaviours, and thus enhanced social support amongst cadets. Social support is negatively associated with depression symptoms and according to the interpersonal emotion regulation model (Marroquín, 2011) prevents maladaptive attributions and appraisals that underpin depression (Beck, 1967). Future research may explore how Mental Fitness AARs conducted in one-on-one versus group settings compare in efficacy and inducing social support amongst peers.

Inconsistent with H3a, the extent instructors were perceived to support resilience training practices did not mediate outcomes. Training condition was not related to perceived instructor encouragement of the resilience training and is inconsistent with the training scholarship suggesting that supervisor behaviours, such as reinforcing and supporting the use of training, is associated with perceived supervisor support for training (Ghosh et al., 2015; Govaerts & Dochy, 2014). A methodological reason for this finding may be the lack of the scale sensitivity to instructor support for resilience training in the iSRT condition. Given that platoons in both conditions were receiving some form of intervention and instructors were aware of the SRT programme, instructors in both conditions may have indicated their support for resilience training. Alternatively, the supervisor behaviours that indicate support for the training are perhaps unclear as research has yet to establish the most effective behaviours for promoting perceived supervisor support for training (Govaerts & Dochy, 2014). Having noted this, greater perceived instructor support for resilience training was associated with a reduction in mental ill-health outcomes and perceived stress. Therefore, in principle greater perceived supervisor support for resilience training may be beneficial, but in this study, there was no evidence it was driven by differences in conditions. Further work is required before discounting supervisor support of the resilience training as a mechanism of effect for a supervisor-led extension.

Inconsistent with H11 and H12, there was no mediation between condition and any performance measure via instructor support for the individual or for the resilience training. This suggests the involvement of other mechanisms distinct to those that are involved in supporting mental health outcomes. For example, cadets in the iSRT received more frequent opportunities to apply self-reflection on coping post-warfighting scenarios resulting in greater training transfer and in turn enhanced performance outcomes in this context (Salas & Cannon-Bowers, 2001). Future work should directly consider the mediating role of frequency of implementing coping self-reflection in the workplace environment (training transfer) in the relationship between supervisor encouragement of training and performance outcomes.

Applied implications and considerations

Resilience training programmes should be provided to employees prior to a stressor period to prevent the deleterious effects of stressors (Chmitorz et al., 2018). Supervisors, who are not mental health subject matter experts, can feasibly support their subordinates' in strengthening their capacities for resilience during the proceeding stressor period by facilitating conversations about stress and coping in an extension to resilience training. For example, health-oriented leadership (i.e., supervisor health-related behaviour and attitudes; Franke & Felfe, 2011) has a demonstrated potential to support trajectories of health during stressor periods (Arnold & Rigotti, 2021). The distinction between the mechanisms of supervisor support for the individual versus supervisor support of the resilience training, suggests that supervisor engagement with workplace initiatives may be effective in promoting mental health during demanding periods, regardless of the approach, as long as supervisors' engagement increases perceptions of supervisory support for the individual. Enhanced perceptions of supervisor support in the immediate interim of a stressor period, may safeguard individual functioning whilst the effects of resilience training eventuate.

This research tests a simple and scalable protocol for debriefing with a view to enhance resilience skills taught via training. Traditional post-event debriefing methods allow individuals and teams to review job skill application to advise areas of development that will improve performance (Salas et al., 2018). Distinctly, Mental Fitness AARs were designed as a method of post-event debriefing that focused on coping with psychological work demands to stimulate insights that refined stressor management. Furthermore, the Mental Fitness AARs encouraged a coaching-styled dialogue between instructors and cadets. Instructors were trained in coaching techniques (e.g., asking open-ended questions and reflecting back with meaning) to ask specific questions about stress and coping (e.g., ‘What personally did you find difficult about this activity? Why?’). This coaching styled conversation about stress and coping, aligned well with features of supportive supervision, particularly, showing interest in subordinate well-being and performance. It may be the case that other resilience training approaches could be extended by a supervisor-led conversation about stress and coping that promotes perceived supervisor support. This is distinct from imparting coping skills (e.g., controlled breathing) that may be perceived as more directive and less akin to a personalized coaching conversation that aligns with supervisor support for the individual.

Strengths, limitations and future directions

Despite rigorous methodology, the study's limitations should be considered in the interpretation of findings and in conducting future work. First, cadet instructors were given a degree of flexibility when conducting Mental Fitness AARs, including the format (one-on-one or group based) and the timing of their conduct. This autonomy provides an ecologically sound test of the intervention and was essential for the feasible implementation of the programme; however, introduced variation to Mental Fitness AARs implementation. A more standardized implementation strategy may potentially induce larger effects than those observed. Second, although intervention instructors were free to conduct Mental Fitness AARs in alternative environments (e.g., barracks), they were not provided implementation instructions for these contexts. As discussed, this may have limited the interventions' effect across domains of performance and future implementations should consider its application to a broader range of contexts. Third, this assessment was undertaken in a unique context. Although the relationship between instructors and cadets may be likened to supervisor–subordinate relationships in other organizational settings on important dimensions (e.g., the provision of day-to-day instruction and performance assessment), the military contains a culture of high-power distance (Soeters et al., 2006). In an organization, power distance is the extent to which members accept that power is unequally distributed (Hofstede, 2001), and results in members' tendency to dutifully follow the instructions of authority. The ability of supervisors to influence the application of skills may be reduced in settings with less power distance. Alternatively, as high-power distance also results in the tendency to maintain social distance between subordinates and leaders, settings with lower power distance that encourage relations between subordinates and leaders may be better able to emphasize supervisor support for the individual to enhance iSRT outcomes (Farh et al., 2007). Future work could seek to apply a similar test in other organizational settings to confirm generalizability. Fourth, we could not determine whether the programme promoted or protected positive psychological health during the stressor period. It was anticipated that these measures would be more vulnerable to ceiling effects and self-report bias, than mental ill-health measures, given that participants are regarded as mentally healthy at commencement and were completing a competitive training programme. Furthermore, the indices of mental ill-health adopted in the current study have demonstrated sensitivity to stressor exposure in past resilience training trials (e.g., Adler et al., 2009). Likewise, these outcomes were measured at the individual level as the SRT and iSRT are founded on a model of individual resilience (Crane, Searle, et al., 2019). As Mental Fitness AARs could be conducted in a group setting and cadets were performing in teams, it is worthwhile for future research to investigate team-level outcomes, especially as the resilience of individual team members does not necessarily transpose to resilience of the team. Fifth, although measures for depression and anxiety were reduced to remove items that lacked validity in the current context, future research should attempt to use multi-item, context-appropriate measures of depression and anxiety. Last, although there were no condition differences in perceived stressor frequency at baseline and we could not include perceived stressor frequency as a covariate in our tests due to limited power given model complexity. Future research with a sufficient sample, may seek to include an objective measure of stressor frequency, and its interaction with time and group. This would facilitate the examination of programme effects on resilience whilst accounting for individual differences in stressor exposure between groups over time.

CONCLUDING EXPLANATIONS

The current superiority trial was the first to evaluate a supervisor-led extension to a systematic self-reflection approach to resilience training, in a military officer training context. The iSRT appears to add benefit to the SRT by promoting resistance to the early deleterious effects of training stressors and by promoting performance in the context it is directly applied to. The study also adds to the literature that highlights the active role that supervisors can espouse in supporting subordinate well-being and performance during stressor periods and exemplifies how resilience training extensions can facilitate supervisor support for the individual to promote training outcomes.

AUTHOR CONTRIBUTIONS

Madison Kho: Conceptualization; data curation; formal analysis; investigation; methodology; project administration; visualization; writing – original draft; writing – review and editing. Eyal Karin: Conceptualization; data curation; formal analysis; methodology; visualization; writing – review and editing. Daniel Gucciardi: Conceptualization; supervision; visualization; writing – review and editing. Monique Crane: Conceptualization; data curation; formal analysis; funding acquisition; investigation; methodology; project administration; resources; supervision; visualization; writing – original draft; writing – review and editing.

ACKNOWLEDGEMENTS

This research was funded by Macquarie University, the Commonwealth Department of Defence, and a Macquarie University Research Excellence Scholarship. The authors express their gratitude to personnel at the Royal Military College (RMC) who assisted with the implementation of the study: Jada Croft, Ryan Pitt, Julie Bodin, Dean Johnson, Robert Ryan, and the RMC second-class cadets and instructors of 2019. Open access publishing facilitated by Macquarie University, as part of the Wiley - Macquarie University agreement via the Council of Australian University Librarians.

CONFLICT OF INTEREST

All authors declare no conflict of interest.

Open Research

DATA AVAILABILITY STATEMENT

The study data are not available as it was obtained from the Australian Department of Defence and its members and is considered confidential in nature.