A Critical Examination of the U.S. Army’s Comprehensive Soldier Fitness Program

  1. 1.  New School of Psychotherapy and Counselling, London, UK

Abstract

This article examines the U.S. Army's Comprehensive Soldier Fitness (CSF) program from a scientific, ethical, and pragmatic viewpoint. CSF is one of the largest single applications of psychological research in history, intended to develop "resilience" in every U.S. Army soldier. I highlight several areas where the available information about the program either suggests the likelihood of specific problems, or is insufficient to allow the research community to evaluate the effectiveness of CSF independently of the claims made by its originators and assurances given by other non-disinterested parties. In particular, I question (a) whether a program based on resiliency training for school-aged children can hope to address the serious mental trauma, including PTSD, faced by soldiers deployed to war zones; (b) whether the instruments used to measure the performance of the program are reliable, valid, and appropriate for the circumstances in which they are being used, and (c) whether the design and delivery of the program takes sufficient account of the conflicting real-world demands placed on the individuals involved. I conclude that the program appears to have a number of potentially problematic aspects that require wider scrutiny from psychological researchers and practitioners.

The back story

This article started life as an essay for my Master's in Applied Positive Psychology course at the University of East London. My teachers gave me some good feedback and encouraged me to turn it into a journal article.

My first port of call was Military Psychology. One reviewer said "I highly recommend publishing the manuscript" while another called it a "very well-written critique." However, the action editor recommended rejection without offering me a chance to rewrite those points (mostly to do with the tone of the writing) which he was not happy with. One infuriating point was when the action editor also chastised me for not including "a brief psychometric survey of the strengths and weaknesses of [the GAT]"-despite the fact that, as I note in the article, the GAT is secret Army property and thus unavailable for anyone to critique. Several other comments basically came down to the same thing: I was not providing a sufficiently empirical critique of a program which is only barely open to empirical scrutiny. Can you say "unfalsifiable"?

Next, I sent a slightly toned-down version of the article to Psychological Services. The rejection letter in this case was considerably nicer and more constructive, but still-I felt-missed the point that any criticism of CSF is limited by the lack of data that the Army makes available. (Amusingly, the action editor suggested that the article might be a better fit at Military Psychology.)

By now, I was getting a little discouraged. I'm not a full-time academic; I'm not even a part-time one, except as a fee-paying student on a modest Master's program at a small school of psychotherapy in London. I didn't want to do a thousand more hours of research into something as well-concealed from view as CSF in order to turn the article into something purely numerical, but I didn't want to make into a political attack on the U.S. military either. My views on U.S. foreign and military policy are complicated; while I am probably on the "dovish" side of the current (and previous) Commander-in-Chief, I'm probably a little to the "hawkish" side of many of those who attacked CSF more or less unconditionally on the basis that it involved psychologists in war. Certainly, one of my aims in critiquing CSF has been to avoid falling into the trap of claiming that "this is immoral, and it won't work anyway." If it won't work, the question of morality-at least, to the extent that this refers to "psychologists helping Americans kill innocent foreigners"-is surely moot. (I do, however, have concerns about the ethicality of the program, both for serving personnel and-to a lesser extent, not least because I am not one myself-the U.S. taxpayers who foot the bill for all this.)

After some reflection, I have decided to publish this article at The Winnower, for a number of reasons. First, because time is passing, and some of the specific details which I was able to obtain are no longer as up-to-the-minute as they were, so I wanted to get this out sooner rather than later. Also, with the ongoing reduction in overseas combat deployments, the U.S. military landscape is changing, which will have its own effects on the demographics of soldiers as well as on the challenges they face (this also is discussed briefly in the article). Second, because I'm interested in the possibilities offered by the format of The Winnower, not just in terms of its review structure, but also to break out of the traditional journal article format. What I've written here is part academic study, part critique, and part polemic. It probably wouldn't fit well into most academic journals, yet it's too complex for most investigative journalism outlets as well. If somebody-perhaps with better access than me, sitting at home in France, to primary data sources-wants to pick up a part of this and run with it, I wish them the very best of luck and offer them any support I can give. Finally, I wish to thank Bea Jauregui and Harris Friedman for their encouragement with this project, and their helpful comments on early drafts of this article.

A Critical Examination of the U.S. Army's Comprehensive Soldier Fitness Program

Since November 2008, the Positive Psychology Center (PPC) at the University of Pennsylvania has been working with the U.S. Army on a large-scale resilience program called Comprehensive Soldier Fitness (CSF), to be delivered to every officer and enlisted soldier in the U.S. Army, including reservists-a total of 1.1 million people at any one time (Cornum, Matthews, & Seligman, 2011, p. 4). This is, by several orders of magnitude, the largest organizational application of positive psychology to date, and will also lead to the creation of one of the largest longitudinal datasets in the history of social science research (Seligman & Fowler, 2011). In view of this, the science, ethics, and effectiveness of the CSF program would seem to be a subject worthy of critical examination by psychological researchers; such an examination is the focus of the current article. After describing the components of CSF, I discuss some apparent scientific problems with its design, before reviewing the results that have been published so far. Finally, I draw some tentative conclusions about what we know so far about CSF and describe some current developments.

My principal intention here is to examine mainly objective questions relating to the scientific issues surrounding the CSF program (although a few peripheral issues that do not seem to have been raised elsewhere, notably to do with the sometimes curious procurement practices associated with the program, are addressed, principally in the endnotes). Indeed, CSF has already been the subject of criticism on numerous other grounds: for directly involving psychologists in war (e.g., Phipps, 2011; Dyckman, 2011), for implicitly supporting the idea of "persistent war as a mode of human progress" (Jauregui, in press, p. 31), for the contract for its implementation having been awarded without a competitive tendering process (Benjamin, 2010[i]), for allegedly being an unconstitutional front for (evangelical) Christian indoctrination (Leopold, 2011[ii]), and for being conducted without the informed consent of the majority of its participants, thus violating ethical norms (Eidelson, Pilisuk, & Soldz, 2011). Many of these objections are clearly associated with specific political or moral agendas; indeed, Brig. Gen. Rhonda Cornum, the first director of CSF, remarked that the program's critics were "primarily motivated by an anti-war opinion" (Bartlett, 2011, p. A11). In contrast to these themes, I believe that the issues of scientific theory and practice that I raise in this article should be of concern to all psychologists, regardless of their moral or political standpoint.

Background

Before discussing the scientific issues, I will briefly describe the way in which CSF has been presented to the scientific community until now, as well as its basic structure. Both of these set the stage for understanding the scientific questions that I raise later.

Public presentation of CSF

CSF was formally introduced to the psychological research community via the January 2011 issue of American Psychologist, which contained 13 largely uncritical articles documenting the program in considerable detail (including, for example, a four-page, psychology-free description of the software platform used; Fravell, Nasser, & Cornum, 2011), and to the general public in May 2011 in Seligman's popular book Flourish (Seligman, 2011a). This was followed by a selection of critical comments in the October 2011 issue of American Psychologist (e.g., Dyckman, 2011; Krueger, 2011), accompanied by a reply from Seligman (2011b). However, since then, a scholarly silence seems to have fallen. A PsycINFO search on June 25, 2013 identified just nine articles or book chapters matching the term "comprehensive soldier fitness" anywhere in the full text that have appeared since the above-mentioned comments in American Psychologist. Of these nine, six consist mostly of uncritical descriptions of CSF whose authors are either Army personnel or otherwise affiliated to the program, while just three mention CSF in a remotely critical way. Friedman and Robbins (2012) concentrated principally on moral questions, while Sharma and Sharma (2012) noted simply, of the "spiritual" dimension of CSF, that "Questions far outnumber answers" (p. 264). Only S. L. Smith (2013) has directly queried whether the program is likely to meet its expected outcomes, asking whether attempting to inculcate resilience in soldiers might not, in some circumstances, have the opposite effect. In summary, very little original, critical research on the general scientific validity of CSF has been published to date in peer-reviewed fora; even the U.S. Army's own official reports on the interim findings of the program (Lester, Harms, Bulling, Herian, & Spain, 2011a, 2011b; Lester, Harms, Herian, Krasikova, & Beal, 2011; Harms, Herian, Krasikova, Vanhove, & Lester, 2013) have only been published on U.S. Government websites. This seems quite remarkable given the ambitious scale of the program, involving a budget many times larger than anything that most psychological researchers will see in their entire careers, and its portentous implications for positive psychology and indeed a whole raft of other "positive" social interventions hypothesized by Seligman and Fowler (2011, p. 85).

Description of CSF

CSF was developed to address concerns within the U.S. Army about mental health issues among soldiers (Seligman, 2011a, Chapter 8). The Army's procurement documentation for the CSF project mentioned a number of specific areas of concern, including posttraumatic stress disorder (PTSD), suicide, sexual assault, substance abuse, and family violence (U.S. Army, 2009). Of these, PTSD is mentioned more often than any other issue in the CSF literature; this could perhaps be because of its extensive historical media coverage (often linked to the experiences of Vietnam veterans), or simply because of the high rates of PTSD among soldiers returning from combat missions to Iraq or Afghanistan, estimated to be approximately 12-16% (Litz & Schlenger, 2009). The psychological challenges facing Army personnel are considered so important that participation in CSF has been made mandatory for soldiers and reservists; it is also available, on an optional basis, to their families and to "Army civilians" (non-military personnel who work for the Army). However, members of these latter groups do not appear to be very interested in the program at present; Lester et al. (2013) reported that less than 1% of those eligible to volunteer to participate in CSF had done so. Nevertheless, this represents an improvement on the Army unit described by Carr et al. (2013), for whom participation in Master Resilience Training was made voluntary, resulting in a participation rate of zero.

The participant's view of Comprehensive Soldier Fitness is summarized in the program's logo (shown in Figure 1), the principal element of which is a standard five-pointed Army star, with each point, conveniently, corresponding to one of five "fitnesses"[iii]. The first of these, "physical," encompasses standard Army physical training; its inclusion, as well as the strapline "Strong Minds-Strong Bodies," seems to convey the message that the "C" in CSF represents a transition from the traditional military concentration on physical fitness, to a holistic approach that melds the physical with the psychological. The other four points of the star represent the psychological "fitnesses" of CSF, which are defined as "emotional," "family," "social," and "spiritual." According to Peterson, Park, and Castro (2011), these "fitnesses"-chosen initially, it appears, by discussion among psychologists and Army officers-also happen to correspond to distinct psychometric factors, although they provided no numerical data to support this assertion (which, as will be shown, appears to be questionable).


Figure 1 CSF and CAF logos

Left: Original U.S. Army "Comprehensive Soldier Fitness" logo (2009-2012)

Middle: New U.S. Army "Comprehensive Soldier and Family Fitness" logo (August 2012-present)

Right: U.S. Air Force Comprehensive Airman Fitness logo (2011-present)

CSF comprises several functional components (Casey, 2011). The first is a psychometric instrument called the Global Assessment Tool (GAT), a 105-item questionnaire that CSF participants complete using a computer. The full list of items that make up the GAT has not been published, but a summary of its subscales was provided by Lester, Harms, Herian, et al. (2011, p. 10), from which it appears that about 60% of the items are based on subsets of previously validated measures, including the PANAS (Watson, Clark, & Tellegen, 1988) and the PHQ-9 (Kroenke, Spitzer, & Williams, 2001); for a full list, see Lester, Harms, Bulling, et al. (2011a, pp. 29-31). Peterson et al. (2011) described the development of the new items and subscales only in general terms; no numerical data were provided to justify their claims of validity or reliability.

The second main component of CSF is a series of "Comprehensive Resilience Modules" (CRMs). A CRM is a 20-minute long online distance-learning presentation, interspersed with multiple-choice questions to test the trainee's understanding of the material presented. Depending on which source one consults (e.g., Pargament & Sweeney, 2011; Lester, Harms, Herian, et al., 2011), it appears that there are between four and 10 modules for each of the four psychological "fitnesses." Relatively little information, even by the standards of secrecy of the CSF project, has been made available about the CRMs. Thus, for example, at no point in the CSF literature is it documented that the CRMs have been largely recycled from the "Battlemind" program, which was the Army's immediate precursor to CSF; this fact was only revealed in a recent National Academies report on military mental health programs, including CSF (Institute of Medicine, 2014).

The third principal component of CSF is "Master Resilience Training." The premise of this is that experienced soldiers (typically senior sergeants) will be trained by positive psychologists, first, to become more resilient themselves, and second, by qualifying as "Master Resilience Trainers" (MRTs), to be able in turn to deliver resilience training-on subjects such as self-regulation, optimism, explanatory styles, and the cultivation of gratitude (Reivich, Seligman, & McBride, 2011)-to soldiers within their Army units. Resilience training is also being integrated into general Army leadership training courses (Casey, 2011).

Scientific Issues

Seligman (2011a) claimed that "positive psychology... uses tried-and-true methods of measurement, of experiments, of longitudinal research, and of random-assignment, placebo-controlled studies to evaluate which interventions actually work and which are bogus" (p. 71). However, there are several areas where CSF appears to fall somewhat short of these high standards, as discussed in the sections below.

Issues with the design and deployment of the GAT.

In keeping with the rapid pace at which the CSF project was rolled out, the GAT was designed in just three months (Peterson et al., 2011), without the extensive feedback from peer review and multiple rounds of testing that would normally be associated with the creation of any new scale (Krueger, 2011). Despite Seligman's (2011b) assurances that the GAT was "not developed sloppily" (p. 646), the consequences of its rapid development are apparent in a number of areas. For example, Peterson et al. (2011) noted that the data provided by GAT participants was substantially negatively skewed from the outset, but chose to frame this as "a psychometric annoyance trumped by the theoretical point that most soldiers-as expected-were doing well" (p. 15). However, this appears to gloss over a substantial ceiling effect that has potentially serious implications for the longitudinal application of the GAT, especially as the anticipated result is an improvement over time. For example, the three items from the "Adaptability" subscale, used by Fravell et al. (2011, Figure 2) to illustrate CSF's reporting facilities, show that around 70% of the responses to each item consist of the top or next-to-top option on the 5-point Likert scale, leaving little room for improvement. Further issues with the GAT are highlighted in the section entitled "Preliminary results of CSF."

A further cause for concern is the way in which GAT results are interpreted. Peterson et al. (2011, pp. 14-15) provided an example of results for an officer whose scores were high on some items and low on others. However, as Figure 1 on p. 15 shows, the areas of relative strength and weakness for an individual are defined in terms of percentiles, meaning that they depend on the results of everyone else who has taken the GAT. Thus, the officer in the example only scored around the 10th percentile on "work engagement," but by definition, there will always be 10% of respondents who score at or below this percentile on any given measure. Thus, every year, and for every component of the GAT, some percentage-whose value depends only on where the reporting threshold is set-of subjects will be told that they need to improve on that component, regardless of any prior improvements in the absolute level of that component that those subjects, or the Army as a whole, might have made. Confirmation of this approach can be found in Lester, Harms, Bulling, et al. (2011a), in which an apparently positive Likert-scale score of 3.84 out of 5 is shown with a red bar ("facing some significant challenges") because it is slightly lower than the average score for the comparison group (see Figure 2). Thus, the GAT will always "reveal" that large numbers of soldiers are facing "significant challenges"-and, hence, are deemed to be in need of further resilience training-by a process that appears to consist of circular reasoning.[iv]

Figure 2 Example GAT soldier report (Lester, Harms, Bulling, et al., 2011a, p. 35)

The "Social" score (outlined in orange) shows that a score of just 2% less than the mean of the comparison group, regardless of the absolute value (cf. "Spiritual") is considered a "red bar" result, meaning that the subject is "facing significant challenges" (Lester, Harms, Bulling, et al., 2011a, p. 32). On this basis, using the standard deviations shown on p. 40 of the same report, 45% of participants would be expected to have a red bar on this dimension.

Participant bias.

Another cause for concern is the particular set of demand characteristics (Orne, 1959) that might be caused by the fact that for the most important group of participants (i.e., soldiers and reservists), participation is mandatory, with the possibility of disciplinary sanctions for non-compliance. There seems to be no evidence that the designers of CSF have attempted to calculate, or otherwise take into account, the possible effects of such forced participation in psychometric testing. Their apparent lack of experience and expertise in this area is perhaps not surprising, given that any research program-except, apparently, CSF-must normally pass the approval of an institutional review board (which could be expected to apply very close scrutiny to any proposed study in which participation was not entirely voluntary). Thus, aside from the obvious ethical issues around informed consent (Eidelson et al., 2011), mandatory participation causes a major practical problem in the interpretation of GAT results because a unique confounding variable of unknown magnitude has been introduced. Almost 50 years ago, Masling (1966) noted that subjects would frequently bring their own "games" into psychological research processes. If this tendency-which Masling named the "Screw You Effect" (p. 96)-is common for typical research subjects, who have volunteered to participate and may even be receiving some form of compensation for their time, it seems likely that it may be even more so for involuntary participants. Furthermore, for many soldiers, the GAT might well be seen as just another of the apparently endless measurement and appraisal exercises to which they are regularly subjected by the Army (MacLeish, 2013).[v] Since those other exercises are not conducted anonymously, with poor results potentially having adverse career outcomes (U.S. Department of Defense, 2002), soldiers might not believe-despite assurances to the contrary from the program's managers (Fravell et al., 2011)-that their GAT scores are confidential[vi] and that there will be no negative consequences for "incorrect" responses. Sagalyn (2012) reported that CSF data analysts believed that "only [emphasis added] 16 percent of the answers were due to the fact that the soldiers did not trust the survey" (para. 34). It seems reasonable to question the reliability of longitudinal results claiming to show improvements of at most 2.1% (Lester, Harms, Herian, et al., 2011), if the researchers themselves believe that 16% of the answers have either been invented at random or otherwise represent "enhanced" versions of the truth. It is tempting to imagine that a perceived need to be seen as "doing well" might explain, at least partially, the negative skew noted earlier.

The tendency of individual soldiers to act in their own best interest is not the only source of risk to the validity of the GAT data. Although individual results are officially described as being confidential, it is the stated intention of the GAT's designers to make aggregated data available to "commanders" (of unspecified rank), with the prospect of remedial interventions for "units" (of unspecified size) with low scores, or, conversely, the assignment of units that are considered especially resilient to "difficult" operations (Peterson et al., 2011, p. 16). This clearly has the potential to create incentives for officers to ensure that the soldiers under their command "demonstrate" particular levels of resilience, in order to position their unit(s) in a favorable (but, perhaps, not too favorable) light.

Lack of control groups and pilot testing.

Given the scale and ambition of CSF, it seems remarkable that no provision has been made for a substantial, long-term control group to allow the outcomes of the training interventions to be properly evaluated, and that no meaningful form of pilot testing was conducted to see whether the various components of the program had measurable effects. According to Seligman (2011a, p. 163), this is a consequence of a personal decision taken by Army Chief of Staff Gen. George Casey that CSF was to be applied to every soldier, without exception, and as soon as possible. Indeed, as Lester, McBride, Bliese, and Adler (2011) noted, the very existence of a control group was considered to pose a moral problem: "...how could we ethically justify withholding resilience training from soldiers slated for combat duty?" (p. 78). This argument in effect presupposes that the training is effective, and implies that to withhold it would be the equivalent of sending troops into battle without other items of military equipment for which demonstrations of their fundamental utility via controlled trials have traditionally been waived, such as helmets or parachutes (although see G. C. S. Smith & Pell, 2003). Lester, McBride, et al. (2011) claimed that, because of the Army's limited capacity to train MRTs, a natural "wait-list" control group would be formed by those units that did not yet have MRTs in place to provide resilience training, but this hardly constitutes a proper control since those units would have been exposed to all of the other components of the CSF program, and the timing of the transition from "control" to "treatment" conditions would be out of the researchers' hands.

Use of the Penn Resiliency Program as a basis.

CSF, and especially its MRT component, is substantially modeled (Cornum et al., 2011) on the Penn Resiliency Program (PRP; Gillham, Brunwasser, & Freres, 2008). Indeed, the requirement for compatibility with the PRP was one of the reasons given for the award of the CSF contract to the PPC without competitive tendering. The waiver document used to justify that decision stated that a number of studies of the PRP "have concluded that significant positive effects are sustained and performance of participants is generally improved" (U.S. Army, 2009, p. 4). However, it seems unclear whether the PRP, in the current state of knowledge and without further testing, is really an appropriate basis for a large-scale military program. First, the PRP is designed for children aged 10-14, a group whose psychological characteristics and daily sources of stress might be expected to be rather different from those of a group of adults of whom 85% are male with an average age of 29 (Defense Manpower Data Center, 2008), especially when subjected to the extreme conditions experienced by soldiers in wartime. Second, the results obtained from the studies mentioned above are considerably less impressive than the authors of the waiver document apparently believed. Brunwasser, Gillham, and Kim (2009) conducted a meta-analysis of these studies and found inconsistent results and small effect sizes (Glass's d+ values from .11 to .21). The abstract of that article concluded, "Future PRP research should examine ... whether PRP is effective when delivered under real-world conditions," suggesting that these authors, one of whom was a designer of the PRP, were not necessarily convinced that it was ready to be used as the basis of a massive training program such as CSF.

A key characteristic of PRP (and, hence, CSF) is that it is applied equally to an entire population, with no specific emphasis on those for whom an initial testing process appears to show risk factors. However, there have been numerous examples of long-term universal psychological programs with preventative intent that have had either no effect or even, in some cases, the opposite effect to what was hoped for, such as unchanged (Ringwalt, Ennett, & Holt, 1991) or increased (Teasdale, Stephens, Sloboda, Grey, & Stephens, 2009) cigarette and alcohol use among junior high school students, and increased rather than decreased delinquency (McCord, 2003). Bonanno, Westphal, and Mancini (2011) reviewed a number of suicide-reduction programs and concluded that those aimed at the entirety of a given population were typically ineffective; only those programs that pre-screened participants to identify those at risk were beneficial overall. They also noted that, by increasing the saliency of suicide among participants, such universal programs could even be potentially harmful. Indeed, other writers have also questioned whether CSF could be actively dangerous for its subjects. Eidelson and Soldz (2012) suggested that soldiers who imagined that their CSF training provided them with "protection" against trauma might become more reckless with their own safety, or that of others. S. L. Smith (2013) argued that soldiers who found themselves unable to cope with negative feelings despite having been "made resilient" by CSF training might feel marginalized and less able to ask for support, and consequently be less able to adequately resolve potentially PTSD-inducing feelings such as guilt, than if no resilience training had been provided. A further consideration is the possible negative effect on soldiers of exposing their problems to others without proper therapeutic scrutiny and supervisory follow-up; two weeks' training, including a brief introduction to Ellis's (1962) ABC model (as described by Harms et al., 2013), arguably does not constitute sufficient preparation in psychotherapeutic techniques to enable MRTs to handle difficult issues that might arise, for example, during an exercise in the middle of a group training session.

Relevance of resilience training to PTSD.

One of the stated aims of CSF (e.g., Cornum et al., 2011, p. 5) is to reduce the incidence of post-traumatic stress disorder (PTSD) among soldiers, but it is not clear whether this is a realistic aim, or indeed whether the designers of CSF have fully appreciated the complex nature of PTSD. Reivich et al. (2011) referred to PTSD as "a nasty combination of depressive and anxiety symptoms" (p. 26), but this description greatly oversimplifies the condition. Simms, Watson, and Doebbeling (2002) reviewed several models of PTSD and concluded that the best fit was a model having four factors (intrusions, avoidance, hyperarousal, and dysphoria), with only the last of these being significantly correlated with depression and anxiety. Thus, much of PTSD consists of symptoms whose prevention is not addressed by the PRP, or indeed anything else that comes under the umbrella of positive psychology. The idea that techniques that have demonstrated, at best, marginal effects in reducing depressive symptoms in school-age children could also prevent the onset of a condition that is associated with some of the most extreme situations with which humans can be confronted is a remarkable one that does not seem to be backed up by empirical evidence.

Indeed, although 11 of the 13 articles in the January 2011 edition of American Psychologist mentioned the term "posttraumatic stress," none described an empirically-validated mechanism by which CSF might reduce the incidence of PTSD. It was implied, however, that the harnessing of posttraumatic growth (PTG) might be part of the solution; Seligman and Fowler (2011, p. 84) reported preliminary discussions about CSF in which Seligman had suggested that the aim of a prevention program should be to move soldiers from PTSD to PTG, but proposed no method by which this trivially desirable outcome might be achieved. Similarly, Tedeschi and McNally (2011) asked-but left unanswered-the question of whether PTG could be facilitated among combat veterans. These authors cited numerous studies showing the occurrence of PTG alongside PTSD among people who had been exposed to adversity, but all of these concerned individuals who had principally been passive victims of traumatic events (e.g., cancer patients, or civilians in war zones). However, PTSD in soldiers is not just the result of experiences of intense danger to oneself or seeing harm come to one's comrades (consistent with the popular view of the term "shell shock"). Rather, in many cases PTSD arises from guilt at what a soldier has done to harm others, or perhaps not done to save them (Henning & Frueh, 1997). This phenomenon, named "moral injury" by Litz et al. (2009; see also Maguen et al., 2010; Maguen & Litz, 2012) is likely to be accentuated if those harmed are non-combatants, an increasingly likely possibility in the era of what R. A. Smith (2006) has named "War amongst the people," characterized by low-intensity combat against irregular forces, often in urban and semi-urban environments. This raises questions about the value of teaching soldiers how to find meaning in adversity, or to seek alternative explanations for why bad things have happened to them, when in many cases they know themselves to be the authors of those adverse events, such as the soldier ordered to run over Iraqi children in the way of her armored vehicle "because stopping is not an option" (Paulson & Krippner, 2010, pp. 12-13).

Role of MRTs.

An issue that does not seem to have been extensively reported during the development of CSF is the demands placed on the Master Resilience Trainers (MRTs). As a result of taking on the responsibility of the MRT role, these NCOs might find themselves in ambiguous situations, thus potentially reducing their effectiveness as trainers (compared to professional psychologists) to a far greater degree than in the analogous case of using community members in the PRP (Brunwasser et al., 2009, p. 1050). As senior sergeants, MRTs typically owe their rank, along with the authority and prestige that it confers, to a demonstrated commitment to conveying orders from commanding officers to the soldiers in their units, while emphasizing to those soldiers the importance of complying without hesitation or questioning; indeed, historically, sergeants have often had the job of getting soldiers to overcome their innate reluctance to do something as basic as firing their weapons at the enemy (Grossman, 2009). The traditional view of the Army drill sergeant in popular culture (e.g., Lehrer, 1959; gibbysgarage, 2010) does not allow much room for empathy or tolerance of ambiguity; although that may not be an accurate representation of the psychological makeup of contemporary NCOs, the problem remains that the role with which sergeants are charged by the Army is precisely one of "encouraging" soldiers to do what the Army wants, which those same soldiers may not always have experienced in the past as being something that will necessarily enhance their emotional well-being. Having as a resilience teacher-imparting wisdom on how to "Hunt the good stuff" (one of the motivational slogans of CSF) to improve one's positive emotions, or to challenge irrational beliefs about the causes of adverse events-the same individual who also ensures one's less-creative compliance with detailed regulations in matters such as drills, uniform, and curfews could arguably lead to confusion and doubt for many soldiers. While MRTs are not meant to provide individual psychotherapy, it seems implausible that the best person to assure the effective delivery of even the most basic training on human emotional functioning would be someone whose official organizational role at all other times vis-à-vis his or her "students" excludes, more or less by definition, anything remotely resembling "unconditional positive regard." (Analogous reasoning presumably explains why Army chaplains, for example, have no other specific duties that would involve them having commanding authority over soldiers.) Readers interested in questions of power and legitimacy in therapeutic relationships might also find it an interesting exercise to consider the dynamics involved when a commissioned officer wishes to consult the battalion/company MRT-whom the officer almost certainly outranks-for advice on personal resilience.

Training or research?

Earlier, when discussing the possible confounding effects of participation in CSF being mandatory for serving personnel, I noted that this also had an ethical dimension. Eidelson et al. (2011) have discussed some of the technicalities of whether or not informed consent is strictly required according to the letter of the rulebooks of various organizations, such as the APA (American Psychological Association, 2010, p. 6) and the Department of Defense (2011, p. 2). But it seems that the designers of CSF have sidestepped this question entirely, by positioning the program as merely another form of Army training:

"CSF is not research [emphasis added]. It has the same status as training programs that require all soldiers to attend classes about how to recognize signs of suicide and sexual harassment, to do morning physical training, how to resist psychologically when captured, or why to wear safety belts when driving. These programs do not require informed consent." (Seligman, 2011b, p. 646)

It is interesting to contrast this statement with Seligman and Fowler's (2011) enthusiastic declaration that "One million soldiers taking the GAT is an unprecedented database for the prospective longitudinal study [emphasis added] of the effects of psychological variables on physical health, mental health, and performance" (p. 85). The two appear to be difficult to reconcile, unless one accepts the apparent implication that a longitudinal study can somehow be deemed not to constitute research when this suits the purposes of the authors of the study. Indeed, this "non-research" project appears to have recently evolved further, with data from the GAT becoming part of something called the "Person-Event Data Environment" (PDE), described as a "virtual electronic warehouse of medical, psychological, health, training, and personnel information, which has been gathered on over one million Army soldiers," with Seligman and Cornum as co-PIs (Chronicle of Higher Education, 2013); one wonders how soldiers who do not wish their confidential psychological data to be combined with their (non-confidential) health information and other service records are to register their objections.

Even if these ethical issues are ignored, however, there remains a major potential confounding factor when conclusions are to be drawn about the effectiveness of CSF, namely the general turnover of personnel within the Army, which historically runs at around 17% (Congressional Budget Office, 2006). This means that even within the initial three-year CSF contract period, approximately half a million new people will join the Army and start CSF training and evaluation. However, it cannot be assumed that the ethnic, socio-economic, or attitudinal composition of successive cohorts of recruits will remain constant over time. Changes in economic circumstances, recruitment targeting policies, requirements for operational skills, and the perceived degree of danger (and its reverse side, adventure) to which soldiers may be exposed as U.S. foreign policy evolves, are all likely to cause substantial changes in the average baseline psychological profile of newly-recruited soldiers. Without rigorous measurement and discounting of those factors, which can only be done at the point at which the individual joins the Army (and is effectively impossible for spouses and other family members), any conclusions about the effect, in isolation, of CSF training is likely to be of limited validity; however, no indication has been given in the CSF literature as to how this baseline measurement is to be performed, if at all.

Lack of transparency.

A final, overall concern with CSF is its inscrutability. Only a few tantalizing glimpses of the questions constituting the GAT have been made public. The complete GAT questionnaire itself is proprietary to the U.S. Army (N. S. Park, personal communication, January 15, 2013) and as such, although it does not appear to be a classified document in the sense of Executive Order 13526 (Obama, 2009), has not been made available to the academic research community. Of course, many studies, particularly those including a large ecological component such as is arguably the case with CSF, are difficult to replicate for logistical reasons, but this does not obviate the need for as much transparency as possible in the reporting of the tools and methods used. Similarly, the contents of the Comprehensive Resilience Modules do not seem to have been made available, which again prevents any meaningful assessment or criticism of their design and pedagogical value. Even allowing for the need for some degree of confidentiality in a military environment, this lack of transparency is both disappointing and worrying, especially given the vast number of serving personnel and civilians with minimal security clearance to whom CSF is being applied. While the intention of the scientists who contributed to the development of CSF was surely not to create an unfalsifiable, self-justifying process whose validity cannot effectively be questioned by outsiders, there seems to be a significant danger of precisely that occurring, with consequent risks not only to the practice of psychological science, but also the U.S. taxpayer and, ultimately, serving military personnel themselves, who may be receiving ineffective or even inappropriate training and advice.

Preliminary Results of CSF

To date, four official reports have been published by the Army's Research Facilitation Team. The first of these (Lester, Harms, Bulling, et al., 2011a) demonstrated that soldiers who committed suicide, tested positive for illegal drugs, or were convicted of violent crimes, had significantly lower GAT scores than other soldiers. However, even the results of those who committed suicide are substantially skewed, with mean scores of 3.40 out of 5 (versus 3.76 for the overall sample). Furthermore, the difference between those who committed suicide and the rest of the sample corresponds to about 0.4 standard deviations, meaning that the typical suicide victim would be in the 35th percentile overall, with nearly 400,000 other soldiers scoring lower than him or her. This suggests that any potential use of the GAT to predict suicides, and perhaps to target prophylactic interventions, is likely to have limited success.

However, the report by Lester, Harms, Bulling, et al. (2011a) revealed some interesting information about the psychometric structure of CSF. Table C-1 on p. 36 shows the correlations between the GAT subscales and their associated "fitness" dimensions, with "Spiritual Fitness" being correlated .619 with "Emotional Fitness" and .546 with "Social Fitness," and these last two being correlated .693 with each other. These high correlations are in direct contrast to the findings reported by Peterson et al. (2011) that all four CSF psychological "fitnesses" loaded on distinct factors during testing of the GAT. This suggests, for example, that it may not be appropriate from a purely scientific point of view for "Spiritual Fitness"-which has been a highly controversial component of CSF (e.g., Leopold, 2011)-to be included as a separate factor; however, it might be that a decision to drop this dimension would be politically difficult.

The second report on CSF (Lester, Harms, Bulling, et al., 2011b) identified significant correlations between GAT results and "positive outcomes," principally promotions. However, promotions in any organization-and especially the "below zone" (the Army's term for "fast track") promotions that figured prominently in the report-typically require positive (and, inevitably, subjective) reports from one's superiors. Thus, these correlations might simply reflect the fact that people with better "resilience" characteristics-such as positive emotions or engagement-make a better impression on their superior officers, or perhaps that those who are motivated by promotion take extra care to give "positive" answers on the GAT if they suspect that these may (despite assurances to the contrary) be examined as part of the promotion procedure. Additionally, in any organization of a finite size, the number of promotions is limited; in such a zero-sum game, if every officer were to objectively improve his or her resilience by the same amount, this would have no overall effect on their promotion prospects. Thus, the reporting of the correlation between "resilience" and promotions does not seem to have much scientific value.

The third and most extensive report on CSF (Lester, Harms, Herian, et al., 2011) purported to analyze the impact of Master Resilience Training on "psychological fitness," as measured by self-report data from the GAT. The authors compared the results of 3,218 subjects in a "control" group, whose units had no Master Resilience Trainers in place yet, with 6,739 subjects in a "treatment" group, consisting of units where MRTs had been deployed after receiving their training. However, this report appears to exhibit a number of methodological deficiencies. First, different tables (e.g., Tables 4, B1, B2) show different means for what should be the same data; one of the authors (D. Krasikova, personal communication, December 22, 2012) explained that the means shown are not the raw means of the sample data, but the "estimated marginal means" output by SPSS, which makes any meaningful checking of the work by scholars who do not have access to the (confidential!) raw data almost impossible. Second, the authors claimed significance for results of certain "dimensions" (corresponding to "fitnesses") as well as some of the subscales that compose them, which inflates the number of elements for which significant results are reported by, in effect, counting some of them twice (see Figure 3). Third, the principal discussion revolves around differences between the control and treatment groups at a particular point in time (Table 4), when-as a consequence of the lack of random assignment-the two groups clearly differ in substantial ways that were not controlled for (most notably, that soldiers in the treatment group were more likely to have been deployed during the months immediately preceding the measurement of their GAT scores, as shown in


Figure 3 GAT Time 2 report (Lester, Harms, Herian, et al., 2011, p. 15)

1. The (very small) effect sizes are circled in green.

2. The "Social Fitness" dimension (outlined in purple), whose F-statistic is reported as significant (p=.050), is merely the aggregate of four subscales (outlined in pink), whose F-statistics are reported significant at ps=.183, .000, .666, and .263, respectively.

Figure 1 on p. 12). Fourth, the basis for the claimed effects of MRT training (pp. 15-16) consists principally of noting those individual subscale scores where a statistically significant result was obtained; however, in the absence of previously stated hypotheses, it is impossible to distinguish this from cherry picking. No explanation is given, for example, as to why the reader should be impressed by a barely-significant improvement in "Social Fitness" in Table 4, while no such improvement appears to have taken place in "Family Fitness." Fifth, as pointed out by Gelman (personal communication, June 11, 2013), it is not clear why the overall null hypothesis implied by the way in which the treatment and control group are presented and compared-namely, that the treatment has no effect-is a valid choice (see also Gelman, in press).

Even if these issues are ignored, however, it is clear that the effect sizes of the presence of MRTs are extremely small (partial eta squared values of .002 or less; see Figure 3)[vii]. As Gelman (2009) has pointed out, in any study with a large enough sample size-a description that certainly applies to CSF-one will almost inevitably find a substantial number of small effect sizes with p-values less than .05 among the various subscales. The authors of the report attempted to play down the implications of this by noting that, in public health settings, small changes in overall behavior can have significant global outcomes, but this does not appear to be relevant to CSF where the changes are to self-reported emotions and attitudes, rather than substantive actions such as smoking cessation (Eidelson & Soldz, 2012)

A careful reading of the text of the third report (Lester, Harms, Herian, et al., 2011) reveals two important pieces of information about the state of the CSF program. The first is that once MRTs have been trained, there appears to be no formal tracking of how much training they subsequently provide within their units, or of who attends that training. Information about the training that had been conducted in each unit was collected by a survey of MRTs that revealed that almost a third of those responding had not, in fact, performed any training at all, and that even when they had, the effects were very small (pp. 20-22). In other words, the claimed higher resilience of the "treatment" group-units with MRTs deployed-did not necessarily reflect the outcomes of classroom training delivered by the MRTs, but only their presence in their respective units, as if resilience is something that is likely to be transferred by osmosis[viii]. Nevertheless, the Army News Service felt able to make the confident claim that "The Master Resilience Training aspect of Comprehensive Soldier Fitness is working well" (Vergun, 2012).

The second revelation from Lester, Harms, Herian, et al. (2011), in a footnote on p. 9, is that the Comprehensive Resilience Modules (CRMs) have had no measurable impact; indeed, as of December 2011, these modules were "undergoing a significant revision." This could be simply because soldiers do not take them very seriously. Anecdotal reports (e.g., anonymous U.S. soldier, personal communication, December 3, 2012) suggest that if the trainee gives an incorrect answer to one of the multiple-choices questions in a CRM, the correct response is indicated and the module continues. Although there are officially no negative consequences for giving too many incorrect responses, it is reported that soldiers regularly collaborate with each other to "succeed" on CRM tests by sitting in a group at multiple workstations, exchanging the correct answers, and taking turns at attempting new questions for the first time. An anonymous member of the CSF team (personal communication, December 12, 2012) indicated that new CRMs were under development, in the hope that this would "right the ship." Lester et al. (2013) reported, in a positive tone, that 160,000 soldiers have taken at least one CRM to date, but given the number of individuals who have been enrolled in CSF (presumably at least 1.1 million, plus however many have been recruited to the Army to replace those who leave), this does not seem to be a very high figure; indeed, taken together with the complete failure of voluntary participation in CSF mentioned by Carr et al. (2013), this would seem to raise questions about soldiers' spontaneous enthusiasm for measures that it is claimed will improve their resilience.

The fourth and, to date, final report on CSF (Harms et al., 2013) concentrated on outcome measures. Specifically, it reported analyses of statistics concerning diagnoses of mental health problems and substance abuse problems across two groups of soldiers, of which only one had undergone resilience training. Although the report by Harms et al. runs to 24 pages, excluding appendices, and reports the analysis methods used in considerable detail, the reader will search in vain for an answer to the straightforward question of whether the two groups differed significantly on the outcome variables, as listed in Table 3 on p. 17. A few moments with a chi-square calculator reveals that for mental health problems, there was no significant difference between the "resilience training" and "no-training" groups, χ² (1, N = 7,230) = 1.428, two-tailed p = .23. For substance abuse, the difference was statistically significant, χ² (1, N = 7,230) = 26.482, two-tailed p < .0001, but the effect size was extremely small (Φ = 0.06). This did not stop the Army's public relations department (e.g., U. S. Army, 2013) from portraying the results as positive and "significant" (with the latter word not being preceded by "statistically").

In summary, of the main components of CSF, the Master Resilience Trainers have had only a very small effect on (self-reported) resilience and in some cases are not even providing any training at all, and the Comprehensive Resilience Modules have performed even worse. Only the GAT continues in its original form, but the combination of skewed data, very small effect sizes, and the various confounding factors described earlier, makes it difficult to assert that soldiers are becoming much more resilient, even as measured by their self-reports; indeed, Harms et al. (2013, p. 21) acknowledged some of the deficiencies of the GAT that were discussed earlier in this article. Meanwhile, since the launch of CSF, combat deployments have continued, with soldiers continuing to be exposed to the kinds of severe psychological and social problems that CSF was intended to address; it ought, therefore, to be possible to see whether the program has had a substantial effect in the areas that make the most difference to soldiers and their families. While rates of incidence of PTSD or family violence among (ex-)soldiers do not appear to be tracked on an ongoing basis in a way that would facilitate longitudinal comparisons, careful records are kept of every case of in-service suicide. However, the results are not encouraging. Briggs (2013a) noted that Army suicides (182) exceeded combat deaths (176) in 2012, with the suicide rate up by at least 54 percent since 2007 (Briggs, 2013b). Similarly, Carr et al. (2013) reported that, following delivery of Master Resilience Training to almost all personnel at a U.S. military facility in Afghanistan, levels of resilience as measured on a well-validated scale (CD-RISC; Connor & Davidson, 2003) actually declined, while reported usage of prescription medication for stress or sleep problems approximately doubled. Of course, it is possible to argue that any or all of these results might have been even worse without CSF, but there certainly does not appear to be any evidence that the program is, as yet, reducing the incidence of suicide or PTSD, or increasing personal resilience. Steenkamp, Nash, & Litz (2013) have argued that CSF developers have as yet not provided any empirical or theoretical basis substantiating the mechanisms that may lead to the prevention or reduction of PTSD. In this regard, it is interesting to compare the barely-hedged rhetoric of Seligman and Fowler (2011) concerning what they implied as the entirely likely beneficial outcomes of CSF with the following sober evaluation from a group of authors who have worked with the program since its inception:

"Comprehensive Soldier Fitness is not a panacea-for anything. The program will not bring about an end to low base rate behavioral problems, such as suicide and violent crime within the Army. It will not cure Posttraumatic Stress Disorder (PTSD). It will not solve the Army's alarmingly high number of soldiers who are prescribed psychotropic medication for behavioral health problems. It will not cure addiction of any kind ... It will not prevent a divorce from happening or make a soldier a great parent." (Lester et al., 2013, p. 271)

Although Lester et al. went on to claim that CSF would help some (unspecified) percentage of soldiers to cope with the problems listed in the paragraph quoted above, the change in tone is palpable. One can only speculate as to whether the Army's senior commanders would have been in such a hurry to roll out the CSF program without pilot testing had that prognosis appeared in the initial proposal document.

Conclusions

CSF is without doubt a highly ambitious program, whose stated aims are (within the political context of which they are a part) difficult to fault per se. However, as discussed above, it is not at all clear that either the science of positive psychology in its current state, or the structure of the program itself, will be able to deliver on CSF's promises. Neither the state of theory nor the evidence base seems to justify the claim that increasing soldiers' resilience will prevent mental health problems among those exposed to the physical horrors and moral dilemmas of combat, and the operational constraints of the Army environment do not seem conducive to performing empirical research on the outcomes of resilience training in a properly scientific manner.

It appears that some of the designers of the program may have been aware of its limitations from the start. Concerns about CSF's potential effectiveness were expressed by Gottman, the designer of the "Family Fitness" component, who remarked, "What these soldiers deal with on a daily basis is not like common, everyday American stress ... We can't hope to solve this problem with a couple of online modules" (Novotney, 2009, p. 40). Bartlett (2011) reported that researchers Park and Algoe also had misgivings about the lack of pilot studies or control groups. Even the recently-retired Brig. Gen. Rhonda Cornum[ix], in a television interview, refused to be drawn on whether CSF could help prevent suicides (British Forces News, 2012).

Since the end of 2012, CSF has been rebranded as "Comprehensive Soldier and Family Fitness" or CSF2; whether the figure 2 represents the extra "F" of Family, or signals a shift to a new "version number," is (perhaps deliberately) unclear. The image of an armed soldier on the logo has been joined by the silhouette of a (nuclear) family, and the strapline has been changed to "Building Resilience and Enhancing Performance" (see Figure 1).

Leipold (2012) reported that the spouses of soldiers are to be trained as MRTs, to be able to pass on the principles of resilience to other families. CSF has a friendly face on social media, with the @ArmyCSF2 account tweeting upbeat messages, accompanied by hashtags such as #ReadyandResilient or #HTGS ("Hunt the Good Stuff") several times per day to its 1,600 followers. A long-running tendering process for a new contractor to supply CSF2 services (which, this time, is on a competitive basis, at least nominally; Federal Business Opportunities, 2013a[x], 2013b, 2013c) may finally reach a conclusion in 2014. CSF/CSF2 is clearly here to stay, despite being criticized for its lack of effectiveness by people considerably more qualified than the present author (Institute of Medicine, 2014). As such, it seems clear that more scrutiny of this project by psychologists is needed, not only to ensure that the quality of the associated science does not begin to take a back seat to the influence of politics on the reporting of outcomes that inevitably accompanies any large-scale, top-down project on whose success numerous livelihoods and professional careers depend, but also to assure the people funding the program (i.e., U.S. taxpayers) that the program represents a worthwhile investment.

References

Advisory Committee on Human Radiation Experiments (1995). Final report. Retrieved from http://www.hss.energy.gov/healthsafety/ohre/roadmap/achre/report.html

Anderson, R. (2004). Home front: The government's war on soldiers: A report on how America's weapons, medicines, and bureaucracies of mass destruction harm our troops and veterans. Atlanta, GA: Clarity Press.

Bartlett, T. (2011, November 4). Soldiers of optimism: Is new Army psychology program simply a shot in the dark? Chronicle of Higher Education, A1, A10-A11.

Benjamin, M. (2010). "War on terror' psychologist gets giant no-bid contract. Salon.com. Retrieved from http://www.salon.com/2010/10/14/army_contract_seligman/

Bonanno, G. A., Westphal, M., & Mancini, A. D. (2011). Resilience to loss and potential trauma. Annual Review of Clinical Psychology, 7, 511-535. doi:10.1146/annurev‑clinpsy‑032210‑104526

Briggs, W. (2013a). Military suicide rate hit record high in 2012. Retrieved from http://usnews.nbcnews.com/_news/2013/01/14/16510852-military-suicide-rate-hit-record-\n high-in-2012

Briggs, W. (2013b). The enemy within: Soldier suicides outpaced combat deaths in 2012. Retrieved from http://usnews.nbcnews.com/_news/2013/01/03/16309351-the-enemy-within-soldier-suicides\n -outpaced-combat-deaths-in-2012

British Forces News [BritishForcesNews] (2012, February 8). Top US soldier on being captured in Iraq 07.02.2012 [Video file]. Retrieved from https://www.youtube.com/watch?v=3MUL3IktYDM

Brunwasser, S. M., Gillham, J. E., & Kim, E. S. (2009). A meta-analytic review of the Penn Resiliency Program's effect on depressive symptoms. Journal of Consulting and Clinical Psychology, 77, 1042-1054. doi:10.1037/a0017671

Carr, W., Bradley, D., Ogle, A. D., Eonta, S. E., Pyle, B. L., & Santiago, P. (2013). Resilience training in a population of deployed personnel. Military Psychology, 25, 148-155. doi:10.1037/h0094956

Casey, G. W. (2011). Comprehensive Soldier Fitness: A vision for psychological resilience in the U.S. Army. American Psychologist, 66, 1-3. doi:10.1037/a0021930

Chronicle of Higher Education (2013). Postdoctoral research fellow for Army data health analysis [job advertisement]. Retrieved July 15, 2014 from http://chronicle.com/jobs/0000788904-01/

Congressional Budget Office (2006). Recruiting, retention, and future levels of military personnel. Retrieved from http://www.cbo.gov/sites/default/files/cbofiles/ftpdocs/76xx/doc7626/10-05-recruiting\n .pdf

Connor, K. M., & Davidson, J. R. (2003). Development of a new resilience scale: The Connor-Davidson Resilience Scale (CD-RISC). Depression and Anxiety, 18, 76-82. doi:10.1002/da.10113

Cornum, R., Matthews, M. D., & Seligman, M. E. P. (2011). Comprehensive Soldier Fitness: Building resilience in a challenging institutional context. American Psychologist, 66, 4-9. doi:10.1037/a0021420

Defense Manpower Data Center (2008). Active duty demographic profile [Electronic slide presentation]. Retrieved from http://www.slideshare.net/pastinson/us-military-active-duty-demographic-profile-prese\n ntation

Dyckman, J. (2011). Exposing the glosses in Seligman and Fowler's (2011) straw-man arguments. American Psychologist, 66, 644-645. doi:10.1037/a0024932

Eidelson, R. J., Pilisuk, M., & Soldz, S. (2011). The dark side of Comprehensive Soldier Fitness. American Psychologist, 66, 643-644. doi:10.1037/a0025272

Eidelson, R., & Soldz, S. (2012). Does Comprehensive Soldier Fitness work? CSF research fails the test. Retrieved from http://www.ethicalpsychology.org/Eidelson-&-Soldz-CSF_Research_Fails_the_Test.pdf

Ellis, A. (1962). Reason and emotion in psychotherapy. Oxford, UK: Lyle Stuart.

Federal Business Opportunities (2010). U--Comprehensive Soldier Fitness, Master Resilience Training: Solicitation number W91WAW-10-R-0060. Retrieved from https://www.fbo.gov/?s=opportunity&mode=form&id=d633d6632aced319716a8a1e4951a8ef

Federal Business Opportunities (2013a). R--HQDA G 3/5/7 Comprehensive Soldier and Family Fitness Program contract solicitation in support of educational services for Master Resilience Training (Levels 1 to 4) and associated educational training materials:Solicitation number W91CRB13R0041. Retrieved July 15, 2014 from https://www.fbo.gov/?s=opportunity&mode=form&id=b6f096870618d4c8102f1ba3890550fa

Federal Business Opportunities (2013b). Solicitation number: W91CRB13R0041. Retrieved July 15, 2014 from https://www.fbo.gov/index?s=opportunity&mode=form&id=cbe706ff0bf0ac222e45fc24a01d1338

Federal Business Opportunities (2013c). U--Notification of Intent to sole source-W91CRB13R0041. Retrieved July 15, 2014 from https://www.fbo.gov/index?s=opportunity&mode=form&id=559506fc34b60ba7fbf2af28d714d761

Fravell, M., Nasser, K., & Cornum, R. (2011). The Soldier Fitness Tracker - Global delivery of Comprehensive Soldier Fitness. American Psychologist, 66, 73-76. doi:10.1037/ a0021632

Friedman, H. L., & Robbins, B. D. (2012). The negative shadow cast by positive psychology: Contrasting views and implications of humanistic and positive psychology on resiliency. The Humanistic Psychologist, 40, 87-102. doi:10.1080/08873267.2012.643720.

Gelman, A. (2009). "The sample size is huge, so a p-value of 0.007 is not that impressive." Retrieved from http://andrewgelman.com/2009/06/18/the_sample_size/

Gelman, A. (in press). The connection between varying treatment effects and the crisis of unreplicable research: A Bayesian perspective. Management Review.

gibbysgarage (2010, July 5). Geico Insurance Drill Sergeant Therapist Commercial [Video file]. Retrieved file://localhost/from http/::www.youtube.com:watch%3Fv=WEbzM2FUP9s

Gillham, J. E., Brunwasser, S. M., & Freres, D. R. (2008). Preventing depression in early adolescence: The Penn Resiliency Program. In J. R. Z. Abela & B. L. Hankin (Eds.), Handbook of depression in children and adolescents (pp. 309-322). New York, NY: Guilford Press.

Grossman, D. (2009). On killing: The psychological cost of learning to kill in war and society. New York, NY: Back Bay.

Harms, P. D., Herian, M. N., Krasikova, D. V., Vanhove, A., & Lester, P. B. (2013). The Comprehensive Soldier and Family Fitness Program evaluation report #4: Evaluation of resilience training and mental and behavioral health outcomes. Retrieved from http://csf2.army.mil/supportdocs/TR4.pdf

Henning, K. R., & Frueh, B. C. (1997). Combat guilt and its relationship to PTSD symptoms. Journal of Clinical Psychology, 53, 801-808.

Hobbs, W. R., & Fowler, J. H. (2014). Prediction of mortality using on-line, self-reported health data: Empirical test of the Realage score. PLoS ONE, 9, e86385. doi:10.1371/journal.pone.0086385

Institute of Medicine (2014). Preventing psychological disorders in service members and their families: An assessment of programs. Washington, DC: The National Academies Press.

Jauregui, B. (in press). World fitness: The US Army family and the positive science of persistent war. Public Culture. Draft retrieved from http://www.academia.edu/6613856/World_Fitness_The_US_Army_Family_and_the_Positive_Sci\n ence_of_Persistent_War

Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16, 606-613. doi:10.1046/j.1525‑1497.2001.016009606.x

Krueger, J. I. (2011). Shock without awe. American Psychologist, 66, 642-643. doi:10.1037/a0025080

Lakens, D., & Evers, E. R. K. (2014). Sailing from the seas of chaos into the corridor of stability: Practical recommendations to increase the informational value of studies. Perspectives on Psychological Science, 9, 278-292. doi:10.1177/1745691614528520

Lehrer, T. (1959). It makes a fellow proud to be a soldier [Song]. Retrieved from http://www.youtube.com/watch?v=J0L_rD7CCe4

Leipold, J. D. (2012). Changes to Comprehensive Soldier Fitness focus on families. Retrieved from http://www.army.mil/article/86008/Changes_to_Comprehensive_Soldier_Fitness_focus_on_f\n amilies/

Leopold, J. (2011). Army's ""spiritual fitness'' test comes under fire. Truthout. Retrieved from http://truth-out.org/news/item/268:army's-"spiritual-fitness"-test-comes-under-fire

Lester, P. B., Harms, P. D., Bulling, D. J., Herian, M. N., & Spain, S. M. (2011a). Evaluation of relationships between reported resilience and outcomes: Report #1: Negative outcomes (suicide, drug use, & violent crimes). Retrieved from http://www.dtic.mil/dtic/tr/fulltext/u2/a538618.pdf

Lester, P. B., Harms, P. D., Bulling, D. J., Herian, M. N., & Spain, S. M. (2011b). Evaluation of relationships between reported resilience and outcomes: Report #2: Positive performance outcomes in officers (promotions, selections, & professions). Retrieved from http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA542229

Lester, P. B., Harms, P. D., Herian, M. N., Krasikova, D. V., & Beal, S. J. (2011). The Comprehensive Soldier Fitness program evaluation: Report #3: Longitudinal analysis of the impact of Master Resilience Training on self-reported resilience and psychological health data. Retrieved from http://dma.wi.gov/dma/news/2012news/csf-tech-report.pdf

Lester, P. B., McBride, S., Bliese, P. D., & Adler, A. B. (2011). Bringing science to bear: An empirical assessment of the Comprehensive Soldier Fitness program. American Psychologist, 66, 77-81. doi:10.1037/a0022083

Lester, P. B., McBride, S., & Cornum, R. L. (2013). Comprehensive Soldier Fitness: Underscoring the facts, dismantling the fiction. In R. R. Sinclair & T. W. Britt (Eds.), Building psychological resilience in military personnel: Theory and practice (pp. 267-309). Washington, DC: APA Press.

Litz, B. T., & Schlenger, W. E. (2009). PTSD in service members and new veterans of the Iraq and Afghanistan wars: A bibliography and critique. PTSD Research Quarterly, 20, 1-7.

Litz, B. T., Stein, N., Delaney, E., Lebowitz, L., Nash, W. P., Silva, C., & Maguen, S. (2009). Moral injury and moral repair in war veterans: A preliminary model and intervention strategy. Clinical Psychology Review, 29, 695-706. doi:10.1016/j.cpr.2009.07.003

MacLeish, K. (2013). Making war at Fort Hood: Life and uncertainty in a military community. Princeton, NJ: Princeton University Press.

Maguen, S., Lucenko, B. A., Reger, M. A., Gahm, G. A., Litz, B. T., Seal, K. H., ...Marmar, C. R. (2010). The impact of reported direct and indirect killing on mental health symptoms in Iraq war veterans. Journal of Traumatic Stress, 23, 86-90. doi:10.1002/jts.20434

Maguen, S., & Litz, B. T. (2012). Moral injury in veterans of war. PTSD Research Quarterly, 23, 1-6.

Masling, J. (1966). Role-related behavior of the subject and psychologist and its effect upon psychological data. In D. Levine (Ed.), The Nebraska symposium on motivation (pp. 67-103). Lincoln, NB: University of Nebraska Press.

Matsumoto, G. (2010). Vaccine A: The covert government experiment that's killing our soldiers-and why GI's are only the first victims. New York, NY: Basic Books.

McCord, J. (2003). Cures that harm: Unanticipated outcomes of crime prevention programs. Annals of the American Academy of Political and Social Science, 587, 16-30. doi:10.1177/0002716202250781

Novotney, A. (2009). With help from psychologists, a new U.S. Army program puts psychological well-being on par with physical fitness. APA Monitor on Psychology, 40(11), 40.

Obama, B. (2009). Executive Order 13526:Classified National Security Information. Retrieved from http://ise.gov/sites/default/files/EO13526.pdf

Orne, M. T. (1959). The demand characteristics of an experimental design and their implications. Paper read at American Psychological Association conference, Cincinnati.

Pargament, K. I., & Sweeney, P. J. (2011). Building spiritual fitness in the army: An innovative approach to a vital aspect of human development. American Psychologist, 66, 58-64. doi:10.1037/a0021657

Paulson, D. S., & Krippner, S. (2010). Haunted by combat: Understanding PTSD in war veterans, including women, reservists, and those coming back from Iraq. Westport, CT: Praeger Security International.

Peterson, C., Park, N., & Castro, C. A. (2011). Assessment for the U.S. Army Comprehensive Soldier Fitness program. American Psychologist, 66, 10-18. doi:10.1037/a0021658

Phipps, S. (2011). Positive psychology and war - An oxymoron. American Psychologist, 66, 641-642. doi:10.1037/a0024933

Reivich, K. J., Seligman, M. E. P., & McBride, S. (2011). Master Resilience Training in the U.S. Army. American Psychologist, 66, 23-34. doi:10.1037/a0021897

Ringwalt, C. L., Ennett, S. T., & Holt, K. D. (1991). An outcome evaluation of Project DARE (Drug Abuse Resistance Education). Health Education Research, 6, 327-337. doi:10.1093/her/6.3.327

Sagalyn, D. (2012). Health experts question Army report on psychological training. Retrieved from http://www.pbs.org/newshour/updates/military/jan-june12/csf_training_01-02.html

Seligman, M. E. P. (2011a). Flourish: A visionary new understanding of happiness and well-being. New York, NY: Free Press.

Seligman, M. E. P. (2011b). Helping American soldiers in time of war: Reply to comments on the Comprehensive Soldier Fitness special issue. American Psychologist, 66, 646-647. doi:10.1037/a0025156

Seligman, M. E. P., & Fowler, R. D. (2011). Comprehensive Soldier Fitness and the future of psychology. American Psychologist, 66, 82-86. doi:10.1037/a0021898

Sellke, T., Bayarri, M. J., & Berger, J. O. (2001). Calibration of p values for testing precise null hypotheses. The American Statistician, 55, 62-71. doi:10.2307/2685531

Sharma, S., & Sharma, M. (2012). Low intensity conflict stress in soldiers: Building coping skills and resources. Psychological Studies, 57, 260-268. doi:10.1007/s12646‑011‑0142‑6

Simms, L. J., Watson, D., & Doebbeling, B. N. (2002). Confirmatory factor analyses of posttraumatic stress symptoms in deployed and nondeployed veterans of the Gulf war. Journal of Abnormal Psychology, 111, 637-647. doi:10.1037/0021‑843X.111.4.637

Smith, G. C. S., & Pell, J. P. (2003). Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. British Medical Journal, 327(7429), 1459-1461. doi:10.1136/bmj.327.7429.1459

Smith, R. A. (2006). The utility of force:The art of war in the modern world. London, UK: Penguin.

Smith, S. L. (2013). Could Comprehensive Soldier Fitness have iatrogenic consequences? A commentary. Journal of Behavioral Health Services & Research, 40, 242-246. doi:10.1007/s11414‑012‑9302‑2

Steenkamp, M. M., Nash, W. P., & Litz, B. T. (2013). Post-Traumatic Stress Disorder: Review of the Comprehensive Soldier Fitness program. American Journal of Preventive Medicine, 44, 507-512. doi:10.1016/j.amepre.2013.01.013

Teasdale, B., Stephens, P. C., Sloboda, Z., Grey, S. F., & Stephens, R. C. (2009). The influence of program mediators on eleventh grade outcomes for seventh grade substance users and nonusers. Drug and Alcohol Dependence, 102(1-3), 11-18. doi:10.1016/j.drugalcdep.2008.11.010

Tedeschi, R. G., & McNally, R. J. (2011). Can we facilitate posttraumatic growth in combat veterans? American Psychologist, 66, 19-24. doi:10.1037/a0021898

U.S. Army (2009). Justification review document for other than full and open competition: U.S. Army Master Resiliency Trainer program. Retrieved from https://www.fbo.gov/utils/view?id=f58e39c7f8a259b848fba32e52eed57a

U. S. Army (2013, April 18). Stand-To: Today's focus: Comprehensive Soldier and Family Fitness Technical Report #4. Retrieved from http://www.army.mil/standto/archive_2013-04-18/

U.S. Department of Defense (2002). Instruction number 1308.3: DoD physical fitness and body fat programs procedures. Retrieved from http://www.dtic.mil/whs/directives/corres/pdf/130803p.pdf

Vergun, D. (2012). Study concludes Master Resilience Training effective. Retrieved from http://www.army.mil/article/72431/Study_concludes_Master_Resilience_Training_effectiv\n e/

Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063-1070. doi:10.1037/0022‑3514.54.6.1063


[i] The waiver document (U.S. Army, 2009), reference code CSFMRTJA, specified a contract value of $665,760, although the announcement of the contract awarded by the Department of Defense to the PPC (Federal Business Opportunities, 2010) specify a three-year value of $31,350,000. It appears that a separate waiver document may exist to justify the continuing waiver of competitive tendering requirements despite a near 50-fold increase in the value of the contract-the contract announcement refers to code CSFMSTJA, but this is not available for download. Interestingly, no notification of the contract (number W91WAW-10-D-0018) has ever been published on the DoD's contract awards website (http://www.defense.gov/contracts/archive.aspx), despite its value considerably exceeding the $6.5 million requirement for contracts to be so disclosed. All this is, of course, a peripheral matter, principally of interest to U.S. taxpayers (a group of which the author is not a member).

[ii] Soldiers wishing to improve their "Spiritual Fitness" within the CSF program are directed to http://www.spiritfit.army.mil/. Here they can indicate their "Religious Preference", after which they are directed to appropriate "Resources". It is interesting to compare the results of choosing, say, "Christian"/"Catholic", versus "Other"/"Atheist". The first leads to a page, hosted on the same "Spiritual Fitness" site, containing many links of interest to Catholics. The second leads directly to a single external page (i.e., outside the .mil Internet domain, at the popular general-interest site wikihow.com), entitled "How to become [sic] an Atheist". Other non-Christian options for "Religious Preference" include "Dont'Know" [sic], which leads to a page entitled "What Kind of Christian Are You?"; "No Preference", which leads to a quiz called "Belief-O-Matic"; and "None", which leads to an intermediate page containing a link entitled "Go To The Spiritual Fitness Module In Comprehesive" [sic]. This part of the "Spiritual Fitness" site appears to have been written in considerable haste, and without substantial input from experts in comparative religious studies.

[iii] The U.S. Air Force's (USAF) equivalent resilience program, "Comprehensive Airman Fitness" (CAF), has only four "fitnesses", with the Army's "Family fitness" being apparently considered superfluous or at least expendable (which could be considered slightly problematic in view of the Army's subsequent decision to particularly emphasize the "Family" dimension in a later revision of CSF, as described elsewhere in this article). The logo for CAF (see Figure 1) is centered around the USAF "Prop and Wings" emblem, which conveniently divides its surrounding circle into four parts, thus again allowing one "fitness" to fit into each part. It would, of course, be entirely mischievous to suggest that either service might have allowed graphical convenience to influence the choice of the number of dimensions of its respective resilience program.

[iv] In late 2013, the GAT was updated as "GAT 2.0." An official website at http://csf2.army.mil/takethegat.html documents the enhancements that have been made in this version. It is unclear whether any of the issues described in this article have been addressed by this update. An interesting new feature is RealAge¨ [sic], described by the above-mentioned website as "a metric that looks at all five dimensions of strength and tells you your biological age compared to your calendar age." This metric appears to be a proprietary development by a company co-owned by the popular U.S. television doctor, Dr. Mehmet Oz; see Hobbs and Fowler (2014) for a discussion of the empirical validation of RealAge.

[v] An additional potential cause of skepticism among soldiers is the U.S. Army's history of conducting various forms of experimental research on its personnel without always taking the trouble to inform, let alone obtain consent from, the individuals concerned (e.g., Advisory Committee on Human Radiation Experiments, 1995, Chapter 10). Whatever the truth of claims made in popular works by authors such as Anderson (2004) and Matsumoto (2010) that these experiments continue to the present day, the average soldier could be forgiven for wondering whether CSF is just an experiment in manipulation, regardless of the true intentions of its instigators and managers.

[vi] As Jauregui (2013, note 24) has pointed out, the GAT Terms of Use explicitly state that the US Government may "inspect and seize data stored on this [system]" and that "data stored ... are subject to routine monitoring, interception, and search, and may be disclosed or used for any USG authorized purpose". Although a later clause attempts to provide a specific confidentiality exception for data "related to personal representation or services by attorneys, psychotherapists, or clergy, and their assistants", it is not at all clear whether GAT data is subject to the psychotherapist-client privilege.

[vii] One of the effects claimed as significant by Lester, Harms, Herian, et al. (2011), namely "Social Fitness" (line 10 of their Table 4, p. 15) is associated with a p-value that is reported as being .050. As Lakens and Evers (2014) have pointed out, the high statistical power associated with the large sample size in this case means that such a p-value actually constitutes evidence in favor of the null hypothesis; see also Sellke, Bayarri, & Berger (2001).

[viii] The extent to which individual MRTs have fully absorbed the implications of the demanding role to which they have been assigned-which would appear, potentially at least, to be that of de facto psychological counselor to the soldiers in their unit-is not clear. There seems to be no mention in the CSF literature of any pass/fail criteria for the two-week training program that MRTs undergo.

[ix] Brig. Gen. Cornum's current job title is "Director of Health Strategy" at TechWerks, LLC (Pensacola, Florida), the company that provides the GAT technical infrastructure. The founder, President, and CEO of TechWerks, retired Lt. Col. Mike Fravell, was, according to his own LinkedIn profile, the "senior advisor to the Army Chief of Staff on the development and implementation of an information technology strategy for Comprehensive Soldier Fitness" until he retired from the Army in August 2009.

[x] Amusingly, the Q&A section of this web page discloses that the first version of the "Performance Work Statement" (PWS) document (which specifies the work to be performed under the contract) to appear online invited potential suppliers to confirm that they could deliver services in Philadelphia, PA, a city with no U.S. Army base within 30 miles, which-presumably coincidentally-just happens to be the location of the current supplier, the PPC.

Reviews

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.