Ann Brunet v. City Of Columbus, Ohio

58 F.3d 251 68 Fair Empl.Prac.Cas. (BNA) 518 Ann BRUNET, Denise Sachs, on behalf of themselves and the class they represent, Plaintiffs-Appellants, Guy E. Tucker, James T. Meyer, Stuart J. Tudor, Joseph S. Hilleary, on behalf of themselves and the class they represent, Plaintiffs/Intervenors-Appellees, v. CITY OF COLUMBUS, OHIO; Columbus Civil Service Commission; Dana Rinehart; Alphonse Montgomery, Defendants-Appellees. No. 94-3072. United States Court of Appeals, Sixth Circuit. Argued May 22, 1995. Decided June 28, 1995. Rehearing and Suggestion for Rehearing En Banc Denied Aug. 11, 1995.* Alexander M. Spater (briefed), Kathaleen B. Schulte (argued and briefed), Spater, Gittes, Schulte & Kolman, Columbus, OH, for plaintiffs-appellants. Richard A. Frye (argued and briefed), Schwartz, Kelm, Warren & Ramirez, Columbus, OH, for intervenors-appellees. Ronald J. O'Brien (argued and briefed), City Attorney's Office for the City of Columbus, Columbus, OH, for defendants-appellees. Before: KENNEDY, WELLFORD, and MILBURN, Circuit Judges. MILBURN, J., delivered the opinion of the court, in which KENNEDY, J., joined. WELLFORD, J. (pp. 14-20), delivered a separate dissenting opinion. MILBURN, Circuit Judge. — 1 — Plaintiffs Ann Brunet and Denise Sachs, female firefighters with the Columbus Division of Fire ("CDF"), and the class they represent ("the Brunet plaintiffs"), appeal the district court's order, entered upon remand from this court, that sets forth the basis upon which the defendants City of Columbus, Ohio; Columbus Civil Service Commission; Dana Rinehart; and Alphonse Montgomery (collectively "the City") may hire firefighters in this action brought under Title VII of the Civil Rights Act of 1964, 42 U.S.C. Secs. 2000e et seq. On appeal, the issues are (1) whether the district court improperly narrowed the scope of the issue on remand, and (2) whether the district court committed clear error in finding that sufficient evidence supported the City's use of rank-order hiring based on equal weighting of an applicant's scores on a cognitive ability test ("CAT") and a physical capability test ("PCT"). For the reasons that follow, we affirm. I. — 2 — The facts and proceedings leading up to this appeal are thoroughly discussed in our opinion in Brunet v. City of Columbus, 1 F.3d 390 (6th Cir.1993), cert. denied, --- U.S. ----, 114 S.Ct. 1190, 127 L.Ed.2d 540 (1994) ("Brunet I "). Accordingly, we shall set forth only the facts that are relevant to the specific issues involved in this appeal. — 3 — This case was filed in 1984 by female firefighter applicants who alleged that the City of Columbus, Ohio's entry level firefighter examination illegally discriminated against them on the basis of gender. At that time, applicants were ranked based on their combined scores on the CAT and the PCT, weighted at 70% and 30%, respectively. In 1986, the district court held that the 1984 PCT violated Title VII because it had a disparate impact on female applicants and because the City had failed to demonstrate that the PCT, as it was then designed, was job related. Thus, the district court enjoined the City from hiring any new firefighters until a new PCT could be validated and enjoined the use of rank-order hiring until this selection method could be validated. Brunet v. City of Columbus, 642 F.Supp. 1214, 1249-50 (S.D.Ohio 1986), appeal dismissed, 826 F.2d 1062 (6th Cir.1987), cert. denied, 485 U.S. 1034, 108 S.Ct. 1593, 99 L.Ed.2d 908 (1988). — 4 — In response, the City developed a new 1986 PCT and submitted to the district court a content-valid study that verified that the skills tested by the PCT were important to a firefighter's job. On May 21, 1987, the district court held that the new PCT was not content valid because it included a task that had an adverse impact on female applicants and that was not critical to the performance of a firefighter, namely, the hose hoist event. The district court further held that absent this noncritical task, the 1986 PCT was content valid, and it directed the City to hire male and female firefighters in proportion to their numbers in the pool of candidates that passed the firefighter examination, scoring the PCT on a pass-fail basis. The parties appealed the district court's rulings, and while these appeals were pending, the City initiated settlement discussions with the Brunet plaintiffs that culminated in a February 27, 1989 consent decree. The consent decree provided that both the CAT and the PCT would be scored on a pass-fail basis and that male and female applicants would be hired in proportion to the number of applicants of each gender who passed the test. — 5 — On July 26, 1991, a class of white male applicants referred to as the Tucker plaintiffs filed a complaint challenging the constitutionality of the consent decree on equal protection grounds. On March 18, 1992, the district court held that the consent decree was unconstitutional and enjoined the City from selecting firefighters in the manner provided by the consent decree. The City informed the district court that in the absence of the consent decree, it planned to use a gender-neutral, strict rank-order selection process. The district court determined that in order for the City to use rank-order selection, it had to present evidence showing a linear relationship between the test scores on the PCT and job performance. In response, the City submitted a concurrent criterion-related study prepared by Dr. Frank J. Landy, an industrial psychologist. Based on this study, the district court determined in its July 24, 1992 memorandum opinion that there was a correlation between a higher score on the PCT and better job performance. Thus, the district court validated the use of strict rank-order selection based on equal weighting of the PCT and the CAT. — 6 — In Brunet I, the Brunet plaintiffs appealed, among other things, the district court's order validating the City's use of strict rank-order selection. They argued that the PCT was not appropriate for ranking purposes and that comparable selection devices existed that were as "substantially equally valid" as strict rank-order hiring and that would have less of a disparate impact on female applicants. We reversed the district court's order validating rank-order hiring and remanded the case, requiring the district court to determine whether there was any support for the equal weighting of the CAT and the PCT, and if so, whether the City explored alternative weightings of the CAT and the PCT to determine if any alternative weighting was substantially equally valid and had a less adverse impact on female applicants. — 7 — On remand, the district court stated that "[t]he remand is limited to the issue of alternatives to strict rank ordering based on an equal weighting of the CAT and PCT.... It is apparent that the remand resulted from the appellate court's inability to find anything in the record justifying the equal weighting of the CAT and the PCT." J.A. 172. In this connection, the district court entered an order on January 5, 1994, in which it concluded that there was sufficient evidence in the record to support strict rank ordering based on equal weighting of the CAT and the PCT and that rank ordering based upon a different weighting of the CAT and the PCT would be inappropriate. This timely appeal followed. II. A. — 8 — The Brunet plaintiffs argue that the district court improperly narrowed the scope of the issue on remand from this court. Specifically, they argue that the district court erred when it failed on remand to require the City to demonstrate that it had explored and considered alternatives to strict rank-order hiring that might have a less adverse impact on female applicants. The district court rejected the Brunet plaintiffs' interpretation of the issue on remand because such an interpretation "would in effect completely reopen the litigation and that would be contrary to the language and spirit of the court of appeals' opinion." J.A. 171. — 9 — Pursuant to the mandate rule, lower courts must "adhere to the commands of a superior court." United States v. Moored, 38 F.3d 1419, 1421 (6th Cir.1994). — 10 — "[U]pon remand of a case for further proceedings after a decision by the appellate court, the trial court must 'proceed in accordance with the mandate and the law of the case as established on appeal.' The trial court must 'implement both the letter and the spirit of the mandate, taking into account the appellate court's opinion and the circumstances it embraces.' " — 11 — Moored, 38 F.3d at 1421 (quoting United States v. Kikumura, 947 F.2d 72, 76 (3d Cir.1991) (citations omitted); see also Jones v. Lewis, 957 F.2d 260, 262 (6th Cir.), cert. denied, --- U.S. ----, 113 S.Ct. 125, 121 L.Ed.2d 80 (1992); Petition of United States Steel Corp., 479 F.2d 489, 493 (6th Cir.), cert. denied, 414 U.S. 859, 94 S.Ct. 71, 38 L.Ed.2d 110 (1973). In Brunet I, we reviewed the district court's determination that "there were no alternative selection devices [to strict rank-order hiring] that have less of a disparate impact on females and are 'substantially equally valid' methods of choosing qualified firefighters." Brunet I, 1 F.3d at 411. In reversing the district court's order validating strict rank-order hiring, we stated: — 12 — [U]nder the Uniform Guidelines, 29 C.F.R. Sec. 1607.3 B, before implementing a process of strict rank-order hiring from one list, a process that has an adverse impact upon women, the City is obligated to conduct its own investigation of viable alternatives with lesser or no impact on the female applicants. "[B]efore utilizing a [selection] procedure that has an adverse impact on minorities, the City has an obligation pursuant to the Uniform Guidelines to explore alternative procedures and to implement them if they have less adverse impact and are substantially equally valid to rank ordering." Officers for Justice v. Civil Service Commission, 979 F.2d 721, 728 (9th Cir.1992) (citing the Uniform Guidelines, 29 C.F.R. Sec. 1607.3 B). There is nothing in the record indicating the City explored alternatives to strict rank ordering or that the District Court looked to see that the City had done so. We believe that this was error. — 13 — Brunet I, 1 F.3d at 412 (emphasis in original). However, our advice to the district court did not stop there. We went on to offer the following specific guidance: — 14 — We reiterate that a selection procedure that ranks only on the basis of CAT scores is not acceptable. However, we have found nothing in the record that requires the CAT and the PCT to be weighted equally. In 1980 and 1984, the PCT constituted 30 percent of an applicant's score; the CAT weighed in at 70 percent. The City should be required to demonstrate why the CAT, which arguably is more predictive than the PCT, should not be weighted more than the PCT. We suspect that this change would result in less of an adverse impact on women, although it will not eliminate it. — 15 — Brunet I, 1 F.3d at 412 (emphasis in original) (footnote omitted). In a footnote to this passage, we observed, "It may be that in the voluminous record [the issue of equal weighting of the CAT and the PCT] was addressed, but we are unable to find it." Id. at 412 n. 10. — 16 — Our opinion in Brunet I, when read in its entirety, makes clear that on remand the district court was to determine whether there was any support for the equal weighting of the CAT and the PCT, and if so, whether the City explored alternative weightings of the CAT and the PCT to determine if any alternative weighting was substantially equally valid and had a less adverse impact on female applicants. Our opinion in Brunet I and the procedural history of this case permit no other conclusion. Any other interpretation would make our specific guidance with respect to the weightings surplusage. — 17 — Moreover, it was not this court's intent that the district court be required to consider all possible alternative hiring procedures because many such procedures had already been considered during this case's earlier proceedings. In fact, in Brunet I, we held that the district court did not err in determining that the alternatives to rank-order hiring proposed by the Brunet plaintiffs (i.e., eliminating the PCT or scoring the PCT on a pass-fail basis) were not substantially equally valid. — 18 — In conclusion, our mandate cannot be read to require the City to go back to the drawing board to evaluate alternative hiring procedures that the City has not already used or that the Brunet plaintiffs had not already proposed. Instead, our intent was that the district court require the City to demonstrate that alternative weightings of the CAT and the PCT had been evaluated and that there was support for the conclusion that no other weighting which was substantially equally valid had a less adverse impact on female applicants. Accordingly, the district court did not err in defining the scope of the issue on remand. B. — 19 — The Brunet plaintiffs also argue that the district court erred in concluding that there was sufficient evidence to support the equal weighting of the CAT and the PCT. We review a district court's findings of fact for clear error. Anderson v. City of Bessemer City, 470 U.S. 564, 573, 105 S.Ct. 1504, 1511, 84 L.Ed.2d 518 (1985); Brunet I, 1 F.3d at 410. Under this standard of review, "[w]here there are two permissible views of the evidence, the factfinder's choice between them cannot be clearly erroneous." Anderson, 470 U.S. at 574, 105 S.Ct. at 1511. — 20 — The district court found that there was sufficient evidence to support the equal weighting of the CAT and the PCT. It relied on the 1985-1986 Job Analysis prepared by Dr. Frank J. Landy ("the Landy Job Analysis"), an expert in industrial psychology, and Dr. Landy's testimony at the May 21, 1992 hearing wherein he explained the rationale for equal weighting. The Landy Job Analysis states: Overview — 21 — As a result of this job analysis, it was possible to develop a test plan that covered both the physical and cognitive abilities required for the job of Columbus firefighter. As can be seen from the data in Appendix M, approximately 50% is cognitive in nature. — 22 — J.A. 173 (January 5, 1994 Order, quoting the Landy Job Analysis). The relevant portion of Dr. Landy's testimony at the May 21, 1992 hearing was as follows:BY MR. BROWELL: — 23 — Q. Dr. Landy, regarding the matter of how you weight P.C.T. scores with the C.A.T. scores to come up with the total scores, to the best of my recollection, Dr. Cranny characterizes that as arbitrary, 50-50 calculation or set up was arbitrary. — 24 — Is that true in your opinion? — 25 — A. Absolutely not. The evidence in Brunet is exhaustive with respect to how we came from the job analysis and the ability analysis in which we weighted very, very closely-- — 26 — MR. SPATER: Your Honor, this is beyond the scope. — 27 — THE COURT: No, you got into this. Overruled. — 28 — THE WITNESS:--in which we weighted very closely each of the job dimensions in terms of its importance, and that 50-50 split was a very carefully calculated, not arbitrary, capricious or casual number, and anything that disturbs it, which changed it to 60-40 or 70-30 would be clearly open to objection because it contradicts the job analysis and how we should be testing them. — 29 — J.A. 173 (January 5, 1994 Order, quoting transcript of May 21, 1992 hearing at 82).1 The district court found that this evidence was sufficient to support the equal weighting of the CAT and the PCT. — 30 — (1) — 31 — The Brunet plaintiffs argue that the evidence on which the district court relied is contradicted by the City's own evidence. They point to the Ingram/Kriska Job Analysis ("the Ingram/Kriska report") completed in 1980, which was introduced by the City at the trial in 1986 to justify the change in scoring the PCT from pass-fail to ranking. After reviewing the report, the district court noted that the report's recommendation that applicants be ranked on the basis of their combined scores on the CAT and the PCT was, "at least in part, unsupported by the report," Brunet v. City of Columbus, 642 F.Supp. at 1238, and concluded that the Ingram/Kriska report "only weakly supports one of its central conclusions: that superior physical ability distinguishes superior from average firefighters." Id. at 1240. The district court rejected the report's conclusion because insufficient evidence supported the use of the PCT for ranking purposes, id. at 1238, and it enjoined the use of strict rank-order selection, id. at 1250. — 32 — However, the Brunet plaintiffs' reliance on the district court's early interpretation of this report is misplaced. After the court entered that opinion, and before this court heard the appeal, the parties entered into a consent decree and dismissed the appeal. Thereafter, the Tucker plaintiffs intervened, and the case was reassigned to a different district judge. On July 24, 1992, following a trial, the district court issued an opinion setting aside the consent decree and concluding that the PCT was appropriate for ranking purposes. We affirmed that finding in Brunet I. Brunet I, 1 F.3d at 409-11. Therefore, we reject the Brunet plaintiffs' argument. — 33 — In addition, the Brunet plaintiffs claim that the Concurrent Criterion-Related Validity Report prepared by Dr. Landy ("the Landy report") contradicts the district court's finding that the CAT and the PCT should be equally weighted. Specifically, the Brunet plaintiffs note that the Landy report shows that the CAT is more predictive of job performance than the combined scores of the CAT and the PCT. Thus, they assert that it follows that the CAT should be weighted more heavily than the PCT. — 34 — The Brunet plaintiffs argue that the district court erred in relying on the Landy Job Analysis, a content validity analysis,2 because it was contradicted by the Landy report, a criterion-related validity study.3 They claim that "when the issue at hand is the predictive quality of a testing procedure, evidence which purports to support the content validity of a test must give way to contradictory evidence concerning what distinguishes between superior and average firefighters or predicts better performers." Appellants' Brief at 17. Thus, the Brunet plaintiffs assert that because the question of whether different weightings should be given to the CAT and the PCT goes to the predictiveness of the two tests, it was inappropriate for the district court to rely on a content validity analysis in light of the Landy report.4 — 35 — In concluding that using strict rank-order hiring based on equal weighting of the CAT and the PCT was supported by the record, the district court stated: — 36 — [T]he Court finds by a preponderance of the evidence that the City has justified rank order scoring on the basis of equal weighting of the PCT and CAT because the abilities necessary to properly perform the job of firefighter are evenly distributed between physical abilities and cognitive abilities. The Court further finds that rank ordering based upon a different weighting of the CAT and PCT would violate the concept that the content of the examination should be representative of the content of the job. The Court further finds that a procedure that selects applicants who are ranked by giving the CAT greater weight than the PCT is not substantially equally valid as a procedure that chooses applicants ranked by giving equal weight to the CAT and PCT. — 37 — J.A. 174-75. As earlier stated, in concluding that sufficient evidence supported the equal weighting of the CAT and the PCT, the district court relied, in part, on Dr. Landy's testimony at the May 21, 1992 hearing where Dr. Landy defended equal weightings for the CAT and the PCT as necessary to maintain accuracy. At this hearing regarding the appropriateness of using the PCT scores for ranking purposes, Dr. Landy discussed the results of the Landy report, his criterion-related validity study, and concluded that the evidence required that the CAT and the PCT be weighted equally. Therefore, the record contained the testimony of an expert, who had prepared both a content-validity analysis and a criterion-related validity study, which supported equal weightings of the CAT and the PCT. In light of all of the evidence in the record, including that upon which the district court explicitly relied, we do not believe that the district court's findings are clearly erroneous. — 38 — (2) — 39 — In a related argument, the Brunet plaintiffs urge that this court required the district court to do more than simply find evidence in the record to support equal weighting of the CAT and the PCT and that the district court erred in failing to conduct a hearing to take additional evidence. "[A] district court's decision about whether to reopen the record after a remand should not be disturbed absent an abuse of discretion." Yashon v. Gregory, 737 F.2d 547, 556 (6th Cir.1984); see also State Indus., Inc. v. Mor-Flo Indus., Inc., 948 F.2d 1573, 1577 (Fed. Cir.1991) ("Absent contrary instructions, a remand for reconsideration leaves the precise manner of reconsideration--whether on the existing record or with additional testimony or other evidence--to the sound discretion of the trial court."). — 40 — In Brunet I, we did not require that additional evidence be taken because we recognized that the record might contain evidence supporting the equal weighting of the CAT and the PCT. See Brunet I, 1 F.3d at 412 n. 10. Instead, we directed the district court, which was more familiar with the voluminous record in this case, to ascertain whether there was sufficient evidence to support the equal weighting. We did not preclude the district court from taking additional evidence if it determined that the evidence in the record was insufficient to support the equal weighting, but we did not require the district court to do anything other than point to sufficient evidence in the record, assuming such evidence exists. — 41 — On remand, the district court concluded that "[w]hile the evidence [in the record] is not extensive, it is sufficient to permit this Court to make a finding on the issue which the court of appeals has remanded to it." J.A. 174. As already discussed, we agree that the evidence on which the district court relied was sufficient to dispose of the issue on remand. Therefore, the district court did not abuse its discretion in declining to reopen the record. III. — 42 — For the reasons stated, the district court's order is AFFIRMED. — 43 — WELLFORD, Circuit Judge, dissenting. — 44 — I concur in this difficult case as to the scope of our remand in Brunet I. I believe, however, that there was insufficient evidence to support the district court's finding that ranking applicants based on a hiring device that weighted the CAT more heavily than the PCT was not "substantially equally valid" to ranking applicants based on a hiring device that weights the two tests equally. I therefore DISSENT. I. STANDARD OF REVIEW — 45 — On remand, the district court concluded that "a procedure that selects applicants who are ranked by giving the CAT greater weight than the PCT is not substantially equally valid as a procedure that chooses applicants ranked by giving equal weight to the CAT and PCT." In support of its finding, the district court relied on Dr. Landy's 1985 content validity study1 and Dr. Landy's testimony at an evidentiary hearing on May 21, 1992. At the hearing, Dr. Landy restated his conclusions from the content validity study and explained that the firefighters' test equally weighted the CAT and PCT because the job analysis (i.e., content validity study)2 indicated that the skills necessary to perform the job were equally divided between cognitive and physical attributes. Noting that the fifty-fifty split was not selected arbitrarily, Dr. Landy concluded that, if the CAT was weighted sixty or seventy percent, it "would be clearly open to objection because it [would] contradict[ ] the job analysis." Though both the majority and district court opinions treat Dr. Landy's testimony and content validity study as separate pieces of evidence, I find the substance of the testimony and study to be indistinguishable. Dr. Landy's testimony added nothing to the content validity evidence in the job analysis, which merely showed that physical and cognitive skills are evenly represented in the job of a firefighter. The district court's factual findings cannot be set aside unless clearly erroneous. FED.R.CIV.P. 52(a). Clear error exists " 'when although there is evidence to support [a finding], the reviewing court on the entire evidence is left with the definite and firm conviction that a mistake has been committed.' " Anderson v. City of Bessemer City, 470 U.S. 564, 573, 105 S.Ct. 1504, 1511, 84 L.Ed.2d 518 (1985) (quoting United States v. United States Gypsum Co., 333 U.S. 364, 395, 68 S.Ct. 525, 542, 92 L.Ed. 746 (1948)). This court cannot reverse the district court "simply because it is convinced that it would have decided the case differently." Id. "Where there are two permissible views of the evidence, the factfinder's choice between them cannot be clearly erroneous." Id. at 574, 105 S.Ct. at 1511. Thus, we must affirm if the content validity evidence supports a reasonable inference that giving greater weight to the CAT would render the firefighters' test less predictive of future performance than the present test. II. SCOPE OF REMAND — 46 — At the outset, it is necessary to identify the precise issue that was before the district court on remand. It is undisputed that the PCT has an adverse impact on female applicants when used for rank-order hiring. Thus, under Uniform Guideline Sec. 1607.3 A, the City had to demonstrate that ranking candidates based on their PCT scores was job related. It did so to the district court's satisfaction, and we affirmed. See 1 F.3d 390, 410-11 (6th Cir.1993), cert. denied, --- U.S. ----, 114 S.Ct. 1190, 127 L.Ed.2d 540 (1994). As we noted in Brunet I, however, the inquiry does not end there. Under Uniform Guideline Sec. 1607.3 B, the City had to demonstrate that ranking candidates based on a hiring device that weighted the CAT more heavily than the PCT was not "substantially equally valid" to a hiring procedure that weighted the CAT and PCT equally. — 47 — How, then, could the City satisfy its burden of proof? The district court concluded that the City had justified equal weighting because the evidence in the record showed that "the abilities necessary to properly perform the job of a firefighter are evenly distributed between physical and cognitive abilities." I find the district court's reasoning flawed as a matter of logic. Evidence of content validity only establishes that a hiring device "is representative of important aspects of performance on the job." 29 C.F.R. Sec. 1607.16 D (emphasis added). The Uniform Guidelines provide that — 48 — [e]vidence which may be sufficient to support the use of a selection procedure on a pass/fail ... basis may be insufficient to support the use of the same procedure on a ranking basis.... Thus, if a user decides to use a selection procedure on a ranking basis, and that method of use has a greater adverse impact than use on an appropriate pass/fail basis[,] ... the user should have sufficient evidence of validity and utility to support use on a ranking basis. — 49 — Id. Sec. 1607.5 G. Therefore, where ranking is at issue, the focus is not on whether a hiring device is representative of the job but whether it properly measures those aspects of future performance that distinguish the superior employee from the average employee. See Gilbert v. City of Little Rock, 799 F.2d 1210, 1215 (8th Cir.1986). "A test may have enough validity for making gross distinctions between those qualified and unqualified for a job, yet may be totally inadequate to yield passing grades that show positive correlation with job performance." Guardians Ass'n of New York City Police Dep't v. Civil Serv. Comm'n, 630 F.2d 79, 100 (2d Cir.1980), cert. denied, 452 U.S. 940, 101 S.Ct. 3083, 69 L.Ed.2d 954 (1981). Simply because a hiring procedure facially mirrors the distribution of skills needed to perform a particular job does not necessarily indicate that the selection procedure will differentiate between those skills that contribute most to superior performance. Cf. Williams v. Vukovich, 720 F.2d 909, 924 (6th Cir.1983). Indeed, the Uniform Guidelines require that, "[w]here a selection procedure supported solely or primarily by content validity is used to rank job candidates, the selection procedure should measure those aspects of performance which differentiate among levels of job performance." 29 C.F.R. Sec. 1607.14 C(9). Dr. Landy's content validity study clearly supports the conclusion that only fifty percent of the skills needed to be a firefighter are cognitive. To reiterate, however, simply because cognitive and physical skills are both prerequisites to performance of the job does not negate the possibility that cognitive skills more often distinguish the superior firefighter from the merely qualified firefighter. Therefore, the City could not simply show on remand that the firefighters' test, as it is presently configured, is representative of the job (i.e., content valid). — 50 — Likewise, the City could not rely on evidence which merely established that giving equal weight to the CAT and PCT was valid for ranking purposes. In other words, the City could not simply show that better performance on the present test correlates with better performance on the job. The issue on remand was not whether equal weighting was predictive of job performance, but whether equal weighting was the only weighting that would yield substantially the same degree of predictability. Even if the content validity study conclusively established--and I do not believe it did--that ranking candidates based on a hiring device that gives equal weight to the CAT and PCT is predictive of varying levels of job performance, the study is not probative as to whether ranking applicants based on a hiring device that gives greater weight to the CAT is less predictive. — 51 — Instead, the City had to demonstrate that ranking candidates based on a hiring device that weighted the CAT fifty-five or sixty percent, for example, is not as predictive of future performance as equal weighting. There was but one way to accomplish this end. The City had to offer evidence that cognitive and physical abilities equally contribute to superior performance as a firefighter. III. CONTENT VALIDITY EVIDENCE — 52 — The district court approved equal weighting of the CAT and PCT because it found that giving greater weight to the CAT "would violate the concept that the content of the examination should be representative of the content of the job." Implicit in the district court's conclusion is the assumption that cognitive and physical ability equally contribute to superior performance on the job. Although Dr. Landy's content validity study suggests that cognitive and physical skills are equally represented in the job of a firefighter, it does not demonstrate that cognitive and physical skills equally contribute to superior performance. — 53 — In Brunet I, we approved the use of rank-order hiring based on an applicant's PCT score. See 1 F.3d at 411. In doing so, however, we expressly declined to rely on Dr. Landy's 1985 content validity study. Id. at 410. Instead, we concluded that the results of Dr. Landy's 1987 concurrent criterion-related study3 provided a sufficient statistical correlation between PCT scores and future job performance to permit the City to rank applicants based on their PCT scores. Id. at 411. Thus, the content validity study merely demonstrated that physical and cognitive skills are equally necessary to perform the job of a firefighter. I am aware of no evidence, however, that the job analysis established that physical and cognitive skills equally contribute to superior job performance. I believe that, in Brunet I, we declined to rely on the content validity study to support rank-order hiring, because the job analysis did not indicate which skills are more important in distinguishing superior performance. — 54 — Thus, while evidence of content validity is relevant to show that the firefighters' test is representative of the job, it does not support the district court's conclusion that ranking candidates based on a hiring device that gives greater weight to the CAT than the PCT is less predictive of future performance than the present test. As a consequence, I cannot agree that the content validity evidence was probative on remand. — 55 — I dissent with some reluctance because this controversy has been dragging on for far too long and my disagreement with the majority is on a technical and complex issue. Nevertheless, the City had an affirmative obligation, under Uniform Guideline Sec. 1607.3 B, to demonstrate that giving greater weight to the CAT would render the firefighters' test less predictive of future performance than the present test. Yet, the content validity evidence relied upon by the district court does not support that conclusion. Moreover, the majority acknowledges that there was substantial evidence in the record--evidence which the district court failed to even consider on remand--which tended to support the Brunet plaintiffs' claim that cognitive skills are more predictive of future performance than physical skills. For example, the 1980 job analysis performed by the City's chief of testing (the Ingram/Kriska Job Analysis) and the concurrent-criterion related validity report (Landy Report) seem to show that cognitive skills are more related to superior performance than either physical skills or a combination of both. Indeed, the Landy Report contradicts Dr. Landy's own testimony. — 56 — Despite this empirical evidence, the majority affirms the district court's finding because Dr. Landy testified that assigning greater weight to the CAT would violate the job analysis and, impliedly, render the test unrepresentative of the job. Dr. Landy's testimony would indeed be probative if there was some evidence indicating that the job analysis demonstrated that physical and cognitive skills equally contribute to superior performance as a firefighter. I am unaware, however, of any evidence in the record to that effect. The 1985 job analysis only establishes which skills are prerequisites for the job and does not draw conclusions about which skills distinguish the superior from adequate firefighter. Furthermore, while we agreed in Brunet I that the Landy Report provided a sufficient basis for rank-order hiring from the PCT, we never held that the Landy Report supported equal weighting. Id. In fact, we noted, in ordering remand, that the CAT was arguably more predictive of superior job performance. Id. at 412. — 57 — Under the circumstances, I believe evidence of content validity was not enough. The problem in this case is not with the quantity of the evidence, but rather with the character of the evidence. Absent some proof that the job analysis showed that physical and cognitive ability equally contribute to superior performance, I would remand to the district court with instructions to hold a hearing, at which, the parties could present evidence on the single question of whether cognitive skills disproportionately contribute to superior performance as a firefighter. Footnotes [-s] * Wellford, Circuit Judge, would grant rehearing for the reasons stated in his dissent [1] The Brunet plaintiffs argue that Dr. Landy's testimony at the May 21, 1992 hearing was improper rebuttal testimony that should have been excluded. However, they failed to raise this issue in their first appeal in Brunet I. Therefore, the Brunet plaintiffs did not preserve this issue for appeal, and we need not consider it for the first time at this late date [2] A content validity analysis determines whether a test accurately represents the skills that are important to a particular job [3] A criterion-related validity study examines the correlation between one's score on a test and one's subsequent success on the job (that is, how accurately the test predicts one's performance on the job) [4] The Brunet plaintiffs also argue that the district court's reliance on a content validity analysis was improper because the district court had previously required a criterion-related validity study to validate the use of rank-order hiring [1-1] 1 A content validity study establishes that a hiring device "is representative of important aspects of performance on the job for which the candidates are to be tested." 29 C.F.R. Sec. 1607.5 B (emphasis added) [2-1] 2 The Uniform Guidelines define a job analysis as a "detailed statement of work behaviors and other information relevant to the job." 29 C.F.R. Sec. 1607.16 K [3-1] 3 Criterion-related validity studies show that "the selection procedure is predictive of or significantly correlated with important elements of job performance." 29 C.F.R. Sec. 1607.5 B