Firefighters Institute for Racial Equality v. City of St. Louis

ROSS, Circuit Judge. In this consolidated action, black firefighters and the Department of Justice allege the existence of racially discriminatory practices in the St. Louis City Fire Department. In the first action the Firefighters Institute for Racial Equality (F.I.R.E.) and several named plaintiffs represent a class who are presently employees or who seek employment with the St. Louis Fire Department. The United States subsequently filed suit under Title VII of the Civil Rights Act of 1964 seeking redress for a “pattern or practice” of discrimination as well as for individuals not represented by F.I.R.E. Both actions were filed pursuant to 42 U.S.C. §§ 1981, 1983 and 2000e et seq. The principal defendant, the City of St. Louis, is joined in its argument by the Intervenors who represent a class of nonblack employees and candidates for employment in the fire department. Appeal is taken on a number of issues, but no appeal has been taken with respect to the examination for firefighter which is the entry level position in the fire department.1 The F.I.R.E.-Appellants do contest the City’s promotional practices with regard to the fire captain’s exam and the battalion chief’s exam. Also charged as unlawful are exclusion of blacks from firehouse eating arrangements known as “supper clubs,” and the failure to promote a black individual, George Horne, to a fire captain position. F.I.R.E. also contests the amount of attorney’s fees which the district court indicated it would award. Of these issues, the United States appeals on only two: the promotional exam for fire — 509 — captain and the supper club discrimination issue. This court considers the latter as the principal claims, and reverses with respect to them. The district court is affirmed with respect to the battalion chief exam and in the matter of George Horne. Fire Captain’s Examination The position of fire captain is the first level supervisory job in the St. Louis Fire Department. According to the findings of the district court, fire captains are responsible for the supervision of a group of men and equipment on a particular work shift. The in-service training of the firefighters under his command is a significant part of the fire captain’s job. According to the City’s validation study, the fire captain leads his company at the fire scene. Promotion to the fire captain’s position is dependent on a composite score developed from three measurements of an individual’s qualifications. For candidates on the 1974 eligibility list which is at issue here, the “written test” was given a 45 percent weight as was an “experience and training score.” The “service rating” score was weighted as 10 percent of the composite. Attaining the rank of fire captain is a highly sought-after and competitive goal of both blacks and whites. From the 1974 procedure now under scrutiny a total of 453 persons are seeking the higher position of captain, while only approximately 18 persons are needed to fill vacancies during the two-year life of the eligibility list. The highest ranking black man ranks as number 55 out of a total of 189 on the list. As a prerequisite for consideration all applicants for fire captain must have served five years as a firefighter.2 The experience and training score, which comprised 45 percent of the total score, is also a function of length of service with the fire department. According to the district court, points are awarded for each month of experience with the department. In 1974 all applicants with ten or more years experience received the maximum score of 45 points. Eighty-one percent of those who made the 1974 eligibility list for the captain’s position received the maximum number of points for experience and training. While the experience and training score is largely quantitative, the “service rating” is qualitative, and reflects the individual’s last supervisory rating prior to announcement of the written exam. On this measurement of qualification, 93 percent of the persons on the eligibility list received scores in the narrow range between 7.8 and 9.2. At trial Dr. O’Leary, the City’s test analyst and expert witness, testified that these two work-related ratings were included because he felt that experience on the job and the quality of that experience were important factors in evaluating potential fire captains.3 It appears that as a practical matter these two scores carry less weight than their assigned value indicates. For many of those 189 persons who made the eligibility list, these two experience-based scores are closely clustered, making the written exam of much greater weight in determining final rank than the allotted weight of 45 percent. It is not disputed here that the 1974 written exam for the fire captain’s position adversely affected black candidates as a whole. The district court concluded that — 510 — the statistical evidence presented established that the test had a disparate impact on blacks.4 It is a distinguishing feature of a Title VII cause of action that discriminatory impact suffices to establish a prima facie showing of discrimination. The recent case of Washington v. Davis, 426 U.S. 229, 239, 96 S.Ct. 2040, 48 L.Ed.2d 597 (1976) establishes that a law or other official act is not unconstitutional solely because it has a racially disproportionate impact regardless of whether it reflects a racially discriminatory purpose. However, Congress’ statutory standard for Title VII, where discriminatory purpose need not be proved, is unshaken by the Washington decision. Id. at 246-47, 96 S.Ct. 2040.5 It is now a familiar principle that Title VII was not meant to preclude the use of testing devices, and that what is forbidden is the controlling use of such tests “unless they are demonstrably a reasonable measure of job performance.” Griggs v. Duke Power Co., 401 U.S. 424, 436, 91 S.Ct. 849, 856, 28 L.Ed.2d 158 (1971). Once a racially adverse impact is demonstrated, the burden of proof shifts to the employer to prove the job relatedness of the exam he has utilized. Albemarle Paper Co. v. Moody, 422 U.S. 405, 425, 95 S.Ct. 2362, 45 L.Ed.2d 280 (1975). Accepted professional methods of “validating” exams for their job-relatedness are found in the EEOC Guidelines published in 29 C.F.R. § 1607.5 (1975). The Supreme Court has said of these test validation techniques: The EEOC Guidelines are not administrative “regulations” promulgated pursuant to formal procedures established by the Congress. But, as this Court has heretofore noted, they do constitute “[t]he administrative interpretation of the Act by the enforcing agency,” and consequently they are “entitled to great deference.” (Citation omitted). Albermarle Paper Co. v. Moody, supra, 422 U.S. at 431, 95 S.Ct. at 2378. Though it has been argued here that the EEOC Guidelines, which refer to the standards of the American Psychological Association (APA), should be considered “guidelines only ” these standards have often been sanctioned as a means by which courts may professionally evaluate the validity of employment tests when called upon to do so.6 It is also true that in this case Dr. O’Leary, the City’s expert who developed and validated the test, purportedly considered and reviewed the APA and EEOC publications as guidelines in preparing the examination. Accepted validation techniques under these standards include two forms of criterion-related validity, plus the content and construct validity methods. Criterion-related studies involve the correlation of job — 511 — performance with success on an examination. Predictive validation requires a comparison between an applicant’s test scores and subsequent on-the-job performance as an employee; concurrent validation methods correlate the test scores of present employees vis-a-vis their present job performance. Vulcan Society of New York City Fire Dept. v. Civil Service Commission, 490 F.2d 387, 394 (2d Cir. 1973). These empirical methods are, of course, dependent on statistical correlations as proof of reliability and validity. Content validity, the technique chosen by Dr. O’Leary for justification of the fire captain’s exam, generally requires that the examination reflect a representative sample of the knowledge or behavior that will be used in performance of the job. The F.I.R.E.-Appellants have argued on this appeal that a content validity study should not have been undertaken by the City and that this type of test should be used only when a criterion-related study has proved to be technically infeasible.7 This argument is now undermined by the Supreme Court’s recent observation in Washington v. Davis, 426 U.S. 229, 96 S.Ct. 2040, 48 L.Ed.2d 597 (1976) that “[i]t appears beyond doubt by now that there is no single method for appropriately validating employment tests for their relationship to job performance.” Id. at 247 n. 13, 96 S.Ct. at 2051 (emphasis added).8 Although in this court’s opinion content validation, if properly done could be an acceptable means of evaluation for the City to undertake, it is no more acceptable than a criterion-related test, especially if such criterion-related test involves concurrent validation testing methods with present fire captains. Constructing a content valid exam and proof of its validity requires as a first step a thorough analysis of the job to be performed. The district court concluded, and this court does not challenge the finding, that Dr. O’Leary’s analysis of the fire captain’s job was thorough and complete.9 It is in fact the fatal flaw in the validation study that the test Dr. O’Leary devised did not reflect his findings in the job analysis. The captain’s exam admittedly failed to test the one major job attribute that separates a firefighter from a fire captain, that of supervisory ability. From the interviews conducted for the job analysis, the City’s expert determined that almost 43 percent of a fire captain’s time was spent in supervision, a higher percentage of time than on any other single element. Of the six tasks ranked as the “most important,” supervision was fourth. There was no attempt to test supervisory skills prior to selecting the new captains. The City admits this lack of proper testing for supervisory ability but claims the best method of — 512 — doing it, through an Assessment Center approach, is too expensive. The EEOC Guidelines, accept evidence of content validity for tests “that consist of suitable samples of the essential knowledge, skills or behaviors composing the job in question.” 29 C.F.R. § 1607.5(a) (1975) (emphasis added). Similarly, the APA Standards, which the EEOC Guidelines refer the reader to, clearly warn that: An employer cannot justify an employment test on grounds of content validity if he cannot demonstrate that the content universe includes all, or nearly all, important parts of the job. American Psychological Association, Standards for Educational and Psychological Tests 29 (1974) (emphasis added). It is clear that this court’s objection to this test is similar to the objections of other courts. In a decision where Blacks and Hispanics challenged the content validity of a correction officer’s exam, the court also questioned the test’s lack of comprehensiveness: More serious perhaps than specific item flaws is the fact that, regardless whether 34-944 adequately tests the attributes it is intended to measure, it fails to examine a number of traits, skills and abilities which witnesses for both sides singled out as important to the Sergeant job. Among these are leadership, understanding of inmate resocialization, ability to empathize with persons from different backgrounds, and ability to cope with crisis situations. Kirkland v. New York State Dept. of Correctional Services, 374 F.Supp. 1361, 1378 (S.D.N.Y.1974), aff'd in relevant part, 520 F.2d 420 (2d Cir. 1975). In Vulcan Society of New York City Fire Department, Inc. v. Civil Service Commission, supra, 360 F.Supp. at 1274, aff’d, 490 F.2d 387 (2d Cir. 1973) the court quite simply stated: “[a]n examination has content validity if the content of the examination matches the content of the job.”10 Though the district court was “convinced” from the job analysis that the “areas tested sufficiently identify suitable candidates for promotion,” it is this court’s opinion that an erroneous legal standard was applied in reaching that conclusion. The job analysis here may have appeared impressive in relation to those challenged in other cases, but a good analysis in any situation is of little use when the examination fails to reflect what is found in the job analysis. The test is not content valid. In short, even a common sense concept of content validity, aside from EEOC and APA Standards, requires that an important and distinguishing attribute be tested in some manner to find the best qualified applicants. Here, where the exam failed to test a job component comprising over 40 percent of the employee’s time, the inference of discrimination has not been rebutted with a finding of the exam’s “job-relatedness.” Both experts agree that there is no good pen and paper test for evaluating supervisory skills. In his validation study Dr. O’Leary had anticipated that supervisory ability would be judged after an employee had been selected and placed on the job through the use of a “six months working test period.” According to the validation study, the individual’s supervisory abilities would be “closely scrutinized” and his performance evaluated on a pass/fail basis. Whatever merit this idea may have as a means of eliminating unfit employees after they are chosen, it cannot substitute for a valid selection method utilized at the outset to fairly pick the best employees in a nondiscriminatory manner. The many who are — 513 — not picked for the “working test period” obviously have no opportunity to compete or to raise their rank by a demonstration of their ability. Significantly, the City’s director of personnel testified that though the working test portion has “always existed” he did not know whether a fire captain had ever been eliminated because of his performance during that period. Used in this manner, the probationary period would prove even less valuable as a means of selection. Both experts also testified at trial concerning an excellent method of supervisory evaluation known as the Assessment Center technique. Dr. O’Leary had himself used it for the deputy and fire chief examination in St. Louis, labeling it as “one of the most effective methods” available. He described the Assessment Center as a selection procedure which uses individual and group exercises that simulate job responsibilities while assessors evaluate a candidate’s performance. The Assessment Center was apparently rejected for choosing fire captains because of the large number of persons who wish to take that test. The evidence indicates that the assessment technique takes at least one day and costs as much as $500 per person. Dr. Barrett, who is enthusiastic about this approach as well, has suggested a means for reducing the expense through the use of a content valid screening test. As the court understands his testimony, the written test would be a screening device only, eliminating those persons who obviously did not possess the requisite job knowledge to perform at the captain’s level. The Assessment Center could then be used to rank those persons who successfully complete the written exam. Dr. Barrett, however, also testified that this cutoff score would need to be relatively low. As Dr. Barrett testified, another possible screening device is performance ratings given from the lower level job. This court clearly does not have enough evidence in the record or testing expertise to devise a complete remedy for testing supervisory skills using the Assessment Center or any other method. Other courts have dealt with the problem and in similar causes of action concerning invalid exams have directed “executive or administrative officials to live up to their responsibilities and to prepare and conduct an examination consonant with the Fourteenth Amendment.” Vulcan Society of New York City Fire Dept., Inc. v. Civil Service Commission, supra, 360 F.Supp. at 1278. Other courts have also dealt with the necessity of testing supervisory skill where that attribute was critical. In affirming the district court’s rejection of New York City’s exam for school principals, the court of appeals said: The [district] judge did not outlaw other written examinations or indicate that none could be created to test more fairly the qualities necessary for a supervisory job. It may well be that new testing procedures will be devised by the parties themselves and be approved by the district court. Chance v. Board of Examiners, 458 F.2d 1167, 1179 (2d Cir. 1972) (footnotes omitted). This court will take the same general approach of urging the parties on remand to devise a test of supervisory skills to be approved by the district court. The one caveat is that the final test must be validated in accordance with the published EEOC Guidelines. This may be accomplished by devising a content valid test or by a concurrent criterion-related validity test. The Assessment Center is a concededly good device, but the court will not at this point require that approach to be the sole method finally used. Whatever test is used should provide equal reliability and validity. Because of the difficulty of devising a test properly reflecting the supervisory skills of the applicants, it is possible that a criterion-related concurrent validation test coupled with a limited use of the Assessment Center would meet the guidelines. Cost to the City is one factor which may be considered in deciding whether to use the Assessment — 514 — Center technique,11 but it may not be the sole deciding factor. The district court shall have continuing jurisdiction until a valid exam is devised by the parties to these cases, if possible, and may require reports or take evidence on testing procedures as it deems necessary. The final plan will be subject to that court’s approval. Both appellants also argue that the test has a number of “flawed items” which invalidate the written exam even for the areas of job knowledge it has attempted to cover. At trial Dr. Barrett had criticized many of the individual questions that were used on the test.12 Though Dr. Barrett gave examples of each of his specific criticisms, his testimony does not render the trial court clearly erroneous. When asked if each item on a test had to be valid in order for the whole test to be valid, he replied that it did not. He previously admitted not having made a study of the entire fire captain’s exam to determine the total number of poor and unrelated questions. This court agrees that some improvement could be made in this area. Supper Clubs The second principal claim concerns the exclusion of blacks from the “supper clubs.” Supper clubs are informal eating arrangements among on-duty firefighters at firehouses in the St. Louis Fire Department. Cooking facilities, stove, refrigerator, and cabinets for storage, are provided by the City for the use of its on-duty personnel. Each supper club provides for its own utensils and condiments and buys food for the shared meals. A cook is chosen, who the district court found, “has the discretion and authority to determine who is a member of the club.” The clubs are not organized or regulated by the Fire Department. As a finding of fact the district court concluded that blacks have been excluded from many of these clubs. These exclusions, the court found, frequently result in blacks, where a minority in a firehouse, cooking and eating apart from their white associates. The district court felt that such segregation was “offensive” and “incomprehensible” but concluded that because no Fire Department directive or order promoted the exclusivity, the court would not intervene. The district court did indicate that if the problem persisted it would seek a solution. The existence of segregated supper clubs was accepted as a fact by the district court and that fact is accepted here. This court as well finds the exclusion of black co-workers by whites highly offensive, and regards the situation as one which the Fire Department could remedy by appropriate regulations. In Rogers v. Equal Employment Opportunity Commission, 454 F.2d 234, 238 (5th Cir. 1971), cert. denied, 406 U.S. 957, 92 S.Ct. 2058, 32 L.Ed.2d 343 (1972), the court in determining that the Commission possessed “the statutory authority to investigate psychological fringes in an employment relationship,” discussed the scope of Title VII’s authority to alleviate race discrimination; This language evinces a Congressional intention to define discrimination in the broadest possible terms. Congress chose neither to enumerate specific discriminatory practices, nor to elucidate in extenso the parameter of such nefarious activities. Rather, it pursued the path of wisdom by being unconstrictive, knowing that constant change is the order of our — 515 — day and that the seemingly reasonable practices of the present can easily become the injustices of the morrow. Time was when employment discrimination tended to be viewed as a series of isolated and distinguishable events, manifesting itself, for example, in an employer’s practices of hiring, firing, and promoting. But today employment discrimination is a far more complex and pervasive phenomenon, as the nuances and subtleties of discriminatory employment practices are no longer confined to bread and butter issues. As wages and hours of employment take subordinate roles in management-labor relationships, the modern employee makes ever-increasing demands in the nature of intangible fringe benefits. * * * * * * * * * * * Therefore, it is my belief that employees’ psychological as well as economic fringes are statutorily entitled to protection from employer abuse, and that the phrase “terms, conditions, or privileges of employment” in Section 703 is an expansive concept which sweeps within its protective ambit the practice of creating a working environment heavily charged with ethnic or racial discrimination. * * * One can readily envision working environments so heavily polluted with discrimination as to destroy completely the emotional and psychological stability of minority group workers, and I think Section 703 of Title VII was aimed at the eradication of such noxious practices. See also Wilson v. Woodward Iron Co., 362 F.Supp. 886, 896 (N.D.Ala.1973). The language in Rogers is apposite to the situation in this case. The City provides the cooking facilities in each firehouse for use by its on-duty personnel as part of their employment; it is clear that city officials have been made aware of the segregated eating arrangements in the firehouses for sometime and have protested their inability to solve the problem to the district court. On remand of this case to the district court, that court should renew the interest it had indicated it had in an ongoing review of the supper club problem by supervising the Department’s promulgation of new regulations. Those regulations should provide that use of city facilities by supper clubs may not continue in a discriminatory and segregated manner. In other words the supper clubs may not use City kitchen facilities if they refuse membership to blacks. In this way the City may comport with its duty to provide a nondiscriminatory working environment; additionally, the inclusion of blacks and the reduction of racial tension in firehouses cannot help but aid the City as an employer where the job at hand requires the close cooperation of its employees and a concerted team effort. Battalion Chief’s Exam and the Matter of George Horne The F.I.R.E.-Appellants have appealed the district court’s decision that a prima facie case of racial discrimination was not shown with respect to the use of the battalion chief’s examination. We affirm. It is undisputed that results from the exam showed no disparate racial impact. Two of the three blacks who took the examination passed it and at least one will be áppointed to the higher level supervisory position. F.I.R.E. argues, however, that a comparison of the racial composition of the employment pool with the lack of black battalion chiefs clearly suffices for a prima facie showing of racial discrimination. This argument is rejected. In Carter v. Gallagher, 452 F.2d 315, 323 (8th Cir. 1971) this court held that demographical “[statistical evidence can make a prima facie case of discrimination” in a situation where an all white 535 man Fire Department operated in a large city with a 6.44 percent black population. In St. Louis, eleven percent of the existing force is black, and blacks will be hired at a 50 percent rate on the entry level pursuant to the district court’s order. In the case on which F.I.R.E. makes its primary claim for support, the court used census figures to supplement “meager exam statistics” which had shown a racial — 516 — disparity. See Boston Chapter, N.A.A.C.P., Inc. v. Beecher, 504 F.2d 1017, 1020 (1st Cir. 1974). This is obviously not the case here. Likewise, the district court is affirmed in the matter of George Horne. The district court found that a union official had written a letter to Mr. Horne in which he told him that six vacancies would be available. The district court also found that a Deputy Chief had told Horne he would speak to the Fire Chief concerning a promotion. However, only four new fire captains were requisitioned by the Fire Department at that time, eliminating Horne from consideration prior to the expiration of the eligibility list. F.I.R.E. points to no evidence which suggests that racial discrimination was the Department’s motive in making a request for four new captains. In fact, George Horne was sixth on the eligibility list, and a white man who was fifth and would precede Horne was also passed over for promotion. As a final matter, St. Louis has raised the issue of whether or not the district court erred in not dismissing the suit brought by the United States under 42 U.S.C. §§ 2000e-5(f) and 2000e-6(b). It is the City’s contention that the United States could only participate in this suit as an intervenor; that to bring their own suit is duplicative and contrary to the intent of the statute. The United States argues that St. Louis is precluded from raising this issue because of failure to cross-appeal; that the suit by the United States is brought on the basis of 47 charges filed with the EEOC, 30 of which were not included in the F.I.R.E. complaint; that the United States also proceeded under the pattern and practice authority of § 2000e-6 on behalf of the “public interest” and accordingly, presents policy considerations different from those presented by private litigants; and finally that the consolidation of the cases provides an equivalent practical effect. We agree with each of the reasons advanced by the United States, (with the exception of the second reason) and, in view of the consolidation of the cases for trial, and the fact that F.I.R.E. raised the same questions as the United States on appeal, we do not consider our determination to be in conflict with EEOC v. Missouri Pacific Railroad Co., 493 F.2d 71 (8th Cir. 1974). A question is also raised concerning the jurisdiction of the district court to enter its nunc pro tunc order of June 28, 1976.13 Without passing on this question we direct that at the time the district court reassumes jurisdiction of the cases on remand, it should reenter its order as of that date. Also at that time the district court should award appropriate attorney’s fees to F.I. R.E. for work prior to this appeal. The court had previously indicated it was without jurisdiction to do so after an appeal was taken. Attorney’s fees under Title VII may be awarded pursuant to 42 U.S.C. § 2000e-5(k). The fees awarded in this case, both on remand and upon completion of the balance of the case, should be in a fair and reasonable amount in accordance with twelve guidelines set by the court in Johnson v. Georgia Highway Express, Inc., 488 F.2d 714 (5th Cir. 1974). In our opinion the award of $3,000 suggested by the trial court is grossly inadequate under these guidelines for work done prior to appeal. Accordingly, the judgment of the district court is affirmed in part, and reversed and remanded in part for further proceedings consistent with the views expressed in this opinion. Attorney’s fees for this appeal will be awarded to F.I.R.E. upon submission of affidavits relating to the guidelines described in Johnson v. Georgia Highway Express, Inc., supra, 488 F.2d 714. Footnotes [1] The firefighter exam was shown to have had a disparate racial effect and was not validated. See nunc pro tunc order 2 (June 28, 1976). The order was entered on June 28 and amended the April 9, 1976 memorandum opinion which had included the parties partial consent decree on the entry level issue. On June 28 the court issued an order pursuant to the stipulation of the parties granting relief similar to the partial consent decree. The June 28 order required defendants inter alia to try to achieve a 50 percent hiring rate for blacks in filling vacancies at the entry level over the next five years. [2] F.I.R.E. has alleged that the change of the in-service requirement from five to seven years from 1967 up until 1974, when it was changed back to five years, adversely affected black applicants. However, the change, which was instituted both times on the recommendations of consultants, admittedly affected white candidates as well as black. [3] Dr. O’Leary also testified that determining the relative weight of the three scores was a matter of judgment and he knew of no mathematical or statistical procedure to objectively determine the weight to be accorded each criteria. This opinion was refuted to a certain extent by the plaintiff’s expert, Dr. Barrett, who indicated that weighting is best arrived at by some empirical means such as a criterion-related validity study. In any event, it has not been argued on this appeal that the use of an experience and training score or a service score has resulted in a discriminatory impact on black candidates. [4] The mean score for blacks on this exam was 69.72; the mean for whites was 76.59. Of those blacks taking the exam 25.5 percent received a passing score; 43.6 percent of the whites passed. [5] In 1972 Congress amended 42 U.S.C. § 2000e and included state and local governments, such as the City of St. Louis, under the rubric of “employers” subject to that Act. Intervenors in this appeal, citing National League of Cities v. Usery, 426 U.S. 833, 96 S.Ct. 2465, 49 L.Ed.2d 245 (1976) have strongly contested Congress’ power to dispense with the intent requirement in Title VII cases where state and local governments act as employers. The Cities case, supra, limiting Congress’ power to impose wage and hour standards for state and local employees under the commerce clause, is undoubtedly inapposite. But see Fitzpatrick v. Bitzer, 427 U.S. 445, 96 S.Ct. 2666, 49 L.Ed.2d 614 (1976). In any event, the court determines that it need not reach this issue. Counsel for the Intervenors has admitted under the court’s questioning that the constitutional issue was not raised below. [6] See, e. g., Douglas v. Hampton, 168 U.S.App. D.C. 62, 512 F.2d 976 (1975): “These guidelines have been cited with approval by the Supreme Court, followed by all courts dealing with these issues, and recognized as controlling in at least one circuit. We think it unwise to depart from these accepted principles at this stage in the development of the law concerning equal employment opportunity.” Id. at 986 (footnote omitted). See also, Kirkland v. New York St. Dept. of Correctional Serv., 520 F.2d 420, 426 (2d Cir. 1975); Vulcan Soc’y of New York City Fire Dept., Inc. v. Civil Serv. Comm'n, 360 F.Supp. 1265, 1273 n. 23 (S.D.N.Y.1973). [7] See 29 C.F.R. § 1607.5(a) (1975). [8] Newly proposed regulations provide that “[f]or the purposes of satisfying these guidelines users may rely upon criterion related validity studies, content validity studies, or construct validity studies * * 41 Fed.Reg. 29018 (1976). [9] Twenty-seven fire captains were interviewed in the preparation of this job analysis. Those • interviewed were asked to describe the elements of the job and rank their relative importance. Captains were questioned concerning the positive and negative critical incidents of the job, and the essential and desirable qualities of a fire captain. A pool of questions for the exam were obtained through the International Governmental Personnel Management Association; according to Dr. O’Leary, a series of items tailor-made by the department were also included. Questions on the exam were distributed according to the importance of each area of knowledge. The best items were selected after review by several technical and test advisors. See United States v. City of St. Louis, 410 F.Supp. 948, 954 (E.D.Mo.1976). Dr. Barrett’s main objection to the job analysis was that in his opinion the descriptions of job activities and skill levels were incomplete. He also acknowledged, however, that the 27-person sample, interview technique, and cataloging of information used by Dr. O’Leary were reasonable methods. Considering the district court’s evaluation of the two experts’ credibility, this court will not disturb the lower court’s conclusions about the job analysis. Compare the job analysis in this case with the cursory preparation disapproved in Vulcan Soc’y of New York City Fire Dept., Inc. v. Civil Serv. Comm’n, supra, 360 F.Supp. at 1275. [10] The court continued: “It is essential that the examination test these attributes both in proportion to their relative importance on the job and at the level of difficulty demanded by the job.” 360 F.Supp. at 1274. See also Douglas v. Hampton, 168 U.S.App.D.C. 62, 512 F.2d 976 „ (1975). “ ‘Content’ validity is established when the content of the test closely approximates the tasks to be performed on the job by the applicant.” Id. at 984 (emphasis added) (footnote omitted). See also Bridgeport Guardians, Inc. v. Members of Bridgeport Civil Serv. Comm’n, 482 F.2d 1333, 1338 (2d Cir. 1973). Dr. O’Leary said at trial: “Content validity is validity demonstrated when one can show that the content of his predictors * * * are very close to the content of the job that the person is going to be performing.” (Emphasis added). [11] If a screening device is used in conjunction with the Assessment Center, development of a fair cutoff score is obviously important. The EEOC Guidelines provide: It is expected that each operational cutoff score will be reasonable and consistent with normal expectations of proficiency within the work force or group on which the study was conducted. 29 C.F.R. § 1607.6 (1975). [12] Dr. Barrett objected to “tenuous linkage” in some questions; questions where a correct answer did not guarantee adequate performance on the job; and questions that gave a premium to the test-wise individual. Other items, he said, called for “esoteric information” not related to job performance. [13] See note 1, supra.