Kaplan v. United States

OPINION BRUGGINK, Judge. Plaintiff, Dr. Kathleen Kaplan, appearing pro se, filed her complaint on January 27, 2014. Plaintiff is a former employee of the Air Force Office of Scientific Research (“AFOSR”), which is part of the Air Force Research Laboratory (“AFRL”), She brings this suit under the Equal Pay Act, 29 U.S.C. § 206(d)(1) (2012). She contends that while she was employed at AFOSR, she was paid less than males doing comparable work. She claims that she is entitled to additional pay beginning in 2011. We held a three-day trial from December 6 to December 8, 2016. After post-trial briefing, the matter is ready for disposition. Because the’ government has demonstrated that it had in place a merit system that explains the differential in pay, the Laboratory Personnel Demonstration Project, we find against plaintiff. In reaching our finding in favor of the government, we are mindful that Dr. Kaplan appears pro se. BACKGROUND Dr. Kaplan is trained' as a computer scientist. She earned three degrees in computer science: a B.S. from the University of Massachusetts Lowell, a M.S. from the Florida Institute of Technology, and a D.Sc. from George Washington University. Tidal Transcript (“Tr.”) 46; PX 121.2 Dr. Kaplan spent a number of years in academia before moving to AFOSR in August of 2005. I. The Air Force Office of Scientific’ Research The AFOSR, part of the Ah' Force Re-searéh Laboratory, manages the research investment for the United States Air Force. AFOSR’s mission is to “[discover, shape, and champion basic science that profoundly impacts the future Air Force.” PX 145 at KAPL005102. In large part, the agency pursues its mission by directly funding research by others-as well as through certain scholarship and.outreach programs. Over the years, the agency has undergone several reorganizations. The most recent occurred in 2013 and resulted in AFOSR changing from having three “directorates”—the Physics & Electronics Directorate; the Mathematics, Information, & Life Sciences Directorate; and the Aerospace, Chemical & Material Sciences Directorate—to five “divisions,” namely, Dynamical Systems & Control; Quantum & Non-Equilibrium Processes; Information, Decision, & Complex Networks; Complex Material and Devices; and Energy, Power, & Propulsion. II, Dr. Kaplan and Her Alleged Comparators During her tenure at AFOSR, she held three different positions. She began in 2005 as a program manager, which she held from her hire date until January of 2011. Tr. 53, As the result of an Equal Employment Opportunity settlement, Dr. Kaplan left her program manager position and became the deputy director of the Physics & Electronics Directorate on January 10, 2011. Tr. 67. She held her deputy director position until AFOSR completed the 2013 reorganization, which resulted in her assuming a program officer position in the Information, Decision, & Complex Networks division in February of 2013. Plaintiff claims that she is entitled to an increase in pay based on her performance in these three positions. Because of the lapse — 237 — of time between performance, evaluation, and the time for which pay increases became effective, the relevant performance assessment cycles began in October 2009, Dr. Kaplan offers six male comparators— men who she argues received more pay while performing work that required substantially equal skill, effort, and responsibility. Her alleged comparators, two for each position, are Dr. Wu and Dr. Schlossberg for her program manager position, Dr. DeLong and Dr. Blackwood for her deputy director position, and Dr. Bonneau and Dr. Nachman for her program officer position. Defendant does not concede that these men are proper comparators, even if they might have shared a title with plaintiff, because their work did not involve equal skill, effort or responsibility. Defendant also argues that the agency used a merit-based system that linked pay to performance, thus providing a reason for the difference in pay regardless of whether the comparators’ work involved equal skill, effort, or responsibility. In view of the importance of the agency’s novel system of linking pay to performance, known as the Laboratory Personnel Demonstration Project, we begin with an extensive description of that program. III. The Laboratory Personnel Demonstration Project The APRL replaced the General Schedule (“GS”) compensation system with the Laboratory Personnel Demonstration Project (the “Lab Demo”) in March 1997, hence it was in place throughout plaintiffs employment period and its effects would have been reflected in the pay Dr. Kaplan received at the commencement of the years at issue here. Tr. 547. As described in AFRL Manual 36-104 (the “Manual”),3 the Lab Demo “is an improved personnel management system which provides laboratory management, at the lowest practical level, the flexibility, authority, and control to achieve a quality workforce and laboratory.” PX 50 at KAPL000238.4 In relevant part, the Lab Demo replaced the GS 1-15 grade levels used commonly across federal civilian employment with four broadband levels, DR-I through DR-IV, with DR-IV being the highest grade and equivalent to GS-15. Id. at KAPL000241. Dr. Kaplan and each of her alleged male comparators took part in what is known as the “DR pay plan,” which is specific to APRL scientists and engineers (“S & E”). Id. at KAPL000238. In order to “provide an effective, efficient, and flexible method for assessing, compensating, and managing the laboratory workforce,” the Lab Demo implements what is known as a contribution-based compensation system (“CCS”). Id. A contribution is the “measure of the value of what an employee accomplished.” DX 5 at AFKAP032215. As opposed to simply judging an employee’s performance, in assessing an employee’s contribution, the focus is on the positive impact an employee’s actions have with respect to advancing “the purpose, vision, mission, and objectives of the organization,” here, the Air Force. Id. That is, the system seeks to “[n]ever confuse motion with progress,” Tr. 838 (Dr. Christian), nor is mere longevity rewarded for its own sake. Every year, S & E employees are assessed and given scores by their supervisors based on four factors: problém solving, communication, technology management, and teamwork and leadership. Each factor has a corresponding descriptor—often referred to as a “rubric”—that describes the expected level of work specific to the S & E career path and an employee’s particular broadband level. These rubrics are published in the APRL Manual 36-104, to which the employees have access, PX 50 at KAP000353-56; Tr. 157. In a process described in more detail below, the four factor scores are averaged to determine an employee’s Overall Contribution Score (“OCS”), which, in conjunction with the standard pay line (“SPL”),5 serves as the primary — 238 — basis for individual pay increases beyond an employee’s then-current pay. In theory, the CCS allows Lab Demo employees to more quickly reach the equivalent of a GS-15/ step-10 than under the GS system, if their contributions merit such a progression. There are also two types of bonuses that are available to Lab Demo employees. The broadband IV bonus is used when an “employee’s OCS indicates that he/she should receive a base pay increase that would place him/her over the GS-15/step-10 pay cap.”6 Id. at KAPL000310. What is referred to as the CCS bonus is used “to recognize unsustainable contributions, [for example if] the employee had an opportunity in a given year that’s not going to last... they might be recognized with a CCS bonus.” Tr. 579. This gives the employee’s supervisors the ability to reward a one-off contribution without increasing the employee’s expected OCS for the next cycle. According to the Manual, “[t]he annual CCS assessment scoring process begins with input from the employee, which provides an opportunity to state the perceived accomplishments and jevel of contribution.” PX 50 at KAPL000280-81. The assessment cycle begins on October 1 and ends on September 30 of the following year. Id. at KAPL000281. For example, an assessment cycle that runs from October 1, 2016 to September 30, 2017 would determine an employee’s pay for the 2018 calendar year. For that reason, the pay Dr. Kaplan received in January 2011, the beginning of what she asserts is the relevant limitations period, was a reflection of the prior operation of the Lab Demo from 2005 through 2009, insofar as her performance was evaluated. At the beginning of the cycle, supervisors are directed to “provide the broadband level descriptors, factors, and key elements to employees, as well as discuss expectations for the upcoming assessment period so that employees know the basis on which their contribution will be assessed.” Id. The Manual also dictates that there be a mid-cycle review in April or May of a particular cycle. “At this time, the employees’ contributions to the mission will be discussed, as well as future professional development and career opportunities .... The supervisor completes the AFRL Form 279, CCS Feedback, to facilitate and document the mid-cycle review.” Id. The scoring process occurs during the end-of-cycle assessment, which begins when the employee performs a self-assessment, summarizing their contributions in a way that “reflect[s] the impact or result of each activity rather than just listing the activity itself.” Id. These self-assessments are then reviewed by the employee’s first-level supervisor as he or she creates a preliminary assessment, which “is determined using the employee input, to include any team participation, and the supervisor’s personal knowledge of the employee’s overall contribution to the laboratory mission.” Id. at KAPL000282. The first-level supervisor then takes the preliminary assessment to the First-Level Meeting of Managers (“First-Level MoM”), the opportunity for an employee’s first-level supervisor and first-level supervisors of similar employees to “meet with their respective second-level supervisor to review and discuss preliminary assessments, refine them into numerical CCS factor scores, and adjust any factor write-ups based on results of the meeting.” Id. As mentioned above, each factor has a corresponding set of descriptors, or a rubric, that describes the expected contribution specific to the four broadband levels. Supervisors are to use these rubrics in their evaluations. Factor scores can range from 0.0 to 5.9. Contributions that are assessed to correspond with the descriptors associated with the level I broadband receive a score within the range of 0.75 to 1.9. “Contributions meeting level II descriptors are assessed at scores of 2.0 to 2.9, and so forth.” Id. Contributions that fail to meet the descriptors for a level I — 239 — contribution receive a score of 0,0; similarly, contributions relevant to a specific factor that exceed expectations for level IV can receive a score as high as 5.9. In other words, contributions beyond what is expected for an employee at that level are more highly leveraged in terms of scoring. The four factor scores are averaged to reach an employee’s OGS. The Manual provides that [t]he maximum compensation for broadband level IV is the GS-15/Step-10 salary and equates to a total OCS of just below 4.9. If the average of CCS factor scores exceeds 5.25, the total OCS will be set to 5.25 with the individual identified to upper management as having exceeded the maximum contribution defined by the broadband. However, contribution exceeding the level IV descriptors would not commonly occur. Id. After the First-Level MoM, the Pay Pool Manager (“PPM”) takes part in what is referred to as the “PPM MoM.” “The purpose of this meeting is .to ensure consistent application of the CCS process ....” Id. at KAPL000283. The PPM may direct subordinate supervisors to take a second look at scores if he or she finds that the CCS process has not been consistently applied. The PPM approves the scores after they have been finalized. Once the PPM has approved and finalized all assessments, the first-level supervisors are then required to conduct assessment feedbacks with the employee being evaluated. Id. The feedback session is expected to include “a discussion of the contribution statements, factor scores, OCS, developmental and career opportunities, and expectations for the upcoming cycle.” Id. A copy of the “Annual Contribution Evaluation Form,” known as “AFRL Form 280,” is then given to the employee. The AFRL Form 280 “documents decisions made during the MoM to include factor write-ups and factor scores,” Id. The grievance period begins after the employee receives his or her AFRL Form 280. In the absence of a successful grievance, the employee’s actual OCS is then compared to her “expected OCS,” in order to determine whether she is being properly compensated. As mentioned above, the Standard Pay Line can be used to generate an employee’s expected OCS by determining the OCS score that correlates with the employee’s current basic pay. If an employee’s actual OCS is greater than their expected OCS, they are eligible for an increase in pay based on their increased contribution. — 240 — Program Manager Assessment Cycle Expected OCS Actual OCS Delta OCS Resulting Base Salary Total CCS and Broadband IV Bonuses Dr. Kaplan 10/01/09-9/30/10 [ 3 [] [ 3 ■ t 3 f 3 10/01/10-9/30/11 C ] [ 1 r 3 I 3 [] • Dr, Schlossberg 10/01/09-9/30/10 [ ] [ ] [ ] [ 3 [ 3 10/01/10-9/30/11 [ ] [ 3 [ 3 [ 3 r 3 Dr. Wu 10/01/09-9/30/10 [ 3 [ '] [ ] l 3 [ 3 10/01/10-9/30/11 N/A PX22.2M1.32, 175; DX 100, Deputy Director Assessment Cycle Expected OCS Actual OCS Delta OCS Resulting Base Salary Total CCS and Broadband IV Bonuses Dr, Kaplan 10/01/09-9/30/10 t 3 [ 3 [ 3 C 3 [ 3 — 241 — 10/01/10-9/30/i 1 [ 3 [ 3 [ 3 [ 3 [] 10/01/11-9/30/12 13 [ 3 [ 3 [ 3 [3 10/01/12-9/30/13 □ n [] [ J □ Dr. Blackwood 10/01/09-9/30/10 [ 3 [ ] C 3 [ 3 [ 3 10/0 i/10-9/30/11 [ 3 l ■] [ 3 [ 3 r 3 10/01/11-9/30/12 L J [ 3 [ 3 [ 3 n 10/01/12-9/30/1,3 [ 3 [ 3 r i i: 3 n Dr. DeLong 10/01/09-9/30/10 [ 3 [ 3 [3 [ 3 [ 3 10/01/10-9/30/11 f .3 [ 3 [3 [ 3 [ 3 10/01/11-9/30/12 [ 3 [ 3 [J [ 3 [ 3 10/01/12-9/30/13 r ] [ 3 i: .3 [ 3 [ 3 Px 9, 10, 18, 19,24,25,26, 27, 28, 31, 32; DX 100, — 242 — Program Officer Assessment Cycle Expected OCS Actual OCS Delta OCS Resulting Base Salary Total CCS and Broadband IV Bonuses Dr. Kaplan 10/01/11-9/30/12 [ ] [ 3 E 3 E 3 E3 10/01/12-9/30/13 [ ] [ 3 E 3 E 3 E3 10/01/13-9/30/14 [ ] E 3 E 3 E 3 E3 10/01/14-9/30/15 [ I [ 3 E 3 E 3 E3 Dr, Nachman 10/01/11-9/30/12 C ] E 1 E3 t 3 C 3 10/01/12-9/30/13 [ .1 E 3 E3 E 3 E 3 10/01/13-9/30/14 [ ] E 3 E3 E 3 E 3 10/01/14-9/30/15 E ] [ 3 E3 [ 3 E 3 Dr, Bonneau 10/01/11-9/30/12 [ 3 E 3 E3 E -3 E 3 10/01/12-9/30/13 [ 3 E 3 E 3 E 3 E 3 10/01/13-9/30/14 N/A 10/01/14-9/30/15 N/A PX 9, 10, 20,21,36, 37, 39,44, 45,139; DX 100, Sixteen witnesses testified during trial. We also received the transcript of the de bene esse deposition of Colonel Robert Kraus, a former first-level supervisor of Dr. Kaplan. We will list the witnesses in order of their appearance along with a brief biographical description. As relevant, we will discuss the particulars of their testimony in more detail below. On the first day of trial, four witnesses testified. The first was Dr. Kaplan. Second was Dr. Frederica Darema, former director of the Mathematics, Information, and Life Sciences Directorate and current director of — 243 — APOSR. Third was Dr. Thomas Russell, former AFOSR director and the Pay Pool Manager (“PPM”) for the assessment cycles that set Dr. Kaplan’s 2011, 2012, and 2013 pay. Dr. Patrick Carrick was the fourth witness to testify and was the PPM that set Dr. Kap-lan’s pay for 2014. Dr. Kaplan served as Dr. Carrick’s deputy when he was the director of the Physics & Electronics Directorate. On the second day of trial, Dr. Carrick concluded his testimony and the following six witnesses testified. Ms. Lisa Aiken, a paralegal for the Department of Justice and the fifth witness to testify, helped prepare defendant’s exhibit 100—a gathering of the salaries and overall contribution scores of plaintiffs and her comparators, along with a total of their bonuses, if any. Colonel Andrew Szmerekovsky, the sixth witness to testify, was the PPM responsible for the assessment cycle that set Dr. Kap-lan’s pay for 2015. Ms. Michelle Williams testified seventh. She was the chief of the personnel demonstration project office in the headquarters of the APRL at all times relevant to this case. Eighth to testify was Ms. Terry Brailsford, who was at all relevant times the chief of operations management and responsible for human resources. Ms. Mary Jackson was the ninth witness. She was the contribution-based compensation system coordinator for the APOSR Lab Demo System. Dr. Milton Blackwood, one of plaintiffs alleged comparators for her deputy director position, testified tenth. On the third and final day of trial, the remaining six witnesses testified: Dr. Hugh DeLong, one of plaintiffs alleged comparators for her deputy director position (he appeared by videoconference); Dr. Robert Bon-neau, one of plaintiffs alleged comparators for her program officer position; Dr. Arje Nachman, one of plaintiffs alleged comparators for her program officer position; Mr. Raheem Lawal, not an alleged comparator but a program manager beginning in 2005; Dr. Thomas Christian, Jr., the PPM responsible for setting Dr. Kaplan’s pay for 2016 and until December 2016, the director of APOSR; and Lieutenant Colonel Jamie Morrison, the branch chief of the Engineering and Information Sciences branch and Dr. Kaplan’s first-level supervisor beginning in April of 2015. The de bene esse deposition of Colonel Robert Kraus, Dr. Kaplan’s first-level supervisor prior to Lt. Col. Morrison, was admitted as Court Exhibit 1. DISCUSSION The Equal Pay Act provides in relevant part that: No employer having employees subject to any provision of this section shall discriminate, within any establishment in which such employees are employed, between employees on the basis of sex by paying wages to employees in such establishment at a rate less than the rate at which he pays wages to employees of the opposite sex in such establishment for equal work on jobs the performance of which requires equal skill, effort, and responsibility, and which are performed under similar working conditions, except where such payment is made pursuant to (i) a seniority system; (ii) a merit system; (iii) a system which measures earnings by quantity or quality of production; or (iv) a differential based on any other factor other than sex .... 29 U.S.C. § 206(d)(1). Dr. Kaplan believes that, during her service as a program manager, deputy director, and program officer with APOSR, the government violated this provision and did so willfully. Even though the limitations period, assuming a willful violation, goes back no further than January 2011, she can challenge the operation of the Lab Demo project as far back as October of 2009, when the assessment cycle fixing her pay for 2011 began. To successfully prove a violation of the Equal Pay Act, she has the burden to demonstrate that she was paid less than her respective male comparators for a job that required substantially equal skill, responsibility, and -effort. Cooke v. United States, 85 Fed.Cl. 325, 341 (2008). If plaintiff is able to establish her prima facie ease, the burden shifts to defendant to demonstrate by a preponderance of the evidence that the difference in pay is due either to “(i) a seniority system; (ii) a merit system; (iii) a system which measures earnings by quantity or quality of production; or (iv) a differential based on any other factor — 244 — than sex.” 29 U.S.C. § 206(d)(1). The burden on the employer is “a heavy one,” and it must demonstrate that the factor causing the difference in wage is actually the gender-neutral factor identified. Cooke, 85 Fed.Cl. at 347. The government attributes the pay disparity between Dr. Kaplan and her male comparators to the proper functioning of the Lab Demo system, which was in place at all times relevant to this case, and which compensates employees according to the quality of their performance by measuring an employee’s contribution to the AFOSR mission through an organized procedure that uses predetermined criteria. It contends that plaintiff was fairly scored for her contribution under that system, and as a result of her lower contribution to the mission of AFOSR, she continued to be paid less than her alleged comparators. To establish the merit system defense, defendant must demonstrate that the Lab Demo “was an organized and structured procedure by which employees were evaluated systematically and in accordance with predetermined criteria. Under such a merit system, an employer may reward workers for outstanding experience, training, and ability, so long as the resulting salary differential is not based upon sex. ” Raymond v. United States, 31 Fed.Cl. 514, 518 (1994) (internal citations omitted). Further, a merit-based system does not have to be devoid of subjectivity. EEOC v. Aetna Ins. Co., 616 F.2d 719, 726 (4th Cir. 1980) (“An element of subjectivity is essentially inevitable in employment decisions; provided that there are demonstrable reasons for the decision, unrelated to sex, subjectivity is permissible.”). Finally, “[i]f the merit system is not in writing, the employer must also show that the employees were aware of it.”7 Cooke, 85 Fed.Cl. at 347 (citing Aetna, 616 F.2d at 725). There is no question that plaintiff was paid less than the male employees she selected as comparators. Nevertheless, it is unnecessary for us to consider whether plaintiff has established that these men performed work that required substantially equal skill, responsibility, or effort because the government has clearly established the second-listed defense above—the application of a merit system.8 We conclude for reasons set out below that the Lab Demo is a legitimate and comprehensive merit system that adequately explains the difference in pay between Dr. Kaplan and her alleged male comparators. The Lab Demo system is as free from subjectivity as one could reasonably expect a merit system to be. Also, we find that, in assessing both Dr. Kaplan and her alleged male comparators, her supervisors deviated from the Lab Demo procedures only to a minor and inconsequential degree, causing no prejudice to Dr, Kaplan.9 As plaintiff points out, and as can be seen from the chart set out above, plaintiffs beginning salary in 2011, when she began her term as a Deputy Director, was approximately $9,000—512,000 less than the other two deputy directors were making that year, not considering the differences in bonuses. Plaintiff argues that this illustrates her claim of disparate treatment. In response, the government points out that her salary commencing that year was the result of the assessment cycle for October 2009-September 2010, during which time she was not yet a deputy director but was a — 245 — program manager, The Lab Demo contemplates that in such a situation, her default salary would be carried along from the prior position, as adjusted by the assessment results. Although supervisors had the discretion, when someone moved positions in that fashion, to raise the base pay in anticipation of a greater expected contribution, that did not occur in her case. Despite her change in title, her salary as Deputy Director thus was determined by her prior performance in the program manager position. This happened again in February 2013 when her position changed from deputy director to program officer. Her pay as a program officer commencing in that year was fixed by her evaluation for the preceding assessment cycle when she was worldng as a deputy director. There is no reason to think that plaintiff was treated any differently in this respect than anyone else in the agency who moved between positions. All of this was consistent with the manual.10 The mere fact that she held the same title as her higher paid comparators is not an indictment of the system. The relevant inquiry is whether the agency’s application of the assessment process was legitimate. If it was, then the disparity in salary from year to year would continue. Given the nature of the Lab Demo, it is inevitable that employees with the same job title will receive different salaries based on their respective contributions and those differences are cumulative. We also note that the Lab Demo is more agile than the GS system when it comes to recognizing an employee’s impactful work with higher compensation. For example, Dr. Bonneau, an alleged compartator for Dr. Kaplan’s program officer position, joined the AFOSR as a DR-III level program manager in 2007. Tr. 730. While serving as a program manager, Dr. Bonneau quickly raised his OCS score to the [ ] maximum in the assessment cycle beginning October 1, 2010. PX 43. This represented a nearly [] OCS increase from his expected OCS of [ ] from the previous cycle, PX 42. In other words, in two years, Dr, Bonneau was able to raise his Actual OCS by a significantly greater amount than Dr, Kaplan did through all assessment cycles relevant to this litigation.11 The actual fairness and consistency of this somewhat mechanical grading scheme is thus dependent in the first instance on the quality of the first-line supervisors’ scoring and then on whether there was meaningful second-level review. Here we have no reason to question either the inputs—admittedly somewhat subjective—or the higher-level reviews. Through testimony elicited at trial and other evidence in the record, the government was able to demonstrate that each of Dr. Kap-lan’s first-level supervisors, namely, Dr. Car-rick, Colonel Kraus, and Lieutenant Colonel Morrison, used the required rubric and did not consider Dr. Kaplan’s gender while preparing their assessments, Court Ex. 1 at 11-12, 24-26; Tr. 381, 385, 890, 903. For example, Lieutenant Colonel Morrison testified about his use of the Lab Demo rubrics during his assessments as follows: “what I’m doing is I’ve got that rubric in front of me and I’m trying to match what language [from the rubric] best describes the activity that I’ve seen.” Tr. 890. These first-line evaluations, which arguably could be the most problematic in terms of subjectivity, were then reviewed by second-line supervisors, the PPMs. Dr. Russell, Dr. Carrick, Colonel Szmerekovsky, and Dr. Christian testified that both Dr. Kaplan’s and — 246 — her comparators’ OCS scores accurately reflected their contribution, and were in line with those of other employees, Tr. 258-59, 409-12, 492, 830-31. In addition, AFOSR administrators, Ms. Terry Brailsford and Ms. Mary Jackson, explained that they observed the meetings of managers in order to ensure a level of uniform application of the procedures. Ms. Brailsford, the Chief of Operations Management who attended all MoMs since at least 2010, testified that the standardized rubrics were used in the MoMs to assess each employee’s contributions. Tr. 627. Ms. Jackson, the CCS coordinator who also attended all MoMs at all times relevant to this litigation, testified, as did Ms. Brailsford, that gender did not play a role in the Lab Demo process. Tr. 627, 655. Plaintiff cannot point to any credible evidence to rebut this, testimony. In effect, her challenge amounts to no more than an effort to have the court substitute its assessment of her performance for that of her supervisors. Dr. Kaplan, in her post-trial reply brief, identifies a number of instances in which she believes that the government’s i'eliance on the PPMs’ testimony is misplaced due to alleged inconsistencies in their statements at trial. We are not persuaded, however. As an example, plaintiff argues that “Col Szmere-kovsky did not address the ‘red flags’ with Col Kraus using copied Lab Demo instruction text as Dr. Kaplan’s contributions,” Pl.’s Post-Trial Reply 14.12 However, it was only Dr. Kaplan who identified the copying itself to be a “red flag,” not Colonel Szmerekovsky. The only “red flag” that Colonel Szmerekov-sky clearly identified in his testimony was that Colonel Kraus was signaling that Dr. Kaplan’s contribution level in certain areas was below the level at which Dr. Kaplan was expected to contribute as a DR-IV employee. Tr. 469-70. Dr. Kaplan makes a number of arguments attempting to show that the Lab Demo procedures were not followed. In all of them that are actually relevant to this case, plaintiff is either mistaken in fact or plaintiffs comment highlights an inconsequential deviation that we find was not prejudicial to her participation in the Lab Demo process. We hold that plaintiff has not shown that the agency failed to adequately follow the procedures of the Lab Demo. Plaintiff also advances several arguments that we find amount to nothing more than her disagreements with how her contributions were valued through the Lab Demo process. She fails to demonstrate, however, that there was an impermissible level of subjectivity. For example, in her post-trial brief, Dr. Kaplan takes issue with how Dr. Christian only pointed out “one thing” when he “was asked to compare Dr. Kaplan’s impressive communications contributions which only received a [ ] with Dr. Nachman’s less-than-impressive [communication] contributions which received a [ ].” Pl.’s Post-Trial Br. 38. The one contribution that Dr. Kaplan is referencing related to Dr. Nachman’s visit to the Intelligence Advanced Research Projects Activity (“IARPA”). Dr. Christian testified that this communication was particularly important because IARPA exists to support the intelligence community, “so it’s a very important organization doing work that highly benefits the nation.” Tr. 841. Dr. Christian went on to explain that “[t]here is tremendous money that IARPA puts into research, and this allows the Air Force to let them know what we’re doing and inform them.” Id. Dr. Kaplan cites this to illustrate the fact that one contribution can carry a lot of weight while another may have little influence on a factor score, thus showing the “complete subjectivity” of the system. Pl.’s Post-Trial Br. 38. We disagree with her characterization. Indeed, it is difficult to imagine-any merit247 based system passing muster under Dr. Kap-lan’s view. With respect to her position as a program officer, we note that the assessments performed by Dr. Kaplan’s supervisors were corroborated, in part, by unfavorable reviews by the Air Force Scientific Advisory Board (“SAB”) in front of which Dr. Kaplan presented in March 2016.13 Dr. Christian described the SAB as follows: “The SAB is a very ... independent, objective, and I’ll use the term ‘eminent experts.’ They’re from universities. They’re from government laboratories .... And they review every directorate within the research laboratory, of which AFOSR is one.” Tr. 846-47. When Dr. Kaplan assumed the program officer position in February 2013, she inherited the “Software and Systems” portfolio from Dr. Bon-neau. Tr. 740. Relevant here, the SAB reviewed the Software and Systems portfolio when it was under Dr. Bonneau in 2012 and under Dr. Kaplan in 2016. In 2012, Dr. Bon-neau received the highest possible rating from the SAB for the Systems and Software program. PX 44. Under Dr. Kaplan in 2016, the SAB found that it “was the only one to be highly deficient,” and “strongly recommended that no money be put into it until it was restructured.” Tr. 848. Two months later, Dr. Kaplan was placed on a contribution improvement plan based on a finding that her contributions were not commensurate with her pay. PX 213. While we have discussed only a few examples of the evidence plaintiff cites to demonstrate bias, we believe that they are illustrative and are satisfied that the difference in the contribution scores are not the result of an improper amount of subjectivity. The Lab Demo system is structured to limit subjectivity, while recognizing that the assessments of first-line supervisors are to some extent subjective. The First-Level MoM, PPM MoM, and the employees’ ability to later file grievances all work towards introducing objective points of comparison. As Dr. Christian testified, the “second set of eyes” provided by the other supervisors in the FirsL-Level MoM minimize subjectivity. Tr. 835-36. We are also satisfied that the description by Colonel Szmerekovsky, the PPM that set Dr. Kaplan’s pay for 2015, is accurate, He viewed it as his job in the PPM MoM to “ensure the integrity of the system,” Tr. 468, and to make certain that there was a similar or consistent standard when [the supervisors are] looking at the rubric across the board .... And if it’s not consistent, [if] one person appears to be too hard on their people and one person appears to be too easy, we try to normalize it ... through discussion and say, hey, why are you rating this because ... this doesn’t match the rubric or this isn’t consistent with how we’ve been providing ratings. Tr. 519. Finally, plaintiff also challenges what she considers to be inconsistency in the award of bonuses. For example, Dr. Kaplan argues that Dr. Russell, the PPM for the assessment cycles that set Dr. Kaplan’s 2011, 2012, and 2013 pay, could not explain how Dr. Nach-man and Dr. Bonneau, who both received an Overall Contribution Score of [], were awarded different bonuses. However, as the government points out, Dr. Russell was able in a brief examination of Dr. Bonneau’s AFRL Form 280 to identify at least one illustration that Dr. Bonneau was working beyond his portfolio and contributing at levels higher than Dr. Nachman, thus entitling him to a larger bonus. Tr. 212. In any event, while there was some uncertainty regarding the bonuses expressed during trial, we are satisfied that the two types of bonuses were adequately described in the Manual, which was consistent with Ms. Williams’ testimony, among others. See e.g., PX 50 at KAPL000272-73, 000310; Tr. 579. We also note that Dr. Kaplan’s actual OCS never came close to the 4.85 score that would make the broadband IV bonus relevant to her total compensation. Also, she has failed to demonstrate any unsustainable contribution for which she should have been awarded a CCS — 248 — bonus. Accordingly, we see no reason to question her supervisor’s judgment in not awarding her a bonus. CONCLUSION Dr. Kaplan was paid less than her alleged comparators, but even if those men did work performed under similar working conditions that required substantially equal skill, effort and responsibility, the government has satisfied it burden of proving that those pay differences are entirely attributable to the operation of its merit pay system. We have considered all of Dr. Kaplan’s other arguments and find them to be without merit.14 Accordingly, plaintiffs complaint is dismissed. The Clerk of Court is directed to enter judgment in favor of the United States, No costs. Footnotes [2] "PX” refers to admitted exhibits offered by plaintiff: "DX" refers to admitted exhibits offered by defendant. [3] Dr. Kaplan testified that she had access to the Manual and was aware of the Lab Demo system at all times relevant to this litigation. Tr. 157. [4] The page citations for exhibits refer to the Bates pages provided during discovery. [5] The SPL relates an employee's contribution to basic pay for a given year. It is expressed on a graph with possible contribution scores along the x-axis and potential annual basic pay amounts along the y-axis. The SPL is a straight line with a positive slope and can be used to determine an employee’s expected OCS, given her current sala — 238 — ry, as well as to determine whether an employee is over-, under-, or equitably compensated, given her actual contribution. An employee is considered equitably compensated if her pay is within 0.3 OCS above or below the SPL. PX 50 at KAPL000279. [6] We note that Dr. Kaplan never received a score high enough to warrant a broadband IV bonus. [7] We note that the Lab Demo is in writing and that we are not persuaded by plaintiffs arguments that slight deviations or discretionary decisions by the supervisors amounted to a change to some unwritten policy. [8] Not that we have any misgivings about the quality of the government’s evidence on this point. There was ample evidence that plaintiff was not performing the work that required substantially the same skill, effort and responsibility, For example, her work as deputy director was more often administrative in nature, whereas Dr. DeLong served as acting director of his directorate at this time, while also managing a portfolio. Dr. Bonneau, one of her comparators for the program officer position, in addition to his other duties, served as "cochair of the large-scale networking group for the Office of Science and Technology Policy for the White House.” Tr. 737. [9] For example, Dr. Kaplan testified that she only received mid-cycle feedback for two assessment cycles relevant to this litigation. Tr. 78-79. Because this mid-cycle review was not a part of the end-of-cycle assessment process that determined her pay, this deviation from the Lab Demo carries little weight in plaintiff’s attempt to rebut the government’s affirmative defense. [10] The manual gives management the authority to make such a change in pay when an employee moves within the system but does not make it mandatory. PX 50 at KAPL000272. The Manual provides that "[w]hen setting basic pay ... managers must consider an employee's expected level of contribution based upon academic qualifications, competencies, experience ... [and their] current OCS” among other factors. Icl. We decline the opportunity to second guess her supervisors as plaintiff has failed to show how their decision was irrational or otherwise inconsistent with the Manual. [11] While the system allows for more rapid progression, it also contemplates situations where an employee is unable to further increase her contribution. The Manual provides that "[i]t is reasonable to expect an employee's ability to increase contribution to eventually plateau .... In [this] case[], artificial bumps that increase scores (such as from a 2.2 to 2.3) should be avoided if contribution remains the same.” Id. at KAPL000283. [12] This is a reference to Colonel Kraus’s replacement of some of the language from Dr. Kaplan's self-assessment with that of the rubric's description of a lower broadband level during the end-of-cycle assessment that set her pay for 2015. Dr. Kaplan also argues that, since Colonel Kraus copied the language, his assessment was not based on her actual contribution. We are satisfied that this was Colonel Kraus's way of describing her actual contribution and note that her AFRL Form 280 from that year also mentions a number of specific contributions that she made. See PX 37. Moreover, as defendant points out, Ms, Michelle Williams testified that she did not find Colonel Kraus's taldng excerpts from the rubric in the manner that he did to be inappropriate. Tr. 588. [13] We note that a presentation in March of 2016 would be relevant for setting Dr. Kaplan's pay for 2017, and that Dr. Kaplan was removed from her position in November of 2016. We find the SAB’s assessment to be relevant in that it demonstrates the negative impact that Dr. Kaplan had over time on an inherited program. [14] Plaintiff argues that she was denied a number of advancement opportunities that she believes would have helped her bolster her contribution. We are satisfied that her supervisor’s actions in this regard were based on legitimate management considerations and are not a reflection of gender bias. In her post-trial brief, plaintiff also makes a general argument regarding other female employees that were paid less than the male employees they replaced. At trial, Dr. Kap-lan made a relevance objection when defendant's counsel began a line of questioning designed to elicit facts that could rebut this argument. We sustained plaintiff’s objection after she stated that she would not be pursuing such an argument. Tr. 291.