Gordon W. Harvey v. Merit Systems Protection Board

SWYGERT, Senior Circuit Judge. Petitioner Gordon W. Harvey appeals from a final order of the Merit Systems Protection Board (“the Board”) in which the Board found that Harvey had committed three prohibited personnel practices during his tenure as Assistant Inspector General for Audits of the Office of Inspector General for the Department of Energy and, as a sanction, demoted Harvey from the Senior Executive Service for three years. Harvey argues that the Board’s findings are not supported by substantial evidence and are based on erroneous interpretations of the Civil Service Reform Act. 5 U.S.C. §§ 2301 et seq. (1982). Harvey also challenges the Board’s authority to issue the penalty it imposed on him. We agree with Harvey that the Board's findings are not supported by substantial evidence, and we therefore reverse. I The petitioner is a professional auditor and a career civil servant who worked for twenty-two years in a variety of increasingly responsible federal government positions, culminating in his assignment in 1981 to serve as the Assistant Inspector General *539 for Audits (“AIG/A”) of the Department of Energy. He has been a member of the Senior Executive Service since its inception in 1979, and he received training in the rights and obligations of senior executives. Prior to being recruited as AIG/A, Harvey served as Assistant Administrator for Enforcement of the Energy Department’s Economic Regulatory Administration. Harvey was recruited for the AIG/A position in October 1981 by James Richards, Inspector General of the Department of Energy. As the AIG/A, Harvey reported directly to Richards. Richards succeeded Acting Inspector General James Wright, taking office in September 1981. Richards testified that upon assuming office he immediately set out to reorganize the Inspector General’s Office, particularly its audit function, in order to impart a “semblance” of structure to an office that had been a “shambles,” devoid of professional standing and widely perceived as ineffective. Richards directed Harvey to reorganize the office of the AIG/A within sixty days. Harvey assumed the position of AIG/A on October 18, 1981, but he did not report for duty until October 26, 1981. During that week, Harvey studied the organization and staffing of the office, and planned its reorganization. He concluded from information obtained from several sources that the Department of Energy audits office “was virtually paralyzed as an effective audit organization” and that the entire Inspector General’s office was held in low esteem by the Secretary of the Department of Energy. He believed that one route to an effective reorganization of that office was to secure a fresh management team. His ability to employ this route was severely limited because the office only had three members of the Senior Executive Service. He had been informed by Richards that to accomplish the task of reorganization his only flexibility lay with his highest-ranking subordinate, Charles V. Gorsey, Director of Financial and Procurement Audits and one of the three members of the Senior Executive Service who, notwithstanding his management title, was at that time working for a GS-15 employee. Harvey determined that Gorsey should be replaced. Harvey contacted the Department of Energy’s Office of Personnel and was informed that Gorsey was a reemployed annuitant, readily removable without cause. Harvey recommended to Richards that Gorsey be terminated; Richards agreed on October 27, 1981. On that same day, Harvey met with the entire audit staff to inform them of policies he expected to be followed. In particular, he told the staff that he wished all contacts from Congress, the General Accounting Office, and the media to be referred directly to him. After the meeting, he informed Gorsey that Harvey planned to request Gorsey’s termination as a reemployed annuitant. Gorsey responded that he could not be dismissed without cause because he was not an annuitant. There is no dispute that Gorsey's performance under Harvey played no role in this decision and that the action stemmed from Harvey’s need for fresh management. On October 27,1981, Harvey sent Gorsey and Gorsey’s subordinates a memorandum ordering that all official matters be routed to him directly, rather than to Gorsey. He also requested Gorsey to make a “quick study” regarding certain improvements in audit procedures that Harvey believed would be useful to him in connection with the office reorganization. Harvey again contacted the Department of Energy’s Office of Personnel, who informed him on October 30, 1981 that Gorsey had completed one year in his position, and therefore was no longer considered a reemployed annuitant who could be peremptorily terminated. To carry out his original intent to replace Gorsey, Harvey then recommended to Richards that Gorsey be reassigned out of the Inspector General’s Office. Richards again agreed and said that he proposed to ask the Department’s Executive Personnel Board (a body headed by the Deputy Secretary of Energy that is responsible for advising the Secretary as to the assignment of the Department’s Senior Executives) that Gorsey be reassigned. Harvey was subsequently told *540 by the Department of Energy’s Office of Personnel that Gorsey’s reassignment was appropriate and would occur within a few days. After his meeting with Harvey on October 27, 1981, Gorsey contacted the personnel department and engaged a lawyer. He also began to work on the assigned “quick study” four or five days later. He submitted his report, which took him a few days to complete, three weeks later. Gorsey said that the delay was caused by the need to have the report typed. Harvey testified that the report was unsatisfactory and that, as a result, he gave Gorsey a follow-up assignment on November 20, 1981. In the meantime, in November 1981, senior officials at Department of Energy concluded that Gorsey should be reassigned to a position in Chicago. This decision was rescinded after Gorsey stated that he would retire rather than be transferred. On December 7, 1981 Gorsey turned in his follow-up report. According to Gorsey, the two-week delay was again occasioned by the need to have the report typed. Harvey testified that he was again dissatisfied with this product. As a result, and because by then Gorsey still had not been reassigned, Harvey undertook renewed efforts to reassign him. In that same month, a personnel officer told Harvey that he was working on Gorsey’s detail to the Economic Regulatory Administration. Between December 7,1981 and February 10,1982 Harvey assigned Gorsey no day-today tasks because, he testified, he believed that Gorsey was soon to be transferred, that he had been dissatisfied with the two reports Gorsey had already submitted, and that Gorsey had a sufficient backlog of audit reports to rewrite pending his transfer. Gorsey testified, however, that the backlog was in the hands of the branch chiefs who had been instructed “not to have any dealings with Gorsey.” In addition, Gorsey’s mailbox was taken away, he was not invited to staff meetings (even meetings encompassing the entire membership of the audit office), and he was not recognized in the reorganization as being a component of any office. On February 10, 1982 Gorsey was detailed to Department of Energy’s Economic Regulatory Administration. Gorsey subsequently wrote a letter to the Executive Personnel Board, requesting that it rescind his detail to the ERA because there was no work for him to perform. He noted that on February 26, 1982 vacancy appointments were issued for three of the four new Senior Executive Service positions which were created in Harvey’s office as a result of the reorganization and he expressed the belief that he was qualified to fill any one of them. He also arranged for a detail to the House Appropriations Committee. On March 8, 1982 Gorsey’s position as a Senior Executive in the entire Department of Energy was officially declared “surplus” as part of a department-wide, reduction-in-force, and Gorsey was placed in competition with other “surplus” senior executives for available positions within the department. In an effort to preserve his position at the Department of Energy, Gorsey drafted a complaint to the Special Counsel of the Merit Systems Protection Board in which he claimed that he had been deprived of work and detailed to the Economic Regulatory Administration by Harvey, Richards, and others at the department in reprisal for his having “blown the whistle” to various government agencies and the White House about mismanagement in the Inspector General’s Office prior to the arrival of Richards and Harvey. Gorsey gave this draft to a personnel officer at the Department of Energy with directions to show it to Richards. He told the personnel officer that he would not file the complaint with the Board’s Special Counsel if he was reassigned to a permanent position with the Inspector General’s Office. Gorsey offered to accept a downgrade to GS-14. Richards testified that he did not give the document much credence, but rather considered it an attempt to intimidate the personnel office. *541 When Gorsey received no response to this suggestion, he hand-delivered to the Special Counsel a signed complaint, dated March 9, 1982, which was similar, although not identical to the draft complaint. The final version encompassed all allegations of the draft except for the elimination of four lines referring to two other Department of Energy employees. Harvey read the draft complaint sometime before March 22, 1982. He testified that he was “angry” because the document he saw contained a number of misrepresentations about the work Gorsey had been assigned by Harvey and other false allegations regarding acts of reprisal against Gorsey. Harvey further testified that he did not learn until eleven months later — in February 1983 — that Gorsey had actually submitted a complaint to the Special Counsel. He claimed that until that time he believed that Gorsey “had dropped or never filed” the complaint. Gorsey subsequently applied for one of the three vacant Senior Executive positions in Oak Ridge, Tennessee, Washington, D.C., and Germantown, Maryland that had been created in the office of AIG/A (Harvey’s office) as part of the office-wide reorganization mandated by the Inspector General. All three new positions reported directly to the AIG/A, Harvey. J.M. Schulman, Executive Secretary of the Executive Personnel Board, cautioned Richards that he could refuse to accept Gorsey only if he did not meet the “technical qualifications” for the position. Richards solicited Harvey’s opinion as to whether Gorsey was “technically qualified” for any of them. Harvey said that Gorsey was not. Harvey testified that he had been advised by two senior Energy Department personnel officers that “technical qualification” includes the element of “personal integrity.” In his view, Gorsey lacked “superior moral integrity” because he had attempted to “blackmail” the department with a draft complaint that contained numerous misstatements and false allegations, and he also lacked the requisite managerial skills. Harvey therefore advised Richards and the Executive Personnel Board that he had “zero personal confidence that Gorsey could perform effectively in any of these positions.” Harvey testified to two reasons for his view. First, he believed that Gorsey had been in contact with the investigator from the GAO regarding a draft audit report in violation of Harvey’s October 27 instructions that all subordinates refer all outside inquiries to him. Second, Harvey was “angered” by Gorsey’s draft letter to the Special Counsel because he considered it to be inaccurate. Richards agreed with Harvey’s recommendation. As a result, the Executive Personnel Board formally found Gorsey “technically unqualified” for each of the three positions. In so doing, the EPB overruled the Energy Department’s personnel director, who had advised that although Harvey’s appraisal of Gorsey raised issues of “legitimate management concern,” Gorsey nevertheless met the technical qualification requirements for the jobs he sought. As a result, the department did not place Gorsey in any of those jobs, and the Secretary of Energy certified that Gorsey was unqualified for them and would have to be separated from service. The Office of Personnel Management later conducted its review and informed the Department that Gorsey “appeared to be qualified” for the three vacant positions and that none of them could be filled unless either a position was first found for Gorsey or the head of the agency filed an objection to Gorsey’s placement on the ground of a lack of technical qualifications for the vacant jobs within ten days. No objection was filed. The Department elected to try to find another job for Gorsey. Harvey, Richards, and the Personnel Office determined that he might be accommodated as a deputy manager of one of the three regional offices (Germantown, Maryland; Oak Ridge, Tennessee; and Albuquerque, New Mexico) that had been created in the course of the reorganization. Harvey considered the possibility of working out a situation for Gorsey at each office. The regional office *542 located in Germantown was already staffed with two “associate” managers and could not support a senior executive deputy as well. The regional offices in Oak Ridge, Tennessee and Albuquerque, New Mexico had been created with positions for two “assistant managers” each, but those slots had not yet been filled, and the Personnel Office told Harvey that either regional office could support only a single senior executive deputy in lieu of two assistants. Harvey consulted with the manager of the Albuquerque office who, in turn, informed him that he was willing to take Gorsey as his deputy. Harvey therefore recommended that Gorsey be reassigned to a new deputy manager position in Albuquerque. Harvey testified that he made this recommendation to remove Gorsey from the Washington area so that he would no longer have a “base to carry on his disruptive operations.” Richards agreed, and the Personnel Board directed Gorsey’s reassignment to Albuquerque. The Office of Personnel Management’s approval for establishment of the position was obtained. When Gorsey was nominated to OPM for the new position, he was presented as “highly qualified.” To implement the Board’s decision, the Personnel Office created a deputy manager position in Albuquerque by merging the two GS-15 positions, and a reassignment was proposed to Gorsey in July 1982. Gorsey accepted the offer, but before moving to Albuquerque he made use of OPM’s Senior Executive Service job referral mechanism and found another senior executive position at the Department of the Navy in the Washington area. Gorsey left the Department of Energy in August 1982. Following Gorsey’s departure, the senior executive manager position in New Mexico, was abolished. The OPM subsequently conducted a review of the performance appraisal systems and reductions-in-force procedures effected in 1982 as applied to Senior Executive Service. The Office of Personnel Management, noticing numerous irregularities which necessitated corrective action, found that Gorsey should have been placed in any one of the three vacant SES positions within his own office. Gorsey subsequently withdrew his complaint and the Special Counsel official closed the file on September 15, 1982. For reasons that are unclear, the Special Counsel reopened the file in January 1983 and formally filed a disciplinary action against Harvey on August 24, 1983 pursuant to 5 U.S.C. § 1206(g). The complaint alleged that Harvey had committed five prohibited personnel practices in violation of the Civil Service Reform Act, 5 U.S.C. §§ 2302(b)(8)-(11). In Count I the Special Counsel alleged that Harvey violated section 2302(b)(ll) in that, from October 26, 1981 until March 31, 1982, he took, recommended, and caused to be taken personnel actions which significantly changed the duties of Gorsey, thereby effecting Gorsey’s de facto removal from the Senior Executive Service in violation of 5 U.S.C. §§ 3131(7), 3592(a)(2), 3595(b)(3)(A), and 7543 which implement and directly concern the merit system principles found in 5 U.S.C. §§ 2301(b)(2), (5), (6), and (8)(A). Count II alleged that Harvey violated section 2302(b)(8) by recommending that Gorsey not be reassigned to any of the three vacant SES positions in AIG/A and that he be reassigned to an SES position in New Mexico in reprisal for his having “blown the whistle” to congressional investigators, a White House staff member, and Special Counsel. Count III alleged that Harvey violated section 2302(b)(9) by recommending that Gorsey be assigned to New Mexico, not within AIG/A, in reprisal for Gorsey having drafted and mailed a complaint to the Office of Special Counsel alleging various improprieties at DOE and retaliatory actions taken against him. Count IV alleged that Harvey violated section 2302(b)(10) by discriminating against Gorsey on the basis of conduct which did not adversely affect the performance of his duties. Count V alleged a second violation of section 2302(b)(ll) based on Harvey’s supposed reprisal against Gorsey. *543 A hearing on the charges was held in November 1983 before the Board’s administrative law judge. On May 1, 1984 the judge issued his recommended decision in which he found Harvey guilty of two of the five charged prohibited personnel practices. In particular, he found that Harvey had violated section 2302(b)(9) by obtaining Gorsey’s reassignment from Washington to a job in Albuquerque in reprisal for Gorsey’s filing of the complaint, and section 2302(b)(ll) by “deliberately idling” Gorsey from December 7, 1981 until February 10, 1982. He also found that the evidence did not show that Harvey retaliated against Gorsey for Gorsey’s supposed “whistle-blowing,” as alleged in Count II. He dismissed Count IV, alleging discrimination, as duplicitous of the other counts in the complaint, as well as Count V. The Board affirmed in part and reversed in part. It affirmed the judge’s findings that Harvey had violated the Act by “deliberately idling” Gorsey for two months and by retaliating against Gorsey for Gorsey’s exercise of his appeal rights to the Special Counsel. It also affirmed the judge’s finding that Harvey did not retaliate against Gorsey for his alleged whistleblowing. The Board, however, reversed the judge’s decision to dismiss the discrimination charge as duplicitous, and it found that Harvey had discriminated, as charged. It also reversed the judge’s decision to accord Harvey the benefit of the so-called Mount Healthy defense in determining whether Harvey had engaged in retaliatory or discriminatory conduct. See Mount Healthy City School District Board of Education v. Doyle, 429 U.S. 274, 97 S.Ct. 568, 50 L.Ed.2d 471 (1977). In addition, the Board changed the punishment the judge imposed on Harvey from removal from the Senior Executive Service and reduction to grade GS-14 to removal from the Senior Executive Service and demotion to a nonmanagerial position in grade GS-14 for a period of three years. The Board directed that the sanction be put into effect within thirty days and that the Special Counsel report annually to the Board regarding compliance with the penalty. Harvey filed a timely petition for review in this court. His motion for a stay of the Board decision was denied by this court on January 3,1985 and the sanction imposed by the Board became effective on January 6, 1985. II Harvey’s primary contention on appeal is that the Board’s findings in support of its conclusion that he committed the three prohibited personnel practices are not supported by substantial evidence. See 5 U.S.C. § 7703(c) (1982). “Substantial evidence” provides only a narrow scope of judicial review. The reviewing court must affirm the decision of the agency if it is based on such “relevant evidence as a reasonable mind might accept as adequate to support [the] conclusion.” United States v. Carlo Bianchi & Co., 373 U.S. 709, 83 S.Ct. 1409, 10 L.Ed.2d 652 (1963). With the appropriate standard of review in mind, we turn now to a review of the Board’s findings. A. The Board found that Harvey committed a “prohibited personnel practice” in violation of section 2302(b)(ll) of the Act. That section proscribes the taking of personnel action that violates “any law, rule, or regulation implementing, or directly concerning, the merit system principles contained in section 2301 of the Act.” The statutory rubric underlying the charged violation is found in section 2302(a)(2)(A)(x) which provides that “ ‘personnel action’ means ‘any ... significant change in duties or responsibilities which is inconsistent with the employee’s salary or grade level.’ ” In this disciplinary proceeding, the Board found that Harvey had taken “personnel action” within the meaning of section 2302(a)(2)(A)(x) by “significant[ly] changpng] ... [Gorsey’s] duties ... [in a manner that was inconsistent with] ... [Gorsey’s] salary or grade level” by “deliberately idling” Gorsey for two months. The Board ruled that in so doing Harvey had violated a law, section 3131(7) of the *544 Act, which provides that “[t]he Senior Executive Service shall be administered so as ... to protect senior executives from arbitrary and capricious action.” Finally, the Board ruled that section 3131(7) directly concerns the merit principle stated in section 2301(b)(5): “The Federal work force should be used efficiently and effectively.” Distillation of this statutory maze reveals that the Board found that Harvey committed a prohibited personnel practice in that he significantly changed Gorsey’s duties by “deliberately idling” Gorsey for two months, thereby subjecting him to arbitrary and capricious action resulting in an inefficient and ineffective use of Gorsey. Harvey contends that there is no evidence to support the Board’s finding that he “deliberately idled” Gorsey or that he otherwise subjected Gorsey to arbitrary and capricious treatment. We agree. 1 Initially, it should be noted that the administrative law judge found, and the Board agreed, that Harvey did not commit a prohibited personnel practice from October 26, 1981 until December 7, 1981. The administrative law judge found, and the Board agreed, that when Harvey assumed his position as Assistant Inspector General for Audits, Gorsey had no supervisory duties, and in fact worked for a GS-15. Gorsey merely reviewed and rewrote the audit reports of subordinates. The administrative law judge’s conclusion that Harvey significantly changed Gorsey’s duties by deliberately idling him is based solely on Harvey’s removing Gorsey, on October 27, 1981, from the supervisory chain of command by directing Gorsey’s three branch chiefs to route all official matters directly to Harvey and on Harvey's failure to ensure, after December 7, 1981, that Gorsey had audit reports to rewrite. Presumably, if Harvey had continued to assign Gorsey menial projects, without monitoring Gorsey’s progress on them, or had ensured that once Gorsey completed the two projects he had audits to rewrite pending Gorsey’s reassignment, he would not have significantly changed Gorsey’s duties or deliberately idled him. Thus, to the extent that Harvey relieved Gorsey of any supervisory duties on October 27, 1981, the Board is apparently willing to concede that this action did not constitute a significant change in Gorsey’s duties. The OSC, of course, bears the burden of proof by a preponderance of the evidence of every element of the charge. With regard to this charge, the OSC utterly failed in its burden of proof. The word “deliberate” in the critical charge of “deliberately idling” connotes “knowing” or “intentional” conduct. On this score, the record is singularly lacking in any evidence that Harvey knew or should have known that Gorsey had no work to do between December 7, 1981 and February 7, 1982. 2 The Board, however, relies on two pieces of evidence to support its finding: (1) the October 27, 1981 letter Harvey sent to Gorsey requesting Gorsey to undertake a “quick study” and informing Gorsey that three of his subordinates would thereafter be directly reporting to Harvey, and (2) Gorsey’s statement in his deposition that the backlog remained in his subordinates’ hands. Neither of these pieces of evidence provides any support for the Board’s conclusion. The October 27, 1981 letter supports only an inference, tenuous at best, that Harvey precluded Gorsey from receiving any new audit reports to rewrite. It does *545 not show that Harvey was aware or should have been aware that Gorsey had no backlog or that the backlogged audit reports were in the hands of Gorsey’s subordinates. In fact, it does not even support the Board’s contention that the backlog was in the hands of Gorsey’s subordinates. The fact that Harvey made no exception for Gorsey’s subordinates to submit their draft reports to Gorsey for editing and reworking is similarly lacking in probative value on the issue of whether Harvey reasonably believed that Gorsey had many reports on which he could work. 3 Gorsey’s testimony similarly does not show that Harvey was aware that Gorsey had no work left to do because of Harvey’s directive rerouting official business. In this regard, it is significant that the record is devoid of any evidence that Gorsey ever complained to Harvey or anyone else before February 10, 1982, when he was finally detailed out of the office, and for another month thereafter, that he had no work to do or that any of Gorsey’s subordinates complained about the backlog or rerouted it to another supervisor for rewriting. Since the memorandum ordered all subordinates’ work directed to Harvey, presumably the backlog would have been routed to Harvey, but there is no evidence that Harvey himself rewrote the reports or directed them to others to rewrite. In contrast to this complete lack of evidence showing that Harvey knew or should have known that Gorsey had no work, there is evidence in the record, apart from Harvey’s testimony, to show that Harvey could have reasonably believed that Gorsey did have a significant backlog of reports to rewrite and edit. Thus, we are compelled to conclude that there is no relevant evidence in the record from which a reasonable person could conclude that Harvey deliberately idled Gorsey. The Board further found that Harvey subjected Gorsey to arbitrary and capricious treatment by deliberately idling Gorsey and in making him a supemumerary in the AIG/A. We have already dealt with the lack of evidence to support the finding of deliberately idling. We agree, however, with the Board that the evidence supports the inference that Harvey made Gorsey a supernumerary, although not necessarily an idle one. We cannot agree, however, that the evidence demonstrates that by making Gorsey a supernumerary, Harvey subjected Gorsey to arbitrary and capricious treatment. We first note several undisputed and significant facts. Uncontradicted evidence demonstrates that the AIG/A office was in a chaotic state, and desperately in need of reorganization, that Richards ordered Harvey to set the office in order within sixty days, that Harvey was informed that his only flexibility with senior management lay with Senior Executive Service members, and that Harvey was further informed that Richards had been so dissatisfied by Gorsey’s performance that he had made Gorsey a subordinate, to a grade GS-15, rather than place him in charge. Significantly, the Special Counsel conceded that the office was undergoing a needed reorganization at this time and it did not argue that Harvey could not request Gorsey’s reassignment without regard to Gorsey’s managerial ability or that Harvey’s reassignment request was motivated by improper concerns. On December 24, 1981 Gorsey’s job and the jobs of all other senior managers in the office of AIG/A were abolished when the office reorganization was implemented. On January 8, 1982 Richards formally reported to the Executive Personnel Board that Gorsey was “surplus” and that he could not be assigned within the Inspector General’s Office, and he again recommended that Gorsey be placed elsewhere. In addition to these undisputed facts, the record contains Harvey’s uncontradicted testimony that he did not assign Gorsey any new tasks because he was dissatisfied with Gorsey’s work and because he believed that Gorsey would soon be reassigned. We believe that the only reason *546 able conclusion from these facts is that Harvey exercised his considered managerial judgment in making Gorsey a supernumerary. He reasonably assumed that Gorsey no longer had a place in the AIG/A and operated under that assumption. The Board, however, discredited Harvey’s contention that he gave Gorsey no new tasks because he was dissatisfied with Gorsey’s two memoranda on the ground that.Harvey never recommended any performance-based action against Gorsey. This is a slender reed on which to hang such a finding. Harvey had little reason to recommend such action against Gorsey because he expected him to be reassigned out of the office. Cf. Fucik v. United States, 655 F.2d 1089, 1097, 228 Ct.Cl. 379 (1981). Moreover, Harvey’s November 20, 1981 memo to Gorsey and a subsequent memo indicated that Harvey was not entirely satisfied with Gorsey’s projects. The Board further rejected Harvey’s assertion that he reasonably believed that Gorsey would soon be transferred out of the office. The Board supports this rejection on the ground that Harvey presented no credible evidence that he had reason to believe that the personnel action with respect to Gorsey would be completed “imminently.” Harvey testified that he was told by Department of Energy personnel officers that Gorsey would be soon transferred out of the office. The Special Counsel presented no contrary evidence. It also presented no evidence on the usual length of time necessary to effectuate a transfer or any evidence of any suspicious circumstances that would have led a reasonable manager to be aware that the transfer would not occur “imminently.” To be sure, an agency is entitled to reject a witness’ testimony as incredible. Yet in this case the Board apparently is willing to concede that Harvey did consult the Department’s personnel office with respect to the propriety of the proposed action against Gorsey and was in regular contact with that office regarding Gorsey’s transfer to another position. It is also undisputed that Harvey’s supervisor declared Gorsey as “surplus” in late December. We believe that the Board was not entitled to reject Harvey’s testimony, there being no contrary evidence and indeed there being supporting evidence that Gorsey was made a supernumerary by Richards and that Harvey was regularly tracking Gorsey’s proposed reassignment. Harvey simply cannot be penalized under this statutory framework for lacking the knowledge that the OSC and the Board only obtained by benefit of hindsight. In short, the Board’s findings on this charge amount to sheer speculation without foundation in the record. The burden was on the Special Counsel to prove the charge by a preponderance of evidence. It was not on the petitioner to disprove it by filling the Special Counsel’s evidentiary void. Moreover, the OSC was required to show not that Harvey failed to make the most efficient use of Gorsey, but that Harvey knowingly deprived Gorsey of all job duties and subjected him to arbitrary and capricious treatment. This it failed to do. The only reasonable inference to be drawn from the evidence is that Harvey did have legitimate management reasons for attempting to remove Gorsey from his office, but that perhaps he did not achieve it in the most efficient and graceful manner. We note that on appeal the Board makes no serious effort to defend this finding on the basis of record evidence. It simply asserts that “Harvey, for no sound management reason, deprived his only Senior Executive Service member of meaningful work for a two-month period” and “Harvey cannot seriously argue that what he did with respect to Gorsey was permissible management.” These conclusory assertions are an insufficient basis on which to sustain the charge. B. Count III charged Harvey with violating section 2302(b)(9) of the Act which reads: (b) Any employee who has the authority to take, direct others to take, recommend, or approve any personnel action, *547 shall not, with respect to such authority— * * * sfe * * (9) take or fail to take any personnel action ... as a reprisal for the exercise of any appeal right granted by any law, rule or regulation. To establish a violation of this section, the OSC had to prove by a preponderance of the evidence that (1) Gorsey engaged in an activity protected by the statute, (2) Gorsey was subsequently treated adversely, (3) Harvey knew or had constructive knowledge that Gorsey had engaged in protected activity, (4) that there was a causal connection between the protected activity and the personnel action, and (5) the protected activity was a significant factor in the taking of the action. In re Frazier, 1 M.S.P.B. 159, 1 M.S.P.R. 163 186 (1979), aff'd, Frazier v. Merit Systems Protection Board, 672 F.2d 150 (D.C.Cir.1982). The Board, affirming the administrative law judge, found that as a reprisal for Gorsey’s “appeal” to the Special Counsel, Harvey “had denied Gorsey consideration for three available SES positions [German-town, Maryland; Oak Ridge, Tennessee; and Albuquerque, New Mexico] and contrived a position in Albuquerque, New Mexico to remove Gorsey from the Washington area.” The Board defined the appeal as the draft of an undated letter to the Special Counsel which Gorsey had written and which he delivered after some modification to the Office of the Special Counsel on March 9, 1982. In that draft, Gorsey allegedly complained that after Harvey’s arrival in the department he was given little, if any, work to perform in reprisal for his whistleblowing activities. Harvey raises three challenges to this charge, only one of which we address. 4 We need not, today, define the precise contours of a claim of retaliatory conduct on the part of a supervisor. We need only observe that previously we have decided that retaliation is to be defined broadly to include any action designed to punish an employee for exercising his protected rights or to deter him from exercising those rights. See, e.g., Donovan v. Stafford Constr. Co., 732 F.2d 954, 959-60 (D.C.Cir.1984). Thus, in this case, to affirm the Board we need only find that there is substantial evidence in the record to show that Harvey took action in order to punish Gorsey for having exercised a protected appeal right or to deter him from exercising that right at a later date. In this regard, Harvey contends that the evidence only shows that he had distinct, valid managerial reasons for his actions and that therefore there was no showing that there was a causal connection between Gorsey’s protected activities and Harvey’s actions or that Gorsey’s protected activity was a significant factor in the taking of the action. We agree. The Board’s finding on this charge rests almost exclusively on its conclusion that Harvey admitted he “retaliated” because of Gorsey’s draft letter. The record does not bear out this finding. Harvey admitted only that he became “angry.” He later testified that when Gorsey was being considered for one of the three open Senior Executive Service positions, his recommendation was “unfavorable” to Gorsey. Because of its importance to our review, we quote pertinent parts of Harvey’s testimony on which the Board expressly relied in making its finding that Harvey admitted he retaliated: I read the letter to — the draft letter to the Special Counsel which Mr. Gorsey *548 gave to the Department. On little things in there, Mr. Gorsey made a statement that he was given no work to do, that he was given two small tasks by me and it only took him two to three days. He was given a task by me on the 27th of — his first task on the 27th of November which, by the way, was an addendum task and not a make work task, and he was right. It should have only taken him two to three days. He finished that task some 25 days later in late November. At that point in time, I gave Mr. Gorsey another task, which in his draft of it that he passed around the Department, he said it only took him three or four days. He did not complete that task but sometime in mid to late December. It certainly made me wonder about an individual who was apparently flippant with charges. I have testified before that I know for a fact that these charges concerning me are absolutely false. If an individual is going to make these allegations, in small matters, small matters, he is untruthful. I wonder where he may be on the larger issues. How does this relate to my selection? I consider technical competence, as I mentioned, to be broader than what the 0PM subsequently decided it to be. By the way, the OPM would not have had any knowledge of the process that I went through. But I certainly considered technical competence in the way this job was advertised to go beyond the pure paper academic experience qualifications. Under that broad definition of technical competence, I believe that Mr. Gorsey did not measure up on that job, and that’s why I recommended he not be in those jobs. Earlier Harvey had testified, “It’s a job where you are obtaining and dealing with often times, confidential information and that a person in that job needs to have the demonstrated integrity beyond what might normally be expected of other jobs.” As the Board concedes, there was no contradictory evidence. This evidence must be viewed for what it actually shows: Harvey’s action in not recommending Gorsey for the SES openings was not in retaliation for Gorsey’s exercise of his appeal rights or an attempt to deter him from exercising those rights, but was a management decision about Gorsey’s qualifications, both technical and personal. To be sure, that management decision was based on an opinion that Harvey formulated of Gorsey based in part on Gorsey’s allegations in his draft complaint and his efforts to use the draft to preserve his job. In that regard, of course, there is some link between Harvey’s actions and Gorsey’s protected conduct. But it is not the type of prohibited link covered by the Act. See Starrett v. MSPB, 792 F.2d 1246 (4th Cir.1986). Formulating an adverse opinion of an employee, based upon what he has written and thereby not recommending him for certain jobs, is not the same as taking action against an employee in an attempt to thwart his exercise of his protected rights. If it were, it would mean that one in an executive position can never exercise his considered judgment in making personnel recommendations when asked to do so when that judgment is based on anything even tangentially related to the exercise of protected conduct. See, e.g., Starrett, 792 F.2d at 1253 n. 12. 5 *549 The Board, however, attempts to bolster its position by reciting evidence to show that Gorsey’s personal qualifications were not a valid basis on which to withhold recommendation, that the OPM found later that Gorsey should have been referred to one of those three positions, and that the fact that Harvey’s recommendation was accepted by others should be discounted because the decisions made by the Executive Personnel Board were routinely improper. This evidence is irrelevant for it has no probative value on the issue of whether Harvey’s actions against Gorsey were motivated by a desire to punish Gorsey for exercising his statutorily-protected rights. Simply put, whether the Department of Energy’s referral system was improperly or ineffectively run in and of itself sheds no light on Harvey’s state of mind. This is particularly true when the Board does not even attempt to show that Harvey had a hand in that referral system or to dispute Harvey’s contention that others solicited Harvey’s opinion of Gorsey, including his opinion as to Gorsey’s “personal integrity” or that others generally concurred in Harvey’s evaluation of Gorsey. Similarly, the Board is clearly correct that the evidence supports the inference that Harvey and others contrived a position in Albuquerque to remove Gorsey from the Washington, D.C. area to prohibit Gorsey from carrying out his “disruptive operations” activities. But again this evidence, read in the context of the entire record, does not support the inference that Harvey was motivated by prohibited retaliatory concerns. Harvey’s reference to “disruptive operations” was to Gorsey’s supposed communications with Schaffer and his efforts to preserve his position within the AIG/A. The relevant passages from the record support this conclusion: I believed by then that the reassignment was not going to happen. For some reason, Mr. McCormick’s attempts to reassign had failed. By then a lot of water had gone over the dam. In his letter to the Office of the Special Counsel, Gorsey had acknowledged his surreptitious involvement with the congressional investigator, and had indicated there were five or six others involved. My reading of the draft letter which he had provided to various people in the Department revealed a man who was rather “flippant” about charging his bosses with irregularities. I certainly know the charges he made about me to be false. Therefore, it was my judgment that Gorsey was not someone I could trust to carry out my policies and programs, especially at a level reporting directly to me. His letter revealed a management philosophy that runs counter to my own. There were, however, two potential positions that were considered for Mr. Gorsey. The Germantown position was available; however, it was a senior management position. It didn’t make good management sense to me to place Gorsey in the Germantown billet and give him a base to potentially carry on the disruptive actions that he had admitted to in the draft letter to the Office of the Special Counsel. The other position was as Deputy Regional Manager, Western Regional Audit Office, Office of the Inspector General, Albuquerque, New Mexico. We combined two GS-15 Assistant Regional Manager positions and created an SES Deputy Manager slot for Gorsey. There was a job to do there and Gorsey would be under the supervision of a strong manager. It is my understanding that Gorsey verbally accepted this offer. When Gorsey left the Department for a position in the Department of Navy, the Deputy Manager’s position was abolished and the two Assistant Regional Manager positions were reestablished. ****** Q: You told Mr. Russell that one of the reasons that you didn’t want to reassign Mr. Gorsey, either in your shop, in the District of Columbia, or to Germantown was that you didn’t want him to have a base to continue his operations. A. And I think I said his disruptive operations. *550 Q: Yes, I think you did. What operations of his did you consider to be disruptive? A. Gorsey’s recommendation as I learned it and also as I observed it, either through documents that were there or the GAO situation I just described to you, was that basically he did not subscribe to an office that had structure, that he seemed to go around assigned leadership within his own organization if he didn’t get his way or what he perceived to be the right way, as evidenced by the GAO involvement. I think there’s reasonable evidence that he would go outside the organization if he doesn’t get his way. Q: You had considerable evidence, at that time, that he would go to the Congress when he thought he should also, didn’t you? That Harvey wished to remove Gorsey from within Harvey’s direct sphere of influence cannot be doubted. But neither can it be doubted that Harvey’s conduct was not motivated by Gorsey’s protected activities, but rather by other personal activities (that were intertwined with Gorsey’s protected activities) that Harvey found to be unbecoming of an SES employee. As a result, it is of no consequence in this disciplinary proceeding whether or not Harvey was correct in his assessment of Gorsey’s activities because even if he was wrong, improper motivation was still lacking. See Starrett, 792 F.2d at 1255. Based on the foregoing, although we are inclined to believe that there cannot even be a valid inference from the evidence that Harvey’s conduct was motivated by personal spite or “to get even,” we are absolutely certain that there is no evidence in the record to support any inference that Harvey’s actions were motivated by a desire either to punish Gorsey for exercising or deterring him from exercising his appeal rights. It is for those reasons that we must reject the Board’s finding that Harvey violated section 2302(b)(9) of the Act. The “substantial evidence” test requires no less. C Count IV of the Special Counsel’s complaint charged Harvey with violating section 2302(b)(10) by discriminating against Gorsey on the basis of conduct that did not adversely affect Gorsey’s job performance. The Special Counsel alleged that the prohibited actions of Harvey toward Gorsey charged in Counts II and III also constituted a violation of section 2302(b)(10). The administrative law judge dismissed the charge and in his recommended decision wrote: In support of this count petitioner [Special Counsel] has incorporated by reference the allegations of fact recited in the prior three counts of this complaint____ [T]he separate but identical facts upon which these three other counts are individually predicated are now combined to give rise to a fourth— The cumulation of offenses based upon the same conduct amounts to multiplicity. 6 The Board reversed the administrative law judge’s dismissal, stating in its decision: Count four charged that Harvey had violated section 2302(b)(10) by significantly changing Gorsey’s duties (as alleged in count one); and by recommending against Gorsey’s reassignment to any of three vacant SES positions in the Washington, D.C., area and by recommending Gorsey’s reassignment to New Mexico (as alleged in counts two and three). Count four does not duplicate the first three counts. Although Special Counsel relies in count four on the same alleged retaliatory conduct by Harvey; that is, denial of the SES position and transfer to *551 Albuquerque, his conduct must be analyzed under the provisions of section 2302(b)(10). There is much to be said for the administrative law judge’s view that the inclusion of Count IV was impermissible because of multiplicity of charges in this quasi-criminal proceeding. Although the Board, to support Count IV, cited subsidiary facts different from those it used to support Count II, the ultimate findings of a prohibited action on the part of Harvey are identical in both counts, as the Board admits. For that reason, we are tempted to agree with the administrative law judge; however, we believe that there are stronger reasons upon which to base the dismissal of the count. The Board’s analysis of the facts it cites concerning Gorsey’s conduct and Harvey’s reaction to support Count IV does not permit a holding that section 2302(b)(10) was violated. The Board focuses on a draft audit report prepared by the Inspector General’s Office prior to Harvey’s appointment as Assistant Inspector General for Audits. Apparently, Thomas Shaffer, an investigator for the General Accounting Office, became aware of the report’s existence and sought to read it. On the day after his appointment as Assistant Inspector General, Harvey informed his staff, including Gorsey, that, to carry out a policy promulgated by Richards, all communications between his office and the GAO were to be channeled through him and that Harvey was also to be informed of all communications from the GAO. On November 25, 1981, Harvey was told by Shaffer that he had been communicating directly with Gorsey and Joseph Ferri, another auditor in the Inspector General’s Office. There is testimony in the record that Shaffer had received the draft report early in 1981 and that neither Harvey nor Shaffer referred to the report during this November 25 discussion. Rather, they discussed future communications between the GAO and the Inspector General for Audits. There is contradictory testimony that Shaffer informed Harvey that he had discussed the draft report with both Gorsey and Ferri, and that when questioned by Harvey, Ferri stated that he had not talked to Shaffer about the report. In light of Ferri’s denial, the Board found that Harvey “leaped to the conclusion that Gorsey had revealed the report contrary to orders” and that Harvey considered Gorsey insubordinate. It found that Harvey believed incorrectly that Gorsey revealed the report, that he improperly concluded that Gorsey lacked integrity, and that he therefore did not recommend Gorsey for one of the three Senior Executive Service regional positions. The Board concluded, “Harvey must have retaliated against Gorsey for conduct that did not adversely affect his job performance or the performance of others” because Gorsey did not engage in such conduct. There is no logic in the Board’s position. The Board apparently concedes that Harvey could have taken personnel action against Gorsey if Gorsey had in fact released the report or disclosed its contents in contravention of Harvey’s directive. This concession is well taken. Section 2302(b)(10) proscribes discrimination against an employee only on the basis of that employee’s conduct which does not adversely affect performance. If Gorsey had communicated with Shaffer about the draft report, and in so doing had disobeyed Harvey’s order, that conduct would have been directly related to his performance as an employee in the Inspector General’s audit division. Perforce it would not have been included within the coverage of section 2302(b)(10). The legislative history supports this view. That history makes clear that section 2302(b)(10) speaks only to an employee’s conduct totally unrelated to his job performance, such as a conviction for crime or sexual propensity. For example, the Conference Report reads in part: The conferees intend that only conduct of the employee or applicant that is related to the duties to be assigned to an employee or applicant or to the employee’s or applicant’s performance or the performance of others may be taken into consid *552 eration in determining that employee’s suitability or fitness. Conviction of a crime which has no bearing on the duties to be assigned to an employee or applicant or on the employee’s or applicant’s performance or the performance of others may not be the basis for discrimination for or against an employee or applicant. Rather, the Board contends that because Gorsey did not violate Harvey’s order, and Harvey undertook no efforts to ascertain whether Gorsey had in fact violated the order before formulating an opinion of Gorsey, Harvey discriminated against Gorsey by taking personnel action against him. We disagree. In our view, it is simply irrelevant whether Harvey erroneously, either honestly or recklessly, believed that Gorsey had violated Harvey’s directive, for Harvey’s actions still would not be covered by section 2303(b)(10). The conduct that Harvey, whether rightly or wrongly, ascribed to Gorsey was, as we have demonstrated, performance-related. The focus of section 2302(b)(10) is on the nature of the alleged conduct, not on whether or not the conduct was actually engaged in. If, for example, Harvey had recommended against Gorsey because he believed that Gorsey was a convicted felon, he would have committed a prohibited practice regardless of whether Gorsey was in fact a convicted felon. Whatever are the merits of the Board’s position that a supervisor should investigate whether an employee actually engaged in improper performance-related conduct before taking personnel action, that position has not been incorporated into the Act by Congress and hence cannot form the basis for disciplinary action. Our determination that section 2302(b)(10) is not applicable should end the matter. We shall assume, however, that the Board’s strained interpretation of section 2302(b)(10) is somehow warranted; even then Count IV must fall. The Board found that Gorsey’s alleged insubordination formed the basis of Harvey’s conclusion that Gorsey lacked “integrity.” Harvey’s conclusion was justified on the equivocal evidence presented at the hearing, even if he was mistaken in his belief that Gorsey had talked to Shaffer about the draft report. Harvey should not be punished for an honest mistake and, most certainly, if he was not mistaken. His belief that Gorsey lacked “integrity” and therefore should not be recommended for one of the available SES positions was a managerial decision which was honestly arrived at. It was his duty as Gorsey’s supervisor to furnish his evaluation of Gorsey’s qualifications. As a responsible official he could not remain silent. The petitioner raises other issues such as the applicability of the Mount Healthy defense in disciplinary proceedings brought under the Act and whether the Board exceeded its authority in imposing certain penalties against the petitioner. Because of our determination that none of the charges upon which Harvey was found guilty of a prohibited personnel action is supported by substantial evidence, we need not reach these other issues. The petition for review is granted and the Board is directed to dismiss its complaint and to take such further action as is necessary to restore Harvey to the position he retained prior to the imposition of the penalty. Footnotes [1] Harvey also contends that he had no ability to take or recommend any "personnel action” within the meaning of section 2302(a)(2) of the Act. He also contends that section 3131(7) is not a law that implements or directly concerns any merit system principle. Although we agree with Harvey that the route taken by the Board to define this violation of section 2302(b)(ll) is circuitous, at best, we believe it is not necessary to reach these issues. For even assuming that the Board is correct in juxtaposing these sections to create a prohibited personnel practice, the charge is not supported by substantial evidence. [2] In addition, we note that the OSC offered no proof as to what Gorsey’s duties actually were. All we know from the record is that when Harvey assumed command, Gorsey had no supervisory duties and that he rewrote audit reports. Nothing in the record shows whether Gorsey had any other duties. [3] In fact, the opposite inference is more appropriate. Had Harvey been aware that the backlog was in the hands of the subordinates, he presumably would have made the exception. [4] The record shows that Harvey did not learn of the actual filing of the letter to the Special Counsel until February 1983, long after Gorsey had voluntarily left the Department of Energy. Because either actual or constructive knowledge of an employee’s exercise of his appeal rights is essential to a violation of section 2302(b)(9), Harvey argues that any personnel actions he may have taken in not recommending Gorsey for reassignment to any of the three regional SES offices could not have had their source in an appeal of which he had no knowledge. Harvey also contends that Gorsey’s conduct "consisted of blackmail and defensive maneuvering, not exercise of a statutorily protected appeal right” and, therefore, Harvey’s reaction could not constitute prohibited reprisal. [5] In Starrett, the Fourth Circuit observed: [S]uch a difference, valid as it may be in some contexts, may not support the use of a standard of causation which inadequately protects personnel ... who are forced to make scores of personnel decisions which may peripherally or accidentally involve situations where employees have engaged in protected practice. The Act was designed to prohibit retaliation or reprisals by personnel because of improper consideration of protected actions. The standard of proof used must insure that the motivation for the adverse action was an improper one. There is no support in the legislative history of the Act for any particular standard of causation, but the standard used by the Board for any action under the Act, whether a "corrective action" under 5 U.S.C. § 1206(c)(1)(B) or a “disciplinary action” under 5 U.S.C. § 1206(g), must not be so loose or weak as to punish those not motivated by improper purposes. Id. at 1253 n. 12. [6] The judge further noted: That the pleadings in this complaint are prolix is further manifest by my conclusion that Count Four is inherently duplicitous as well. Duplicity is the charging of more than one offense in a single count. In Count Four, Harvey is charged with (1) the impermissible removal of Gorsey from SES; and (2) discriminatory personnel actions against Gorsey.