In re The City of New York

JOSÉ A. CABRANES, Circuit Judge: The question presented is whether we should issue a writ of mandamus to overturn an order of the United States District Court for the Southern District of New York (Richard J. Sullivan, Judge) granting a motion to compel the production of certain sensitive intelligence reports prepared by undercover officers of the New York City Police Department (“NYPD”). In answering that question, we are called upon to examine the circumstances in which the so-called “law enforcement privilege” must yield to the needs of a party seeking discovery in a civil action. Plaintiffs-respondents (“plaintiffs” or “respondents”) are protesters and other persons who were arrested, detained, and fingerprinted after demonstrating at the 2004 Republican National Convention (“RNC”) in New York City. They brought the underlying suits under 42 U.S.C. § 1983 and state law claiming that their arrest and treatment at the hands of the NYPD violated the United States Constitution and New York law. During pretrial discovery proceedings, plaintiffs brought a motion to compel the City to produce roughly 1800 pages of confidential reports created by undercover NYPD officers who were investigating potential security threats in the months before the RNC. (Using the NYPD’s parlance, we refer to these 1800 pages of reports as the “Field Reports” or simply the “Reports.”) The City opposed the motion to compel by asserting, among other things, that the documents were protected from disclosure by the law enforcement privilege. Magistrate Judge James C. Francis IV, assigned by the District Court to address all “[g]eneral [p]retrial” matters in the litigation, 1 granted plaintiffs’ motion to compel. The City filed objections to that decision with Judge Sullivan, see Fed.R.Civ.P. 72(a), and Judge Sullivan affirmed Magistrate Judge Francis’s order in its entirety, MacNamara v. City of N.Y., Nos. 04 Civ. 9216, 04 Civ. 7922, 04 Civ. 7921, 2009 WL 4789421 (S.D.N.Y. Dec.14, 2009). (For the remainder of this opinion, we will refer to the rulings of Magistrate Judge Francis and Judge Sullivan collectively as the rulings of the “District Court.”) The City then filed this petition for a writ of mandamus seeking relief from the order granting plaintiffs’ motion to compel. We hold that the City’s petition presents an “exceptional circumstance[ ]” warranting the “extraordinary remedy” of a writ of mandamus. Cheney v. U.S. Dist. Court for Dist. of Columbia, 542 U.S. 367, 380, 124 S.Ct. 2576, 159 L.Ed.2d 459 (2004) (internal quotation marks omitted). We reach that conclusion for the following reasons: *929 First, a writ of mandamus is the only “adequate means” for the City to seek review of the District Court’s order and thereby prevent the irreparable harm that the City — and thus the public — would suffer from the disclosure of the Field Reports. See id. (internal quotation marks omitted). In particular, we reject the idea that the secrecy of the Field Reports can be protected by disclosing them on an “attorneys’ eyes only” basis and filing them “under seal.” We have no trouble concluding, therefore, that an appeal after a final judgment is not an “adequate means” for the City to “attain the relief [it] desires.” Id. (internal quotation marks omitted). Second, because we have never before addressed the circumstances in which the law enforcement privilege must yield to a party’s need for discovery, this petition presents “novel and significant questions] of law ... whose resolution will aid in the administration of justice.” In re S.E.C. ex rel. Glotzer, 374 F.3d 184, 187 (2d Cir.2004) (internal quotation marks omitted). We are therefore “satisfied,” in “the exercise of [our] discretion,” that “the writ is appropriate under the circumstances.” Cheney, 542 U.S. at 381, 124 S.Ct. 2576. Third, we conclude that the City has a “clear and indisputable” right to the writ, id. (internal quotation marks omitted), because the District Court indisputably “abused its discretion” in making three distinct errors. Specifically, after determining that the law enforcement privilege applied, the District Court indisputably erred in failing to apply a “strong presumption against lifting the privilege.” Dellwood Farms v. Cargill, Inc., 128 F.3d 1122, 1125 (7th Cir.1997). The District Court also indisputably erred in failing to require that plaintiffs show a “compelling need” for the Field Reports. Cf. Marriott Int’l Resorts, L.P. v. United States, 437 F.3d 1302, 1307 (Fed.Cir.2006). Finally, the District Court made a clearly erroneous assessment of the evidence when it found that plaintiffs’ need for the Field Reports outweighed the public’s interest in their secrecy. Each of these errors may, on its own, merit a writ of mandamus, but taken together, they easily establish that the City has a “clear and indisputable” right to the writ. Accordingly, we grant the City’s petition for a writ of mandamus; we vacate the December 10, 2009 order of the District Court; 2 and we instruct the District Court to deny plaintiffs’ motion to compel the production of the Field Reports. BACKGROUND In February 2003, Mayor Michael Bloomberg announced that the 2004 Republican National Convention would be held in New York City. Immediately thereafter, the NYPD began developing plans to maintain order and safety during the event. The NYPD was mindful of the City’s status as a prime target of international terrorism, and the NYPD was keenly aware that a large political event could attract anarchist violence and unlawful civil disobedience. To address those concerns, David Cohen, the NYPD’s Deputy Commissioner of Intelligence (“Commissioner Cohen”), researched the security threat posed by violent protests at other large political events. Based on his analysis of the chaos and violence caused by recent protests in other metropolitan areas, including recent large-scale protests in Seattle and Miami, Commissioner Cohen concluded that even a small extremist element could trigger spi *930 raling violence at large political demonstrations. 3 App. to Pet. for Writ of Mandamus (hereinafter “App.”) 126-29 (Cohen Decl., June 6, 2007, Ex. A at 8-11). Accordingly, Commissioner Cohen and the NYPD’s Intelligence Division recognized that the NYPD needed a strategy to avoid disorder and violence during the RNC. Commissioner Cohen and the Intelligence Division began to form that strategy by researching potential extremist groups and their plans for disrupting the RNC. Much of the research consisted of combing through publicly available information on the Internet for plans to disrupt the RNC with violence or unlawful civil disobedience. The Intelligence Division compiled the results of that research into 600 pages of so-called “End User Reports.” Additionally, some members of the Intelligence Division went undercover and infiltrated various organizations to determine whether these organizations had devised plans to disrupt the RNC. The undercover officers prepared the Field Reports to memorialize what they had learned in various meetings and discussions with members of the infiltrated organizations. 4 With the help of Commissioner Cohen’s analysis and briefing regarding the threats outlined in the End User Reports, the NYPD’s RNC Executive Committee — consisting of Police Commissioner Raymond Kelly and other high-ranking NYPD officials — formed a policing strategy for the RNC known as the Mass Arrest Processing Plan (“the Plan”). App. 266-67 (Def. Rule 72(a) Objections, Sept. 19, 2008, at 7-8). Of greatest importance here, the Plan instructed NYPD officers not to issue summonses but to arrest all RNC-related disorderly protestors. 5 The Plan also required' that all arrestees be fingerprinted to verify the identity of those in custody. Id. There were 800,000 individuals who protested in New York City during the RNC. See Tr. of Oral Argument, Apr. 19, 2010, at 19. Plaintiffs were among the 1200 of those persons who were arrested and processed according to the Plan. Resp. at 6. Many plaintiffs were detained overnight, even though they were only charged with minor, noncriminal offenses, all of which were eventually dismissed. *931 In the fall of 2004, plaintiffs filed suit in the District Court under 42 U.S.C. § 1983 and state law alleging that the City and various NYPD officers (collectively “the City” or “defendants”) had violated plaintiffs’ rights under the United States Constitution and New York law. They claimed, among other things, that NYPD officers had arrested them without cause, subjected them to unreasonably long periods of detention, and fingerprinted them without authorization. In addition to the suits filed by plaintiffs, a number of other actions were brought in the District Court by RNC protestors. One such action asserts claims on behalf of a putative class of approximately 1200 people arrested at the RNC. Over forty similar actions were consolidated in the District Court for “discovery purposes.” Schiller, No. 04 Civ. 7922, Docket Entry No. 33 (S.D.N.Y. September 21, 2005). As a result, all discovery orders — such as the one at issue here — address the scope of discovery for literally dozens of cases and determine the documents that will be reviewed by over fifty attorneys. See Tr. of Oral Argument, Feb. 23, 2010, at 7. The discovery dispute that forms the basis of this petition began mere days before the close of discovery, when the City amended its mandatory disclosures under Federal Rule of Civil Procedure 26(a) and produced to plaintiffs the 600 pages of End User Reports. See Schiller v. City of N.Y., Nos. 04 Civ. 7922, 04 Civ. 7921, 2007 WL 735010, at * 1-2 (S.D.N.Y. Mar.12, 2007). Because the Reports had been turned over so late in the discovery process, Magistrate Judge Francis allowed plaintiffs to “reopen discovery in order to explore fully the newly disclosed information.” Id. at *5. Plaintiffs then deposed Commission Cohen and learned that the NYPD had in its possession the undercover Field Reports prepared in anticipation of the RNC. 6 Plaintiffs requested that the City produce the Field Reports, as well as “[a]ll documents” containing information about “individuals, groups, or entities connected to demonstrations expected to take place in New York City in conjunction with the Republican National Convention.” Schiller v. City of N.Y., Nos. 04 Civ. 7922, 04 Civ. 7921, 2007 WL 1461378, at *2 (S.D.N.Y. May 18, 2007) (quoting “Plaintiffs Sixth Request for Documents”). The City objected to the request on the grounds that it was (1) overbroad, (2) not likely to lead to the discovery of admissible evidence, and (3) seeking information protected by the law enforcement privilege. Id. Plaintiffs filed a motion to compel the City to produce the Field Reports. Id. at *1. In a series of rulings, Magistrate Judge Francis first determined that the Field Reports were broadly relevant to plaintiffs’ case. Id. at *2-3. Next, Magistrate Judge Francis determined that the Field Reports could, notwithstanding the law enforcement privilege, be disclosed to plaintiffs with certain information contained in the Reports redacted. See Schiller v. City of N.Y., 244 F.R.D. 273, 279, 281 (S.D.N.Y.2007). He then redacted certain portions of documents and ordered that they be disclosed on an “attorneys’ eyes only” basis. Id. at 281. He directed plaintiffs to store the documents at the office of the New York Civil Liberties Union. Id. at 279 n. 11. *932 The City filed objections to that order with Judge Sullivan. See App. 255; Fed.R.Civ.P. 72(a). 7 Relying on Commissioner Cohen’s declarations, the City argued that the intelligence documents were not relevant to plaintiffs’ case, that the law enforcement privilege protected the documents from disclosure, and that the Magistrate Judge’s procedures for protecting the documents — redacting them and disclosing them on an “attorneys’ eyes only” basis — were inadequate to protect the documents’ secrecy. See App. 274, 300. In December 2009, Judge Sullivan rejected the City’s arguments and affirmed the Magistrate Judge’s order in a careful and conscientious memorandum and order. MacNamara, 2009 WL 4789421. The City then filed this petition for a writ of mandamus and moved in this Court for a stay of the District Court’s order pending consideration of the petition. On January 27, 2010, Judge Wesley granted a temporary stay, and on March 12, 2010, a panel of our Court granted a stay pending consideration of the petition. 8 We now consider the City’s petition for a writ of mandamus. DISCUSSION The All Writs Act empowers this Court — as well as “all courts established by Act of Congress” — to “issue all writs necessary or appropriate in aid of their respective jurisdictions and agreeable to the usages and principles of law.” 28 U.S.C. § 1651(a). One such writ is the writ of mandamus, an “extraordinary remedy” that has been used “both at common law and in the federal courts ... to confine the court against which mandamus is sought to a lawful exercise of its prescribed jurisdiction.” Cheney, 542 U.S. at 380, 124 S.Ct. 2576 (brackets and internal quotation marks omitted). Although we do not, in this context, employ “an arbitrary and technical definition of ‘jurisdiction,’ ” we nevertheless issue a writ of mandamus only in “exceptional circumstances amounting to a judicial ‘usurpation of power’ or a ‘clear abuse of discretion.’ ” Id. (citations and some internal quotation marks omitted); see also Sims v. Blot, 534 F.3d 117, 132 (2d Cir.2008) (“A district court has abused its discretion if it [has] based its ruling on an erroneous view of the law or on a clearly erroneous assessment of the evidence or [has] rendered a decision that cannot be located within the range of permissible decisions.” (brackets, citations, and internal quotation marks omitted)). There are three conditions that must be established before a writ of mandamus may issue (we list these conditions in the order in which we discuss them below): (1) “the party seeking issuance of the writ must have no other adequate means to attain the relief [it] desires”; (2) “the issuing court, in the exercise of its discretion, must be satisfied that the writ is appropriate under the circumstances”; *933 and (3) the petitioner must demonstrate that the “right to issuance of the writ is clear and indisputable.” Cheney, 542 U.S. at 380-81, 124 S.Ct. 2576 (brackets, citations, and internal quotation marks omitted); accord Kerr v. U.S. Dist. Court for N. Dist. of Cal., 426 U.S. 394, 403, 96 S.Ct. 2119, 48 L.Ed.2d 725 (1976). Each of those conditions has been established in the situation presented here. I. The City Has “No Other Adequate Means to Obtain the Relief [It] Desires” The City seeks a writ of mandamus to overturn an order of the District Court requiring the City to disclose, in discovery proceedings, 1800 pages of undercover Field Reports to over fifty attorneys representing a putative class of some 1200 individuals who were arrested at the RNC. See Resp. at 6. We are satisfied that a writ of mandamus is the only “adequate” means for the City to challenge that order. Cheney, 542 U.S. at 380, 124 S.Ct. 2576. First, it is clear that the City cannot challenge the District Court’s order by means of an interlocutory appeal. As the Supreme Court has recently clarified, the “collateral order doctrine does not extend to disclosure orders adverse to” a claim of privilege. Mohawk Indus., Inc. v. Carpenter, — U.S. -, 130 S.Ct. 599, 609, — L.Ed.2d -(2009) (holding that a party may not bring an interlocutory appeal of an order denying a claim of attorney-client privilege). Instead, the Court has made clear that a party who loses a claim of privilege must pursue other “avenues of review apart from collateral order appeal,” including, “in extraordinary circumstances,” a “petition [to] the court of appeals for a writ of mandamus.” Id. at 607 (explaining that mandamus remains available in the event of “a particularly injurious ... privilege ruling”). Second, a certified appeal under 28 U.S.C. § 1292(b) is not an “adequate means” for the City to challenge the District Court’s order. 9 The certification of an interlocutory appeal under that statute, which is entirely a matter of discretion for the District Court, is appropriate only when “an immediate appeal from the order may materially advance the ultimate termination of the litigation.” 28 U.S.C. § 1292(b). Although the Supreme Court has recognized that the review of some discovery orders might involve “controlling question[s] of law, the prompt resolution of which may materially advance the ultimate termination of the litigation,” Mohawk Indus., 130 S.Ct. at 607 (internal quotation marks omitted), the discovery order at issue here would not have qualified for § 1292(b) certification because prompt appellate review of this order would not speed the District Court’s consideration of the merits of the parties’ claims or defenses — for no matter how the discovery dispute is resolved, the District Court must then proceed to resolve the merits of plaintiffs’ claims, whether by trial or by dispositive motion. Accordingly, a decision on the discovery dispute at issue here will not *934 “materially advance the ultimate termination of the litigation,” id., and thus § 1292(b) is not an “adequate means” for the City to obtain the relief it desires. Third, a contempt order is not an “adequate means” for the City to challenge the District Court’s ruling. Of course, were the City not a party, it could obtain relief by disobeying the District Court’s order and inviting civil contempt sanctions, which are immediately appealable when imposed on nonparties. See Lamb v. Cramer, 285 U.S. 217, 219-21, 52 S.Ct. 315, 76 L.Ed. 715 (1932); United States v. Johnson, 801 F.2d 597, 599 (2d Cir.1986) (“[A] non-party witness may immediately appeal an order of civil contempt.”). But because the City is a party, it can only appeal a civil contempt sanction after a final judgment. See Fox v. Capital Co., 299 U.S. 105, 107, 57 S.Ct. 57, 81 L.Ed. 67 (1936); Johnson, 801 F.2d at 599 (“[A] party to a pending proceeding may not appeal from an order of civil contempt except as part of an appeal from a final judgment.”). We acknowledge that a party to an action may immediately appeal an order of criminal contempt, and thus the City could theoretically (a) disobey the District Court’s order, (b) be held in criminal contempt, and then (c) bring an immediate appeal. See In re Christensen Eng’g Co., 194 U.S. 458, 459-61, 24 S.Ct. 729, 48 L.Ed. 1072 (1904); In re Air Crash at Belle Harbor, N.Y. on Nov. 12, 2001, 490 F.3d 99, 104 (2d Cir.2007). But because the choice of sanctions — civil or criminal— is vested in the discretion of the District Court, see In re Weiss, 703 F.2d 653, 664 (2d Cir.1983), the City does not know whether the District Court would punish its disobedience with an appealable criminal sanction or an “onerously coercive civil contempt sanction with no means of review until the perhaps far distant day of final judgment,” 15B Charles Alan Wright, Arthur R. Miller & Edward C. Cooper, Federal Practice and Procedure § 3914.23, at 146 (2d ed.1992). We therefore conclude, along with our sister Circuits, that the “uncertainty” of seeking a criminal contempt order “bespeaks its inadequacy in this case.” In re Sealed Case No. 98-8077, 151 F.3d 1059, 1065 (D.C.Cir.1998); see also Sporck v. Peil, 759 F.2d 312, 315 & n. 4 (3d Cir.1985) (holding, in the context of a mandamus petition, that the disobedience- and-contempt path is not an “adequate means to obtain the relief sought”). Finally, an appeal after a final judgment is not, in the situation presented, an “adequate means” for the City to obtain the relief it seeks. If the City hands over the Field Reports now, the sensitive information contained in the Reports will, upon final judgment, “already have been exposed.” In re von Bulow, 828 F.2d 94, 99 (2d Cir.1987). As Judge Kearse has succinctly explained, “a remedy after final judgment cannot unsay the confidential information that has been revealed.” Sims, 534 F.3d at 129 (internal quotation marks and punctuation omitted). 10 The inadequacies of an appeal after a final judgment are especially apparent here, where the information at issue involves a police department’s undercover investigations. 11 See In *935 re Dep’t of Investigation of the City of N.Y., 856 F.2d 481, 484 (2d Cir.1988) (noting that one purpose of the law enforcement privilege is to protect witnesses and law enforcement personnel). We also reject the idea that an appeal after a final judgment is an adequate remedy for the City, an idea that is based upon the assumption that the District Court’s protective order will successfully prevent the individual plaintiffs, and the general public, from gaining access to the Field Reports as this action proceeds to judgment. Indeed, it has frequently been asserted — both here and in similar cases— that confidential information can be protected by means of two measures: disclosure of information on an “attorneys’ eyes only” basis and the filing of information “under seal.” See, e.g., MacNamara, 2009 WL 4789421, at *5. For the reasons that follow, we hold that neither of those measures is adequate to protect the secrecy of the Field Reports in this context. A. Disclosure on an “Attorneys’ Eyes Only” Basis Is Inadequate The disclosure of confidential information on an “attorneys’ eyes only” basis is a routine feature of civil litigation involving trade secrets. See Fed.R.Civ.P. 26(c)(1)(G) (“The court may, for good cause, issue an order to protect a party or person requiring that a trade secret or other confidential research, development, or commercial information not be revealed or be revealed only in a specified way .... ”). 12 The purpose of this form of lim *936 ited disclosure is to prevent a party from viewing the sensitive information while nevertheless allowing the party’s lawyers to litigate on the basis of that information. Even if the “attorneys’ eyes only” procedure works in some commercial litigation, as well as some criminal cases, see note 12, ante, the consequences of accidental disclosure are too severe to employ the procedure here. If a party in a commercial suit obtains a competitor’s trade secrets, at worst the party will gain an unfair financial advantage over his competitor. Though such an injury is serious, it involves money, not public safety, and it can usually be remedied by an injunction or money damages. Here, however, accidental disclosure of the Field Reports risks undermining important NYPD investigatory procedures and thereby endangering the safety of law enforcement personnel and countless New York residents. Not only is that injury more severe, it is far more difficult to remedy. Furthermore, when a party in a commercial suit gains access to trade secrets and uses them to his advantage, it is, in many cases, obvious that an unauthorized disclosure has occurred. That is not the case, however, for confidential law enforcement information, for there is often only ambiguous evidence that an undercover operation had been undermined — an unexpected event impairs an investigation, an informant is suddenly isolated from his collaborators, or a once-valuable source begins to provide disinformation. Thus, if confidential law enforcement information is disclosed on an “attorneys’ eyes only” basis, the police may never know if their undercover operations have been compromised by an unauthorized disclosure of that information. What is more, when a party in a commercial suit profits from an unauthorized disclosure of trade secrets, his competitor often has little trouble identifying the source of the disclosure. Yet it is, by contrast, far more difficult to identify the source of an unauthorized disclosure of law enforcement information. In this very litigation, for example, the City suspects that materials disclosed on an “attorneys’ eyes only” basis (pursuant to the District Court’s protective order) were used as the source of a newspaper article discussing the secret operations of the NYPD. Jim Dwyer, City Police Spied Broadly Before G.O.P. Convention, N.Y. Times, Mar. 25, 2007, at Al. When the City sought an order directing each of the more than fifty attorneys for the plaintiffs to attest to their knowledge of any such disclosure, Magistrate Judge Francis instead directed that the City first establish its own blamelessness by “provid[ing] declarations from everyone [on the defendants’ side] who had access to the Intelligence Documents.” Schiller v. City of N.Y., No. 04 Civ. 7921, 2007 WL 1623108, at *4 (S.D.N.Y. June 5, 2007). 13 Even if the City had been allowed to question the plaintiffs’ lawyers, the source might still have remained elusive, and in any event, pursuing the source of the disclosure would have arguably raised significant issues regarding the rights of the press. See generally N.Y. Times Co. v. Gonzales, 459 F.3d 160, 162 (2d Cir.2006). For these reasons, we hold that, in the circumstances of this case, allowing the disclosure of the Field Reports on an “attorneys’ eyes only” basis would provide insufficient protection for the confidential *937 law enforcement information at issue. 14 Accordingly, the “attorneys’ eyes only” procedure does not adequately mitigate the harm that could be caused by requiring the City to wait for a final judgment before appealing the District Court’s order. B. Filing Under Seal Is Likewise Inadequate We also doubt that filing the Field Reports under seal would adequately address the City’s legitimate concerns. Courts are public institutions accustomed to making their files open to all comers, and their methods of preserving confidentiality are relatively unsophisticated and altogether too fallible. The use of pseudonyms such as “John Doe,” for example, can sometimes be more an invitation to publicity than a shield of privacy. In addition, “sealed” materials submitted for review only by a judicial officer usually pass through many hands, and the chances of disclosure — and difficulties in locating the source of the disclosure — increase exponentially with each pair of hands involved. To make our point, we recite a few recent and illuminating episodes, starting with incidents outside our Circuit. Just weeks ago, redacted portions of a motion filed by Rod Blagojevich, the former Governor of Illinois, seeking a subpoena to compel President Obama to testify at Blagojevich’s upcoming criminal trial accidentally became publically accessible for several hours on the website of the United States District Court for the Northern District of Illinois. John Kass, Blagojevich Drags Obama into His Three-Ring Circus, Chi. Trib., Apr. 23, 2010, at 2 (“While this is believed to have been a technical glitch ... this is Chicago. Do you believe in coincidence?”). Sadly, that disclosure was not an aberration. Consider just a few examples: In a Pennsylvania state court proceeding, materials that had been filed under seal as part of a high-profile divorce proceeding were inadvertently posted for several days on a Pennsylvania county court’s publicly accessible website. See Steve Twedt, Scaife Demands Documents’ Return, Pittsburgh Post-Gazette, Sept. 23, 2007, at Al; see also Gary Rotstein, Unclear How Often Sealed Cases Opened by Accident, Pittsburgh Post-Gazette, Sept. 23, 2007, at A1. A clerk in the United States District Court for the Eastern District of North Carolina inadvertently turned over a “sealed” settlement agreement along with unsealed materials to a reporter who asked to see the file in a pollution lawsuit. See B. Drummond Ayres Jr., Press Freedom at Issue in Use of ‘Sealed’ File by Paper and Reporter, N.Y. Times, February 21, 1998, at A6. And documents containing the name of the woman who accused the basketball star Kobe Bryant of sexual assault, though filed “under seal,” were twice posted on the publicly accessible website of the Colorado State Judicial Branch, and the full transcript of a “sealed” hearing in the same case was inadvertently sent to seven news organizations by email. Kirk Johnson, Name of Bryant Case Accuser Is Again Mistakenly Released, N.Y. Times, July 29; 2004, at A16. Often such disclosures are benign; sometimes, however, they risk the gravest of consequences. For example, a recent post-9/11 detention case, the existence of which was to be sealed altogether, was revealed when it was listed inadvertently on the public docket of the United States *938 Court of Appeals for the Eleventh Circuit. Linda Greenhouse, News Groups Seek to Open Secret Case, N.Y. Times, Jan. 5, 2004, at A12. And a “sealed” affidavit in a visa fraud case describing evidence related to the 9/11 attacks was mistakenly unsealed, a mistake that was promptly reported in a major newspaper. See David Johnston, Officials Say Search Finds Hijacking-Related Material, N.Y. Times, May 10, 2002, at A38. Several similar instances have occurred in our own Circuit, which often witnesses high-profile litigation covered by a large and intrepid press corps. For example, the name of a plaintiff in a “sealed” case challenging the FBPs issuance of a “national security letter” was disclosed when a website operated by the United States District Court for the District of Connecticut inadvertently made the name available to the public. Alison Leigh Cowan, Judges Question Patriot Act in Library and Internet Case, N.Y. Times, Nov. 3, 2005, at B5. See generally Doe v. Gonzales, 386 F.Supp.2d 66 (D.Conn.2005). Similarly, the subjects of a grand jury investigation were revealed when a “John Doe” case in the United States District Court for the Eastern District of New York was listed under the real name on the argument calendar. Andrew Blum, Defense Work in Pan Am Case Probed[:] Possible Perjury, Fraud and Litigation Abuse Examined, Nat’l L.J., Sept. 18, 1995, at A6. See generally In re Richard Roe, Inc., 68 F.3d 38 (2d Cir.1995). And in the first World Trade Center bombing case in the United States District Court for the Southern District of New York, a “sealed” motion to identify a confidential informant was inadvertently disclosed. Alison Mitchell, Files Say U.S. Had Informer in Bomb Case, N.Y. Times, May 11, 1993, at B1. Indeed, our own Court is not unblemished in this regard. As reported in the Washington Post: [N]ew details about the FBI’s allegedly aggressive [interrogation] tactics ... were included in a ruling briefly issued ... by the U.S. Court of Appeals for the 2nd Circuit, which reinstated a civil lawsuit brought ... against the FBI and [an FBI agent]. In an unusual move, however, the appeals court withdrew the first opinion within minutes on Thursday and issued a second opinion Friday, with the details of [plaintiff] Higazy’s allegations removed. Dan Eggen, Second Court Ruling Redacts Information About Interrogation, Wash. Post, Oct. 25, 2007, at A10; Higazy v. Templeton, 505 F.3d 161, 166 (2d Cir.2007) (“This opinion has been redacted because portions of the record are under seal”). Even though the opinion in Higazy was withdrawn “within minutes,” the disclosure could not be undone. See, e.g., Jim Dwyer, Roots of False Confession: Spotlight Is Now on the F.B.I., N.Y. Times, Oct. 31, 2007, at Bl. 15 In light of how often there are all-too-human lapses with material filed “under seal,” we cannot conclude with confidence that filing the Field Reports under seal would protect the information. Even if the Field Reports were not “filed” on the District Court’s docket at all, were they disclosed to plaintiffs’ counsel they would inevitably be referenced in the parties’ affidavits, memoranda of law, Rule 56.1 statements, and the like — not to mention *939 future District Court decisions — and we cannot conclude with confidence that filing those documents under seal would protect the confidential information contained in the Field Reports. In sum, because the City cannot bring an immediate appeal under the collateral order doctrine or by inviting an order of civil contempt, there are only three avenues by which the City can challenge the disclosure of the Field Reports: (1) a certified interlocutory appeal under 28 U.S.C. § 1292(b), (2) an immediate appeal of an order of criminal contempt, and (3) an appeal after a final judgment. For the reasons set forth above, we hold that neither a certified interlocutory appeal under 28 U.S.C. § 1292(b) nor an order of criminal contempt provides an “adequate means” for the City to challenge the District Court’s discovery order. In addition, we hold that neither disclosure on an “attorneys’ eyes only” basis nor the filing of documents “under seal” is suitable to protect the confidential law enforcement information at issue here, and thus an appeal after a final judgment is not an “adequate means” for the City to vindicate its claim to law enforcement privilege. We conclude, as a result, that the first condition for a writ of mandamus is established: “the party seeking issuance of the writ [has] no other adequate means to attain the relief [it] desires.” Cheney, 542 U.S. at 380, 124 S.Ct. 2576. II. A Writ of Mandamus Is “Appropriate Under the Circumstances” We now turn to whether mandamus is “appropriate under the circumstances.” Cheney, 542 U.S. at 381, 124 S.Ct. 2576. Although we have “expressed reluctance” to issue writs of mandamus to overturn discovery rulings, see Glotzer, 374 F.3d at 187 (internal quotation marks omitted), we have recognized that when a discovery question “is of extraordinary significance or there is extreme need for reversal of the district court’s mandate before the case goes to judgment,” mandamus can serve as a legitimate “escape hateh[] from the finality rule,” Am. Express Warehousing, Ltd. v. Transamerica Ins. Co., 380 F.2d 277, 282 (2d Cir.1967). The Supreme Court has also recognized, as recently as 2009, the ability of mandamus to “serve as [a] useful safety valve[ ] for promptly correcting serious errors” or “a particularly injurious ... privilege ruling.” Mohawk Indus., 130 S.Ct. at 607-08 (internal quotation marks omitted). To determine whether mandamus is “appropriate” in the context of a discovery ruling, we look primarily for “ ‘the presence of a novel and significant question of law ... and ... the presence of a legal issue whose resolution will aid in the administration of justice.’ ” 16 Glotzer, 374 F.3d at 187 (quoting United States v. Coppa, 2Q1 F.3d 132, 137-38 (2d Cir.2001)); see also Sims, 534 F.3d at 128-29 (“Although mandamus is generally unavailable as a means of reviewing district court discovery orders, the writ is appropriate to *940 review discovery orders that involve privilege where (i) the petition raises an issue of importance and of first impression; (ii) the petitioner’s privilege will be lost if review must await final judgment; and (iii) immediate resolution will avoid the development of discovery practices or doctrine undermining the privilege.”) (quoting In re Long Island Lighting Co., 129 F.3d 268, 270 (2d Cir.l997)(“L/LCO ”)). Both of those factors are present in this petition, as it raises several “novel and significant question[s] of law,” as well as “legal issue[s] whose resolution will aid in the administration of justice.” Glotzer, 374 F.3d at 187. A. This Petition Raises “Novel and Significant Question[s] of Law” We are mindful that “an allegedly incorrect application of a well-developed principle” does not, by itself, give rise to “ ‘such a novel and important issue as to warrant mandamus review.’ ” LILCO, 129 F.3d at 271 (quoting In re W.R. Grace & Co., 984 F.2d 587, 589 (2d Cir.1993)). But because this petition requires us to clarify so many aspects of our standard for addressing claims of law enforcement privilege, we conclude that this petition raises sufficiently “novel and significant question[s] of law” to justify a writ of mandamus. Glotzer, 374 F.3d at 187. 17 We have rarely considered the scope of the law enforcement privilege; our last case to do so was decided over twenty years ago. See Dep’t of Investigation, 856 F.2d at 481. There is, as a result, a critical aspect of our jurisprudence that remains undeveloped: although we have provided some guidance about the kind of information to which the law enforcement privilege applies, see, e.g., id. at 483-86, we have never had an occasion to address how a court should proceed once it establishes that the information at issue is subject to the privilege. That is, because the law enforcement privilege is a qualified privilege, not an absolute privilege, there are circumstances in which information subject to the privilege must nevertheless be disclosed. We have never addressed how a court should determine whether such a circumstance exists, and it is that critical aspect of our jurisprudence that we are required to address in this petition. In particular, as set forth in greater detail below, this petition requires us to clarify (1) the legal standard for evaluating whether the law enforcement privilege should give way to a party’s need for discovery, (2) the factors to be weighed in applying that standard, (3) whether there is a presumption against disclosure, and (4) if there is such a presumption, the extent of the showing that a party must make to overcome it. Each of those issues presents a “novel ... question of law.” Glotzer, 374 F.3d at 187. Each of those issues also presents a “significant question of law.” Id. (emphasis added). Although it is rarely raised on appeal, the law enforcement privilege plays a critical role in litigation involving the government. The purpose of the privi *941 lege, as we have observed in the past, is “to prevent disclosure of law enforcement techniques and procedures, to preserve the confidentiality of sources, to protect witness and law enforcement personnel, to safeguard the privacy of individuals involved in an investigation, and otherwise to prevent interference with an investigátion.” Dep’t of Investigation, 856 F.2d at 484; accord United States v. Amodeo, 44 F.3d 141, 147 (2d Cir.1995). Those goals are of such importance that the privilege, though it developed at common law from executive privilege, 18 has been largely incorporated into both New York state 19 and federal 20 statutory law. Dep’t of In *942 vestigation, 856 F.2d at 483 & nn. 1-2. It is hard to imagine, therefore, many “question[s] of law” that carry greater “significan[ce]” than the question of when the goals of the law enforcement privilege must give way to a party’s need for discovery. Glotzer, 374 F.3d at 187. Accordingly, we hold that this petition presents sufficiently “novel and significant question[s] of law” to justify a writ of mandamus. Id. B. Resolving the Issues Raised in this Petition “Will Aid in the Administration of Justice” In determining whether a petition presents a “legal issue whose resolution will aid in the administration of justice,” id., we think it is appropriate to consider “the administration of justice” both in terms of a district court applying the law and in terms of a potential litigant organizing his affairs ,in the shadow of the law. As for the district courts, we have previously recognized, that the resolution of a novel and significant privilege question in a mandamus proceeding will “aid the administration of justice.” von Bulow, 828 F.2d at 99. Addressing the merits of this petition, for example, may “forestall future error in trial courts” by correcting a privilege determination with a potentially broad applicability and influence. Id. (internal quotations omitted); see also LILCO, 129 F.3d at 271; Chase Manhattan Bank, N.A. v. Turner & Newall, PLC, 964 F.2d 159, 163-64 (2d Cir.1992). Indeed, by clarifying the circumstances under which the law enforcement privilege must (and must not) yield to a party’s need for discovery, we will provide guidance for the courts of our Circuit in an important, yet underdeveloped, area of law. As for potential litigants, we have recognized that, for a privilege to serve its intended function, potential litigants must be able to predict which of their materials will be protected by the privilege. See von Bulow, 828 F.2d at 100; cf. George A. Davidson & William H. Voth, Waiver of the Attorney-Client Privilege, 64 Or.L.Rev. 637, 639 (1986) (describing the “need for certainty and predictability inherent in the purpose of the [attorney-client] privilege”); id. at 666 (“Decisions on questions of privilege may have a far more profound effect on the administration of justice than the decisions on questions of 'substantive law to which judges give most of their attention.”). Otherwise, potential litigants may become overly cautious in creating materials so as to not risk disclosing sensitive information in future litigation. If we were to decline to grant this petition, we would risk discouraging law enforcement agencies from conducting undercover investigations (or from keeping records of those investigations). See Coppa, 267 F.3d at 139 (concluding that mandamus was only adequate relief where a pretrial ruling requiring “early disclosure of the identities of potential witnesses could undermine undercover operations and ongoing investigation involving these witnesses”); cf. In re Dep’t of Investigation of City of N.Y., 851 F.2d 65, 68-69 (2d Cir.1988) (finding an issue important enough to warrant mandamus because “[f]ederal law enforcement officials should have ample guidance as to whether defendants will have access to the files of cooperating state and local officials before entering into joint investigations,” but denying petition because *943 nonparty petitioner could appeal immediately if held in contempt). Moreover, if we require the disclosure of the Field Reports, other police officers may be less willing to become undercover agents if they fear that their identities may be disclosed in court proceedings. Accordingly, we hold that this petition presents “legal issue[s] whose resolution will aid in the administration of justice,” Glotzer, 374 F.3d at 187, not only in terms of district courts applying the law, but also in terms of police departments predicting which of their documents will be subject to discovery in future litigation. In sum, this petition requires us to clarify several critical aspects of our jurisprudence on the law enforcement privilege, for deciding the petition requires us to describe the circumstances under which the law enforcement privilege must give way to a party’s need for discovery. This petition, therefore, raises several “novel and significant questions] of law ... whose resolution will aid in the administration of justice.” Id. We therefore hold that mandamus is “appropriate under the circumstances.” Cheney, 542 U.S. at 381, 124 S.Ct. 2576. III. The City Has a “Clear and Indisputable” Right to the Writ Finally, we address whether the City has shown that its right to the writ is “clear and indisputable.” Id. (internal quotation marks omitted). Because a writ of mandamus is a “drastic and extraordinary remedy reserved for really extraordinary causes,” we issue the writ only in “exceptional circumstances amounting to a judicial usurpation of power or a clear abuse of discretion.” Id. at 380, 124 S.Ct. 2576 (internal quotation marks omitted) (emphasis supplied); see also Mohawk Indus., 130 S.Ct. at 607 (noting that mandamus may also be appropriate “when a disclosure order ... works a manifest injustice” (emphasis added)). A district court “abuses” 21 its discretion if it (1) bases its ruling on an “erroneous view of the law,” (2) makes a “clearly erroneous assessment of the evidence,” or (3) renders “a decision that cannot be located within the range of permissible decisions.” Sims, 534 F.3d at 132 (internal quotation marks omitted). Thus, we issue a writ of mandamus if a district court “clear[ly] and indisputably]” abuses its discretion in one of those three ways. For the- reasons set forth below, we conclude that the District Court “indisputably]” erred in three respects. Although the District Court correctly determined that the law enforcement privilege applied to the Field Reports, the District Court “indisputably]” adopted “erroneous view[s] of the law,” Cheney, 542 U.S. at 381, 124 S.Ct. 2576; Sims, 534 F.3d at 132, when it set forth the legal standard to determine whether the privilege was over *944 come by plaintiffs need for discovery: First, the District Court failed to apply a “strong presumption against lifting the privilege.” Dellwood Farms, 128 F.3d at 1125. Second, the District Court failed to require that plaintiffs show a compelling need for the Field Reports. Third, in addition to those legal errors, the District Court “indisputably]” made a “clearly erroneous assessment of the evidence,” Cheney, 542 U.S. at 381, 124 S.Ct. 2576; Sims, 534 F.3d at 132, when it held that plaintiffs’ need for the Field Reports outweighed the public’s interest in maintaining their secrecy. Accordingly, we hold that the City has shown that it has a “clear and indisputable” right to a writ of mandamus. Id. at 381, 124 S.Ct. 2576. A. The Law Enforcement Privilege Applies to the Field Reports When assessing a claim of law enforcement privilege, a district court must first determine if the law enforcement privilege applies to the documents at issue. We adopt the holding of the District of Columbia Circuit that the party asserting the law enforcement privilege bears the burden of showing that the privilege applies to the documents in question, In re Sealed Case, 856 F.2d 268, 271-72 (D.C.Cir.1988). To meet this burden, the party asserting the law enforcement privilege must show that the documents contain information that the law enforcement privilege is intended to protect. Such protected information includes information pertaining to “law enforcement techniques and procedures,” information that would undermine “the confidentiality of sources,” information that would endanger “witness and law enforcement personnel [or] the privacy of individuals involved in an investigation,” and information that would “otherwise ... interfere[ ] with an investigation.” Dep’t of Investigation, 856 F.2d at 484. “An investigation,” however, “need not be ongoing for the law enforcement privilege to apply as ‘the ability of a law enforcement agency to conduct future investigations may be seriously impaired if certain information’ ” is revealed to the public. Nat’l Congress for P.R. Rights ex rel. Perez v. City of N.Y., 194 F.R.D. 88, 95 (S.D.N.Y.2000) (quoting Morrissey v. City of N.Y., 171 F.R.D. 85, 90 (S.D.N.Y.1997)); see also Halpern v. FBI, 181 F.3d 279, 294 (2d Cir.1999). The law enforcement privilege clearly applies to the documents at issue here. The Field Reports, even in their redacted form, contain detailed information about the undercover operations of the NYPD. This information clearly relates to “law enforcement techniques and procedures.” Dep’t of Investigation, 856 F.2d at 484. Moreover, providing information about the nature of the NYPD’s undercover operations will only hinder the NYPD’s ability to conduct future undercover investigations. Additionally, even the redacted documents contain some information that could disclose the identity of an NYPD undercover officer. Pulling any individual “thread” of an undercover operation may unravel the entire “fabric” that could lead to identifying an undercover officer. This could present a risk to the safety and effectiveness of that officer and would likely provide additional information about how the NYPD infiltrates organizations, thereby impeding future investigations. Because we conclude (1) that the City has shown that the Field Reports contain information regarding “law enforcement techniques and procedures,” id., and the identity of undercover officers, and (2) that disclosure of the Field Reports could undermine the safety of law enforcement *945 personnel and would likely undermine “the ability of a law enforcement agency to conduct future investigations,” Morrissey, 171 F.R.D. at 90, we hold that the City has met its burden to show that the law enforcement privilege applies to the Field Reports. B. Plaintiffs’ Need for the Field Reports Does Not Overcome the Strong Presumption Against Disclosure We have concluded that the law enforcement privilege applies to the Field Reports. But the law enforcement privilege is qualified, not absolute, In re Sealed Case, 856 F.2d at 272, so determining the applicability of the privilege does not complete our inquiry. Instead, as the District of Columbia Circuit has held, “[t]he public interest in nondisclosure must be balanced against the need of a particular litigant for access to the privileged information.” Id. We agree, and we adopt this sensible balancing test as the law of our Circuit. Once a court has determined that the law enforcement privilege applies, we agree with the Seventh Circuit that “there ought to be a pretty strong presumption against lifting the privilege.” Delkoood Farms, 128 F.3d at 1125. 22 To rebut that presumption, the party seeking disclosure must show (1) that its suit is “non-frivolous and brought in good faith,” (2) that “the information sought is [not] available through other discovery or from other sources,” and (3) that the information sought is “important]” to the party’s case. See Friedman v. Bache Halsey Stuart Shields, Inc., 738 F.2d 1336, 1343 (D.C.Cir.1984) (internal quotation marks omitted). With respect to the importance of the information sought, we hold that a “compelling need” is required. Cf. Marriott Int’l Resorts, 437 F.3d at 1307 (emphasis added) (discussing the deliberative process privilege). Parties seeking disclosure of information protected by the law enforcement privilege must make those three showings to overcome the strong presumption against disclosure. 23 Of course overcoming the presumption does not end the inquiry. Rather, once the presumption has been overcome, a court must still balance “[t]he public interest in nondisclosure ... against the need of a particular litigant for access to the privileged information.” In re Sealed Case, 856 F.2d at 272. In other words, demonstrating a “compelling need” does not automatically entitle a litigant to privileged information. Rather, disclosure is required only if that compelling need outweighs the public interest in nondisclosure. *946 In this case, the parties do not dispute that plaintiffs’ suit is “non-frivolous and brought in good faith” and that “the information sought” — among other things, the names of the organizations that the NYPD infiltrated — “is [not] available through other discovery or from other sources.” Friedman, 738 F.2d at 1343. The parties only dispute whether plaintiffs have shown a “compelling need” for the documents. Marriott Int’l Resorts, 437 F.3d at 1307. The City asserts that plaintiffs have, at most, an attenuated need for the Field Reports; plaintiffs assert that they have a substantial need for the Field Reports. Determining plaintiffs’ need for the Field Reports requires us to review the general contours of the parties’ putative arguments at trial. Plaintiffs intend to challenge the City’s mass arrest policy during the RNC. In its defense, the City intends to assert that the threats facing the RNC and the public justified the policy. Plaintiffs, in turn, intend to argue that the threats were insufficient to justify the mass arrest policy. At issue, then, is the seriousness of the threat facing the RNC and the City in the summer of 2004. Whether the information in the Field Reports contradicts the information in the End User Reports determines whether plaintiffs have a compelling need for the Field Reports. As the District Court succinctly noted, “to the extent that the documents reveal a threat that would have legally justified the City’s challenged policies, they will support the City’s defense; to the extent they do not, they will undercut that defense.” MacNamara, 2009 WL 4789421, at *4. The City has previously released to plaintiffs 600 pages of End User Reports revealing the threat level to the City, so the only possible use of the confidential Field Reports to plaintiffs is to undermine or otherwise cast doubt upon the assessment of the threat revealed in the 600 pages already released. If the Field Reports do not contradict the threats revealed in the End User Reports, then plaintiffs have no need for the documents. The obvious corollary, then, is that plaintiffs would need the Field Reports if they in any significant way contradicted or cast doubt upon the End User Reports. We have reviewed the Field Reports and the End User Reports, 24 and we conclude that the information in the Field Reports in no way undermines, contradicts, or casts doubt upon the information in the End User Reports. In fact, much of the information in the Field Reports reinforces the City’s assertions that the public faced a substantial threat of disruption and violence during the RNC. Because the Field Reports do not undermine the information previously disclosed to plaintiffs, we have no difficulty in concluding that plaintiffs do not have a need, much less a compelling need, for the Field Reports. Plaintiffs argue that, regardless of their showing of need, the law enforcement privilege should not protect these documents from disclosure because the City cannot “put[ ] forth the surveillance operation as its defense ... [and] at the same time invoke the law-enforcement privilege to block the plaintiffs from discovering information about the operation that bears on the validity of that defense.” Resp. 33. We agree that, as a general matter, a *947 party cannot use materials as a “sword” in its defense “while using privileges attaching to [materials relied upon for that defense] as a ‘shield.’ ” John Doe Co. v. United States, 350 F.3d 299, 302 (2d Cir.2003). But because that general proposition is “rooted in fairness,” “we have also cautioned against broad generalizations, stressing that whether fairness requires disclosure ... is best decided on a case by case basis, and depends primarily on the specific context in which the privilege is asserted.” Id. (alterations and internal quotation marks omitted). Here, the City does not intend to use the Field Reports as a “sword,” because it has expressly disavowed reliance on the information contained in the Field Reports to support its claim that there was a substantial risk of mayhem during the RNC. Pet. 27. Moreover, even if we were to agree with plaintiffs’ claim that the City is using the Field Reports as a “sword” — that is, selectively disclosing Reports helpful to its defense— we would conclude, in “the specific context in which the privilege is asserted,” John Doe Co., 350 F.3d at 302 (internal quotation marks omitted), that it is not unfair for the law enforcement privilege to protect the Field Reports because the information in the Field Reports does not contradict or undermine the information in the End User Reports. Accordingly, we hold that plaintiffs’ need for the Field Reports is not “compelling,” Marriott Int’l Resorts, 437 F.3d at 1307, and, in any event, certainly does not outweigh the public’s substantial interest in nondisclosure as a means to preserve the integrity of the NYPD’s undercover operations. We therefore hold that plaintiffs have not overcome the strong presumption against disclosure. C. The District Court Erred “Indisputably]” Because we have now resolved several novel legal questions that were unsettled when the District Court considered this case, we pause briefly to reiterate exactly which of the District Court’s errors justify a writ of mandamus in this context. First, although the District Court correctly determined that the law enforcement privilege applied to the Field Reports, the District Court “indisputably]” adopted an “erroneous view of the law,” Cheney, 542 U.S. at 381, 124 S.Ct. 2576; Sims, 534 F.3d at 132, when it failed to apply a “strong presumption against lifting the privilege,” Dellwood Farms, 128 F.3d at 1125. Second, the District Court “indisputably]” adopted an “erroneous view of the law,” Cheney, 542 U.S. at 381, 124 S.Ct. 2576; Sims, 534 F.3d at 132, when it failed to require that plaintiffs show a “compelling need” for the Field Reports, Marriott Int’l Resorts, 437 F.3d at 1307. Third, and finally, the District Court “indisputabl[y]” made a “clearly erroneous assessment of the evidence,” Cheney, 542 U.S. at 381, 124 S.Ct. 2576; Sims, 534 F.3d at 132, when it held that plaintiffs’ need for the Field Reports outweighed the strong presumption against disclosure. Moreover, we conclude that the District Court’s order — directing a department of the executive branch of the City of New York to disclose information that could risk the safety of its employees and the public — was a clear abuse of discretion. See Cheney, 542 U.S. at 380, 124 S.Ct. 2576. We agree with the Seventh Circuit that “control of criminal investigations is the prerogative of the executive branch, subject to judicial intervention only to protect rights.” Dellwood Farms, 128 F.3d at 1126. Although evaluating a claim of law enforcement privilege is unquestionably a proper and important judicial function, we think that an intrusion into the executive *948 branch’s historic control over criminal investigations that unreasonably jeopardizes public safety amounts to a clear abuse of discretion, if not “a judicial usurpation of power.” See Cheney, 542 U.S. at 380, 124 S.Ct. 2576. Each of the District Court’s errors may, on its own, warrant a writ of mandamus, but taken together, they readily establish that the City has shown an “clear and indisputable” right to the writ. See id. at 381, 124 S.Ct. 2576. IY. How to Analyze Claims of Law Enforcement Privilege In our Circuit, a writ of mandamus is appropriate when the writ will “aid in the administration of justice.” Glotzer, 374 F.3d at 187. Thus, in an effort to best aid the administration of justice in our Circuit, we briefly summarize our holdings in this opinion and explain how district courts should analyze claims of law enforcement privilege. First, the party asserting the law enforcement privilege bears the burden of showing that the privilege indeed applies to the documents at issue. In re Sealed Case, 856 F.2d at 272. To show that the privilege applies, the party asserting the privilege must demonstrate that the documents contain information that the law enforcement privilege is intended to protect. Specifically, the party asserting the privilege must show that the documents in question contain (1) information pertaining to “law enforcement techniques and procedures,” Dep’t of Investigation, 856 F.2d at 484, (2) information that would undermine “the confidentiality of sources,” id., (3) information that would endanger “witness and law enforcement personnel,” id., (4) information that would undermine “the privacy of individuals involved in an investigation,” id., or (5) information that would seriously impair “the ability of a law enforcement agency to conduct future investigations,” Morrissey, 171 F.R.D. at 90. Once the party asserting the privilege successfully shows that the privilege applies, the district court must balance the public interest in nondisclosure against “the need of a particular litigant for access to the privileged information.” In re Sealed Case, 856 F.2d at 272. There is a “strong presumption against lifting the privilege.” Dellwood Farms, 128 F.3d at 1125. To rebut that strong presumption, the party seeking disclosure bears the burden of showing (1) that the suit is “non-frivolous and brought in good faith,” Friedman, 738 F.2d at 1343, (2) that “the information sought is [not] available through other discovery or from other sources,” id., and (3) that the party has a “compelling need for the privileged information,” Marriott Int’l Resorts, L.P., 437 F.3d at 1307. If the presumption against disclosure is successfully rebutted (by a showing of, among other things, a “compelling need”), the district court must then weigh the public interest in nondisclosure against the need of the litigant for access to the privileged information before ultimately deciding whether disclosure is required. To assess both the applicability of the privilege and the need for the documents, the district court must ordinarily review the documents in question. As described in Part I.C, filing documents under seal may inadequately protect particularly sensitive documents. Thus, rather than require that the parties file the potentially privileged documents with the court, the district court may, in the exercise of its informed discretion and on the basis of the circumstances presented, require that the party possessing the documents appear ex parte in chambers to submit the documents for in camera review by the judge, after which the materials can be returned *949 to the custody of that party. See, e.g., In re Grand Jury Subpoenas Dated March 19, 2002 and August 2, 2002, 318 F.3d 379, 386 (2d Cir.2003) (describing the presentation of documents for in camera review as a “practice both long-standing and routine in cases involving claims of privilege” and citing illustrative cases); United States v. Wolfson, 55 F.3d 58, 60-61 (2d Cir.1995) (noting, in the criminal context, that “the prescribed procedure for resolving [a] dispute [as to whether certain confidential documents are subject to discovery] is to provide the documents to the district court for in camera review” and that “[t]he district court normally returns such documents to the party that submitted them in camera ”). Some documents may be so sensitive that they should not be left in a judge’s chambers overnight. In those circumstances, the court may, in its discretion, direct the party who submitted the documents to retrieve them each evening and to return them to the judge if and when necessary. By following this commonsensical procedure for especially sensitive documents, the party with the strongest incentives — and presumably the best policies and tools — to maintain the confidentiality of the documents can retain control over them. This arrangement minimizes the likelihood of inadvertent disclosures of sensitive information. 25 If the district court determines that the law enforcement privilege does not protect the documents at issue, the documents must be disclosed. In an effort to minimize the effects of disclosure, however, the district court may order that the documents be “revealed only in a specified way.” Fed.R.Civ.P. 26(c)(1)(G). Although the court is free to tailor the protective order to the circumstances presented, the court may wish to consider making the documents available only on an “attorneys’ eyes only” basis or requiring that the documents — and other submissions that reference- them — be filed under seal. As we discuss above, although those procedures are fallible, they are better than nothing. CONCLUSION In sum, we hold as follows: (1) A writ of mandamus is the only “adequate means” for the City to “attain the relief [it] desires” — namely, reversal of the District Court’s thoughtful and thorough discovery order requiring the City to disclose 1800 pages of undercover Field Reports. Cheney v. U.S. Dist. Court for Dist. of Columbia, 542 U.S. 367, 380, 124 S.Ct. 2576, 159 L.Ed.2d 459 (2004). In particular, we hold that, in the circumstances presented, disclosing the Field Reports on an “attorneys’ eyes only” basis and filing the Reports “under seal” will not adequately protect the secrecy of the Reports. Thus, we hold that an appeal after a final judgment is not an “adequate means” for. the City to challenge the District Court’s order. We also hold that (a) a certified interlocutory appeal under 28 U.S.C. § 1292(b) and (b) an immediate appeal of an order of criminal contempt are not “adequate means” for the City to challenge the order. (2) A writ of mandamus is “appropriate under the circumstances,” Cheney, 542 U.S. at 381, 124 S.Ct. 2576, because the City’s petition, inter alia, raises several “novel and significant questions] of law ... whose resolution will aid in the admin *950 istration of justice.” In re S.E.C. ex rel. Glotzer, 374 F.3d 184, 187 (2d Cir.2004). Most significantly, the petition requires us to clarify the circumstances under which the law enforcement privilege must give way to a party’s need for discovery — an issue never before decided by our Court. (3) To clarify when the law enforcement privilege must give way to a party’s need for discovery, (a) The party asserting the law enforcement privilege bears the burden of showing that the privilege applies to the documents in question. In re Sealed Case, 856 F.2d 268, 271-72 (D.C.Cir.1988). (b) To determine whether the privilege must give way, a court must balance “[t]he public interest in nondisclosure” against “the need of a particular litigant for access to the privileged information.” Id. at 272. ' (c) In balancing those interests, a court must apply a “strong presumption against lifting the [law enforcement] privilege.” Dellwood Farms, Inc. v. Cargill, Inc., 128 F.3d 1122, 1125 (7th Cir.1997). (d) To rebut that “strong presumption,” the party seeking disclosure must show: (i) that its suit is “non-frivolous and brought in good faith,” Friedman v. Bache Halsey Stuart Shields, Inc., 738 F.2d 1336, 1343 (D.C.Cir.1984); (ii) that “the information sought is [not] available through other discovery or from other sources,” id.) and (iii) that the information sought is “important” to the party’s case, id. Information is sufficiently important only if the party seeking its disclosure demonstrates a “compelling need.” Cf. Marriott Int’l Resorts, L.P. v. United States, 437 F.3d 1302, 1307 (Fed.Cir.2006) (emphasis added). (4) Applying those standards to the circumstances presented, (a) The City has met its burden of showing that the law enforcement privilege applies to the Field Reports because the Reports, even as redacted by the District Court, contain detailed information about the NYPD’s undercover “law enforcement techniques and procedures.” In re Dep’t of Investigation of the City of N.Y., 856 F.2d 481, 484 (2d Cir.1988). (b) Plaintiffs do not have a “compelling need” for the Field Reports because the Reports do not contradict, undermine, or otherwise cast doubt upon the End User Reports — the documents upon which the City will rely in defending its arrest and fingerprinting procedures. (c) Because plaintiffs do not have a “compelling need,” they have not overcome the “strong presumption” against lifting the law enforcement privilege. (5) The City’s right to a writ of mandamus is “clear and indisputable” because the District Court committed three errors that amounted to a “clear abuse of discretion,” if not a “judicial usurpation of power.” Cheney, 542 U.S. at 380-81, 124 S.Ct. 2576. (a) The District Court indisputably committed legal error when it failed to apply a strong presumption against lifting the law enforcement privilege. (b) The District Court indisputably committed legal error when it failed to require that plaintiffs show a compelling need for the Field Reports. (c) The District Court indisputably made a clearly erroneous assessment of the evidence when it found that plaintiffs’ need for the Field Reports outweighed the public’s interest in maintaining the integrity of the NYPD’s undercover operations and the safety of NYPD officers. *951 For the foregoing reasons, despite the admirable care of the District Court in dealing with these sensitive and complex questions, we grant the City’s petition for a writ of mandamus; we vacate the December 10, 2009 order of the District Court; and we instruct the District Court to deny plaintiffs’ motion to compel the production of the Field Reports. Footnotes [1] See Schiller v. City of N.Y., No. 04 Civ. 7922, Docket Entry No. 40 (S.D.N.Y. Oct. 14, 2005) (referring the underlying action to Magistrate Judge Francis for "General Pretrial” purposes, including "scheduling, discovery, nondispositive pretrial motions, and settlement”). [2] Although the District Court's order is dated December 10, 2009, it was not entered on the District Court's docket until December 14, 2009. [3] To reach this conclusion, Commissioner Cohen analyzed the "ratio of violence and civil disorder to law[-]abiding protest[e]r crowd size” during large political events hosted in eight cities between 1999 and 2004. Pet. 5 (citing Cohen Decl., June 6, 2007, Ex. A at 8-11). For example, during the World Trade Organization meetings held in Seattle in 1999, roughly two hundred anarchist vandals turned anti-globalization protests into violent riots, leading to a collapse in the World Trade Organization meetings. See App. 126-27 (Cohen Decl., June 6, 2007, Ex. A at 8-9); see also Kim Murphy, Anarchists Deployed New Tactics in Violent Seattle Demonstrations, L.A. Times, Dec. 16, 1999, at A3. These riots were later called the "Battle in Seattle.” See, e.g., Robert A. Jordan, Battle in Seattle Sent a Message, Boston Globe, Dec. 7, 1999, at D4. [4] Both the End User Reports and the Field Reports are a part of the record before us because they were part of the record upon which the District Court issued the discovery order at issue. The End User reports were submitted to the District Court as Exhibit B to the September 19, 2008 declaration of Assistant Corporation Counsel Tonya Jenerette. The Field Reports were submitted to the District Court for in camera review on June 20, 2007. Schiller, No. 04 Civ. 7922, Docket Entry No. 232 (S.D.N.Y. June 21, 2007). [5] A summons is "[a] writ or process commencing the plaintiff's action and requiring the defendant to appear and answer,” Black’s Law Dictionary 1574 (9th ed.2009), presumably at some later date, while an arrest would take the individual into custody immediately, id. at 124 (defining "arrest” as “the taking or keeping of a person in custody by legal authority”). See N.Y.Crim. P. Law §§ 150.10, 150.20; see also Bryant v. City of N.Y., 404 F.3d 128 (2d Cir.2005). [6] Plaintiffs have explicitly stated that they are not challenging the legality of those undercover operations in the underlying actions. Plaintiffs challenge only their arrest, detainment, and fingerprinting which resulted from their conduct at the RNC. [7] This Rule provides in relevant part: When a pretrial matter not dispositive of a party's claim or defense is referred to a magistrate judge to hear and decide, the magistrate judge must ... issue a written order stating the decision. A party may serve and file objections to the order within 14 days after being served with a copy. A party may not assign as error a defect in the order not timely objected to. The district judge in the case must consider timely objections and modify or set aside any part of the order that is clearly erroneous or is contrary to law. Fed.R.Civ.P. 72(a). [8] The panel consisted of Judges Cabranes and Parker, as well as Judge Stefan R. Underhill, of the United States District Court for the District of Connecticut, who was sitting by designation. See Order, Mar. 12, 2010. [9] This statute provides in relevant part: When a district judge, in making in a civil action an order not otherwise appealable under this section, shall be of the opinion that such order involves a controlling question of law as to which there is substantial ground for difference of opinion and that an immediate appeal from the order may materially advance the ultimate termination of the litigation, he shall so state in writing in such order. The Court of Appeals which would have jurisdiction of an appeal of such action may thereupon, in its discretion, permit an appeal to be taken from such order, if application is made to it within ten days after the entry of the order. ... 28 U.S.C. § 1292(b). [10] In Sims v. Blot, we granted mandamus relief to the petitioner who, like the City here, challenged a district court’s determination that confidential information sought in discovery was not privileged and therefore must be disclosed. 534 F.3d at 142. Our decision to do the same here is consistent with this recent and important precedent of our Court. [11] We do not know whether the information contained in the field reports pertains to ongoing investigations. After reviewing the Field Reports, however, we are confident that the techniques the undercover officers used to investigate organizations before the RNC— techniques that are outlined in the Field Re *935 ports — are currently in use in NYPD undercover operations. [12] In addition, the disclosure of information on an "attorneys' eyes only” basis is occasionally a feature of criminal cases in which the government is required to disclose classified national security information in order to protect " 'a criminal defendant's right to present a meaningful defense.’ " See, e.g., In re Terrorist Bombings of U.S. Embassies in E. Afr., 552 F.3d 93, 126 (2d Cir.2008) (quoting United States v. Aref, 533 F.3d 72, 79 (2d Cir.2008)). The use of the procedure in those cases does not show that it is appropriate here. As an initial matter, when classified material is disclosed on an "attorneys' eyes only” basis in criminal trials, a defendant’s attorneys must "possess!] the security clearances necessary to review and inspect such material.” Id. Here, plaintiffs have made no showing that any of the fifty attorneys representing them has a security clearance — or other form of credential — vouching for their trustworthiness. Of course, we have no basis to question — and do not question — the integrity or competence of the attorneys involved in this litigation. But we are not able to make blanket statements about the reliability of counsel merely based on their membership in the bar, for the record of the bar in such matters is not without blemish. Indeed, some attorneys have deliberately disclosed confidential information. See, e.g., United States v. Stewart, 590 F.3d 93 (2d Cir.2009); Bob Egelko, Lawyer Admits Leaking BALCO Testimony, S.F. Chron., Feb. 15, 2007, at A1. Moreover, mere negligence of an attorney — rather than deliberate malfeasance- — could result in the unintended disclosure of confidential information. Furthermore, we are presented here with a discovery request by a plaintiff in a civil action, not a defendant in a criminal action. We have no trouble concluding that a plaintiff's right to discovery in a civil action is less fundamental than a criminal defendant's constitutional right “ ‘to present a meaningful defense.' " In re Terrorist Bombings, 552 F.3d at 126 (quoting Aref, 533 F.3d at 79). In any event, as we explain in Part IV below, the disclosure of information on an “attorneys’ eyes only” basis is an appropriate measure insofar as the information must, in fact, be disclosed. It is, nonetheless, a deeply flawed procedure that cannot fully protect the secrecy of information in this case; it merely mitigates — to some degree — -the possibility of unauthorized disclosure. Thus, the fact that "attorneys’ eyes only” disclosure has been used as a mitigating procedure in criminal cases — or in other civil litigation — does not alter our analysis. [13] It was never determined who made the unauthorized disclosure. As the City asserts in its petition, "the violation of the court’s confidentiality order escaped any consequences.” Pet. 55. [14] Again, in reaching this conclusion, we do not question the integrity or competence of the attorneys involved in this litigation. [15] The examples cited in the foregoing paragraphs all refer to unintentional, and perhaps negligent, disclosures of sealed information. Much less commonly, a court employee can deliberately disclose sealed material. See, e.g., Former Court Clerk Admits Disclosing Information in Newton Case, The Day (New London), May 2, 2006, at C7; Leonard Buder, Former Court Clerk Admits Giving Information to Mob, N.Y. Times, May 30, 1987, § 1, at 35. [16] A writ of mandamus may also be available if there is an " ‘extreme need for reversal of the district court’s mandate.’ ” Glotzer, 374 F.3d at 187 (quoting von Bulow, 828 F.2d at 97). For example, the Supreme Court has granted the writ “to restrain a lower court when its actions would threaten the separation of powers by 'embarrassing] the executive arm of the Government' or result in the 'intrusion by the federal judiciary on a delicate area of federal-state relations.' ” Cheney, 542 U.S. at 381, 124 S.Ct. 2576 (citations omitted) (quoting Ex parte Peru, 318 U.S. 578, 588, 63 S.Ct. 793, 87 L.Ed. 1014 (1943); Will v. United States, 389 U.S. 90, 95, 88 S.Ct. 269, 19 L.Ed.2d 305 (1967)). [17] We note that Cheney requires only that we consider whether the writ of mandamus is “appropriate under the circumstances.” Cheney, 542 U.S. at 381, 124 S.Ct. 2576. If a petition raises a novel question of law, it is more likely that we will conclude that mandamus is appropriate. Applying Cheney, however, a petition need not present a novel question of law to warrant the writ of mandamus; ihe writ need only be “appropriate under the circumstances.” Id. The writ of mandamus could be appropriate, for example, if a district court ruling flagrantly misapplies a well-settled principle of law. In that instance, the petition might not present a novel question, but the writ of mandamus may nevertheless be "appropriate under the circumstances.” Id. [18] The law enforcement privilege is related to, and indeed an outgrowth of, the executive privilege long recognized at common law. The law enforcement privilege "shares with those [privileges] typically labeled 'executive’ a justification rooted in the need to minimize disclosure of documents whose revelation might impair the necessary functioning of a department of the executive branch.” Black v. Sheraton Corp. of Am., 564 F.2d 531, 541-42 (D.C.Cir.1977). The interests that underlie the privilege are "rooted in common sense as well as common law” and, like other common law privileges, the law enforcement privilege is subject to “pragmatic adjustment to the needs of sound government.” Id. at 542. In 1957 the Supreme Court explicitly recognized an "informer’s privilege” — that is, the ”[g]overnment’s privilege to withhold from disclosure the identity of persons who furnish information of violations of law to officers charged with enforcement of that law.” Roviaro v. United States, 353 U.S. 53, 59, 77 S.Ct. 623, 1 L.Ed.2d 639 (1957). As we and many of our sister Circuits have noted, however, the law enforcement privilege is not limited to protecting the identity of informers. See Dep’t of Investigation, 856 F.2d at 484; see also In re U.S. Dep’t of Homeland Sec., 459 F.3d 565, 569 (5th Cir.2006) (collecting cases and noting that "case law has acknowledged the existence of a law enforcement privilege beyond that allowed for identities of confidential informants”). The law enforcement privilege was also recognized in the proposed Federal Rules of Evidence promulgated by the Supreme Court in 1974. Although specific rulés governing the various common law privileges were ultimately rejected by Congress in favor of the broad and flexible Rule 501, see Fed.R.Evid. 501 (providing that the rules of privilege "shall be governed by the principles of the common law as they may be interpreted by the courts of the United States in light of reason and experience”), the rejected Rules provide considerable guidance to courts attempting to discern the "principles of the common law.” Cf. United States v. Gillock, 445 U.S. 360, 367-68, 100 S.Ct. 1185, 63 L.Ed.2d 454 (1980) (noting that the absence of a proposed Rule covering a particular privilege "suggest[s] that the claimed privilege was not thought to be ... indelibly ensconced in our common law”). Two proposed rules encompassed concepts now embodied in the law enforcement privilege. Proposed Rule 509, titled "Secrets of State and Other Official Information,” specifically covered, among other things, “investigatory files compiled for law enforcement purposes.” See 3 Jack B. Weinstein & Margaret A. Berger, Weinstein’s Federal Evidence § 509.01 (Joseph M. McLaughlin ed., Matthew Bender 2d ed.2001). Proposed Rule 510, titled "Identity of Informer,” permitted the "government or a state or a subdivision thereof ... to refuse to disclose the identity of a person who has furnished information relating to or assisting in an investigation of a possible violation of law to a law enforcement officer.” See id. § 510.01. Thus, despite its recent origins, the law enforcement privilege is grounded in well-established doctrine and is widely recognized by the federal courts. [19] For example, under New York’s Freedom of Information Law, many records created by law enforcement agencies are exempt from disclosure. See N.Y. Pub. Off. Law § 87(2)(e) (McKinney 2001). This statutory protection, of course, does not apply in the litigation context, but the clear exemption of certain law enforcement materials from disclosure under state law suggests that the New York state legislature explicitly recognizes the importance of maintaining the confidentiality of certain law enforcement materials. [20] The Federal Freedom of Information Act, or FOIA, contains an exemption for "records or information compiled for law enforcement *942 purposes” provided that, disclosure of such information could have harmful consequences. See 5 U.S.C. § 552(b)(7). [21] The word "abuse” in the "abuse of discretion” standard is an unfortunate — and inaccurate — term of art. When a district court abuses its discretion, it involves nothing as heinous as abuse. Indeed, a so-called abuse of discretion often involves something quite common and unavoidable in a system of adjudication: a “view of the law” that is simply "erroneous.” Sims, 534 F.3d at 132. It is, to be sure, a serious matter when a district court reaches a "clearly erroneous assessment of the evidence” or makes a ruling that "cannot be located within the range of permissible decisions.” Id. But such is the stuff of ordinary appellate review. An appellate court sometimes disagrees with a district court, even when the appellate court applies a deferential standard of review. That does not mean that the district court has acted arbitrarily or committed any form of "abuse” in the ordinary sense of the word. [22] We conclude that there is a "strong presumption against lifting the privilege" because, as the Seventh Circuit noted, "[t]he heart of our concern [over lifting the privilege] is with the principle that control of criminal investigations is the prerogative of the executive branch, subject to judicial intervention only to protect rights.” Dellwood Farms, 128 F.3d at 1126. Determining that law enforcement materials are subject to disclosure, then, intrudes into the province of the executive branch of federal, state, or local governments. We do not take making such an intrusion lightly. [23] We are mindful that the ultimate burden of demonstrating the law enforcement privilege is on the party asserting the privilege — here, the City, see Friedman, 738 F.2d at 1341 — but we nevertheless expect the opposing party to establish its “compelling need” for the information. See generally Tuite v. Henry, 98 F.3d 1411, 1418 (D.C.Cir.1996) (implicitly assuming that the "need” factor is to be established by party seeking disclosure); Friedman, 738 F.2d at 1341 (same). This is so because the party seeking disclosure of information is uniquely situated to explain to a court why it wants and needs that information. [24] We reiterate that when the District Court determined that plaintiffs need the Field Reports, it made implicit factual findings as to the contents of the Field Reports and how the Field Reports could prove useful in litigation. To determine whether the District Court indisputably abused its discretion, we review those factual findings for clear error. To accomplish this, we have carefully examined the Field Reports and End User Reports in camera. [25] In some, though not necessarily all, cases an appropriately general docket entry memorializing the ex parte proceeding can be entered in the public records of the district court so long as it does not compromise the interests of the party holding the confidential information, or the public.