January 22, 2025

Why Open Source Intelligence Could Be Hurting the Future of National Security with Jeff Rogg, author of The Spy and The State

The historical evolution of American intelligence offers a lens through which to understand today’s complex national security challenges. As the information landscape becomes increasingly vast and nuanced, tracing the trajectory of intelligence—from its origins to its current form—provides insights into the interplay between secrecy, trust, and the public-private partnerships that define its practice.

My conversation with Jeff Rogg, the  author of the forthcoming book, The Spy and The State, helped me to examine these historical precedents. Together we uncovered lessons on navigating technological disruption, fostering trust, and balancing transparency with national security imperatives.

From Ivy League Elitism to Broader Recruitment

Early American intelligence was deeply rooted in exclusivity, exemplified by the "pale, male, and Yale" archetype that characterized early CIA recruitment. "There was a reason for this," Jeffrey Rogg notes, "because, if you think about it with secrecy, who do you trust? Intelligence is very much a trust business."

This trust inherent in elite social networks was initially considered indispensable for a field reliant on secrecy. Yet this approach had inherent limitations, fostering accusations of elitism and exclusion. The democratization of intelligence recruitment—spurred by cultural and political shifts in the 1960s and 70s—highlighted the necessity of diversity in perspectives and expertise.

Today, intelligence faces a similar challenge as it seeks to adapt to an increasingly decentralized information environment. The recruitment of a more diverse workforce that reflects the complexities of modern global interactions can help ensure that intelligence analysis remains robust and relevant. As the information age democratizes access to data, intelligence agencies must evolve their trust-building mechanisms to account for both technological advancements and a broader societal spectrum.

Public-Private Collaboration: Historical Precedents and Modern Imperatives

Public-private collaboration in intelligence is not new. During World War II, academia and private industry played pivotal roles in cryptographic breakthroughs, such as the work of Alan Turing and the Bletchley Park team in the UK, which had analogous efforts in the U.S. The Cold War further institutionalized these relationships, exemplified by partnerships in signals intelligence and defense technology.

In the modern era, the private sector often outpaces government agencies in technological innovation, especially in artificial intelligence, cybersecurity, and data analytics. Companies like OpenAI and Filter Labs, which analyze open-source data, highlight the blurred lines between private innovation and public application. However, as Jeffrey Rogg emphasizes, "The intelligence community wants all the benefits of the relationship with none of the costs." The question of how to balance utility with privacy remains paramount, as private companies must navigate the risks of being co-opted by state agendas while safeguarding proprietary data and public trust.

Adapting to Technological Disruption

Periods of technological disruption have historically tested the adaptability of intelligence communities. The introduction of satellite imagery, for instance, revolutionized intelligence during the Cold War, shifting reliance from human intelligence (HUMINT) to technical intelligence (TECHINT). These shifts often exposed vulnerabilities in existing frameworks but also created opportunities for innovation.

Today’s disruptions—ranging from the proliferation of open-source intelligence (OSINT) to the challenges posed by social media and deepfake technologies—demand a similar willingness to adapt. "Open-source intelligence is not new," Rogg explains, "but today it’s doing something that can threaten the intelligence profession...it sort of indicates to people, policymakers, CEOs, that anyone can do intelligence because it’s open source." Intelligence professionals must distinguish between information and actionable intelligence while navigating an environment where the line between truth and manipulation is increasingly blurred. As Rogg suggests, the professionalization of intelligence—focusing on rigorous analysis and ethical decision-making—is crucial to maintaining credibility and efficacy.

Balancing Transparency and Secrecy

The tension between transparency and secrecy has been a recurring theme in American intelligence history. The Ramparts scandal of 1967, which revealed CIA involvement with the National Student Association, marked a turning point in public trust. Subsequent reforms, including the creation of the Church Committee and the Privacy and Civil Liberties Oversight Board, sought to institutionalize oversight and accountability.

In an era of ubiquitous information, the balance between secrecy and transparency has become even more precarious. Overclassification risks eroding public trust, while underclassification may compromise national security. As Rogg emphasizes, "Secrecy is a source of bureaucratic power. Controlling that information even from each other...is a source of power." Fostering trust requires intelligence agencies to demonstrate accountability not only to policymakers but also to the American public. Transparency initiatives, such as declassification programs and public engagement efforts, can help demystify intelligence and strengthen civil-intelligence relations.

The Role of Academia in Intelligence

The relationship between academia and intelligence has evolved from close collaboration to periods of estrangement and back again. During World War II and the early Cold War, academic institutions were integral to intelligence efforts, with many scholars transitioning into government service. However, revelations of covert CIA funding for academic organizations in the 1960s created a schism that persisted for decades.

Today, the resurgence of academic interest in intelligence studies offers an opportunity to rebuild trust and mutual understanding. By incorporating intelligence history and ethics into curricula, universities can prepare students to engage thoughtfully with national security issues. "Making American intelligence history a core part of the curriculum is crucial," Rogg argues, "because you don’t even know where you came from, and yet you’re trying to tackle the problems of today."

Reforms for the Future

To strengthen civil-intelligence relations, three reforms stand out:

  1. Enhancing Education and Historical Awareness: Making intelligence history a core component of professional training can provide practitioners with a deeper understanding of their field’s evolution and its ethical implications. "You’d be surprised how many intelligence officers don’t know their own history," Rogg notes.
  2. Reforming Congressional Oversight: Bipartisan and comprehensive oversight is essential to ensuring that intelligence agencies operate within legal and ethical boundaries while addressing the needs of a diverse society. Rogg highlights, "We need more comprehensive and bipartisan intelligence oversight that is also focused on transparency."
  3. Empowering Transparency Boards: Expanding the role of entities like the Privacy and Civil Liberties Oversight Board can help bridge the gap between secrecy and accountability, enabling informed public discourse. "Give them declassification authority across the intelligence community," Rogg suggests, noting that such a move would challenge entrenched secrecy.

The historical evolution of American intelligence underscores the importance of adaptability, ethical accountability, and public engagement. As Jeffrey Rogg aptly states, "The first principle of what you’re doing is don’t destroy that which you’re trying to defend."

By learning from the past, intelligence agencies can navigate the complexities of today’s information environment while maintaining the trust of the people they serve. In an age where information is both a tool and a weapon, the principles of transparency, diversity, and professionalism are more critical than ever.

Find the signal in the noise of global information. Delivered to your inbox every wednesday.
Copyright © Filter Labs