In Pittsburgh, researchers with a Pentagon contract are teaching computers to scan software for bugs and turn them automatically into weapons. In a converted textile mill in New Hampshire, programmers are testing the combat potential of coding errors on a digital bombing range.
Across the US, a new league of defence contractors is mining the foundation of the Internet for glitches that can be turned to the country's strategic advantage. They are part of a cybermilitary industrial complex that has grown up in more than a dozen states and employs thousands of civilians, according to 15 people who work for contractors and the government. The projects are so sensitive their funding is classified, and so extensive a bid to curb their scope will be resisted not only by intelligence agencies but also the world's largest military supply chain.
"We're in an arms race," said Chase Cunningham, the National Security Agency's former chief cryptologic technician. The competition to find exploitable bugs before an enemy does is as intense as "the space race and the Cold War combined."
The US has poured billions of dollars into an electronic arsenal built with so-called zero-day exploits, manipulations of missteps or oversights in code that can make anything that runs on a computer chip vulnerable to hackers. They go far beyond flaws in web encryption like SSL and OpenSSL, which the NSA has exploited for years without warning the public about it, according to people with knowledge of the matter.
The agency's stockpile of exploits runs into the thousands, aimed at every conceivable device, and many are not disclosed even to units within the agency responsible for defending US government networks, people familiar with the program said.
Under a directive made public April 11, after Bloomberg News reported the NSA's utilisation of the infamous Heartbleed bug -- a use the agency denied -- the White House said exploits should in most cases be disclosed so computer users can protect themselves.
Michael Daniel, the White House cybersecurity coordinator, said in a blog post this week that "building up a huge stockpile of undisclosed vulnerabilities while leaving the Internet vulnerable and the American people unprotected would not be in our national security interest."
He said the US would continue to develop and use those vulnerabilities to protect the country, however, and that the administration has established "a disciplined, rigorous and high-level decision-making process" when it comes to deciding whether to keep the flaws secret or disclose them so they can be fixed.
The NSA referred to the White House blog in response to a request for comment.
Because the White House directive said there should be exceptions for national security, the impact it will have is uncertain: Using just about any computer bug as a weapon can be justified as the Web plays an increasingly central role in intelligence gathering and kinetic conflict. During his confirmation hearing, Navy Vice Admiral Michael S Rogers, director of the NSA and the US Cyber Command, said it would be hard to imagine an international crisis not involving digital weaponry.
It is also hard to imagine the US increasingly sophisticated cyberspying and cyberwar operations without its deep arsenal of software exploits, according to current and former arms makers. Those operations would be slower and more susceptible to detection without zero days.
And like giving up sophisticated missiles and bombers, giving up an arsenal of highly valuable computer exploits could leave the country more vulnerable in a future national security crisis, those experts say. Pentagon contractors are working on developing them for the CIA, the Army's recently activated 780th Military Intelligence Brigade, the NSA’s Tailored Access Operations corps and other units. The military's cyberwarriors are trained at places like the US Navy's Center for Information Dominance in Pensacola, Florida.
As conventional military spending has been cut back and funding for cyber operations ramped up, Lockheed Martin Corp (LMT), Northrop Grumman Corp (NOC) and others better known for jets and tanks are retooling for a new generation of armaments.
"Of countries that are developing industrial-strength cyber capabilities, certainly the US is in the lead," said Nate Freier, a research professor at the US Army War College in defence and military strategy. "The question is whether we understand it well enough to use it without encountering a significant amount of blowback or unintended consequences."
Spy agencies develop exploits themselves, buy them from contractors or steal them from rivals. And the arsenal must be constantly refreshed, as the software at which they’re aimed is updated and fixed. Some exploits are used for years; others only for a short time before the flaws on which they are based are patched.
There are more than bugs in the government's arsenal: Hackers at the NSA's Tailored Access Operations, or TAO, have more than 1,000 special tools to aid them in stealing data or manipulating a rival's electronics. As described by three people briefed on the technology, the tools enable rapid, mix-and-match attack capabilities against the most widely used computers, servers and software.
If TAO wants to switch on a microphone in a computer running Microsoft Windows, covertly recording conversations of anyone nearby, a custom module does the job. If it needs to hijack the system that communicates between computers and the controls that operate train-track switches or dam flow gates, there is a plug-in for that too. Dozens of the plug-ins can be loaded onto a single digital warhead, customised to the needs of the mission at hand.
The private sector provides support at all levels. When intelligence agencies were looking three years ago for holes in commercial software that runs video conferencing systems, they reached out to several contractors. Endgame Inc, an Arlington, Virginia-based company that once specialized in weaponising software bugs, provided the exploits, allowing US spies to tap into the systems, according to a person familiar with the contract.
Sara Conneighton, a spokeswoman for Endgame, which is also working on commercial security applications, declined to comment.
Most defence contractors have launched cyber-weapons programs; many have grown through acquisitions of boutique security and Internet-intelligence firms.
Northrop Grumman, which makes the Global Hawk drones, purchased Essex Corp, a NSA supplier, in 2006. Combat radio-maker Harris Corp (HRS) purchased Crucial Security Inc, which develops hacking tools, in 2009.
"Cyber money is not shrinking the way the rest of the defence budget is," said Dave Aitel, chief executive officer of Immunity Inc, an offensive security firm in Miami. "That means that all the big beltway bandits must invest heavily to build their cyber teams and that this market is going to continue to grow."
In Obama's proposed fiscal 2014 budget, the money for cyber operations jumped 20%, to $4.5 billion, and the Pentagon placed it on a list of priority programs, an unclassified comptroller's presentation shows.
The Defence Department foresees spending $26.6 billion on cybersecurity efforts for the five years ending in 2019, and requested $5.06 billion for fiscal year 2015, according to Pentagon budget documents.
Of all the major defence contractors, Raytheon Co may be best positioned to take advantage of the shift through a little-noticed acquisition made six years ago. In 2008, it bought SI Government Solutions, the brainchild of a former computer science professor at the Florida Institute of Technology who developed a method to rapidly scan software code to find flaws that could be exploited by hackers. Since then, the Raytheon division has grown to become one of the most prolific US cyber arms makers, said several people familiar with the subsidiary.
Its engineers develop exploits not only for computers but for every conceivable device with a microchip, from heating and air conditioning systems to printers to industrial computers used in manufacturing, according to a person who was recruited by the company and received a detailed description of the program.
For the CIA, SI Government Solutions specializes in ways to gain access to computers that are not connected to the Internet, according to a second person familiar with the subsidiary's government contracts. That includes the use of technology that can surreptitiously transfer data from so-called air-gapped computers, which often contain a rival country's most sensitive secrets.
Jason B Kello, a Raytheon spokesman, declined to comment on the company's role or to identify its clients. Ed Wallach, who recruits prospective employees for SI Government Solutions, said high demand and the need for specialised skills makes his job difficult, even though the 250-person subsidiary offers a more lucrative benefits package than the rest of Raytheon.
"We may not be building ships, but it’s clear this is one area where the government is willing to spend money," he said.
Finding bugs requires the creativity of human researchers as well as the power of computers, which relentlessly pound software programs with unpredictable data to spot a possible malfunction, a technique called fuzzing. The Finnish company Codenomicon was using an advanced fuzzing engine when it detected Heartbleed earlier this month, according to Mikko Varpiola, a Codenomicon co-founder. Researchers from Codenomicon and Google Inc. reported both the flaw and a fix for it.
ForAllSecure, a Pittsburgh company founded by researchers from Carnegie Mellon University, has a Pentagon contract to teach computers to scan for software vulnerabilities and automatically generate attack code. Its product - called Mayhem - has been used to analyze more than 37,000 off-the-shelf software programs and found 14,000 bugs in them, including 152 for which the company has developed exploits, said David Brumley, an assistant professor in computer science and engineering at Carnegie Mellon University who is leading the work. Without automation, it can take months to turn a coding mistake into a weapon.
If the US wanted to use an exploit to gather intelligence and not disclose the underlying error’s existence, Brumley said he would not object. "We have to be free to be able to strategically and appropriately use cyber weapons," Brumley said.
Unlike Tomahawk missiles, which do one thing and with a high degree of reliability, even the best-crafted exploits are unpredictable, because computer systems and software can come in almost endless combinations of configurations that could foil attacks. Siege Technologies, a startup in a converted 19th century textile mill along the Merrimack River in Manchester, New Hampshire, is working to change that with more than $10 million in contracts from the Department of Defence and other agencies.
For the last four years, Siege has been developing an algorithm that predicts the likelihood of a cyberattack’s success, a process that entails running attack code through thousands of test cases to generate models of how effective it would be in the real world, whether it is breaking into power grids to hacking mobile phones. The company's main product is Eprouvette, named after military equipment used to test the strength of gunpowder.
Siege Technologies is considering enhancements that would provide real-time feedback as to whether an exploit actually hit its target, said company founder Jason Syversen, whose background is in cryptography and hacking. Military commanders "want a smoking crater to prove an attack was successful," he said. "We don’t have that in cyber."
Syversen said he set out to create the equivalent of the military's so-called probability of kill metric, a statistical analysis of whether an attack is likely to succeed.
"I feel more comfortable working on electronic warfare," he said. "It's a little different than bombs and nuclear weapons -- that's a morally complex field to be in. Now instead of bombing things and having collateral damage, you can really reduce civilian casualties, which is a win for everybody."