In the wake of the case of National Security Agency leaker Edward Snowden, the company that conducted the background investigation for his security clearance is under heavy scrutiny. USIS has received significant media attention and is under investigation by the inspector general of the U.S. government’s Office of Personnel Management. While such investigations of government contractors happen frequently, this particular case is interesting in that it concerns background investigations, which are considered by many to be a critical component of protecting national security.

However, background investigations as currently designed and conducted are fundamentally flawed. They are more effective at providing bureaucratic cover than they are at catching spies or preventing moles from penetrating the system.

As a bit of disclosure, I have had several U.S. government background investigations conducted on me. I was first subject to full-field background investigations as an Army Intelligence Officer and then later as a Diplomatic Security Service special agent. I have also conducted background investigations, first as a Diplomatic Security Service special agent domestically and while assigned to a U.S. embassy overseas, and then later as a contract investigator for the State Department. I have also been interviewed by scores of investigators conducting background investigations on my friends, colleagues and former interns and employees. All this to say that my experience as a subject and source in various investigations, and as an investigator who has conducted hundreds of background investigations myself, has given me some insight into the problems of the current process.

National security background investigations in the United States date back to efforts to prevent potential Nazi and communist spies from infiltrating the government. Today, every federal government employee is subjected to a basic criminal and credit history check as part of the hiring process. Those employees who require access to classified information as part of their duties will also be subjected to a national security background check. These can vary from a perfunctory national agency records check for confidential or secret clearances to full-field background checks for access to top-secret material and sensitive compartmented information. Some agencies, such as the CIA and FBI, also mandate full lifestyle polygraph examinations as part of the clearance process. People holding a security clearance are also subjected to a periodic reinvestigations.

One of the biggest problems with personnel security investigations is the sheer number of them that are conducted. The State Department, which has fewer than 19,000 U.S. employees, conducts some 20,000 of them each year. When one considers the 240,000 employees at the Department of Homeland Security and the 3.23 million employed by the Department of Defense, as well as myriad other agencies and contractors with government security clearances, the number of personal security investigations conducted each year is staggering.

Going back to the State Department example, with only about 2,000 Diplomatic Security Service special agents worldwide, and the vast majority of them assigned to duties other than background investigations, the bulk of the investigations are conducted by contractors. But beyond sheer manpower considerations, most special agents simply do not view background investigations as important. Speaking from personal experience, I was far more motivated to investigate a bombing attack against an embassy or arresting a drug smuggler, terrorist or pedophile for passport fraud than I was to interview a potential employee’s neighbors or former bosses. Because background investigations are not exciting and sexy, conducting them is also not considered particularly career enhancing compared to other assignments. I know from talking with friends from other agencies that agents there had similar views. Even many investigators working for investigation companies view their jobs more as stepping stones to other occupations, such as being a federal agent.

This disdain for background investigations by the people conducting them is compounded by the people who manage personnel security investigations. Because of the massive volume of cases involved, program managers tend to be focused on speed, efficiency and cost far more than on thoroughness. They encourage investigators to do the absolute minimum to meet the requirements as outlined in Executive Order 12968, which sets the standards for access to classified information, rather than to focus on really investigating and understanding the background of the subject of the investigation. In the case of a for-profit company like USIS, the bottom line is also a factor. Streamlining the efficiency of the process means doing the very least amount of work required to complete the specific task, thus maximizing profit and satisfying the board of directors.

Agents and investigators who sense something is not right and want to dig deeper and interview additional developed sources are frequently criticized for missing deadlines or conducting unnecessary interviews. Contract investigators who do this are also often accused of milking the system. Many investigators therefore find it easier to just do the minimum required rather than have to go back to their supervisors to get permission to conduct additional interviews beyond what was initially assigned to them.

There is a very real tension between managers who want streamlined investigations done within deadlines and good investigators who want to conduct a thorough investigation. A good investigator is methodical and persistent in his craft, but methodical and persistent investigations often take longer and cost more than investigation program managers will allow. Good investigators also frequently want to follow hunches and instincts, and it is very hard to convince managers to permit that type of latitude, and so the investigations rarely go very deep.

Even the interviews tend to be shallow. Most investigators merely read Privacy Act and Freedom of Information Act warning statements followed by a list of obligatory, scripted questions rather than really probe during an interview.

While most agencies and companies do not permit investigators to conduct phone interviews except in exigent circumstances, some investigators have become masters of eliciting people to request telephonic interviews. By conducting interviews by phone, a contract investigator can cut down on travel time and complete more interviews more quickly — thus making more money per hour. I live in a very remote area, and most investigators do not want to make the effort to drive out to interview me in person, so I often get this approach by investigators who are conducting investigation on friends and former employees. Since I know the game, when I am solicited for a telephonic interview, I refuse and make the investigator drive out to talk to me in person.

The problem with telephonic interviews is that the people being interviewed via telephone are far less likely to provide derogatory information than people interviewed in person. I experienced this several times while conducting split investigations, in which another investigator and I both identified and interviewed the same developed source. In one case, I interviewed a woman who provided some incredibly significant derogatory information about a subject indicating he had lied about his background. During our interview, she also told me that another investigator had interviewed her. When I asked her if she had also related this critical information to the other investigator, she said no, because he had only conducted a telephonic interview and had not asked her any specific questions that would have led to the information.

Dishonest investigators are also known to fabricate interviews they never conducted. While some investigators do this out of greed, others do it out of frustration when they have attempted to reach a source several times and a manager is pressuring them to close out a case. A good investigator will resist this pressure, because he or she has the mindset that the source may be able to provide a critical piece of information needed to break open the case. The poor investigator just sees the interview as a yet another meaningless task to complete, and therefore feels less guilt about the fabrication. Such conduct is criminal, and several investigators have been prosecuted in recent years for such activity. Agencies and companies are supposed to audit the work of investigators to protect against such things, but this auditing also takes time and money — according to the Washington Post, one of the things USIS is being investigated for is failure to conduct sufficient audits.

I have found that the prevailing attitude among managers and investigators is that background investigations are perfunctory tasks that only require the minimum work necessary to check off the prerequisite boxes and get the investigation over as quickly — and as superficially — as possible. This means that the subject of the investigation can greatly influence its scope just by the information he or she provides — or withholds — in questionnaires and interviews.

Moreover, the background investigations currently being conducted were developed decades ago when the world was a far different place. They are not really designed to deal with the complexity of the modern world. In the Internet age, people no longer must to go to Klan meetings, neo-Nazi rallies, anarchist bookstores or jihadist mosques to be exposed to such ideas or radicalized by them. In much the same way that people can become grassroots terrorists through such a self-radicalization process, they can also become a grassroots intelligence threat. The low-key methods of recruitment and radicalization mean that this potential threat may not be visible to others. This very real limitation is then compounded by the fact that personal security investigations have become meaningless bureaucratic processes that lack the ability to uncover the type of information required to catch an infiltrator who does not want to be caught.

The investigations are also only looking for very specific behaviors in the subject’s past, such as criminal behavior, debt, mental health issues or drug use. These things are merely outward indicators, not real insights into the subject. Besides, a lack of these indicators does not necessarily mean that the subject will not compromise national security in the future, especially if the person is motivated toward espionage or a massive public disclosure of classified information by ideology or ego. Even many of the people motivated by money have done so more out of greed than by any demonstrable history of financial problems.

Furthermore, with the transient nature of today’s society, where people frequently change jobs and addresses, neighbors and co-workers simply do not know people as well as they did in the 1950s. I have interviewed countless neighbors and co-workers who had absolutely no relevant knowledge of the subject I was investigating. Yet these interviews were deemed sufficient to check off a required box. The problem is that a person who does not know their neighbor is simply in no position to provide a reason that neighbor should not be considered suitable for a position of trust and confidence with the U.S. government.

Polygraphs are not much more effective. As we’ve discussed in relation to previous cases, polygraph examinations work well when administered to honest people, but they are not very helpful in cases where an individual has no compunction against lying or where the subject has received training in deceiving the polygraph. Even in cases where the polygraph is effective on a subject, it will only catch past nefarious behavior — it cannot detect or prevent future espionage.

The U.S. government is spending billions of dollars on a national security background investigation system that is archaic and largely ineffective. The system works on honest people or stupid spies, but is quite simply not very effective in uncovering the lies of clever, duplicitous people. It is also clearly not designed to anticipate future behavior. The Office of Personnel Management’s inspector general is attempting to hold USIS accountable for some allegations of sub-standard performance, but who is holding the entire system accountable? The only thing the current system does well is provide cover for bureaucrats who can point to a completed investigation as proof they are not culpable for a security breach.