This is my thinking as well. A lot of what spy tech is, is the ability to throw a warrant or NS letter into someone's face and force them to open the gate. The mythical uber-competent security services seem to be a pretty questionable premise. The few criminals caught via these tools are almost always the low hanging fruit of unconnected crazies, who seem to be harmless cranks and not real terrorists. Five eyes didn't stop or predict Snowden, Assange/Manning, Boston bombers, the rise of ISIS, Putin's advance on Crimea and Eastern Ukraine, the massive cyberwar Russia and China have declared on us, etc. In fact, all these things were major surprises to these security services, at least from what we can tell.
There's a lot of federal tax money spent on what is cargo cult science and cargo cult intelligence. The NSA has the ability to buy a lot of PhD's but they are stuck with the same batch of incompetent and profiteer federal contractors the rest of the government is stuck with.
It can't help that the Secret Service is losing top talent to private security firms like Academi (aka Blackwater), which draws a parallel with parts of the tech industry.
It makes me wonder exactly how much worse it is in the Intelligence industry. I agree with others in this thread, given the secrecy involved I expect it to be orders of magnitude worse than what we see in the private sector.
It also makes me wonder about the private Intelligence industry, about how much better they must be than their government counterparts, and about how they work together with government to "fight terror" and other demons. Combine that with the idea that government IT is losing talent to private firms in the same way... Maybe we should be more worried about private Intelligence firms than we currently are.
These days too many of them are a bit intimidated by the mere presence and aura of people in uniform, intel people, cops etc...Years ago, many of them had served in the military or a had a very good understanding of what they can really do (not what they claim they can do) how they worked/the types of people behind them(JFK, Churchill etc) so were rightly sceptical of their claims. I mean, has anyone ever met a cop/spook/politician who didn't want more power to do X, more money to do Y, more people, more secrecy etc?
One of the main problems for intel agencies is that they are merely one conduit to policy-makers, amongst many these days. Ultimately it is those policy-makers who need to (and increasingly are incapable of) using this information for strategic decision-making...Tactically it is easy to say "arrest/shoot that guy," because they get kudos and less blame if some thing goes wrong. Strategically it is harder to say "don't shoot that guy, keep him alive and in 5 years we can negotiate with him." It took the British a long time to learn how to do this in Northern Ireland, and it was only when they did that they managed to start to move towards peace with the IRA. I'm not sure would that be possibly in the bloggy, politically polarised, short-term, tabloid, Sky/Fox News type of world we live in these days.
After all, even if they did predict the rise of ISIS, Boston, it is the policy-maker who must decide what action to take. Difficult when:
a) There are sooo many warnings coming across their desk
b) Limited resources
C) Not many real policy instruments available. For example, the US public would not have been up for attacking for acting on intel available about ISIS 18 months ago, thus that policy option was thus not available.
>The few criminals caught via these tools are almost always the low hanging fruit of unconnected crazies
This is true of the criminals caught via FBI counterterrorism's more traditional police work that ends in civilian courts.
It remains entirely possible that the intelligence establishment enjoys some success in penetrating more sophisticated groups, but doesn't write press releases about it (since it will be acted upon with further covert operations to get further into those organizations, or with secret assassinations.)
There's a lot of federal tax money spent on what is cargo cult science and cargo cult intelligence. The NSA has the ability to buy a lot of PhD's but they are stuck with the same batch of incompetent and profiteer federal contractors the rest of the government is stuck with.