Russian Hacking and Bayes’ Theorem, Part 3

Time to deal with some objections, now that the previous parts are inked. Sam Biddle over at The Intercept seems quite aware of the evidence, as he adds details I didn’t:

  • The attacker or attackers registered a deliberately misspelled domain name used for email phishing attacks against DNC employees, connected to an IP address associated with APT 28/Fancy Bear.

  • Malware found on the DNC computers was programmed to communicate with an IP address associated with APT 28/Fancy Bear.

  • Metadata in a file leaked by “Guccifer 2.0″ shows it was modified by a user called, in cyrillic, “Felix Edmundovich,” a reference to the founder of a Soviet-era secret police force. Another document contained cyrillic metadata indicating it had been edited on a document with Russian language settings.

  • Peculiarities in a conversation with “Guccifer 2.0″ that Motherboard published in June suggests he is not Romanian, as he originally claimed.

  • The domain was registered by a person using the same email service as the person who registered a misspelled domain used to send phishing emails to DNC employees.

  • Some of the phishing emails were sent using Yandex, a Moscow-based webmail provider.

  • A link believed to have been used by APT 28/Fancy Bear in the past was also used against Podesta.

That’s Not How Any of This Works

But almost immediately, he starts downplaying them.

Viewed as a whole, the above evidence looks strong, and maybe even damning. But view each piece on its own, and it’s hard to feel impressed.

That’s not how induction works, though. Let’s look at the Higgs particle; in the end, the LHC did 300 trillion rounds of collisions to produce this graph.

Figure 3, depicting observed dual photons. From,

See that bump? That wee little fluctuation in two-photon detections was the smoking gun for the Higgs boson. But let’s apply Biddle’s reasoning. The Higgs can decay several ways, yet each one can be the result of another particle’s decay as well. So how do we know this isn’t another particle?

On top of that, the bump is pretty tiny. It’s roughly a fluctuation of 15% from the expected value, an increase of maybe 100 events. Pull any one event out, and it wouldn’t be convincing. Meanwhile, there’s another dataset bump near the 136 GeV range, yet that one apparently doesn’t count.

We thus conclude this evidence is lacking for the Higgs boson. Yet it was barely a year later that the notoriously conservative Nobel committee handed out a Nobel Prize for its discovery.

Biddle’s reasoning must be flawed, and indeed it is: he’s asserting that every piece of evidence in isolation must be convincing, and that if the likelihood of finding a bit of evidence under a hypothesis is weak then it cannot be evidence for that hypothesis. Both are false, and the case of the Higgs demonstrates why. The total sum of events can be much stronger evidence for the Higgs than any one event, and weak likelihood can become strong evidence if alternative explanations are even less likely. While many physicists were indeed cagey about calling this the Higgs boson (CERN states “results obtained in the various search channels are consistent with the expectations for the SM [Standard Model] Higgs boson” in their initial release), the next-best explanations were a new particle that no-one had thought was as likely, or a massive statistical fluke by two separate groups of scientists. It’s telling that they invited Peter Higgs to the party, you don’t do that unless you have a decent level of confidence in the results.

Likewise, if Biddle thinks the sum total of the evidence is “strong” or “damning,” in his words, then that counts a lot more than the weakness of any one piece of evidence. Nor is it enough to point to weaknesses in the evidence and say that reduces the likelihood of a Russian state actor behind the hack, you need to show how the evidence is more likely given another hypothesis.

Thumb on the Scales

Biddle seems desperate to disbelieve the Kremlin did it, though.

In its 2014 reveal of the group, the high-profile security firm FireEye couldn’t even blame Russia without a question mark in the headline: “APT28: A Window into Russia’s Cyber Espionage Operations?” The blog post itself is remarkably similar to arguments about the DNC hack: technical but still largely speculative, presenting evidence the company “[believes] indicate a government sponsor based in Moscow.” Believe! Indicate! We should know already this is no smoking gun. FireEye’s argument that the malware used by APT 28 is connected to the Russian government is based on the belief that its “developers are Russian language speakers operating during business hours that are consistent with the time zone of Russia’s major cities.”

Slight problem: that blog post links to a 48-page report which details APT28’s history of doing intelligence gathering against governments and NGO that the Kremlin has vocally opposed or view as enemies, yet has shown no interest in earning a profit. Computer programmers are notorious for working odd, erratic hours, because we don’t need to be in a specific physical location to get work done. If APT28 were a hacking collective you’d expect their malware to be developed at bizarre hours of the day; if APT28 were part of a stuffy government bureaucracy, you’d expect their working hours to coincide with business hours.

Biddle is being selective with the evidence, downplaying and even misrepresenting it in order to reach a predetermined conclusion. This is bizarre, coming from a journalist. But it’s not the worst card he plays.

Asking for the Impossible

It must be stated plainly: The U.S. intelligence community must make its evidence against Russia public if they want us to believe their claims. The integrity of our presidential elections is vital to the country’s survival; blind trust in the CIA is not. […]

I asked Jeffrey Carr what he would consider undeniable evidence of Russian governmental involvement: “Captured communications between a Russian government employee and the hackers,” adding that attribution “should solely be handled by government agencies because they have the legal authorization to do what it takes to get hard evidence.”

This is so naïve I have trouble seeing how it passed an editor. Marcus Ranum offers a good hint as to why:

I pointed this problem out during my “cyberwar is bullshit” talk at RSA conference in 2012: once you begin using your cyberweapons, they become subject to commercial pressures: and competitive analysis.

This guarantees that cyberweapons will have (relatively) short lifespans, and they’ll have the same problem that copy-protect and other digital rights management systems have: in order to work, you have to give them to the enemy, which means they are subject to examination and dissection. […]

Consider Duqu: we now know it was the ancestor of Stuxnet. We now know Stuxnet was an NSA TAO cyberweapon. Therefore, we can conclude that any systems prior to December 1, 2011 that showed prior infection by Duqu were infected by NSA TAO, Q.E.D.

Now think of what happens next. Suppose someone publishes a full public dissection of your hacking tool, showing a clear fingerprint for identifying it, correctly linking it to prior uses, and from those uses works out it probably comes from the organisation you work for. Do you:

A) Carry on using the tool, making it easy for anyone who’s read the report to track you in future, or
B) Use the published information to obliterate the unique fingerprint, and hope that the original authors can’t find a common thread they missed before, or
C) Stop using the tool, and start doing almost the opposite of what the published report claims you do?

Publicly revealing the evidence of hacking is a great way to convince people of its truth, but it’s also a great way to lose the ability to track the hackers. This is why the police never reveal their evidence until they absolutely have to at trial. This is why the FBI will let people they think are consuming child pornography walk free. This is why the CIA “cannot confirm or deny,” because even a single bit of information can reveal volumes. It is never in a government’s interest to explain the details of an investigation, especially when the target of the investigation is part of another government.

Commercial security companies have different interests, though. They need money from customers and investors to operate, so it’s in their interest to provide proof of their hacking savvy. If making those details public causes their opponents to switch to other tools, that merely generates more business for them. So private security companies tend to be more liberal with details.

The Double-Edged Sword, in Action

Remember the French election? The Kremlin has offered millions of Euros to France’s National Front. That party’s leader, Marine Le Pen, has been quite pro-Russia, most notably being one of a few foreign leaders to support the annexation of Crimea. Her main opponent during the runoff was Emmanuel Macron, who isn’t nearly as cosy with the Kremlin. Less than a day before the French media were forbidden to discuss the election, someone dumped a tonne of Macron’s emails to the intertubes, and Russia-aligned people and/or bots dutifully spread it around. This fits the pattern of recent Russian disinformation campaignswith one exception.

The head of the French government’s cyber security agency, which investigated leaks from President Emmanuel Macron’s election campaign, says they found no trace of a notorious Russian hacking group behind the attack. In an interview in his office Thursday with The Associated Press, Guillaume Poupard said the Macron campaign hack “was so generic and simple that it could have been practically anyone.”

Suppose a Russian government agency was behind the DNC and Macron hacks. How likely would it be for them to read all the reports on the DNC hack produced by Crowdstrike, SecureWorks, Fidelis, ThreatConnect, and so on, then use the report details to alter their techniques and wipe away their fingerprints for the Macron hack?

Now suppose a Chinese hacking group, or any group trying to pin the blame on Russia, were behind both hacks. How likely would it be for them to read all the reports on the DNC hack produced by Crowdstrike, SecureWorks, Fidelis, ThreatConnect, and so on, then use the report details to alter their techniques and wipe away their planted fingerprints for the Macron hack?

Now remember Putin keeps mentioning “patriotic Russians” when talking about hacks like these. You know, a group which would share the same language, culture, and political ideals as any official Russian government hackers. Who are very difficult to tell apart from government hackers, and for good reason.

Play The Tapes

I know, I know. Carr called for “Captured communications between a Russian government employee and the hackers” in specific, yet I’ve been talking about executable code and hacking methods. In my defence, more people call for the latter than the former, and about the only thing I need to change in my analysis is my analogy.

The F.B.I. affidavit said reporters for the news outlet, which it also did not name, had approached the N.S.A. with questions for their story and, in the course of that dialogue, provided a copy of the document in their possession. An analysis of the file showed it was a scan of a copy that had been creased or folded, the affidavit said, “suggesting they had been printed and hand-carried out of a secured space.”

The N.S.A.’s auditing system showed that six people had printed out the report, including Ms. Winner. Investigators examined the computers of those six people and found that Ms. Winner had been in email contact with the news outlet, but the other five had not.

Publishing a conversation, a date it took place, and the people who said it dramatically narrows the number of people who could have intercepted it. Carr’s request will result in dead NSA agents and the loss of further on-the-ground intelligence. No sane spy agency would grant it, so you’re going to have to rely on trust.

Don’t trust any of the US intelligence agencies? I already factored that into my earlier analysis.

Is this great evidence? Hell no, it’s entirely possible Putin is lying, and given the history of KGB and FSB it’s probable. But all that does is blunt the magnitude of the likelihoods, it doesn’t change their direction [away from the Kremlin].

So much for Biddle’s article. I’m out of time to deal with Ranum‘s contributions, though, so let’s go into overtime.