Hillary Clinton’s Email Server – Some Things


One of my commenters in another posting, asks:

What do you think of the notion, given the incompetence of the government in keeping hackers out of their computer systems, that The Secretary’s private email system might actually been no worse than the State Department’s system, which, apparently, is known to have been hacked into?

As I understand it, the FBI has no evidence that her private system was, in fact hacked but they, apparently, do have evidence that the State Department’s computer system was invaded by hackers (as by the way as has the Pentagon’s).

There are a lot of issues to cover and I’m going to jump on some of them scatter-shot, then see if I can conclude with a summary of my views, along with some supporting facts.*

First: the fact that security elsewhere in the government sucks is no excuse for Clinton to have decided to bypass State Departments’ email systems. We know that before Clinton became Secretary she and her circle communicated by Blackberry phones (B Clinton liked his, too!) When she became Secretary, Clinton apparently tried to get the NSA to authorize her use of Blackberry phones in a SCIF (Sensitive Compartmented Information Facilities – AKA: “place where secret stuff is stored and nobody takes electronic media in or out except Chelsea Manning or Aldrich Ames or John Walker or certain CIA directors and whatever”)  One data-point we can get from this is that somewhere along this time-line someone explained to the new Secretary what a SCIF was and why Blackberries were a problem and probably a few other basic things regarding classified material management.

It’s hugely unlikely that Clinton didn’t already know this stuff because she was part of the White House for the entire B Clinton administration and would surely have encountered security before. Some of the redacted emails from the FBI investigation apparently show that State Department staff and NSA had some energetic discussions around the topic of Clinton’s email and Blackberry.

First point: Clinton is not ignorant about this stuff.

Clinton didn’t like what NSA said so she ignored them and did her own thing with her own private communications infrastructure. Just like former CIA heads and Secretaries of State and presidents have done before.

Some things about Presidential Records

Clinton’s not president so presidential records are a non-issue. However, there was another Clinton that was president, and Clinton was in The White House at the time. B Clinton also liked his Blackberry and (apparently) the home mail server was initially set up for him. I say “apparently” because Clinton is already lying and “When did this all start?” would be a good thing to be vague about. A critical piece of this puzzle is the report from State Department’s Inspector General which indicated Clinton hadn’t complied with The Records Keeping Act and hadn’t apparently gotten permission to use her private server. That’s beaureaucratic butt-covering for “we couldn’t stop her.”

The Records Keeping Act is not about Presidential Records. It’s about agency records and it was signed by President Obama in 2011. The White House press release about it says some stuff like:

Sec. 2Agency Commitments to Records Management Reform.  (a)  The head of each agency shall:

 (i)    ensure that the successful implementation of records management requirements in law, regulation, and this memorandum is a priority for senior agency management;

 (ii)   ensure that proper resources are allocated to the effective implementation of such requirements; and

 (iii)  within 30 days of the date of this memorandum, designate in writing to the Archivist of the United States (Archivist), a senior agency official to supervise the review required by subsection (b) of this section, in coordination with the agency’s  Records Officer, Chief Information Officer, and General Counsel.

Blah blah blah. It doesn’t apply to you. But “Head of each agency” includes US State Department. Clinton was the Secretary of State – head of a major agency – at the time when Obama signed the act. That’s somewhat relevant, in retrospect. It didn’t affect Clinton’s behavior, of course.

The reason that the Presidential Records rules were put in place was because of Iran-Contra and the Reagan administration’s convenient but unfortunate loss of a bunch of evidence that would have probably shown that the big guy knew about things he didn’t want to be known to know about. When B Clinton came into office, there was the usual turmoil in the White House (including – apparently – staffers from the previous administration running around breaking the ‘C’ keys off of computer keyboards)  the office staff who make things work are not political appointees, though: so while the new Clinton administration had its own ideas about how to do things, the IT department wanted them to use the PROFS mail system running on a VAX/VMS machine.** To put it mildly, that sucked. So like many presidents and department heads, B Clinton did his own thing. As had Bush before him. As had Reagan before him, and Ford and Nixon, etc.

So, while President Obama was busy promising openness and accountability in government, heads of major agencies continued to do what they wanted to, subject to bureaucratic hand-wringing as documented in the State Department Inspector General’s report and the email discussion with NSA.

Did The State Department’s Security Suck?

Of course it did.

Here’s an example of how badly State Department’s security sucked: The US Secretary of State was doing her official duties using a privately configured email server that was not under professional management. Everyone at State who was swapping emails about State Department business with the boss-triple-prime knew their emails were not going to a State Department server under State Department governance.

Let me talk briefly about IT audit. IT audit is your server logs and transaction logs. It’s the records of who, what, when, how much. Stuff like that. Any system that is under a reasonable governance regime will be keeping logs.

Normally, we’d expect something like:
secstate@us.state.gov -> hil@clintonemail.com so that official communications by the Secretary of State would bounce from the official mail exchanger at US State Department to – wherever. But there’d be logs of that. There’d be logs on that mail exchanger server and there’d be logs in the firewalls and all those logs would add up neatly unless there had been a great deal of messages deleted off some server (in which case the server might show 50,000 emails and the logs 200,000 meaning 150,000 emails had deleted themselves)

It doesn’t appear that Secretary of State Clinton used her @state address at all. In fact it appears that that address was never made or set up – probably by some IT specialist at US State Department who specifically did not want to create that account and put the forwarder in.

Clinton just used her @clintonemail.com address. That means that any email she was transacting as head of the US State Department was neatly bypassing any IT governance at US State Department’s servers. Just the way Hillary wanted it to.

More to the point, it meant that lots of people who sent and received emails with the Secretary of State were sending them and receiving them from @clintonemail.com

If people are sitting around pretending that it’s a shocking revelation that we just now discovered this was going on: it was in every email.

Another serious issue this raises: Clinton claims she deleted some personal emails. Under a reasonable generous interpretation of “personal” – and the kind of fine-parsing politicians enjoy – it was personal email if it was to the @clintonemail.com address, QED. Oops, she was accidentally being US Secretary of State with that account but, well, she never had an @state.us.gov address.

Did The FBI’s Reaction To It Suck?

Of course it did.

The FBI’s job (among other things) is to enforce governance on handling of classified data. It’s not some IT person at State Department’s job to arrest the boss-triple-prime for carrying out agency business with moderate (at best) security instead of using the security mechanisms that were available and were approved for handling classified material.

The FBI didn’t do anything about any of this from 2009 (when the issue was publicized) and waited until there was an election cycle in progress. The FBI is playing politics, in other words. That shouldn’t shock anyone at all. The FBI has been involved in various types of chicanery for various presidents – it depends on whether the director likes the president (i.e.: was installed by the president) or not, etc.

Did Clinton’s Personal Email Server Suck?

Of course it did.

The thing was running Microsoft Exchange 2010 and Outlook Web App, so we know it was running on a Windows box of some flavor. Was it running on the same Windows box that was set up for B Clinton in 2008? Then it might be running Vista or XP. (shudder)

Want to get an idea how good the security of Exchange 2010 is? Outlook Web Access (OWA) is about the same. Reading between the lines, it sounds like the Clinton server was receiving inbound SSL/TLS from the internet into OWA so that (whoever) could read their email through the web interface (or download it) As a security guy, my flesh crawls slightly, but if it were done carefully and were well-managed, it might be OK.

Vulnerabilities in Exchange 2010

Vulnerabilities in Exchange 2010

This is why you want professional IT management on security critical servers: so they don’t get installed, forgotten about, and eventually their software decays to the point where anyone who wants to can walk right into it.

As someone who’s done incident response, security system design, and served as a legal expert in security cases, I would be comfortable saying unequivocally that there was more than a bit of negligence shown. But that it was endemic in agency communications and it has been endemic in presidential communications and it’s inconsistent – if not ridiculous – to go after Clinton. There ought to be heads rolling all over the place, because it appears to me that anyone who swapped an email with Clinton between 2009 and 2015 ought to have known she was doing her email through a private server.

Second point: “@clintonemail.com” is a giveway and was a giveaway since 2009.

What Else Do We Know?

Nothing. I don’t, you don’t and the FBI probably doesn’t, either.

A system with no logs, no firewall logs, no vulnerability assessments, etc – sitting in someone’s basement?*** We have no idea if it was being used as a dropbox by every hacker on the internet, or if it was largely ignored.

When the FBI says “someone might have accessed the server” why, yes, that’s true. But they have no way of knowing anything about that, apparently, and they’re playing politics if they pretend to unless they present evidence. And it’d have to be better evidence than the usual (wave hands) “Chinese hackers!” (FWIW I did a series on how Attribution Is Hard here and here)

One scary possibility is that the FBI was able to get some information from NSA about who/what has been going into and out of the @clintonemail server. Since those communications were being done from mobile devices, their traffic would have almost certainly been vacuumed up with all the rest of the country’s mobile device communications. If the FBI had that bit of knowledge, then the inevitable question would be: “why didn’t you mention this earlier?”

The FBI is a branch of the Department of Justice and its stock in trade is presenting criminal cases and convicting criminals. As such, the FBI cannot plausibly plead that they don’t understand how to make their case with evidence. Nor can they plausibly claim that they don’t know what “trying in the court of public opinion” is.

FBI sucks. They’re playing politics. But they’ve always sucked and they’ve always played politics so it seems unfair to complain now.

My Opinion

I’ve already embedded it in my answer: I think they all suck.

If the State Department was going to have a “BYOD” (Bring Your Own Device) IT security policy, they ought to have formalized it and then decided what they could do to separate things appropriately so there wouldn’t be this kind of problem. Or, they could have written the boss-triple-prime up for mis-handling secret information, which happens all the time to underlings but not to oligarchs.

I’ve said elsewhere: records acts and governance are the citizens’ recognition that their politicians occasionally sneak off to line their own pockets, make illegal deals, or violate the laws that have been passed to rein them in. When you see a politician deliberately and knowingly setting things up so that their actions are unrecorded, that ought to be a red flag. Unless, of course, you favor secret government by oligarchs who feel beholden to nobody and who cheerfully bypass records keeping because, you know, that stuff’s for the little people.

I’ll hold my nose and vote for Clinton, of course. But this affair boils my blood because – as usual – the republicans have gone barking and woofing off after something irrelevant because they can’t complain about the actual problem because their guy did it too. I wish Clinton had just said that, BTW: “I’m just doing what Ronald Reagan did. STFU.” Oh, OK then.


(* The wikipedia page about the affair appears to be technically accurate. So let’s treat that as a resource for facts.)

(** Around this time, I fall into the scene. I was brought in to build an inbound email server for president@whitehouse.gov emails and was chartered by DARPA to research and propose ‘advanced communications security’ for the new administration. It didn’t take anyone at DARPA very long to realize that we proposed to herd chickens, so we just focused on the email server and getting a firewall in place. So I built that – the first whitehouse.gov email server lived next to my desk in Glenwood, MD where I worked, and president@whitehouse.gov’s email went down to Washington via FEDEX on floppies covered with my chicken-scratch. I hope they are in the National Archives somewhere…

I got briefings about “Presidential Records” and the TL;DR form was: “don’t say anything in a meeting where there is a minutes-taker or a schedule, if you need to talk to someone grab them in the hall.”)

(*** In fairness: it may be there are, in fact, exhaustive logs and the system had excellent security. Sadly, that’s probably not the case. Gmail’s security would certainly be vastly better, but gmail keeps everything and Clinton wouldn’t want that.)

 

Comments

  1. AT says

    The Russians have a proven history of being very skilled at owning Exchange email servers. From the Climate Research Unit’s email fiasco ( https://en.wikipedia.org/wiki/Climatic_Research_Unit_email_controversy ) to the latest DNC email madness ( https://www.washingtonpost.com/world/national-security/russian-government-hackers-penetrated-dnc-stole-opposition-research-on-trump/2016/06/14/cf006cb4-316e-11e6-8ff7-7b6c1998b7a0_story.html ). I would put the probability of the Russians having a full backup of ALL of Clinton’s emails at nearly 100%. Based upon the selective leakage in the DNC case, I think that the Russians are going to follow the ‘death by a thousand cuts’ strategy to just bleed Clinton over the next few months, maybe with a spectacular leak about corruption within the Clinton foundation as the grand finale.

    From a legal/policy precedent, the Clinton email fiasco sets a very bad policy precedent for political players in the future. In the current age of computing ubiquity, the levers that we as citizens used to have to keep government officials honest through transparency have all been sheered off at the floor. With the statutes written the way they are (requiring intent to prosecute for mishandling of classified info), with DOJ officials who are political appointee puppets, with Congress’ powers being short circuited and the Judicial Branch becoming a defacto extension of the White House, we are heading in a direction where a tyrant could easily turn the Executive Branch into something very, very ugly. Remember NSA? Executive Branch. DOJ? Executive Branch. CIA? Executive Branch. Pentagon? Executive Branch. All of the power of mass surveillance and projection of force has been concentrated in the Executive Branch with very few controls left to other branches to check them.

    Bottom line for me, I don’t see much difference between Trump and Clinton. Clinton will probably win through her sheer powers of corruption and graft. And, with the state of the Legislative and Judicial Branches of government, there’s a whole lotta badness that she can effect. Nixon may be the last President who ever was/will be held to account for his Chief Executive misdeeds. Who will hold anyone to account in the future? Wouldn’t it be ironic if the Russians played spoiler and took all of the stolen data that they had with the intent of destabilizing the Executive Branch? They’re a sneaky bunch!

  2. says

    AT@#1:
    The Russians have a proven history of being very skilled at owning Exchange email servers.

    It looks to me like anyone who did a vuln scan against the damn thing ought to have been able to walk into it unless the patches were up to date.
    Now, if the FBI posted a vuln scan of the box, that’d be alternately amusing and fascinating. I hope that, a few decades from now, the person who was running that box shows up at a security conference to do a talk about it. For all we know, it was someone really good. Or the lowest bidder.

  3. AT says

    I saw a post on some social media feed a few months ago with a person claiming that they were ‘on the team’ which managed the security of the server in question. As they were pressed about details regarding their security approach, it was obvious that their idea of ‘security’ was installing a default Windows Server 2008 build, change some passwords, install Exchange 2010 and “turn on a bunch of firewalls” to make sure no one got in. I don’t think the Clintons operate on the lowest bidder concept, they probably paid through the nose because some friend of some friend needed the work, and they those people found some dude off the street to do the work.

  4. says

    AT@#3:
    it was obvious that their idea of ‘security’ was installing a default Windows Server 2008 build, change some passwords, install Exchange 2010 and “turn on a bunch of firewalls” to make sure no one got in.

    I hope they turned on the firewall in the FIOS home router. That’s a super good firewall, yup, yup, for sure.

  5. Pierce R. Butler says

    Clinton not only followed Reagan’s precedent, she used his excuse – of being too damn ignorant to know what was going on in her own office for year after year.

    No wonder the Dems & Repubs look so much alike any more…

  6. colnago80 says

    As I have said previously here and on other blogs, what the Congressional committees should be investigating is the state of computer security at government agencies. That’s the real scandal here.

    It’s hugely unlikely that Clinton didn’t already know this stuff because she was part of the White House for the entire B Clinton administration and would surely have encountered security before. Some of the redacted emails from the FBI investigation apparently show that State Department staff and NSA had some energetic discussions around the topic of Clinton’s email and Blackberry.

    First point: Clinton is not ignorant about this stuff.

    Michael Heath makes a contrary claim in a comment on Ed. Brayton’s blog that, in fact, both Clintons are ignorant of technology.

    I agree it was bad behavior. But I’ve long observed that both Clintons are ignorant of technology. They try and talk a good game about it, but it comes across as “2 Corinthians”.

    I don’t know what he bases this opinion on but he’s one of the most reliable commentors over there (despite the fact that he doesn’t like me).

    http://www.patheos.com/blogs/dispatches/2016/07/06/no-clintons-emails-is-nothing-like-petraeus/

  7. colnago80 says

    Re #8

    As war criminals go, the IAF looks pretty pathetic compared to the folks in Syria and Iraq.

  8. says

    Colnago80@#:
    Michael Heath makes a contrary claim in a comment on Ed. Brayton’s blog that, in fact, both Clintons are ignorant of technology.

    I am not responsible for what Michael Heath says.

    What does “ignorant of technology” mean? Doesn’t know how to use a Blackberry, or doesn’t understand how a Blackberry works? Perhaps Clinton is ignorant of technology but if that’s the case she has people to be not ignorant for her and a good manager has to listen to their experts. In this case, Clinton appears to have not gotten the answer she wanted from one set of experts (the NSA and State Department’s IT) and gone and looked for another who’d tell her what she wanted to hear. I’ve worked for executives that “find the ‘yes'” myself and anyone who’s worked in a dysfunctional company or agency knows how that game’s played. Here’s the thing: you can’t be ignorant to play that game, either.

  9. says

    Colnago80@#9:
    That’s your one and only yellow flag warning. If you start turning threads into rants about your favorite tribe, and trumpeting your ethical ignorance with false moral equivalencies, I’ll ban you.

  10. lanir says

    Technical ignorance is not a barrier to making an effective and informed decision. I end up explaining computers and sometimes security issues to technically ignorant people on occasion. Unless they’re not listening, people usually understand the gist of what I’m telling them. All it usually takes is the right amount of focus and maybe a bit of analogy. Because hitting them with every detail at once is confusing and analogies allow them to leverage their understanding of some other situation and apply it to the current issue.

    Actually the hardest people to explain things to are the ones that know just enough to think they understand most of it already. But that happens with any topic. It sometimes feels unreasonable but it makes a lot of logical sense. If they think they have the right information already but what you say doesn’t fit the framework they already have, you must first prove that their current information is lacking somehow. It only becomes a problem when they get attached to their views.

    Sometimes in when people get overly attached to their views you also get into issues over who is an expert on a subject but that gets into a big mess. The benefit to understanding the logic necessary to navigate this is it’s useful all over the place. The bad news is “getting it” doesn’t seem to be very common.

  11. says

    lanir@#13:
    Technical ignorance is not a barrier to making an effective and informed decision. I end up explaining computers and sometimes security issues to technically ignorant people on occasion. Unless they’re not listening, people usually understand the gist of what I’m telling them.

    Yup. I do that a lot, too. And my experience matches yours: if someone isn’t “getting it” it’s usually because they don’t want to get it. We’ve probably both seen too many occasions when an executive will “look for the ‘yes'” by asking around until someone says what they want to hear.

Leave a Reply