Ibrahim Diallo was fired from his job by a machine algorithm that, once set in motion, no one in his company seemed to be able to override however high they were in the administrative structure.
The story of Mr Diallo’s sacking by machine began when his entry pass to the Los Angeles skyscraper where his office was based failed to work, forcing him to rely on the security guard to allow him entry.
…
Then he noticed that he was logged out of his work system and a colleague told Mr Diallo that the word “Inactive” was listed alongside his name.
His day got worse. After lunch – and a 10-minute wait for a co-worker to let him back into his office – he was told by his recruiter that she had received an email saying his contract was terminated. She promised to sort out the problem.
The next day he had been locked out of every single system “except my Linux machine” and then, after lunch, two people appeared at his desk. Mr Diallo was told that an email had been received telling them to escort him from the building.
His boss was confused but helpless as Mr Diallo recalls: “I was fired. There was nothing my manager could do about it. There was nothing the director could do about it. They stood powerless as I packed my stuff and left the building.”
…“All the necessary orders are sent automatically and each order completion triggers another order. For example, when the order for disabling my key card is sent, there is no way of it to be re-enabled.
“Once it is disabled, an email is sent to security about recently dismissed employees. Scanning the key card is a red flag. The order to disable my Windows account is also sent. There is also one for my Jira account. And on and on.”
Diallo tells the whole story here. The process had begun when his manager got laid off and left without setting in motion the renewal of Diallo’s contract and the transition to a new computer system resulted in him being ‘terminated’ by default.
Once the order for employee termination is put in, the system takes over. All the necessary orders are sent automatically and each order completion triggers another order. For example, when the order for disabling my key card is sent, there is no way of it to be re-enabled. Once it is disabled, an email is sent to security about recently dismissed employees. Scanning the key card is a red flag. The order to disable my Windows account is also sent. There is also one for my JIRA account. And on and on. There is no way to stop the multi-day long process. I had to be rehired as a new employee. Meaning I had to fill up paperwork, set up direct deposit, wait for Fedex to ship a new key card.
He was out of work for three weeks while the issue was investigated and resolved. His story is disturbing because he and his superiors kept getting computer generated emails ordering them through the processes of his termination even though no human was involved in making the decisions.
Our machine overlords have arrived.
Well. Nothing is terrifying about this story. Nothing at all.
In what kind of work environment is it normal to just do whatever an email tells you to do, irrespective of its sender? Or were the emails disguised to look like they came from humans? I have so many questions!
A process like that that has no provision for interruption or even cancellation is idiotic. You may not want to easily cancel the process to avoid unauthorized actions but not seeing that a process might had been activated unintentionally is very shortsighted and shows very poor design and management.
It’s a bit like Kafka’s The Castle, albeit moving from the inside out.
Mano reported the story here concisely and correctly, but the positioning is overhyped. The key fact is that no machine “decided” on firing our protagonist.
User provisioning and deprovisioning workflows can get complex very quickly. What we have here is a bad workflow implementation, as others mentioned. “Deprovisioning by expiration” is sometimes necessary but you’d hope there are touch points for intervention.
This is not even unique to computing. You can find the same issue with conveyor belt systems. While I work in IT now and understood what was going on here, before I was reliably getting IT jobs I worked at a warehouse that had a conveyor belt and scanners everywhere. It was a great system for the most part and made the casual, everyday workflow much less arduous. But the moment something slipped past without a clear scan, you’d have to stop the whole works and run all over the building to find the tray that didn’t scan and lug it back to the front.
Uninterruptible flows with no off-ramps for exception handling are generally awful design. But it’s also difficult to plan well and put off ramps in the right places too. That tends to only happen if your flow is very, very simple.
@Michael Sternberg, No. 5
I would agree if there had been a quick and easy way for humans to intervene in the process.
What if the situation were somehow transported to a hospital where patients “were locked out” when their medical bills crossed a threshold?
Jeff Hess
Have Coffee Will Write
I am reminded of Gordon Dickson’s short story “Computers Don’t Argue”. But this, apparently, was real.
“A process like that that has no provision for interruption or even cancellation is idiotic.”
A process like that that’s set in motion because someone fails to do something (ie is passive) iso actually doing something (ie being active) is asking for problems like this one. MS hasn’t stressed it nearly enough, so here it s:
“his manager … left without setting in motion the renewal “\
Keyword: without.