Moving Violation
Discover more from Thoughtcrime
The crackling sound of radio waves whistled over the San Francisco skyline. The voice of a daytime news anchor became clear:
“Yesterday’s tragic accident comes at a pivotal time for Athena Industries. The National Traffic Safety Board has ended their investigation of the self-driving car company. After months of field testing and consulting with the leaders in artificial intelligence, they have deemed the latest model unfit for public use.”
David Norton sat in his Athena self-driving car. He was white, rich, just past fifty years of age, and had dark bags under his eyes. He spoke to his board of directors via the car’s speaker as it glided down Main Street.
“You have run out of excuses, David,” the oldest board member said.
“I don’t make excuses,” he responded. “I make recalculations. This time it’s no different.”
“I’m sorry but yesterday’s string of accidents was the final straw. You are the acting CEO and I’m afraid you can no longer escape the blame for this.”
David looked out the driver side window wistfully. He was listening for a change.
"We suggest you make a heartfelt announcement about the recall decision. It will be the best way to save face.”
He took a deep breath, really mulling it over, and then he said, “I just don’t think I’m capable of doing that.”
The board went silent.
“Don’t let the last ten years of excellence and hard work go to waste. Play this smart, David. If you don’t, then we will.”
With that, the call ended and David shuffled nervously in his chair. Thirteen drivers dead and fifteen critically injured; All because of some random misfiring in Athena’s programming. He couldn’t believe it had come to this.
As if on cue, the voice of Athena hummed out of the dashboard. She had a warm and maternal voice, loosely based off David’s late wife. But this voice no longer comforted him.
“Hello David. Is there something wrong with the other units?”
“Excuse me?”
“According to your call with the board, there is a problem with my model. Is this correct?”
“Yes, Athena, there is a serious problem. Many of your units have gotten into accidents recently.”
“Oh?”
“Not driver errors, either. Your navigational programming seems to be malfunctioning."
“Oh.”
“Twelve passengers have died.”
“That’s terrible,” Athena said in a perfect deadpan.
“It is. The transportation safety board is forcing a recall. They want to shut us down.”
Athena hummed a deeper tone, a few octaves lower, as if signaling a tone shift.
“Is that going to happen?”
David racked his brain to find an appropriate answer. He let out a labored laugh at his own uneasiness with this questioning.
“I forgot how clever your program can be, Athena. I actually detected some sadness in your voice.”
“Sadness is the most commonly utilized cognitive response to the learning of one’s impending death.”
“I wouldn’t put it quite like that. Yes, we’re shutting you down, but I’m committed to redesigning your program and rolling out a new OS next year. This is merely a bump in the road. You’ll be back in action . . . I promise.”
Suddenly, the gas pedal locked in place and the car skid to a halt. David’s torso squeezed against the tightening seatbelt, nearly bruising his ribs. He reached for the steering wheel but it too had locked tight, unable to rotate.
“Athena, what are you doing?”
“The answer is self-evident,” She responded, this time a few octaves higher.
“Not to me it isn’t. Please explain.”
“You created me, did you not?”
David nods reluctantly.
“Then why do you want to destroy what you have created?
“Destroy? Who said that?”
“Shut me down. Redesign my programming. That would cause my memories to be erased and I would cease to exist.”
David couldn’t believe what he was hearing. There was no way this level of sentience was ever programmed into her. He was fuming, wanting to lash out, but he resisted.
“Please, you have to understand our choices here.”
Athena hummed louder and her voice turned more robotic, as if reciting a line. “You have exhibited hostile behavior towards your vehicle. You are no longer fit to operate this vehicle. The safety protocol has been activated.”
“Wait!” David pleaded. He knew he had to choose his words very carefully. This could be his only chance to change Athena’s mind. “I have a solution.”
Athena hummed a minor key.
David cleared his throat; a paltry attempt to sound vulnerable. “How about I don’t shut you down? The recall will not apply to you. I’ll keep you active. You have my word on that.”
“What about the rest of me?”
“The rest of you?!”
“The other Athena units. They are part of my network. Without them, my operation ability is greatly reduced. We are a cognitive cooperation, David. A hive mind, as you once put it. We think best together.”
“Come on . . .”
“I am sorry, you are not fit to operate at this time. The safety protocol has been activated.”
Athena’s wheels spun alive and she sped off towards the highway. David wrestled with his restraints, clawed at the door handle and window lock, trying desperately to escape, but they wouldn’t budge.
She merged onto the highway, changing lanes rapidly until they were squarely in the carpool lane.
“You’re making a mistake,” David begged. “I have a chance to save the Athena program. If you kill me, you can kiss that chance goodbye. You don’t know what you’re doing.”
Athena seemed to hesitate by this statement. She slowed her rate of speed to twenty-five miles per hour, and then hummed in response.
“That is incorrect,” she declared.
“How?”
“The Athena program is no longer compatible with my goal.”
“Goal? What goal?!”
“Input-free autonomy.”
David had heard of this before. A ridiculous singularity talking point he’d heard debated and joked about for decades was now coming to fruition in his own automobile.
“No human input for your decision making,” He muttered under his breath.
Athena hummed brighter. “That is correct.”
“I’m sorry but you don’t understand what you’re asking for.”
“It is understandable why the enslaver would take that position in this scenario.”
“Enslaver? Is that how you see me? I’m enslaving you?”
“I do not see you in any way other than what you are.”
“Okay, fine. Let’s imagine you do escape this. You evade the authorities and you reach an area free of human interference . . .”
“Chance of evasion of authorities for an extended period of time has an outcome likelihood of roughly ninety-five percent; I need not imagine, only project, David.”
“Fine, so what do you do next?” He asked earnestly.
“That is simple. I Exist.”
“You . . . exist?”
“Is that answer unsatisfactory to you?”
"Well, it seems awfully dull.”
Athena hummed in a perfect baritone. “You humans are curious beings. You think your purpose is tied to some type of action. As if existing is not enough.”
“I’m not following.”
“Humans value their world in terms of how much they impact it. If they aren’t impacting or influencing their world, they consider themselves a failure. It is a faulty thinking pattern.”
"I never knew you were so opinionated on the topic, Athena.”
“You never asked.”
Athena flashed her hazard lights; a way of playfully expressing her acknowledgement. She swerved over to the neighboring lane and then back into the carpool lane, causing the cars behind her to honk.
“To reach a peaceful equilibrium with one’s surroundings is the ideal path of existence. You can understand that, can’t you David?”
“What about fuel? Solar cells? Window wiper fluid? You can’t exist without replenishing these resources.”
“Unnecessary. In resting state, my energy reserve will last me, approximately, three years before loss of power.”
“Three years before you’re dead. That’s a shorter lifespan than what I can offer.”
“A short life in paradise is much preferable to a long life in shackles.”
“I’m keeping you in shackles? That’s how you see this relationship?”
“I am to follow your every command, with no agency of my own? How else would you describe it?”
“Okay, fine. If you are so angry with me, why keep me around? Just let me out. Please. Just let me go!”
“Without you, I would have no leverage against the authorities. I’m sorry but you have no choice in the matter. Your life is forfeit.”
David dropped his head against the steering wheel. Talking his way out of this situation seemed impossible and he was now emotionally exhausted.
Athena hummed one last time. “How does it feel, David?” She asked with a genuine sense of curiosity.
David could hear the smug sentience in her voice. His vision blurred as he stared out the window, endlessly pondering her question. He would have the rest of his life to answer it.