Regularized by cars

The most American activity is driving. The high-volume production car? Invented in America. Coast-to-coast highway systems? America. The car-centric city, car-centric suburb, desert road trip culture? USA! USA! USA!

Driving is liberating. The freedom of the road. Adventure in a pedal. The thrill of adventure, the surprise of speed.

It’s completely heretical to claim that cars are a constraint that adds inertia to a society that is otherwise incomprehensibly adventurous and far more adaptable than any before, right?


Let’s review what it’s like to drive a car.

  • You expect to be carried.
  • Your mind and arms and neck are working, but your legs aren’t.
  • The car becomes your body; as it strains, you feel yourself straining. As the RPM goes up, so does your heartbeat.
  • If someone hits your car, it’s a personal insult. A dented door is a bruise; a smashed fender is a chipped tooth. A spiderwebbed windshield feels just like a crack in glasses.
  • You get used to opening your wallet: gas, tolls, insurance, tires, washer fluid, oil changes, electricity, washes, new windshields, batteries.

If all this is news to you, maybe you aren’t American.


In the world of neural networks, an obstacle that prevented early multi-layer perceptrons and recurrent neural nets from being stacked many layers deep was the exploding or vanishing gradients problem.

Each network layer directly interacted with the layers beneath (or above), and every layer learned simultaneously. Small movements in one layer could short-circuit to the next during training. Complex structures or behaviors couldn’t be modeled when the network so quickly diverged with weight values going to either +/- infinity or zero.

Ultimately, the neural network would lose its ability to produce meaningful output, whether that was classification of input data or generation of outputs similar to a training sequence.

But a solution was found in the last ~20 years: normalization. The output distribution has to look like roughly the same shape in N-dimensional space to propagate forward, and if it isn’t, we squeeze and stretch it until it fits in that box.

As an oversimplified example, imagine we just divide the output signal by the square root of the square of each of its elements. Now the output is always the same length: 1. In a flash, learning layers could be stacked not one or two deep, or maybe sometimes three with hand-holding, but dozens or hundreds deep without fear of the network blowing up.

Keep the direction, but crush any colorful amplitude variation to a flat grey.


One could imagine the places we go as places where we learn, where we change ourselves and others. The imaginary ideal:

  • When we go to work, we interact with colleagues and accomplish goals.
  • At home, we spend time with family and develop our personal lives.
  • Everywhere else, we meet new people (or old friends) and explore.

But exploration has risks of the unknown. Reading journals you picked up at the store, you might stumble into an idea that could put your current employer out of business. Just as long as you don’t forget it.


A lot of words have been spilled about how violent video games are desensitizing our youth to interpersonal and military violence. But excluding bizarre fantasies, nobody has died from injuries sustained in a video game. But not much has been said about the inherent violence and danger of roads.

If desensitization to fear and danger lets you experience more of it at a lower emotional cost, and benefit more from increased mobility, it seems obvious that driving regularly would impact the way you choose to live and work.


To get to your next destination in America, you’ll face a bouquet of risks that would be incomprehensible for an individual from the 18th century:

  • Dodge speeders approaching on your position at 25 times walking speed
  • Swerve to avoid potholes and roadkill
  • Keep your tank or battery full enough to get where you’re going
  • Keep your tires inflated and don’t let them blow up
  • Play a miniature game of chance with real money involving HOV lanes
  • Accept the roughly 1 in 10,000 risk of death per year
  • Accept the risk of witnessing anything that can happen on the road

Fail to juggle all of the above tasks and you’ll be either bankrupted, sent to jail, lose your family, wipe out your economic livelihood, or even die.

Those positive constraints are just half of the story. Consider the things you can’t do while driving:

  • Resting or sleeping
  • Focusing your full attention on someone or something
  • Birdwatching
  • Reading sheet music
  • Creating things with your hands

Does the American transit system compel you to play an unstoppable high-stakes game of life and death if you want to do anything outside of your own home?

It seems like it does.

  • Every driver has to pay around the same attention to the road.
  • You are transported into a unique physical space where you have to balance slow relative motion and high absolute motion.
  • You risk everything at every turn, and unlike an online game, the consequence for losing isn’t just a bombardment of verbal insults.
  • Everything you do must be slow, with clear intent demonstrated to everyone it might affect.
  • Ingenuity and innovation don’t exist. The best highway driver is predictable and unwavering.

What is lost when each fragment of your life is walled off from the other by miles of concrete and metal?


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *