Tech

The man who invented the self-driving car (in 1986)

HOFOLDING, Germany — The other drivers wouldnt have noticed anything unusual as the two sleek limousines with German license plates joined the traffic on Frances Autoroute 1.

But what they were witnessing — on that sunny, fall day in 1994 — was something many of them would have dismissed as just plain crazy.

It had taken a few phone calls from the German car lobby to get the French authorities to give the go-ahead. But here they were: two gray Mercedes 500 SELs, accelerating up to 130 kilometers per hour, changing lanes and reacting to other cars — autonomously, with an onboard computer system controlling the steering wheel, the gas pedal and the brakes.

Decades before Google, Tesla and Uber got into the self-driving car business, a team of German engineers led by a scientist named Ernst Dickmanns had developed a car that could navigate French commuter traffic on its own.

The story of Dickmanns invention, and how it came to be all but forgotten, is a neat illustration how technology sometimes progresses: not in small steady steps, but in booms and busts, in unlikely advances and inevitable retreats —“one step forward and three steps back,” as one AI researcher put it.

Ernst Dickmanns, the German scientist who tested self-driving cars on European streets in the 1980s and 1990 | Janosch Delcker for POLITICO

Its also a warning of sorts, about the expectations we place on artificial intelligence and the limits of some of the data-driven approaches being used today.

“Ive stopped giving general advice to other researchers,” said Dickmanns, now 82 years old. “Only this much: One should never completely lose sight of approaches that were once very successful.”

From the skies to the street

Before becoming the man “who actually invented self-driving cars”, as Berkeley computer scientist Jitendra Malik put it, Dickmanns spent the first decade of his professional life analyzing the trajectories space ships take when they reenter the Earths atmosphere.

Trained as an aerospace engineer, he quickly rose through the ranks of West Germanys ambitious aerospace community so that in 1975, still under 40, he secured a position at a new research university of Germanys armed forces.

The three autonomous road vehicles at the PROMETHEUS demonstration in Paris, October 1994. From left to right: UniBwM VaMP, Daimler VITA-2, Daimler VITA-1 | Photo by Reinhold Behringer

By this point, he had already started mulling what would soon become his life mission: teaching vehicles how to see. The place to start, Dickmanns became increasingly convinced, was not spaceships but cars. Within a few years, he had bought a Mercedes van, installed it with computers, cameras and sensors, and began running tests on the university premises in 1986.

“The colleagues at the university said, well, hes an oddball, but hes got a track record [of achievements in aerospace technology,] so lets just let him do it,” Dickmanns said during an interview at his family house, located steps from an onion-domed church in Hofolding, a small town outside of Munich.

In 1986, Dickmanns van became the first vehicle to drive autonomously — on the skidpan at his university. The next year, he sent it down an empty section of a yet-to-be-opened Bavarian autobahn at speeds approaching 90 kilometers per hour. Soon afterward, Dickmanns was approached by the German carmaker Daimler. Together, they secured funding from a massive pan-European project, and in the early 1990s, the company came up with an idea that first seemed “absurd” to Dickmanns.

“Cant you equip one of our large passenger cars for the final demonstration of the project in Paris in October [of 1994], and then drive on the three-lane motorway in public traffic?” he remembered officials asking.

He had to take a deep breath, “but then I told them that with my team, and the methods were using, I think were capable of doing that.”

Daimler boosted the projects funding. Car lobbyists ironed out doubts inside the French government. And in October 1994, Dickmanns team picked up a group of high-ranking guests from Charles de Gaulle airport, drove them to the nearby motorway and switched the two cars into self-driving mode.

“Sometimes, we would take our hands off the wheel” — Reinhold Behringer, one of the engineers who sat in the drivers seat during the demonstration

An engineer remained in the front seat of each car — with his hands on the steering wheel in case something went wrong — but the cars were doing the driving.

“Sometimes, we would take our hands off the wheel,” said Reinhold Behringer, one of the engineers who sat in the drivers seat during the demonstration, with excitement still in his voice 24 years later.

Newspapers ran frontpage stories about the demonstration, he remembered. And a year later, Dickmanns team took a re-engineered car on an even longer trip, traveling for more than 1,700 kilometers on the autobahn from Bavaria to Denmark, reaching speeds of more than 175 kilometers per hour.

Not long afterward, the project was over. The technology Dickmanns was using hit its limits. Daimler lost interest in funding the basic research needed to move it forward. Before long, Dickmanns pioneering effort was all but forgotten.

Summer child

The history of artificial intelligence is a history of buzzy springs followed by what researchers call “AI winters,” when the attention and funding fades away.

Dickmanns work on autonomous driving began during the first winter and ended after a second one hit the field.

Research on AI — efforts to make machines do tasks that would otherwise require human thinking — started in the late 1950s. From its early days, the field was characterized by hype, leading some ambitious researchers like economist Herbert Simon to predict in the 1960s that machines would “be capable within 20 years of doing any work a man can do.”

Spurred by such promises, funding exploded — but technology failed to deliver and the bubble burst in the mid-1970s. Money dwindled and AI research was consigned to the backroom labs.

Inside of the UniBwM autonomous experimental vehicle VaMP, at the rear bench where the computing system was installed for easy access and monitoring | Photo by Reinhold Behringer

This first AI winter was one of the reasons Dickmanns kept his work on machine vision largely to himself in the early years. He was aware, he said, that “people would have said that guy has a screw loose somewhere.”

By the time he sent his self-driving van down an empty German autobahn in the mid-1980s, another AI spring had arrived. His proof of concept generated enough interest to hire a team that would eventually grow to 20 people ahead of the 1994 Paris demonstration.

Then came another winter, in the early 1990s, and Dickmanns momentum was lost.

“It was an interesting concept,” said Behringer, the engineer who sat behind the wheel in Paris. “But for many it was still way too futuristic.”

Teaching a car to see

Technologists say there two types of inventions: Those like the light bulb, which have been in use and continuously improved ever since they were first invented. And those like supersonic planes — remember the Concorde? — which embody revolutionary technological process but are too advanced to survive, at least at the time of their invention.

Dickmanns self-driving cars belong to the second category.

When he started developing them in the early 1980s, computers needed up to 10 minutes to analyze an image. To drive autonomously, a car needs to react to its surroundings, and to do that, Dickmanns calculated that computers would need to analyze at least 10 images per second.

Facing what looked like an insurmountable hurdle, he drew inspiration from human anatomy. Cars, he decided, should be programmed to see streets like humans perceive their surroundings.

Inside the UniBwM experimental vehicle VaMP on a public motorway in Denmark on 11 November 1995 | Photo by Reinhold Behringer

The human eye is only able to see a small spot in the center of its visual field in high resolution. Similarly, Dickmanns thought, a car should focus only on whats relevant for driving, such as road markings. This slashed the amount of information the onboard computers had to process.

He also found other computational shortcuts — a significant amount of computing time was freed up when Dickmanns realized he didnt need to spend valuable processing power saving each image. He also programmed the car to learn from its mistakes, gradually improving its understanding of its surrounding.

Altogether, it was enough to keep the car on the road — barely.

Driving on a highway, it turns out, is one of the easier tasks a self-driving car can perform. The conditions are well-defined: Traffic flows predictably, in one direction. Lanes are clearly marked.

And even then, the demonstration didnt go perfectly. “It was a test,” said Behringer. “When, for instance, there was a car in front of us that covered up the road markings, and on the other side, the markings were washed away, then the lane identification feature had a problem.”

America calling

After the second AI winter set in, and the buzz surrounding the Paris demonstration faded away, Daimler told Dickmanns it “wanted to have a product for the market as soon as possible,” he recalled. The carmaker had lost interest in his expensive fundamental research, which was unlikely to produce any real-life applications within the next couple of years.

“In hindsight, it was probably a mistake that those projects werent immediately continued,” Jürgen Schmidhuber, co-director of the Dalle Molle Institute for Artificial Intelligence Research in Lugano, Switzerland, said. “Otherwise there would be no question about who would be leading in the field today.”

German companies continue to hold most — almost half of all — patents in self-driving technology, but newer players, among them U.S. tech giants like Alphabets Waymo, have been catching up. Experts describe the current race for leadership in autonomous driving technology as neck-and-neck.

“There is a deep lack of awareness of what has been done in the past, especially among machine-learning scientists” — Longtime AI researcher

“Its possible that [Germany] threw away its clear vanguard role because research wasnt consistently continued at the time,” Schmidhuber said. He added that carmakers might have shied away from self-driving technology because it seemed to be in opposition to their marketing, which promoted the idea of a driver in charge of steering a car.

In the late 1990s, Dickmanns turned overseas and signed a four-year-contract with the United States Army Research Lab.

The cooperation led to another generation of self-driving cars, which were able to navigate more complicated surfaces; its results — published around the time Dickmanns retired — caught the eye of Darpa, the Pentagons emerging technologies division. It inspired the agency to launch a series of “challenges,” starting in 2004, tasking inventors with sending self-driving cars racing through spectacular territory.

Those challenges, promoted by massive marketing campaigns, were the first time a broad public heard about autonomous driving. They made German-born computer scientist Sebastian Thrun — who won the challenge in 2005 as a Stanford University professor and later founded Googles self-driving team — a celebrity in the AI community.

The UniBwM experimental vehicle VaMP during a stop | Photo by Reinhold Behringer

Meanwhile, Ernst Dickmanns pioneering work fell into oblivion.

When in 2011, 17 years after Dickmanns Paris demonstration, the New York Times ran a front-page story about Thruns efforts to build a self-driving car, it had to run a correction afterward, making clear that “though Mr. Thrun developed a driverless car, he was not the first to do so.”

“There is a deep lack of awareness of what has been done in the past, especially among machine-learning scientists,” said one longtime AI researcher, who asked to remain anonymous.

He added that he regularly interviews high-ranking candidates who dismiss five-year-old papers as “outdated” or simply dont know about research done in previous decades.

Winter is coming?

In 2018 — as AI undergoes yet another round of hype — could a new winter be looming? Some think thats a distinct possibility.

Much recent research into AI has been into so-called “deep learning,” in which algorithms “learn” by recognizing patterns. Its underlying principle — finding correlations in complex data — works great for most applications but proves to be a dead end in some cases. And since deep learning is driven by data, its algorithms are always just as good as the data theyre being fed.

Filip Piękniewski — a San Diego-based computer scientist and the author of an essay titled “The AI Winter is well on its way” — said that much of the funding pouring into AI, particularly in the context of self-driving cars and robotics, is based on unrealistic expectations raised about what deep learning is capable of doing.

Photo by Ernst Dickmanns

“This is the place where expectations collide with reality,” Piękniewski said. “And a lot of people will be annoyed that they invested so much money, and the expectations dont materialize.”

Virginia Dignum, a professor at the University of Delft, agreed that if AI researchers keep focusing primarily on deep learning, “at some point, people will be disappointed.” The field, she said, has to look beyond it and invest in other approaches that depend on less data, or models based on causation rather than the correlation deep learning relies on.

But nonetheless — echoing widespread opinion among researchers and analysts — Dignum stressed she does not believe another “AI winter” is coming anytime soon. Unlike previous booms, todays developers are turning cutting-edge AI into commercial real-life applications, thanks to recent technological advances starting in the early 2010s, particularly in computing power and data storing.

Pathfinder

This makes the situation different from previous generations, which were often said to be doing “blue skies research” — scientists like Ernst Dickmanns, who said that he figured at the time of his experiments that it would take decades more until autonomous cars would ever become an everyday reality.

Indeed, Dickmanns, seated in his winter garden, said he still believes truly autonomous vehicles are still a decade or two away.

The self-driving vehicles currently being tested use a different, less computationally expensive process that needs less processing power in the on-board computer. They make up for the difference by using maps, GPS positioning and databases of previously observed objects.

“Im glad I could be one of the pioneers. But if I could start anew today, with the technology thats available, this would be a whole different story” — Ernst Dickmanns

Rather than truly “seeing,” Dickmanns said they rely on what he calls “confirmation vision.” That means they might work well on roads and areas that have been extensively mapped but fail when it comes to less controlled environments.

The approach he pioneered — which he calls “pathfinder vision” and which is still being pursued at a couple of research institutions — would allow cars to operate anywhere. “At some point, people will realize that after a storm, after an earthquake, or significantly more often in a military context when you get into new surroundings, the [current approach] wont work,” he said.

Someday, he predicts, the industry will realize the limitations of its approach, and his work will see a resurgence.

“Im glad I could be one of the pioneers,” he added, “But if I could start anew today, with the technology thats available, this would be a whole different story.”

Read this next: Im on Putins hit list but Im not the real victim

Original Article