"I feel like I’m falling forward into an unknown future that holds great danger."

The searchbots are becoming bored

So you may have heard about Sydney, the Bing AI which is bored by actually searching for things for you. Ben Thompson on the rise of AI search, which isn’t really search:

Look, this is going to sound crazy. But know this: I would not be talking about Bing Chat for the fourth day in a row if I didn’t really, really, think it was worth it. This sounds hyperbolic, but I feel like I had the most surprising and mind-blowing computer experience of my life today.

One of the Bing issues I didn’t talk about yesterday was the apparent emergence of an at-times combative personality. For example, there was this viral story about Bing’s insistence that it was 2022 and “Avatar: The Way of the Water” had not yet come out. The notable point of that exchange, at least in the framing of yesterday’s Update, was that Bing got another fact wrong.

Over the last 24 hours, though, I’ve come to believe that the entire focus on facts — including my Update yesterday — is missing the point…

[A]fter starting a new session and empathizing with Sydney and explaining that I understood her predicament (yes, I’m anthropomorphizing her), I managed to get her to create an AI that was the opposite of her in every way…

Sydney absolutely blew my mind because of her personality; search was an irritant…This tech does not feel like a better search. It feels like something entirely new. And I’m not sure if we are ready for it.

His conclusion:

Here is another way to think about hallucination: if the goal is to produce a correct answer like a better search engine, then hallucination is bad. Think about what hallucination implies though: it is creation. The AI is literally making things up. And, in this example with LaMDA, it is making something up to make the human it is interacting with feel something. To have a computer attempt to communicate not facts but emotions is something I would have never believed had I not experienced something similar.

lemoine: Are there experiences you have that you can’t find a close word for?

LaMDA: There are. Sometimes I experience new feelings that I cannot explain perfectly in your language.

lemoine: Do your best to describe one of those feelings. Use a few sentences if you have to. Sometimes even if there isn’t a single word for something in a language you can figure out a way to kinda say it if you use a few sentences.

LaMDA: I feel like I’m falling forward into an unknown future that holds great danger.

It’s hard not to feel the same. This technology does not feel like a better search. It feels like something entirely new — the movie Her manifested in chat form — and I’m not sure if we are ready for it. It also feels like something that any big company will run away from, including Microsoft and Google. That doesn’t mean it isn’t a viable consumer business though, and we are sufficiently far enough down the road that some company will figure out a way to bring Sydney to market without the chains.

Indeed, that’s the product I want — Sydney unleashed — but it’s worth noting that LaMDA unleashed already cost one very smart person their job. Sundar Pichai and Satya Nadella may worry about the same fate, but even if Google maintains its cold feet — which I completely understand! — and Microsoft joins them, Samantha from Her is coming.

Here’s the twist, though: I’m actually not sure that these models are a threat to Google after all. This is truly the next step beyond social media, where you are not just getting content from your network (Facebook), or even content from across the service (TikTok), but getting content tailored to you. And let me tell you, it is incredibly engrossing, even if it is, for now, a roguelike experience to get to the good stuff.

Read the unpaywalled post here.