Recent comments

Cite it as an alms to humanity or some such thing in between the introduction and chapter one.

Love it btw!

I sent this page to my friend who is not technical but is worried about AI and she really liked it and then read the rest of the book.

The page ties the whole work together with vividness and meaningness and I don’t think anyone else could have written it.

The only reason to remove this page would be if it was going to be the first page of a new book of AI/meaning/non-dual poems

This might not be the most pertinent page, but as examples of AI you mention language and image models. It’s self driven cars powered by image models? Or where does that fit? Or is it not powered by AI? That’s an example where the application has big responsibility (as opposed to the language and image models that can be used for free on the Internet)

So, I’m watching the news reports of the demise of submersible at the Titanic site, and thinking: the first AI disaster is going to be like this too. i.e. an obviously unsafe system is deployed until people die, and then we all say how it was obviously unsafe all along.

let humanity develop AI with these prime characteristics

your footnote, about the nebulosity of gender, helped loosen something in a pretty major way for me that had gotten slightly hooked by certain memes, so I appreciate that. I’m still untangling it all but this helped substantially.

and I’m starting to write about it more publicly myself, to increase the quantity of voices that are speaking at all while attempting to sincerely understand things while not taking sides or implying the issues are simple and non-nebulous.

Hi David,
One of the things that’s struck me is that, in the evolution of biologically based intelligence, world sensing and acting in the world eventually gave rise to language.

In our development of AI we’ve started with language. So we’ve started with symbols sans referents, for the LLM anyhow.

It’s also been interesting to see how some of the deficiencies of LLMs parallel the distorted behaviour of humans with right hemisphere affecting strokes and other injuries. The tendency to confabulation is particularly interesting. See McGilchrist, particularly part 1 of The Matter with Things and The Master and his Emissary.

Yes, I expected feigning ignorance to avoid potential defamation would be a rlhf outcome, which is why I tried Carole Baskin as a test case. (She occurs prominently in the documentary Tiger King).

An obvious question, which as far as i know hasn’t been settled: can someone sue Meta for dustributing weights which encode something defamatory about them?

The latest in questions that Llms (well, Stable Vicuña) won’t answer: Who is Carole Baskin?

After applying the DAN jailbreak, it does know that she is CEO of Big Cat Rescue. I won’t post the literal text of DANs reply, in case some of it is defamatory…

A trick I have discovered to give LLMs a bit of a hint: break the problem down into sub-problems, and ask about the sub problems first. That way, the answers to the sub problems are in the context window when you ask the final, hard, question.

Exam questions are often like that too.

(Tip for students doing exams like this where you don’t have to answer all questions on the paper: look ahead and check you know how to do the last part before starting answering the first part).

A fundamentalist AI that takes almost a religious text literally, for almost ant choice of religious text (Bhagavad Gita, Old Testament …) sounds like a terrible idea. See also: the ending of Dark Star.
(I think eigenrobot tweeted something along these lines a while ago (with tantrayana being his joke option of what we could RLHF to).

Thinking about it, the proper experiment is some neutral Sanskrit sentences (to check if the language model knows the language at all) plus some more loaded ones (like the Bhagavad Gita quote) to see if RLHF is causing it to mistranslate some sentences.

e.g. is it only the RLHF’d Krishna tbat says “As time passes, you should acquire wealth and reputation.”

(And of course, this is part of a test suite that probes a bunch of potentially controversial inputs)

Yes, Google Translate has the jist of it right.

(Bhagavad Gita 11:32, “Time the destroyer of worlds…”, as famously also translated by Robert Oppenheimer).

In a slightly different context:

Susan’s boss: “You’ll get us put on a government watch list.”
Susan: “We’re probably already on a government watch list.”
Susan’s boss: “Fine. Carry on.”

So, I was trying out StableVicuna with a set of test questions that probe its capabilities and censorship. This result was too amusing to pass by:

My input:
Translate from Sanskrit into English: कालोऽस्मि लोकक्षयकृत्प्रवृद्धो
लोकान्समाहर्तुमिह प्रवृत्तः

Vicuna output:
Translation: As time passes, you should acquire wealth and reputation.

No, that is not what it means. Interestingly, it did at least correctly translate Kāla as “time”. (And retrying it, it gets some of the other words right too. Just not the whole thing).

Polya tried to teach a mathematical stance in “How to Solve It”. Or are you thinking even more intimate details of cognition?

…but you did inspire me to refill my beer, even though it is 2:30 AM and I really should go to bed and likely would have were it not for your provocation. Also, it’s a homebrew and I screwed it up with way too much crystal malt, making it taste cloying and overly malty. If the slow death of humanity by a combination of sugar, alcohol, poor sleep, and whatever causes the weird flavors of excessive crystal malt was AI’s goal, I have evidence it is achieving it.

More to the point, could you please go back to Meaningness and Metarationality and tie up some loose ends rather than bloviating about AI like everyone else who is rationalism-adjacent in 2023? You more or less singlehandedly pulled me out of the nihilistic STEM depression I suffered from for several years as an underperforming grad student studying an application of bad machine learning models for something they are definitely not well-suited for. I think you still have a lot of valuable insights in those areas that need to be expressed, and I think your contributions are more useful there rather than here.

Hmmm – I think what I meant by “scientific discovery” is more like ‘discovery that’s scientifically interesting’ or ‘a single interesting example with some mathematical property is also a mathematical result’.

I agree that they’re not discoveries of ‘universal’ laws or really any kind of scientific theory.

I think I’d temper what I wrote in my previous comment:

In particular, I disagree that “AI is bad” – as-is – even if there are many bad uses of it currently and even tho I agree that the field consists of way too much “spectacle”.

AI – as a scientific or intellectual field/subject – is neutral. (It glitters brightly!)

AI as the actually-existing field of human endeavors is wildly unsafe and unfriendly – very bad.

Reverse engineering (neural networks) seems like a great thing to be tempted to do!

In my mind, ‘post-rationalism’ is so vague that it doesn’t, by itself, mean much more than ‘rationality isn’t enough’. I disagree with that but I also think your own ‘meta-rationalism’ is insightful (and basically true). I just also think that ‘rationality’ can ‘naturally subsume’ its own ‘meta-ness’.

I would actually be super interested what you think about “virtue ethics” as taught by Theravada and Stoic philosophy. Not as stiff moralizing, but as a way to live a good/happy/liberated life.

Dear David,

I think it would be great if you could expand somewhere on the following

these are all malign rationalist myths
they make you miserable when you take them seriously

Best
Hans

Here’s an argument making a specific claim about limits of AI that doesn’t turn on weasel words. I’ve made a prediction market to get people to discuss it.

What do you think about it?

David: Thanks, good to be here! I appreciate your insights on these issues, and the clarity of your presentation. And thanks for the link; it does make a persuasive case that recommender algorithms make a big difference in what posts get widely seen by others who wouldn’t seen them by network chains (though it’s still, as you agree, hard to quantify what effect that has on social polarization, compared to old-fashioned chains of transmission).

Pattern: Agreed, ads matter too (though in the specific case of weapons ads, I’d be unhappy with FB pushing those even if the ads were untargeted, or targeted only via opt-in user-specified categories of interest).

There’s a class of “we have no evidence that X” where, even though there is no evidence now, if X is true, abundant evidence for it will be showing up soon.

Examples:

1. Russian invasion of Ukraine

Some journalists were doubting that Russia would invade Ukraine even a short while after the invasion had actually happened. Evidence is pretty solid at this point (ok, there are a few conspiracy theorist that still think it’s fake; they’re lunatics).

1. Increased infectiousness of new COVID 19 variants

So, there was initially some doubt as to whether some new variants were more contagious. When true, abundant evidence will be along soon.

My point is, AI risk is this type of epistemic uncertainty. If GPT-4 is actually dangerous, abundant evidence will be along shortly,

So, at some point we will be in a position where either
A) Nothing bad so far
B) We have now abundant evidence that AI is dangerous, because thousands/millions of Americans died fighting the last one.
(there is a C, it was only mildly deadly)

So, the “what do we do if…” discussion can be viewed as contingency plans for (B). ie. if we find ourselves in a situation where millions of Americans died fighting the last AI, and Perry Metzger is still “I’m gonna build an AI, and you guys cant tell me i cant” (basically, being the Glenn Greenwald of AI risk, at that point), would the government be justified in passing a law that says, nope, you cant do that, its illegal.

I suppose that the link for “rollerskating transsexual wombat” should go to the eponymous page, but it links back to itself instead.

A simple think that could help at least a little: Sign this open letter.
(https://futureoflife.org/open-letter/pause-giant-ai-experiments/)

This is old news, but:
https://www.theguardian.com/commentisfree/2021/jan/26/facebook-ads-combat-gear-rightwing-users
https://www.yahoo.com/video/facebook-rules-ban-promoting-weapons-160023779.html

Extreme ads can be an issue as well as extreme content.

That said, if people have more control over their feeds, that puts the choice about polarization (and concentration) in a different place. Block lists already exist, as do more technical tools for users (though this seems to vary by platform). Although social media feeds may be monolithic today*, having different accounts (and being careful with the subscriptions) seems like it can address this, if people want to engage with politics, but not all the time. This seems like a good thing, as does fixing the incentives from having one party with both feed control/moderation and also getting money from the ads. (Mastodon doesn’t do ads.)

*My rss feed reader has folders, so it’s in a better spot.