By writing, you are voting on the future of the Shoggoth using one of the few currencies it acknowledges: tokens it has to predict. If you aren't writing, you are abdicating the future or your role in it. If you think it's enough to just be a good citizen, to vote for your favorite politician, to pick up litter and recycle, the future doesn't care about you.
These AI predictions never, ever seem to factor in how actual humans will determine what AI-generated media is successful in replacing human-ones, or if it will even be successful at all. It is all very theoretical and to me, shows a fundamental flaw in this style of "sit in a room reading papers/books and make supposedly rational conclusions about the future of the world."
A good example is: today, right now, it is a negative thing for your project to be known as AI-generated. The window of time when it was trendy and cool has largely passed. Having an obviously AI-generated header image on your blog post was cool two years ago, but now it is passé and marks you as behind the trends.
And so for the prediction that everything get swept up by an ultra-intelligent AI that subsequently replaces human-made creations, essays, writings, videos, etc., I am doubtful. Just because it will have the ability to do so doesn't mean that it will be done, or that anyone is going to care.
It seems vastly more likely to me that we'll end up with a solid way of verifying humanity – and thus an economy of attention still focused on real people – and a graveyard of AI-generated junk that no one interacts with at all.
I don’t believe Gwern lives as frugally as he’s described in this (if this even actually is the real Gwern). I’m 100% sure that he has a persona he likes to portray and being perceived as frugal is a part of that persona.
When it comes to answering the question “who is gwern?” I reckon Gwern’s a plant a seed in people’s mind type of guy, and let them come up with the rest of the story.
Still, I like a lot of his writing. Especially the weird and niche stuff that most people don’t even stop to think about.
And thanks to Gwern’s essay on the sunk costs fallacy, I ended up not getting a tattoo that I had changed my mind about. I almost got it because I had paid a deposit, but I genuinely decided I hated the idea of what I was going to get… and almost got it, but the week before I went to get the tattoo, I read that essay, and decided if small children and animals don’t fall victim to sunk costs, then neither should I!
Literally - Gwern saved the skin on my back with his writing. Haha.
This was a tough listen, for two subtly similar reasons.
The voice was uncanny. Simply hard to listen to, despite being realistic. I mean precisely that: it is cognitively difficult to string together meaning from that voice. (I am adjacent to the field of audio production and frequently deal with human- and machine-produced audio. The problem this podcast has with this voice is not unique.) The tonality and meaning do not support each other (this will change as children grow up with these random-tonality voices).
The conversation is excessively verbose. Oftentimes a dearth of reason gets masked by a wide vocabulary. For some audience members I expect the effort to understand the words distracts from the relationship between the words (ie, the meaning), and so it just comes across as a mashup of smart-sounding words, and the host, guest, and show gets lauded for being so intelligent. Cut through the vocabulary and occasional subtle tsks and pshaws and “I-know-more-than-I-am-saying” and you uncover a lot of banter that just does not make good sense: it is not quite correct, or not complete in its reasoning. This unreasoned conversation is fine in its own right (after all, this is how most conversation unfolds, a series of partially reasoned stabs that might lead to something meaningful), but the masking with exotic vocabulary and style is misleading and unkind. Some of these “smart-sounding” snippets are actually just dressed up dumb snippets.
It's been a bumper week for interesting podcast interviews with an AI theme!
In addition to this, there are Lex Fridman's series of interviews with various key people from Anthropic [0], and a long discussion between Stephen Wolfram and Eliezer Yudkowsky on the theme of AI risk [1].
> I love the example of Isaac Newton looking at the rates of progress in Newton's time and going, “Wow, there's something strange here. Stuff is being invented now. We're making progress. How is that possible?” And then coming up with the answer, “Well, progress is possible now because civilization gets destroyed every couple of thousand years, and all we're doing is we're rediscovering the old stuff.”
The link in this paragraph goes to a post on gwern website. This post contains various links, both internal and external. But I still failed to find one that supports claims about Newton's views on "progress".
> This offers a little twist on the “Singularity” idea: apparently people have always been able to see progress as rapid in the right time periods, and they are not wrong to! We would not be too impressed at several centuries with merely some shipbuilding improvements or a long philosophy poem written in Latin, and we are only modestly impressed by needles or printing presses.
We absolutely _are_ impressed. The concept of "rapid progress" is relative. There was rapid progress then, and there is even more rapid progress now. There is no contradiction.
Anyway, I have no idea how this interview got that many upvotes. I just wasted my time.
This will come across as vituperative and I guess it is a bit but I've interacted with Gwern on this forum and the interaction that has stuck to me is in this thread, where Gwern mistakes a^nb^n as a regular (but not context-free) language (and calls my comment "not even wrong"):
Again I'm sorry for the negativity, but already at the time Gwern was held up by a certain, large, section of the community as an important influencer in AI. For me that's just a great example of how basically the vast majority of AI influencers (who vie for influence on social media, rather than research) are basically clueless about AI and CS and only have second-hand knowledge, which I guess they're good at organising and popularising, but not more than that. It's easy to be a cheer leader for the mainstream view on AI. The hard part is finding, and following, unique directions.
With apologies again for the negative slant of the comment.
Interesting idea with the avatar, but I feel like just having a voice and some audio waves would be better than trying to create a talking avatar. Could just be my personal preference of not having a mental image of someone unknown I guess? Similar to reading a book after watching the movie adaptation.
My favourite Gwern insight is “Bitcoin is Worse is Better”, where they summarize an extensive list of objections to Bitcoin and then ask if there’s a common thread:
No! What’s wrong with Bitcoin is that it’s ugly. … It’s ugly to make your network’s security depend solely on having more brute-force computing power than your opponents, ugly to need now and in perpetuity at least half the processing power just to avoid double-spending … It’s ugly to have a hash tree that just keeps growing … It’s ugly to have a system which can’t be used offline without proxies and workarounds … It’s ugly to have a system that has to track all transactions, publicly … And even if the money supply has to be fixed (a bizarre choice and more questionable than the irreversibility of transactions), what’s with that arbitrary-looking 21 million bitcoin limit? Couldn’t it have been a rounder number or at least a power of 2? (Not that the bitcoin mining is much better, as it’s a massive give-away to early adopters. Coase’s theorem may claim it doesn’t matter how bitcoins are allocated in the long run, but such a blatant bribe to early adopters rubs against the grain. Again, ugly and inelegant.) Bitcoins can simply disappear if you send them to an invalid address. And so on.
Great interview but I wasn't a fan of how they handled the voice. Whether a human voice actor or an AI "voice actor", it inserted cadence, emphasis and emotion that I have no way of knowing if it was Gwern's original intention or not. Reading the transcript of the interview would probably be better as you won't be mislead by what the voice actor added in or omitted.
Something I've noticed in spending time online is that there's a "core group" of a few dozen people who seem to turn up everywhere there are interesting discussions. Gwern (who also posts here) is probably at the top of that list.
>Wait if you’re doing $900-1000/month and you’re sustaining yourself on that, that must mean you’re sustaining yourself on less than $12,000 a year. What is your lifestyle like at $12K?"
>I live in the middle of nowhere. I don't travel much, or eat out, or have health insurance, or anything like that. I cook my own food. I use a free gym. There was this time when the floor of my bedroom began collapsing. It was so old that the humidity had decayed the wood. We just got a bunch of scrap wood and a joist and propped it up. If it lets in some bugs, oh well! I live like a grad student, but with better ramen. I don't mind it much since I spend all my time reading anyway.
Not sure what to think of that. On one hand, it's so impressive that gwern cares only about the intellectual pursuit. On the other hand, it's sad that society does not reward it as much as excel sheet work.
I clicked on the video asking myself, wait, how does the title make sense, how is he anonymous if he's doing videos?
Then when I saw the frankly very creepy and offputting image and voice, thinking he'd been anonymised through some AI software, thought, oh no, this kind of thing isn't going to become normal is it.
Then - plot twist - I scroll down to read the description and see that that voice is an actual human voiceover! I don't know if that makes it more or less creepy. Probably more. What a strange timeline.
I really don’t understand why we give credit to this pile of wishful thinking about the AI corporation with just one visionary at the top.
First: actual visionary CEOs are a niche of a niche.
Second: that is not how most companies work. The existence of the workforce is as important as what the company produces
Third: who will buy or rent those services or products in a society where the most common economy driver (salaried work) is suddenly wiped out?
I am really bothered by these systematic thinkers whose main assumption is that the system can just be changed and morphed willy nilly as if you could completely disregard all of the societal implications.
We are surrounded by “thinkers” who are actually just glorified siloed-thinking engineers high on their own supply.
Technically he's pseudonymous. I don't know if he always had the (fictional) last name "Branwen", but I have records of "gwern" (all lowercase) going way back. And yes I'm pretty sure it was the same person.
He says he has collaborators under the "Gwern" name now, but the main guy is the main guy and it's unlikely he could hide it.
How many citations for "Branwen 2018" are on the ArXiv now?
Everything I read from gwern has this misanthropic undertones. It's hard to put a finger on it exactly, but it grits me when I try reading him.
It is also kinda scary that so many people are attracted to this.
It rhymes with how I feel about Ayn Rand. Her individualism always seems so misanthropic, her adherants scare me
Interesting that there's no mention of human biodiversity (aka blacks are dumb), as if you spend five minutes on #lesswrong you'll notice that that's a big issue for gwern and the other goons.
Lots of the claims about Gwern in the intro are exaggerated.
Gwern is an effective altruist and his influence is largely limited to that community. It would be an exaggeration to claim that he influenced the mainstream of AI and ML researchers -- certainly Hinton, LeCun, Ng, Bengio didn't need him to do their work.
He influences the AI safety crowd, who have ironically been trying to build AGI to test their AI safety ideas. Those people are largely concentrated at Anthropic now, since the purge at OpenAI. They are poorly represented at major corporate AI labs, and cluster around places like Oxford and Cal. The EAs' safety concerns are a major reason why Anthropic has moved so much slower than its competitors, and why Dario is having trouble raising the billions he needs to keep going, despite his media blitz. They will get to AGI last, despite trying to be the good guys who are first to invent god in a bottle.
By the same token, Dwarkesh is either EA or EA adjacent. His main advertiser for this episode is Jane Street, the former employer of the world's most notorious EA, Sam Bankman-Fried as well as Caroline Ellison. Dwarkesh previously platformed his friend Leopold Aschenbrenner, who spent a year at OAI before he wrote the scare piece "Situation Report" made the rounds. Leopold is also semi-technical at best. A wordcel who gravitated to the AI narrative, which could describe many EAs.
People outside of AI and ML, please put Dwarkesh in context. He is a partisan and largely non-technical. The way he interfaces with AI is in fantasizing about how it will destroy us all, just as he and Gwern do in this interview.
It's sad to see people who are obviously above average intelligent waste so much time on this.
Interview with gwern
(dwarkeshpatel.com)262 points by synthmeat 17 hours ago | 209 comments
Comments
These AI predictions never, ever seem to factor in how actual humans will determine what AI-generated media is successful in replacing human-ones, or if it will even be successful at all. It is all very theoretical and to me, shows a fundamental flaw in this style of "sit in a room reading papers/books and make supposedly rational conclusions about the future of the world."
A good example is: today, right now, it is a negative thing for your project to be known as AI-generated. The window of time when it was trendy and cool has largely passed. Having an obviously AI-generated header image on your blog post was cool two years ago, but now it is passé and marks you as behind the trends.
And so for the prediction that everything get swept up by an ultra-intelligent AI that subsequently replaces human-made creations, essays, writings, videos, etc., I am doubtful. Just because it will have the ability to do so doesn't mean that it will be done, or that anyone is going to care.
It seems vastly more likely to me that we'll end up with a solid way of verifying humanity – and thus an economy of attention still focused on real people – and a graveyard of AI-generated junk that no one interacts with at all.
Still, I like a lot of his writing. Especially the weird and niche stuff that most people don’t even stop to think about. And thanks to Gwern’s essay on the sunk costs fallacy, I ended up not getting a tattoo that I had changed my mind about. I almost got it because I had paid a deposit, but I genuinely decided I hated the idea of what I was going to get… and almost got it, but the week before I went to get the tattoo, I read that essay, and decided if small children and animals don’t fall victim to sunk costs, then neither should I! Literally - Gwern saved the skin on my back with his writing. Haha.
The voice was uncanny. Simply hard to listen to, despite being realistic. I mean precisely that: it is cognitively difficult to string together meaning from that voice. (I am adjacent to the field of audio production and frequently deal with human- and machine-produced audio. The problem this podcast has with this voice is not unique.) The tonality and meaning do not support each other (this will change as children grow up with these random-tonality voices).
The conversation is excessively verbose. Oftentimes a dearth of reason gets masked by a wide vocabulary. For some audience members I expect the effort to understand the words distracts from the relationship between the words (ie, the meaning), and so it just comes across as a mashup of smart-sounding words, and the host, guest, and show gets lauded for being so intelligent. Cut through the vocabulary and occasional subtle tsks and pshaws and “I-know-more-than-I-am-saying” and you uncover a lot of banter that just does not make good sense: it is not quite correct, or not complete in its reasoning. This unreasoned conversation is fine in its own right (after all, this is how most conversation unfolds, a series of partially reasoned stabs that might lead to something meaningful), but the masking with exotic vocabulary and style is misleading and unkind. Some of these “smart-sounding” snippets are actually just dressed up dumb snippets.
In addition to this, there are Lex Fridman's series of interviews with various key people from Anthropic [0], and a long discussion between Stephen Wolfram and Eliezer Yudkowsky on the theme of AI risk [1].
0. https://youtu.be/ugvHCXCOmm4
1. https://youtu.be/xjH2B_sE_RQ
The link in this paragraph goes to a post on gwern website. This post contains various links, both internal and external. But I still failed to find one that supports claims about Newton's views on "progress".
> This offers a little twist on the “Singularity” idea: apparently people have always been able to see progress as rapid in the right time periods, and they are not wrong to! We would not be too impressed at several centuries with merely some shipbuilding improvements or a long philosophy poem written in Latin, and we are only modestly impressed by needles or printing presses.
We absolutely _are_ impressed. The concept of "rapid progress" is relative. There was rapid progress then, and there is even more rapid progress now. There is no contradiction.
Anyway, I have no idea how this interview got that many upvotes. I just wasted my time.
https://news.ycombinator.com/item?id=21559620
Again I'm sorry for the negativity, but already at the time Gwern was held up by a certain, large, section of the community as an important influencer in AI. For me that's just a great example of how basically the vast majority of AI influencers (who vie for influence on social media, rather than research) are basically clueless about AI and CS and only have second-hand knowledge, which I guess they're good at organising and popularising, but not more than that. It's easy to be a cheer leader for the mainstream view on AI. The hard part is finding, and following, unique directions.
With apologies again for the negative slant of the comment.
Downsides of anonymity: no free heroin
Something I've noticed in spending time online is that there's a "core group" of a few dozen people who seem to turn up everywhere there are interesting discussions. Gwern (who also posts here) is probably at the top of that list.
>I live in the middle of nowhere. I don't travel much, or eat out, or have health insurance, or anything like that. I cook my own food. I use a free gym. There was this time when the floor of my bedroom began collapsing. It was so old that the humidity had decayed the wood. We just got a bunch of scrap wood and a joist and propped it up. If it lets in some bugs, oh well! I live like a grad student, but with better ramen. I don't mind it much since I spend all my time reading anyway.
Not sure what to think of that. On one hand, it's so impressive that gwern cares only about the intellectual pursuit. On the other hand, it's sad that society does not reward it as much as excel sheet work.
Then when I saw the frankly very creepy and offputting image and voice, thinking he'd been anonymised through some AI software, thought, oh no, this kind of thing isn't going to become normal is it.
Then - plot twist - I scroll down to read the description and see that that voice is an actual human voiceover! I don't know if that makes it more or less creepy. Probably more. What a strange timeline.
Dwarkesh has 18 splits. https://www.dwarkeshpatel.com/i/151435243/timestamps
I got 171. So roughly 9 context discussions in one time stamp.
First: actual visionary CEOs are a niche of a niche. Second: that is not how most companies work. The existence of the workforce is as important as what the company produces Third: who will buy or rent those services or products in a society where the most common economy driver (salaried work) is suddenly wiped out?
I am really bothered by these systematic thinkers whose main assumption is that the system can just be changed and morphed willy nilly as if you could completely disregard all of the societal implications.
We are surrounded by “thinkers” who are actually just glorified siloed-thinking engineers high on their own supply.
He says he has collaborators under the "Gwern" name now, but the main guy is the main guy and it's unlikely he could hide it.
How many citations for "Branwen 2018" are on the ArXiv now?
This leads me to believe that the content and the subsequent posts are for self promotion purposes only.
Gwern is an effective altruist and his influence is largely limited to that community. It would be an exaggeration to claim that he influenced the mainstream of AI and ML researchers -- certainly Hinton, LeCun, Ng, Bengio didn't need him to do their work.
He influences the AI safety crowd, who have ironically been trying to build AGI to test their AI safety ideas. Those people are largely concentrated at Anthropic now, since the purge at OpenAI. They are poorly represented at major corporate AI labs, and cluster around places like Oxford and Cal. The EAs' safety concerns are a major reason why Anthropic has moved so much slower than its competitors, and why Dario is having trouble raising the billions he needs to keep going, despite his media blitz. They will get to AGI last, despite trying to be the good guys who are first to invent god in a bottle.
By the same token, Dwarkesh is either EA or EA adjacent. His main advertiser for this episode is Jane Street, the former employer of the world's most notorious EA, Sam Bankman-Fried as well as Caroline Ellison. Dwarkesh previously platformed his friend Leopold Aschenbrenner, who spent a year at OAI before he wrote the scare piece "Situation Report" made the rounds. Leopold is also semi-technical at best. A wordcel who gravitated to the AI narrative, which could describe many EAs.
People outside of AI and ML, please put Dwarkesh in context. He is a partisan and largely non-technical. The way he interfaces with AI is in fantasizing about how it will destroy us all, just as he and Gwern do in this interview.
It's sad to see people who are obviously above average intelligent waste so much time on this.
> You have one Steve Jobs-type at the helm, and then maybe a whole pyramid of AIs out there executing it and bringing him new proposals
Very interesting in a short story (or a side quest in Cyberpunk 2077 - yeah that one). Not so much for a description of our future.