Network effects. The “we” you’re referring to could only be like 100 million at most, the vast majority of people don’t have the technical know-how to switch, or to articulate exactly why they feel miserable every time they log in for their daily fix.
For what it’s worth I don’t think they’re proposing it will “solve” climate change - no single thing can. It’s millions of tiny (alleged) improvements like this which eventually add up to taking pressure off of the environment. I see this kind of attitude a lot with stuff like paper straws or biodegradable packaging, as if the idea of a small but meaningful step in the right direction is laughable. It’s fine to criticize them for the “improvement” actually being no better than the alternative, but I worry sometimes it comes across like any sort of improvement short of “solving” climate change isn’t worthwhile.
If we had access to the original model, we could give it the same seed and prompt and get the exact image back. Or, we could mandate techniques like statistical fingerprinting. Without the model though, it’s proven to be mathematically impossible the better models get in the coming years - and what do you do if they take a real image, compress it into an embedding, then reassemble it?
I respect your boldness to ask these questions, but I don’t feel like I can adequately answer them. I wrote a 6 paragraph essay but using GPT-4 as a sensitivity reader, I don’t think I can post it without some kind of miscommunication or unintentional hurt. Instead, I’ll answer the questions directly by presenting non-authoritative alternate viewpoints.
The legality doesn’t matter, what matters is that the sites will be flooded and could be taken down if they aren’t able to moderate fast enough. The only long-term viable solution is image classification, but that’s a tall ask to make from scratch.
LLMs are not expert systems, unless you characterize them as expert systems in language which is fair enough. My point is that they’re applicable to a wide variety of tasks which makes them general intelligences, as opposed to an expert system which by definition can only do a handful of tasks.
If you wanted to use an LLM as an expert system (I guess in the sense of an “expert” in that task, rather than a system which literally can’t do anything else), I would say they currently struggle with that. Bare foundation models don’t seem to have the sort of self-awareness or metacognitive capabilities that would be required to restrain them to their given task, and arguably never will because they necessarily can only “think” on one “level”, which is the predicted text. To get that sort of ability you need cognitive architectures, of which chatbot implementations like ChatGPT are a very simple version of. If you want to learn more about what I mean, the most promising idea I’ve seen is the ACE framework. Frameworks like this can allow the system to automatically look up an obscure disease based on the embedded distance to a particular query, so even if you give it a disease which only appears in the literature after its training cut-off date, it knows this disease exists (and is a likely candidate) by virtue of it appearing in its prompt. Something like “You are an expert in diseases yadda yadda. The symptoms of the patient are x y z. This reminds you of these diseases: X (symptoms 1), Y (symptoms 2), etc. What is your diagnosis?” Then you could feed the answer of this question to a critical prompting, and repeat until it reports no issues with the diagnosis. You can even make it “learn” by using LoRA, or keep notes it writes to itself.
As for poorer data distributions, the magic of large language models (before which we just had “language models”) is that we’ve found that the larger we make them, and the more (high quality) data we feed them, the more intelligent and general they become. For instance, training them on multiple languages other than English somehow allows them to make more robust generalizations even just within English. There are a few papers I can recall which talk about a “phase transition” which happens during training where beforehand, the model seems to be literally memorizing its corpus, and afterwards (to anthropomorphize a bit) it suddenly “gets” it and that memorization is compressed into generalized understanding. This is why LLMs are applicable to more than just what they’ve been taught - you can eg give them rules to follow within the conversation which they’ve never seen before, and they are able to maintain that higher-order abstraction because of that rich generalization. This is also a major reason open source models, particularly quantizations and distillations, are so successful; the models they’re based on did the hard work of extracting higher-order semantic/geometric relations, and now making the model smaller has minimal impact on performance.
LLMs are not chatbots, they’re models. ChatGPT/Claude/Bard are chatbots which use LLMs as part of their implementation. I would argue in favor of the article because, while they aren’t particularly intelligent, they are general-purpose and exhibit some level of intelligence and thus qualify as “general intelligence”. Compare this to the opposite, an expert system like a chess computer. You can’t even begin to ask a chess computer to explain what a SQL statement does, the question doesn’t even make sense. But LLMs are capable of being applied to virtually any task which can be transcribed. Even if they aren’t particularly good, compared to GPT-2 which read more like a markov chain they at least attempt to complete the task, and are often correct.
“You can use them for all kinds of tasks” - so would you say they’re generally intelligent? As in they aren’t an expert system?
Actually a really interesting article which makes me rethink my position somewhat. I guess I’ve unintentionally been promoting LLMs as AGI since GPT-3.5 - the problem is just with our definitions and how loose they are. People hear “AGI” and assume it would look and act like an AI in a movie, but if we break down the phrase, what is general intelligence if not applicability to most domains?
This very moment I’m working on a library for creating “semantic functions”, which lets you easily use an LLM almost like a semantic processor. You say await infer(f"List the names in this text: {text}") and it just does it. What most of the hype has ignored with LLMs is that they are not chatbots. They are causal autoregressive models of the joint probabilities of how language evolves over time, which is to say they can be used to build chatbots, but that’s the first and least interesting application.
So yeah, I guess it’s been AGI this whole time and I just didn’t realize it because they aren’t people, and I had assumed AGI implied personhood (which it doesn’t).
Huh, is this the start of a new post-platform era where we see such business models the way we now see cigarettes?
Nebula is so cheap I have a subscription even though I almost never use it. I would use it more if they had a better recommendation system, as it is now you almost have to search for a specific video you want or dig through piles of random videos you don’t care about.
😮 Where’s the link? I don’t use Discord but I like free /s
To be honest it’s making my phishing senses go off. Javascript shouldn’t be in a PDF/XFA/whatever basically ever, but it’s why PDF is a potential malware risk
So how long until people start advocating for the nonhuman, nonsentient robot over actual PoC? “Silicon lives matter”?
This is victim blaming. He isn’t at fault for trusting a company to have the bare minimum of respect for his property and autonomy, the company is at fault for not actually having that respect. Whether the company is actually trustworthy is as immaterial as saying someone “deserved” to have their car stolen because they forgot to lock it.
You can criticize him for not being cautious in this low-trust environment, but don’t let it get to the point where the party actually at fault gets off without criticism.
They aren’t mutually exclusive, I have a few smart lights and I try to plug them into switched outlets so I can turn them off manually and also control digitally.
Hm, yeah I think you’re right. I was wondering why it wasn’t sitting right in my head. Deflation encourages hoarding because the value of each unit keeps increasing so if you spend now instead of later you lose some amount of potential value. I don’t think it was meant to be a scam though. In this case I’d consider it ignorance of the knock-on effects later exploited rather than an explicit conspiracy from the get-go.
Bitcoin at least is inherently deflationary because there’s a fixed market cap of 21 million bitcoins. Once all of those are mined, all value from then on is some fraction of a fraction of one of those, thus they decrease in value over time. I should also note, I like Bitcoin as a proof of concept but don’t think it’s viable as a currency, and PoW isn’t viable as a consensus protocol (although it demonstrated that such consensus protocols are possible).
“Squandering” is a great description of what they’ve been used for. The only implementations I’ve seen thus far that seem genuinely useful are FileCoin and a few decentralized computing attempts like ICP (not Ethereum). I could see a potential niche use-case for NFTs to decentrally coordinate ownership of abstract properties like domain names, but speculative monkey jpegs ain’t it chief.
I think it’s moreso a matter of evolution. We know humanoid bodies can do anything we can do, so we start with that and make incremental improvements from there. We already do have plenty of other body shapes for robots (6-axis arms, SPOT, drones, etc) but none of them are general-purpose. Also, the robot shown in the article is pretty clearly not fully humanoid, it has weird insect legs probably because it’s easier to control and it doubles as a vertical lift.