Towards a Conceptual Framework for Artificial General Intelligence
date
Feb 8, 2023
slug
conceptual-framework-agi
author
status
Public
tags
Unedited
Theory
summary
type
Post
thumbnail
updatedAt
Feb 12, 2023 10:21 AM
What is wrong with LLMs today?
It seems that LLMs have a lot of “facts” encoded into them — they can talk about a huge variety of topics in detail. But, they don’t really have much thought process behind the words they generate; they just regurgitate and re-format the huge amount of information they already know into something approximating an acceptable answer to the prompt. Ask ChatGPT to play a game of chess and it will fail extremely quickly. The notation format is perfect, but the moves make no sense.
So what explains the gap? I think the answer may lie in the lack of “world knowledge” by these LLMS.
How much world knowledge can you truly derive from text?
ChatGPT has probably seen on the order of hundreds of thousands of chess games written in text notation. However, it still utterly fails at playing chess. The downfall of self-attention transformer models is that they can’t connect the rules about chess to the moves written down.
A human can probably learn a lot about chess just by reading at it. So what is the difference between how a human approaches reading a chess game vs. how a model approaches reading a chess game?
There is a lot of implicit information missing from text and human communication: when humans read a chess game, they know that they are looking at a game of chess, so they infer that the rules of chess apply to the moves that are being played.
To a human, “1. d4” is sufficient to know the board state after the first move is made.
To a computer, it would have to be given the following string of text: “White makes the first move since the player playing white always makes the first move in a game of chess. White moves its pawn on the d2 square to the d4 square, since pawns can advance one or two ranks when moving it for the first time, and only one rank per move thereafter, unless it is capturing a piece, which it can only do diagonally forward by one move left or right, or it reaches the rank on the other side of the board, in which case it is ‘promoted’ to a knight, bishop, rook, or queen. A pawn can also not move if it is ‘pinned’, meaning that moving the pawn would put the player’s king into check, which is not allowed. Since none of the black pieces are pinning the d2 pawn, and both the squares in front of it on d3 and d4 are clear, white is allowed to play pawn d2 to d4 and chooses to do so.”
And then if you wanted to have the LLM do anything except play random legal moves, you would then have to feed it fifty paragraphs for each move describing all the possible combinations and responses by both players, and an explanation about why one particular move is a good one.
THIS is why most engines up until recently have been hand-coded. They first generate a list of legal moves and make a minimax tree (with lots of optimizations such as alpha-beta pruning). They don’t even consider illegal moves and you don’t have to explain to them the rules of chess.
So why is AlphaGo Zero, the chess-playing neural network, still so good at playing chess? Well for starters, it still only ever plays legal moves, so it didn’t have to learn what the rules are. But also, it has played itself so many times that it brute forces some general rules for making a good move.
For a human that is new to driving, they do not need to know that driving off a cliff will lead to their car falling some distance and destroying both the car and human. But a neural network will drive a car off a cliff ten thousand times (or more) before it finally realizes that driving a car off a cliff is a bad idea. Almost all humans have never driven off a cliff but they know the outcome, since they know about the concept of gravity from their experience with the world.
This brings me to the first concept I want to highlight: interaction. I think it is very important for neural networks to be able to interact with the world in order to be good. Neural networks need to be able to see the impacts of what they do in the world in order to evaluate if what they did was good or bad. A singular human could theoretically be born into an empty world and invent most of mathematics, since the mathematicians that discovered today’s mathematics were fellow humans with approximately the same brain structure and abilities as the single person, but yet the single person would probably not be able to invent all of mathematics even given a very long time. This is because collaboration and interaction with other people provides feedback on your own ideas that you would not have thought about on your own.
I propose a new kind of neural network, which I call, a Dialectical Neural Network (or maybe DLLM). We start with a minimal language model that has some understand of language generation. Then we embed that language model into a larger model, so that the larger neural network would only dedicate a small amount of parameters towards language generation and the rest would be left for rule inference, which is the important part. Person A gives an argument for something. Then, the DLLM B will try to generate a compelling argument for the other side of the argument. Then, A and B go back and forth discussing why the arguments were good or bad (in terms of logic, validity, soundness, not fallacies).
Another concept I want to highlight: summarization.
After this back and forth, B generates a summary about what they learned from the back and forth. This serves as a set of rules going forward. For example, a rule might be, “When I argued about the issue of homelessness with my friend, I failed to take into account that they are generally more left-leaning on political issues. Next time, I might try framing an outcome that I desire in terms of his leftist viewpoint, and try finding more common ground between us.”
Some language models probably already have okay performance in this task.
Then B’s ruleset is updated using some sort of rule updating function, and the process repeats again, except B’s argument is generated using the updated ruleset.
Humans learn through having a large number of interactions in the world, and this kind of mimics that experience of having a large number of interactions in the world.
Interaction can be thought of as a thesis (A) and an antithesis (B). Summarization can be thought of as the synthesis, the new thing that is born out of the interaction between the thesis and antithesis.
Humans are good at ideas but not memory. Machines are extraordinary at memory but terrible at ideas. I think this explains why LLMs are leaning into the strengths of computers, and they are consuming huge amounts of memory and compute power. But as I explained, I think this is somewhat of a red herring and there is a limit to how good these models can get due to the lack of explanation for inferred rules in text.