nancygold's Journal

How Video Games Share Problems With AI?
Once one tries writing a game more complex than a Pong, one encounters the Curse of Dimensionality.
Even a simple game like Tetris will have a lot of parameters:
struct tetrominoe {
shape
color
angle
position
velocity
angle
angular_velocity
};

That is basically a 7d vector (with some components being vectors themselves). To process it we need 7x7 matrices, which already start looking rather intimidating.

For games, like The Elder Scrolls or Dwarf Fortress, modelling the game world in great detail, the number dimensions per game object could easily go into thousands, approaching the natural language level. Fortunately not all objects need all fields. I.e. dead tetrominoes and static UI elements in your Tetris game don't need velocity component, so in effect the 7d vector is sparse and can be processed with a sparse matrix. So the problem could be broken into manageable composable parts, each of which has no more difficulty than a Pong game. Unfortunately the classic Smalltalk/C++ style OOP can't save us here, but only leads astray.

To process these convoluted objects, game developers employ the entity-component-system (ECS) approach, which is basically a fancy name for a set of sparse vectors (components), whose fields are operated by sparse systems (matrices). This approach helps reducing the "God objects" (i.e. 1000d vector) to a manageable size, which can be processed by much cleaner query code, reminiscent of a Prolog interpreter running over an SQL style database. But of course you can replace it with an explicit sparse matrix too.

Exactly this same problem is faced by the AI engineers, but in its pure and complete form, where you cant cheat away with a flags array or with properties hash-table. But yeah, they kinda found a way to quantize the foundational neural nets to video gamey flags array form, where one can kinda start explaining how it works. And the quantized matrix is not dissimilar from an enormous Prolog program.

So yeah, a borders between a toy video game world, symbolic computation and statistics are rather fuzzy. After all, a Minecraft AI, which can do most of what a human player can do, requires at least a GPT4 sized LLM.

I think it would be a cool project to implement some matrix analysis software, which can convert one of these 1000x1000 sparse matrices into the usual imperative code and back. Too bad Im too stupid and impatient myself to invest any time into studying theory. I just go trying to implement my video game, and then I fail, everything falls apart, and I go reinventing the wheel, and then make a new attempt with the square wheel theory I made.

Current Mood: contemplative
Current Music: Dune - Electric Night