[Most Recent Entries] [Calendar View] [Friends View]
Thursday, September 26th, 2024
| Time | Event |
| 5:16p | The Barnes-Hut Algorithm for LLMs So currently the main issue with LLMs is that they are bound by the N^2 law, where N is the context size. The problem appears to be very similar to simulating gravity. Yet for gravity we can use the Barnes-Hut approximation, where far away entities are grouped together as a single entity. Therefore the question: is it possible to adapt Barnes-Hut to the attention matrix of LLMs? I think people are trying to do something similar (hierarchical approaches, summaries and tree of thought), but they still don't see the forest behind the trees. I.e. that the language modelling is basically a physics problem. All these N^2 connectednes/pathfinding problems appear to be expressible as gravity over some distance function, which itself a sorting problem, which can be solved by a N*log(N) algorithm. Current Mood:  amused |
| 8:44p | Unity I don't use Unity, but I do use a few placeholder assets from Unity store. Now Unity stored doesn't allow downloading assets without Unity installed. Unity is a huge 7 gigabyte bloated piece of shit. Although games made with Unity tend to be even bigger. Installing it alone is not enough to download assets either. One has to also create a project. Which in my case fails with "compilation errors" Would would it ever need to compile an empty template project? The Unity Editor also tends to crash even in idle, after importing assets. That is some GNU/Linux level stability. To all stock assets creators: please publish them at better stores. Like itch.io. Unity should just die already.   Current Mood:  annoyed |
| << Previous Day |
2024/09/26 [Calendar] |
Next Day >> |