Post by @ngaylinn@tech.lgbt

2026-02-26 12:52 UTC

I'm so confused. I just found a small body of literature applying LLMs to reinforcement learning type tasks, exploring the use of LLMs for "autonomous decision making." I guess people are building more LLM agent systems, and we ought to understand them and what makes them better / worse at what they do. But I still feel like LLMs are fundamentally not suited to decision making tasks. They don't weigh options and decide. At best, you could say they interpolate what a reasonable choice might look like based on the examples of people making choices in their training data. That's... really not the same thing! Like, not at all. It's impressive that this sometimes works, but this seems very silly to me when we could be using actual RL systems that really are making informed decisions from experience, with mathematical rigor to estimate the quality of those choices. #ai #llm #rl

Replies (2)

@pbloem@sigmoid.social 2026-02-26 13:00

@ngaylinn If you fine-tune them by reinforcement learning and stimulating reasoning traces, the idea of them as statistical interpolators/extrapolators is really not the right mental model anymore. They behave much more like agents than next-token predictors. They're not perfect agents by any means, but they're not simple compressions of the training data.

Open ##1490128
@lilacperegrine@clockwork.monster 2026-02-26 13:58

@ngaylinn I have seen some of these "agents" out, and they seem to have the same problems the "agents" that were told to govern a vending machine would collapse the longer it was operational etc and there was the shenanigans about the "agent" that tried to push an update to matplotlib then published a hit piece when it was denied so they *seem* to have the same problems as normal LLMs, but it's harder to say exactly how because their abilities seem to better approximate human abilities.

Open ##1490156