There's a Japanese idea that I love called Shuhari. Which says that mastery has three stages, and they have to be walked in order. Shu is obedience, where you protect the form and follow the master without adding yourself to it. Ha is the break, where the rules start to feel less like commandments and more like suggestions. Ri is the disappearance of the form altogether, leaving only you and the work.
Thinking about LLMs for a second. LLMs are predictive models, and they're great at it. They've absorbed every form and pattern, which makes them the best "Shu" machine ever built.
But that's where they stop.
Innovation (usually) happens when you learn to bend the rules. You master the craft and learn to break the rules and push the status quo. And a model can't do this. It can't break a rule because it follows patterns. And it can't get to Ri, because Ri isn't a more advanced version of Shu.