How should LLMs be used?
As the LLM tools available get increasingly better, the temptation to outsource more of your own work also increases. This raises an important question. How much of your work should you allocate to the models? Or, if you are inclined to ask optimizing questions, what division of work between you and some LLM is optimal? For me, the answer depends on two concepts.
The first concept relates to core competencies. Technologies in general have degraded some of our abilities. For example, with smart phones and map applications, we don’t really need to know how to navigate anymore. Want to know the ideal travel route to home, to your friend’s place, or to Italy? Just check the available tools. But most of us have accepted this deal, where we choose to rely on the apps and not our own capacities. I would argue the reason is that few of us find navigation to be a part of our core competency. For the same reason, almost nobody practices manual calculation.
I define core competency to mean an ability that you have an aptitude for, which you potentially or already excel at, and which you use to provide value to yourself or others. However, as pointed out earlier, modern technologies have capabilities which make the aptitude and excellence part questionable—we simply cannot match computers at calculation or chess anymore. But I believe that humans have and will have competencies that will not be outmatched by ML models for a long time.
I find creativity the most robust and lasting ability that will distinguish us and the improving models. We uniquely have the capacity to understand sentient life and imagine and create technologies, art, training programs and language that has the capacity to improve the world. But creativity is not something that develops without sustained and conscious effort; therefore, I find it imperative that we develop our own understanding to inform and improve our ability to create new beautiful, useful and interesting things.
The number one question for me, when using LLMs, is whether my core competencies are degraded or enhanced by them. As an example, I consider myself to be a decent thinker and a writer, and therefore I am very careful when using generative tools for text production. I could, for example, just ask Claude Opus to write me this essay, but I believe that the outcome would be just worse compared to my own work. Even worse, I would forego an opportunity to write the essay myself and improve my abilities further.
The second concept relates to the quality and impacts of the output. Here we need to understand what the scope of the output is to decipher how the models should be used. As an example, I began learning to develop mathematical animations so I can enhance the educational content on this website. But my workflow is heavily reliant on the usage of an LLM, and the development of the animation consists mostly of prompt driven work. The reason for this is that the product is a video and the information value of the video is what matters. I’ve yet to come up with a reason why I should care if the python code behind the video is horrible and unreliable, if the video serves its purposes.
But impact of the output is very different in other cases, such as delivering software to a client or implementing it in a self-driving car. Faulty software can have huge costs, such as serious brand harm for a company or actual human lives. Therefore, the effects of the output need to be carefully considered when using generative tools. In other words, when the scope of the output becomes more significant, we need and should implement more robust systems to ensure the quality of the output.
In my field, this question effectively reduces to the question of software quality. I’m a huge advocate for adopting quality improving frameworks, such as DevOps principles, clean coding practices and test-driven development. Thankfully, software quality was an issue long before I was even born, so there is a rich background of theory and practical knowledge that assists in this field.
In my post about a meme with Bilbo Baggins using ChatGPT, I also argued that we should be mindful of our intentions when using LLMs. Getting access to increasingly efficient cognition can boost our worst instincts, and nobody wants to a mass scale empowerement of our worst ideas. Luckily, the remedy is also offers an opportunity to improve the best ability that all of us have—the ability to monitor and view our own thinking critically.
In summary, when using LLMs we should ask whether the usage degrades or improves our core competencies, what the potential outcomes of the work are and how to ensure quality, and what our intentions are and whether they are worth pursuing at all.
Claude Opus 5.5 grammar fix suggestions accepted.