How Simon Willison uses LLMs to help him code

LLM
coding
tips
A great article filled with Simon’s deep experience in coding with LLMs.
Published

March 11, 2025

https://simonwillison.net/2025/Mar/11/using-llms-for-code/

Personal takeaways

  • Why use LLMs for code
    • they are not as lazy as you! They will do the important boring stuff, like write correct code, comments and tests.
    • Also, speed! Makes development faster.
  • Choose core tools based on being able to safely run and iterate on code.
  • Use tools that run code for you, e.g. Claude Artifacts.
  • Provide examples of what you want as output.
  • Context is king. Errors that an LLM makes becomes part of its context so start again…
  • Explore ideas first. Explore requirements, then possible functionality, then possible implementation. (Requirements spec. → Functional spec. → Technical spec.)
  • For pro code:
    • Get LMM to write tests with pytest.
    • In technical spec, write the function signature (the header) e.g. 
    • def multiply(x: int, y: int) -> int:
  • Remember to account for training cut-off dates — they won’t know about latest features.
  • Use established, boring tech!
  • Use LLMs to answer questions about codebases.
  • Connect Claude to GitHub.

Set reasonable expectations

  • Use LLMs to augment your abilities.
  • LLMs make mistakes, sometimes big ones.
  • The mistakes can be weird and not the kind a human would make.
  • Don’t anthropomorphize LLMs.
  • Some things they just cannot do.

Account for training cut-off dates

  • Find out the cut-off date of the model you are using.
  • Can be a relatively long time ago.
  • Influences what libraries they are familiar with.
  • It’s good to stick to boring, established technology!

Context is king

  • The most important thing!
  • Context isn’t just the prompt, it’s the whole conversation.
  • If an LLM starts making mistakes, it is often best to start afresh.
  • Context can also include:
    • uploaded files
    • current editor session.
    • Claude allows import from GitHub
  • LLM tools that obscure context are less useful.
    • Directly with ChatGPT and Claude is good.
  • Try writing a simpler version first, then add complexity.
  • It’s good to start with existing code to seed the context.
    • You could seed with several full examples.

Thoughts from me: Previous code responses become part of the context going forwards, so code with errors in it actually becomes part of the context — the AI will revert changes that fix errors because its context contains the errors. This is one reason why it is good to start again every so often, it removes code with errors from the context.

Ask the LLM for options

  • What are the potential ways this could be implemented?
  • Which option is the best?
  • Get the LLM to do this research phase.
    • e.g. ask what libraries are best to do X.
      • ask for demonstrations.
  • Make a prototype first.

Thoughts from me: Requirements specification, then function, then technical

Tell them exactly what to do

  • Once you have done the research, then tell them exactly what to do.
  • Be very specific.
  • Write the function signature (the header) e.g. 
    • def multiply(x: int, y: int) -> int:
    • This tells it the name to use, the variables and their types, and the return type.
  • You can then ask it to write tests using pytest
  • “Now write me the tests using pytest”

You have to test what it writes!

  • You cannot trust LLM code, you must read it and test it yourself.

Get it to rewrite

  • A bad initial result isn’t a failure, it’s a starting point for directing the model.

Use tools that can run the code for you

  • Stick to ones that run code in a safe sandbox.
  • ChatGPT Code Interpreter — executes Python code directly in a Kubernetes sandbox VM managed by OpenAI.
    • You can connect to Google Cloud storage but this only seems to be an option in the web interface.
  • Claude Artifacts — Builds HTML+JavaScript+CSS web application. Very locked down.
    • “Claude artifacts can load most code that is available via cdnjs, making them great for quickly trying out new libraries.” source.
  • ChatGPT Canvas — like Claude Artifacts.

A little more dangerous:

Vibe-coding is a great way to learn

Detailed example using Claude Code

  • Provide examples of what you want.
  • Specify you want mobile friendly html.

Be ready for the human to take over

  • LLMs are no replacement for human intuition and experience

Bonus: answering questions about codebases

  • LLMs are great about answering questions about code.
  • Use it on your own codebases or other people’s.
  • Zero risk, good for understanding and finding issues.