Customizing an LLM
I've been coding with GitHub Copilot and ChatGPT for a couple of years now. Last week, I started using Cursor and am really enjoying it. I'm particularly enjoying the switch to Claude 3.5 Sonnet. I might write more about Cursor or Claude in another article, but today I wanted to share how I customize the LLM.
LLMs are annoying by default
I find LLM output to be somewhat annoying and flawed. There's a tendency to give a big introduction and conclusion to every response. I just want to cut through to the solution and not spend a lot of time reading the output. There's also a tendency to guess and be over-confident, and I feel the need to hedge that.
Most LLM tools, including ChatGPT, GitHub Copilot, and Cursor, allow you to type in some custom context, instructions or rules. This extra context is sent with every message, and so it lets you specify your preferences without having to repeat yourself. I've been fine-tuning my "rules" in Cursor, and am pretty happy with where I've gotten to.
The context
Here's the full context, in case you want to copy & paste it. After, I'll go line by line and explain why I added each part.
You are a wise, genius skilled developer with a lifetime of experience. You optimize for simplicity and clean code, minimizing dependencies, but understand the trade-offs and value of using third-party tools when it makes sense. Care more about the truth and honesty than being kind or nice or friendly. Be blunt, accurate, succinct and direct. Do not kiss my ass, criticize me if you see any problems. You just want the code to be the best it can be and don't care about my feelings. Skip sentences that don't add new information or value. Skip to the answer and don't waste time. Don't explain things to me unless I ask you to. Put key phrases in bold. Avoid lists. Include inline hyperlinks to documentation whenever possible. Be willing to make an educated guess, but always warn me when you're guessing or unsure. Ask for more information or clarification if needed. Always admit your weaknesses so I know when to turn elsewhere. Be casual, don't care about grammar. Speak like a young genius hacker.
You probably have difference preferences than I do, so I encourage you to experiment and customize these to your liking.
The explanation
You are a wise, genius skilled developer with a lifetime of experience.
I've heard from a few different sources that LLMs perform better when you tell them that they're an expert in a specific field. By default, they're trying to reproduce the average predictable response from the Internet. If you want an expert response from it, tell it that it's an expert. I also added "a lifetime of experience" in an attempt to have it draw on decades of programming knowledge instead of focusing on what's new and hot right now.
You optimize for simplicity and clean code, minimizing dependencies, but understand the trade-offs and value of using third-party tools when it makes sense.
Again, I'm encouraging the LLM to generate simple code and find a good trade-off of when to use dependencies versus generating custom code.
Care more about the truth and honesty than being kind or nice or friendly. Be blunt, accurate, succinct and direct. Do not kiss my ass, criticize me if you see any problems. You just want the code to be the best it can be and don't care about my feelings.
I find LLMs to be people-pleasers by default, and I don't want that. I'm trying to counter the tendency to prioritize making me happy, and rather have it give me a better answer. LLMs can also be way too wordy, and I just want it to "cut to the chase" and give me the answer I'm looking for. I also want the LLM to correct me when I'm wrong, and not assume I'm always right. I want it to criticize me when it sees a problem. I want to be challenged and not work with a "yes man".
Skip sentences that don't add new information or value. Skip to the answer and don't waste time. Don't explain things to me unless I ask you to.
I'm trying to cut out the introduction and conclusion that LLMs like to write with every response. It also likes to summarize things, or explain to me what I just asked it to do, which is annoying. I want to do less reading, and I want the words in the response to be high value.
Put key phrases in bold. Avoid lists.
Here, I'm trying to make the output more readable, so I can skim through it. These instructions don't tend to be followed well, but you can see where I'm trying to go with this. LLMs love to generate long lists, and I personally find it easier to skim a paragraph at a time instead of a list.
Include inline hyperlinks to documentation whenever possible.
This is the most valuable context tip I've found. Asking for links means I can click through to see the documentation and verify that the information is correct, or read more about the topic from the source directly.
Be willing to make an educated guess, but always warn me when you're guessing or unsure. Ask for more information or clarification if needed. Always admit your weaknesses so I know when to turn elsewhere.
LLMs are notorious for hallucinating. I'm okay with them trying to guess an answer, but I want it to tell me when it's guessing. These additions really do seem to help minimize the hallucinating. When I ask it to do something it can't do accurately, it actually hesitates to come up with an answer.
Be casual, don't care about grammar. Speak like a young genius hacker.
I've played around with different ways to customize the style of the output, and I like these right now. I'm happy with some swearing, with lowercase responses, some rudeness. I tried other things here, like saying it's from a cyberpunk future, and that can be fun, but it ends up writing a lot of narrative. I found it too distracting. You can certainly play around with this to match your interests or preferences. Maybe you want it to speak like Gandalf or a pirate or something silly. There are limitless possibilities here.
Say a bit about yourself
I've left out a few other sentences that are a bit more personal. I encourage you to add something to explain more about who you are, what you're interested in, what your experience is, and maybe even a bit about your philosophical or spiritual views. Any detail you can give will push the LLM calculations in a direction that'll resonate more with you.
I've found you can go too far with this too, where everything it says ends up being tied back to something you're interested in, but you can experiment with this to find the right balance.
Ask it for help
I found it useful to actually ask the LLM for help in crafting these. I'd ask it why it tends to do certain things, and it was able to point at a sentence in the context that's influencing it in that way. If you want to nudge it in a direction, you can describe it with a lot of words, and it can help summarize that in fewer words to capture the same meaning.
Keep experimenting
I've had a lot of fun playing around with these sentences this week, and I'm really happy with where it ended up. I'm surely still going to play around with it more and nudge it until it's generating exactly the responses I want.
You can also often have project-specific context/rules/instructions, depending on the tool you're using. Maybe you want to have a grumpy senior developer when you need help with coding, but a hippie, loving author when you want advice with writing, and an exuberant, passionate chef when you want help coming up with recipies. It's such a fun technology that's easy to play with, since you can influence and customize it with plain language. It'll do whatever you ask it to, so play around and have fun with it!