Goldilocks was the lead of a software development team. She needed to review pull requests from three of her team members.

The first developer's code was a mess. It relied on some deprecated features of an outdated library. The few modules were long and complex, trying to do too many different things. There were no tests, so it was impossible to be sure the code was bug-free. The architecture needed to run on a single server, so it could never scale up. There was no way to know whether it did what it was supposed to do.

The second developer's code was also a mess. The system was built using some brand new libraries and coding paradigms. The system comprised of a dozen different interconnected microservices. There was a very thorough test suite, testing every implementation detail. The system included infrastructure as code, but couldn't run on a single computer. There was no way to know whether it did what it was supposed to do.

The third developer's code was just right. It used the latest versions of libraries the team was familiar with. The system was split up into a dozen simple modules. It was obvious what each module did, and how it fit within the business requirements. There were a few tests for the core functionality, so she knew that it was working. The system was easy to get running, but was simple enough to scale up infinitely. It was very easy to understand, and to know that it did what it was supposed to do.

Goldilocks then had a meeting with the three developers.

She told the first developer their code was under-engineered. She said they should take some time to simplify it and make it easier for other developers to understand and work with it.

She told the second developer their code was over-engineered. She said they should take some time to simplify it and make it easier for other developers to understand and work with it.

She said well done to the third developer and approved the pull request.

The Problem

If you have servers in AWS doing a high volume of AWS service requests, you may come across some rare but frustrating sporadic credential errors like these:

"Unable to locate credentials"

or if you're using aws-sdk in Node.js:

"CredentialsProviderError: Could not load credentials from any providers"

I'm not totally sure why these errors happen, but typically I see them happen across multiple services, accounts and regions around the same time, which leads me to believe that there can be some sporadic flakiness in the metadata service used for fetching IAM credentials.

I tried using metadata retries and other configuration parameters to prevent this, but they didn't seem to make any difference.

The Solution

Looking for a solution, I found this buried in the AWS documentation for instance metadata retrieval:

"If you're using the IMDS to retrieve AWS security credentials, avoid querying for credentials during every transaction or concurrently from a high number of threads or processes, as this might lead to throttling. Instead, we recommend that you cache the credentials until they start approaching their expiry time."

Now, I don't think this throttling was the source of all the errors I was seeing, but it may be playing a role. Maybe the metadata service tolerance for throttling changes over time as demand changes, I don't know.

Either way, this gave me an idea to write a bash script to cache the IAM credentials in ~/.aws/credentials so they could be used by both the AWS CLI, and also any Node.js or Python clients accessing the AWS services:

#!/bin/bash

IMDS_URL="http://169.254.169.254/latest/meta-data/iam/security-credentials/"
AWS_CREDENTIALS_PATH="~/.aws/credentials"
PROFILE_NAME="default"

# 4.5 minutes, because new credentials appear 5 minutes before expiry
EXPIRY_BUFFER=270

get_aws_credentials() {
    local role_name=$(curl -s $IMDS_URL)
    local credentials_url="${IMDS_URL}${role_name}"
    local response=$(curl -s $credentials_url)

    local access_key_id=$(echo $response | jq -r '.AccessKeyId')
    local secret_access_key=$(echo $response | jq -r '.SecretAccessKey')
    local token=$(echo $response | jq -r '.Token')
    local expiration=$(echo $response | jq -r '.Expiration')
    local expiration_time=$(date -d "$expiration" +%s)

    echo "[$PROFILE_NAME]" > $AWS_CREDENTIALS_PATH
    echo "aws_access_key_id = $access_key_id" >> $AWS_CREDENTIALS_PATH
    echo "aws_secret_access_key = $secret_access_key" >> $AWS_CREDENTIALS_PATH
    echo "aws_session_token = $token" >> $AWS_CREDENTIALS_PATH
    echo "expiration = $expiration_time" >> $AWS_CREDENTIALS_PATH
}

should_fetch_credentials() {
    if [[ ! -f $AWS_CREDENTIALS_PATH ]]; then
        return 0
    fi

    local expiration_time=$(grep 'expiration' $AWS_CREDENTIALS_PATH | cut -d ' ' -f 3)
    local current_time=$(date +%s)

    if (( $current_time + $EXPIRY_BUFFER > $expiration_time )); then
        return 0
    fi

    return 1
}

if should_fetch_credentials; then
    get_aws_credentials
fi

Since the credentials have to be refreshed every few hours, I set it up to run in a cron job every minute, to check if the expiration time has come:

* * * * * /home/ec2-user/credentials.sh > /dev/null 2>&1

Voila! No more credential errors! I hope that helps. Let me know if you've run into the same errors, and if you found this approach useful.

If a user sees an error message on your web server, how do you find out?

Does the user report it directly, if they're friendly enough? Do you read about it on social media, if they're frustrated enough? Or do you receive a notification directly from the server?

I remember one of my first jobs at Strato, a web hosting company in Germany. When we deployed a new version of our content management system, we'd log onto the web server and tail the logs to see if any errors appeared. We wanted to see if we broke anything.

I remember scrolling through this web server error log. I'd see hundreds of different errors from before we deployed. When I asked the team about them, they weren't really sure about them. "Nobody has complained about these things," they said.

Seeing errors and warnings in logs made me uncomfortable. Something was broken, and we weren't doing anything about it. I made it my mission and took a few days to go and fix every error I could. I felt that unhandled errors were unacceptable. I wanted the error log to be empty.

Ever since then, I've always set up error monitoring on any server I manage. Whenever an error is logged, I want it emailed to me immediately. I usually run a cron job that scans the error log and sends out emails any new entries every minute. It might strip out some common, unavoidable networking errors. But for any unexpected errors, I want to be the first to know.

Whenever a user trips over an obscure bug, I know before the user has time to tell me. When things break badly, and the error log starts filling up, I know immediately.

Whether you're running Apache, Nginx or serverless functions, have you looked at your error logs lately? If not, go take a look. You might be surprised what you find.

Make it your team's goal to get those errors down to zero. Set up a cron job so those errors get sent to your inbox. Or, sign up for a log monitoring service that makes this easier for you.

Don't wait for your users to complain.

You don't need permission to write the highest quality code you can. You don't need permission to design a reliable server architecture that won't crash. You don't need permission to develop a suite of tests to ensure bugs are caught early. You don't need permission to upgrade your dependencies, to ensure your system stays secure and modern.

Your boss, manager or client will never ask you to take time to refactor your code. They'll never ask you to set up a test suite for the code you wrote. They'll never ask you to upgrade your framework.

It's not that they don't want you to do things to improve the quality of the code. It's because they expect you to write high quality code from the start. They believe you to know what it takes to design a reliable system. They trust you to build web apps that last.

They'll never ask you to make sure your code has no bugs. They'll never ask you to make sure the new system doesn't crash. They'll never ask you to make sure your code will be understood by other developers. It all goes without saying.

So, don't put these things off for later. Don't put writing tests on the backlog. Don't put refactoring on your list of nice-to-haves. Don't put improving the reliability of your servers on the wishlist. Don't wait for permission.

Do these things every day. Make them a part of your process. You don't need permission.

I started using ChatGPT when it came out a few months ago. It was mind blowing to chat with a computer and have it feel almost like a real person.

Some people are talking about how it's going to replace all sorts of jobs, including software developers. I'm not sure about that. But I have found some ways that it can definitely make our jobs easier, if we understand how it works and use it cautiously.

Understanding the limitations

Like all Large Language Models (LLMs), ChatGPT has been trained on a massive quantity of text from the Internet. Its basically a function which takes a context as input, including your prompt and the rest of your chat log, up to a limit of roughly 2,000 words. Based on that context, it is trying to make an educated guess of what should come next. Specifically, it's trying to predict what the people who trained it would have voted as the best response.

So when you're using it for coding, or anything else, always keep in mind that it is a guessing machine. Yes, it has been trained with a large amount of information, but not all that information is correct or up to date, and, most importantly, it's very good at completely making things up while exuding confidence.

It's amazing that ChatGPT can run some statistical analysis on words written by humans, and suddenly people are wondering if this thing is going to take over the world (no), take our jobs (maybe), or become self aware (definitely not). Statistically analyzing text becomes an excellent way to imitate humans, and come up with text that looks extremely plausible. GPT probably stands for "Guessing Plausible Text" (j/k).

Unfortunately, in programming, plausible doesn't cut it. We need things to be precisely accurate. It can quickly become frustrating to use an LLM as a tool to help you with programming, because it's wrong so often.

There's still hope. I've found it can still be a very powerful and helpful tool, even for developers.

Researching with ChatGPT

I think a very natural but dangerous way to use ChatGPT is as a search engine, to ask it factual questions. The problem here is that it's like playing "two truths and a lie". A lot of what it says is certainly true, but there's absolutely no way to know which parts are completely made up.

Even knowing this, I find myself using it this way anyway, but with a caveat. You need to treat ChatGPT as if it's your know-it-all friend who will go on and on confidently about any topic, even ones he is actually clueless about. I've learned about lots of new tools and features with ChatGPT, and some of them really did exist!

One trick is to ask for references. This is as simple as adding "Give references." to your prompts, or asking for them after. For coding topics, ChatGPT will usually be able to give you URLs you can click on to specific official documentation, and that is very useful.

Clicking those links to follow-up is absolutely critical here, because very often ChatGPT has told me how to do something using some specific API or function, and it has turned out to have been making it up. These situations did not save me any time, it actually wasted my time.

All that said, I love how ChatGPT can introduce me to all sorts of things I've never heard of before. Searching on Google would have required me clicking on dozens of semi-related pages and skimming through. ChatGPT is excellent at summarizing content, so you can take advantage of that.

Here's where ChatGPT can really shine: Let's say you have some specific software architectural challenge in front of you and you're not sure how to approach it. Open up ChatGPT and write it out in as much detail as you can.

"I need to build on online web-based chat interface. There will be a small number of users, and I'm not sure which database to use to manage this. I'm using AWS for web hosting and I'm hoping to find a serverless solution to save money. I'm familiar with JavaScript and Python. What are some tech stacks I could use for this? Provide references."

Seconds later, you'll have a list of options, some of which you may not have heard of, and links to read more about each one. If there's one you like, or if you have any follow up questions, you can just say "Tell me more about #2". Or you can provide more detail with your specific requirements to refine it's suggestions.

You always need to be careful, because I find that the more specific you get, the more likely you're going to encourage it to make up something that doesn't exist. Always ask for references, and don't make a decision until you've followed up on other websites to verify what ChatGPT says.

Transforming code and text

There are some low-risk and highly effective uses of ChatGPT, and transforming content is one of them. You can paste in some code or text, and ask it to rewrite it in some specific way. In these cases, it seems much less likely to make an error, and if it does make a mistake, you should be able to recognize it and refine your request quickly.

I've pasted in a JavaScript file with two dozen constant strings defined, and asked to convert all the variable names to uppercase. At first it converted both the variable names and the string contents to uppercase, so I had to be more specific and tell it to leave the strings alone. Then it completed it quickly and accurately, saving me a few mindless minutes.

I've pasted in an email from a client with a list of described menu options, plus a snippet of Svelte code with a few placeholders in the menu, and asked ChatGPT to add all the menu options into the code. It handled this very well.

You can ask it to rewrite a short function from JavaScript to Python, and it will do a good job of this as well, though it can make some mistakes depending on the complexity or the length of the code.

If you ever have these sorts of straightforward boring text transformation jobs in front of you, and your IDE isn't up to the job, try asking ChatGPT to do it for you, and save the headache.

Understanding & improving code

ChatGPT is excellent at summarizing any type of content, and that includes code.

Just paste in a chunk of code and it'll be able to tell you what the code does. You can ask it to add inline comments to the code for you too, though Copilot is quite good at this too.

If you get a weird error message, ChatGPT might be able to give you an explanation of why the error might have happened, and some possible ways to fix the error. Unlike a webpage, you can ask follow up questions in realtime and get feedback to help you find a solution.

I've also had success pasting in a freshly written function or module, and asking ChatGPT to suggest improvements. It's told me ways to improve error handling, or some cases I hadn't thought of where things might break. It's even found a few bugs in my code, and showed me how to fix them. If you work alone, it's nice to use ChatGPT for feedback and review, and maybe you'll learn something new too.

Coding with ChatGPT

ChatGPT is very capable of writing code. However, like everything else it does, it often makes mistakes.

In my experience, the code written by ChatGPT is rarely perfect on the first try. Very often, the code will try to do something that isn't possible, or misunderstand what was being asked of it. I guess that's true of code written by humans too.

When you're asking ChatGPT to write code for you, it's up to you to run the code and paste back any error messages or problems into the chat, asking for fixes. In a way, it's like the roles are reversed. You're no longer the programmer, but stuck between the AI and the compiler. I have to say, this is not a very fun place to be. I would much rather just make changes to the code myself, than to try different prompts until ChatGPT is able to generate the right code. Often it's faster to type the code you specifically want and need than to type some prompts and wait to see if ChatGPT has made it correct.

It's almost like working with a junior developer, except that a junior developer is capable of learning and improving and eventually becoming a senior developer. ChatGPT, on the other hand, isn't learning anything from you over the long term. It might learn from you in the short term, but remember, the context of an LLM is limited, and that means that it will soon forget the suggestions you made for improvement.

If, on the other hand, you're new to programming, then ChatGPT is going to be extremely helpful and time saving. I've seen lots of new developers have great success using ChatGPT in this way, to do things they don't know how to do. I believe ChatGPT and similar tools will enable a lot more people to get into coding, and that's really exciting.

Even as an experienced developer, we're always learning new things. Having ChatGPT lead the way and provide feedback in a new programming language or library can be extremely helpful. Just be wary that it's very likely to make mistakes, so you still need to understand what the code is doing. Never trust code written by an AI, just as you wouldn't trust any code you find on the Internet. Ultimately, code generated by an LLM is coming from code from the Internet, security issues and all.

Fortunately, ChatGPT makes some of this easier for you. As mentioned above, you can ask ChatGPT to explain the code it's written, or look for bugs. Sometimes it's worth doing this with the code it just generated. It's kind of funny how that's possible. Since it generates a word at a time, it can't often go back and fix its own mistakes during generation. So if you ask it if it made any mistakes, sometimes it'll be able to spot the mistake right away and write a better version.

Ask for small, simple code snippets

To be honest, I haven't enjoyed having ChatGPT generate large amounts of code for me. It hasn't seemed to saved me much time, it just changed how I spent my time. I've had more success asking it to do smaller, more limited things.

It's really good at writing SQL queries for you. Paste in the table schema and tell it what you're looking to query. You can also be specific about which programming language and library you're using to connect to the database. I think this will be very helpful to a lot of people.

It can also generate things like regular expressions, or other complex code, based on your description. More detail is always better here, including specific examples of edge cases.

Ask it to generate some boilerplate code for you, to give you a head start. Or, paste in the specifications from your manager and have it attempt a first draft for you to use as a starting point. Depending on your skill level, you might prefer to move into your editor and do the rest of the coding from here.

It's important that you're able to quickly test what it generated and verify that it works as expected. You can even paste in some code and ask ChatGPT to generate some unit tests for you. You can use it with Test Driven Development, pasting in some unit tests and ask it to write the code. You can even ask ChatGPT to generate some test code alongside any other code it generates, by including in your prompt something like "Write tests for the code too."

Comparison to Copilot

As I've written about before, I really enjoy using GitHub Copilot, and it helps me to be more productive. Copilot also uses GPT, but it's doing so in a more focused way that automatically takes your code into its context. It's very good at suggesting code while you're writing it, suggesting comments for your code, or generating code based on your comments. ChatGPT hasn't at all replaced my use of Copilot. If anything, it has made me appreciate Copilot more, and encouraged me to use Copilot in more creative ways. I've found myself bringing up the Copilot suggestions panel more often, to see the variety of suggestions available, and very often there are some better and more useful snippets available in here.

For some reason, using Copilot is less misleading. When Copilot makes a wrong suggestion, it doesn't bother me. Perhaps it's because there's no confidence here, everything is just a "suggestion".

ChatGPT is of course better at discussing and explaining things in plain language. Microsoft is already planning to integrate a chat interface into Copilot, so-called "GitHub Copilot X". You can sign up for the beta if you want to get early access to Copilot chat. I'm really looking forward to this, as it'll likely be a lot more useful for coding than ChatGPT currently is.

It's not a human

It's very important to keep in mind that ChatGPT is not a person. It's a statistically-driven guessing machine.

Like a human, it makes mistakes, but it won't tell you how sure or unsure it is about being right.

Like a human, it's trying to generate responses it thinks you'll like, but it has no feelings and will never be your friend.

Like a human, it has biases that it's not aware of and can't articulate, but it's incapable of growing and learning from you over time.

It can be hard to talk to a machine like this without all the baggage we've picked up from talking to real humans. I find myself saying "please" and "thank you" when I really don't need to.

I think we need to create a new place in our brains for interacting with things like this.

It's ok to be a bit blunt and succinct. Often it's necessary to be extra explicit, and state things that might otherwise seem obvious. You don't need to spare the feelings of these guessing machines. You need to tell it whenever it's wrong and ask it to fix its mistakes. You can tell it to "be succinct", to "skip unnecessary phrases" and "just output the code" and other commands which speed it up and tailor the output to your preferences. You may need to repeat these phrases regularly, and you may likely find some new patterns that work well for you.

Try it for yourself, have fun

I've outlined some of the approaches that have worked for me, but I suggest you try it out yourself and see what works for you. I think it's worth experimenting and finding a way for ChatGPT and other AI tools to help you out in your work.

These tools should make your life better, and make work more fun. The goal isn't just to save time, but to enjoy the process.

When you're feeling stuck, you can use ChatGPT as a mentor to help you get unstuck. When you want to bounce some ideas off someone, ChatGPT can give you helpful suggestions.

Save the fun coding stuff for yourself, and leave the boring parts for ChatGPT.