Make, Bash, and a scripting language of your choice

---

Creating AWS Resources…let me count the ways

You need to create an S3 bucket, an SQS queue, an IAM policy and a few other AWS resources. But how?…TIMTOWTDI

The Console

• Pros: visual, immediate feedback, no tooling required, great for exploration
• Cons: not repeatable, not version controllable, opaque, clickops doesn’t scale, “I swear I configured it the same way”

The AWS CLI

• Pros: scriptable, composable, already installed, good for one-offs
• Cons: not idempotent by default, no state management, error handling is manual, scripts can grow into monsters

CloudFormation

• Pros: native AWS, state managed by AWS, rollback support, drift detection
• Cons: YAML/JSON verbosity, slow feedback loop, stack update failures are painful, error messages are famously cryptic, proprietary to AWS, subject to change without notice

Terraform

• Pros: multi-cloud, huge community, mature ecosystem, state management, plan before apply
• Cons: state file complexity, backend configuration, provider versioning, HCL is yet another language to learn, overkill for small projects, often requires tricks & contortions

Pulumi

• Pros: real programming languages, familiar abstractions, state management
• Cons: even more complex than Terraform, another runtime to install and maintain

CDK

• Pros: real programming languages, generates CloudFormation, good for large organizations
• Cons: CloudFormation underneath means CloudFormation problems, Node.js dependency

…and the rest of crew…

Ansible, AWS SAM, Serverless Framework - each with their own opinions, dependencies, and learning curves.

Every option beyond the CLI adds a layer of abstraction, a new language or DSL, a state management story, and a new thing to learn and maintain. For large teams managing hundreds of resources across multiple environments that overhead is justified. For a solo developer or small team managing a focused set of resources it can feel like overkill.

Even in large organizations, not every project should be conflated into the corporate infrastructure IaC tool. Moreover, not every project gets the attention of the DevOps team necessary to create or support the application infrastructure.

What if you could get idempotent, repeatable, version-controlled infrastructure management using tools you already have? No new language, no state backend, no provider versioning. Just make, bash, a scripting language you’re comfortable with, and your cloud provider’s CLI.

And yes…my love affair with make is endless.

We’ll use AWS examples throughout, but the patterns apply equally to Google Cloud (gcloud) and Microsoft Azure (az). The CLI tools differ, the patterns don’t.

---

A word about the AWS CLI --query option

Before you reach for jq, perl, or python to parse CLI output, it’s worth knowing that most cloud CLIs have built-in query support. The AWS CLI’s --query flag implements JMESPath - a query language for JSON that handles the majority of filtering and extraction tasks without any additional tools:

# get a specific field
aws lambda get-function \
    --function-name my-function \
    --query 'Configuration.FunctionArn' \
    --output text

# filter a list
aws sqs list-queues \
    --query 'QueueUrls[?contains(@, `my-queue`)]|[0]' \
    --output text

--query is faster, requires no additional dependencies, and keeps your pipeline simple. Reach for it first. When it falls short - complex transformations, arithmetic, multi-value extraction - that’s when a one-liner earns its place:

# perl
aws lambda get-function --function-name my-function | \
    perl -MJSON -n0 -e '$l=decode_json($_); print $l->{Configuration}{FunctionArn}'

# python
aws lambda get-function --function-name my-function | \
    python3 -c "import json,sys; d=json.load(sys.stdin); print(d['Configuration']['FunctionArn'])"

Both get the job done. Use whichever lives in your shed.

---

What is Idempotency?

The word comes from mathematics - an operation is idempotent if applying it multiple times produces the same result as applying it once. Sort of like those ID10T errors…no matter how hard or how many times that user clicks on that button they get the same result.

In the context of infrastructure management it means this: running your resource creation script twice should have exactly the same outcome as running it once. The first run creates the resource. The second run detects it already exists and does nothing - no errors, no duplicates, no side effects.

This sounds simple but it’s surprisingly easy to get wrong. A naive script that just calls aws lambda create-function will fail on the second run with a ResourceConflictException. A slightly better script wraps that in error handling. A truly idempotent script never attempts to create a resource it knows already exists.

And it works in both directions. The idempotent bug - running a failing process repeatedly and getting the same error every time - is what happens when your failure path is idempotent too. Consistently wrong, no matter how many times you try. The patterns we’ll show are designed to ensure that success is idempotent while failure always leaves the door open for the next attempt.

Cloud APIs fall into four distinct behavioral categories when it comes to idempotency, and your tooling needs to handle each one differently:

Case 1 - The API is idempotent and produces output

Some APIs can be called repeatedly without error and return useful output each time - whether the resource was just created or already existed. aws events put-rule is a good example - it returns the rule ARN whether the rule was just created or already existed. The pattern: call the read API first, capture the output, call the write API only if the read returned nothing.

Case 2 - The API is idempotent but produces no output

Some write APIs succeed silently - they return nothing on success. aws s3api put-bucket-notification-configuration is a good example. It will happily overwrite an existing configuration without complaint, but returns no output to confirm success. The pattern: call the API, synthesize a value for your sentinel using && echo to capture something meaningful on success.

Case 3 - The API is not idempotent

Some APIs will fail with an error if you try to create a resource that already exists. aws lambda add-permission returns ResourceConflictException if the statement ID already exists. aws lambda create-function returns ResourceConflictException if the function already exists. These APIs give you no choice - you must query first and only call the write API if the resource is missing.

Case 4 - The API call fails

Any of the above can fail - network errors, permission problems, invalid parameters. When a call fails you must not leave behind a sentinel file that signals success. A stale sentinel is worse than no sentinel - it tells Make the resource exists when it doesn’t, and subsequent runs silently skip the creation step. The patterns: || rm -f $@ when writing directly, or else rm -f $@ when capturing to a variable first.

---

The Sentinel File

Before we look at the four patterns in detail, we need to introduce a concept that ties everything together: the sentinel file.

A sentinel file is simply a file whose existence signals that a task has been completed successfully. It contains no magic - it might hold the output of the API call that created the resource, or it might just be an empty file created with touch. What matters is that it exists when the task succeeded and doesn’t exist when it hasn’t.

make has used this pattern since the 1970s. When you declare a target in a Makefile, make checks whether a file with that name exists before deciding whether to run the recipe. If the file exists and is newer than its dependencies, make skips the recipe entirely. If the file doesn’t exist, make runs the recipe to create it.

For infrastructure management this is exactly the behavior we want:

my-resource:
    @value="$$(aws some-service describe-resource \
            --name $(RESOURCE_NAME) 2>&1)"; \
    if [[ -z "$$value" || "$$value" = "ResourceNotFound" ]]; then \
        value="$$(aws some-service create-resource \
            --name $(RESOURCE_NAME))"; \
    fi; \
    test -e $@ || echo "$$value" > $@

The first time you run make my-resource the file doesn’t exist, the recipe runs, the resource is created, and the API response is written to the sentinel file my-resource. The second time you run it, make sees the file exists and skips the recipe entirely - zero API calls.

This brings us to the || rm -f $@ discipline. If the API call fails for any reason, the sentinel file is immediately removed. Without this a failed create leaves an empty or partial sentinel file. Make sees the file exists on the next run, skips the recipe, and the resource is never created. An idempotent bug - consistently broken, silently, forever.

One more pattern worth noting - test -e $@ || echo "$$value" > $@. This writes the sentinel only if it doesn’t already exist. Combined with the initial query this means we never rewrite a sentinel unnecessarily, avoiding redundant API calls on every make invocation. The sentinel is written exactly once - on the first successful run - and never touched again.

---

The Four Patterns

Armed with the sentinel file concept and an understanding of the four API behavioral categories, let’s look at concrete implementations of each pattern.

---

Pattern 1 - Idempotent API with output

The simplest case. Query the resource first - if it exists capture the output and write the sentinel. If it doesn’t exist, create it, capture the output, and write the sentinel. Either way you end up with a sentinel containing meaningful content.

The SQS queue creation is a good example:

sqs-queue:
    @queue="$$(aws sqs list-queues \
        --query 'QueueUrls[?contains(@, `$(QUEUE_NAME)`)]|[0]' \
        --output text --profile $(AWS_PROFILE) 2>&1)"; \
    if echo "$$queue" | grep -q 'error\|Error'; then \
        echo "ERROR: list-queues failed: $$queue" >&2; \
        exit 1; \
    elif [[ -z "$$queue" || "$$queue" = "None" ]]; then \
        queue="$(QUEUE_NAME)"; \
        aws sqs create-queue --queue-name $(QUEUE_NAME) \
            --profile $(AWS_PROFILE); \
    fi; \
    test -e $@ || echo "$$queue" > $@

Notice --query doing the filtering work before the output reaches the shell. No jq, no pipeline - the AWS CLI extracts exactly what we need. The result is either a queue URL or empty. If empty we create. Either way $$queue ends up with a value and the sentinel is written exactly once.

The EventBridge rule follows the same pattern:

lambda-eventbridge-rule:
    @rule="$$(aws events describe-rule \
            --name $(RULE_NAME) \
            --profile $(AWS_PROFILE) 2>&1)"; \
    if echo "$$rule" | grep -q 'ResourceNotFoundException'; then \
        rule="$$(aws events put-rule \
            --name $(RULE_NAME) \
            --schedule-expression "$(SCHEDULE_EXPRESSION)" \
            --state ENABLED \
            --profile $(AWS_PROFILE))"; \
    elif echo "$$rule" | grep -q 'error\|Error'; then \
        echo "ERROR: describe-rule failed: $$rule" >&2; \
        exit 1; \
    fi; \
    test -e $@ || echo "$$rule" > $@

Same shape - query, create if missing, write sentinel once.

---

Pattern 2 - Idempotent API with no output

Some APIs succeed silently. aws s3api put-bucket-notification-configuration is the canonical example - it happily overwrites an existing configuration and returns nothing. No output means nothing to write to the sentinel.

The solution is to synthesize a value using &&:

lambda-s3-trigger: lambda-s3-permission
    @function_arn=$$(cat lambda-function | perl -MJSON -n0 -e \
        '$$l=decode_json($$_); print $$l->{Configuration}->{FunctionArn}'); \
    config="{LambdaFunctionConfigurations => \
        [{LambdaFunctionArn => q{$$function_arn}, Events => [qw($(S3_EVENT))]}]}"; \
    config="$$(perl -MJSON -e "printf q{\"%s\"}, encode_json($$config)")"; \
    trigger="$$(aws s3api put-bucket-notification-configuration \
        --bucket $(BUCKET_NAME) \
        --notification-configuration $$config \
        --profile $(AWS_PROFILE) && echo $$config)"; \
    if [[ -n "$$trigger" ]]; then \
        test -e $@ || echo "$$trigger" > $@; \
    else \
        rm -f $@; \
    fi

The && echo $$config is the key. If the API call succeeds the && fires and $$trigger gets the config value - something meaningful to write to the sentinel. If the API call fails && doesn’t fire, $$trigger stays empty, and the else branch cleans up with rm -f $@.

This is also where a useful trick emerges for generating shell-safe JSON from a scripting language. The AWS CLI needs the JSON wrapped in double quotes as a single shell argument. Rather than fighting with shell escaping at the point of use, we bake the quotes into the generated value at the point of creation:

# perl
config="$(perl -MJSON -e "printf q{\"%s\"}, encode_json({...})")"

# python
config="$(python3 -c "import json; print('\"' + json.dumps({...}) + '\"')")"

TIMTOWTDI.

---

Pattern 3 - Non-idempotent API

Some APIs are not idempotent - they fail with a ResourceConflictException or similar if the resource already exists. aws lambda add-permission and aws lambda create-function are both in this category. There is no “create or update” variant - you must check existence first and only call the write API if the resource is missing.

The Lambda S3 permission target is a good example:

lambda-s3-permission: lambda-function s3-bucket
        @permission="$$(aws lambda get-policy \
                --function-name $(FUNCTION_NAME) \
                --profile $(AWS_PROFILE) 2>&1)"; \
        if echo "$$permission" | grep -q 'ResourceNotFoundException' || \
           ! echo "$$permission" | grep -q s3.amazonaws.com; then \
            permission="$$(aws lambda add-permission \
                --function-name $(FUNCTION_NAME) \
                --statement-id s3-trigger-$(BUCKET_NAME) \
                --action lambda:InvokeFunction \
                --principal s3.amazonaws.com \
                --source-arn arn:aws:s3:::$(BUCKET_NAME) \
                --profile $(AWS_PROFILE))"; \
        elif echo "$$permission" | grep -q 'error\|Error'; then \
            echo "ERROR: get-policy failed: $$permission" >&2; \
            exit 1; \
        fi; \
        if [[ -n "$$permission" ]]; then \
            test -e $@ || echo "$$permission" > $@; \
        else \
            rm -f $@; \
        fi

A few things worth noting here…

• get-policy returns the full policy document which may contain multiple statements - we check for the presence of s3.amazonaws.com specifically using ! grep -q rather than just checking for an empty response. This handles the case where a policy exists but doesn’t yet have the S3 permission we need.
• The sentinel is only written if $$permission is non-empty after the if block. This covers the case where get-policy returns nothing and add-permission also fails - the sentinel stays absent and the next make run will try again.
• We pipe errors to our bash variable to detect the case where the resource does not exist or there may have been some other error. When other failures are possible 2>&1 combined with specific error string matching gives you both idempotency and visibility. Swallowing errors silently (2>/dev/null) is how idempotent bugs are born.

---

Pattern 4 - Failure handling

This isn’t a separate pattern so much as a discipline that applies to all three of the above. There are two mechanisms depending on how the sentinel is written.

When the sentinel is written directly by the command:

aws lambda create-function ... > $@ || rm -f $@

|| rm -f $@ ensures that if the command fails the partial or empty sentinel is immediately cleaned up. Without it Make sees the file on the next run and silently skips the recipe - an idempotent bug.

When the sentinel is written by capturing output to a variable first:

if [[ -n "$$value" ]]; then \
    test -e $@ || echo "$$value" > $@; \
else \
    rm -f $@; \
fi

The else rm -f $@ serves the same purpose. If the variable is empty - because the API call failed - the sentinel is removed. If the sentinel doesn’t exist yet nothing is written. Either way the next make run will try again.

In both cases the goal is the same: a sentinel file should only exist when the underlying resource exists. A stale sentinel is worse than no sentinel.

Note also that our Makefiles set .SHELLFLAGS := -ec which causes make to exit immediately if any command in a recipe fails. This means commands that don’t write to $@ - like aws sqs create-queue - don’t need explicit failure handling. make will die loudly and the sentinel won’t be written.

---

Conclusion

Creating AWS resources can be done using several different tools…all of them eventually call AWS APIs and process the return payloads. Each of these tools has its place. Each adds something. Each also has a complexity, dependencies, and a learning curve score.

For a small project or a focused set of resources - the kind a solo developer or small team manages for a specific application - you don’t need tools with a high cognitive or resource load. You can use those tools you already have on your belt; make,bash, [insert favorite scripting language here], and aws. And you can leverage those same tools equally well with gcloud or az.

The four patterns we’ve covered handle every AWS API behavior you’ll encounter:

• Query first, create only if missing, write a sentinel
• Synthesize output when the API has none
• Always check before calling a non-idempotent API
• Clean up on failure with || rm -f $@

These aren’t new tricks - they’re straightforward applications of tools that have been around for decades. make has been managing file-based dependencies since 1976. The sentinel file pattern predates cloud computing entirely. We’re just applying them to a new problem.

One final thought. The idempotent bug - running a failing process repeatedly and getting the same error every time - is the mirror image of what we’ve built here. Our goal is idempotent success: run it once, it works. Run it again, it still works. Run it a hundred times, nothing changes. || rm -f $@ is what separates idempotent success from idempotent failure - it ensures that a bad run always leaves the door open for the next attempt rather than cementing the failure in place with a stale sentinel.

Your shed is already well stocked. Sometimes the right tool for the job is the one you’ve had hanging on the wall for thirty years.

---

Further Reading

• “Advanced Bash-Scripting Guide” - https://tldp.org/LDP/abs/html/index.html
• “GNU Make” - https://www.gnu.org/software/make/manual/html_node/index.html
• Dave Oswald, “Perl One Liners for the Shell” (Perl conference presentation): https://www.slideshare.net/slideshow/perl-oneliners/77841913
• Peteris Krumins, “Perl One-Liners” (No Starch Press): https://nostarch.com/perloneliners
• Sundeep Agarwal, “Perl One-Liners Guide” (free online): https://learnbyexample.github.io/learn_perl_oneliners/
• AWS CLI JMESPath query documentation: https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-filter.html