Today I’m releasing version 2.8.0 of
Amazon::API.
This version of Amazon::API is a significant milestone. It represents
the first release in a series of planned releases that seeks to
achieve parity with the Python Botocore library.
Botocore library parity means that
Amazon::APIcan consume the same Botocore service metadata and implement the same low-level AWS client behaviors needed to construct, sign, send, and interpret requests correctly.It does not mean reproducing every convenience API exposed by Boto3. Features such as waiters, transfer managers, resources, and higher-level helpers only require that
Amazon::APIexpose the underlying primitives needed to implement them.
Over the last year, with the assistance of AI, I’ve been able to
accelerate development of my lightweight interface to Amazon’s
APIs. The existing model-driven architecture of Amazon::API, already
built around Botocore metadata, is now approaching the AWS SDK for
Perl I imagined in 2017.
The result is a Perl AWS API that is much closer to the capabilities
of the Python Botocore library. It also moves Perl closer to the
broader goal of first-class AWS support that the community has been
pursuing since Paws was released in 2015.
Botocore parity is a daunting task for a Perl developer.
The challenge comes from the sheer number of Botocore behaviors that
one developer must recognize and implement. Those implementations then
have to be tested. The places where they differ from Boto3 results
must be analyzed by tracing through metadata and code. Each discovery
must finally be turned into a test and a fix. AI tools such as Claude
and ChatGPT have made it practical for me to work through that surface
area much more quickly, while I continue to make the architectural and
design decisions that determine what Amazon::API should become.
AI has been especially useful in the less glamorous but time-consuming parts of the work: debugging serialization problems, improving documentation, and writing or expanding unit tests. Those are exactly the areas where a second set of eyes, rapid iteration, and the ability to compare behavior across a large codebase can accelerate development.
It has also slowed things down at times. AI can hallucinate interfaces that do not exist, spend too much time nit-picking perfectly reasonable constructions or proposing paranoid guardrails, and head down dead ends when important context has been lost or forgotten. Using it effectively still requires knowing the codebase and recognizing when a suggestion is technically incorrect or runs counter to the architectural philosophy underpinning the entire work. A developer must be willing to question it and push back hard before AI sends one down a rabbit hole.
AI has also been extremely useful in helping me understand the broader AWS ecosystem. Implementing an API operation requires knowing how to construct the request correctly, but that is very different from understanding why a service behaves the way it does or what problem a particular capability is intended to solve. Endpoint resolution is a good example: implementing the rules is one task; understanding the use cases behind regional, account-specific, ARN-derived, or service-specific endpoints is another. AI has helped bridge that gap between implementing AWS behavior and understanding the ecosystem that gave rise to it.
AI’s limitations are real, but so are the gains. On balance, AI has accelerated this work far more than it has hindered it.
Amazon::API was not created in response to Paws. Paws had
already been on CPAN for a couple of years when I began formal
development of Amazon::API in 2017, but Paws was still very much a
project under active development. Its own documentation cautioned that
the SDK should not yet be considered stable, even though it was
already being used in production.
I was solving a similar problem independently, with a different set of
design goals. Amazon::API itself did not reach CPAN until December
of 2018.
Amazon::API was not an attempt to reinvent the wheel. It’s a
different wheel that was taking shape while Paws was still evolving.
Although Paws and Amazon::API attempt to solve the same problem,
they take different approaches. Amazon::API started light and
remains that way today. It keeps the runtime small, keeps AWS service
definitions independently updatable, and returns ordinary Perl data
instead of building a large generated object model around every
service.
That separation is particularly important in an ecosystem where AWS
service definitions change constantly. A new operation, shape, or
pagination rule does not necessarily require a new Amazon::API
release. Individual service distributions can instead be regenerated
against newer Botocore metadata, allowing the AWS API definitions to
remain current without coupling their release cycle to that of the
runtime.
Paws and Amazon::API both make use of Botocore metadata, but they
diverge sharply in how they package and expose it. Paws generates a
broad Perl object model representing AWS services, requests,
responses, and associated types. Amazon::API uses generated service
definitions with a comparatively small runtime and normally returns
plain Perl data structures.
Because those generated service distributions are independent of the runtime, an application can install only the AWS services it actually needs and update those definitions independently as AWS evolves.
Amazon::API is therefore aimed at Perl applications where a
lightweight, data-oriented interface, independently current AWS
service definitions, and a small production footprint are important.
Version 2.8.0 increases the stability and robustness of the API. It includes serialization fixes, a suite of tests that exercise the full set of Botocore-derived protocol and pagination test suites, endpoint rule-set resolution, and updated documentation. Additionally, features and capability gaps that still exist have been articulated in a formal roadmap.
The result is a much clearer picture of what Amazon::API supports
today, what remains to reach broader Botocore parity, and what is
intentionally outside the scope of the core runtime. The table below
summarizes that current state.
| Capability | Status | Notes |
|---|---|---|
| Service/operation model support | Implemented | Generated from Botocore service metadata; independently updatable |
| Request serialization/deserialization | Implemented | Existing protocols supported; Smithy RPC protocols are planned separately |
| Endpoint rule-set resolution | Implemented | 2.8.0 milestone; modeled endpoint context and auth metadata supported for SigV4 |
| SigV4 signing | Implemented | Includes modeled signing name, signing region, and disableDoubleEncoding |
| SigV4a | Planned (2.9.0) | Required for broader modern AWS endpoint/auth parity |
| S3 compatibility | Planned | Much closer after 2.8.0; full support depends on remaining auth and service-specific behavior |
| S3 Express | Planned (2.10.0) | Requires session auth and v4-s3express support |
| Smithy RPC protocols | Planned (3.0.0) | Includes rpcv2Cbor and related protocol behavior |
| Pagination | Implemented | Model-driven/compiler-backed |
| Checksums / payload-signing behavior | Planned (3.0.0) | Requires parity audit against Botocore handlers and protocol traits |
| Streaming / event streams | Planned (3.0.0) | Needed for applicable Smithy and streaming operations |
| Botocore handler semantics | Planned (3.0.0) | Systematic audit needed to identify remaining request/response mutations |
| Credential provider behavior | Out of scope | Owned by Amazon::Credentials; parity should be evaluated separately |
| Retry policy | Out of scope | Execution policy rather than request/protocol correctness |
| Waiters / transfer helpers / resources | Out of scope | Amazon::API should expose the primitives needed to implement them above the core runtime |
The remaining work is being broken into a small number of focused releases, each intended to close a specific set of gaps on the way toward broader Botocore parity.
| Version | Feature/Capability | Target Date | Status |
|---|---|---|---|
| 2.8.0 | Endpoint rule sets + modeled SigV4 signing | 9/26 | Released |
| 2.9.0 | SigV4a | 10/1 | - |
| 2.10.0 | S3 Express | 10/31 | - |
| 3.0.0 | Smithy protocol work + handler audit/fixes | 12/31 | - |
Amazon::API is tested at several different levels because no single test strategy is sufficient for a runtime that interprets Botocore service metadata.
The ordinary unit and regression suite exercises specific runtime behavior: request construction, serialization, deserialization, pagination, URI handling, endpoint context evaluation, service hooks, and signing.
A second layer uses protocol test cases derived from the Botocore test
corpus. These tests provide known request and response examples across
the AWS protocol families and allow Amazon::API behavior to be checked
against expectations derived from Botocore.
The corpus is not treated as exhaustive. When live AWS testing exposed a behavior or edge case that was not represented by the existing corpus, I added a focused regression test or custom corpus case.
Finally, selected requests are validated against live AWS services. This is particularly useful for endpoint resolution and signing, where a request can appear correct locally but still be rejected by AWS.
The current 2.8.0 testing picture looks like this:
| Test Area | Status | Notes |
|---|---|---|
| Unit and regression suite | Passing | Covers core runtime behavior including serialization, deserialization, pagination, request construction, URI handling, endpoint context, service hooks, and signing |
| Botocore protocol corpus | Passing | 671 tests across 15 protocol test files |
| Pagination | Passing | Dedicated coverage for model-driven paginator compilation and runtime evaluation |
| Endpoint rule-set evaluation | Passing | Covers rule evaluation, endpoint context sources and precedence, scalar template handling, and service endpoint selection |
| S3Control endpoint and URI behavior | Passing | Covers ARN-driven endpoint context, region selection, and percent encoding of non-greedy URI labels |
| S3 request construction | Passing | Covers virtual-hosted and force-path-style addressing, the S3 service hook, and final request URL composition |
| S3 greedy URI handling | Passing | Covers {Key+} behavior, preserving / while percent encoding other reserved characters |
| S3 response deserialization | Passing | Covers the rest-xml root-unwrapping behavior that previously affected some S3 responses |
| Kinesis endpoint resolution | Passing | ListShards with StreamARN resolves to the account-specific control endpoint |
| SigV4 auth propagation | Passing | Endpoint-derived signing name, signing region, and disableDoubleEncoding are propagated through the signing stack |
| Live AWS validation | Selective | Used for endpoint and signing behavior where AWS itself provides the final validation |
| Custom regression coverage | Ongoing | Added whenever real service behavior exposes a case not represented by the existing Botocore corpus |
Passing tests do not prove that every AWS operation is correct. What they do provide is a repeatable way to increase confidence and coverage: Botocore-derived corpus tests establish broad protocol behavior, focused regressions preserve specific fixes and service behaviors, and live AWS calls validate the cases where the service itself is the final authority.
There is still a larger question: why am I investing this much effort in a Perl API for AWS when many Perl developers have moved on to other languages with more complete and current support for AWS?
First, let’s discuss the elephant in the room. Isn’t Perl dying a slow death? I don’t think so. In fact, I think just the opposite might happen, especially if the community continues to enhance its toolchains to keep up with security and advancing technology. I simply refuse to accept the premise that Perl is simply fading away.
AI may be another catalyst that creates something of a renaissance for Perl, especially as it helps refresh, secure, and enhance legacy Perl-based applications. Large bodies of working Perl code that were once expensive or risky to modernize can now be analyzed, refactored, tested, documented, and secured with much greater leverage than was possible even a few years ago. A legacy Perl application still running today is often doing something important enough that replacing it is neither simple nor inexpensive.
And then there are the selfish reasons to build a modern AWS API for Perl. I need a capable, complete Perl AWS SDK. The applications and infrastructure I maintain need more than I can comfortably get from the existing choices, and rewriting those systems in another language simply to gain better AWS support is not an attractive solution.
And finally, I’d like to share my work. Perl has always benefited from
developers publishing the tools they built to solve their own
problems. Much of the software I rely on exists because someone else
chose to do exactly that. Amazon::API is my contribution back to
that tradition.
If you’ve used Paws or other Perl interfaces to AWS, you may still
be wondering what makes Amazon::API worth trying.
Your answer is probably specific to your needs. If you need:
…then Amazon::API may just fit the bill.
Version 2.8.0 is an important milestone, but it is not the end of the work.
The immediate roadmap is to continue closing the remaining gaps
between Amazon::API and Botocore:
…and, of course, to continue auditing Botocore behavior, expanding test coverage, and fixing defects as they are found. Lather, rinse, repeat.
Amazon::APIandAmazon::API::Helpare available from CPAN. Generated AWS service distributions are available from theAmazon::APIDarkPAN (https://cpan.openbedrock.net/orepan2). See https://cpan.openbedrock.net/signature for details on verifying the authenticity ofAmazon::APIservice classes.
The easiest way to try Amazon::API 2.8.0 is with STS and
GetCallerIdentity.
For example, using cpm:
cpm install -L local Amazon::API
cpm install -L local Amazon::API::Help
cpm install -L local --resolver 02packages,https://cpan.openbedrock.net/orepan2 Amazon::API::STS
export PERL5LIB=$(pwd)/local/lib/perl5:$PERL5LIB
export PATH=$(pwd)/local/bin:$PATH
Once installed, Amazon::API::Help can show you the operations
available for a service:
amzn-api-help sts
You can then inspect an individual operation:
amzn-api-help sts GetCallerIdentity
With AWS credentials available in the environment or by letting
Amazon::Credentials
discover them for you, making the call is straightforward:
perl -MAmazon::API::STS -MData::Dumper \
-e 'print Dumper(Amazon::API::STS->new->GetCallerIdentity);'
The result is returned as ordinary Perl data.
From there, install another generated service
distribution and use
amzn-api-help to discover its operations and request and response
shapes.
If you just want to try Amazon::API without installing anything
locally, a pre-built Docker image is available from Docker
Hub. Assuming you have
AWS credentials that can be discovered from your AWS CLI configuration
in $HOME/.aws:
docker run --rm \
-v "$HOME/.aws:/root/.aws:ro" \
-e AWS_PROFILE=my-profile \
rlauer/amazon-api:2.8.0 \
perl -MAmazon::API::STS -MData::Dumper \
-e 'print Dumper(Amazon::API::STS->new->GetCallerIdentity)'
If you would rather see exactly how the environment is assembled, you can build the environment from scratch using the bootstrap script below.
#!/usr/bin/env bash
# copy the script below as 'bootstrap' in your current directory...then:
# docker run --rm -it \
# -v "$(pwd)/bootstrap:/bootstrap:ro" \
# -v "$HOME/.aws:/root/.aws:ro" \
# debian:trixie /bin/bash /bootstrap
RESOLVER="--resolver 02packages,https://cpan.openbedrock.net/orepan2"
apt-get update && \
apt-get install -y --no-install-recommends \
perl libssl3 libexpat1 zlib1g ca-certificates gcc make \
libssl-dev libexpat-dev zlib1g-dev libperl-dev curl && \
curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm \
| perl - install -g App::cpm && \
apt-get clean && rm -rf /var/lib/apt/lists/*
cpm install -L /app/local $RESOLVER Amazon::API::STS
# Substitute your profile below
AWS_PROFILE=sandbox \
perl -I /app/local/lib/perl5 -MAmazon::API::STS -MData::Dumper \
-e 'print Dumper(Amazon::API::STS->new->GetCallerIdentity)';
…then from the command line:
docker run --rm -it \
-v "$(pwd)/bootstrap:/bootstrap:ro" \
-v "$HOME/.aws:/root/.aws:ro" \
debian:trixie /bin/bash /bootstrap
...
...
...
$VAR1 = {
'Arn' => 'arn:aws:iam::123456789012:user/sandbox-developer',
'Account' => '123456789012',
'UserId' => 'AIDA*************MTGW'
};
Previous post: Amazon::S3 v2.1.0 - On Being an S3 Client