Credit: I want to believe...

Today I’m releasing version 2.8.0 of Amazon::API. This version of Amazon::API is a significant milestone. It represents the first release in a series of planned releases that seeks to achieve parity with the Python Botocore library.

Botocore library parity means that Amazon::API can consume the same Botocore service metadata and implement the same low-level AWS client behaviors needed to construct, sign, send, and interpret requests correctly.

It does not mean reproducing every convenience API exposed by Boto3. Features such as waiters, transfer managers, resources, and higher-level helpers only require that Amazon::API expose the underlying primitives needed to implement them.

Why another AWS API for Perl?

Amazon::API is built around a different idea from a traditional generated SDK: keep the runtime small, keep AWS service definitions independently updatable, and return ordinary Perl data instead of building a large generated object model around every service.

That separation is particularly important in an ecosystem where AWS service definitions change constantly. A new operation, shape, or pagination rule does not necessarily require a new Amazon::API release. Individual service distributions can instead be regenerated against newer Botocore metadata, allowing the AWS API definitions to remain current without coupling their release cycle to that of the runtime.

Paws and Amazon::API both make use of Botocore metadata, but they package and expose that information differently. Paws generates a broad Perl object model representing AWS services, requests, responses, and associated types. Amazon::API uses generated service definitions with a comparatively small runtime and normally returns plain Perl data structures.

Because those generated service distributions are independent of the runtime, an application can install only the AWS services it actually needs and update those definitions independently as AWS evolves.

Amazon::API is therefore aimed at Perl applications where a lightweight, data-oriented interface, independently current AWS service definitions, and a small production footprint are important.

Current Status

Capability Status Notes
Service/operation model support Implemented Generated from Botocore service metadata; independently updatable
Request serialization/deserialization Implemented Existing protocols supported; Smithy RPC protocols are planned separately
Endpoint rule-set resolution Implemented 2.8.0 milestone; modeled endpoint context and auth metadata supported for SigV4
SigV4 signing Implemented Includes modeled signing name, signing region, and disableDoubleEncoding
SigV4a Planned (2.9.0) Required for broader modern AWS endpoint/auth parity
S3 compatibility Planned Much closer after 2.8.0; full support depends on remaining auth and service-specific behavior
S3 Express Planned (2.10.0) Requires session auth and v4-s3express support
Smithy RPC protocols Planned (3.0.0) Includes rpcv2Cbor and related protocol behavior
Pagination Implemented Model-driven/compiler-backed
Checksums / payload-signing behavior Planned (3.0.0) Requires parity audit against Botocore handlers and protocol traits
Streaming / event streams Planned (3.0.0) Needed for applicable Smithy and streaming operations
Botocore handler semantics Planned (3.0.0) Systematic audit needed to identify remaining request/response mutations
Credential provider behavior Out of scope Owned by Amazon::Credentials; parity should be evaluated separately
Retry policy Out of scope Execution policy rather than request/protocol correctness
Waiters / transfer helpers / resources Out of scope Amazon::API should expose the primitives needed to implement them above the core runtime

Tentative Roadmap

Version Feature/Capability Target Date Status
2.8.0 Endpoint rule sets + modeled SigV4 signing 9/7 Released
2.9.0 SigV4a 9/26 -
2.10.0 S3 Express 10/16 -
3.0.0 Smithy protocol work + handler audit/fixes. 11/27 -

Testing Status

Amazon::API is tested at several different levels because no single test strategy is sufficient for a runtime that interprets Botocore service metadata.

The ordinary unit and regression suite exercises specific runtime behavior: request construction, serialization, deserialization, pagination, URI handling, endpoint context evaluation, service hooks, and signing.

A second layer uses protocol test cases derived from the Botocore test corpus. These tests provide known request and response examples across the AWS protocol families and allow Amazon::API behavior to be checked against expectations derived from Botocore.

The corpus is not treated as exhaustive. When testing a real AWS service exposed a behavior or edge case that was not represented by the existing corpus, I added a focused regression test or custom corpus case.

Finally, selected requests are validated against live AWS services. This is particularly useful for endpoint resolution and signing, where a request can appear correct locally but still be rejected by AWS.

The current 2.8.0 testing picture looks like this:

Test Area Status Notes
Unit and regression suite Passing Covers core runtime behavior including serialization, deserialization, pagination, request construction, URI handling, endpoint context, service hooks, and signing
Botocore protocol corpus Passing 671 tests across 15 protocol test files
Pagination Passing Dedicated coverage for model-driven paginator compilation and runtime evaluation
Endpoint rule-set evaluation Passing Covers rule evaluation, endpoint context sources and precedence, scalar template handling, and service endpoint selection
S3Control endpoint and URI behavior Passing Covers ARN-driven endpoint context, region selection, and percent encoding of non-greedy URI labels
S3 request construction Passing Covers virtual-hosted and force-path-style addressing, the S3 service hook, and final request URL composition
S3 greedy URI handling Passing Covers {Key+} behavior, preserving / while percent encoding other reserved characters
S3 response deserialization Passing Covers the rest-xml root-unwrapping behavior that previously affected some S3 responses
Kinesis endpoint resolution Passing ListShards with StreamARN resolves to the account-specific control endpoint
SigV4 auth propagation Passing Endpoint-derived signing name, signing region, and disableDoubleEncoding are propagated through the signing stack
Live AWS validation Selective Used for endpoint and signing behavior where AWS itself provides the final validation
Custom regression coverage Ongoing Added whenever real service behavior exposes a case not represented by the existing Botocore corpus

Passing tests do not prove that every AWS operation is correct. What they do provide is a repeatable way to increase confidence and coverage: Botocore-derived corpus tests establish broad protocol behavior, focused regressions preserve specific fixes and service behaviors, and live AWS calls validate the cases where the service itself is the final authority.

Try it

The easiest way to try Amazon::API 2.8.0 is with STS and GetCallerIdentity.

Amazon::API and Amazon::API::Help are available from CPAN. Generated AWS service distributions are available from the Amazon::API DarkPAN at:

https://cpan.openbedrock.net/orepan2

See https://cpan.openbedrock.net/signature for details on verifying the authenticity of Amazon::API service classes.

For example, using cpm:

cpm install Amazon::API
cpm install Amazon::API::Help
cpm install --resolver 02packages,https://cpan.openbedrock.net/orepan2 Amazon::API::STS

Once installed, Amazon::API::Help can show you the operations available for a service:

amzn-api-help sts

You can then inspect an individual operation:

amzn-api-help sts GetCallerIdentity

With AWS credentials available in the environment or through your normal credential configuration, making the call is straightforward:

perl -MAmazon::API::STS -MData::Dumper \
  -e 'print Dumper(Amazon::API::STS->new->GetCallerIdentity);'

The result is returned as ordinary Perl data.

From there, substitute another generated service distribution and use amzn-api-help to discover its operations and request and response shapes.

Try it using Docker

#!/usr/bin/env bash
# copy the script below as 'bootstrap' in your current directory...then:
# docker run --rm -it \
#   -v "$(pwd)/bootstrap:/bootstrap:ro" \
#   -v "$HOME/.aws:/root/.aws:ro" \
#   debian:trixie /bin/bash /bootstrap

RESOLVER="--resolver 02packages,https://cpan.openbedrock.net/orepan2"

apt-get update && \
apt-get install -y --no-install-recommends \
    perl libssl3 libexpat1 zlib1g ca-certificates gcc make  \
    libssl-dev libexpat-dev zlib1g-dev libperl-dev curl && \
    curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm \
        | perl - install -g App::cpm && \
    apt-get clean && rm -rf /var/lib/apt/lists/*

cpm install -L /app/local $RESOLVER Amazon::API::STS

# Substitute your profile below
AWS_PROFILE=sandbox \
perl -I /app/local/lib/perl5 -MAmazon::API::STS -MData::Dumper \
     -e 'print Dumper(Amazon::API::STS->new->GetCallerIdentity)';

…then from the command line:

docker run --rm -it \
  -v "$(pwd)/bootstrap:/bootstrap:ro" \
  -v "$HOME/.aws:/root/.aws:ro" \
  debian:trixie /bin/bash /bootstrap
 ...
 ...
 ...
$VAR1 = {
          'Arn' => 'arn:aws:iam::123456789012:user/sandbox-developer',
          'Account' => '123456789012',
          'UserId' => 'AIDA*************MTGW'
        };

Where this goes next

Version 2.8.0 is an important milestone, but it is not the end of the work.

The immediate roadmap is clear: SigV4a, S3 Express support, the remaining Smithy protocol work, and a broader audit of Botocore behavior. The goal is not to reproduce Boto3 in Perl. It is to make Amazon::API a reliable Perl runtime for the same AWS service descriptions that underpin Botocore.

That distinction matters because Amazon::API takes a different approach to building an AWS SDK.

  • The runtime is small.
  • AWS service definitions can be generated independently.
  • Applications install only the services they need.
  • Requests and responses remain ordinary Perl data.
  • Documentation can exist where developers need it without becoming part of every production deployment.

Those choices did not appear all at once. Amazon::API started nearly nine years ago as something considerably simpler, and adopting Botocore metadata in 2022 changed both the project and the way I think about providing AWS APIs for Perl.

That story deserves its own post.

For now, Amazon::API 2.8.0 is available on CPAN. It is the first release where I am comfortable calling the Botocore-based architecture the foundation for what comes next.


Next post: Reflecting on Mirrors - Part I

Previous post: Phoenix Rising: Why Perl’s Unix Roots Matter More Than Ever