[S3] Feature Request: Support S3 Access Points

[melya]

Hey 👋

Official aws-php-sdk supports presign of s3 access points, here is an example

$command = $awsSdk->getCommand(
   "GetObject",
   [
      "Bucket" => "arn:aws:s3-object-lambda:eu-central-1:000000000000:accesspoint/<accesspoint-name>",
      "Key" => "<object-key>"
   ]
);

$preSignedUrl = $awsSdk->createPresignedRequest($command, "+5 min")->getUri();

So, the bucket like arn:aws:s3-object-lambda:eu-central-1:000000000000:accesspoint/<accesspoint-name>
resolves into https://<accesspoint-name>-000000000000.s3-object-lambda.eu-central-1.amazonaws.com

---

*Real use-case: s3 lambda func generates a thumbnail by accessing the pre-signed URL

Would be great to have this functionality inside async-aws ❤️

Research:

• Access point compatibility with AWS services.
• Access point alias use cases and limitations.
• Using S3 access point ARNs the AWS SDK for PHP Version 3
• Amazon S3 Control: AccessPoint
• Use a presigned URL with Multi-Region Access Points
• AWS PrivateLink for Amazon S3

[jderusse]

did you tried https://async-aws.com/features/presign.html ?

[melya]

Sure, here is an example with using async-aws/s3

$client->presign(\AsyncAws\S3\Input\GetObjectRequest::create([
   "Bucket" => "arn:aws:s3-object-lambda:eu-central-1:000000000000:accesspoint/thumbnail-generator",
   "Key" => "Foo/HappyFace.jpg"
]));

Result of async-aws

https://s3.eu-central-1.amazonaws.com/arn%3Aaws%3As3-object-lambda%3Aeu-central-1%3A000000000000%3Aaccesspoint%2Fthumbnail-generator/Foo/HappyFace.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230622T102450Z&X-Amz-Expires=60&X-Amz-Credential=<accessKeySignature>&x-amz-content-sha256=UNSIGNED-PAYLOAD&X-Amz-SignedHeaders=host&X-Amz-Signature=c0fdd97e5b43239ef754d49304ffc9835617292e3c772bc67e0ba45ae796e099

Result of aws-sdk-php (expected)

https://thumbnail-generator-000000000000.s3-object-lambda.eu-central-1.amazonaws.com/Foo/HappyFace.jpg?X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=<accessKeySignature>&X-Amz-Date=20230622T102816Z&X-Amz-SignedHeaders=host&X-Amz-Expires=60&X-Amz-Signature=d25fc4c6c27ac025152b7691023607e7923bf22d5f503910497c74e53df5eb46

*Here is a original aws BucketEndpointArnMiddleware where actually hostname resolves depending on a Bucket

[melya]

This is last feature I miss to fully switch to async-aws, so I open for contribution )
I need some help here to understand how would you like to implement this from helicopter (architecture) view

[melya]

Kinda MVP with lots of hacks, but generates valid pre-signed url

<?php

declare(strict_types=1);

use AsyncAws\Core\Request;
use AsyncAws\Core\RequestContext;
use AsyncAws\Core\Stream\StreamFactory;
use AsyncAws\S3\Signer\SignerV4ForS3;

final class MVP
{
    private const ARN_PREFIX = "arn:aws:s3-object-lambda:";

    public function __construct(private readonly \AsyncAws\S3\S3Client $client)
    {
    }

    /**
     * @param array{
     *     Bucket: string,
     *     Key: string,
     *     ResponseContentDisposition?: string,
     *     ResponseContentType?: string,
     * } $input
     */
    public function __invoke(array $input, \DateTimeImmutable $expires): string
    {
        $this->assertSupports($input["Bucket"]);

        [$host, $region] = $this->arnToHost($input["Bucket"]);

        $headers = ['content-type' => 'application/xml'];
        if (isset($input["ResponseContentDisposition"])) {
            $query['response-content-disposition'] = $input["ResponseContentDisposition"];
        }

        if (isset($input["ResponseContentType"])) {
            $query['response-content-type'] = $input["ResponseContentType"];
        }

        $endpoint = "{$host}/"
            . \str_replace('%2F', '/', \rawurlencode(\ltrim($input["Key"], "/")))
            . "?"
            . \http_build_query($query, '', '&', \PHP_QUERY_RFC3986);

        $request = new Request('GET', '', $query, $headers, StreamFactory::create(''));
        $request->setEndpoint($endpoint);

        $r = new \ReflectionProperty(\AsyncAws\Core\AbstractApi::class, "credentialProvider");
        $r->setAccessible(true);
        $credentialProvider = $r->getValue($this->client);

        if (null !== $credentials = $credentialProvider?->getCredentials($this->client->getConfiguration())) {
            $signer = new SignerV4ForS3("s3-object-lambda", $region);
            $signer->presign($request, $credentials, new RequestContext(['expirationDate' => $expires]));
        }

        return $request->getEndpoint();
    }

    private function assertSupports(string $arn): void
    {
        if (0 !== \strpos($arn, self::ARN_PREFIX)) {
            throw new \LogicException("Only s3-object-lambda can be provided");
        }
    }

    /** @return array{string, string} */
    private function arnToHost(string $arn): array
    {
        $arn   = \substr($arn, \strlen(self::ARN_PREFIX));
        $parts = \explode(":", $arn, 3);
        if (3 !== \count($parts)) {
            throw new \LogicException("Invalid s3-object-lambda ARN provided");
        }
        $region      = $parts[0];
        $accountId   = $parts[1];
        $accessPoint = \explode("/", $parts[2], 2)[1] ?? $parts[2];

        return ["https://{$accessPoint}-{$accountId}.s3-object-lambda.{$region}.amazonaws.com", $region];
    }
}

[melya]

Hi @jderusse!
I've collected all needed aws documentation in the description (below Research)

Looking forward to your response. Thanks!

[stof]

Have you tried using the access endpoint alias instead of it's ARN ? From my quick reading of the documentation links you provided, I think it might work already.

[melya]

Have you tried using the access endpoint alias instead of it's ARN ? From my quick reading of the documentation links you provided, I think it might work already.

Thanks for your reply!
Yeah, alias works good 👍

Looking back to my feature request - it should be converted into support of Access Points (e.g. Outposts, ObjectLambdaAccessPoint, etc.) out of the box.

Good idea here to mention in the documentation that currently there is no support of S3 Access Points to avoid further issues and misunderstanding

---

Also, in my scenario with thumbnail generator - I provide a custom query parameter size which is used to generate thumbnails on the fly with needed size.
Currently, there is no ability to do this with GetObjectRequest - open for ideas how this could be handled 👀

I've made a workaround and created own GetObjectRequest which accepts CustomQuery?: array<string, string>

final class GetObjectRequest extends Input
{
    /**
     * @param array{
     *   Bucket?: string,
     *   Key?: string,
     *   ...
     *   CustomQuery?: array<string, string>,
     *
     *   @region?: string,
     * } $input
     */
    public function __construct(array $input = [])
    ... 
    public function request(): Request
    {
       ...
       $query += $this->customQuery ?? [];
       
       return new Request('GET', $uriString, $query, $headers, StreamFactory::create($body));
    }
}

[stof]

Is passing custom query parameters supported by the official SDK ?

[stof]

Good idea here to mention in the documentation that currently there is no support of S3 Access Points to avoid further issues and misunderstanding

PRs updating the documentation are welcome.

[melya]

Is passing custom query parameters supported by the official SDK ?

Unfortunately No :)
With official php-sdk it can be done only by adding custom middleware which will append the query later
But in async-aws there is no such possibility at all

[melya]

Good idea here to mention in the documentation that currently there is no support of S3 Access Points to avoid further issues and misunderstanding

PRs updating the documentation are welcome.

👍 I will create a PR later