Hard Coded canned ACL parameter on S3.upload in uploadfs/s3.js copyIn method

[shaunhurley]

Hi Team,

Just spent a chunk of time working through an access issue, thought I would share the context. I don't want to have my website assets bucket publicly accessible directly from the bucket. I also don't want Apostrophe linking to images directly via the bucket URL, nor via HTTP (since it causes HTTPS sites to show up as only 'paritally secure' in the browser)

As such my S3 assets bucket has public access turned off, and is being presented through Cloudfront with access restricted via a Cloudfront "Origin Access Identity" to enable public access via HTTPs:// URLs. Apostrophe is configured with an IAM identity (key / secret) to allow uploadfs to process files in and with the options 'https' and 'cdn' parameters in the app.js file apostrophe-attachments/uploadfs module configuration section.

    'apostrophe-attachments': {
      uploadfs: {
        https: true,
        cdn: {
          enabled: true,
          url: 'https://assets.urbanaxes.com'
        },
        backend: 's3',
        secret: '<IAM Secret>',
        key: '<IAM key>',
        bucket: 'assets.urbanaxes.com',
        region: 'us-east-1'
      }
    }

This configuration results in a generic S3 'AccessDenied' error being returned from Apostrophe when trying to upload files through the CMS.

After investigation and verification that the credentials were working, I determined that you cannot apply the 'public-read' canned ACL to an objects in buckets that have all public access restricted.

Uploadfs currently has the 'public-read' ACL hard coded into the client.upload call params in the s3.js copyIn method (circa line 91 or so).

    var params = {
        ACL: 'public-read',
        Key: cleanKey(path),
        Body: inputStream,
        ContentType: contentType
    };

While I haven't exhaustively tested all the available ACL options, changing the ACL to 'private' successfully resolved the issue / prevented the error from recurring.

Longer term, enabling the ACL to be configurable, via the module configuration and possibly via environment variable, would seem like a reasonable approach?

Thanks,

Shaun

[boutell]

Wouldn't this mean uploadfs can't do its job (serving public media)?

…

On Fri, Apr 17, 2020 at 11:50 AM shaunhurley ***@***.***> wrote: Hi Team, Just spent a chunk of time working through an access issue, thought I would share the context. I don't want to have my website assets bucket publicly accessible directly from the bucket. I also don't want Apostrophe linking to images directly via the bucket URL, nor via HTTP (since it causes HTTPS sites to show up as only 'paritally secure' in the browser) As such my S3 assets bucket has public access turned off, and is being presented through Cloudfront with access restricted via a Cloudfront "Origin Access Identity" to enable public access via HTTPs:// URLs. Apostrophe is configured with an IAM identity (key / secret) to allow uploadfs to process files in and with the options 'https' and 'cdn' parameters in the app.js file apostrophe-attachments/uploadfs module configuration section. 'apostrophe-attachments': { uploadfs: { https: true, cdn: { enabled: true, url: 'https://assets.urbanaxes.com' }, backend: 's3', secret: '<IAM Secret>', key: '<IAM key>', bucket: 'assets.urbanaxes.com', region: 'us-east-1' } } This configuration results in a generic S3 'AccessDenied' error being returned from Apostrophe when trying to upload files through the CMS. After investigation and verification that the credentials were working, I determined that you cannot apply the 'public-read' canned ACL to an objects in buckets that have all public access restricted. Uploadfs currently has the 'public-read' ACL hard coded into the client.upload call params in the s3.js copyIn method (circa line 91 or so). var params = { ACL: 'public-read', Key: cleanKey(path), Body: inputStream, ContentType: contentType }; While I haven't exhaustively tested all the available ACL options, changing the ACL to 'private' successfully resolved the issue / prevented the error from recurring. Longer term, enabling the ACL to be configurable, via the module configuration and possibly via environment variable, would seem like a reasonable approach? Thanks, Shaun — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#55>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27JNRFNLR4ATSM4PKWLRNB3F3ANCNFSM4MK3FYBA> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[shaunhurley]

Not sure I'm understanding you here (or maybe I'm misunderstanding the intent of uploadfs) - the content being uploaded through uploadfs is still publicly accessible, it just has to be accessed through a HTTPS Cloudfront distribution, and not via HTTP / directly from an S3 URL.

These links are all publicly accessible assets (in that there is authentication requirement in front of them) through our Cloudfront distribution:

• https://assets.urbanaxes.com/
• https://assets.urbanaxes.com/attachments/ck94dktx1001384k03jztiqgv-durham.full.jpg
• https://assets.urbanaxes.com/attachments/ck94diqno000b84k09q4q9yzb-boston.full.jpg
• https://assets.urbanaxes.com/attachments/ck94djbaa000p84k0ahmpvpjt-baltimore.full.jpg

vs. a direct S3 http link on a public bucket

• http://urbanaxes.com.s3.us-east-1.amazonaws.com/images/boston.jpg

[boutell]

Ah right, I see why that is a reasonable thing to want to do.

…

On Fri, Apr 17, 2020 at 12:40 PM shaunhurley ***@***.***> wrote: Not sure I'm understanding you here (or maybe I'm misunderstanding the intent of uploadfs) - the content being uploaded through uploadfs is still publicly accessible, it just has to be accessed through a HTTPS Cloudfront distribution, and not via HTTP / directly from an S3 URL. These links are all publicly accessible assets (in that there is authentication requirement in front of them) through our Cloudfront distribution: - https://assets.urbanaxes.com/ - https://assets.urbanaxes.com/attachments/ck94dktx1001384k03jztiqgv-durham.full.jpg - https://assets.urbanaxes.com/attachments/ck94diqno000b84k09q4q9yzb-boston.full.jpg - https://assets.urbanaxes.com/attachments/ck94djbaa000p84k0ahmpvpjt-baltimore.full.jpg vs. a direct S3 http link on a public bucket - http://urbanaxes.com.s3.us-east-1.amazonaws.com/images/boston.jpg — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#55 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27IB3SAAXYB7V3RWPZLRNCBBVANCNFSM4MK3FYBA> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[boutell]

Yes making it configurable is reasonable. It doesn't need to be publicly readable for cloudfront to pick it up and make it publicly readable?

…

On Fri, Apr 17, 2020 at 1:31 PM Tom Boutell ***@***.***> wrote: Ah right, I see why that is a reasonable thing to want to do. On Fri, Apr 17, 2020 at 12:40 PM shaunhurley ***@***.***> wrote: > Not sure I'm understanding you here (or maybe I'm misunderstanding the > intent of uploadfs) - the content being uploaded through uploadfs is still > publicly accessible, it just has to be accessed through a HTTPS Cloudfront > distribution, and not via HTTP / directly from an S3 URL. > > These links are all publicly accessible assets (in that there is > authentication requirement in front of them) through our Cloudfront > distribution: > > - https://assets.urbanaxes.com/ > - > https://assets.urbanaxes.com/attachments/ck94dktx1001384k03jztiqgv-durham.full.jpg > - > https://assets.urbanaxes.com/attachments/ck94diqno000b84k09q4q9yzb-boston.full.jpg > - > https://assets.urbanaxes.com/attachments/ck94djbaa000p84k0ahmpvpjt-baltimore.full.jpg > > vs. a direct S3 http link on a public bucket > > - http://urbanaxes.com.s3.us-east-1.amazonaws.com/images/boston.jpg > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#55 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAH27IB3SAAXYB7V3RWPZLRNCBBVANCNFSM4MK3FYBA> > . > -- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[shaunhurley]

Yes making it configurable is reasonable. It doesn't need to be publicly readable for cloudfront to pick it up and make it publicly readable?

Nope. Making it private just means that an explicit (internal) authentication is required to retrieve objects from the bucket, not to serve it up in response to a request. Cloudfront receives the original anonymous request(s) from a browser. If the asset is cached, job done. If the result is not already cached, while while retrieving the requested asset from the S3 bucket, it attaches a configured AWS 'Origin Access Identity' to the request. The bucket has a policy statement that explicitly allows the OAI to retrieve objects, per below, allowing the object to be retrieved.

        {
            "Sid": "CloudfrontOAIS3AccessPolicy",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity <AAAAAAAAAAAA>"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::assets.urbanaxes.com/*"
        },

[boutell]

Are you interested in submitting a PR for this one?

[shaunhurley]

I can take a run at it, however can you point me in the direction of the module that handles parsing environment variables? I was mainly digging directly into uploadfs and hadn't got that far...

[boutell]

uploadfs should deal with it as an option rather than an environment variable, but you could add support for getting it from an environment variable in the apostrophe-attachments module itself (inside the apostrophe npm module). See the existing APOS_S3_* env vars.

…

On Mon, Apr 20, 2020 at 10:11 AM shaunhurley ***@***.***> wrote: I can take a run at it, however can you point me in the direction of the module that handles parsing environment variables? I was mainly digging directly into uploadfs and hadn't got that far... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#55 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27IE7U32ZRDMQCSUXNDRNRJYBANCNFSM4MK3FYBA> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[shaunhurley]

Are you interested in submitting a PR for this one?

Pull request submitted on uploadfs to address the issue via providing an option, I'll take a look at the environment variable stuff when I get a few minutes spare.