It took some time, but I got self hosted NIP-05 verification working today using AWS Amplify and a Jekyll static site. These are the three key files and contents, all are in the site root directory:

amplify.yml

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - bundle install
    build:
      commands:
        - bundle exec jekyll b
  artifacts:
    baseDirectory: _site
    files:
      - '**/*'
  cache:
    paths: []

customHttp.yml

customHeaders:
  - pattern: '/.well-known/*'
    headers:
      - key: 'Access-Control-Allow-Origin'
        value: '*'

and add this to _config.yml:

include:
  - .well-known