Deploying to AWS Amplify and Lambda from Self Managed GitLab CI/CD

NOTE: Things have changed. See the latest setup in the updated blog post.

Wanting to automate all of our deployments, we had to “find a way”.

After some iteration, here are the final solutions. This assumes you have created the app in the AWS console and just want to be able to deploy new releases.

Amplify

AWS Amplify doesn’t have support for self managed GitLab instances yet. This issue is tracked on AWS Amplify Github.

Create An AWS IAM User For Amplify

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DeployAmplify",
            "Effect": "Allow",
            "Action": "amplify:StartDeployment",
            "Resource": "arn:aws:amplify:REGION:ACCOUNT_ID:apps/APP_ID/*"
        }
    ]
}

You will need to replace REGION, ACCOUNT_ID and APP_ID with the relevant values. You can also go further and limit to a specific branch if desired.

You may wish to grant your user less restrictive permission (for example, if you want the same credentials to be used to deploy multiple apps).

Configure GitLab CI/CD Variables

We chose to configure:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_REGION

Set them to be protected and masked. You could also define your APP_ID and BRANCH_NAME here if you wanted to.

Configure .gitlab-ci.yml

build-frontend:
  stage: build
  image: node:17.9.0-slim
  tags:
    - docker
  script:
    - apt-get update
    - apt-get install zip -y
    - cd web
    - npm install
    - npm run $ENVIRONMENT
    - date > dist/version.html
    - echo "JOB_ID=$CI_JOB_ID" > ../build.env
    - cd dist
    - zip -r ../../dist.zip *
  artifacts:
    paths:
      - dist.zip
    reports:
      dotenv: build.env
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: $CI_COMMIT_BRANCH == "master"
      variables:
        ENVIRONMENT: prod
    - if: $CI_COMMIT_BRANCH == "test" || $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "test"
      variables:
        ENVIRONMENT: test

deploy-frontend-amplify:
  stage: deploy
  image:
    name: amazon/aws-cli
    entrypoint: [""]
  tags:
    - docker
  variables:
    APP_ID: APP_ID
    SOURCE_URL: $CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/$JOB_ID/artifacts/dist.zip?job=build-web&job_token=$CI_JOB_TOKEN
  script:
    - aws amplify start-deployment --app-id $APP_ID --branch-name $BRANCH --source-url $SOURCE_URL
  needs:
    - build-frontend
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: $CI_COMMIT_BRANCH == "master"
      variables:
        BRANCH: master
    - if: $CI_COMMIT_BRANCH == "test" || $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "test"
      variables:
        BRANCH: test

You will need to replace APP_ID, BRANCH_NAME and SOURCE_URL with the relevant values.

That’s it!

Pipeline job succeeded Deployment success

It’s fairly straightforward, but there’s a tiny bit of “magic”.

AWS Amplify requires a publicly accessible URL to download your project. We can achieve this by:

  • Splitting the pipeline into multiple jobs.
  • Storing build artifacts and the build job ID.
  • Using the $CI_JOB_TOKEN to authorize the “private” artifact download endpoint.

Big shout out to Marco and Niklas from the GitLab community for helping us debug/troubleshoot and get this working.

While developing this solution we had some issues with AWS CLI:

root@a1b5517efdf2:/# aws amplify start-deployment --app-id abc123 --branch-name test --source-url "https://www.example.net/wp-content/uploads/2022/08/dist.zip"

Error parsing parameter '--source-url': Unable to retrieve https://www.example.net/wp-content/uploads/2022/08/dist.zip:
'utf-8' codec can't decode byte 0x9c in position 11: invalid start byte

Our solution initially was to use the AWS Ruby SDK:

require 'aws-sdk-amplify'

amplify = Aws::Amplify::Client.new
amplify.start_deployment({
  app_id: ENV["APP_ID"],
  branch_name: ENV["BRANCH_NAME"],
  source_url: ENV["SOURCE_URL"],
})

There may be a scenario where you would prefer to use the SDK, so we felt it was worth including this alternate approach.

It turned out we were using an old version of the CLI, and updating to latest version fixed the issue, allowing us to get rid of the custom Ruby solution.

Lambda

Setting up AWS Lambda is pretty much the same process as Amplify.

We will need to create another IAM user.

Create An AWS IAM User For Lambda

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DeployLambda",
            "Effect": "Allow",
            "Action": "lambda:UpdateFunctionCode",
            "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:PROJECT_NAME*"
        },
    ]
}

You will need to replace REGION, ACCOUNT_ID and PROJECT_NAME with the relevant values.

Configure GitLab CI/CD Variables

Similarly to Amplify, We chose to configure:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_REGION

Set them to be protected and masked. You could also define your APP_ID and BRANCH_NAME here if you wanted to.

Configure .gitlab-ci.yml for Lambda

deploy-backend-lambda:
  stage: deploy
  image: mcr.microsoft.com/dotnet/sdk:6.0
  tags:
    - docker
  script:
    - apt-get update
    - apt-get install awscli zip -y
    - cp $NUGET_CONFIG ./NuGet.Config
    - dotnet publish -c Release --self-contained false -r linux-arm64 -o publish
    - cd publish
    - zip -r ../publish.zip *
    - cd ..
    - aws lambda update-function-code --function-name $FUNCTION_NAME --zip-file fileb://publish.zip
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: $CI_COMMIT_BRANCH == "master"
      variables:
        FUNCTION_NAME: ExampleProject
    - if: $CI_COMMIT_BRANCH == "test" || $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "test"
      variables:
        FUNCTION_NAME: ExampleProjectTest

You will need to replace FUNCTION_NAME with the relevant values.

Pipeline job succeeded

That’s it.