AWS Serverless framework, stack deploy stuck "CREATE_IN_PROGRESS"

Question

I have a problem while using the Serverless framework to deploy my stack to AWS.

I'm running this serverless deploy --stage tst --region eu-west-3 --verbose

Then 2 of my functions get stuck in "CREATE_IN_PROGRESS" then 1 hour later, I have a "UPDATED_FAILED_ROLLBACK_IN_PROGRESS" and "UPDATED_FAILED_ROLLBACK_COMPLETED" status.

 CREATE_IN_PROGRESS - Custom::S3 - MyFunctionDataListenerLambdaCustomS31
 CREATE_IN_PROGRESS - Custom::S3 - MyFunctionData2ListenerLambdaS31

Here is one of my yaml function definition :

MyFunctionDataListenerLambda:
  name: myapp-${sls:stage}-entry-data-listener
  description: 'Triggered by S3 event and start a new step function execution'
  handler: src/features/entry/infrastructure/lambda/myapp-files-listener/handler.handle
  role: MyAppDataListenerLambdaRole
  layers:
    - ${ssm:/${sls:stage}/utils-lambda-layer-arn}
  events:
    - s3:
        bucket: myapp-${sls:stage}-data
        event: s3:ObjectCreated:*
        existing: true
        forceDeploy: true
        rules:
          - prefix: xxx/
          - suffix: .csv
  environment:
    ONE: ${self:custom.redshift.stage.${sls:stage}}
    TWO: myapp-${self:custom.redshift.stage.${sls:stage}}-redshift-cluster
    THREE: myapp
    FOUR: ${self:custom.redshift.stage.${sls:stage}}_myapp
    FIVE: arn:aws:states:${aws:region}:${aws:accountId}:stateMachine:myapp-${sls:stage}-workflow
    SIX: myapp-${sls:stage}-myapp/data

The other function is almost identical

So this problem occurs when I put the node "events"

    - s3:
        bucket: myapp-${sls:stage}-data
        event: s3:ObjectCreated:*
        existing: true
        forceDeploy: true
        rules:
          - prefix: xxx/
          - suffix: .csv`

when I remove it, everything works, the Role "MyAppDataListenerLambdaRole" is defined as follow in another file included in the serverless.yaml

Resources:
  MyAppDataListenerLambdaRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: myapp-${sls:stage}-listener-role
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: myapp-${sls:stage}-listener-policy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - states:StartExecution
                Resource: arn:aws:states:${aws:region}:${aws:accountId}:stateMachine:myapp-${sls:stage}-workflow
              - Effect: Allow
                Action:
                  - s3:GetObject
                  - s3:PutObject
                  - s3:ListBucket
                Resource:
                  - arn:aws:s3:::myapp-${sls:stage}-data
                  - arn:aws:s3:::myapp-${sls:stage}-data/*

I tried to put in my configuration

provider: deploymentMethod: direct # also tried with changesets still stuck

I don't find anything useful online, this is the first time i'm writing here because I'm really stuck with that, please tell me if you need more information. The S3 bucket is created on another project with the CDK

In the CloudFormation events I have "CloudFormation did not receive a response from your Custom Resource."

Normally, serverless outputs some messages if an error occured. Did it just exit? Did you check the Cloudformation console in AWS? Select the deployed stack and check under "Events" and "Ressources" for hints about failures. — SAE
– SAE, Commented Jul 25, 2024 at 16:21
The only output I have from the console in events and ressources is "CREATE_FAILED" and "CloudFormation did not receive a response from your Custom Resource." I have also a requestId but I don't know where to find the associated log — JeanPierre
– JeanPierre, Commented Jul 25, 2024 at 19:01
Log into the AWS Console. From there, go to the CloudFormation console. Also, there's another question with a similar problem: stackoverflow.com/a/73373300/577483 Maybe the Lambda doesn't work as expected. — SAE
– SAE, Commented Jul 25, 2024 at 19:56
Yes everything seems fine on my side, the lambda is deployed and work without the "events" node in the configuration. It's really just when I add the "events" node the deploy is stuck during one hour then I have the CREATE_FAILED with the error CloudFormation did not receive a response from your Custom Resource." — JeanPierre
– JeanPierre, Commented Jul 26, 2024 at 12:22

JeanPierre · Accepted Answer · 2024-07-26 15:04:51Z

0

Ok I found the answer, it is answered starting here https://github.com/serverless/serverless/issues/6771#issuecomment-1916211820 so when we puts the events node, serverless create a specific lambda to do it named "custom-resource-existing-s3" using node as runtime. But if we force the node version higher than 16, the aws-sdk will not be available in this lambda and this will cause the bug. The issue is on the framework side. I was using a custom plugin to force the custom-resource-existing-s3 lambda to use node 20 after disabling it, it works.

answered Jul 26, 2024 at 15:04

JeanPierre

11 bronze badge

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

AWS Serverless framework, stack deploy stuck "CREATE_IN_PROGRESS"

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related