Gitlab job / docker execution hangs on node command forever

Question

We are upgrading our Anguar project from v.17 to v.18.1.0. Everything works nicely, but we get a problem during our CI/CD pipelines.

I have the following commands to run as part of a gitlab pipeline:

image: node:21.4.0
...
    - npm run lint
    - npm run prettier

Since we upgraded to Angular 18.1, this script get stuck and never finishes (no exit code, it just hangs execution indefinetly).

In our gitlab logs:

Running with gitlab-runner 16.8.0 (c72a09b6)
  on docker-runner-63-48-#1 6Bbjxbz5, system ID: s_0f499b5c2941
Preparing the "docker" executor
00:03
Using Docker executor with image node:21.4.0 ...
Authenticating with credentials from /root/.docker/config.json
Pulling docker image node:21.4.0 ...
Using docker image sha256:5aeb975d4a5c53f930d848c6d40af3b11232bf7679acae74201c40d5136dc97c for node:21.4.0 with digest node@sha256:814a6dc5bfd4ecc5ef24652f6b4f2790e9f3552b52ee38a7b51fc4d4c0d6d7fd ...
Preparing environment
00:00
Running on runner-6bbjxbz5-project-705-concurrent-0 via oraclegitlab...
Getting source from Git repository
00:05
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/.../customer-webportal-frontend/.git/
Checking out b1c0df15 as detached HEAD (ref is feature/819-angular-update)...
Removing node_modules/
Removing package-lock.json
Skipping Git submodules setup
Downloading artifacts
00:25
Downloading artifacts for install (818083)...
Downloading artifacts from coordinator... ok        host=gitlab.....hu id=818083 responseStatus=200 OK token=glcbt-64
Executing "step_script" stage of the job script
54:34
Using docker image sha256:5aeb975d4a5c53f930d848c6d40af3b11232bf7679acae74201c40d5136dc97c for node:21.4.0 with digest node@sha256:814a6dc5bfd4ecc5ef24652f6b4f2790e9f3552b52ee38a7b51fc4d4c0d6d7fd ...
$ npm run lint
> [email protected] lint
> ng lint --fix
Node.js version v21.4.0 detected.
Odd numbered Node.js versions will not enter LTS status and should not be used for production. For more information, please see https://nodejs.org/en/about/previous-releases/.
Linting "customer-webportal-fe"...
All files pass linting.
(node:32) ExperimentalWarning: WASI is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
Terminated
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: script timeout context: context deadline exceeded

I tried to execute the same job with registering my PC as a runner (same settings, docker with node v.21.4.0). I get the same output: after the said ExperimentalWarning, it just hangs forever.

forest@forestg:~$ docker logs -f --tail 1000 1b4fcd4e2c32
$ npm run lint

> [email protected] lint
> ng lint --fix

Node.js version v21.4.0 detected.
Odd numbered Node.js versions will not enter LTS status and should not be used for production. For more information, please see https://nodejs.org/en/about/previous-releases/.

Linting "customer-webportal-fe"...

All files pass linting.

(node:32) ExperimentalWarning: WASI is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

If I run these commands manually on the project, directly on my same machine where I have the docker running, but without docker or gitlab-runner, everything is fine. (it executes both commands in ~5 seconds)

forest@forestg:~/projects/.../customer-webportal-frontend$ npm run lint && npm run prettier

> [email protected] lint
> ng lint --fix

Node.js version v21.4.0 detected.
Odd numbered Node.js versions will not enter LTS status and should not be used for production. For more information, please see https://nodejs.org/en/about/previous-releases/.

Linting "customer-webportal-fe"...

All files pass linting.


> [email protected] prettier
> pretty-quick

🔍  Finding changed files since git revision null.
🎯  Found 1 changed file.
✅  Everything is awesome!

The same happens if I upgrade the node version to 22.0.0. What can be the problem here?

It might be better to use the LTS version to eliminate the chances of non-LTS bugs happening. — Sergey
– Sergey, Commented Aug 11, 2024 at 12:16
Given that you don't get the "WASI" warning on your local machine, it seems like the likely cause. Figuring where it comes from and why you don't get it on the local machine might be the key. — Sergey
– Sergey, Commented Aug 11, 2024 at 12:19
It's also unclear where you get the node_modules. I can see that it pulls the artifacts which may be the cache of node_modules saved on a previous stage, but it's only my guess. It's worth checking if it would work without cache when you run npm install as part of the stage. — Sergey
– Sergey, Commented Aug 11, 2024 at 12:21

Tyler2P · Accepted Answer · 2024-08-15 19:10:50Z

2

I suggest trying a LTS node version like 18 or 20 image: node:18.x

npm run lint

npm run prettier

and setting a longer timeout " 1h" for example, other than that, try to show more logs, like using verbosity script:

npm run lint --verbose

npm run prettier --verbose

Or by using interactive session inside the docker container and send the logs, I hope this helps!

edited Aug 15, 2024 at 19:10

Tyler2P

2,37030 gold badges26 silver badges34 bronze badges

answered Aug 11, 2024 at 14:27

aladinebha

395 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ForestG Over a year ago

Thanks, I tried different node versions and verbose output, nothing changed, and there were no extra information in the logs. (Our timeout is already 60 minutes)

judx · Accepted Answer · 2024-08-08 13:46:41Z

1

You might be missing some necessary packages. Before running lint-command, ensure all dependencies are installed by running npm install like so:

image: node:21.4.0
steps:
  - npm install
  - npm run lint
  - npm run prettier

Hope this helps!

answered Aug 8, 2024 at 13:46

judx

4922 silver badges7 bronze badges

1 Comment

ForestG Over a year ago

Thanks, we made sure we have all these available - the npm install was an artifact from a different step, but we executed it in this step as well, same result unfortunately.

subdeveloper · Accepted Answer · 2024-08-10 18:03:39Z

1

I suspect there's some command that is expecting user/console input.

You have not posted your package.json but check try to check if lint command in that file is piping something other than ng --lint.

answered Aug 10, 2024 at 18:03

subdeveloper

2,69416 silver badges26 bronze badges

1 Comment

ForestG Over a year ago

thanks, but no, this is not the issue.

Jaap · Accepted Answer · 2024-09-04 11:26:27Z

1

I came across these issues on Github for angular-eslint:

Under certain circumstances the eslint process hangs after linting. It can be fixed by upgrading all @angular-eslint dependencies to version 17.5.3 or 18.

edited Sep 4, 2024 at 11:26

answered Sep 2, 2024 at 7:58

Jaap

7441 gold badge15 silver badges21 bronze badges

Comments

ForestG · Accepted Answer · 2024-08-14 13:29:16Z

0

None of the above answers helped. We disabled the caching steps for the given job, and the problem went away. We have 0 clue how can that relate to the problem - maybe shared cache between multiple gitlab runner machines can cause this phenomenom.

answered Aug 14, 2024 at 13:29

ForestG

18.2k16 gold badges64 silver badges94 bronze badges

Collectives™ on Stack Overflow

Gitlab job / docker execution hangs on node command forever

5 Answers 5

1 Comment

1 Comment

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

1 Comment

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related