2

We had been having troubles with gitlab runner for couple of days now. Whenever we try to deploy a commit, we are getting the following error (it's the full log):

Running with gitlab-runner 10.8.0 (079aad9e) on LST GitLab Runner 9db4eb2c Using Docker executor with image alpine:3.7 ...

Pulling docker image alpine:3.7 ...

Using docker image sha256:6d1ef012b5674ad8a127ecfa9b5e6f5178d171b90ee462846974177fd9bdd39f for alpine:3.7 ...

Running on runner-9db4eb2c-project-2-concurrent-0 via a7dcae9d7882...

Fetching changes...

HEAD is now at 142ba7a

Merge branch '3378-new-payroll-export-lexware' into test

error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error

fatal: The remote end hung up unexpectedly

ERROR: Job failed: exit code 1

It happens right after pulling docker image. We also tested it on different branches and different stages (we have test and deploy), all of them result with the same error.

Not sure if it's relevant, but we also cannot preform pull request from git repository. It fails with following error:

RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error

The remote end hung up unexpectedly

We tried switching to SSH instead of HTTP, but this also didn't work. Could you please point us in right direction so we can solve it? What could be the problem here?

Improve this question

4 Answers 4

Reset to default
4

The whole problem was connected to disk space. After logging in via SSH, we found out that docker container accumulated a huge log, that took all the space. We removed it and everything was fixed

I guess next step to avoid that problem in the future will be enabling docker to automaticly rotate the logs

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

Check if Increasing the git buffer size to the largest individual file size of your repo solves the problem, for example-

git config --global http.postBuffer 157286400

Ref: https://stackoverflow.com/a/52487332/737440

Also ensure that your server hard drive is not out of space.

Improve this answer

1 Comment

http.postbuffer only works on pushes, not pulls. In addition, the Git documentation recommends against setting it unless you're sure you know you need it.
0

The HTTP 500 error means that you've received an internal server error: the server has encountered a problem and can't continue. Since it's insecure to tell the user what the problem is, the only indication of what went wrong will be in the logs.

If you're using a self-hosted GitLab instance, you should check the logs to figure out what's going on. If you're using gitlab.com, you'd want to open a support ticket with them to see what's happening.

Note that it may be helpful (or not) to run the clone or fetch with GIT_TRACE=1 GIT_CURL_VERBOSE=1, which will print information about the HTTP operation and any response that's being provided.

Improve this answer

2 Comments

Thank you, we ended up opening ticket with gitlab.com I wanted to try enabling trace, but since yesterday someone tried to fix gitlab by restarting it and now it returns error 502...
Thanks again for the answer, problem was solved. I'll post additional comment about how we fixed it
0

In our case, it was Unicorn that was hung. Restarting it solved the issue but it took us a whole day to figure it out.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.