Connection refused when using Ocelot as API gateway in docker compose

I'm playing around with docker and I'm currently trying to set up some microservices with multiple .net 8 API's, a gateway and an angular frontend.

I got 2 api's up and running and working without issue. I also have an angular app running that can do calls to these API's and get data.

Now I wanted to add a gateway in between my API's and the frontend. I decided to use Ocelot for this.

So added this ocelot.json file to the project and did the rest of the setup as it should work.

{
  "Routes": [
    {
      "DownstreamPathTemplate": "/api/v1/todos/tasks",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "todo.api",
          "Port": "8080"
        }
      ],
      "UpstreamPathTemplate": "/api/todos",
      "UpstreamHttpMethod": [ "Get" ]
    },
    {
      "DownstreamPathTemplate": "/api/v1/food/recipes",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "food.api",
          "Port": "5010"
        }
      ],
      "UpstreamPathTemplate": "/api/recipes",
      "UpstreamHttpMethod": [ "Get" ]
    }
  ],
  "Aggregates": [],
  "GlobalConfiguration": {
    "BaseUrl": "http://localhost:5900"
  }
}

I then added everything to my docker compose file

version: '3.4'

services:
  todo.api:
    image: todoapi
    container_name: todo.api
    build:
      context: ./TestProject.Service.Todo
      dockerfile: ./Api/Dockerfile
    ports:
      - "5000:8080"
      - "5001:8081"
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
  todo.database:
    image: postgres:latest
    container_name: todo.database
    environment:
      - POSTGRES_DB=todos
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    volumes:
      - ./.containers/todos-db:/var/lib/postgresql/data
    ports:
      - 5432:5432
  gateway.web:
    image: gatewayweb
    container_name: gateway.web
    build:
      context: ./TestProject.Gateway.Web
      dockerfile: ./Api/Dockerfile
    ports:
      - "5900:8080"
      - "5901:8081"
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
  dbbeaver:
    image: dbeaver/cloudbeaver
    ports: ["8000:8978"]
    hostname: gui
    container_name: dbeaver-container    

  food.api:
    image: foodapi
    container_name: food.api
    build:
      context: ./TestProject.Service.Food
      dockerfile: ./Api/Dockerfile
    ports:
      - "5010:8080"
      - "5011:8081"
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
  food.database:
    image: postgres:latest
    container_name: food.database
    environment:
      - POSTGRES_DB=food
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    volumes:
      - ./.containers/food-db:/var/lib/postgresql/data
    ports:
      - 3366:5432
  frontend:
    image: frontend
    build:
      context: ./TestProject.Frontend/TestProject.frontend
      dockerfile: ./dockerfile
    ports:
      - 4200:80

When I run this everything starts up without issues, but when I try to get the data through the gateway url, I get the following error:

 requestId: 0HN3GO8DRCRJT:00000002, previousRequestId: No PreviousRequestId, message: 'Error Code: ConnectionToDownstreamServiceError Message: Error connecting to downstream service, exception: System.Net.Http.HttpRequestException: Connection refused (food.api:5010)

       ---> System.Net.Sockets.SocketException (111): Connection refused

         at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)

         at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)

         at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)

         at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)

         --- End of inner exception stack trace ---

         at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)

         at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)

         at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)

         at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)

         at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)

         at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)

         at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)

         at Ocelot.Requester.TimeoutDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)

         at Ocelot.Requester.MessageInvokerHttpRequester.GetResponse(HttpContext httpContext) errors found in ResponderMiddleware. Setting error response for request path:/api/recipes, request method: GET'

I first thought I did something wrong in my ocelot setup but when I try this without docker and run the projects in debug mode, the gateway works just fine.

I also tried using localhost:port instead of the name of the service but that just gave me another error:

"Cannot assign requested address (localhost:XXXX)."

I currently have no clue at what could be the cause of this issue, hoping someone experienced something similar and found a solution!