SignatureDoesNotMatch Errors When Using Pre-Signed URLs with MinIO in Docker and Node JS

I am running MinIO inside a Docker Compose setup and using the minio-js SDK to generate pre-signed URLs for accessing files. However, when I attempt to access these URLs, I encounter SignatureDoesNotMatch errors, depending on whether I use minio or localhost in the URL. I've tried multiple approaches, including configuring a custom network in Docker, but the issue persists. I need help understanding why these errors occur and how to resolve them.

I'm working on a project using MinIO to store and serve files. I'm running MinIO inside a Docker container and trying to generate pre-signed URLs for objects stored in a MinIO bucket. The URLs are being generated successfully, but when I try to access the objects using these URLs, I encounter issues.

Details:

• MinIO Setup:

  • MinIO is running inside a Docker container on my local machine.
  • The MinIO service is exposed on the port 9000 and the console on the port 9001.
  • I've configured the MinIO client (mc) to connect to the MinIO server and list and upload objects without issues.

• Docker Compose Configuration:

  • My Docker Compose file defines several services: minio, a Node.js app (projectk), MongoDB, and Redis.

  • The Node.js app uses the minio service as its MINIO_ENDPOINT and attempts to generate pre-signed URLs for accessing objects stored in MinIO.

  • I have tried multiple approaches to resolve the issue, such as setting up a custom network and specifying a static IP address for the MinIO container. These approaches are commented out in the configuration below.

    version: '3.4'
    
    services:
      minio:
          image: minio/minio
          command: server /data --console-address ':9001'
          volumes:
            - minio_data:/data
          ports:
            - "9000:9000" 
            - "9001:9001"
          environment:
            MINIO_ROOT_USER: "minioadmin"
            MINIO_ROOT_PASSWORD: "minioadmin"
          # MINIO_DOMAIN: "localhost"
          # MINIO_SERVER_URL: http://localhost:9000
          # MINIO_BROWSER_REDIRECT_URL: http://localhost:9000
          # MINIO_SERVER_URL: http://localhost:9001
          # CONSOLE_SECURE_TLS_REDIRECT: "off"
          # networks:
          #   my_network:
          #     ipv4_address: 172.22.0.2
    
      projectk:
        image: projectk
        build:
          context: .
          dockerfile: ./Dockerfile
        command: "yarn dev"
        volumes: 
          - .:/usr/src/app
          - /usr/src/app/node_modules
        env_file:
          - ./.env
        environment:
          NODE_ENV: development
          FORCE_COLOR: 1
          MINIO_ENDPOINT: "minio"
        ports:
          - "3001:3001"
        depends_on:
          - minio
          - mongo
          - redis
        # networks:
        #   my_network:
    
      mongo:
        image: mongo:latest
        command: mongod --quiet --logpath /dev/null
        ports:
          - "27018:27017"
        volumes:
          - mongodata:/data/db
        # networks:
        #   my_network:
    
      redis:
        image: redis:latest
        ports:
          - "6379:6379"
        healthcheck:
          test: ["CMD", "redis-cli", "ping"]
        # networks:
        #   my_network:
    
    # networks:
    #   my_network:
    #     driver: bridge
    #     ipam:
    #       config:
    #         - subnet: 172.22.0.0/24
    
    volumes:
      mongodata:
      minio_data:
    
    

• URL Generation:

  • I'm using the presignedUrl method from the minio-js SDK to generate pre-signed URLs for objects in my MinIO buckets.

    const Minio = require("minio");
    
    // Create a MinIO client
    const minioClient = new Minio.Client({
      endPoint: process.env.MINIO_ENDPOINT,
      port: 9000, 
      useSSL: false, 
      accessKey: process.env.MINIO_ACCESS_KEY,
      secretKey: process.env.MINIO_SECRET_KEY,
    });
    
    const generateMinioPresignedUrl = async (bucketName, objectName, expiry = 60 * 60) => {
      try {
        const presignedUrl = await minioClient.presignedGetObject(bucketName, objectName, expiry);
        return presignedUrl;
      } catch (err) {
        console.error("Error generating pre-signed URL:", err);
        return "failed to obtain accessUrl";
      }
    };
    

  • The URLs generated look like this:

    http://minio:9000/users-documents/filename.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256...
    

• Problems Encountered:

  1. When I try to access the pre-signed URL, I can't access it because the minio:9000 endpoint is only accessible inside the Docker container.

  http://minio:9000/users-documents/filename.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256...
  

  2. If I replace minio with localhost in the URL, I get a different error:

     <Error>
       <Code>SignatureDoesNotMatch</Code>
       <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
       <Key>filename.jpg</Key>
       <BucketName>users-documents</BucketName>
       <Resource>/users-documents/filename.jpg</Resource>
       <RequestId>17F0DBA572E215F0</RequestId>
       <HostId>dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8</HostId>
     </Error>
     

Environment Configuration:

• Docker Compose Setup:

  • The MINIO_ENDPOINT in my Node.js application is set to minio. if I use localhost it throws connection error.
  • The MinIO client in my code is configured with the correct access and secret keys, and can successfully connect to MinIO.

• MinIO Client (mc) Configuration:

  • The MinIO client is configured with the correct endpoint and credentials, and I can successfully list and upload objects to my buckets using mc.

What I’ve Tried:

• Verified that the buckets and objects exist and have the correct permissions.
• Configured the bucket policy to allow public access using mc.
• Checked the MinIO server logs for any relevant error messages.
• Attempted to generate and access pre-signed URLs using different configurations (localhost (gives connection error) vs minio).

Questions:

1. Why am I getting AccessDenied or SignatureDoesNotMatch errors when trying to access objects using pre-signed URLs?
2. Is there a specific way I should be configuring the MinIO client or the URL generation to avoid these issues?
3. How should I handle the hostname in the generated URL (minio vs localhost) when accessing MinIO within a Docker container