Label studio and Minio Local Setup

Previousy I tried contacting on slack on the above issue and recieved a reply too.
To summarise my issue, I have set up my label studio locally with minio container separate setup. So when i add source storage it says cannot connect to endpoint url: ``botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: " http://minio:9001/images "`

Replication of my error:

  1. docker compose up -d (docker-compose.yml)
  2. docker compose up -d (minio-compose.yml)
    docker-compose.yml:
version: "3.9"
services:
  nginx:
    build: .
    image: heartexlabs/label-studio:latest
    restart: unless-stopped
    ports:
      - "6060:80"  # Map to port 80 inside the container
      - "6061:443" # Map to port 443 inside the container
    depends_on:
      - app
    environment:
      - LABEL_STUDIO_HOST=${LABEL_STUDIO_HOST:-192.168.1.29}
    #   Optional: Specify SSL termination certificate & key
    #   Just drop your cert.pem and cert.key into folder 'deploy/nginx/certs'
    #      - NGINX_SSL_CERT=/certs/cert.pem
    #      - NGINX_SSL_CERT_KEY=/certs/cert.key
    volumes:
      - ./mydata:/label-studio/data:rw
      - ./deploy/nginx/certs:/certs:ro
      - ./deploy/default.conf:/etc/nginx/nginx.conf  # Mount the custom nginx.conf
    #   Optional: Override nginx default conf
    #      - ./deploy/my.conf:/etc/nginx/nginx.conf
    command: nginx

  app:
    stdin_open: true
    tty: true
    build: .
    image: heartexlabs/label-studio:latest
    restart: unless-stopped
    expose:
      - "6000"
    depends_on:
      - db
    environment:
      - DJANGO_DB=default
      - POSTGRE_NAME=postgres
      - POSTGRE_USER=postgres
      - POSTGRE_PASSWORD=
      - POSTGRE_PORT=5432
      - POSTGRE_HOST=db
      - LABEL_STUDIO_HOST=${LABEL_STUDIO_HOST:-192.168.1.29}
      - JSON_LOG=1
      - LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
      - LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/home/cypher
      - MINIO_ENDPOINT=http://minio:9000
      - MINIO_ACCESS_KEY=minio_admin_do_not_use_in_production
      - MINIO_SECRET_KEY=minio_admin_do_not_use_in_production
      - MINIO_BUCKET=label-studio
      
    #      - LOG_LEVEL=DEBUG
    volumes:
      - ./mydata:/label-studio/data:rw
    command: label-studio-uwsgi

  db:
    image: pgautoupgrade/pgautoupgrade:13-alpine
    hostname: db
    restart: unless-stopped
    # Optional: Enable TLS on PostgreSQL
    # Just drop your server.crt and server.key into folder 'deploy/pgsql/certs'
    # NOTE: Both files must have permissions u=rw (0600) or less
    #    command: >
    #      -c ssl=on
    #      -c ssl_cert_file=/var/lib/postgresql/certs/server.crt
    #      -c ssl_key_file=/var/lib/postgresql/certs/server.key
    environment:
      - POSTGRES_HOST_AUTH_METHOD=trust
    volumes:
      - ${POSTGRES_DATA_DIR:-./postgres-data}:/var/lib/postgresql/data
      - ./deploy/pgsql/certs:/var/lib/postgresql/certs:ro

Minio-compose.yml:

`version: '3'` `services:`
`  minio:`
`    image: ` `docker.io/bitnami/minio:2022`
`    ports:`
`      - '9000:9000'`
`      - '9001:9001'`
`    networks:`
`      - minionetwork`
`    volumes:`
`      - 'minio_data:/data'`
`    environment:`
`      - MINIO_ROOT_USER=your_username`
`      - MINIO_ROOT_PASSWORD=your_pasword` `networks:`
`  minionetwork:`
`    driver: bridge` `volumes:`
`  minio_data:`
`    driver: local`

In minio i created bucket name images and created user, and user grp and policy(copy pasted policy yml as specified in the website:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::images"
]
},
{
"Effect": "Allow",
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::images/*"
]
}
]
} )
Then created access key in user and added access key of the user in specified placeholders but this doesn’t show any change in the error.
Even tried adding access key from outside in general but both doesnt change my error. Even tried http://localhost/:9000, localhost:9001, minio:9000, minio:9001

Hello,

Thank you for providing detailed information about your setup and the issue you’re encountering with connecting Label Studio to your MinIO instance.

Based on the error message:

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://minio:9001/images"

It seems that the Label Studio application container cannot resolve or connect to the MinIO service at the hostname minio on port 9001.

Problem Analysis

  • Separate Docker Networks: Since you are running Label Studio and MinIO in separate Docker Compose files, they are likely on different Docker networks. This means that the app service in your Label Studio setup cannot resolve the hostname minio to connect to the MinIO service.

Solution

To resolve this issue, you need to ensure that both Label Studio and MinIO services are on the same Docker network so they can communicate with each other. Here’s how you can do it:

1. Combine Docker Compose Files

Combine your docker-compose.yml and minio-compose.yml into a single docker-compose.yml file. This ensures that both services are on the same network and can resolve each other’s hostnames.

Updated docker-compose.yml:

version: '3.9'
services:
  nginx:
    build: .
    image: heartexlabs/label-studio:latest
    restart: unless-stopped
    ports:
      - '6060:80'  # Map to port 80 inside the container
      - '6061:443' # Map to port 443 inside the container
    depends_on:
      - app
    environment:
      - LABEL_STUDIO_HOST=${LABEL_STUDIO_HOST:-192.168.1.29}
    volumes:
      - ./mydata:/label-studio/data:rw
      - ./deploy/nginx/certs:/certs:ro
      - ./deploy/default.conf:/etc/nginx/nginx.conf  # Mount the custom nginx.conf
    command: nginx

  app:
    stdin_open: true
    tty: true
    build: .
    image: heartexlabs/label-studio:latest
    restart: unless-stopped
    expose:
      - '6000'
    depends_on:
      - db
      - minio
    environment:
      - DJANGO_DB=default
      - POSTGRE_NAME=postgres
      - POSTGRE_USER=postgres
      - POSTGRE_PASSWORD=
      - POSTGRE_PORT=5432
      - POSTGRE_HOST=db
      - LABEL_STUDIO_HOST=${LABEL_STUDIO_HOST:-192.168.1.29}
      - JSON_LOG=1
      - LABEL_STUDIO_LOCAL_FILES_SERVING_ENABLED=true
      - LABEL_STUDIO_LOCAL_FILES_DOCUMENT_ROOT=/home/cypher
      - MINIO_ENDPOINT=http://minio:9000
      - MINIO_ACCESS_KEY=your_username
      - MINIO_SECRET_KEY=your_pasword
      - MINIO_BUCKET=images
    volumes:
      - ./mydata:/label-studio/data:rw
    command: label-studio-uwsgi

  db:
    image: pgautoupgrade/pgautoupgrade:13-alpine
    hostname: db
    restart: unless-stopped
    environment:
      - POSTGRES_HOST_AUTH_METHOD=trust
    volumes:
      - ${POSTGRES_DATA_DIR:-./postgres-data}:/var/lib/postgresql/data
      - ./deploy/pgsql/certs:/var/lib/postgresql/certs:ro

  minio:
    image: bitnami/minio:2022
    restart: unless-stopped
    ports:
      - '9000:9000'
      - '9001:9001'
    volumes:
      - minio_data:/data
    environment:
      - MINIO_ROOT_USER=your_username
      - MINIO_ROOT_PASSWORD=your_pasword
    command: minio server /data --console-address ":9001"

volumes:
  minio_data:
    driver: local

Explanation:

  • Services:
    • minio: Added the MinIO service to the same Docker Compose file.
    • app: Updated depends_on to include minio.
  • Environment Variables:
    • MINIO_ENDPOINT: Set to http://minio:9000. Since both services are on the same network, the minio hostname can be resolved by the app service.
    • MINIO_ACCESS_KEY and MINIO_SECRET_KEY: Set to match with MinIO credentials.
    • MINIO_BUCKET: Set to images, which is the bucket you created.
  • Volumes:
    • Defined a minio_data volume for MinIO to persist data.

2. Update MinIO Configuration

  • Bucket Creation: Ensure that the images bucket exists in MinIO.
  • User and Policy:
    • Create a user and assign the appropriate policy to allow Label Studio to access the bucket.
    • Use the policy you’ve provided, ensuring it is correctly applied to the user.

Policy Example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::images"]
    },
    {
      "Effect": "Allow",
      "Action": ["s3:DeleteObject", "s3:GetObject", "s3:PutObject"],
      "Resource": ["arn:aws:s3:::images/*"]
    }
  ]
}

3. Rebuild and Start Services

Run the following command to build and start the updated Docker Compose configuration:

docker-compose up -d --build

4. Configure Label Studio Cloud Storage

In Label Studio, set up the S3 storage connection:

  • Go to: Project Settings > Cloud Storage
  • Add Source Storage:
    • Type: AWS S3
    • Bucket Name: images
    • Region Name: (Leave empty or set if required)
    • S3 Endpoint: http://minio:9000
    • Access Key ID: your_username (Same as MINIO_ROOT_USER)
    • Secret Access Key: your_pasword (Same as MINIO_ROOT_PASSWORD)
    • Use SSL: Unchecked (since MinIO is running without SSL)
    • Use Presigned URLs: Unchecked
    • Treat every bucket object as a source file: Check if you want to import all objects
  • Check Connection: Click to verify the connection.
  • Add Storage: Save the configuration.

5. Troubleshooting

If you still encounter issues:

  • Hostnames and Networking:
    • Ensure that the minio hostname is resolvable from the app container. You can test this by entering the app container shell and pinging minio:
docker exec -it <app_container_name> bash
ping minio
  • Alternate Endpoint:
    • If the above does not work, try using host.docker.internal:
      • Update MINIO_ENDPOINT in the app service to:
- MINIO_ENDPOINT=http://host.docker.internal:9000
* In Label Studio Cloud Storage settings, set the S3 Endpoint to `http://host.docker.internal:9000`.
* Note: `host.docker.internal` allows Docker containers to access services running on the host machine.
  • Ports and Firewall:
    • Ensure that ports 9000 and 9001 are not blocked by any firewall settings.

6. Verify Access Keys and Secrets

  • Double-check that the MINIO_ACCESS_KEY and MINIO_SECRET_KEY in your app service match the MinIO credentials.

7. Ensure Correct MinIO Command

  • In your minio service, use the following command to start MinIO with the console address:
command: minio server /data --console-address ":9001"

Additional Information

  • Context from Documentation:
    • According to the Label Studio and MinIO integration guides, using http://minio:9000 as the S3 Endpoint within the same Docker network is standard practice.
    • It is important to disable Use Presigned URLs when MinIO is accessible directly within the Docker network.
  • Common Issues:
    • Hostname Resolution: When services are in different Docker Compose files, they cannot resolve each other’s hostnames because they are on separate networks.
    • Separate Networks: Combining the services into one Docker Compose file places them on the same network, allowing hostname resolution.

Summary

  • Combine Docker Compose Files: Ensure all services are in a single docker-compose.yml to allow networking between them.
  • Configure Environment Variables: Set the correct MINIO_ENDPOINT, MINIO_ACCESS_KEY, and MINIO_SECRET_KEY.
  • Update Label Studio Settings: Configure the S3 storage in Label Studio with the correct endpoint and credentials.
  • Test Connectivity: Verify that the app container can reach the minio service.
  • Check MinIO Policies: Ensure that the policies are correctly set to allow necessary S3 actions.

References

1 Like