Constant logging out with label studio deployed on Cloud Run

Label Studio Support
1

I’ve been having this issue with label studio on Cloud Run where it doesn’t keep the user signed in for very long at all. I think it has to do with the statelessness of Cloud Run, so when there are multiple instances of our service on Cloud Run, there is a chance that at some point you connect to a different instance and your session details are stored on that instance. The reason I’m pretty sure I think this is the issue is because when I set the max-instances of my Cloud Run deployment to 1, I can’t reproduce the issue anymore.

I tried following the persistent storage instructions for GCS, but they don’t seem to work as nothing appears in the bucket I created for it.

Here is my Cloud Run deployment command

gcloud run deploy "$service_name" \
  --image "$image_uri" \
  --platform managed \
  --region "$region" \
  --service-account "$service_account" \
  --set-env-vars "DJANGO_DB=default" \
  --set-env-vars "POSTGRE_NAME=$db_name" \
  --set-env-vars "POSTGRE_USER=$db_user" \
  --set-env-vars "POSTGRE_PORT=$db_port" \
  --set-env-vars "POSTGRE_HOST=$db_host" \
  --set-env-vars "POSTGRE_PASSWORD=$db_password" \
  --set-env-vars "STORAGE_TYPE=gcs" \
  --set-env-vars "STORAGE_GCS_BUCKET_NAME=$bucket_name" \
  --set-env-vars "STORAGE_GCS_PROJECT_ID=$project_id" \
  --vpc-connector "$vpc_connector" \
  --cpu "2" \
  --memory "4Gi" \
  --max-instances "5" \
  --concurrency "5" \
  --timeout "300" \
  --add-cloudsql-instances "$cloud_sql_instance"

And the Dockerfile:

FROM heartexlabs/label-studio

# Define a build argument for the key name
ARG KEY_NAME

# Copy the key from the build context to the container
COPY ./${KEY_NAME} /${KEY_NAME}

# Set the environment variables for persistent storage and import/export GCS connectors
ENV GOOGLE_APPLICATION_CREDENTIALS=/${KEY_NAME}

CMD ["label-studio"]
2

hey @agent88 thanks for posting here.

My experience and knowledge of Cloud Run is kinda limited to bear with me.

From your config, I’m not seeing a SECRET_KEY defined.

The SECRET_KEY is crucial for maintaining session integrity across different instances because it is used by Django to sign session cookies and other security-related tokens.

  • Session Security: The SECRET_KEY is used to sign session cookies. If the key changes, any existing session cookies will become invalid, causing users to be logged out.
  • Cross-Instance Consistency: In a stateless environment like Cloud Run, multiple instances of your application may be running. If each instance has a different SECRET_KEY , the session cookies signed by one instance will not be recognized by another, leading to frequent logouts.
  • Data Integrity: The SECRET_KEY is also used to sign other data, such as CSRF tokens. A consistent key ensures that these tokens remain valid across different instances.By setting a persistent SECRET_KEY , you ensure that all instances of your application use the same key to sign and validate session cookies and tokens, maintaining session integrity and preventing unexpected logouts.

You can add a key by adding this to the Cloud Run deploy command:

–set-env-vars “SECRET_KEY=your_random_secret_key”

And this would be the full command:

gcloud run deploy "$service_name" \
  --image "$image_uri" \
  --platform managed \
  --region "$region" \
  --service-account "$service_account" \
  --set-env-vars "DJANGO_DB=default" \
  --set-env-vars "POSTGRE_NAME=$db_name" \
  --set-env-vars "POSTGRE_USER=$db_user" \
  --set-env-vars "POSTGRE_PORT=$db_port" \
  --set-env-vars "POSTGRE_HOST=$db_host" \
  --set-env-vars "POSTGRE_PASSWORD=$db_password" \
  --set-env-vars "STORAGE_TYPE=gcs" \
  --set-env-vars "STORAGE_GCS_BUCKET_NAME=$bucket_name" \
  --set-env-vars "STORAGE_GCS_PROJECT_ID=$project_id" \
  --set-env-vars "SECRET_KEY=your_random_secret_key" \
  --vpc-connector "$vpc_connector" \
  --cpu "2" \
  --memory "4Gi" \
  --max-instances "5" \
  --concurrency "5" \
  --timeout "300" \
  --add-cloudsql-instances "$cloud_sql_instance"

But like I said, this may or may not be the issue, I’m not 100% certain. Regardless, let me know if this is helpful.

3

Hi @sajarin. Thanks for the reply. That was indeed the issue. Thanks for the explanation as well, makes total sense :slight_smile: