Migrating images and annotations from local to Cloud Storage

Label Studio Support
1

Hello,

I’m new to the world of CV and AI. I’ve been playing around with Label Studio locally, and it’s enabled me to create some really great models and learn a lot over the past month. I’m now in a position where I’d like to productionise one, which means getting all of the data off my local machine, and into the S3.

I’ve got the Cloud Storage sync for Source and Target all working all working with S3, but they only work one-way.

I can’t find any steps that have worked for me to take all of the images and annotations from my local and get it into S3.

Moving all of the images into S3 is easy, but what I’m struggling with is how I can associate my existing annotations with the new images. As far as I can see, each upload has a unique ID, and then each submission of an annotation has a unique ID.

Any help on how I can carry out this task would be greatly appreciated!

Share your setup!
LabelStudio version 1.12.1 installed and run via Homebrew.

Sample migration project attempt

  1. Create a new Object Detection with Bounding Box project call “Local Project”. Import files to the project from a folder on my machine.
  2. Annotate and submit a number of the images
  3. Create a new Object Detection with Bounding Box project call “Remote Project”. Setup the Cloud Storage and sync the same image set but from S3. Check “Treat every bucket object as a source file”.
  4. Sync the Cloud Storage and the same images appear in the project, but they are loaded from S3.
  5. Go back to Local project, click “Export” and select “JSON”.
  6. Following the step found in this Github issue thread, edit the JSON file to point to S3 paths, not relative paths.

I imagine there are other edits that need to happen here. For example, does the file_upload field need to be removed? It points to a hash-prefixed file name which no longer exists.

  1. Once I’ve updated the JSON file, upload it to an annotations folder in S3 and copy the Object URL.
  2. If I then go back to Remote project in Label Studio, click Import, and provide the Object URL I get a Validation error ('content-length') error.
2

I have found a solution to my own problem using the Label Studio API to write a script that does the migration.