How to export segmentation in COCO/YOLO format?

Label Studio Support
,
1

I have encountered a problem with exporting annotations.I have partitioned a dataset with SAM-backend support.When trying to export to COCO format I got an error (KeyError “x”) and exactly the same story when exporting to YOLO format.(First pic)
After reading the documentation and articles, I found out that in order to export the segmentation you need to run SAM with the parameter out_poly=true (Maybe it will be useful for someone to know) on the old partitioning it does not work, only on the one that was partitioned with the parameter out_poly=true.
I created a new project and for the test I loaded 3 images there and made markup(Second and Third pics is a source annotations on image), but when exporting to COCO I get almost empty json file(fourth pic).With export to YOLO format exactly the same problem, text labels are just empty.
When exporting to json and json-min there is no problem.
Note:LabelStudio and SAM-Backend running in Docker.
Used Version:1.5.0

coco
coco680×558 37.6 KB

src_code
src_code793×921 34.6 KB

src_code_2
src_code_2793×921 57.7 KB

export
export1331×1012 75.3 KB

2

hey @YesNile thanks for posting here,

I believe the issue you’re encountering with exporting annotations to COCO and YOLO formats when using the SAM-backend is known. The problem arises because COCO and YOLO formats do not support BrushLabels, which is what SAM generates.

Here are some troubleshooting steps that might help

  1. Ensure out_poly=true Parameter: As you mentioned, using the out_poly=true parameter is necessary for exporting segmentation. This parameter ensures that the output is in a polygon format, which is compatible with COCO and YOLO.
  2. Check for BrushLabels: Verify that the annotations are not in BrushLabels format. If they are, you need to convert them to a format supported by COCO and YOLO, such as PolygonLabels.
  3. Update Label Studio and SAM Backend: Your Label Studio version 1.5.0 is outdated. It’s recommended to update to the latest version to benefit from recent fixes and improvements. The latest version as of now is 1.12.2 for Label Studio.
  4. Exporting to JSON: Since exporting to JSON and JSON-MIN works without issues, you can use these formats as an intermediate step. Export the annotations to JSON, then convert them to COCO or YOLO format using a custom script.
  5. Custom Conversion Script: You can write a script to convert the JSON annotations to COCO or YOLO format. Here is a basic generated boilerplate example of how you might convert BrushLabels to PolygonLabels:

import json
import random
import string
import numpy as np

def mask2poly(mask):
    # Implement the conversion from mask to polygon
    pass

def convert_annotations(input_json, output_json):
    with open(input_json, 'r') as f:
        data = json.load(f)

    for item in data['annotations']:
        if item['type'] == 'brushlabels':
            mask = item['value']['mask']
            poly = mask2poly(mask)
            for p_ in poly:
                label_id = ''.join(random.choices(string.ascii_uppercase + string.ascii_lowercase + string.digits, k=8))
                p_label = np.reshape(p_, (-1, 2)).tolist()
                p_label = [[cw / item['original_width'] * 100, ch / item['original_height'] * 100] for [cw, ch] in p_label]
                result_ = {
                    "from_name": "labels",
                    "to_name": item['to_name'],
                    "original_width": item['original_width'],
                    "original_height": item['original_height'],
                    "image_rotation": 0,
                    "value": {
                        "points": p_label,
                        "polygonlabels": item['value']['labels'],
                    },
                    "type": "polygonlabels",
                    "id": label_id,
                    "readonly": False,
                }
                item['result'].append(result_)

    with open(output_json, 'w') as f:
        json.dump(data, f, indent=4)

Let me know if this helps!