Tasks aren't leaving the labeling queue

Label Studio Support
1

Hi i have a labeling project with two annotators assigned to it. One difficulty i’m facing is that the the tasks i am assigning to the annotators aren’t leaving the queue after I have accepted the annotation. This is problematic because the queue is never cleared and grows linearly with the number of tasks I’ve assigned making it difficult for the annotators to identify open tasks. This doesn’t seem like intended behavior. Can anyone point me in the right direction to solve this problem; that is, to make sure tasks are removed from the labeling queue if an annotators annotation is accepted

2

Hello,

  • If you’re using Manual task assignment (Label Studio Enterprise), what you see is expected: the annotator’s “My tasks” list includes both tasks assigned to them and tasks they already finished. Accepted tasks won’t be served again in the label stream, but they remain on the “assigned to me” list unless you clear the assignment. This is by design in LSE; see the function that computes annotator queues where manually assigned “already done” tasks are included together with assigned ones link to code.

  • If you want tasks to disappear from annotators’ labeling queue after a review is accepted:

    1. use Auto distribution instead of Manual assignment, or

    2. keep Manual assignment but filter the queue to only show “not finished” tasks, or

    3. bulk clear assignments from tasks that have an accepted review.

But I must note that your description leaves much ambiguity:

  1. Are you using Label Studio Enterprise? If yes, it’s better reach us using the support portal: support.humansignal.com
  2. if you are on Enterprise: “the tasks i am assigning to the annotators aren’t leaving the queue after I have accepted the annotation” - do you mean the Label stream queue? Or the Review stream queue? What overlap per a task (“Annotations per task minimum**”**) do you use? If it’s 2, do you mean that the second annotator should not see the accepted task?
  3. if you are on Community opensource: I don’t understand what you mean by “accepted the annotation”.