Sync Video and Timeseries using sync option

Label Studio Support
1

Looking for support on using Label Studio? Ask your questions here, and get answers from members of the community!

Here are a few tips for best results!
Provide a clear, detailed title. The title is the intro to your question. Providing a clear summary of your issue is much more helpful than just “Urgent, help!”

I have setup video and time series data however, it doesn’t appear that the two are synced to one another. I have tried the time series data both with timeColumn and without and it leads to the same result. Any ideas or guidance? The sensor data for the timeline is approx 400Hz, the framerate of the camera is approx 30fps

Share your setup!
To help us understand your issue, it’s helpful to have as much detail as possible about your installation. We suggest including:

What version of Label Studio you’re using (for example, 1.10.0).
1.12.0

How you installed Label Studio (for example, pip, brew, Docker, etc.).
Docker

Your labeling configuration.

<View>
  <Header value="Video and Time-Series Data Synchronization"/>
  
  <Video name="video" value="$video" sync="syncGroup"/>
  
  <TimeSeries name="sensor" value="$timeseriesUrl" valueType="url" sync="syncGroup">
    <Channel column="Gyroscope [rad/s]"
                 units="rad/s"
                 displayFormat=",.1f"
                 strokeColor="#1f77b4"
                 legend="Gyroscope Y"/>
  </TimeSeries>
  
  <Labels name="labels-sensor" toName="sensor">
    	<Label value="Run" background="red"/>
        <Label value="Walk" background="green"/>
        <Label value="Stationary" background="blue"/>
        <Label value="Pothole" background="#f6a"/>
        <Label value="Kerb" background="#351"/>
  </Labels>
</View>

Sample data to help us reproduce the issue.
These are stored in my GCS - happy to share over a screenshare or huddle.

Screenshots, videos, logs, or other details that illustrate the problem.
Video showing issue: label studio time series and video sync - TechSmith Screencast - TechSmith Screencast

The more you can share, the more equipped we’ll be to help you!

2

hey @Kirk thanks for posting your question here from Slack! I had initially thought that this type of labelling config would work but it seems like I mistook it for video-audio syncing.

Syncing Video and Timeseries data was supported in a previous version of Label Studio but it’s not supported anymore in newer versions due to some regressions.

That being said, we’ve gotten a lot of requests for this labelling config from users and are trying to figure out how to implement it in the latest version.

For the time being, if you’re willing to downgrade to an earlier version, there is a guide on how to enable this config to work on an earlier version: Label Studio — Time series video audio labeling

We don’t typically recommend downgrading since it becomes harder for us to give support and you may be missing out on newer features or UI improvements. But if this is critical for your usecase, it may be worth considering.

I wish I had a better answer for you but hopefully this gives some clarity. Happy to help in anyway I can!

3

Thanks for clearing that up Sajarin! Looking forward to seeing this return in a future version - the sync tag logic would work really well (in my humble opinion). I’ll run a downgraded container for now with the other logic and I am more than happy to try out and provide feedback on any preview or beta versions when the feature is re-added.

All the best!

1 Like