VideoVector

The VideoVector tag brings vector annotation capabilities to videos. It works in combination with the <Video/> and the <Labels/> tags, and supports closable paths and skeleton mode with keyframe-based interpolation across video frames.

Use with the following data types: video.

Use with SAM2 for video segmentation

VideoVector is especially powerful when paired with Segment Anything Model 2 (SAM2). Instead of drawing and adjusting vectors on every frame by hand, you place a vector on a keyframe and let SAM2 track and propagate the object across the video, so you only fix up the frames where the model drifts. This combination is where these tags really shine for object tracking and video segmentation workflows.

For a step-by-step setup, see the SAM2 with Videos ML backend tutorial. A dedicated blog post walking through VideoVector + SAM2 is coming soon.

How keyframes work

You annotate a vector on a single frame and Label Studio stores that frame as a keyframe. When you move points, add points, or close the path on a later frame, a new keyframe is created. Label Studio linearly interpolates each vertex position (including bezier control points) between keyframes, so the vector follows your object as the video plays.

Path and point basics

Action	Instruction
Add points	Click on empty space.
Add points to path segments	Press Shift while clicking on a segment that is between two points.
End or exit the path	Press Esc or double-click on the last point you added to the path.
Move points	Simply click a point and drag to reposition.
Delete points	Press Alt or Option and click on an existing point.

Advanced

Closed paths

You can create closed paths to create polygon shapes. To create closed paths, use the closable="true" parameter in your labeling configuration.

Action	Instruction
Close the path	Double-click on your final point. This automatically adds a segment between your first point and final point.
Break closed path	Press Alt or Option and click on a vector segment in a closed path to reopen it. Click on a point to delete the point.

Skeleton

You can create skeleton vectors using the skeleton="true" parameter in your labeling configuration.

When enabled, new points connect to the active point and not the last added point.

Parameters

Parameters

Param	Type	Default	Description
name	string		Name of the element
toName	string		Name of the element to control (video)
[opacity]	number	0.2	Opacity of vector
[fillColor]	string	"#f48a42"	Vector fill color in hexadecimal or HTML color name
[strokeColor]	string	"#f48a42"	Stroke color in hexadecimal
[strokeWidth]	number	2	Width of stroke
[pointSize]	small | medium | large	small	Size of vector handle points
[pointStyle]	rectangle | circle	circle	Style of points
[closable]	boolean	false	Allow closed shapes
[skeleton]	boolean	false	Enables skeleton mode to allow branch paths
[minPoints]	number | none	none	Minimum allowed number of points
[maxPoints]	number | none	none	Maximum allowed number of points
[snap]	pixel | none	none	Snap vector to image pixels
[pointSizeEnabled]	number	5	Size of a point in pixels when shape is selected
[pointSizeDisabled]	number	3	Size of a point in pixels when shape is not selected

Example

Video Vector Annotation

<View>
  <Header>Label the video:</Header>
  <Video name="video" value="$video" />
  <VideoVector name="vector" toName="video" />

  <Labels name="videoLabels" toName="video">
    <Label value="Road" background="#944BFF"/>
    <Label value="Boundary" background="#98C84E"/>
  </Labels>
</View>

Result parameters

Kind: global typedef Returns: VideoVectorRegionResult - The serialized video vector region data in Label Studio format Properties

Name	Type	Description
original_width	number	width of the original video frame (px)
original_height	number	height of the original video frame (px)
image_rotation	number	rotation degree of the video frame (deg)
value	Object
value.sequence	Array.<Object>	array of keyframes; positions between keyframes are interpolated
value.sequence[].frame	number	frame number the keyframe applies to
value.sequence[].enabled	boolean	whether the vector is visible starting at this keyframe
value.sequence[].closed	boolean	whether the vector is closed (polygon) or open (polyline) on this keyframe
value.sequence[].vertices	Array.<Object>	array of point objects with coordinates, bezier curve information, and point relationships
value.labels	Array.<string>	array of label names assigned to this vector (when used with <Labels> or VideoVectorLabels)

Example results JSON export

{
  "original_width": 1920,
  "original_height": 1280,
  "image_rotation": 0,
  "value": {
    "sequence": [
      {
        "frame": 1,
        "enabled": true,
        "closed": false,
        "vertices": [
          { "id": "point-1", "x": 25.0, "y": 30.0, "prevPointId": null, "isBezier": false },
          { "id": "point-2", "x": 75.0, "y": 70.0, "prevPointId": "point-1", "isBezier": false }
        ]
      },
      {
        "frame": 30,
        "enabled": true,
        "closed": false,
        "vertices": [
          { "id": "point-1", "x": 40.0, "y": 45.0, "prevPointId": null, "isBezier": false },
          { "id": "point-2", "x": 80.0, "y": 60.0, "prevPointId": "point-1", "isBezier": false }
        ]
      }
    ],
    "labels": ["Road"]
  }
}