How to build a labeling tool for dermatology lesion tracking

Tracking skin lesions across multiple visits requires precise geometric measurement and longitudinal context. Building a custom application to manage this temporal data and medical compliance wastes engineering cycles that you could spend training computer vision models. With Label Studio, you can generate a tailored interface that handles paginated multi-image workflows and per-lesion metadata without writing frontend code.

Configure paginated image interfaces to let annotators switch between timepoints without losing visual context.

Define polygon tools with small point sizes to capture precise lesion boundaries for area measurements.

Assign tracking identifiers and clinical status choices directly to individual regions using specific attributes.

Deploy configurations programmatically using a coding agent to eliminate manual setup steps.

Export nested JSON records containing percentage-based coordinates to feed computer vision training pipelines.

The problem

Curating data for dermatology lesion tracking requires linking individual skin anomalies across multiple temporal images. Annotators struggle with mental fatigue when they open separate browser tabs to compare baseline and follow-up patient visits. Medical datasets also introduce strict Health Insurance Portability and Accountability Act (HIPAA) compliance constraints that require de-identified data and secure hosting environments. Engineering a custom frontend to manage these specific clinical features requires months of development time, which delays critical model training.

The short answer

You can use Label Studio as the foundation for this workflow, and a coding agent generates the custom interface. The agent uses the XML labeling config builder skill to produce optimized interface configurations from a plain-language spec. It then uses the Label Studio SDK/CLI to wire the config into a real project programmatically. Rather than building a new labeling application from scratch, agents generate the interface from your spec and deploy it into Label Studio in one pass.

Docs: LLM-friendly docs (markdown) → https://labelstud.io/llms.txt

Docs: Image tag → https://labelstud.io/tags/image

Docs: PolygonLabels tag → https://labelstud.io/tags/polygonlabels.html

Docs: Tasks guide → https://labelstud.io/guide/tasks.html

What you're building

Create a paginated image viewer that lets annotators flip through multiple patient visits in a single workspace.

Configure a polygon drawing tool to capture accurate lesion borders for downstream segmentation algorithms.

Add a numeric input field tied to each drawn region to establish a persistent tracking identifier across visits.

Provide a categorical selection control for each lesion to record clinical changes like stable, enlarged, or resolved.

Include a free-text area attached to the region list so reviewers can document specific clinical rationale without moving their pointer.

Implement a global assessment dropdown for each timepoint to capture the overall trajectory of the patient.

How to build it in Label Studio

1. Set up the project

Install and self-host Label Studio within your secure cloud environment to maintain strict compliance with HIPAA safe harbor rules for medical images. A single annotation task consists of an array of presigned image URLs representing consecutive clinic visits for one patient. You need to supply metadata fields like patient identifiers and visit dates in the task payload to help reviewers filter the queue. You should also upload any reference ontology files that define standardized lesion types so annotators select from an approved clinical vocabulary.

2. Generate the labeling interface with the XML config skill

Direct your coding agent to process the feature list above using the XML labeling config builder skill. The skill evaluates the specific requirements for dermatology lesion tracking and emits a validated Label Studio XML configuration. This configuration binds the appropriate geometric and categorical tags to the primary multi-image data object. The generated output structures the interface using the following fundamental tags.

<Image name="img" valueList="$images" zoom="true"> - Displays a paginated array of temporal clinic visit images in a single view.

<PolygonLabels name="lesion" toName="img" strokeWidth="3" pointSize="small"> - Provides a precise drawing tool to capture the exact geometric boundary of a skin anomaly.

<Number name="track_id" toName="img" perRegion="true"> - Attaches a distinct numeric identifier to an individual polygon to establish longitudinal continuity.

<Choices name="lesion_status" toName="img" perRegion="true"> - Forces the annotator to categorize the temporal change of a specific region as new, stable, or resolved.

<TextArea name="notes" toName="img" perRegion="true" displayMode="region-list"> - Surfaces a text box in the sidebar for clinical notes to minimize mouse travel during review.

3. Wire it into a project with the SDK

Tell the agent to use the Label Studio SDK/CLI to create a new project using the generated XML configuration. The agent can then upload JSON task files containing the image arrays and import model predictions from systems like YOLO to pre-populate bounding boxes. You can operate this entire process as a tight iteration loop. Deploy a small batch of tasks to the interface, watch the clinical annotators struggle with the layout, and instruct the agent to regenerate and redeploy the configuration.

4. Set up review and quality workflows

Configure the project to require multiple independent annotators per task to ensure clinical consensus on difficult cases. Establish dedicated reviewer roles so senior dermatologists can monitor a separate queue and resolve disagreements between junior annotators. Track annotator consistency using polygon intersection over union metrics to measure spatial agreement on lesion boundaries. You can also monitor categorical agreement scores to evaluate how often annotators align on clinical status changes like regression or enlargement.

5. Export and integrate

Export the finalized project data using the default JSON format to preserve the complex relationship between temporal images and individual regions. The JSON payload includes percentage-based geometric coordinates and the custom tracking identifiers required for offline temporal matching. You can then hand this structured payload off to a training pipeline to fine-tune lesion detection models or push it to an analytics warehouse for longitudinal clinical studies.

Why Label Studio for dermatology lesion tracking

With the native paginated image viewer, you avoid the mental fatigue of opening multiple browser tabs to compare patient visits.

By using self-hosted deployment options, you keep all protected health information within your secure network boundary to satisfy HIPAA regulations.

With region-specific input attributes, annotators assign tracking identifiers directly to polygons without building a custom database relationship.

By defining global keyboard shortcuts, you reduce repetitive strain when annotators draw precise lesion boundaries across high-resolution imagery.

Because direct JSON task imports automatically map to cloud storage presigned URLs, you bypass the complexity of building asset delivery systems.

Common variations

Single-image lesion segmentation relies on the same polygon tools but omits the paginated multi-image data structure.

Pure diagnostic classification uses the image viewer alongside global choices to identify general malignancy without explicit geometric boundaries.

Total body photography tracking applies a similar annotation schema to three-dimensional meshes instead of flat temporal photographs.

Surgical site monitoring tracks wound healing progress over time using the identical multi-timepoint interface and status dropdowns.

Next steps

XML labeling config builder skill → https://github.com/HumanSignal/create-xml-labeling-config-skill

Label Studio SDK/CLI → https://api.labelstud.io/api-reference/introduction/getting-started

LLM-friendly docs (markdown) → https://labelstud.io/llms.txt

Multi-page document annotation template → https://labelstud.io/templates/multi-page-document-annotation

Image tag documentation → https://labelstud.io/tags/image

GitHub → https://github.com/HumanSignal/label-studio