How to build a labeling tool for pathology whole slide image annotation
Pathology whole-slide image annotation powers the machine learning models that detect tumors and segment tissue compartments. Building an application to handle gigapixel images usually requires significant engineering time. You can bypass this overhead using coding agents and Label Studio to build a customized labeling interface.
Agentic workflows eliminate the manual process of designing specialized medical imaging interfaces.
The XML labeling config builder skill translates natural language specifications into validated interface code.
The Label Studio SDK programmatically creates projects and imports tiled whole-slide images in a single pass.
Review queues and pixel intersection-over-union metrics ensure high inter-annotator agreement among pathologists.
Export pipelines deliver run-length encoded masks directly into model training workflows.
The problem
Pathology whole-slide image annotation is inherently difficult due to the scale of the data. You are dealing with gigapixel images stored in specialized formats like SVS or NDPI that require extensive tiling before web viewing. Pathologists suffer from annotator fatigue when navigating these massive slides without ergonomic pan, zoom, and contrast controls. Complex tissue boundaries make semantic segmentation tedious without dedicated brush tools. Strict HIPAA compliance constraints regarding patient health information mean you cannot simply upload these files to shared public cloud tools. The cost to rebuild a compliant, high-performance medical imaging viewer internally will drain your engineering budget and delay model deployment significantly.
The short answer
You can use Label Studio as your foundation and rely on a coding agent to generate the labeling interface. Rather than building a new labeling application from scratch, agents generate the interface from your spec and deploy it into Label Studio in one pass. First, the agent uses the XML labeling config builder skill to produce optimized interface configurations from a plain-language spec. Second, the agent uses the Label Studio SDK to wire that configuration into a real project programmatically.
Docs:
XML labeling config builder skill → https://github.com/HumanSignal/create-xml-labeling-config-skill
Label Studio SDK/CLI → https://api.labelstud.io/api-reference/introduction/getting-started
LLM-friendly docs (markdown) → https://labelstud.io/llms.txt
What you're building
A high-resolution image viewer presents tiled slide patches with ergonomic pan and zoom behaviors.
Dedicated brightness and contrast controls allow annotators to adjust the visual presentation dynamically.
Brush controls enable fast pixel-level semantic segmentation for tissue compartments like tumor, stroma, and necrosis.
A classification picker captures slide-level attributes and marks unusable images containing scanning artifacts.
A per-region text area captures free-text notes for diagnostic rationale or quality flags.
Interactive pre-annotation overlays render existing machine learning model predictions directly on the tissue.
How to build it in Label Studio
1. Set up the project
You must first install or self-host Label Studio to ensure strict HIPAA compliance for pathology whole-slide image annotation. A single task typically consists of one web-viewable patch derived from an original gigapixel slide using an external tiling pipeline. You need to provide metadata fields for slide coordinates and de-identified patient identifiers to maintain data lineage. Your setup process should also preload reference data like histopathology ontology files to define the exact tissue compartments.
2. Generate the labeling interface with the XML config skill
Hand the specifications from the previous section to a coding agent running the XML labeling config builder skill. The skill interprets your requirements and emits a validated Label Studio XML configuration. This code uses the exact tags required for pathology whole-slide image annotation.
<View> - This tag wraps the layout and groups the specialized medical interface elements together for pathology whole-slide image annotation.
<Image name="wsi" value="$image" zoomControl="true" brightnessControl="true"> - This object tag displays the tiled slide patch and provides essential pan and zoom controls for pathology whole-slide image annotation.
<BrushLabels name="mask" toName="wsi"> - This control tag provides the core painting interface to segment specific tissue compartments during pathology whole-slide image annotation.
<Label value="Tumor" background="rgba(255,0,0,0.6)"> - This tag defines a visual class inside the brush tool to mark specific anomalies in pathology whole-slide image annotation.
<Choices name="flags" toName="wsi"> - This tag allows annotators to select slide-level attributes or mark scanning artifacts common in pathology whole-slide image annotation.
<TextArea name="note" toName="wsi" perRegion="true"> - This control tag captures optional free-text rationale attached to a specific painted region during pathology whole-slide image annotation.
3. Wire it into a project with the SDK
Instruct the agent to use the Label Studio SDK/CLI to create the project and apply the generated configuration. The agent uploads the tiled image tasks into the newly created project via the API. You can also direct the agent to import model predictions from systems like Segment Anything as editable pre-annotations. Run a small batch of images to observe the workflow. Watch annotators struggle with specific interface elements, tell the agent to regenerate the XML configuration to fix those friction points, and redeploy the updated interface.
4. Set up review and quality workflows
Design your review pattern for pathology whole-slide image annotation around multi-annotator overlap to establish ground truth. Configure reviewer queues to catch disagreements and route them to senior pathologists automatically. The critical agreement metrics for this domain include pixel intersection-over-union and span overlap. You can track these agreement metrics in dedicated dashboards using Label Studio Enterprise. These quality workflows ensure your models learn from consistent consensus data.
5. Export and integrate
You will export your completed labels in the default JSON format. The key fields include the run-length encoded brush masks and the percentage-based geometric coordinates for each region. Your data pipeline will convert these run-length encoded masks into NumPy arrays or PNG bitmasks. You then hand these files off to your model training pipeline or analytics warehouse. Convert the coordinates back to absolute pixels before feeding the data into your evaluation harness.
Why Label Studio for pathology whole-slide image annotation
The image tag provides native zoom and contrast controls to mitigate the annotator fatigue caused by navigating massive slide tiles.
Brush tools allow annotators to paint complex tissue boundaries quickly instead of clicking hundreds of polygon points.
Self-hosted deployment options guarantee that protected health information remains entirely within your compliant cloud environment.
Webhooks allow you to trigger downstream active learning loops automatically when a pathologist submits a new annotation.
Enterprise review streams enable you to route low-agreement segmentation tasks to senior staff to resolve complex diagnostic disagreements.
Common variations
Radiology series annotation uses similar image controls but relies on bounding boxes to detect abnormalities.
Cytology single-cell instance labeling uses the same basic image viewer but replaces brush strokes with point or polygon tools.
Document review workflows analyze medical PDF reports using text object tags and relationship links.
Next steps
XML labeling config builder skill → https://github.com/HumanSignal/create-xml-labeling-config-skill
Label Studio SDK/CLI → https://api.labelstud.io/api-reference/introduction/getting-started
LLM-friendly docs (markdown) → https://labelstud.io/llms.txt
Task format guide → https://labelstud.io/guide/task_format
Image tag parameters → https://labelstud.io/tags/image
Export formatting → https://labelstud.io/guide/export
How do you meet HIPAA compliance requirements when ingesting whole-slide images?
You must strip all protected health information from the slide labels and metadata before uploading data to Label Studio. Apply the HIPAA Safe Harbor standard to remove the 18 specified identifiers, such as patient names and accession barcodes. Keep your de-identified images within a self-hosted or strictly controlled private cloud environment.
Can you load gigapixel vendor formats directly into the labeling interface?
Web browsers cannot render massive SVS or NDPI files directly. You need to process the whole-slide images through a tiling pipeline using a library like OpenSlide. This pipeline converts the original gigapixel images into web-viewable tiles that the interface image tag can render efficiently.
How do you configure the review workspace to reduce pathologist fatigue?
Add specific control parameters to the image object tag to handle complex slide navigation. Set the interface to provide native zoom, brightness, and contrast controls. This configuration prevents jittery first renders and gives reviewers the ergonomic tools they need to inspect dense tissue compartments.
How do brush segmentation masks export for model training?
Label Studio exports brush masks as run-length encoded arrays inside the standard JSON output. Your data engineering pipeline must convert these compressed arrays into rasterized PNG bitmasks or NumPy arrays. You also need to convert the percentage-based coordinate outputs back into absolute pixels before training your models.
How do you handle cross-origin resource sharing when serving private slide tiles?
You must configure strict cross-origin resource sharing policies on your external storage buckets to serve private pathology tiles securely. Apply the anonymous cross-origin parameter directly to the interface image tag. This setup prevents browser security blocks while maintaining access control over your sensitive medical datasets.