SXT Segmentation

Soft X-ray Tomography (SXT) segmentation provides specialized tools for cellular structure identification in SXT imaging data. SXT offers advantages including natural contrast, high spatial resolution, and the ability to image thick biological specimens in near-native states.

Key Features:

  • Multi-Structure Recognition: Simultaneous detection of cell membrane and nuclear boundaries

  • Deep Learning Powered: U-Net architectures optimized for SXT’s unique characteristics

  • High Resolution: Capable of resolving fine cellular structures in 3D volumes

  • Robust Performance: Handles SXT’s natural contrast variations and thick specimen artifacts

Model Architecture

SXT segmentation employs specialized U-Net architectures tailored for different cellular components:

Cell and Nucleus Segmentation (run_cell_segmentation)

  • Model Type: Multi-class U-Net (3-class: background, membrane, nucleus)

  • Architecture: U-Net with dropout layers for regularization

  • Input Dimensions: 2D slices resized to 288×480 pixels

  • Output: Per-slice 3-class segmentation maps

  • Preprocessing Pipeline: - Intensity normalization to [0, 255] range - Resize using nearest-neighbor interpolation - Standardization: (pixel_value / 255.0 - 0.456) / 0.224

  • Postprocessing: Slice-by-slice prediction followed by 3D volume assembly

  • Training Data: PBC Consortium pancreatic β-cell SXT datasets

  • Model File: models/sxt/cell_nucleus_unet_best.pth

Mitochondria Segmentation (run_mito_segmentation)

  • Model Type: Binary U-Net

  • Architecture: Adapted from cell segmentation U-Net

  • Input: 2D slices from 3D SXT volumes

  • Output: Binary mitochondrial mask per slice

  • Optimization: Fine-tuned for mitochondrial shape and contrast patterns

  • Model File: models/sxt/mito_unet_best.pth

ISG Segmentation (run_sphere_like_organelle_segmentation)

  • Model Type: Binary U-Net for sphere-like structures

  • Architecture: Specialized for spherical organelle detection

  • Input: 3D SXT volumes

  • Output: Semantic segmentation of ISG population

  • Use Case: Suitable for insulin secretory granules and other spherical organelles

  • Model File: models/sxt/isg_unet_best.pth

Technical Details

U-Net Architecture:

The U-Net models used in SXT segmentation follow a classic encoder-decoder structure with skip connections:

# U-Net for SXT segmentation
Unet(
    n_class=3,           # For cell/nucleus segmentation
    is_dropout=True        # Enable dropout for regularization
)

Data Preprocessing:

SXT data undergoes specific preprocessing steps to optimize for the trained models:

  1. Normalization: Scale intensities to [0, 255] range as uint8

  2. Resizing: Resize to (480, 288) for optimal model input

  3. Standardization: Apply ImageNet normalization parameters (mean=0.456, std=0.224)

Multi-View Processing:

The segmentation pipeline processes 3D volumes slice-by-slice to leverage 2D U-Net models while maintaining 3D context through post-processing assembly.

Core Segmentation API

SXT segmentation now uses the unified Segmenter API for consistent interface across all modalities.

Recommended Usage (Unified API):

from ipa.processing.segmentation import SXTCellSegmenter, SXTMitoSegmenter, SXTISGSegmenter

# Cell and Nucleus Segmentation
cell_seg = SXTCellSegmenter(device='cuda')
cell_seg.load_model()
results = cell_seg.predict(volume)

# Mitochondria Segmentation
mito_seg = SXTMitoSegmenter(device='cuda')
mito_seg.load_model()
results = mito_seg.predict(volume)

# ISG Segmentation
isg_seg = SXTISGSegmenter(device='cuda')
isg_seg.load_model()
results = isg_seg.predict(volume)

For detailed API documentation, see Segmentation Module.

Cell Segmentation for SXT

The run_cell_segmentation function delivers comprehensive analysis capabilities for cellular nuclei, membranes, and combined structural elements in SXT tomographic data through an advanced multi-view deep learning approach.

Example Usage:

from ipa.processing.segmentation import run_cell_segmentation
from ipa.data_loader import UniversalDataLoader

# Load SXT tomogram data
dataid_list = ['784_5']
image_data = {}

for dataid in dataid_list:
    image_path = f'data/sxt_images/Stevens_pancreatic_INS_1E_784_5_pre_rec.mrc'
    data = UniversalDataLoader.load_data(image_path)
    image_data[dataid] = data

# High-precision nuclear and membrane segmentation on SXT tomograms
run_cell_segmentation(
    save_dir="results/mem_nu_seg_with_val/",
    dataid=dataid_list,
    image_data=image_data,  # Use pre-loaded image data
    pool_processes=6,
)

Mitochondria Segmentation for SXT

The run_mito_segmentation function provides specialized capabilities for identifying and segmenting mitochondrial structures in SXT tomographic data.

Example Usage:

from ipa.processing.segmentation import run_mito_segmentation
from ipa.data_loader import UniversalDataLoader

# Load SXT tomogram data
dataid_list = ['784_5']
image_data = {}

for dataid in dataid_list:
    image_path = f'data/sxt_images/Stevens_pancreatic_INS_1E_784_5_pre_rec.mrc'
    data = UniversalDataLoader.load_data(image_path)
    image_data[dataid] = data

# Specialized mitochondrial segmentation for SXT data
run_mito_segmentation(
    save_dir="results/mito_segmentation/",
    dataid=dataid_list,
    image_data=image_data,  # Use pre-loaded image data
    pool_processes=6
)

ISG Segmentation for SXT

The run_sphere_like_organelle_segmentation function provides specialized capabilities for identifying and segmenting insulin secretory granules (ISG) and other sphere-like organelles in SXT tomographic data of pancreatic β-cells.

Example Usage:

from ipa.processing.segmentation import run_sphere_like_organelle_segmentation
from ipa.data_loader import UniversalDataLoader

# Load SXT tomogram data
dataid_list = ['784_5']
image_data = {}

for dataid in dataid_list:
    image_path = f'data/sxt_images/Stevens_pancreatic_INS_1E_784_5_pre_rec.mrc'
    data = UniversalDataLoader.load_data(image_path)
    image_data[dataid] = data

# ISG and sphere-like organelle segmentation for SXT data
run_sphere_like_organelle_segmentation(
    save_dir="results/isg_semantic_mask/",
    dataid=dataid_list,
    image_data=image_data,  # Use pre-loaded image data
    pool_processes=1,
)

Complete Workflow Example

from ipa.processing.segmentation import (
    run_cell_segmentation,
    run_mito_segmentation,
    run_sphere_like_organelle_segmentation
)
from ipa.data_loader import UniversalDataLoader, QuickLogger

# Initialize logging
logger = QuickLogger("sxt_segmentation_workflow", log_dir="logs/")

# Data preparation
dataid_list = ['784_5']
image_data = {}

for dataid in dataid_list:
    image_path = f'data/sxt_images/Stevens_pancreatic_INS_1E_{dataid}_pre_rec.mrc'
    logger.file_in(image_path)
    data = UniversalDataLoader.load_data(image_path)
    image_data[dataid] = data
    logger.step(f"Loaded {dataid}: {data.shape}, dtype: {data.dtype}")

# Run all segmentation tasks
logger.step("Running cell segmentation...")
run_cell_segmentation(
    save_dir="results/cell_segmentation/",
    dataid=dataid_list,
    image_data=image_data,
    pool_processes=6
)

logger.step("Running mitochondria segmentation...")
run_mito_segmentation(
    save_dir="results/mito_segmentation/",
    dataid=dataid_list,
    image_data=image_data,
    pool_processes=6
)

logger.step("Running ISG segmentation...")
run_sphere_like_organelle_segmentation(
    save_dir="results/isg_segmentation/",
    dataid=dataid_list,
    image_data=image_data,
    pool_processes=1
)

logger.step("All segmentation tasks completed successfully")

Cell Segmentation Models

Combined (Mem-Nu) Model for SXT

  • Architecture: Multi-task U-Net with shared encoder, designed for SXT multi-structure segmentation

  • Input Resolution: 288×480 pixels per slice

  • Training Strategy: Joint optimization with balanced loss weighting for SXT data characteristics

  • SXT Adaptations: Unified processing pipeline for multiple SXT-visible cellular structures

  • Optimization: AdamW with learning rate scheduling

Mitochondria Segmentation Model

Mitochondria Model for SXT

  • Architecture: Enhanced U-Net with morphology-aware modules for SXT mitochondrial features

  • Input Resolution: 288×480 pixels per slice

  • Training Strategy: Multi-view fusion with consensus learning on SXT tomographic datasets

  • SXT Adaptations: Custom processing pipeline for SXT mitochondrial contrast and morphology

  • Optimization: Adam with adaptive learning rate

ISG Segmentation Model

ISG (Insulin Secretory Granules) Model for SXT

  • Architecture: U-Net with 3-channel input and 2-class output for binary sphere-like organelle segmentation

  • Input Resolution: Variable resolution with 0.5 scale factor for efficient processing

  • Input Channels: 3 (RGB conversion from grayscale SXT data)

  • Output Classes: 2 (background and ISG/sphere-like organelles)

  • Training Strategy: Binary classification with sigmoid activation and threshold-based prediction

  • SXT Adaptations: - LAC (Linear Absorption Coefficient) normalization factor of 33.33 - Custom preprocessing pipeline for SXT intensity conversion - Sphere-like organelle morphology optimization

  • Preprocessing: - LAC factor normalization: img = img * lac_factor - Intensity clamping: img[img > 1] = 1 - 8-bit conversion: (img - min_val) / max_val * 255

  • Output Threshold: 0.5 (configurable for sensitivity adjustment)

  • Model File: vesicle_mask0.63_processed.pth

Dataset Information for SXT

SXT Cell Segmentation Dataset

  • Source: Soft X-ray Tomography imaging datasets from PBC Consortium (Pancreatic Beta Cell Consortium)

  • Data Access: Available through PBC Dataset Portal

  • Sample Size: 24 SXT tomograms total

  • Image Modality: Soft X-ray Tomography (SXT)

  • Imaging Parameters: Optimized for cellular structure visualization with natural contrast

  • Resolution: High spatial resolution enabling subcellular structure identification

  • Sample Types: Pancreatic β-cells during insulin secretion process

  • Annotation Method: Expert manual annotation on SXT tomographic slices for membrane, nucleus, and mitochondria

  • Data Partition: 18 training / 3 validation / 3 testing samples

Dataset Partition Details

Training Set (18 samples): 766_10, 766_11, 766_2, 766_7, 769_5, 769_7, 783_12, 783_6, 784_4, 784_6, 784_7, 785_7, 822_4, 822_6, 822_7, 842_13, 931_14, 931_9

Validation Set (3 samples): 766_5, 783_5, 842_12

Testing Set (3 samples): 766_8, 784_5, 842_17

SXT Mitochondria Dataset

  • Source: High-resolution SXT tomographic datasets with focus on mitochondrial structures in pancreatic β-cells

  • Sample Size: Same 24 SXT samples with specialized mitochondrial annotations

  • Image Modality: Soft X-ray Tomography optimized for organelle visualization

  • Imaging Parameters: SXT-specific settings for optimal mitochondrial contrast in β-cells

  • Resolution: High-resolution specifications for mitochondrial detail analysis

  • Sample Preparation: Near-native state pancreatic β-cell samples during insulin secretion

  • Annotation Method: Expert manual annotation of mitochondrial boundaries in SXT tomograms

  • Validation Split: Same partition as cell segmentation dataset

ISG Instance Segmentation (Mask R-CNN)

For instance-level ISG segmentation using Mask R-CNN, see: