Recommended MONAI MIL baseline for variable patient-level histopathology bags

[motazkhasawneh]

Hello MONAI community. I am considering MONAI for an efficient patient-level histopathology classification baseline and would appreciate guidance on the most appropriate supported components.

The general setup is public H&E images, multiple slides per patient, variable numbers of crops, and patient-level labels. The goal is one prediction per patient under a limited inference budget. I am interested in comparing a compact trainable encoder plus MIL with frozen pathology-pretrained embeddings plus a small MIL head, while keeping patient and tissue-source groups separate across evaluation folds.

I found the PANDA MIL example: https://github.com/Project-MONAI/tutorials/blob/main/pathology/multiple_instance_learning/panda_mil_train_evaluate_pytorch_gpu.py

Could you point me to:

1. A recommended MONAI example or pattern for extending slide-level MIL to patient-level bags across multiple slides?
2. Supported ways to handle variable bag sizes, padding/masking, and chunked feature extraction without allowing padded crops to affect aggregation?
3. Suitable MONAI MIL aggregators for a fair comparison between frozen embeddings and compact end-to-end training, including any important limitations?

This is a design question, not a bug report. Links to examples, documentation, or relevant papers would be very helpful. I can discuss the general setup, but unpublished methods, results, and research artifacts will remain private. Thank you.

[alleyxd]

Some pointers from reading monai/networks/nets/milmodel.py, which the PANDA example uses:

1. Patient-level bags. MILModel doesn't know about slides. It takes [B, N, C, H, W] and treats the N tiles as one bag. So patient level is just a data question: pool the tiles from all of a patient's slides into one bag (the PANDA dataloader samples a fixed tile_count per slide, and you'd sample per patient instead).

2. Variable bag sizes. There's no mask argument. forward reshapes to B*N tiles and back, so every item in a batch needs the same N, and padded tiles would enter the mean/attention. The practical options are:

   • sample a fixed N per bag (with replacement for small bags), which is what PANDA does
   • use batch size 1 for exact variable bags
   • call model(x, no_head=True) to get the [B, N, D] tile features and write a small masked attention head yourself (set masked logits to -inf before the softmax)

3. Frozen embeddings vs end-to-end. You can reuse the same aggregators on precomputed features by passing them as [B, N, D, 1, 1] with backbone=torch.nn.Flatten() and backbone_num_features=D. The reshape then just passes them through. Custom backbones support mean, max, att and att_trans, but not att_trans_pyramid, which needs the ResNet layer hooks. Using the same mil_mode and the same bag sampling on both sides keeps the comparison fair.

For the attention pooling itself, the reference is Ilse et al., "Attention-based Deep Multiple Instance Learning" (2018). att_trans adds a transformer over the tiles before pooling (Myronenko et al., MICCAI 2021).

[Gianni300x]

One detail for the variable-length comparison: masking only the final pooling logits is sufficient for a custom per-tile attention pooling head, but not for a transformer that has already mixed valid and padded tokens. In the current MILModel implementation, att_trans runs the transformer before attention pooling and neither forward nor calc_head accepts a padding mask. For padded bags, a custom transformer path needs a padding mask at self-attention and masked final pooling; alternatively process each unpadded bag separately. With PyTorch's TransformerEncoder, src_key_padding_mask uses True for padding (documentation). Reject empty bags or handle them separately to avoid an all-masked softmax.

For frozen features, chunking the encoder can limit peak device memory: encode only real tiles in chunks, concatenate their embeddings in patient order, then aggregate the entire patient bag once. Averaging independently pooled chunk predictions is generally a different operation from pooling all tiles together. Use evaluation mode for a frozen encoder, and record the tile-sampling budget for both baselines. I inspected the implementation but have not run this on your data.

[motazkhasawneh]

Thank you for the clarification. Sent from Gmail Mobile

…

On Mon, 9 Nov 2026 at 5:05 PM Gianni Luigi Messina ***@***.***> wrote: One detail for the variable-length comparison: masking only the final pooling logits is sufficient for a custom *per-tile attention pooling* head, but not for a transformer that has already mixed valid and padded tokens. In the current MILModel implementation <https://github.com/Project-MONAI/MONAI/blob/dev/monai/networks/nets/milmodel.py>, att_trans runs the transformer before attention pooling and neither forward nor calc_head accepts a padding mask. For padded bags, a custom transformer path needs a padding mask at self-attention *and* masked final pooling; alternatively process each unpadded bag separately. With PyTorch's TransformerEncoder, src_key_padding_mask uses True for padding ( documentation <https://docs.pytorch.org/docs/stable/generated/torch.nn.TransformerEncoder.html>). Reject empty bags or handle them separately to avoid an all-masked softmax. For frozen features, chunking the encoder can limit peak device memory: encode only real tiles in chunks, concatenate their embeddings in patient order, then aggregate the entire patient bag once. Averaging independently pooled chunk predictions is generally a different operation from pooling all tiles together. Use evaluation mode for a frozen encoder, and record the tile-sampling budget for both baselines. I inspected the implementation but have not run this on your data. — Reply to this email directly, view it on GitHub <#9196?email_source=notifications&email_token=BZS3YDYLEWACQHVRHFTOPIT5YG6J5A5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOJUHEZDCMJSUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-19492112>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BZS3YDZ6FKJ6JFPJEHLUNUT5YG6J5AVCNFSNUABIKJSXA33TNF2G64TZHMZDCNBUHA2TAMBRHNCGS43DOVZXG2LPNY5TCMJQHAZTKNZSUF3AE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Gianni300x]

You're welcome! Glad I could help clarify things.

If my explanation answered your question, would you mind marking it as the accepted answer? It could also help other developers who run into the same question in the future.

Good luck with your project!