[Performance] Batched calibration

[kylesayrs]

Purpose

• Reduce calibration runtime by providing users with the ability to control the calibration batch size
• In the future, multi-gpu calibration can be enabled using torch.distributed tensor parallelism

Prerequisites

• [Performance] [Utils] Refactor embeddings utils #2080
• [Performance] Disable LM Head #2081

Changes

• Add batch_size argument
• Adjust data_collator default to be DataCollatorWithPadding
  • This means that text datasets can be collated without extra work
• Update vision model examples to use a real data collator
  • Most vision processors are like text processors, except that they add an extra batch dim (for some reason) that needs to be stripped
  • Models which resize all images to the same sizes are automatically handled by DataCollatorWithPadding
  • Models which handle images of different sizes must either batch during dataset processing, or else not handle BS > 1

TODO

1. Demonstrate throughput increase from batching
2. Test that all modifiers work with batched samples
3. Test that all modifiers work with padded samples

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.