feat(InformationTheory/KullbackLeibler): finite Kullback-Leibler divergence

[elazarg]

Define klDivFin p q = ∑ i, p i * log (p i / q i) for p q : ι → ℝ on a Fintype, and prove Gibbs' inequality for it. The weights are not assumed to be normalized: (∑ i, p i) - (∑ i, q i) ≤ klDivFin p q needs only nonnegativity and absolute continuity, and 0 ≤ klDivFin p q follows whenever ∑ i, q i ≤ ∑ i, p i.

Two supporting lemmas added to Analysis/SpecialFunctions/Log/NegMulLog: Real.mul_log_div and Real.sub_le_mul_log_div, a quotient form of Real.self_sub_one_le_mul_log.

---

Continuing #42584, another part of https://github.com/elazarg/kraft

klDivFin is a Finset.sum definition, not a specialization of klDiv. klDiv is ℝ≥0∞-valued and defined through llr and Measure.rnDeriv, and Mathlib currently has no formula for Measure.rnDeriv on a discrete space.

Even with that lemma in place, klDivFin is ℝ-valued, so it composes directly with Finset arithmetic in downstream code; and it requires no normalization - Gibbs' inequality here holds under ∑ i, q i ≤ ∑ i, p i, so no probability measures are needed.

Happy to split the two NegMulLog lemmas into a separate PR if needed.

[github-actions]

Welcome new contributor!

Thank you for contributing to Mathlib! If you haven't done so already, please review our contribution guidelines, as well as the style guide and naming conventions. In particular, we kindly remind contributors that we have guidelines regarding the use of AI when making pull requests.

We use a review queue to manage reviews. If your PR does not appear there, it is probably because it is not successfully building (i.e., it doesn't have a green checkmark), has the awaiting-author tag, or another reason described in the Lifecycle of a PR. The review dashboard has a dedicated webpage which shows whether your PR is on the review queue, and (if not), why.

If you haven't already done so, please come to https://leanprover.zulipchat.com/, introduce yourself, and mention your new PR.

Thank you again for joining our community.

[github-actions]

PR summary 2a5c657e96

Import changes for modified files

No significant changes to the import graph

Import changes for all files

Files	Import difference
Mathlib.InformationTheory.KullbackLeibler.Finite (new file)	2031

---

Declarations diff (regex)

+ klDivFin
+ klDivFin_comp_equiv
+ klDivFin_nonneg
+ klDivFin_self
+ mul_log_div
+ sub_le_mul_log_div
+ sum_sub_sum_le_klDivFin

You can run this locally as follows

## from your `mathlib4` directory:
git clone https://github.com/leanprover-community/mathlib-ci.git ../mathlib-ci

## summary with just the declaration names:
../mathlib-ci/scripts/pr_summary/declarations_diff.sh <optional_commit>

## more verbose report:
../mathlib-ci/scripts/pr_summary/declarations_diff.sh long <optional_commit>

The doc-module for scripts/pr_summary/declarations_diff.sh in the mathlib-ci repository contains some details about this script.

Declarations diff (Lean)

✅ Lean-aware diff — post-build, computed from the Lean environment (commit 2a5c657).

• +7 new declarations
• −0 removed declarations

+InformationTheory.klDivFin
+InformationTheory.klDivFin_comp_equiv
+InformationTheory.klDivFin_nonneg
+InformationTheory.klDivFin_self
+InformationTheory.sum_sub_sum_le_klDivFin
+Real.mul_log_div
+Real.sub_le_mul_log_div

---

No changes to strong technical debt.

Increase in weak tech debt: (relative, absolute) = (1.00, 0.00)

Current number	Change	Type (weak)
exposed public sections	4969	1

Current commit 2a5c657e96
Reference commit d8ffbd6ba0

This script lives in the mathlib-ci repository. To run it locally, from your mathlib4 directory:

git clone https://github.com/leanprover-community/mathlib-ci.git ../mathlib-ci
../mathlib-ci/scripts/reporting/technical-debt-metrics.py pr_summary

• The relative value is the weighted sum of the differences with weight given by the inverse of the current value of the statistic.
• The absolute value is the relative value divided by the total sum of the inverses of the current values (i.e. the weighted average of the differences).