feat: add strip_whitespace and regex_replace to DocumentCleaner

[VedantMadane]

Summary

Fixes #8798

This PR expands the functionality of DocumentCleaner with two new parameters:

1. strip_whitespace: bool = False

When True, removes leading and trailing whitespace from document content using Python's str.strip().
Unlike remove_extra_whitespaces, this only affects the beginning and end of the text, preserving internal whitespace (useful for markdown formatting).

2.

regex_replace: dict[str, str] | None = None
A dictionary mapping regex patterns to replacement strings. This allows custom replacements instead of just removal. For example:

• {r'\n\n+': '\n'} replaces multiple consecutive newlines with a single newline
• {r'\s{2,}': ' '} replaces multiple spaces with a single space

Changes

• Added strip_whitespace parameter to \DocumentCleaner.init()
• Added regex_replace\ parameter to DocumentCleaner.init()
• Added _replace_regex()\ method for custom regex replacements
• Added comprehensive unit tests for both new features

Test plan

• Added unit tests for strip_whitespace
• Added unit tests for regex_replace with single/multiple patterns
• Added test for combined usage of both features
• Added test for initialization with new parameters

[vercel]

@VedantMadane is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.