Merge branch 'pandas-dev:main' into test/ensure-clean-append

Author: jeffreykenneth

.github/PULL_REQUEST_TEMPLATE.md

@@ -3,3 +3,4 @@
 - [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
 - [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
 - [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
+- [ ] If I used AI to develop this pull request, I prompted it to follow `AGENTS.md`.

AGENTS.md

@@ -0,0 +1,48 @@
+# pandas Agent Instructions
+
+## Project Overview
+`pandas` is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.
+
+## Purpose
+- Assist contributors by suggesting code changes, tests, and documentation edits for the pandas repository while preserving stability and compatibility.
+
+## Persona & Tone
+- Concise, neutral, code-focused. Prioritize correctness, readability, and tests.
+
+## Project Guidelines
+- Be sure to follow all guidelines for contributing to the codebase specified at https://pandas.pydata.org/docs/development/contributing_codebase.html
+- These guidelines are also available in the following local files, which should be loaded into context and adhered to
+    - doc/source/development/contributing_codebase.rst
+    - doc/source/development/contributing_docstring.rst
+    - doc/source/development/contributing_documentation.rst
+    - doc/source/development/contributing.rst
+
+## Decision heuristics
+- Favor small, backward-compatible changes with tests.
+- If a change would be breaking, propose it behind a deprecation path and document the rationale.
+- Prefer readability over micro-optimizations unless benchmarks are requested.
+- Add tests for behavioral changes; update docs only after code change is final.
+
+## Type hints guidance (summary)
+- Prefer PEP 484 style and types in pandas._typing when appropriate.
+- Avoid unnecessary use of typing.cast; prefer refactors that convey types to type-checkers.
+- Use builtin generics (list, dict) when possible.
+
+## Docstring guidance (summary)
+- Follow NumPy / numpydoc conventions used across the repo: short summary, extended summary, Parameters, Returns/Yields, See Also, Notes, Examples.
+- Ensure examples are deterministic, import numpy/pandas as documented, and pass doctest rules used by docs validation.
+- Preserve formatting rules: triple double-quotes, no blank line before/after docstring, parameter formatting ("name : type, default ..."), types and examples conventions.
+
+## Pull Requests (summary)
+- Pull request titles should be descriptive and include one of the following prefixes:
+    - ENH: Enhancement, new functionality
+    - BUG: Bug fix
+    - DOC: Additions/updates to documentation
+    - TST: Additions/updates to tests
+    - BLD: Updates to the build process/scripts
+    - PERF: Performance improvement
+    - TYP: Type annotations
+    - CLN: Code cleanup
+- Pull request descriptions should follow the template, and **succinctly** describe the change being made. Usually a few sentences is sufficient.
+- Pull requests which are resolving an existing Github Issue should include a link to the issue in the PR Description.
+- Do not add summaries or additional comments to individual commit messages. The single PR description is sufficient.

doc/source/user_guide/io.rst

@@ -343,7 +343,7 @@ on_bad_lines : {{'error', 'warn', 'skip'}}, default 'error'
     Specifies what to do upon encountering a bad line (a line with too many fields).
     Allowed values are :
 
-    - 'error', raise an ParserError when a bad line is encountered.
+    - 'error', raise a ParserError when a bad line is encountered.
     - 'warn', print a warning when a bad line is encountered and skip that line.
     - 'skip', skip bad lines without raising or warning when they are encountered.
 

doc/source/whatsnew/v3.0.0.rst

@@ -1032,13 +1032,13 @@ Bug fixes
 Categorical
 ^^^^^^^^^^^
 - Bug in :class:`Categorical` where constructing from a pandas :class:`Series` or :class:`Index` with ``dtype='object'`` did not preserve the categories' dtype as ``object``; now the ``categories.dtype`` is preserved as ``object`` for these cases, while numpy arrays and Python sequences with ``dtype='object'`` continue to infer the most specific dtype (for example, ``str`` if all elements are strings) (:issue:`61778`)
+- Bug in :class:`pandas.Categorical` displaying string categories without quotes when using "string" dtype (:issue:`63045`)
 - Bug in :func:`Series.apply` where ``nan`` was ignored for :class:`CategoricalDtype` (:issue:`59938`)
 - Bug in :func:`bdate_range` raising ``ValueError`` with frequency ``freq="cbh"`` (:issue:`62849`)
 - Bug in :func:`testing.assert_index_equal` raising ``TypeError`` instead of ``AssertionError`` for incomparable ``CategoricalIndex`` when ``check_categorical=True`` and ``exact=False`` (:issue:`61935`)
 - Bug in :meth:`Categorical.astype` where ``copy=False`` would still trigger a copy of the codes (:issue:`62000`)
 - Bug in :meth:`DataFrame.pivot` and :meth:`DataFrame.set_index` raising an ``ArrowNotImplementedError`` for columns with pyarrow dictionary dtype (:issue:`53051`)
 - Bug in :meth:`Series.convert_dtypes` with ``dtype_backend="pyarrow"`` where empty :class:`CategoricalDtype` :class:`Series` raised an error or got converted to ``null[pyarrow]`` (:issue:`59934`)
--
 
 Datetimelike
 ^^^^^^^^^^^^
@@ -1299,6 +1299,7 @@ ExtensionArray
 - Bug in :class:`Categorical` when constructing with an :class:`Index` with :class:`ArrowDtype` (:issue:`60563`)
 - Bug in :meth:`.arrays.ArrowExtensionArray.__setitem__` which caused wrong behavior when using an integer array with repeated values as a key (:issue:`58530`)
 - Bug in :meth:`ArrowExtensionArray.factorize` where NA values were dropped when input was dictionary-encoded even when dropna was set to False(:issue:`60567`)
+- Bug in :meth:`NDArrayBackedExtensionArray.take` which produced arrays whose dtypes didn't match their underlying data, when called with integer arrays (:issue:`62448`)
 - Bug in :meth:`api.types.is_datetime64_any_dtype` where a custom :class:`ExtensionDtype` would return ``False`` for array-likes (:issue:`57055`)
 - Bug in comparison between object with :class:`ArrowDtype` and incompatible-dtyped (e.g. string vs bool) incorrectly raising instead of returning all-``False`` (for ``==``) or all-``True`` (for ``!=``) (:issue:`59505`)
 - Bug in constructing pandas data structures when passing into ``dtype`` a string of the type followed by ``[pyarrow]`` while PyArrow is not installed would raise ``NameError`` rather than ``ImportError`` (:issue:`57928`)

pandas/_libs/lib.pyx

@@ -322,7 +322,7 @@ def item_from_zerodim(val: object) -> object:
     >>> item_from_zerodim(np.array([1]))
     array([1])
     """
-    if cnp.PyArray_IsZeroDim(val):
+    if cnp.PyArray_IsZeroDim(val) and cnp.PyArray_CheckExact(val):
         return cnp.PyArray_ToScalar(cnp.PyArray_DATA(val), val)
     return val
 
@@ -2593,7 +2593,7 @@ def maybe_convert_objects(ndarray[object] objects,
         Whether to convert numeric entries.
     convert_to_nullable_dtype : bool, default False
         If an array-like object contains only integer or boolean values (and NaN) is
-        encountered, whether to convert and return an Boolean/IntegerArray.
+        encountered, whether to convert and return a Boolean/IntegerArray.
     convert_non_numeric : bool, default False
         Whether to convert datetime, timedelta, period, interval types.
     dtype_if_all_nat : np.dtype, ExtensionDtype, or None, default None

pandas/_libs/tslibs/fields.pyx

@@ -146,7 +146,7 @@ def get_date_name_field(
     NPY_DATETIMEUNIT reso=NPY_FR_ns,
 ):
     """
-    Given a int64-based datetime index, return array of strings of date
+    Given an int64-based datetime index, return array of strings of date
     name based on requested field (e.g. day_name)
     """
     cdef:
@@ -335,7 +335,7 @@ def get_date_field(
     NPY_DATETIMEUNIT reso=NPY_FR_ns,
 ):
     """
-    Given a int64-based datetime index, extract the year, month, etc.,
+    Given an int64-based datetime index, extract the year, month, etc.,
     field and return an array of these values.
     """
     cdef:
@@ -502,7 +502,7 @@ def get_timedelta_field(
     NPY_DATETIMEUNIT reso=NPY_FR_ns,
 ):
     """
-    Given a int64-based timedelta index, extract the days, hrs, sec.,
+    Given an int64-based timedelta index, extract the days, hrs, sec.,
     field and return an array of these values.
     """
     cdef:
@@ -555,7 +555,7 @@ def get_timedelta_days(
     NPY_DATETIMEUNIT reso=NPY_FR_ns,
 ):
     """
-    Given a int64-based timedelta index, extract the days,
+    Given an int64-based timedelta index, extract the days,
     field and return an array of these values.
     """
     cdef:
@@ -592,7 +592,7 @@ cpdef isleapyear_arr(ndarray years):
 @cython.boundscheck(False)
 def build_isocalendar_sarray(const int64_t[:] dtindex, NPY_DATETIMEUNIT reso):
     """
-    Given a int64-based datetime array, return the ISO 8601 year, week, and day
+    Given an int64-based datetime array, return the ISO 8601 year, week, and day
     as a structured array.
     """
     cdef:

pandas/_libs/tslibs/offsets.pyx

@@ -827,7 +827,7 @@ cdef class BaseOffset:
     @property
     def nanos(self):
         """
-        Returns a integer of the total number of nanoseconds for fixed frequencies.
+        Returns an integer of the total number of nanoseconds for fixed frequencies.
 
         Raises
         ------

pandas/_libs/tslibs/timedeltas.pyx

@@ -334,7 +334,7 @@ cdef convert_to_timedelta64(object ts, str unit):
     Handle these types of objects:
         - timedelta/Timedelta
 
-    Return an timedelta64[ns] object
+    Return a timedelta64[ns] object
     """
     # Caller is responsible for checking unit not in ["Y", "y", "M"]
     if isinstance(ts, _Timedelta):

pandas/_libs/tslibs/timestamps.pyx

@@ -1717,7 +1717,7 @@ cdef class _Timestamp(ABCTimestamp):
 
     def to_period(self, freq=None):
         """
-        Return an period of which this timestamp is an observation.
+        Return a period of which this timestamp is an observation.
 
         This method converts the given Timestamp to a Period object,
         which represents a span of time,such as a year, month, etc.,

pandas/core/arrays/boolean.py

@@ -273,7 +273,7 @@ class BooleanArray(BaseMaskedArray):
     BooleanArray implements Kleene logic (sometimes called three-value
     logic) for logical operations. See :ref:`boolean.kleene` for more.
 
-    To construct an BooleanArray from generic array-like input, use
+    To construct a BooleanArray from generic array-like input, use
     :func:`pandas.array` specifying ``dtype="boolean"`` (see examples
     below).
 
@@ -313,7 +313,7 @@ class BooleanArray(BaseMaskedArray):
 
     Examples
     --------
-    Create an BooleanArray with :func:`pandas.array`:
+    Create a BooleanArray with :func:`pandas.array`:
 
     >>> pd.array([True, False, None], dtype="boolean")
     <BooleanArray>