[mdhaber]

Closes gh-489

[ev-br]

Looks reasonable! However, the docstring note just above states

        Notes
        -----
        This method returns the static default device when PyTorch is initialized.
        However, the *current* device used by creation functions (``empty`` etc.)
        can be changed at runtime.

        See Also
        --------
        https://github.com/data-apis/array-api/issues/835

git blame points to #300, which, in turn, points to data-apis/array-api#835 and mentions a "current device" vs "default device" controversy. Do you understand what's going on here?

[mdhaber]

The standard is ambiguous about the distinction between "current device" and "default device". And actually, that note in this function's docstring introduces a third idea, which is like the "initial default device".

import torch
# The "initial default device", what the Note says this function should return,
# is always CPU. There doesn't appear to be an environment variable capable
# of changing it.
torch.get_default_device()  # device(type='cpu')
torch.asarray(1).device  # device(type='cpu')

# the actual "default device" can be changed
torch.set_default_device('cuda')
torch.get_default_device()  # device(type='cuda', index=0)
torch.asarray(1).device  # device(type='cuda', index=0)

# the "current device" can be different from the "default device", apparently?
# and torch's `get_default_device` function actually returns the "current" device
with torch.device(1):
    torch.get_default_device()  # device(type='cuda', index=1)
    # torch.asarray(1).device  # I only have one GPU, but presumably `index=1`
torch.get_default_device()  # device(type='cuda', index=0)

The note proposes that the function will always return the "initial default device", CPU. This seems useless to me and data-apis/array-api#835 (comment). I don't see any reason to think that this is what the function was originally intended to do, though.

I would propose that the English word "default" most naturally maps to the concept we are now calling "current device", that is, where the array will go when a device is not specified. torch.get_default_device() seems to agree.

---

Two conflicting observations:

1. The 2021.12 standard referred to context managers as being able to change the "default device", so in that context "default device" seems to mean what we are now calling "current device"... what torch.get_default_device() returns, and what device an array will actually be placed on when none is specified.
2. The phrase "current device" began to appear in 2023.12, and that is when the __array_namespace_info__ and default_device functions were added.

[mdhaber]

I guess the practical advice one can give is to not use the current version of __array_namespace_info__ .default_device because it is probably not what you mean.

[ev-br]

Thank you for the explanation!

My reading (beyond "ugh, what a mess") would be that it'd be great to sort the "current device" vs "default device" for the next Array API spec version. At a guess, some of it (at least context managers) is related to cupy/cupy#9848 and prior CuPy history.

Would you be interested in championing this for the Array API?

To explain my position: the current behavior in array_api_compat.torch seems to be a bug indeed. What I'm not easy about is making decisions about long-standing issues in the compat layer---especially as long as there's a chance that the spec itself can be made unambiguous.