Skip to content

Add devpod cp to copy files to and from a workspace - #41

Open
renatoathaydes wants to merge 1 commit into
mainfrom
claude/devpod-file-copy-dfa7b9
Open

renatoathaydes wants to merge 1 commit into
mainfrom
claude/devpod-file-copy-dfa7b9

Conversation

@renatoathaydes

Copy link
Copy Markdown
Collaborator

Summary

Adds devpod cp, which copies files and directories between the local machine and a workspace's container, using scp's WORKSPACE:PATH syntax:

devpod cp my-file.txt my-workspace:                         # into the workspace folder
devpod cp ./here/file-here.txt my-workspace:file-in-container.txt
devpod cp --user root ./temp/ my-workspace:/usr/local/      # absolute path, as root
devpod cp my-workspace:/etc/os-release .                    # out of the container
  • A relative path in the workspace is relative to the workspace folder, including a custom workspaceFolder (read from the SSH config entry devpod up writes, falling back to /workspaces/<id>).
  • Where things land follows docker cp: into an existing directory, to the path otherwise. A destination ending in / is created (with parents) if missing, and a source ending in /. copies a directory's contents.
  • It copies as the workspace's user, so they own what's copied in; --user changes it, and permission errors from the container suggest --user root.
  • It runs over SFTP through the same connection devpod ssh makes (pkg/filetransfer holds the copy logic, for both directions).

Each file is written to a private (0600) temporary file next to its destination, given its permissions and modification time, and renamed over the destination. With local Docker the workspace folder is usually the local project folder mounted into the container, so this keeps devpod cp my-file.txt foo: run from that folder from truncating the file, lets read-only files (such as .git/objects) be replaced on a second copy, and keeps private files private while they're written. Symlinks are kept (and converted between \ and / for Windows), names the destination can't hold safely are skipped with a warning, and files are copied 8 at a time.

SFTP server fix

The SSH server in workspaces served SFTP in-process with sftp.WithServerWorkingDirectory, which pkg/sftp (up to the latest v1.13.12) also applies to symlink targets, so any SFTP client (sftp WORKSPACE.devpod, IDE sync) got link -> /home/user/a.txt for a relative a.txt. It now serves each SFTP session from a new hidden devpod helper sftp-server process started in the user's home directory, like OpenSSH's sftp-server. Containers get it the next time a CLI of a new version connects, as the agent is re-injected.

Test plan

  • Unit tests for the copy rules, run both local-to-local and through an in-process SFTP server, plus regression tests for copying a file onto itself, copying a read-only tree twice, a nested missing DEST/, skipped names, the private temporary file, and local vs. container errors (go test -race ./pkg/filetransfer/)
  • Unit tests for WORKSPACE:PATH parsing (including Windows drive letters) and for reading the workspace folder from the SSH config
  • golangci-lint v1.60.1 with CI's flags, go build for windows/amd64
  • Against a real Docker workspace (Alpine, non-root user, custom workspaceFolder): uploads and downloads of files and directories, copying onto itself from the mounted folder, read-only tree twice, relative symlinks, 0600 files, nested missing DEST/, the --user root hint, a relative symlink created with plain OpenSSH sftp, and 2000 small files in ~2.5 s
  • Not tested on Windows or with the Kubernetes provider

🤖 Generated with Claude Code

`devpod cp SRC WORKSPACE:DEST` and `devpod cp WORKSPACE:SRC DEST` copy
files and directories between the local machine and a workspace's
container, with scp's WORKSPACE:PATH syntax. A relative path in the
workspace is relative to the workspace folder (read from the SSH config
entry that `devpod up` writes), and where things land follows the rules
of `docker cp`. It copies as the workspace's user (`--user` to change),
over SFTP through the same connection `devpod ssh` makes.

Each file is written to a private temporary file that is renamed over
the destination, so that copying a file onto itself through the mounted
workspace folder doesn't truncate it, read-only files can be replaced,
and private files are never readable by others. Permissions,
modification times and symlinks are kept, names that the destination
can't hold safely are skipped, and files are copied concurrently.

The SSH server in workspaces now serves each SFTP session from a
`devpod helper sftp-server` process started in the user's home
directory, like OpenSSH's sftp-server. Serving it in-process with
sftp.WithServerWorkingDirectory made relative symlink targets absolute,
for any SFTP client.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Transfers can expose private contents through permissive temporary-file and directory creation, while unvalidated source basenames can escape the destination directory and overwrite files on Windows.

4 open issues
What changed in this PR

Adds devpod cp to the CLI for copying files and directories between the local machine and workspace containers over the existing SSH connection.

Changes:

  • Adds workspace path parsing, custom workspace-folder resolution, and --user selection.
  • Introduces shared SFTP copy logic with metadata preservation, symlink handling, atomic file replacement, and regression tests.
  • Moves SFTP sessions into helper processes to preserve relative symlink targets, and documents the new command.
File Description
pkg/​ssh/​server/​sftp_handler.go Starts a separate SFTP helper in the user’s home directory.
pkg/​ssh/​config.go Reads the workspace folder from SSH configuration.
pkg/​ssh/​config_test.go Tests workspace-folder flag parsing.
pkg/​filetransfer/​fs.go Adds local and SFTP filesystem adapters.
pkg/​filetransfer/​copy.go Implements copy rules, traversal, metadata handling, and concurrent transfers.
pkg/​filetransfer/​copy_test.go Tests transfer semantics and regression cases.
go.mod Marks two existing dependencies as direct dependencies.
docs/​pages/​developing-in-workspaces/​connect-to-a-workspace.mdx Documents copy syntax, destination rules, and user selection.
cmd/​root.go Registers the cp command.
cmd/​helper/​sftp_server.go Adds the SFTP server helper command.
cmd/​helper/​helper.go Registers the SFTP helper.
cmd/​cp.go Implements command parsing, connection setup, and transfer execution.
cmd/​cp_test.go Tests local and workspace path parsing, including Windows paths.
CHANGELOG.md Records the copy command and relative-symlink fix.

Review effort: Balanced

Comment thread pkg/filetransfer/copy.go
if dstExists && !dstInfo.IsDir() && (dstMustBeDir || srcInfo.IsDir()) {
return Stats{}, fmt.Errorf("%s is not a directory", dstPath)
}
target := dstPath
Comment thread pkg/filetransfer/copy.go
}

c.log.Debugf("Creating directory %s", e.dstPath)
err := c.dst.Mkdir(e.dstPath)
Comment thread pkg/filetransfer/fs.go
}

func (s SFTPFS) CreateNew(p string) (io.WriteCloser, error) {
f, err := s.Client.OpenFile(p, os.O_WRONLY|os.O_CREATE|os.O_EXCL)
Comment thread pkg/filetransfer/copy.go
}
defer in.Close()

tmp := c.dst.Join(e.dstDir, "."+c.dst.Base(e.dstPath)+".devpod-cp-"+randomSuffix())

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants