⚠️ This issue respects the following points: ⚠️
Bug description
When uploading a large file as chunked v2 mode, with a nextcloud server using S3 as external storage (not primary storage), the server does not proceed to chunk v2 and fallback to chunk v1. Thus the file is in the server disk twice the size (one as chunks in "upload" of the user data, one for aws-php-s3 in /tmp to upload to s3).
Steps to reproduce
- deploy nextcloud 34.0.1 with nginx and fpm
- use rclone fork that enabled chunk v2 (https://github.com/antoinetran/rclone/tree/fix-8321-nextcloud-chunk-v2-upload)
- upload using rclone image
Expected behavior
When uploading as chunk v2 mode (see https://docs.nextcloud.com/server/latest/developer_manual/client_apis/WebDAV/chunking.html), the chunks should be uploaded directly to s3 backend instead of disk as intermediate data.
Nextcloud Server version
34
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.5
Web server
Nginx
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Fresh Nextcloud Server install
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
Configuration report
{
"system": {
"htaccess.RewriteBase": "\/",
"memcache.local": "\\OC\\Memcache\\APCu",
"apps_paths": [
{
"path": "\/var\/www\/html\/apps",
"url": "\/apps",
"writable": false
},
{
"path": "\/var\/www\/html\/custom_apps",
"url": "\/custom_apps",
"writable": true
}
],
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"openmetrics_allowed_clients": [
"127.0.0.1",
"10.42.0.0\/16",
"10.43.0.0\/16"
],
"check_data_directory_permissions": false,
"forwarded_for_headers": [
"HTTP_X_FORWARDED_FOR"
],
"skeletondirectory": "\/dev\/null",
"filelocking.enabled": false,
"debug": false,
"enable_previews": false,
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"password": "***REMOVED SENSITIVE VALUE***",
"port": 6379
},
"upgrade.disable-web": true,
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"localhost",
"my-nextcloud.globalstorage-nextcloud",
"globalstorage-nextcloud-dcc-fi-csc-okd-pm-sys2.2.rahtiapp.fi"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "34.0.1.2",
"overwrite.cli.url": "http:\/\/localhost",
"instanceid": "***REMOVED SENSITIVE VALUE***",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory"
}
List of activated Apps
Enabled:
- appstore: 1.0.0
- cloud_federation_api: 1.18.0
- dav: 1.39.0
- federatedfilesharing: 1.24.0
- files: 2.6.0
- files_external: 1.26.0
- files_sharing: 1.26.0
- lookup_server_connector: 1.22.0
- oauth2: 1.22.0
- profile: 1.3.0
- provisioning_api: 1.24.0
- serverinfo: 6.0.0
- settings: 1.17.0
- theming: 2.9.0
- twofactor_backupcodes: 1.23.0
- user_ldap: 1.25.0
- viewer: 7.0.0-dev.0
- workflowengine: 2.16.
Nextcloud Signing status
Results
=======
- core
- EXTRA_FILE
- tmp/.cache/fontconfig/7ef2298fde41cc6eeb7af42e48b7d293-le64.cache-9
- tmp/.cache/fontconfig/CACHEDIR.TAG
- tmp/.cache/fontconfig/3830d5c3ddfd5cd38a049b759396e72e-le64.cache-9
- tmp/.cache/fontconfig/4c599c202bc5c08e2d34565a40eac3b2-le64.cache-9
- tmp/.cache/fontconfig/d589a48862398ed80a3d6066f4f56f4c-le64.cache-9
- customconfig.log
Nextcloud Logs
Additional info
No response
Bug description
When uploading a large file as chunked v2 mode, with a nextcloud server using S3 as external storage (not primary storage), the server does not proceed to chunk v2 and fallback to chunk v1. Thus the file is in the server disk twice the size (one as chunks in "upload" of the user data, one for aws-php-s3 in /tmp to upload to s3).
Steps to reproduce
Expected behavior
When uploading as chunk v2 mode (see https://docs.nextcloud.com/server/latest/developer_manual/client_apis/WebDAV/chunking.html), the chunks should be uploaded directly to s3 backend instead of disk as intermediate data.
Nextcloud Server version
34
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.5
Web server
Nginx
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Fresh Nextcloud Server install
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
Configuration report
{ "system": { "htaccess.RewriteBase": "\/", "memcache.local": "\\OC\\Memcache\\APCu", "apps_paths": [ { "path": "\/var\/www\/html\/apps", "url": "\/apps", "writable": false }, { "path": "\/var\/www\/html\/custom_apps", "url": "\/custom_apps", "writable": true } ], "trusted_proxies": "***REMOVED SENSITIVE VALUE***", "openmetrics_allowed_clients": [ "127.0.0.1", "10.42.0.0\/16", "10.43.0.0\/16" ], "check_data_directory_permissions": false, "forwarded_for_headers": [ "HTTP_X_FORWARDED_FOR" ], "skeletondirectory": "\/dev\/null", "filelocking.enabled": false, "debug": false, "enable_previews": false, "memcache.distributed": "\\OC\\Memcache\\Redis", "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "password": "***REMOVED SENSITIVE VALUE***", "port": 6379 }, "upgrade.disable-web": true, "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "localhost", "my-nextcloud.globalstorage-nextcloud", "globalstorage-nextcloud-dcc-fi-csc-okd-pm-sys2.2.rahtiapp.fi" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "34.0.1.2", "overwrite.cli.url": "http:\/\/localhost", "instanceid": "***REMOVED SENSITIVE VALUE***", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory" }List of activated Apps
Nextcloud Signing status
Nextcloud Logs
No relevant logAdditional info
No response