Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
2b0f11e
Remove fog-aws gem and all fog AWS blobstore support
kathap Jul 14, 2026
36f87d7
Fix broken specs after fog-aws removal
kathap Jul 14, 2026
936ba9c
Fix FogClient spec and LocalClient tmpdir cleanup
kathap Jul 14, 2026
0baaa1d
Revert changes to .rubocop_cc.yml
kathap Jul 14, 2026
0924232
comment adjusted
kathap Jul 14, 2026
698ab0e
fix: memoize FakeStorage#directories so stub expectations hold
kathap Jul 14, 2026
d4349f3
Repair FakeStorage test stub for fog_client_spec
kathap Jul 14, 2026
259575c
Fix delegate
kathap Jul 15, 2026
92a3cca
Fix OutsideOfExampleError by scoping LocalClient stub to specific ins…
kathap Jul 15, 2026
2733ecb
Fix tests
kathap Jul 15, 2026
ff084f7
Remove fog completely
kathap Jul 15, 2026
fbf160f
fix tests
kathap Jul 15, 2026
890f43e
Revert test_config to local-temp-storage and restore 302 download exp…
kathap Jul 15, 2026
1b70f87
Memoize blobstore clients in DependencyLocator; fix BlobNotFound with…
kathap Jul 16, 2026
a503a3d
Fix dependency locator
kathap Jul 16, 2026
108952f
Don't call logger in at_exit cleanup hook
kathap Jul 16, 2026
c290209
Fix fog-aws removal artifacts
kathap Jul 16, 2026
c8c0993
Fix upload tests
kathap Jul 16, 2026
97450cc
fix specs
kathap Jul 16, 2026
c8c0bdf
Change failed job output logs
kathap Jul 16, 2026
5f53011
Fix tests
kathap Jul 16, 2026
123a111
Fix tests
kathap Jul 16, 2026
d1353b8
Remove deleted file from rubocop
kathap Jul 17, 2026
8f8d1dc
Clean up fog references in storage_cli_client and fix doc spec stubs
kathap Jul 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ Lint/SuppressedException:
- 'lib/cloud_controller/database_uri_generator.rb'
- 'lib/vcap/pid_file.rb'
- 'spec/unit/actions/v2/route_create_spec.rb'
- 'spec/unit/lib/cloud_controller/blobstore/fog/fog_client_spec.rb'
- 'spec/unit/lib/services/service_brokers/service_broker_registration_spec.rb'
- 'spec/unit/models/services/service_broker_spec.rb'

Expand Down Expand Up @@ -581,7 +580,6 @@ Style/HashSlice:
- 'app/controllers/services/service_instances_controller.rb'
- 'app/controllers/services/service_plans_controller.rb'
- 'app/presenters/system_environment/service_binding_presenter.rb'
- 'lib/cloud_controller/blobstore/fog/fog_blob.rb'
- 'lib/cloud_controller/blobstore/storage_cli/storage_cli_blob.rb'
- 'lib/cloud_controller/blobstore/webdav/dav_blob.rb'
- 'lib/cloud_controller/telemetry_logger.rb'
Expand Down
3 changes: 0 additions & 3 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,6 @@ gem 'actionview', '~> 8.1.3'
gem 'activemodel', '~> 8.1.2'
gem 'railties', '~> 8.1.1'

gem 'fog-aws'
gem 'fog-core', '~> 2.6.0'

gem 'cf-uaa-lib', '~> 4.0.10'

group :db do
Expand Down
18 changes: 0 additions & 18 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -90,22 +90,6 @@ GEM
fluent-logger (0.10.0)
logger (~> 1.6)
msgpack (>= 1.0.0, < 2)
fog-aws (3.33.2)
base64 (>= 0.2, < 0.4)
fog-core (~> 2.6)
fog-json (~> 1.1)
fog-xml (~> 0.1)
fog-core (2.6.0)
builder
excon (~> 1.0)
formatador (>= 0.2, < 2.0)
mime-types
fog-json (1.3.0)
fog-core
multi_json (~> 1.10)
fog-xml (0.1.5)
fog-core
nokogiri (>= 1.5.11, < 2.0.0)
formatador (1.2.3)
reline
google-protobuf (4.35.1)
Expand Down Expand Up @@ -481,8 +465,6 @@ DEPENDENCIES
digest-xxhash
factory_bot (~> 6.5)
fluent-logger
fog-aws
fog-core (~> 2.6.0)
googleapis-common-protos (>= 1.8.0)
hashdiff
httpclient
Expand Down
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ When deployed via capi-release only:
| Provider | `blobstore_type` | Backends | Notes |
|----------|------------------|-----------------------------------------------|-------|
| Storage CLI | `storage-cli` | S3, S3-compatible, GCS, Azure, Alibaba Cloud | |
| Fog | `fog` | AWS, GCS, Alibaba Cloud, OpenStack, Local/NFS | **Default.** Local/NFS not recommended for production |
| WebDAV | `webdav` | WebDAV servers | |
| Local | `local`, `local-temp-storage` | Filesystem, NFS | Development and testing only |

Expand Down
4 changes: 0 additions & 4 deletions config/bosh-lite.yml
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,6 @@ resource_pool:
uri:
key_pair_id:
private_key: ""
fog_connection: {}

packages:
blobstore_type: webdav
Expand All @@ -153,7 +152,6 @@ packages:
uri:
key_pair_id:
private_key: ""
fog_connection: {}

droplets:
blobstore_type: webdav
Expand All @@ -168,7 +166,6 @@ droplets:
uri:
key_pair_id:
private_key: ""
fog_connection: {}
max_staged_droplets_stored: 5

buildpacks:
Expand All @@ -184,7 +181,6 @@ buildpacks:
uri:
key_pair_id:
private_key: ""
fog_connection: {}

db_encryption_key: db-encryption-key

Expand Down
8 changes: 0 additions & 8 deletions config/cloud_controller.yml
Original file line number Diff line number Diff line change
Expand Up @@ -182,29 +182,21 @@ resource_pool:
minimum_size: 1
resource_directory_key: "spec-cc-resources"
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}

packages:
app_package_directory_key: "cc-packages"
max_package_size: 42
max_valid_packages_stored: 42
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}

droplets:
droplet_directory_key: cc-droplets
max_staged_droplets_stored: 42
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}

buildpacks:
buildpack_directory_key: cc-buildpacks
blobstore_type: local-temp-storage
fog_connection: {}
fog_aws_storage_options: {}

db_encryption_key: "asdfasdfasdf"
database_encryption:
Expand Down
12 changes: 6 additions & 6 deletions decisions/0014-storage-clis-for-blobstore-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,14 @@
| Alibaba Cloud| ✅ Implemented | Uses [`storage-cli`](https://github.com/cloudfoundry/storage-cli/tree/main/alioss) with native type `alioss` |

**Configuration Migration Status:**
- The `blobstore_provider` field accepts both native storage-cli type names AND legacy fog names
- **Recommended:** Use native storage-cli type names (azurebs, s3, gcs, alioss)
- **Legacy fog names** (AzureRM, AWS, Google, aliyun) still supported for backwards compatibility
- The `blobstore_provider` field accepts native storage-cli type names (azurebs, s3, gcs, alioss)
- **Legacy fog names** (AzureRM, AWS, Google, aliyun) are no longer supported — fog gems have been fully removed
- **WebDAV/dav intentionally excluded** until fully supported
- **Timeline:** Legacy fog name support to be removed May 2026


## Context

Cloud Controller uses the fog gem family to interface with blobstores like Azure, AWS, GCP, and Alibaba Cloud.
Cloud Controller previously used the fog gem family to interface with blobstores like Azure, AWS, GCP, and Alibaba Cloud.
These Ruby gems are largely unmaintained, introducing risks such as:
* Dependency on deprecated SDKs (e.g., Azure SDK for Ruby)
* Blocking Ruby version upgrades
Expand Down Expand Up @@ -79,11 +77,13 @@ The `bosh-azure-storage-cli` needs to be extended with the following commands:

Other providers (AWS, GCP, Alibaba Cloud) will follow. Each will require equivalent blobstore clients and support for the above commands.
This will eventually allow us to remove all fog related gems from Cloud Controller.
> ✅ **All fog gems have been fully removed** (fog-aws, fog-core, fog-azure-rm, fog-google, fog-aliyun).
> ✅ **fog-aws and fog-core have been fully removed** as of this change.


## Consequences

* Enables the removal of `fog-azure-rm` and all other fog related gems
* ✅ Enabled the removal of all fog related gems — fog-aws, fog-core, fog-azure-rm, fog-google, and fog-aliyun have all been removed
* Reduces long-term maintenance burden and potential security issues
* Allows providers to be migrated independently
* Increases initial complexity during migration phase
Expand Down
12 changes: 12 additions & 0 deletions lib/cloud_controller/blobstore/blob.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@ module CloudController
module Blobstore
class Blob
CACHE_ATTRIBUTES = %i[etag last_modified created_at content_length].freeze

def internal_download_url
raise NotImplementedError
end

def public_download_url
raise NotImplementedError
end

def local_path
raise NotImplementedError
end
end
end
end
31 changes: 0 additions & 31 deletions lib/cloud_controller/blobstore/client_provider.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
require 'cloud_controller/blobstore/client'
require 'cloud_controller/blobstore/retryable_client'
require 'cloud_controller/blobstore/fog/fog_client'
require 'cloud_controller/blobstore/error_handling_client'
require 'cloud_controller/blobstore/webdav/dav_client'
require 'cloud_controller/blobstore/local/local_client'
Expand All @@ -18,8 +17,6 @@ def self.provide(options:, directory_key:, root_dir: nil, resource_type: nil)
provide_local(options, directory_key, root_dir, use_temp_storage: true)
when 'storage-cli'
provide_storage_cli(options, directory_key, root_dir, resource_type)
when 'fog', nil, ''
provide_fog(options, directory_key, root_dir)
else
provide_webdav(options, directory_key, root_dir)
end
Expand All @@ -28,34 +25,6 @@ def self.provide(options:, directory_key:, root_dir: nil, resource_type: nil)
class << self
private

def provide_fog(options, directory_key, root_dir)
cdn_uri = HashUtils.dig(options[:cdn], :uri)
cdn = CloudController::Blobstore::Cdn.make(cdn_uri)

client = FogClient.new(
connection_config: options.fetch(:fog_connection),
directory_key: directory_key,
cdn: cdn,
root_dir: root_dir,
min_size: options[:minimum_size],
max_size: options[:maximum_size],
aws_storage_options: options[:fog_aws_storage_options]
)

logger = Steno.logger('cc.blobstore')

# work around https://github.com/fog/fog/issues/3137
# and Fog raising an EOFError SocketError intermittently
# and https://github.com/fog/fog-aws/issues/264
# and https://github.com/fog/fog-aws/issues/265
# and intermittent GCS blobstore download errors
errors = [Excon::Errors::BadRequest, Excon::Errors::SocketError, SystemCallError,
Excon::Errors::InternalServerError, Excon::Errors::ServiceUnavailable, OpenSSL::OpenSSLError]
retryable_client = RetryableClient.new(client:, errors:, logger:)

Client.new(ErrorHandlingClient.new(SafeDeleteClient.new(retryable_client, root_dir)))
end

def provide_local(options, directory_key, root_dir, use_temp_storage:)
client = LocalClient.new(
directory_key: directory_key,
Expand Down
42 changes: 0 additions & 42 deletions lib/cloud_controller/blobstore/fog/cdn.rb

This file was deleted.

40 changes: 0 additions & 40 deletions lib/cloud_controller/blobstore/fog/fog_blob.rb

This file was deleted.

Loading
Loading