diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a31f05..9225746 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,22 @@ ## [Unreleased] +## [0.10.0] - 2026-10-21 +- Drop the `faraday` and `graphlient` dependencies in favor of Ruby's built-in `Net::HTTP` (#56) +- Fix `query.get` failing with "interface conversion: interface {} is nil, not int" on Weaviate >= 1.26.4. Only the arguments that are passed are sent in GraphQL queries (#63) +- Add `proxy:` option to `Weaviate::Client.new`. The `http_proxy`/`https_proxy` env vars are respected by default (#55) +- Add `client.collections` (`Weaviate::Collections`) to match Weaviate's terminology. `client.schema` and `Weaviate::Schema` remain as aliases (#69) +- API keys are filtered out of request logs +- Fix `collections.update` failing with 422 on immutable settings: the changes are now merged into the current collection definition, since Weaviate replaces the whole definition on update +- Fix `objects.exists?` raising on missing objects instead of returning `false` +- Fix `objects.batch_create(tenant:)` being ignored by Weaviate: the tenant is now set on each object +- Fix `query.aggs` failing with "groupBy must be a list, instead got: " when `group_by`/`object_limit` are not passed + +### Breaking +- HTTP errors are raised as `Weaviate::HTTPError` subclasses (e.g. `Weaviate::ResourceNotFound`, `Weaviate::ServerError`) instead of `Faraday::Error` subclasses. Network failures raise `Weaviate::ConnectionFailed`/`Weaviate::TimeoutError` +- GraphQL errors are raised as `Weaviate::GraphQLError` (a `Weaviate::Error` subclass) whose message is the joined error messages +- `client.connection` returns a `Weaviate::Connection` and `client.graphql` returns a `Weaviate::GraphQL` instead of Faraday/Graphlient clients +- The `adapter:` option is deprecated and ignored + ## [0.9.2] - 2024-10-01 - Weaviate::Client constructor accepts customer logger: to be passed in diff --git a/Gemfile b/Gemfile index 097075c..f5b5abf 100644 --- a/Gemfile +++ b/Gemfile @@ -9,4 +9,6 @@ gem "rake", "~> 13.0" gem "rspec", "~> 3.0" gem "standard", "~> 1.25.0" -gem "graphql-client", "~> 0.19.0" +gem "racc" # required by standard, previously pulled in through graphlient + +gem "ostruct" # used in specs, no longer a default gem on Ruby >= 3.5 diff --git a/Gemfile.lock b/Gemfile.lock index 5e599ed..0699c70 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -1,52 +1,21 @@ PATH remote: . specs: - weaviate-ruby (0.9.2) - faraday (>= 2.0.1, < 3.0) - graphlient (>= 0.7.0, < 0.9.0) + weaviate-ruby (0.10.0) + logger GEM remote: https://rubygems.org/ specs: - activesupport (7.1.3) - base64 - bigdecimal - concurrent-ruby (~> 1.0, >= 1.0.2) - connection_pool (>= 2.2.5) - drb - i18n (>= 1.6, < 2) - minitest (>= 5.1) - mutex_m - tzinfo (~> 2.0) ast (2.4.2) - base64 (0.2.0) - bigdecimal (3.1.6) byebug (11.1.3) coderay (1.1.3) - concurrent-ruby (1.2.3) - connection_pool (2.4.1) diff-lcs (1.5.0) - drb (2.2.0) - ruby2_keywords - faraday (2.7.10) - faraday-net_http (>= 2.0, < 3.1) - ruby2_keywords (>= 0.0.4) - faraday-net_http (3.0.2) - graphlient (0.7.0) - faraday (~> 2.0) - graphql-client - graphql (2.2.6) - racc (~> 1.4) - graphql-client (0.19.0) - activesupport (>= 3.0) - graphql - i18n (1.14.1) - concurrent-ruby (~> 1.0) json (2.6.3) language_server-protocol (3.17.0.3) + logger (1.7.0) method_source (1.0.0) - minitest (5.21.2) - mutex_m (0.2.0) + ostruct (0.6.3) parallel (1.22.1) parser (3.2.1.1) ast (~> 2.4.1) @@ -56,7 +25,7 @@ GEM pry-byebug (3.10.1) byebug (~> 11.0) pry (>= 0.13, < 0.15) - racc (1.7.3) + racc (1.8.1) rainbow (3.1.1) rake (13.0.6) rdiscount (2.2.7.4) @@ -91,26 +60,26 @@ GEM rubocop (>= 1.7.0, < 2.0) rubocop-ast (>= 0.4.0) ruby-progressbar (1.13.0) - ruby2_keywords (0.0.5) standard (1.25.3) language_server-protocol (~> 3.17.0.2) rubocop (~> 1.48.1) rubocop-performance (~> 1.16.0) - tzinfo (2.0.6) - concurrent-ruby (~> 1.0) unicode-display_width (2.4.2) yard (0.9.37) PLATFORMS arm64-darwin-21 arm64-darwin-23 + arm64-darwin-24 + ruby x86_64-darwin-19 x86_64-darwin-21 x86_64-linux DEPENDENCIES - graphql-client (~> 0.19.0) + ostruct pry-byebug (~> 3.9) + racc rake (~> 13.0) rdiscount rspec (~> 3.0) diff --git a/README.md b/README.md index 0d60922..249af59 100644 --- a/README.md +++ b/README.md @@ -40,15 +40,27 @@ client = Weaviate::Client.new( url: 'https://some-endpoint.weaviate.network', # Replace with your endpoint api_key: '', # Weaviate API key model_service: :openai, # Service that will be used to generate vectors. Possible values: :openai, :azure_openai, :cohere, :huggingface, :google_palm - model_service_api_key: 'xxxxxxx' # Either OpenAI, Azure OpenAI, Cohere, Hugging Face or Google PaLM api key + model_service_api_key: 'xxxxxxx', # Either OpenAI, Azure OpenAI, Cohere, Hugging Face or Google PaLM api key + proxy: 'http://user:pass@localhost:8888', # Optional. Defaults to the http_proxy/https_proxy env vars + logger: Logger.new($stdout) # Optional. Requests and responses are logged with API keys filtered out ) ``` -### Using the Schema endpoints +### Errors + +All errors inherit from `Weaviate::Error`: + +- `Weaviate::HTTPError` - Weaviate responded with a 4xx/5xx status (`#status` and `#body` are available). Subclasses: `Weaviate::BadRequestError`, `Weaviate::UnauthorizedError`, `Weaviate::ForbiddenError`, `Weaviate::ResourceNotFound`, `Weaviate::ConflictError`, `Weaviate::UnprocessableEntityError`, `Weaviate::ServerError` +- `Weaviate::GraphQLError` - a GraphQL query returned errors (`#errors` is available) +- `Weaviate::ConnectionFailed` / `Weaviate::TimeoutError` - Weaviate could not be reached + +### Using the Collections endpoints + +`client.collections` was previously called `client.schema`, which still works as an alias. ```ruby -# Creating a new data object class in the schema -client.schema.create( +# Creating a new collection +client.collections.create( class_name: 'Question', description: 'Information from a Jeopardy! question', properties: [ @@ -70,21 +82,21 @@ client.schema.create( vectorizer: "text2vec-openai" ) -# Get a single class from the schema -client.schema.get(class_name: 'Question') +# Get a single collection +client.collections.get(class_name: 'Question') -# Get the schema -client.schema.list() +# List all collections +client.collections.list() -# Update settings of an existing schema class. +# Update settings of an existing collection. # Does not support modifying existing properties. -client.schema.update( +client.collections.update( class_name: 'Question', description: 'Information from a Wheel of Fortune question' ) # Adding a new property -client.schema.add_property( +client.collections.add_property( class_name: 'Question', property: { "dataType": ["boolean"], @@ -92,14 +104,14 @@ client.schema.add_property( } ) -# Inspect the shards of a class -client.schema.shards(class_name: 'Question') +# Inspect the shards of a collection +client.collections.shards(class_name: 'Question') -# Remove a class (and all data in the instances) from the schema. -client.schema.delete(class_name: 'Question') +# Remove a collection (and all data in the instances). +client.collections.delete(class_name: 'Question') -# Creating a new data object class in the schema while configuring the vectorizer on the schema and on individual properties (Ollama example) -client.schema.create( +# Creating a new collection while configuring the vectorizer on the collection and on individual properties (Ollama example) +client.collections.create( class_name: 'Question', description: 'Information from a Jeopardy! question', properties: [ @@ -140,9 +152,9 @@ client.schema.create( }, ) -# Creating named schemas +# Creating collections with named vectors -client.schema.create( +client.collections.create( class_name: 'ArticleNV', description: 'Articles with named vectors', properties: [ @@ -306,6 +318,15 @@ client.query.get class_name: 'Question', where: '{ operator: Like, valueText: "S client.query.get class_name: 'Question', fields: 'answer question category _additional { id }', after: "3c5f7039-37f3-4244-b3e2-8f4a083e448d", limit: "1" +# Limiting results by distance or certainty: pass `distance` or `certainty` along with `near_text`, `near_vector` +# or `near_object`, and request them back via `_additional`. +client.query.get( + class_name: 'Question', + fields: 'question _additional { distance certainty }', + near_text: '{ concepts: ["biology"], distance: 0.6 }', # or certainty: 0.7 + limit: 5 +) + # Named vector query - uses targetVectors query_title = client.query.get( class_name: 'ArticleNV', @@ -403,10 +424,10 @@ client.ready? ### Tenants -Any schema can be multi-tenant by passing in these options for to `schema.create()`. +Any collection can be multi-tenant by passing in these options to `collections.create()`. ```ruby -client.schema.create( +client.collections.create( # Other keys... multi_tenant: true, auto_tenant_creation: true, diff --git a/lib/weaviate.rb b/lib/weaviate.rb index f8290d0..01eb063 100644 --- a/lib/weaviate.rb +++ b/lib/weaviate.rb @@ -1,12 +1,15 @@ # frozen_string_literal: true require_relative "weaviate/version" +require_relative "weaviate/error" module Weaviate autoload :Base, "weaviate/base" autoload :Client, "weaviate/client" - autoload :Error, "weaviate/error" - autoload :Schema, "weaviate/schema" + autoload :Connection, "weaviate/connection" + autoload :Collections, "weaviate/collections" + autoload :Schema, "weaviate/collections" + autoload :GraphQL, "weaviate/graphql" autoload :Meta, "weaviate/meta" autoload :Objects, "weaviate/objects" autoload :OIDC, "weaviate/oidc" diff --git a/lib/weaviate/client.rb b/lib/weaviate/client.rb index ed8da57..af4c845 100644 --- a/lib/weaviate/client.rb +++ b/lib/weaviate/client.rb @@ -1,11 +1,10 @@ # frozen_string_literal: true -require "faraday" -require "graphlient" +require "logger" module Weaviate class Client - attr_reader :url, :api_key, :model_service, :model_service_api_key, :adapter, :logger + attr_reader :url, :api_key, :model_service, :model_service_api_key, :proxy, :logger API_VERSION = "v1" @@ -18,21 +17,30 @@ class Client google_palm: "X-Palm-Api-Key" } + # @param url [String] The Weaviate instance URL + # @param api_key [String] The Weaviate API key + # @param model_service [Symbol] The service used to generate vectors, one of the API_KEY_HEADERS keys + # @param model_service_api_key [String] The API key for the model service + # @param proxy [String] Proxy URL, e.g. "http://user:pass@localhost:8888". Defaults to the http_proxy/https_proxy env vars. + # @param logger [Logger] Logger for requests and responses + # @param adapter [Object] Deprecated and ignored, Faraday is no longer used def initialize( url:, api_key: nil, model_service: nil, model_service_api_key: nil, - adapter: Faraday.default_adapter, - logger: nil + proxy: nil, + logger: nil, + adapter: nil ) validate_model_service!(model_service) unless model_service.nil? + warn "[weaviate-ruby] The `adapter:` option is deprecated and ignored, Faraday is no longer used." unless adapter.nil? @url = url @api_key = api_key @model_service = model_service @model_service_api_key = model_service_api_key - @adapter = adapter + @proxy = proxy @logger = logger || Logger.new($stdout) end @@ -40,8 +48,13 @@ def oidc Weaviate::OIDC.new(client: self).get end + def collections + @collections ||= Weaviate::Collections.new(client: self) + end + + # @deprecated Weaviate now refers to schema classes as collections. Use {#collections} instead. def schema - @schema ||= Weaviate::Schema.new(client: self) + collections end def meta @@ -81,43 +94,27 @@ def query end def graphql - headers = {} - - if model_service && model_service_api_key - headers[API_KEY_HEADERS[model_service]] = model_service_api_key - end - - if api_key - headers["Authorization"] = "Bearer #{api_key}" - end - - @graphql ||= Graphlient::Client.new( - "#{url}/#{API_VERSION}/graphql", - headers: headers, - http_options: { - read_timeout: 20, - write_timeout: 30 - } - ) + @graphql ||= Weaviate::GraphQL.new(client: self) end def connection - @connection ||= Faraday.new(url: "#{url}/#{API_VERSION}/") do |faraday| - if api_key - faraday.request :authorization, :Bearer, api_key - end - faraday.request :json - faraday.response :logger, logger, {headers: true, bodies: true, errors: true} - faraday.response :json, content_type: /\bjson$/ - faraday.response :raise_error - faraday.adapter adapter - - faraday.headers[API_KEY_HEADERS[model_service]] = model_service_api_key if model_service && model_service_api_key - end + @connection ||= Weaviate::Connection.new( + url: "#{url}/#{API_VERSION}", + headers: headers, + logger: logger, + proxy: proxy + ) end private + def headers + headers = {} + headers["Authorization"] = "Bearer #{api_key}" if api_key + headers[API_KEY_HEADERS[model_service]] = model_service_api_key if model_service && model_service_api_key + headers + end + def validate_model_service!(model_service) unless API_KEY_HEADERS.key?(model_service) raise ArgumentError, "Invalid model service: #{model_service}. Acceptable values are: #{API_KEY_HEADERS.keys.join(", ")}" diff --git a/lib/weaviate/schema.rb b/lib/weaviate/collections.rb similarity index 70% rename from lib/weaviate/schema.rb rename to lib/weaviate/collections.rb index 8bf9bf9..cbfd703 100644 --- a/lib/weaviate/schema.rb +++ b/lib/weaviate/collections.rb @@ -1,16 +1,17 @@ # frozen_string_literal: true module Weaviate - class Schema < Base + # Manage collections (formerly known as "classes" in the Weaviate schema). + class Collections < Base PATH = "schema" - # Dumps the current Weaviate schema. The result contains an array of objects. + # Lists all collections in the Weaviate schema. def list response = client.connection.get(PATH) response.body end - # Get a single class from the schema + # Get a single collection def get(class_name:) response = client.connection.get("#{PATH}/#{class_name}") @@ -21,7 +22,7 @@ def get(class_name:) end end - # Create a new data object class in the schema. + # Create a new collection. def create( class_name:, description: nil, @@ -35,7 +36,7 @@ def create( module_config: nil, inverted_index_config: nil, replication_config: nil, - vector_config: nil # added for named vector support + vector_config: nil ) response = client.connection.post(PATH) do |req| req.body = {} @@ -57,13 +58,13 @@ def create( req.body["invertedIndexConfig"] = inverted_index_config unless inverted_index_config.nil? req.body["replicationConfig"] = replication_config unless replication_config.nil? - req.body["vectorConfig"] = vector_config unless vector_config.nil? # Added for multi vector support + req.body["vectorConfig"] = vector_config unless vector_config.nil? end response.body end - # Remove a class (and all data in the instances) from the schema. + # Remove a collection (and all data in the instances). def delete(class_name:) response = client.connection.delete("#{PATH}/#{class_name}") @@ -74,10 +75,9 @@ def delete(class_name:) end end - # Update settings of an existing schema class. - # TODO: Fix it. - # This endpoint keeps returning the following error: - # => {"error"=>[{"message"=>"properties cannot be updated through updating the class. Use the add property feature (e.g. \"POST /v1/schema/{className}/properties\") to add additional properties"}]} + # Update settings of an existing collection. + # Weaviate replaces the whole collection definition on update, so the provided settings are merged + # into the current definition. Immutable settings and existing properties cannot be changed. def update( class_name:, description: nil, @@ -87,27 +87,29 @@ def update( module_config: nil, properties: nil, inverted_index_config: nil, - replication_config: nil + replication_config: nil, + vector_config: nil ) + changes = { + "description" => description, + "vectorIndexType" => vector_index_type, + "vectorIndexConfig" => vector_index_config, + "vectorizer" => vectorizer, + "moduleConfig" => module_config, + "properties" => properties, + "invertedIndexConfig" => inverted_index_config, + "replicationConfig" => replication_config, + "vectorConfig" => vector_config + }.compact + response = client.connection.put("#{PATH}/#{class_name}") do |req| - req.body = {} - req.body["class"] = class_name unless class_name.nil? - req.body["description"] = description unless description.nil? - req.body["vectorIndexType"] = vector_index_type unless vector_index_type.nil? - req.body["vectorIndexConfig"] = vector_index_config unless vector_index_config.nil? - req.body["vectorizer"] = vectorizer unless vectorizer.nil? - req.body["moduleConfig"] = module_config unless module_config.nil? - req.body["properties"] = properties unless properties.nil? - req.body["invertedIndexConfig"] = inverted_index_config unless inverted_index_config.nil? - req.body["replicationConfig"] = replication_config unless replication_config.nil? + req.body = deep_merge(get(class_name: class_name), JSON.parse(changes.to_json)) end - if response.success? - end response.body end - # Adds one or more tenants to a class. + # Adds one or more tenants to a collection. def add_tenants( class_name:, tenants: @@ -119,13 +121,13 @@ def add_tenants( response.body end - # List tenants of a class. + # List tenants of a collection. def list_tenants(class_name:) response = client.connection.get("#{PATH}/#{class_name}/tenants") response.body end - # Remove one or more tenants from a class. + # Remove one or more tenants from a collection. def remove_tenants( class_name:, tenants: @@ -140,7 +142,7 @@ def remove_tenants( response.body end - # Add a property to an existing schema class. + # Add a property to an existing collection. def add_property( class_name:, property: @@ -154,7 +156,7 @@ def add_property( response.body end - # Inspect the shards of a class + # Inspect the shards of a collection def shards(class_name:) response = client.connection.get("#{PATH}/#{class_name}/shards") response.body if response.success? @@ -173,10 +175,19 @@ def update_shard_status(class_name:, shard_name:, status:) private + def deep_merge(current, changes) + current.merge(changes) do |_key, current_value, new_value| + (current_value.is_a?(Hash) && new_value.is_a?(Hash)) ? deep_merge(current_value, new_value) : new_value + end + end + def validate_status!(status) unless %w[READONLY READY].include?(status.to_s.upcase) raise ArgumentError, 'status must be either "READONLY" or "READY"' end end end + + # @deprecated Weaviate now refers to schema classes as collections. Use {Collections} instead. + Schema = Collections end diff --git a/lib/weaviate/connection.rb b/lib/weaviate/connection.rb new file mode 100644 index 0000000..9ca4472 --- /dev/null +++ b/lib/weaviate/connection.rb @@ -0,0 +1,145 @@ +# frozen_string_literal: true + +require "json" +require "net/http" +require "uri" + +module Weaviate + # A minimal HTTP client built on Net::HTTP + class Connection + Request = Struct.new(:params, :headers, :body) + + Response = Struct.new(:status, :reason_phrase, :headers, :body, keyword_init: true) do + def success? + (200..299).cover?(status) + end + end + + METHODS = { + get: Net::HTTP::Get, + head: Net::HTTP::Head, + post: Net::HTTP::Post, + put: Net::HTTP::Put, + patch: Net::HTTP::Patch, + delete: Net::HTTP::Delete + } + + ERRORS = { + 400 => BadRequestError, + 401 => UnauthorizedError, + 403 => ForbiddenError, + 404 => ResourceNotFound, + 409 => ConflictError, + 422 => UnprocessableEntityError + } + + FILTERED_HEADERS = /authorization|api-key/i + + attr_reader :url, :headers, :logger, :proxy + + def initialize(url:, headers: {}, logger: nil, proxy: nil) + @url = url.chomp("/") + @headers = headers + @logger = logger + @proxy = proxy && URI(proxy) + end + + METHODS.each_key do |method| + define_method(method) do |path, &block| + request(method, path, &block) + end + end + + private + + def request(method, path) + req = Request.new({}, {}, nil) + yield req if block_given? + + uri = URI("#{url}/#{path}") + uri.query = URI.encode_www_form(req.params) unless req.params.nil? || req.params.empty? + + http_request = METHODS.fetch(method).new(uri) + headers.merge(req.headers).each { |k, v| http_request[k] = v } + + unless req.body.nil? + http_request["Content-Type"] = "application/json" + http_request.body = req.body.is_a?(String) ? req.body : req.body.to_json + end + + log_request(http_request, uri) + + http_response = perform(uri, http_request) + response = build_response(http_response) + + log_response(response, http_response) + + raise_error!(response) unless response.success? + + response + end + + def perform(uri, http_request) + Net::HTTP.start( + uri.host, + uri.port, + proxy ? proxy.host : :ENV, + proxy&.port, + proxy&.user, + proxy&.password, + use_ssl: uri.scheme == "https" + ) do |http| + http.request(http_request) + end + rescue Net::OpenTimeout, Net::ReadTimeout, Net::WriteTimeout => e + raise Weaviate::TimeoutError, e.message + rescue SocketError, SystemCallError, IOError, OpenSSL::SSL::SSLError => e + raise Weaviate::ConnectionFailed, e.message + end + + def build_response(http_response) + body = http_response.body.to_s + + if http_response["Content-Type"].to_s.match?(/\bjson\b/) && !body.empty? + body = JSON.parse(body) + end + + Response.new( + status: http_response.code.to_i, + reason_phrase: http_response.message, + headers: http_response.to_hash, + body: body + ) + end + + def raise_error!(response) + error_class = ERRORS.fetch(response.status) do + (response.status >= 500) ? ServerError : ClientError + end + + raise error_class.new(response: response) + end + + def log_request(http_request, uri) + return if logger.nil? + + logger.info("request") { "#{http_request.method} #{uri}" } + logger.info("request") { format_headers(http_request.each_header.to_h) } + logger.info("request") { http_request.body } unless http_request.body.nil? + end + + def log_response(response, http_response) + return if logger.nil? + + logger.info("response") { "Status #{response.status}" } + logger.info("response") { format_headers(http_response.each_header.to_h) } + logger.info("response") { http_response.body } unless http_response.body.to_s.empty? + end + + def format_headers(headers) + headers.map do |k, v| + "#{k}: #{k.match?(FILTERED_HEADERS) ? "[FILTERED]" : v.inspect}" + end.join("\n") + end + end +end diff --git a/lib/weaviate/error.rb b/lib/weaviate/error.rb index 5f936c0..98dbca7 100644 --- a/lib/weaviate/error.rb +++ b/lib/weaviate/error.rb @@ -3,4 +3,56 @@ module Weaviate class Error < StandardError end + + # Raised when Weaviate cannot be reached + class ConnectionFailed < Error + end + + # Raised when a request times out + class TimeoutError < ConnectionFailed + end + + # Raised when Weaviate responds with a 4xx or 5xx status + class HTTPError < Error + attr_reader :response + + def initialize(message = nil, response: nil) + @response = response + super(message || "the server responded with status #{response&.status}") + end + + def status + response&.status + end + + def body + response&.body + end + end + + class ClientError < HTTPError; end + + class BadRequestError < ClientError; end + + class UnauthorizedError < ClientError; end + + class ForbiddenError < ClientError; end + + class ResourceNotFound < ClientError; end + + class ConflictError < ClientError; end + + class UnprocessableEntityError < ClientError; end + + class ServerError < HTTPError; end + + # Raised when a GraphQL query returns errors + class GraphQLError < Error + attr_reader :errors + + def initialize(errors) + @errors = errors + super(errors.map { |e| e["message"] }.join(", ")) + end + end end diff --git a/lib/weaviate/graphql.rb b/lib/weaviate/graphql.rb new file mode 100644 index 0000000..29eb4e0 --- /dev/null +++ b/lib/weaviate/graphql.rb @@ -0,0 +1,19 @@ +# frozen_string_literal: true + +module Weaviate + class GraphQL < Base + PATH = "graphql" + + # Execute a GraphQL query and return the "data" portion of the response. + def execute(query) + response = client.connection.post(PATH) do |req| + req.body = {query: query} + end + + errors = response.body["errors"] + raise Weaviate::GraphQLError.new(errors) if errors && !errors.empty? + + response.body["data"] + end + end +end diff --git a/lib/weaviate/objects.rb b/lib/weaviate/objects.rb index 7429607..90f542a 100644 --- a/lib/weaviate/objects.rb +++ b/lib/weaviate/objects.rb @@ -66,10 +66,12 @@ def batch_create( ) validate_consistency_level!(consistency_level) unless consistency_level.nil? + # Weaviate expects the tenant on each object rather than on the batch + objects = objects.map { |object| {tenant: tenant}.merge(object) } unless tenant.nil? + response = client.connection.post("batch/#{PATH}") do |req| req.params["consistency_level"] = consistency_level.to_s.upcase unless consistency_level.nil? req.body = {objects: objects} - req.body["tenant"] = tenant unless tenant.nil? end response.body @@ -109,6 +111,8 @@ def exists?( end response.status == 204 + rescue Weaviate::ResourceNotFound + false end # Update an individual data object based on its uuid. diff --git a/lib/weaviate/query.rb b/lib/weaviate/query.rb index 6b1e247..3e3a324 100644 --- a/lib/weaviate/query.rb +++ b/lib/weaviate/query.rb @@ -20,12 +20,15 @@ def get( bm25: nil, ask: nil ) - response = client.graphql.execute( + data = client.graphql.execute( get_query( class_name: class_name, - tenant: tenant, fields: fields, + after: after, + tenant: tenant, + limit: limit, autocut: autocut, + offset: offset, sort: sort, where: where, near_text: near_text, @@ -35,14 +38,9 @@ def get( with_hybrid: with_hybrid, bm25: bm25, ask: ask - ), - after: after, - limit: limit, - offset: offset + ) ) - response.original_hash.dig("data", "Get", class_name) - rescue Graphlient::Errors::ExecutionError => error - raise Weaviate::Error.new(error.response.data.get.errors.messages.to_h) + data.dig("Get", class_name) end def aggs( @@ -57,23 +55,21 @@ def aggs( near_object: nil, group_by: nil ) - response = client.graphql.execute( + data = client.graphql.execute( aggs_query( class_name: class_name, - tenant: tenant, fields: fields, + tenant: tenant, where: where, + object_limit: object_limit, near_text: near_text, near_vector: near_vector, near_image: near_image, - near_object: near_object - ), - group_by: group_by, - object_limit: object_limit + near_object: near_object, + group_by: group_by + ) ) - response.original_hash.dig("data", "Aggregate", class_name) - rescue Graphlient::Errors::ExecutionError => error - raise Weaviate::Error.new(error.response.data.aggregate.errors.messages.to_h) + data.dig("Aggregate", class_name) end def explore( @@ -88,29 +84,29 @@ def explore( near_image: nil, near_object: nil ) - response = client.graphql.execute( + # `after` is accepted for backwards compatibility, but Explore does not support it + data = client.graphql.execute( explore_query( fields: fields, + limit: limit, + offset: offset, sort: sort, where: where, near_text: near_text, near_vector: near_vector, near_image: near_image, near_object: near_object - ), - after: after, - limit: limit, - offset: offset + ) ) - response.original_hash.dig("data", "Explore") - rescue Graphlient::Errors::ExecutionError => error - raise Weaviate::Error.new(error.to_s) + data["Explore"] end private def explore_query( fields:, + limit: nil, + offset: nil, where: nil, near_text: nil, near_vector: nil, @@ -118,21 +114,20 @@ def explore_query( near_object: nil, sort: nil ) - client.graphql.parse <<~GRAPHQL - query( - $limit: Int, - $offset: Int - ) { - Explore ( - limit: $limit, - offset: $offset, - #{(!near_text.nil?) ? "nearText: #{near_text}" : ""}, - #{(!near_vector.nil?) ? "nearVector: #{near_vector}" : ""}, - #{(!near_image.nil?) ? "nearImage: #{near_image}" : ""}, - #{(!near_object.nil?) ? "nearObject: #{near_object}" : ""}, - #{(!where.nil?) ? "where: #{where}" : ""}, - #{(!sort.nil?) ? "sort: #{sort}" : ""} - ) { + arguments = build_arguments( + limit: integer_value(limit), + offset: integer_value(offset), + nearText: near_text, + nearVector: near_vector, + nearImage: near_image, + nearObject: near_object, + where: where, + sort: sort + ) + + <<~GRAPHQL + { + Explore#{arguments} { #{fields} } } @@ -142,8 +137,11 @@ def explore_query( def get_query( class_name:, fields:, - autocut: nil, + after: nil, tenant: nil, + limit: nil, + autocut: nil, + offset: nil, where: nil, near_text: nil, near_vector: nil, @@ -154,29 +152,27 @@ def get_query( ask: nil, sort: nil ) - client.graphql.parse <<~GRAPHQL - query( - $after: String, - $limit: Int, - $offset: Int, - ) { + arguments = build_arguments( + after: string_value(after), + limit: integer_value(limit), + offset: integer_value(offset), + autocut: integer_value(autocut), + tenant: string_value(tenant), + nearText: near_text, + nearVector: near_vector, + nearImage: near_image, + nearObject: near_object, + hybrid: with_hybrid, + bm25: bm25, + ask: ask, + where: where, + sort: sort + ) + + <<~GRAPHQL + { Get { - #{class_name}( - after: $after, - limit: $limit, - offset: $offset, - #{(!autocut.nil?) ? "autocut: #{autocut}" : ""}, - #{(!tenant.nil?) ? "tenant: \"#{tenant}\"" : ""}, - #{(!near_text.nil?) ? "nearText: #{near_text}" : ""}, - #{(!near_vector.nil?) ? "nearVector: #{near_vector}" : ""}, - #{(!near_image.nil?) ? "nearImage: #{near_image}" : ""}, - #{(!near_object.nil?) ? "nearObject: #{near_object}" : ""}, - #{(!with_hybrid.nil?) ? "hybrid: #{with_hybrid}" : ""}, - #{(!bm25.nil?) ? "bm25: #{bm25}" : ""}, - #{(!ask.nil?) ? "ask: #{ask}" : ""}, - #{(!where.nil?) ? "where: #{where}" : ""}, - #{(!sort.nil?) ? "sort: #{sort}" : ""} - ) { + #{class_name}#{arguments} { #{fields} } } @@ -189,32 +185,50 @@ def aggs_query( fields:, tenant: nil, where: nil, + object_limit: nil, near_text: nil, near_vector: nil, near_image: nil, - near_object: nil + near_object: nil, + group_by: nil ) - client.graphql.parse <<~GRAPHQL - query( - $group_by: [String], - $object_limit: Int, - ) { + arguments = build_arguments( + objectLimit: integer_value(object_limit), + groupBy: group_by && Array(group_by).map(&:to_s).to_json, + tenant: string_value(tenant), + nearText: near_text, + nearVector: near_vector, + nearImage: near_image, + nearObject: near_object, + where: where + ) + + <<~GRAPHQL + { Aggregate { - #{class_name}( - objectLimit: $object_limit, - groupBy: $group_by, - #{(!tenant.nil?) ? "tenant: \"#{tenant}\"" : ""}, - #{(!near_text.nil?) ? "nearText: #{near_text}" : ""}, - #{(!near_vector.nil?) ? "nearVector: #{near_vector}" : ""}, - #{(!near_image.nil?) ? "nearImage: #{near_image}" : ""}, - #{(!near_object.nil?) ? "nearObject: #{near_object}" : ""}, - #{(!where.nil?) ? "where: #{where}" : ""} - ) { + #{class_name}#{arguments} { #{fields} } } } GRAPHQL end + + # Only arguments that were provided are included in the query. + # Weaviate fails on explicit nulls, e.g. "interface conversion: interface {} is nil, not int". + def build_arguments(**arguments) + arguments = arguments.reject { |_, value| value.nil? } + return "" if arguments.empty? + + "(#{arguments.map { |name, value| "#{name}: #{value}" }.join(", ")})" + end + + def integer_value(value) + Integer(value) unless value.nil? + end + + def string_value(value) + value.to_s.to_json unless value.nil? + end end end diff --git a/lib/weaviate/version.rb b/lib/weaviate/version.rb index adfc996..6c3ea33 100644 --- a/lib/weaviate/version.rb +++ b/lib/weaviate/version.rb @@ -1,5 +1,5 @@ # frozen_string_literal: true module Weaviate - VERSION = "0.9.2" + VERSION = "0.10.0" end diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index 2bd8198..ed5ad11 100644 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -1,5 +1,6 @@ # frozen_string_literal: true +require "ostruct" require "weaviate" RSpec.configure do |config| diff --git a/spec/weaviate/backups_spec.rb b/spec/weaviate/backups_spec.rb index 9d858d4..e2c5ee6 100644 --- a/spec/weaviate/backups_spec.rb +++ b/spec/weaviate/backups_spec.rb @@ -16,7 +16,7 @@ let(:response) { OpenStruct.new(success?: true, body: backup_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:post) + allow_any_instance_of(Weaviate::Connection).to receive(:post) .with("backups/filesystem") .and_return(response) end @@ -36,7 +36,7 @@ let(:response) { OpenStruct.new(success?: true, body: backup_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("backups/filesystem/my-first-backup") .and_return(response) end @@ -54,7 +54,7 @@ let(:response) { OpenStruct.new(success?: true, body: backup_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:post) + allow_any_instance_of(Weaviate::Connection).to receive(:post) .with("backups/filesystem/my-first-backup/restore") .and_return(response) end @@ -74,7 +74,7 @@ let(:response) { OpenStruct.new(success?: true, body: backup_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("backups/filesystem/my-first-backup/restore") .and_return(response) end diff --git a/spec/weaviate/classifications_spec.rb b/spec/weaviate/classifications_spec.rb index cb21b7f..d20e4ce 100644 --- a/spec/weaviate/classifications_spec.rb +++ b/spec/weaviate/classifications_spec.rb @@ -16,7 +16,7 @@ let(:response) { OpenStruct.new(success?: true, body: classification_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:post) + allow_any_instance_of(Weaviate::Connection).to receive(:post) .with("classifications") .and_return(response) end @@ -37,7 +37,7 @@ let(:response) { OpenStruct.new(success?: true, body: classification_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("classifications/1") .and_return(response) end diff --git a/spec/weaviate/client_spec.rb b/spec/weaviate/client_spec.rb index 2f00c7c..8374284 100644 --- a/spec/weaviate/client_spec.rb +++ b/spec/weaviate/client_spec.rb @@ -27,9 +27,39 @@ end end + describe "#collections" do + it "returns a collections client" do + expect(client.collections).to be_a(Weaviate::Collections) + end + end + describe "#schema" do - it "returns a schema client" do - expect(client.schema).to be_a(Weaviate::Schema) + it "returns a collections client" do + expect(client.schema).to be_a(Weaviate::Collections) + end + end + + describe "#connection" do + it "sends the auth and model service headers" do + client = described_class.new( + url: "http://localhost:8080", + api_key: "abc", + model_service: :openai, + model_service_api_key: "123" + ) + + expect(client.connection.url).to eq("http://localhost:8080/v1") + expect(client.connection.headers).to eq( + "Authorization" => "Bearer abc", + "X-OpenAI-Api-Key" => "123" + ) + end + + it "accepts a proxy" do + client = described_class.new(url: "http://localhost:8080", proxy: "http://user:pass@proxy.local:8888") + + expect(client.connection.proxy.host).to eq("proxy.local") + expect(client.connection.proxy.port).to eq(8888) end end @@ -38,7 +68,7 @@ let(:response) { OpenStruct.new(body: fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("meta") .and_return(response) end @@ -66,7 +96,7 @@ let(:response) { OpenStruct.new(status: 200) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with(".well-known/ready") .and_return(response) end @@ -80,7 +110,7 @@ let(:response) { OpenStruct.new(status: 200) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with(".well-known/live") .and_return(response) end @@ -107,7 +137,7 @@ let(:response) { OpenStruct.new(body: fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with(".well-known/openid-configuration") .and_return(response) end diff --git a/spec/weaviate/schema_spec.rb b/spec/weaviate/collections_spec.rb similarity index 74% rename from spec/weaviate/schema_spec.rb rename to spec/weaviate/collections_spec.rb index 70c728b..bae4c7a 100644 --- a/spec/weaviate/schema_spec.rb +++ b/spec/weaviate/collections_spec.rb @@ -2,7 +2,7 @@ require "spec_helper" -RSpec.describe Weaviate::Schema do +RSpec.describe Weaviate::Collections do let(:client) { Weaviate::Client.new( url: "http://localhost:8080", @@ -10,16 +10,21 @@ model_service_api_key: "123" ) } - let(:schema) { client.schema } + let(:schema) { client.collections } let(:class_fixture) { JSON.parse(File.read("spec/fixtures/class.json")) } let(:classes_fixture) { JSON.parse(File.read("spec/fixtures/classes.json")) } let(:shard_fixture) { JSON.parse(File.read("spec/fixtures/shards.json")) } + it "is aliased as Weaviate::Schema for backwards compatibility" do + expect(Weaviate::Schema).to eq(described_class) + expect(client.schema).to be(client.collections) + end + describe "#list" do let(:response) { OpenStruct.new(body: classes_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("schema") .and_return(response) end @@ -33,7 +38,7 @@ let(:response) { OpenStruct.new(success?: true, body: class_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("schema/Question") .and_return(response) end @@ -48,7 +53,7 @@ let(:response) { OpenStruct.new(success?: true, body: class_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:post) + allow_any_instance_of(Weaviate::Connection).to receive(:post) .with("schema") .and_return(response) end @@ -80,7 +85,7 @@ context "when auto_tenant_creation passed" do before do @captured_request = nil - allow_any_instance_of(Faraday::Connection).to receive(:post) do |_, path, &block| + allow_any_instance_of(Weaviate::Connection).to receive(:post) do |_, path, &block| expect(path).to eq("schema") req = OpenStruct.new(body: {}) block.call(req) @@ -105,7 +110,7 @@ context "named vector config" do before do @captured_request = nil - allow_any_instance_of(Faraday::Connection).to receive(:post) do |_, path, &block| + allow_any_instance_of(Weaviate::Connection).to receive(:post) do |_, path, &block| expect(path).to eq("schema") req = OpenStruct.new(body: {}) block.call(req) @@ -157,7 +162,7 @@ let(:response) { OpenStruct.new(success?: true, body: "") } before do - allow_any_instance_of(Faraday::Connection).to receive(:delete) + allow_any_instance_of(Weaviate::Connection).to receive(:delete) .with("schema/Question") .and_return(response) end @@ -170,20 +175,45 @@ end describe "#update" do - let(:response) { OpenStruct.new(success?: true, body: class_fixture) } + let(:current) { + { + "class" => "Question", + "description" => "Information from a Jeopardy! question", + "invertedIndexConfig" => {"indexTimestamps" => true, "bm25" => {"b" => 0.75, "k1" => 1.2}}, + "properties" => [{"name" => "question", "dataType" => ["text"]}] + } + } before do - allow_any_instance_of(Faraday::Connection).to receive(:put) + @captured_request = nil + + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("schema/Question") - .and_return(response) + .and_return(OpenStruct.new(success?: true, body: current)) + + allow_any_instance_of(Weaviate::Connection).to receive(:put) do |_, path, &block| + expect(path).to eq("schema/Question") + req = OpenStruct.new(body: nil) + block.call(req) + @captured_request = req + OpenStruct.new(success?: true, body: req.body) + end end - it "returns the schema" do + it "merges the changes into the current definition" do response = schema.update( class_name: "Question", - description: "Information from a Wheel of Fortune question" + description: "Information from a Wheel of Fortune question", + inverted_index_config: {bm25: {b: 0.8}} ) + expect(response.dig("class")).to eq("Question") + expect(@captured_request.body).to eq( + "class" => "Question", + "description" => "Information from a Wheel of Fortune question", + "invertedIndexConfig" => {"indexTimestamps" => true, "bm25" => {"b" => 0.8, "k1" => 1.2}}, + "properties" => [{"name" => "question", "dataType" => ["text"]}] + ) end end @@ -193,7 +223,7 @@ let(:response) { OpenStruct.new(success?: true, body: class_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:post) + allow_any_instance_of(Weaviate::Connection).to receive(:post) .with("schema/Question/tenants") .and_return(response) end @@ -215,7 +245,7 @@ let(:response) { OpenStruct.new(success?: true, body: shard_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("schema/Question/shards") .and_return(response) end @@ -229,7 +259,7 @@ let(:response) { OpenStruct.new(success?: true, body: shard_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:put) + allow_any_instance_of(Weaviate::Connection).to receive(:put) .with("schema/Question/shards/xyz123") .and_return(response) end diff --git a/spec/weaviate/connection_spec.rb b/spec/weaviate/connection_spec.rb new file mode 100644 index 0000000..34e44b1 --- /dev/null +++ b/spec/weaviate/connection_spec.rb @@ -0,0 +1,123 @@ +# frozen_string_literal: true + +require "spec_helper" +require "stringio" + +RSpec.describe Weaviate::Connection do + let(:log) { StringIO.new } + let(:connection) { + described_class.new( + url: "http://localhost:8080/v1/", + headers: {"Authorization" => "Bearer secret", "X-OpenAI-Api-Key" => "sk-secret"}, + logger: Logger.new(log) + ) + } + + def http_response(klass, code, body: nil, content_type: "application/json") + response = klass.new("1.1", code.to_s, "Message") + response["Content-Type"] = content_type + response.instance_variable_set(:@body, body) + response.instance_variable_set(:@read, true) + response + end + + def stub_http(response) + http = instance_double(Net::HTTP) + allow(http).to receive(:request) do |req| + @captured_request = req + response + end + allow(Net::HTTP).to receive(:start) do |*args, **kwargs, &block| + @start_args = args + @start_kwargs = kwargs + block.call(http) + end + end + + it "sends query params and parses JSON responses" do + stub_http(http_response(Net::HTTPOK, 200, body: '{"objects": []}')) + + response = connection.get("objects") do |req| + req.params["class"] = "Question" + req.params["limit"] = 10 + end + + expect(@captured_request).to be_a(Net::HTTP::Get) + expect(@captured_request.path).to eq("/v1/objects?class=Question&limit=10") + expect(@captured_request["Authorization"]).to eq("Bearer secret") + expect(@start_args.first(3)).to eq(["localhost", 8080, :ENV]) + expect(@start_kwargs).to eq(use_ssl: false) + expect(response.success?).to eq(true) + expect(response.status).to eq(200) + expect(response.body).to eq("objects" => []) + end + + it "encodes hash bodies as JSON and passes string bodies through" do + stub_http(http_response(Net::HTTPOK, 200, body: "{}")) + + connection.post("objects") { |req| req.body = {"class" => "Question"} } + expect(@captured_request.body).to eq('{"class":"Question"}') + expect(@captured_request["Content-Type"]).to eq("application/json") + + connection.post("schema/Question/tenants") { |req| req.body = '[{"name": "a"}]' } + expect(@captured_request.body).to eq('[{"name": "a"}]') + end + + it "returns an empty string body for empty responses" do + stub_http(http_response(Net::HTTPNoContent, 204, content_type: nil)) + + response = connection.delete("objects/Question/123") + expect(response.status).to eq(204) + expect(response.body).to eq("") + end + + it "raises an error for 4xx responses" do + stub_http(http_response(Net::HTTPNotFound, 404, body: '{"error": [{"message": "not found"}]}')) + + expect { connection.get("schema/Missing") }.to raise_error(Weaviate::ResourceNotFound) { |e| + expect(e).to be_a(Weaviate::ClientError) + expect(e).to be_a(Weaviate::Error) + expect(e.status).to eq(404) + expect(e.body).to eq("error" => [{"message" => "not found"}]) + } + end + + it "raises an error for 5xx responses" do + stub_http(http_response(Net::HTTPServiceUnavailable, 503, body: "")) + + expect { connection.get(".well-known/ready") }.to raise_error(Weaviate::ServerError) + end + + it "wraps connection failures" do + allow(Net::HTTP).to receive(:start).and_raise(Errno::ECONNREFUSED) + + expect { connection.get("meta") }.to raise_error(Weaviate::ConnectionFailed) + end + + it "wraps timeouts" do + allow(Net::HTTP).to receive(:start).and_raise(Net::ReadTimeout) + + expect { connection.get("meta") }.to raise_error(Weaviate::TimeoutError) + end + + it "connects through the configured proxy" do + connection = described_class.new(url: "https://example.weaviate.network/v1", proxy: "http://user:pass@proxy.local:8888") + stub_http(http_response(Net::HTTPOK, 200, body: "{}")) + + connection.get("meta") + + expect(@start_args).to eq(["example.weaviate.network", 443, "proxy.local", 8888, "user", "pass"]) + expect(@start_kwargs).to eq(use_ssl: true) + end + + it "filters secrets from the logs" do + stub_http(http_response(Net::HTTPOK, 200, body: "{}")) + + connection.get("meta") + + expect(log.string).to include("GET http://localhost:8080/v1/meta") + expect(log.string).to include("authorization: [FILTERED]") + expect(log.string).to include("x-openai-api-key: [FILTERED]") + expect(log.string).not_to include("secret") + end +end diff --git a/spec/weaviate/health_spec.rb b/spec/weaviate/health_spec.rb index 692248a..6573f83 100644 --- a/spec/weaviate/health_spec.rb +++ b/spec/weaviate/health_spec.rb @@ -15,7 +15,7 @@ describe "#live" do before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with(".well-known/live") .and_return(response) end @@ -27,7 +27,7 @@ describe "#ready" do before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with(".well-known/ready") .and_return(response) end diff --git a/spec/weaviate/nodes_spec.rb b/spec/weaviate/nodes_spec.rb index 0bf596d..0e581a6 100644 --- a/spec/weaviate/nodes_spec.rb +++ b/spec/weaviate/nodes_spec.rb @@ -17,7 +17,7 @@ } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("nodes") .and_return(response) end diff --git a/spec/weaviate/objects_spec.rb b/spec/weaviate/objects_spec.rb index 52a84f3..e34f086 100644 --- a/spec/weaviate/objects_spec.rb +++ b/spec/weaviate/objects_spec.rb @@ -18,7 +18,7 @@ let(:response) { OpenStruct.new(success?: true, body: object_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:post) + allow_any_instance_of(Weaviate::Connection).to receive(:post) .with("objects") .and_return(response) end @@ -41,7 +41,7 @@ let(:response) { OpenStruct.new(body: {"objects" => objects_fixture}) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("objects") .and_return(response) end @@ -56,7 +56,7 @@ let(:response) { OpenStruct.new(success?: true, status: 204, body: "") } before do - allow_any_instance_of(Faraday::Connection).to receive(:head) + allow_any_instance_of(Weaviate::Connection).to receive(:head) .with("objects/Question/123") .and_return(response) end @@ -68,13 +68,21 @@ ) expect(response).to eq(true) end + + it "returns false when the object does not exist" do + allow_any_instance_of(Weaviate::Connection).to receive(:head) + .with("objects/Question/456") + .and_raise(Weaviate::ResourceNotFound.new(response: OpenStruct.new(status: 404))) + + expect(objects.exists?(class_name: "Question", id: "456")).to eq(false) + end end describe "#get" do let(:response) { OpenStruct.new(success?: true, body: object_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:get) + allow_any_instance_of(Weaviate::Connection).to receive(:get) .with("objects/Question/123") .and_return(response) end @@ -92,7 +100,7 @@ let(:response) { OpenStruct.new(success?: true, body: "") } before do - allow_any_instance_of(Faraday::Connection).to receive(:delete) + allow_any_instance_of(Weaviate::Connection).to receive(:delete) .with("objects/Question/123") .and_return(response) end @@ -110,7 +118,7 @@ let(:response) { OpenStruct.new(success?: true, body: batch_delete_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:delete) + allow_any_instance_of(Weaviate::Connection).to receive(:delete) .with("batch/objects") .and_return(response) end @@ -132,7 +140,7 @@ let(:response) { OpenStruct.new(success?: true, body: object_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:patch) + allow_any_instance_of(Weaviate::Connection).to receive(:patch) .with("objects/Question/123") .and_return(response) end @@ -155,7 +163,7 @@ let(:response) { OpenStruct.new(success?: true, body: object_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:put) + allow_any_instance_of(Weaviate::Connection).to receive(:put) .with("objects/Question/123") .and_return(response) end @@ -178,7 +186,7 @@ let(:response) { OpenStruct.new(success?: true, body: objects_fixture) } before do - allow_any_instance_of(Faraday::Connection).to receive(:post) + allow_any_instance_of(Weaviate::Connection).to receive(:post) .with("batch/objects") .and_return(response) end @@ -203,5 +211,25 @@ ]) expect(response.count).to eq(2) end + + it "sets the tenant on each object" do + captured_request = nil + allow_any_instance_of(Weaviate::Connection).to receive(:post) do |_, path, &block| + req = OpenStruct.new(params: {}, body: nil) + block.call(req) + captured_request = req + response + end + + objects.batch_create(tenant: "a", objects: [ + {class: "Question", properties: {answer: "42"}}, + {class: "Question", tenant: "b", properties: {answer: "43"}} + ]) + + expect(captured_request.body).to eq(objects: [ + {tenant: "a", class: "Question", properties: {answer: "42"}}, + {tenant: "b", class: "Question", properties: {answer: "43"}} + ]) + end end end diff --git a/spec/weaviate/query_spec.rb b/spec/weaviate/query_spec.rb index f77c22b..622e414 100644 --- a/spec/weaviate/query_spec.rb +++ b/spec/weaviate/query_spec.rb @@ -1,7 +1,6 @@ # frozen_string_literal: true require "spec_helper" -require "graphlient" RSpec.describe Weaviate::Query do let(:client) { @@ -13,9 +12,25 @@ } let(:query) { client.query } + before do + @captured_query = nil + + allow_any_instance_of(Weaviate::Connection).to receive(:post) do |_, path, &block| + expect(path).to eq("graphql") + req = OpenStruct.new(body: nil) + block.call(req) + @captured_query = req.body[:query] + response + end + end + + def squish(str) + str.gsub(/\s+/, " ").strip + end + describe "#get" do let(:response) { - double(original_hash: { + OpenStruct.new(body: { "data" => { "Get" => { "Question" => [ @@ -29,27 +44,6 @@ }) } - let(:graphql_query) { - <<-GRAPHQL - query { - Get { - Question(nearText: { concepts: ["biology"] }, limit: 1) { - question - category - } - } - } - GRAPHQL - } - - before do - allow_any_instance_of(Graphlient::Client).to receive(:parse) - .and_return(graphql_query) - - allow_any_instance_of(Graphlient::Client).to receive(:execute) - .and_return(response) - end - it "returns the query" do data = query.get( class_name: "Question", @@ -62,12 +56,65 @@ expect(data.count).to eq(1) expect(data.first["category"]).to eq("SCIENCE") expect(data.first["question"]).to eq("In 1953 Watson & Crick built a model of the molecular structure of this, the gene-carrying substance") + expect(squish(@captured_query)).to eq( + '{ Get { Question(limit: 1, tenant: "tenant_name", nearText: { concepts: ["biology"] }) { question, category } } }' + ) + end + + it "omits arguments that were not passed" do + query.get( + class_name: "Question", + fields: "question", + where: '{ operator: Equal, valueText: "1-dev", path: ["company"] }' + ) + + expect(squish(@captured_query)).to eq( + '{ Get { Question(where: { operator: Equal, valueText: "1-dev", path: ["company"] }) { question } } }' + ) + end + + it "omits the argument list entirely when no arguments are passed" do + query.get(class_name: "Question", fields: "question") + + expect(squish(@captured_query)).to eq("{ Get { Question { question } } }") + end + + it "quotes the after cursor and supports distance in near_vector" do + query.get( + class_name: "Question", + fields: "question _additional { distance }", + after: "3c5f7039-37f3-4244-b3e2-8f4a083e448d", + offset: 0, + near_vector: "{ vector: [0.1, 0.2], distance: 0.3 }" + ) + + expect(squish(@captured_query)).to eq( + '{ Get { Question(after: "3c5f7039-37f3-4244-b3e2-8f4a083e448d", offset: 0, nearVector: { vector: [0.1, 0.2], distance: 0.3 }) { question _additional { distance } } } }' + ) + end + + context "when the query returns errors" do + let(:response) { + OpenStruct.new(body: { + "data" => {"Get" => {"Question" => nil}}, + "errors" => [{"message" => "Cannot query field \"foo\" on type \"Question\"."}] + }) + } + + it "raises a GraphQLError" do + expect { + query.get(class_name: "Question", fields: "foo") + }.to raise_error(Weaviate::GraphQLError, "Cannot query field \"foo\" on type \"Question\".") { |e| + expect(e).to be_a(Weaviate::Error) + expect(e.errors.first["message"]).to include("Cannot query field") + } + end end end describe "#aggs" do let(:response) { - double(original_hash: { + OpenStruct.new(body: { "data" => { "Aggregate" => { "Question" => [ @@ -81,28 +128,6 @@ }) } - let(:graphql_query) { - <<-GRAPHQL - query { - Aggregate { - Question(nearText: { concepts: ["biology"] }) { - meta { - count - } - } - } - } - GRAPHQL - } - - before do - allow_any_instance_of(Graphlient::Client).to receive(:parse) - .and_return(graphql_query) - - allow_any_instance_of(Graphlient::Client).to receive(:execute) - .and_return(response) - end - it "returns the query" do data = query.aggs( class_name: "Question", @@ -113,40 +138,34 @@ expect(data.count).to eq(1) expect(data.first["category"]).to eq("SCIENCE") expect(data.first["question"]).to eq("In 1953 Watson & Crick built a model of the molecular structure of this, the gene-carrying substance") + expect(squish(@captured_query)).to eq( + '{ Aggregate { Question(nearText: { concepts: ["biology"] }) { question, category } } }' + ) + end + + it "passes group_by and object_limit" do + query.aggs( + class_name: "Question", + fields: "meta { count }", + group_by: ["category"], + object_limit: "10" + ) + + expect(squish(@captured_query)).to eq( + '{ Aggregate { Question(objectLimit: 10, groupBy: ["category"]) { meta { count } } } }' + ) end end describe "#explore" do let(:response) { - double(original_hash: { + OpenStruct.new(body: { "data" => { "Explore" => [{"certainty" => "0.9999", "class_name" => "Question"}] } }) } - let(:graphql_query) { - <<-GRAPHQL - query { - Explore( - limit: 1 - nearText: { concepts: ["biology"] } - ) { - certainty - className - } - } - GRAPHQL - } - - before do - allow_any_instance_of(Graphlient::Client).to receive(:parse) - .and_return(graphql_query) - - allow_any_instance_of(Graphlient::Client).to receive(:execute) - .and_return(response) - end - it "returns the query" do response = query.explore( fields: "certainty className", @@ -157,6 +176,15 @@ expect(response.count).to eq(1) expect(response.first["certainty"]).to eq("0.9999") expect(response.first["class_name"]).to eq("Question") + expect(squish(@captured_query)).to eq( + '{ Explore(limit: 1, nearText: { concepts: ["biology"] }) { certainty className } }' + ) + end + + it "ignores after, which Explore does not support" do + query.explore(fields: "className", near_vector: "{ vector: [0.1] }", after: "abc") + + expect(squish(@captured_query)).to eq("{ Explore(nearVector: { vector: [0.1] }) { className } }") end end end diff --git a/weaviate.gemspec b/weaviate.gemspec index 412bdc7..9c5918a 100644 --- a/weaviate.gemspec +++ b/weaviate.gemspec @@ -35,8 +35,7 @@ Gem::Specification.new do |spec| # For more information and examples about making a new gem, check out our # guide at: https://bundler.io/guides/creating_gem.html - spec.add_dependency "faraday", ">= 2.0.1", "< 3.0" - spec.add_dependency "graphlient", ">= 0.7.0", "< 0.9.0" + spec.add_dependency "logger" spec.add_development_dependency "pry-byebug", "~> 3.9" spec.add_development_dependency "yard" spec.add_development_dependency "rdiscount" # for github-flavored markdown in yard