Skip to content

Added CREATE OPERATOR and CREATE AGGREGATE - #3059

Open
Hydrocharged wants to merge 1 commit into
mainfrom
daylon/extension-objects
Open

Added CREATE OPERATOR and CREATE AGGREGATE#3059
Hydrocharged wants to merge 1 commit into
mainfrom
daylon/extension-objects

Conversation

@Hydrocharged

Copy link
Copy Markdown
Collaborator

This adds support for the CREATE OPERATOR and CREATE AGGREGATE statements, which we need to support additional emulated extensions.

@Hydrocharged
Hydrocharged requested a review from zachmu August 10, 2026 06:43
@github-actions

Copy link
Copy Markdown
Contributor
Main PR
covering_index_scan_postgres 2059.55/s 2094.39/s +1.6%
groupby_scan_postgres 152.70/s 155.55/s +1.8%
index_join_postgres 675.59/s 674.47/s -0.2%
index_join_scan_postgres 852.47/s 854.49/s +0.2%
index_scan_postgres 32.58/s 32.56/s -0.1%
oltp_delete_insert_postgres 878.94/s 830.52/s -5.6%
oltp_insert 786.52/s 753.91/s -4.2%
oltp_point_select 3562.92/s 3515.80/s -1.4%
oltp_read_only 3537.83/s 3486.81/s -1.5%
oltp_read_write 2702.40/s 2608.26/s -3.5%
oltp_update_index 805.28/s 776.21/s -3.7%
oltp_update_non_index 833.30/s 819.19/s -1.7%
oltp_write_only 1896.57/s 1886.92/s -0.6%
select_random_points 2178.01/s 2169.37/s -0.4%
select_random_ranges 1635.22/s 1624.17/s -0.7%
table_scan_postgres 31.95/s 32.44/s +1.5%
types_delete_insert_postgres 878.39/s 840.40/s -4.4%
types_table_scan_postgres 14.43/s 14.69/s +1.8%

@itoqa

itoqa Bot commented Aug 10, 2026

Copy link
Copy Markdown

Ito QA test results
Commit: 0fb36f1: 18 test cases ran, 17 passed ✅, 1 additional finding ⚠️.

Summary

Coverage spans database object creation and lifecycle management, query parsing and operator behavior, aggregate edge cases involving empty and null inputs, cross-session visibility, restart and branch-merge persistence, and extension materialization. It includes normal workflows, boundary conditions, invalid definitions, name resolution conflicts, cleanup, and conflict handling, with the exercised behavior broadly healthy.

Safe to merge — the only failure is a pre-existing medium-severity compatibility issue unrelated to this PR, with no regressions or PR-attributable failures identified. The unrelated issue is a flag for later rather than a merge blocker.

Tests run by Ito

View full run

Result Severity Type Description
Aggregate The grouped totals and empty-group result match the transition function's declared behavior. A NULL input is passed through because the transition function is not marked strict, and the fixture expects the affected group to return NULL.
Aggregate The aggregate returned the expected values and reported its final result as text. Invalid transition and combine definitions were rejected, and no bad aggregate entries were saved.
Aggregate Invalid name collisions were rejected, replacing the aggregate changed the query result to 70, and the drop commands cleaned up the aggregate as expected.
Aggregate Empty groups, all-null groups, and one-row groups returned the expected values. Separate groups kept separate state, and final results kept the text type.
Catalog A new session could run the newly created aggregate and see the same catalog entries as the first session. After the objects were dropped, no procedure rows remained.
Extension Creating the extension makes its custom type, array type, routines, casts, operators, and aggregate usable in the target schema.
Extension The retry scenario was not run because the local database service was unavailable and the test-only extension was not registered in the long-lived server. Source review found a possible partial-write risk, but there is no runtime evidence that a real application failure occurred.
Operator The new operator returned 5 for 9 <-> 4, appeared in the operator catalog, and was removed successfully.
Operator Invalid operator definitions were rejected with clear errors, and none of the attempted operators appeared in the catalog afterward.
Operator The same operator name returned 5 or 13 based on schema order. An untyped expression with two matching text types returned a clear ambiguity error.
Operator A user-defined plus operator did not replace the built-in behavior, while a distance operator still used the user-defined function. An unavailable operator returned the expected not-found error.
Operator Boolean operator pairs kept the correct commutator and negator links, even when recreated in the opposite order. Invalid HASHES metadata was rejected for a non-boolean operator.
Parser All six distance operators returned the expected values for literal numbers, table columns, NULL values, nested expressions, and nearby built-in operators.
Parser All six distance operators kept their identity in literals, table columns, NULL-containing filters, parenthesized conditions, and a nested expression in a fresh session.
Persistence The aggregate definition still worked after Doltgres restarted, and it remained visible in the catalog. The operator check could not be completed because its fixture statement was rejected during setup and the target service was unavailable for a rerun.
Persistence The aggregate kept its branch-specific value before merging, and both branches returned the merged value afterward.
Persistence Merging branches with different aggregate definitions returned a clear error, and each branch kept its own result afterward.
⚠️ Medium severity Catalog The regular overload calls returned the expected values, but the aggregate could not be created with the standard integer and bigint type names. The catalog check then stopped because the schema cast to regnamespace was not supported.
Additional Findings Details

These findings are unrelated to the current changes but were observed during testing.

🟡 Aggregate overloads reject common type names
  • Severity: Medium Medium severity
  • Description: The regular overload calls returned the expected values, but the aggregate could not be created with the standard integer and bigint type names. The catalog check then stopped because the schema cast to regnamespace was not supported.
  • Impact: Users who create aggregate functions with common PostgreSQL type names cannot complete that workflow, and catalog tools cannot inspect those types. Regular function overloads still work, and using canonical type names can work around the issue.
  • Steps to Reproduce:
    1. Create a schema named cat2.
    2. Create cat2.mix(integer), cat2.mix(text), and cat2.add_state(integer, integer) functions.
    3. Run CREATE AGGREGATE cat2.mix(bigint) (SFUNC = cat2.add_state, STYPE = integer, INITCOND = '0').
    4. Observe the error type "integer" does not exist.
    5. Run a catalog query that filters pg_proc with p.pronamespace = 'cat2'::regnamespace.
    6. Observe the error unable to resolve type regnamespace.
  • Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
  • Code Analysis: The CREATE AGGREGATE path in server/analyzer/resolve_type.go:68-86 calls resolveType for the state and argument types before server/node/create_aggregate.go:102-110 builds the function signature. resolveType in server/analyzer/resolve_type.go:355-371 asks the type collection for the literal type name. core/typecollection/typecollection.go:333-357 resolves relative names by comparing the requested name with typ.ID.TypeName(), not with DoltgresType.InternalName. The built-in int32 and int64 definitions in server/types/int32.go:26-60 and server/types/int64.go:26-60 use IDs int4 and int8 while only storing integer and bigint as InternalName values, so integer and bigint are not found by this resolver. The same registry issue is visible in server/types/globals.go:265-274, where regnamespace is present but mapped to Unknown, causing a lookup to return an unresolved type. The smallest fix is to register integer and bigint as aliases to the int4/int8 built-ins and provide a resolved regnamespace type (including its array entry if catalog arrays need it); no change to aggregate overload merging is needed for this failure.
Evidence Package

Tip

Reply with @itoqa to send us feedback on this test run.

@github-actions

Copy link
Copy Markdown
Contributor
Main PR
Total 42090 42090
Successful 18940 18948
Failures 23150 23142
Partial Successes1 5340 5325
Main PR
Successful 44.9988% 45.0178%
Failures 55.0012% 54.9822%

${\color{lightgreen}Progressions (8)}$

aggregates

QUERY: SELECT sum2(q1,q2) FROM int8_tbl;

case

QUERY: CREATE OPERATOR = (procedure = inline_eq,
                   leftarg = foodomain, rightarg = foodomain);
QUERY: CREATE OPERATOR = (procedure = ad_eq,
                   leftarg = arrdomain, rightarg = arrdomain);

create_aggregate

QUERY: create aggregate sum2(int8,int8) (
   sfunc = sum3, stype = int8,
   initcond = '0'
);

drop_if_exists

QUERY: DROP AGGREGATE IF EXISTS test_aggregate_exists(int);
QUERY: DROP AGGREGATE IF EXISTS no_such_schema.foo(int);

polymorphism

QUERY: CREATE AGGREGATE mysum2(anyelement,anyelement) (SFUNC = sum3,
  STYPE = anyelement, INITCOND = '0');

subselect

QUERY: create operator = (procedure=bogus_int8_text_eq, leftarg=int8, rightarg=text);

Footnotes

  1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct.

@zachmu zachmu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks fine. The mixing of serialization and in-memory storage concerns can be a followup. A larger problem is that this makes the compatibility tests fail, because we're always writing the new flatbuffer fields even when they're empty. They need to be conditionally written only when they have content, otherwise this breaks every existing customer (rather than just the ones who start using this new feature, which will be a small minority).

Once that's fixed I'm good to approve.

}

// AddOperator adds a new operator.
func (pgo *Collection) AddOperator(ctx context.Context, o Operator) error {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little odd that adding an operator also serializes it to the underlying map. That combines two very separate concerns kind of inappropriately. Do all the collection types work this way?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two do at least. It would be really nice to separate the concerns of serialization from basic in-memory representation in all our root object collections.

),
Auth: vitess.AuthInformation{
AuthType: auth.AuthType_CREATE,
TargetType: auth.AuthTargetType_TODO,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this do anything / work?

}
}
isAggregate := len(aggOverloads) > 0 || len(AggregateCatalog[name]) > 0
if err = addBuiltInOverloads(overloadTree, name); err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this line should go below the early return block immediately below


// String implements the interface sql.ExecSourceRel.
func (c *CreateAggregate) String() string {
return "CREATE AGGREGATE"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to include a name in these String funcs, they're useful in debugging

Same comment for other types in the node package

},
Assertions: []ScriptTestAssertion{
{
Query: `CREATE OPERATOR <%> (LEFTARG = text, RIGHTARG = text, FUNCTION = op_ci_eq, COMMUTATOR = <%>, HASHES, MERGES);`,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not obvious what hashes and merges do here or if we're just testing that they are stored, a comment would help

Expected: []sql.Row{},
},
{
Query: `SELECT ROW(1, 2)::op_pair <+> ROW(3, 4)::op_pair;`,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work in a query on table rows, maybe in an aggregate func?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants