[#600] prevent case-sensitive testing with BETWEEN where infeasible

[d-w-moore]

Tests using the BETWEEN operator are now skipped if they are predicated on being case sensitive and if the installed object catalog ignores alphabetic case for query conditions dependent on lexical order (as tested by BETWEEN, >, >=, <, and <=.)

[korydraughn]

Why does case_sensitive_testing_feasible use a tuple to check cls._csq on line 64?

_query_columns_case_sensitive_by_default returns a boolean which removes the need for a tuple. Am I misunderstanding?

[d-w-moore]

oh. The in (True,) thing? I could have just said == True, if that's any better. Lots of folk are allergic to that, but in Python it's meaningful because of the False != None gotcha.

I expect people to object, it's part of software developer psychology to hate it when people write Python that contains things like is True or == True. I don't mind arguing with an actual human because you can convince them usually to see the logic.

But what I also expect is that codacy and ruff will heartily object and I'll be wasting a good half hour, in aggregate, dealing with the fallout. But when I write that type of code, I mean it exactly the way I wrote it... Lol

[korydraughn]

Yep. What you typed felt very intentional. Just trying to understand the python-isms.

Is the in (True,) style a common pattern in python?

[d-w-moore]

no, actually. Just my own perverse variation : ) . It kind of screams, "please review. and change as appropriate."

[d-w-moore]

In the early review process, I like having humans look at things. Then, and only afterward, let codacy and ruff have their turn. Saves a lot of useless machinations.

[korydraughn]

It just occurred to me. Why does the PRC have tests which care about the database in use at all? Tests which care about that sort of thing should be part of the server.

[d-w-moore]

It just occurred to me. Why does the PRC have tests which care about the database in use at all? Tests which care about that sort of thing should be part of the server.

This doesn't care about which DB, just whether the server is evincing default case-insensitive behavior at the outset. If it is, there's no point in running the BETWEEN tests that will predictably error out.

If you agree more with Alan's temporary solution, in which we just comment those tests out so that they never run, I'm fine with that too. In that case, we just close this PR and forget about it.

I rather prefer a situation in which these tests run by default. Unless they shouldn't. Since Postsgres is the rule for the usual test scenario (i.e. our current set of PRC-test Github Actions) I prefer to keep the changes of this PR. But that is only my preference.

[alanking]

It just occurred to me. Why does the PRC have tests which care about the database in use at all? Tests which care about that sort of thing should be part of the server.

Are you suggesting that this might be a bug in the server? It does seem odd that GenQuery would yield different results depending on database flavor. Kind of defeats the purpose a little bit.

[d-w-moore]

This doesn't care about which DB, just whether the server is evincing default case-insensitive behavior at the outset. If it is, there's no point in running the BETWEEN tests that will predictably error out.

I think what I've done in this PR is simply allow the BETWEEN operator to be tested independently of the LT, GT type comparisons that many on the Internet have asserted are equivalent (but see my caveat at the end of this comment.) So, we end up doing what I think was intended : make sure the operator is actually doing as-intended in interactions between client and server.

Of course, it will be moot if we find out that sometimes x BETWEEN a b is not semantically equivalent to

  a <= x <= b || b <= x <= a

in GenQuery/SQL.

[d-w-moore]

It just occurred to me. Why does the PRC have tests which care about the database in use at all? Tests which care about that sort of thing should be part of the server.

Are you suggesting that this might be a bug in the server? It does seem odd that GenQuery would yield different results depending on database flavor. Kind of defeats the purpose a little bit.

Not a server bug. Kory's original point was that BETWEEN is passed right to the database. aka, it's not the server's job to mediate between client expectations and various flavors of DB. Only to interface. True, @korydraughn ?

[trel]

Looks like postgres and mysql will both return False if a is not less than b.

https://dev.mysql.com/doc/refman/8.4/en/comparison-operators.html#operator_between :

mysql> SELECT 2 BETWEEN 1 AND 3, 2 BETWEEN 3 and 1;
        -> 1, 0

https://www.postgresql.org/docs/current/functions-comparison.html#FUNCTIONS-COMPARISON-PRED-TABLE :

datatype BETWEEN datatype AND datatype → boolean
Between (inclusive of the range endpoints).
2 BETWEEN 1 AND 3 → t
2 BETWEEN 3 AND 1 → f

postgres offers BETWEEN SYMMETRIC to handle that case

https://www.postgresql.org/docs/current/functions-comparison.html#FUNCTIONS-COMPARISON-PRED-TABLE :

datatype BETWEEN SYMMETRIC datatype AND datatype → boolean
Between, after sorting the two endpoint values.
2 BETWEEN SYMMETRIC 3 AND 1 → t

So, i'm not sure the point of discussion about case-insensitivity... do we want to offer that? feels confusing to me.

[korydraughn]

Are you suggesting that this might be a bug in the server? It does seem odd that GenQuery would yield different results depending on database flavor. Kind of defeats the purpose a little bit.

@alanking I don't necessarily think it's a bug. I'm pointing out that addressing behavioral differences for the BETWEEN keyword across databases feels like something that should be part of the core server tests rather than the PRC.

I think what I've done in this PR is simply allow the BETWEEN operator to be tested independently of the LT, GT type comparisons that many on the Internet have asserted are equivalent (but see my caveat at the end of this comment.) So, we end up doing what I think was intended : make sure the operator is actually doing as-intended in interactions between client and server.

@d-w-moore I need to hear more. Does this make the PRC special?

Not a server bug. Kory's original point was that BETWEEN is passed right to the database. aka, it's not the server's job to mediate between client expectations and various flavors of DB. Only to interface. True, @korydraughn ?

True. Now, I'm wondering if we should enforce our own rules around BETWEEN.

That said, let's discuss offline.