Skip to content

Conversation

@Jolanrensen
Copy link
Collaborator

This PR contains a handful of fixes and optimizations regarding Nothing(?) in column types:

  • Fixes DataFrame with Nothing? column is detected as FormattedFrame<*>? in notebooks #1546 and adds a helpful note to colsOf to prevent future confusion
  • DataColumn.empty() created a Unit-typed column. I have no idea why, I made it Nothing, which better represents that emptyCol.get(0): Nothing will always throw an exception. Added DataColumn.emptyOf<T>() if you need a specifically typed empty column.
  • #1546 also discovered that exceptions in update provided little info, so I added UpdateException for this.
  • While inspecting Column DataCollector for null-filled or empty columns, I rewrote ::class-logic to typeOf's (for efficiency), and added redundancy checks:
    • (If the data collector should produce a Nothing column, it can only do that if it has no values; so the resulting column must be empty)
    • (if the data collector should produce a Nothing? column, it can only do that if all its values are null)

I also checked usages of colsOf() querying for nullable types to see if similar issues like #1546 existed. Fortunately, in most cases, including null-filled columns are not an issue! All statistics functions simply skip nulls, for instance.

There are some unfortunate cases, like:

dataFrameOf(
    "a" to columnOf(1, 2, 3, null),
    "b" to columnOf(null, null, null, null),
    "c" to columnOf(7, 3, 2, 65),
).fillNulls { colsOf<Int?>() }.with { 0 }

which throw java.lang.IllegalArgumentException: Can not add value of class kotlin.Int to column of type kotlin.Nothing?. Value = 0 because column b: Nothing? is included by colsOf<Int?>(). However, hopefully, the note to add .filter { !it.allNulls() } is found :). Alternatively, we could decide to add an argument colsOf<Int?>(exactType = true) in the future.

…urate than `Unit` as getting something out of this column will always result in an exception, not "just run", like `Unit` does. Added emptyOf for if you need a specific type
…dded redundancy checks for nothing types in column creation
@Jolanrensen Jolanrensen added this to the 1.0.0-Beta4 milestone Nov 6, 2025
@Jolanrensen Jolanrensen requested a review from koperagen November 6, 2025 20:06
@Jolanrensen Jolanrensen added bug Something isn't working enhancement New feature or request labels Nov 6, 2025
Copy link
Collaborator

@AndreiKingsley AndreiKingsley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! I've missed that behavior a few times.

*
* __NOTE:__ Null-filled columns of type [Nothing?][Nothing] will be included when selecting [`colsOf`][colsOf]`<T?>()`.
* This is because [Nothing][Nothing] is considered a subtype of all other types in Kotlin.
* To exclude these columns, call `.`[filter][ColumnsSelectionDsl.filter]` { !it.`[allNulls][DataColumn.allNulls]`() }`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why colsOf<T?> except colsOf<Nothing> is not an option here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing cannot be used as reified argument

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just have a look what we have to do to achieve typeOf<Nothing>() in dataframe XD

internal val nothingType: KType = typeOf<List<Nothing>>().arguments.first().type!!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we would need to have a shortcut for that nothingCols() for instance, but I'm not sure it's common enough

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or, you know, they type

colsOf<T?>() except colsOf(typeOf<List<Nothing>>().arguments.first().type!!)

you know, elegant

@Jolanrensen Jolanrensen merged commit 4f8c867 into master Nov 10, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DataFrame with Nothing? column is detected as FormattedFrame<*>? in notebooks

4 participants