Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partition tools don't correctly interact with LocalDate type #2302

Open
niloc132 opened this issue Nov 26, 2024 · 0 comments
Open

Partition tools don't correctly interact with LocalDate type #2302

niloc132 opened this issue Nov 26, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@niloc132
Copy link
Member

Description

LocalDate instances can be used by PartitionedTables, or partitioned Tables (iceberg, parquet, etc), and may be displayed in the UI. The user will then expect to select a partition to display only data from that partition. However, the UI seems to be using dh.FilterValue.ofNumber(...) to try to create partition values

PartitionedTable

Steps to reproduce

  1. In the web IDE, create a PartitionedTable with a LocalDate column:
    from deephaven import empty_table
    pt = empty_table(5).update_view(["I=i", "Date=java.time.LocalDate.ofEpochDay(ii)"]).partition_by("Date")
  2. View the PartitionedTable widget.

Expected results
A partition selector dropdown should be visible, and there should be no errors in the JS console.

Actual results
There is no way to select a partition (only to merge the table), and an error is present:

Uncaught (in promise) Error: java.lang.NumberFormatException: For input string: "1970-01-05"
    at NumberFormatException.createError (dh-core.js:1472:10)
    at NumberFormatException.initializeBackingError (dh-core.js:1498:46)
    at NumberFormatException.Throwable_0 (dh-core.js:1429:8)
    at NumberFormatException.Exception_1 (dh-core.js:1523:18)
    at NumberFormatException.RuntimeException_1 (dh-core.js:1541:18)
    at NumberFormatException.IllegalArgumentException_0 (dh-core.js:1555:25)
    at new NumberFormatException (dh-core.js:41262:33)
    at Object.__parseAndValidateDouble (dh-core.js:40739:16)
    at Object.ofNumber (dh-core.js:24934:27)
    at C.makeFilterRawValue (TableUtils.js:1399:27)
    at C.makeNullableEqFilter (TableUtils.js:1412:36)
    at IrisGridPartitionSelector.js:220:34
    at Array.map (<anonymous>)
    at IrisGridPartitionSelector.js:218:57
    at Generator.next (<anonymous>)
    at ri (IrisGridPartitionSelector.js:3:102)
    at o (IrisGridPartitionSelector.js:4:194)
    at IrisGridPartitionSelector.js:4:364
    at new Promise (<anonymous>)
    at IrisGridPartitionSelector.js:4:97
    at _o.updatePartitionOptions (IrisGridPartitionSelector.js:234:6)
    at _o.componentDidUpdate (IrisGridPartitionSelector.js:75:12)
    at het (react-dom.production.min.js:219:502)
    at $et (react-dom.production.min.js:259:160)
    at e.unstable_runWithPriority (scheduler.production.min.js:18:343)
    at ag (react-dom.production.min.js:122:325)
    at vv (react-dom.production.min.js:252:279)
    at gF (react-dom.production.min.js:243:371)
    at react-dom.production.min.js:123:115
    at e.unstable_runWithPriority (scheduler.production.min.js:18:343)
    at ag (react-dom.production.min.js:122:325)
    at C5e (react-dom.production.min.js:123:61)
    at Tp (react-dom.production.min.js:122:428)
    at Vm (react-dom.production.min.js:237:203)
    at Object.enqueueSetState (react-dom.production.min.js:133:211)
    at w3.setState (react.production.min.js:12:369)
    at IrisGrid.js:1446:18
    at Object.$lambda$3 (dh-core.js:12276:5)
    at Function.onInvoke_127 (dh-core.js:24649:17)
    at lambda (dh-core.js:195:22)
    at Array.forEach (<anonymous>)
    at Object.$fireEvent (dh-core.js:12255:15)
    at Object.$refire (dh-core.js:29373:106)
    at TableViewportSubscription.notifyUpdate_0 [as notifyUpdate] (dh-core.js:29472:11)
    at Object.$onDataChanged (dh-core.js:28706:15)
    at AbstractTableSubscription$1methodref$onDataChanged$Type.onDataChanged (dh-core.js:28850:11)
    at WebBarrageSubscription$ViewportImpl.applyUpdates_1 [as applyUpdates] (dh-core.js:22270:27)
    at Object.$onFlightData (dh-core.js:28723:48)
    at Function.apply_157 (dh-core.js:28877:11)
    at lambda (dh-core.js:195:22)
    at dh-internal.js:1:350285
    at Array.forEach (<anonymous>)
    at dh-internal.js:1:350264
    at dh-internal.js:1:11539
    at Array.forEach (<anonymous>)
    at e.rawOnMessage (dh-internal.js:1:11501)
    at dh-internal.js:1:9314
    at Array.forEach (<anonymous>)
    at e.onTransportChunk (dh-internal.js:1:9189)
    at Object.$onMessage (dh-core.js:25125:35)
    at MultiplexedWebsocketTransport$3methodref$onMessage$Type.handleEvent_2 [as handleEvent] (dh-core.js:25268:10)

Table with LocalDate partitions

Steps to reproduce

  1. In the web IDE, create a PartitionedTable with a LocalDate column, write it to disk as a parquet table in partitions, and read it back as a single partition. Note that this will work, ddeasdfe

from deephaven.parquet import write_partitioned, read, ParquetFileLayout
from deephaven import empty_table
from deephaven.column import col_def, ColumnType
from deephaven import dtypes
pt = empty_table(5).update_view(["I=i", "Date=java.time.LocalDate.ofEpochDay(ii)"]).partition_by("Date")

write_partitioned(pt, destination_dir='/tmp/table_with_partitions')

table_with_partitions = read('/tmp/table_with_partitions', table_definition=[
col_def('I', data_type=dtypes.int32),
col_def("Date", data_type=dtypes.LocalDate, column_type=ColumnType.PARTITIONING)
])

  1. View table table_with_partitions.

Expected results
A partition selector dropdown should be visible, and there should be no errors in the JS console.

Actual results
Instead of seeing the table and the partition selector, we see an error message:

Unable to open table. Error: Error displaying table: Error: java.lang.NumberFormatException: For input string: "1970-01-05"

This is the same error as the PartitionedTable case, but hiding the table itself.


In both cases the web UI seems to be trying to serialize the LocalDate via Filters

@niloc132 niloc132 added the bug Something isn't working label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant