Skip to content

bug: in bigquery to_polars doesn't respect timestamp units #11790

@aandrestrumid

Description

@aandrestrumid

What happened?

In bigquery timestamp and datetime have got microseconds precision (us).

When executing a query with to_pyarrow it is accurately reflected in the resulting schema, using microseconds.

When using to_polars, I get timestamp columns with nanosecond precision. It should be microseconds.

import ibis
import polars as pl
import pyarrow as pa

QUERY = 'SELECT TIMESTAMP "2025-01-01 12:34:56" AS ts'

connection = ibis.bigquery.connect()
query = connection.sql(QUERY)

arrow_table = query.to_pyarrow()
assert arrow_table["ts"].type == pa.timestamp("us", "UTC")
from_arrow = pl.from_arrow(arrow_table)
assert from_arrow["ts"].dtype == pl.Datetime("us", "UTC")

direct_from_ibis = query.to_polars()
assert direct_from_ibis['ts'].dtype == pl.Datetime("ns", "UTC") # should be us

What version of ibis are you using?

11.0.0

What backend(s) are you using, if any?

bigquery

Relevant log output

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIncorrect behavior inside of ibis

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions