bug: dropping a column from a table with >5 columns fails with sqlglot=28.0.0

[luca-ferreri]

What happened?

When using Ibis with sqlglot=28.0.0, we encounter an error after dropping a column from a table that originally has more than five columns. For instance, the following

import ibis
import pandas as pd


df = pd.DataFrame(
    [["a", 1, 2, 5, 6], ["b", 3, 4, 7, 8]],
    columns=["one", "two", "three", "four", "five"],
    index=[5, 6],
)
t = ibis.memtable(df)
t.execute()
t = t.drop("three")
t.execute()

results in ValueError: schema names don't match input data columns

while we have no problem if the table has less than five columns

What version of ibis are you using?

11.0.0

What backend(s) are you using, if any?

duckdb==1.4.2

Code of Conduct

• I agree to follow this project's Code of Conduct

[JonAnCla]

Thanks this is an issue that I came across too and filed this ticket for as there are potentially other breaking changes in sqlglot 28.x: #11767

Like you I also noticed that the problem only occurs when drop is used on tables with >5 columns. If that optimisation is done on the sqlglot side then maybe its a problem there rather than in ibis, but I've not had a chance to investigate further yet :)

[samlalwani]

I had to replace all .drop( with .select( and modify all column selectors to get around this issue.

[luca-ferreri]

I had to replace all .drop( with .select( and modify all column selectors to get around this issue.

In my case, the workaround was simpler: I just pinned sqlglot to a previous version (<= 27.27.0).