Skip to content

Conversation

@asg017
Copy link
Owner

@asg017 asg017 commented Jan 11, 2025

refs #165

.load dist/vec0
create virtual table vec_items using vec0(
  vector float[1]
);

insert into vec_items(rowid, vector)
  select value, json_array(value) from generate_series(1, 100);

select vec_to_json(vector), distance
from vec_items
where vector match '[1]'
  and k = 5;

/*
┌─────────────────────┬──────────┐
│ vec_to_json(vector) │ distance │
├─────────────────────┼──────────┤
│ '[1.000000]'        │ 0.0      │
│ '[2.000000]'        │ 1.0      │
│ '[3.000000]'        │ 2.0      │
│ '[4.000000]'        │ 3.0      │
│ '[5.000000]'        │ 4.0      │
└─────────────────────┴──────────┘
*/


select vec_to_json(vector), distance
from vec_items
where vector match '[1]'
  and k = 5
  -- the new magic
  and distance > 4.0;

/*
┌─────────────────────┬──────────┐
│ vec_to_json(vector) │ distance │
├─────────────────────┼──────────┤
│ '[6.000000]'        │ 5.0      │
│ '[7.000000]'        │ 6.0      │
│ '[8.000000]'        │ 7.0      │
│ '[9.000000]'        │ 8.0      │
│ '[10.000000]'       │ 9.0      │
└─────────────────────┴──────────┘
*/

TODO

  • tests
  • docs
  • edge cases - when multiple items have same distance calculations
  • should distances be f64?

@asg017 asg017 mentioned this pull request Jan 31, 2025
vlasky added a commit to vlasky/sqlite-vec that referenced this pull request Nov 28, 2025
Implements WHERE constraints on the distance column in KNN queries, enabling
cursor-based pagination and range queries. Based on upstream PR asg017#166 by Alex
Garcia with completion and enhancements.

Features:
- Supports GT, GE, LT, LE operators on distance column
- Works with all vector types (float32, int8, bit)
- Compatible with partition keys, metadata, and auxiliary columns
- Multiple constraints can be combined (e.g., distance >= 3.0 AND distance <= 6.0)

Implementation:
- Added VEC0_IDXSTR_KIND_KNN_DISTANCE_CONSTRAINT to idxStr encoding
- Distance filtering applied during KNN search before top-k selection
- Cast f64 to f32 for comparison to match internal precision

Enhancements over original PR:
- Fixed variable shadowing in inner loops (i -> j)
- Added comprehensive test coverage (15 tests)
- Fixed bit/int8 vector type handling in tests
- Documented precision handling and pagination caveats

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Alex Garcia <alex@alex.garcia>
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants