This document describes errors you might encounter when you generate a ScaNN index. Examples of errors and recommended fixes are also provided.
Cannot create ScaNN index with empty table
Description
When you try to generate an index on a table with no data, or if you try to truncate a table with a ScaNN index generated on it, the following error occurs:
ERROR: Cannot create ScaNN index "[INDEX_NAME]" for table "[TABLE_NAME]", error: FAILED_PRECONDITION: Cannot create ScaNN index "[INDEX_NAME]" with empty table "[TABLE_NAME]". Once the table is populated with data, create the index. See documentation to bypass this validation.
Sample queries that cause the error
-
Query Example A
create table t1 ( a INT , b VECTOR ( 512 )); CREATE TABLE create index on t1 using ScaNN ( b cosine ) with ( num_leaves = 10 , quantizer = 'sq8' ); -
Query Example B
truncate t1 ;
Recommended fix
- Before you generate a ScaNN index, make sure that your table is populated with embedding vectors.
- To create an index on a table with no data, you can enable deferred index creation. For more information, see Create a deferred index for empty or nearly empty tables .
Cannot create ScaNN index
Description
When you try to generate an index on a table with few rows populated, the following error occurs:
ERROR: Cannot create ScaNN index "[INDEX_NAME]" for table "[TABLE_NAME]", error: INVALID_ARGUMENT: Number of rows ([ACTUAL]) must be larger than ([EXPECTED]).
Sample query that causes the error
create
table
t1
(
a
INT
,
b
VECTOR
(
512
));
CREATE
TABLE
insert
into
t1
select
(
random
()
*
1
e9
)::
int
,
random_vector
(
512
)
from
generate_series
(
1
,
5
);
INSERT
0
5
create
index
on
t1
using
scann
(
b
cosine
)
with
(
num_leaves
=
100
,
quantizer
=
'sq8'
);
Recommended fix
- Before you generate a ScaNN index, make sure that your table is populated with
embedding vectors. The required number of rows depends on the index creation mode:
- For
AUTOmode, we recommend at least 10,000 rows. - For
MANUALmode, we recommend that the number of rows in the table is greater than the value defined in thenum_leavesparameter.
- For
- To create an index on a table with an insufficient number of rows, you can enable deferred index creation. For more information, see Create a deferred index for empty or nearly empty tables .
Cannot create ScaNN index for a table with only one row
Description
This error happened because ScaNN (and most ANN indexes like IVFFlat) needs a training set to build its internal structures (clusters/leaves). When you insert exactly one row , you get the following error:
ERROR: Cannot create ScaNN index "[INDEX_NAME]" for table "[TABLE_NAME]", error: INVALID_ARGUMENT: Number of samples "[NUM_SAMPLES]" must be larger than 1 and the number of leaves "[NUM_LEAVES]"
Sample query that causes the error
create
table
t1
(
a
INT
,
b
VECTOR
(
512
));
CREATE
TABLE
insert
into
t1
select
(
random
()
*
1
e9
)::
int
,
random_vector
(
512
)
from
generate_series
(
1
,
1
);
INSERT
0
1
create
index
on
t1
using
scann
(
b
cosine
)
with
(
num_leaves
=
1
,
quantizer
=
'sq8'
);
Recommended fix
Make sure that your table is populated with embedding vectors before you generate a ScaNN index. We recommend that the number of rows in the table is:
- Greater than 1
- Greater than the value defined in the
num_leavesparameter
Cannot create an automatically-tuned ScaNN index for a table with less than 10k rows
Description
This error occurs when you try to create an automatically-tuned ScaNN index on a table that has less than 10k rows. When you try to do so, you receive the following error:
"ERROR: Cannot create ScaNN index "products_embedding_idx" for table "products", error: FAILED_PRECONDITION: Number of rows: 100 must be 10000 or more."
Sample query that causes the error
CREATE
INDEX
product_index
ON
product
USING
scann
(
embedding
cosine
)
Recommended fix
Ensure that your table has at least 10k rows before trying to create an automatically-tuned ScaNN index. If you need to override this limit for development or testing purposes, see Force index creation on empty or small tables .
Cannot create ScaNN index on parent partition table
Description
If you created partitioned tables from a parent table, then creating a ScaNN index on the parent table generates the following error:
ERROR: Cannot create ScaNN index "[INDEX_NAME]" for parent partition table
"[TABLE_NAME]". Create ScaNN indexes on the child tables instead. See
documentation to bypass this validation.
Sample query that causes the error
create
table
t1
(
a
INT
,
b
VECTOR
(
512
))
partition
by
range
(
a
);
CREATE
TABLE
CREATE
TABLE
t1_one_ten
PARTITION
of
t1
for
values
from
(
1
)
to
(
10
);
CREATE
TABLE
insert
into
t1_one_ten
select
(
random
()
*
1
e9
)::
int
,
random_vector
(
512
)
from
generate_series
(
1
,
100
);
INSERT
0
100
CREATE
TABLE
t1_eleven_twenty
PARTITION
of
t1
for
values
from
(
11
)
to
(
20
);
CREATE
TABLE
insert
into
t1_eleven_twenty
select
(
random
()
*
1
e9
)::
int
,
random_vector
(
512
)
from
generate_series
(
1
,
100
);
INSERT
0
100
create
index
on
t1
using
scann
(
b
cosine
)
with
(
num_leaves
=
10
,
quantizer
=
'sq8'
);
Recommended fix
- You can't generate a
ScaNNindex on the parent table of a partitioned table. - You must generate the
ScaNNindexes on the partitioned table. - Try bypassing this validation by following the steps in Create a ScaNN index .
