deno postgres.js embeddings query in deno under supabase functions

Question

Trying to run an embeddings query on data stored from openAI. Currently using Supabase functions, using the following library. Any ideas on how to fix this?

OpenAI Embeddings response is successful -

const [{ embedding }] = embeddingResponse.data.data;
const query_embedding = embedding;

import postgres from 'https://deno.land/x/postgresjs/mod.js'


const res = await sql`
select
id,
content,
  1 - (documents.embedding <=> ${query_embedding}) as similarity
from documents
where 1 - (documents.embedding <=> ${query_embedding}) > ${similarity_threshold}
order by documents.embedding <=> ${query_embedding}
limit ${match_count}

`;

On execution, the following error returns. Please note, pg_vector extension has already been enabled an i'm able to save embeddings successfully -

Error... PostgresError: malformed vector literal: "0.0013529072,-0.01248068,-0.021260735,-0.009094394,-0.0059045693,0.04160703,-0.028719138,-0.018231653,-0.005847417,-0.0039685275,0.02343253,0.020260567,-0.0124663925,-0.009144402,-0.013738035,0.028161902,0.02916207,0.00086130627,0.010994715,-0.015588349,0.00782275,-0.0022450222,0.003548814,-0.024804192,-0.009637343,0.008608597,0.015702654,-0.033634257,-0.01856028,-0.027047427,0.032262594,0.006036734,-0.012052037,-0.0009858808,-0.019074652,-0.006551107,0.00666184,0.0058259843,0.021789396,0.020774938,-0.0035988223,0.006126035,0.013023629,0.031205272,-0.011916299,0.014002366,-0.020103397,-0.05046567,-0.0022110878,0.019017499,0.035948932,0.0016172376,0.016074145,0.005100862,0.0026808102,0.017088601,0.0016190236,-0.0064903824,-0.005561654,-0.000671542,0.027804699,0.0044936165,-0.016774263,0.023303937,0.0015582991,0.01573123,0.016902857,-0.008672894,0.0055973744,-0.012752155,0.025447156,0.015245433,-0.010523207,-0.022275193,0.023089616,-0.006783289,-0.042778656,0.00006451983,-0.012730722,0.010273164,-0.004147129,-0.011473367,0.021475058,0.009851664,-0.010101707,-0.0142952725,0.016202739,0.010051698,-0.010808969,-0.00961591,0.022803852,0.026518766,0.006436802,0.009687351,0.0027969012,-0.004782951,-0.026447326,0.018574568,0.002771897,-0.0052651754,-0.0076298607,-0.007937055,-0.006904738,-0.0023039607,-0.022289481,-0.018303093,-0.018688872,0.008122801,0.022546668,-0.021089278,-0.013216519,0.042435743,-0.00572954,-0.02237521,-0.020560617,-0.015102552,-0.00805136,0.0033041297,-0.0024039776,-0.026604496,0.012845027,0.0044936165,0.013016486,-0.033348493,0.008322835,0.015702654,-0.046379264,-0.011787706,-0.018874617,0.0034845173,0.076241456,0.017817296,0.02176082,0.0035005915,-0.0016243816,0.050151333,-0.012930756,-0.0032130429,-0.0128164515,-0.023546835,0.004125697,0.010901842,0.0049186884,0.013273671,-0.01856028,0.015602636,0.040635437,0.0076227165,-0.020274855,-0.014459586,0.009865953,-0.0024986365,0.040121064,-0.019631889,0.0039042311,-0.019674754,-0.046436418,0.0013707674

Schema definition is as following -

create table documents (
  id bigserial primary key,
  content text,
  embedding vector (1536)
);

#supabase #deno #openai #postgresjs

Rodrigo Rubio · Accepted Answer · 2023-04-11 01:55:32Z

2

Managed to get it working. See working version below, ended up using "JSON.stringify".

  const res = await sql`
    select
    id,
    content,
      1 - (documents.embedding <=> ${JSON.stringify(query_embedding)}) as similarity
    from documents
    where 1 - (documents.embedding <=> ${JSON.stringify(query_embedding)}) > ${similarity_threshold}
    order by documents.embedding <=> ${JSON.stringify(query_embedding)}
    limit ${match_count}
  `;

answered Apr 11, 2023 at 1:55

Rodrigo Rubio

1,7603 gold badges16 silver badges30 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

dshukertjr · Accepted Answer · 2023-04-10 23:47:38Z

0

You need [] around your vector values.

import postgres from 'https://deno.land/x/postgresjs/mod.js'


const res = await sql`
select
id,
content,
  1 - (documents.embedding <=> [${query_embedding}]) as similarity
from documents
where 1 - (documents.embedding <=> [${query_embedding}]) > ${similarity_threshold}
order by documents.embedding <=> [${query_embedding}]
limit ${match_count}
`;

answered Apr 10, 2023 at 23:47

dshukertjr

19.2k11 gold badges71 silver badges101 bronze badges

3 Comments

dshukertjr Over a year ago

The reason why the example in this Supabase blog article does not need brackets around the embeddings is because the embedding argument that is passed has a type of vector, so SLQ knows it's a vector, where as in your case, you are just passing it as raw string, so you have add any necessary things to format it correctly.

Rodrigo Rubio Over a year ago

The "query_embedding" parameter already gets passed as an array. At first i thought it was due to the missing quotes and tried the following, but no dice. ``` const query = query_embedding.map(i => i.toString()); ```

Rodrigo Rubio Over a year ago

also tried the following ${query_string}::vector and no luck.

Collectives™ on Stack Overflow

deno postgres.js embeddings query in deno under supabase functions

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related