text-embedding-3-small
model and stored in Nile with the relevant chunks of the transcripts.
The response to each question is generated by querying the code snippets table using pg_vector extension, and then sending the relevant documents to
the Llama 3.1 7B model, deployed on A100 GPU from Modal. The response is streamed back to the user in real-time.
Because of Nile’s virtual tenant databases, the retrieved transcripts will only be from the tenant the user selected.
Nile validates that the user has permissions to view this tenant’s data. No risk of accidentally retrieving code that belongs to the wrong tenant.
embedding
column is of type vector(1536)
. Vector type is provided by the pg_vector extension for storing embeddings.
The size of the vector has to exactly match the number of dimensions in the model you use.
The table has tenant_id
column, which makes it tenant aware. By storing embeddings in a tenant-aware table, we can use Nile’s built-in tenant
isolation to ensure that sales transcripts won’t leak between tenants.
If all went well, you’ll see the new table in the panel on the left hand side of the query editor. You can also see Nile’s built-in tenant table next to it.
You can also explore the schema in the “Schema Visualizer”
OPENAI_API_KEY
and EMBEDDING_MODEL
(e.g. text-embedding-3-small
).
create-next-app
for this:
.env
file and add your Nile connection string:
pip install -r requirements.txt
git lfs install
ssh
to use the key you just uploaded to Hugging Face when accessing the repo. You’ll need to put this snippet in your .ssh/config
file:cp -r sales-transcripts/data .
dev
mode, which will start a modal app and a web server.
This will let you see the logs in the console, and it will automatically re-deploy if you make changes.
select * from users
in the query editor.
Once you choose a tenant, you select a sales transcript and can ask questions about the transcript.