Vector DB Pinecone Internal Concepts and Code Snippets

This document explores the inferred internal concepts of Pinecone, a vector database, and provides illustrative code snippets using the Python client library to demonstrate its usage.

Internal Concepts of Pinecone (Inferred)

Code Snippets (Python Client)

1. Initialization and Index Creation

import pinecone
import os

# Initialize Pinecone connection
api_key = os.environ.get("PINECONE_API_KEY")
environment = os.environ.get("PINECONE_ENVIRONMENT")

pinecone.init(api_key=api_key, environment=environment)

# Define index name and dimension
index_name = "my-vector-index"
dimension = 128

# Create an index (if it doesn't exist)
if index_name not in pinecone.list_indexes():
    pinecone.create_index(index_name, dimension=dimension, metric="cosine")

# Connect to the index
index = pinecone.Index(index_name)

print(f"Connected to index: {index_name}")

Initializes the Pinecone connection and creates a new index if it doesn’t already exist.

2. Upserting Vectors with Metadata

import numpy as np

# Sample vectors and metadata
vectors_to_upsert = [
    ("vec1", np.random.rand(dimension).tolist(), {"genre": "fiction", "year": 2020}),
    ("vec2", np.random.rand(dimension).tolist(), {"genre": "science fiction", "year": 2023}),
    ("vec3", np.random.rand(dimension).tolist(), {"genre": "fiction", "year": 2021}),
]

# Upsert the vectors
index.upsert(vectors=vectors_to_upsert)

print("Vectors upserted successfully.")

Upserts vectors with their corresponding metadata into the Pinecone index.

3. Querying for Nearest Neighbors with Filtering

# Query vector
query_vector = np.random.rand(dimension).tolist()

# Query with a filter
query_results = index.query(
    vector=query_vector,
    top_k=2,
    filter={"genre": {"$eq": "fiction"}},
    include_values=True,
    include_metadata=True
)

print("Query Results:")
for match in query_results.matches:
    print(f"  ID: {match.id}, Score: {match.score}, Values: {match.values[:5]}..., Metadata: {match.metadata}")

Performs a nearest neighbor search with a metadata filter applied.

4. Fetching Vectors by ID

# IDs of vectors to fetch
ids_to_fetch = ["vec1", "vec3"]

# Fetch the vectors
fetched_vectors = index.fetch(ids=ids_to_fetch)

print("\nFetched Vectors:")
for id, vector_data in fetched_vectors.vectors.items():
    print(f"  ID: {id}, Values: {vector_data.values[:5]}..., Metadata: {vector_data.metadata}")

Retrieves vectors and their metadata based on their unique IDs.

5. Deleting Vectors

# IDs of vectors to delete
ids_to_delete = ["vec2"]

# Delete the vectors
index.delete(ids=ids_to_delete)

print("\nVector 'vec2' deleted successfully.")

Deletes specific vectors from the index using their IDs.

6. Deleting All Vectors (Use with Caution)

# Delete all vectors in the index
index.delete(delete_all=True)

print("\nAll vectors deleted from the index.")

Deletes all vectors from the index. Use this operation with caution.

7. Describing the Index

# Get information about the index
index_description = pinecone.describe_index(index_name)
print("\nIndex Description:")
print(index_description)

Retrieves and displays information about the specified Pinecone index.

Please remember that the internal implementations of Pinecone are proprietary and may differ from these inferred concepts. This explanation is based on the observable behavior and common practices in vector database technology.