LangChain

Install Necessary Libraries

Install LangChain along with auxiliary libraries for embedding and vector store.

Run `pip install langchain langchain-openai langchain-community chromadb` in a virtual Python environment.

Use a separate virtual environment for the project to avoid library version conflicts.

Load and Split Source Documents

Read internal documents (PDF, docx, txt) and split them into smaller segments for embedding.

Use `DirectoryLoader` to load files, then `RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)` to split segments.

Choose a chunk_size of around 300-500 tokens to balance sufficient context and retrieval accuracy.

Create Embeddings and Store in Vector Store

Convert text segments into vectors and store them in a local or cloud vector database.

Use `OpenAIEmbeddings()` combined with `Chroma.from_documents(docs, embedding=embeddings)` to create and store the vector store.

For large datasets or scaling needs, consider switching Chroma local to Pinecone or pgvector.

Build Retriever and Query Chain

Connect the vector store with the LLM through a RetrievalQA chain.

Initialize `retriever = vectorstore.as_retriever()` and then use `RetrievalQA.from_chain_type(llm=llm, retriever=retriever)`.

Limit `k` (number of segments returned) to about 3-5 to avoid providing too much context that may confuse the LLM.

Test with Real Questions

Run the chain with real questions that internal users may ask.

Call `chain.invoke({'query': 'What is the warranty policy for product X?'})` and check if the response is from the correct source document.

Log the questions and the retrieved document segments for easier debugging when responses are incorrect.

Monitor and Debug Using LangSmith

Enable tracing to view the details of each processing step of the chain when results are unexpected.

Set the environment variable `LANGCHAIN_TRACING_V2=true` and `LANGCHAIN_API_KEY`, then view the trace on the LangSmith dashboard.

Use tracing to identify if the retrieval returned the wrong document segment — the most common cause of incorrect answers.