Debug LangChain Agents with LangSmith Tracing

When a LangChain agent returns the wrong answer or loops through tool calls, finding the cause requires visibility into each step the agent took. LangSmith is an observability platform for LLM applications that records traces of those steps. According to the LangChain documentation at https://docs.langchain.com/langsmith/trace-with-langchain, tracing a LangChain application requires no extra code: once tracing is configured, developers run their LangChain code as normal and each invocation is logged automatically.

Setting Up Tracing

LangSmith tracing for LangChain is configured through environment variables rather than a separate tool. The documentation lists the core variables: LANGSMITH_TRACING set to true, LANGSMITH_API_KEY set to an API key, and LANGSMITH_PROJECT to name the project, which defaults to default when unspecified. Applications running outside the US region also set LANGSMITH_ENDPOINT.

API keys are created from the LangSmith account settings under API Keys. The docs note that signing up requires no credit card and supports logging in with Google, GitHub, or email.

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=<your-api-key>
export LANGSMITH_PROJECT=agent-debugging

With these variables in place, invoking a chain or agent logs the trace without further changes to the code.

Controlling and Annotating Runs

For finer control, the documentation describes several programmatic options. In Python, developers can use the tracing_context context manager or pass a LangChainTracer callback; the JavaScript and TypeScript SDKs accept a LangChainTracer instance as a callback. Tracing can also work without environment variables by constructing a Client with an API key and URL and passing it through the tracing context or tracer.

Several features make traces easier to investigate later. A custom run name can be set with run_name in the run configuration. Metadata and tags supplied through RunnableConfig are inherited by all child runnables, which helps when filtering traces from a specific environment or user. Custom and accessible run IDs allow a particular run to be queried directly. Distributed tracing links runs across different services, and the LangSmith SDK’s traceable decorator interoperates with LangChain runs.

Making Sure Traces Are Recorded

Because Python tracing runs in a background thread, traces can be lost when a program exits before they finish sending. The documentation recommends calling wait_for_all_tracers() in Python or awaitAllCallbacks() in JavaScript and TypeScript, or setting LANGCHAIN_CALLBACKS_BACKGROUND to false, so that pending traces are flushed.

Reviewing Traces

Once recorded, traces are reviewed in the LangSmith interface or through its API. The observability documentation states that traces can be filtered, exported, shared, and compared via the UI or API. Beyond individual traces, LangSmith supports dashboards and alerts for tracking quality over time, automation rules and webhooks, and feedback collection through annotation queues. These tools give developers a record of agent decisions to inspect when an agent behaves unexpectedly, rather than relying on print statements scattered through the code.