EN ▾
Get API key

LangChain Agentic AI Documentation: Tools and JSON Mode

Most langchain agentic ai documentation assumes a vendor's own model class, but any OpenAI-compatible endpoint works with ChatOpenAI once you set a base URL. This guide wires LangChain to this API and builds a tool-calling agent step by step.

Updated

Key points

  • Use ChatOpenAI with base_url set to this API and model set to uncensored.
  • Tool calling and JSON mode are supported, so bind_tools and response_format work as documented.
  • Cap agent loops, validate tool arguments and keep max_concurrency at or below 8 on a full key.
  • Streaming over SSE returns token usage at the end, which helps you track the cost of each agent run.
Robotic arm moving cubes through tool stations on a looping track, representing a LangChain agent loop

What you need before you start

If you are reading langchain agentic ai documentation and wondering what changes when the model is not from the vendor in the examples, the answer is very little. An agent in LangChain is a loop: the model reads the conversation, decides whether to call a tool, your code runs the tool, and the result goes back to the model until it produces a final answer. The model endpoint only has to support chat completions with tool calling, and this API does.

You need three things: an API key from the key page, Python 3.10 or newer, and the packages below. The base URL is https://api.unlimitedaiapi.com/v1 and the model ID is uncensored. It is one open-weight model run by an independent provider, so there is no model list to choose from; GET /v1/models returns it for client discovery.

pip install -U langchain langchain-openai
export UNLIMITED_API_KEY="your-key-here"

Step 1: connect ChatOpenAI to the endpoint

LangChain's ChatOpenAI class takes a custom base_url. Everything else about it stays the same, including invoke, stream, batch and async variants.

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="uncensored",
    base_url="https://api.unlimitedaiapi.com/v1",
    api_key=os.environ["UNLIMITED_API_KEY"],
    temperature=0.2,
)

print(llm.invoke("Reply with one short sentence.").content)

If this prints a sentence, authentication, routing and the model name are all correct. Auth is the standard Authorization: Bearer header, which the class adds for you. A common mistake is leaving off /v1 in the base URL, which usually makes the first call fail.

Step 2: define tools and bind them

Tools are plain Python functions with a docstring. The docstring and the type hints become the schema the model sees, so write them like you are explaining the function to a new colleague.

import os
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

@tool
def get_order_status(order_id: str) -> str:
    '''Look up the status of an order by its ID, for example A100.'''
    return {"A100": "shipped", "A101": "processing"}.get(order_id, "not found")

llm = ChatOpenAI(
    model="uncensored",
    base_url="https://api.unlimitedaiapi.com/v1",
    api_key=os.environ["UNLIMITED_API_KEY"],
    temperature=0,
)
llm_with_tools = llm.bind_tools([get_order_status])

ai = llm_with_tools.invoke("Where is order A100?")
print(ai.tool_calls)

The printed list holds a call with the name get_order_status and arguments such as {"order_id": "A100"}. Under the hood LangChain sends your function as an entry in the tools array of the chat completions request, and the API supports tools, tool_choice and streamed tool_calls. To force a tool call, pass tool_choice to bind_tools.

Why the temperature is low

Agents chain several model calls, so one malformed argument can derail a run. A temperature of 0 to 0.2 makes tool selection more repeatable. Raise it only for the final answer step if you want more varied prose.

Step 3: run the agent loop

Writing the loop yourself takes fifteen lines and shows exactly what happens at each step, which helps when something goes wrong. Tool results go back to the model as ToolMessage objects, which become role: "tool" messages on the wire.

import os
from langchain_core.messages import HumanMessage, ToolMessage
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

@tool
def get_order_status(order_id: str) -> str:
    '''Look up the status of an order by its ID, for example A100.'''
    return {"A100": "shipped", "A101": "processing"}.get(order_id, "not found")

llm = ChatOpenAI(
    model="uncensored",
    base_url="https://api.unlimitedaiapi.com/v1",
    api_key=os.environ["UNLIMITED_API_KEY"],
    temperature=0,
)
tools = {"get_order_status": get_order_status}
llm_with_tools = llm.bind_tools(list(tools.values()))

messages = [HumanMessage("Where is order A100, and what about A101?")]
for _ in range(5):                      # hard cap on loop length
    ai = llm_with_tools.invoke(messages)
    messages.append(ai)
    if not ai.tool_calls:
        break
    for call in ai.tool_calls:
        result = tools[call["name"]].invoke(call["args"])
        messages.append(ToolMessage(content=str(result), tool_call_id=call["id"]))

print(ai.content)

Note the range(5). An agent without a step limit can loop forever on a confusing tool result, and every pass costs input tokens for the whole conversation so far. Five is a reasonable ceiling for simple tasks; raise it deliberately.

Using the prebuilt agent helper

Recent LangChain versions ship a helper that runs this loop for you. The name and import path have moved between releases, so confirm them against the installed version's docs. In the current 1.x line it looks like this:

import os
from langchain.agents import create_agent
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

@tool
def get_order_status(order_id: str) -> str:
    '''Look up the status of an order by its ID, for example A100.'''
    return {"A100": "shipped", "A101": "processing"}.get(order_id, "not found")

llm = ChatOpenAI(
    model="uncensored",
    base_url="https://api.unlimitedaiapi.com/v1",
    api_key=os.environ["UNLIMITED_API_KEY"],
    temperature=0,
)

agent = create_agent(llm, tools=[get_order_status], system_prompt="You are a concise support assistant.")
result = agent.invoke({"messages": [{"role": "user", "content": "Where is order A100?"}]})
print(result["messages"][-1].content)

Use the helper when you want the standard pattern, and the hand-written loop when you need custom stopping rules, logging per step, or tool results that you want to filter before the model sees them.

Structured output with JSON mode

Agents often need to hand data to other code, such as a plan, a classification or a list of extracted fields. JSON mode forces the reply to be a valid JSON object. In LangChain you can pass it through bind:

import json
import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="uncensored",
    base_url="https://api.unlimitedaiapi.com/v1",
    api_key=os.environ["UNLIMITED_API_KEY"],
    temperature=0,
)
json_llm = llm.bind(response_format={"type": "json_object"})

msg = json_llm.invoke(
    "Extract the city and country from: 'I moved to Lyon, France last spring.' "
    "Answer as JSON with the keys city and country."
)
print(json.loads(msg.content))

Mention the word JSON in the prompt and describe the keys you expect, as in the example. JSON mode guarantees syntax, not your schema, so still validate the parsed object (with Pydantic, for example) and handle a missing key.

Limits, streaming and cost for agent runs

Agents make many calls in a short time, so the account limits matter more than for a simple chatbot. A full key allows 8 concurrent requests and 300 requests per minute. A trial key allows 2 and 60. If you fan out with batch or run several agents at once, cap the parallelism in the config:

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="uncensored",
    base_url="https://api.unlimitedaiapi.com/v1",
    api_key=os.environ["UNLIMITED_API_KEY"],
)
prompts = ["Name one prime number.", "Name one even number.", "Name one square number."]
answers = llm.batch(prompts, config={"max_concurrency": 8})
for a in answers:
    print(a.content)

Use 2 instead of 8 on the trial key. For interactive agents, llm.stream(...) returns tokens over SSE as they are generated, and the usage numbers arrive at the end of the stream.

Cost is easy to estimate. A 5-step run that averages 4,000 input and 300 output tokens per step uses 20,000 input and 1,500 output tokens in total, which is $0.003 + $0.00075 = $0.00375 per run at $0.15 and $0.50 per 1M tokens. The pricing page has the rates, and the docs list the request parameters and headers, including X-Request-Id for tracing a single failing call.

Designing tools the model can use

Most agent failures trace back to the tools, not the loop. A few rules help with any model:

  • Keep each tool narrow. A tool called get_order_status works better than one called database_query that takes free-form text.
  • Use simple argument types: strings, integers and short enums. Deeply nested objects give the model more ways to get a field wrong.
  • Return short, plain results. A tool that dumps 50,000 characters into the conversation fills the context window and raises the cost of every later step.
  • Return errors as text, such as "order_id must look like A100", instead of raising exceptions that end the run.

Test the agent on a handful of real prompts with the $0.50 trial credit before you scale up. New users get it with one click on the key page, and it is valid for 7 days.

Pitfalls to check first

  • Tool call never appears. Make the docstring specific, lower the temperature, or force the call with tool_choice.
  • Invalid arguments. Validate inside the tool and return a clear error string, so the model can correct itself on the next step.
  • Context growth. The window is 100,000 tokens including the reply. Trim old tool output before it fills the window.

Questions and answers

Which LangChain class works with this API?

ChatOpenAI from langchain-openai. Set base_url to https://api.unlimitedaiapi.com/v1, model to uncensored and pass your key as api_key.

Does the API support tool calling for agents?

Yes. It accepts the tools and tool_choice parameters, returns tool_calls (also when streaming), and takes results back as messages with role tool.

Can LangChain parse structured output through this API?

Yes, through JSON mode: set response_format to a json_object type. It guarantees valid JSON syntax, so validate the fields yourself.

How many agents can I run in parallel?

A full key allows 8 concurrent requests and 300 per minute. The trial key allows 2 concurrent and 60 per minute.

Your key is one form away

Create an account, copy the key, change the base URL. That is the whole setup.

Get API key