Skip to content
Get an API key

Docs

Quickstart

Get an API key, install an SDK and stream a reply from Llama 3.3 70B Instruct. It takes about five minutes, and a new account starts with $5 of credit.

  1. Create a key

    Create a Weir account to get $5 of credit. No card required.

    In the dashboard, open API keys, choose Create key and name it after the app that will use it. Weir shows the key once, so copy it before you close the dialog.

    Keep the key in an environment variable instead of in your code. Every example on this page reads WEIR_API_KEY:

    Terminal
    export WEIR_API_KEY="paste-your-key-here"

    On Windows PowerShell, the same line is $env:WEIR_API_KEY = "paste-your-key-here".

  2. Install the SDK

    Weir has SDKs for Python and Node, published on Weir's own package index. Install the one that matches your project:

    pip install --index-url https://pypi.weir.example/simple weir

    Both SDKs point at https://api.weir.example/v1 by default, so there is nothing to configure. Using curl or another HTTP client instead? Send the same requests to that address.

    Override the base URL

    To send requests somewhere else, such as a local mock or a proxy, set the base URL when you create the client:

    import os
from weir import Weir

client = Weir(
    api_key=os.environ["WEIR_API_KEY"],
    base_url="http://localhost:8080/v1",
)

  3. Make your first call

    Send one chat message to Llama 3.3 70B Instruct. The model ID is llama-3-3-70b. To use a different chat model, change that one string; every model page lists its ID.

    import os
from weir import Weir

client = Weir(api_key=os.environ["WEIR_API_KEY"])

response = client.chat.completions.create(
    model="llama-3-3-70b",
    messages=[{"role": "user", "content": "What does a KV cache do?"}],
)
print(response.choices[0].message.content)

    The response is JSON. The reply is in choices[0].message.content, and usage counts the tokens you are billed for:

    Response
    {
  "id": "cmpl_8f3a1c",
  "object": "chat.completion",
  "model": "llama-3-3-70b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "A KV cache keeps the attention keys and values of tokens the model has already read, so each new token reuses them instead of recomputing the whole prompt."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 17,
    "completion_tokens": 62,
    "total_tokens": 79
  }
}

    Llama 3.3 70B Instruct costs $0.54 per 1M input tokens and $0.72 per 1M output tokens, so the 79 tokens above cost about $0.000054.

    If the call fails

    StatusWhat it meansWhat to do
    401The key is missing or wrong.Check that WEIR_API_KEY is set in the shell that runs your code.
    404There is no model with that ID.Copy the ID from the model's page.
    429You went over your request limit.Wait for the number of seconds in the Retry-After header, then retry.

  4. Stream the response

    A chat reply can take a few seconds to finish. With streaming, tokens arrive as they are generated, so your app can start showing text right away. Llama 3.3 70B Instruct has a median time to first token of 212 ms.

    Set stream to true. The SDKs give you an iterator of chunks; each chunk carries the next piece of text in choices[0].delta.content.

    import os
from weir import Weir

client = Weir(api_key=os.environ["WEIR_API_KEY"])

stream = client.chat.completions.create(
    model="llama-3-3-70b",
    messages=[{"role": "user", "content": "What does a KV cache do?"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

    With curl, the -N flag turns off output buffering so you see the chunks as they come in. The response is a stream of server-sent events: one data: line per chunk with a blank line after it, and data: [DONE] at the end. The last chunk before it has an empty delta and a finish_reason. Shortened to three chunks, it looks like this:

    Event stream
    data: {"id":"cmpl_8f3a1c","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"A KV"}}]}

data: {"id":"cmpl_8f3a1c","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" cache"}}]}

data: {"id":"cmpl_8f3a1c","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

    Embedding and image models do not stream; they return in one piece.

  5. Next steps

    You have a key, an SDK and a streamed reply. From here: