Skip to content
Get an API key

Live · serverless · 3 regions

Llama 3.1 8B Instruct

by Meta

A small, fast model for classification, extraction and short replies where cost per request matters most.

  • chat
  • FP8
  • dedicated
Serverless priceUSD
Input
$0.10per 1M tokens
Output
$0.14per 1M tokens

Billed per token, no minimums.

Reference

Specifications

Model ID
llama-3-1-8b
Endpoint
POST /v1/chat/completions
Publisher
Meta
Tasks
chat
Served context
128K tokens
Quantization
FP8
Decode speed
240 tok/smedian, one request
p50 first token
74 mslast 24 hours
Regions
us-east · us-west · eu-west
Dedicated
1× NVIDIA H100 per replica

Last 24 hours

Latency and speed

Median of each 15-minute step on the shared pool, measured at the API edge. Time to first token includes queueing; decode speed is for one request.

Time to first token · ms

74 msmedian

p95 98 ms

Decode speed · tok/s

240 tok/smedian

p5 200 tok/s

Show the data
Median of each 15-minute step over the last 24 hours, newest first
StepTime to first tokenDecode speed
15m ago75 ms246 tok/s
30m ago77 ms242 tok/s
45m ago77 ms231 tok/s
1h ago77 ms232 tok/s
1h 15m ago77 ms236 tok/s
1h 30m ago76 ms239 tok/s
1h 45m ago79 ms232 tok/s
2h ago78 ms234 tok/s
2h 15m ago97 ms206 tok/s
2h 30m ago77 ms232 tok/s
2h 45m ago77 ms239 tok/s
3h ago74 ms243 tok/s
3h 15m ago73 ms241 tok/s
3h 30m ago76 ms232 tok/s
3h 45m ago77 ms240 tok/s
4h ago78 ms240 tok/s
4h 15m ago74 ms238 tok/s
4h 30m ago76 ms238 tok/s
4h 45m ago74 ms239 tok/s
5h ago77 ms244 tok/s
5h 15m ago73 ms243 tok/s
5h 30m ago74 ms237 tok/s
5h 45m ago79 ms234 tok/s
6h ago78 ms233 tok/s
6h 15m ago76 ms244 tok/s
6h 30m ago75 ms236 tok/s
6h 45m ago72 ms238 tok/s
7h ago73 ms239 tok/s
7h 15m ago72 ms244 tok/s
7h 30m ago71 ms244 tok/s
7h 45m ago71 ms244 tok/s
8h ago74 ms242 tok/s
8h 15m ago77 ms235 tok/s
8h 30m ago71 ms240 tok/s
8h 45m ago76 ms243 tok/s
9h ago70 ms248 tok/s
9h 15m ago74 ms246 tok/s
9h 30m ago76 ms238 tok/s
9h 45m ago98 ms200 tok/s
10h ago72 ms244 tok/s
10h 15m ago73 ms241 tok/s
10h 30m ago75 ms245 tok/s
10h 45m ago74 ms243 tok/s
11h ago71 ms251 tok/s
11h 15m ago73 ms239 tok/s
11h 30m ago72 ms240 tok/s
11h 45m ago70 ms243 tok/s
12h ago69 ms254 tok/s
12h 15m ago104 ms198 tok/s
12h 30m ago72 ms239 tok/s
12h 45m ago74 ms243 tok/s
13h ago73 ms242 tok/s
13h 15m ago70 ms250 tok/s
13h 30m ago68 ms256 tok/s
13h 45m ago73 ms239 tok/s
14h ago71 ms245 tok/s
14h 15m ago69 ms252 tok/s
14h 30m ago67 ms254 tok/s
14h 45m ago67 ms248 tok/s
15h ago68 ms254 tok/s
15h 15m ago70 ms250 tok/s
15h 30m ago73 ms244 tok/s
15h 45m ago73 ms241 tok/s
16h ago70 ms244 tok/s
16h 15m ago70 ms245 tok/s
16h 30m ago70 ms244 tok/s
16h 45m ago72 ms252 tok/s
17h ago72 ms247 tok/s
17h 15m ago71 ms244 tok/s
17h 30m ago73 ms249 tok/s
17h 45m ago74 ms238 tok/s
18h ago68 ms245 tok/s
18h 15m ago103 ms195 tok/s
18h 30m ago72 ms250 tok/s
18h 45m ago68 ms253 tok/s
19h ago98 ms197 tok/s
19h 15m ago74 ms239 tok/s
19h 30m ago91 ms213 tok/s
19h 45m ago98 ms200 tok/s
20h ago101 ms200 tok/s
20h 15m ago69 ms245 tok/s
20h 30m ago72 ms244 tok/s
20h 45m ago75 ms234 tok/s
21h ago76 ms239 tok/s
21h 15m ago74 ms235 tok/s
21h 30m ago76 ms238 tok/s
21h 45m ago99 ms197 tok/s
22h ago76 ms237 tok/s
22h 15m ago75 ms234 tok/s
22h 30m ago75 ms244 tok/s
22h 45m ago76 ms239 tok/s
23h ago78 ms239 tok/s
23h 15m ago76 ms239 tok/s
23h 30m ago73 ms236 tok/s
23h 45m ago73 ms239 tok/s
24h ago77 ms239 tok/s

API

Make a request

The same request shape as every other model on /v1/chat/completions. Switching between them is a change to one string.

curl https://api.weir.example/v1/chat/completions \
  -H "Authorization: Bearer $WEIR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3-1-8b",
    "messages": [
      { "role": "user", "content": "What does a KV cache do?" }
    ]
  }'
POST /v1/chat/completionsapi.weir.example

Dedicated

Run it on GPUs reserved for you

Replica
1× H100
Replica rate
$2.49 / hour
Peak throughput
30,000 tok/s
Break-even, 1 replica
15.1B tok/mo

Replicas reserved for you, billed by the second, with a 99.95% monthly uptime SLA. Break-even is where one replica costs the same as serverless, at half input and half output tokens.

Run on dedicated GPUsCompare costs

Compare

Similar models