Llama 3.1 8B Instruct
by MetaReleased Jul 23, 2024
meta-llama/llama-3.1-8b-instruct- Calls tools
- Answers in a JSON schema
- Caches long prompts
Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 8B instruct-tuned version is fast and efficient. It has demonstrated strong performance compared to...
- Input / 1M tokens
- $0.06
- Output / 1M tokens
- $0.096
- Context
- 131K
- Longest answer
- 118K
What it costs
What each call is charged to the token's AI credit.
| Billed for | You pay |
|---|---|
| Input tokens | $0.06 / 1M |
| Output tokens | $0.096 / 1M |
| Cached input, read | $0.03 / 1M |
A typical call
A short chat turn: 2,000 tokens in, 500 out.
- per call
- $0.0002
- calls per $20
- 119K
What it can do
From its listing. The router passes every field through.
- Reads imagesNo
- Reads files and PDFsNo
- Hears audioNo
- Calls toolsYes
- Answers in a JSON schemaYes
- Reasons before it answersNo
- Searches the webNo
- Caches long promptsYes
- Draws imagesNo
Call it
Any router key works, in the format your SDK already speaks.
import OpenAI from "openai";const ai = new OpenAI({ baseURL: "https://relayfor.si/api/v1", apiKey: process.env.RELAYFOR_ROUTER_KEY,});const reply = await ai.chat.completions.create({ model: "meta-llama/llama-3.1-8b-instruct", messages: [{ role: "user", content: "Welcome the 1,000th holder in one line." }],});Specifications
What its maker and providers list for it.
- Reads
- Text
- Writes
- Text
- Context
- 131,072 tokens
- Longest answer
- 117,964 tokens
- Intelligence index
- 6.9 (#197 of 220 rated)
- Knowledge cutoff
- Dec 31, 2023
- Released
- Jul 23, 2024
- Moderation
- None
Request fields
The fields its providers accept, passed through as you send them.
- frequency_penalty
- logit_bias
- logprobs
- max_tokens
- min_p
- presence_penalty
- repetition_penalty
- response_format
- seed
- stop
- structured_outputs
- temperature
- tool_choice
- tools
- top_k
- top_logprobs
- top_p
Questions
How much does Llama 3.1 8B Instruct cost?
$0.06 per million input tokens and $0.096 per million output tokens, paid from the token's AI credit. A call of 2,000 tokens in and 500 out costs about $0.0002.
How do I call Llama 3.1 8B Instruct?
Make a router key for one of your tokens on the dashboard, point any OpenAI or Anthropic SDK at https://relayfor.si/api/v1 and send the model id meta-llama/llama-3.1-8b-instruct. The token's AI credit pays for each call.
How long a prompt can Llama 3.1 8B Instruct take?
Up to 131,072 tokens of context, and up to 117,964 tokens in one answer.
Are my prompts stored?
No. The RFS Router passes each call through and keeps only its token counts and cost, for the token's usage page.
More from Meta
Output price per 1M tokens
Similar in price
Output price per 1M tokens
