41. SKU breakdown

Daalu’s billing page has a “SKU picker” with four cards. This chapter explains what each one represents, what it costs, and how to think about which ones your team consumes.

A SKU here is a class of inference call. Different SKUs have different cost profiles because the work involved is different.

The four SKUs

1. Classifier

Short, simple LLM calls used to make routing decisions and quick classifications.

• Typical input: 100–500 tokens.
• Typical output: 1–10 tokens.
• Examples: deciding whether a question needs the long-context model, classifying an alert by team, scoring complexity.
• Per-call cost (cloud): ~$0.0001.
• Per-call latency (local): ~50 ms.
• Volume: very high; thousands of these per active user per day.

This SKU is the workhorse. Most of Daalu’s internal LLM use is classifier-tier, and it’s the SKU that benefits most from local inference.

2. Chat (default)

Standard Assistant turns and agent reasoning steps.

• Typical input: 1k–4k tokens.
• Typical output: 200–800 tokens.
• Examples: answering an operator’s question, generating an alert explanation, writing a daily briefing.
• Per-call cost (cloud): ~$0.01.
• Per-call latency (local): ~200–400 ms.
• Volume: high; the bulk of an operator’s intentional interaction.

3. Chat (large context)

Long-context tasks the local 8B can’t handle.

• Typical input: 10k–100k tokens.
• Typical output: 500–2k tokens.
• Examples: summarizing 100 logs from a noisy incident, reviewing a multi-file diff, processing a daily-report blob.
• Per-call cost (cloud): ~$0.10–0.50.
• Always cloud (local model max context is 8k).
• Volume: low; tens to hundreds per active user per day.

4. Embedding

Vector embeddings for semantic search.

• Typical input: 200–1000 tokens.
• Typical output: a fixed-size vector.
• Examples: indexing runbooks for retrieval, finding similar past alerts.
• Per-call cost (cloud): ~$0.00001.
• Per-call latency (local): ~30 ms.
• Volume: very high during indexing; low during interactive use.

What the Billing page shows

For each SKU, the page shows:

• This month — how many calls of this SKU.
• Cloud-equivalent cost — what it would have cost if all calls had been cloud.
• Local-saved cost — what you actually saved by routing local.
• Net cloud bill — what you actually paid for cloud inference (calls that did spill).

These numbers update in near-real-time on the page.

How to read the savings

Most tenants see something like:

The pattern: classifier and chat default mostly run local; chat (large) always runs cloud. Embeddings are almost free either way.

Net cloud bill ≈ $500. If you had no local, it would be ≈ $1000. Your GPU paid for half.

Picking the right model size locally

The model you serve locally affects the SKU breakdown:

• 8B INT4 — fast and cheap. Excellent for classifier and chat default. Maxes at 8k context.
• 70B INT4 — closer to GPT-4 quality on hard chat prompts. Slower, costs more in VRAM. Max context still 8k typically (configurable up to 32k with VRAM cost).

Picking 70B locally means more chat prompts succeed locally without spillover; chat-large still goes cloud.

For most teams, 8B is the right starting point. Step up to 70B when you find your team is hitting the cloud-spillover threshold often.

SKU pinning

In Settings → LLM router → Per-SKU rules admins can:

• Force a SKU to always go cloud.
• Force a SKU to always go local (request fails if local is down).
• Pick which cloud provider for cloud calls of this SKU.

Useful patterns:

• Classifier always local, even if quality dips. Frees up budget for chat-quality.
• Chat-large always to provider X. When you have a preferred long-context model.
• Embedding always local. Locks in cost.

Quotas and overage

If your plan includes a token allowance, the allowance applies across SKUs proportionally. The billing page shows what percentage of the included allowance you’ve consumed.

Overage is billed at the per-SKU rates listed on the pricing page.

What’s next

Chapter 42 covers spend analysis and forecasting — how to project the next few months’ bill and spot anomalies before they become a problem.