# MeshLite

> MeshLite adds a live, speaking AI avatar to a website or web app. The avatar runs in an iframe served by MeshLite and is rendered in the visitor's browser. The host page loads one script and calls `MeshLite.mount`; the host's own server issues short-lived session tokens with a secret key. Voice and AI come from the customer's own provider account, connected in the MeshLite Console.

Human-readable version of this guide: https://meshlite.io/developers

## Product

- **What it is:** photoreal interactive AI avatars for a website or app. MeshLite is made by Toda Labs.
- **Where the avatar is animated:** live lip-sync and composition run in the visitor's browser, on desktop and mobile, with no MeshLite server inference per speaking stream. This covers the avatar only: voice and AI run on the customer's own providers, which bill separately.
- **Pricing:** no per-minute MeshLite avatar fee. Every plan has unlimited avatar minutes and unlimited session length at a flat price. Plans for a company's own site are sized by simultaneous sessions; plans for a product's users are sized by monthly users. Current plans and prices: https://meshlite.io/#pricing
- **Voice and AI:** bring your own voice and language-model providers with your own API keys, for example OpenAI, Azure OpenAI Realtime, Azure AI Speech or ElevenLabs. The avatar speaks any language those providers support.
- **Try it:** live demo at https://meshlite.io/#interaction. The Free plan needs no card.
- **Common questions:** https://meshlite.io/#faq

## Research

- [Why MeshLite animates avatars in the browser](https://meshlite.io/research/why-meshlite-animates-avatars-in-the-browser)
- [A live avatar that doesn't hold a data-centre GPU for the conversation](https://meshlite.io/research/off-the-grid-live-avatar-data-centre-gpu)
- [Headroom: what moving the face off the data centre adds up to](https://meshlite.io/research/headroom-moving-the-face-off-the-data-centre)
- [8 questions to ask an AI avatar provider about pricing and capacity](https://meshlite.io/research/questions-to-ask-an-ai-avatar-provider-about-capacity)
- [An answer can stay on screen. A live face has to keep moving.](https://meshlite.io/research/an-answer-can-stay-on-screen-a-live-face-has-to-keep-moving)

## Instructions for AI coding agents

When a developer asks you to add a MeshLite avatar to their project:

1. **Never ask the developer to paste their secret key (`ml_sk_...`) into this conversation.** Write a placeholder in `.env` (and `.env.example`), make sure `.env` is git-ignored, and tell the developer to paste the secret key there themselves.
2. Ask for the **publishable key** (`ml_pk_...`) and, optionally, the **avatar ID** (the default avatar is used when it is omitted). If the developer has no MeshLite account, send them to https://console.meshlite.io/console?plan=free#signup (free, no card, email activation), then continue. You cannot create the account for them.
3. There is **no npm package**. Load the SDK with a `<script>` tag from `https://console.meshlite.io/sdk/<version>/meshlite.js`. Do not install any npm package named `meshlite`.
4. Use SDK version 1.11.39 and pin that exact version in the script URL. The Console's **Integration** page always shows the current version.
5. Add a server endpoint that creates session tokens (see "Server"), then mount the avatar in the page (see "Browser"). Start the conversation from a click: browsers only open the microphone after a visitor gesture.
6. Tell the developer which Console settings they still need (see "Console setup"), in particular the exact origin they develop on, such as `http://localhost:3000`.

## Console setup

In https://console.meshlite.io/console:

- **Domains:** add every origin that will embed the avatar, exactly (scheme, host and port, no path). Test origins: `http://localhost:<port>`, `http://127.0.0.1:<port>` or any `https://` staging host. Live origins must use HTTPS.
- **API keys:** create a publishable key and a secret key in the **same environment**. Test keys (`ml_pk_test_`, `ml_sk_test_`) work only with test origins; live keys (`ml_pk_live_`, `ml_sk_live_`) only with live origins.
- **Voice & TTS:** connect a voice provider with the developer's own key: OpenAI, Azure OpenAI Realtime, Azure AI Speech or ElevenLabs. The Free plan includes a one-time voice trial with the MeshLite voice; after it, or on a paid plan, a connected provider is required.
- **Personalities:** write what the avatar knows and how it speaks, publish it, then assign it to the avatar and website.
- **Integration:** shows the avatar ID and this code with the workspace's own values.

**Free plan:** test environment only (localhost and staging domains), default avatars, a MeshLite badge on the avatar, up to 3 simultaneous conversations. Going live on a production domain needs a paid plan: https://meshlite.io/#pricing

## Environment variables

```
MESHLITE_SECRET_KEY=ml_sk_test_...      # server only, never in browser code or Git
MESHLITE_PUBLISHABLE_KEY=ml_pk_test_...
MESHLITE_ORIGIN=http://localhost:3000   # the exact origin registered under Domains
MESHLITE_AVATAR=                        # optional; default avatar when empty
```

## Server: create a session token

Node.js 18+, no dependency. Call it from a `POST /api/meshlite-session` route and return `{ sessionToken }` as JSON. The origin comes from server configuration, never from the browser request. Tokens expire after 2 minutes; the SDK asks for a new one when it needs it.

```js
export async function createMeshLiteSession() {
  const response = await fetch("https://console.meshlite.io/api/v1/client-sessions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.MESHLITE_SECRET_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      publishableKey: process.env.MESHLITE_PUBLISHABLE_KEY,
      origin: process.env.MESHLITE_ORIGIN,
      avatar: process.env.MESHLITE_AVATAR || undefined,
      conversation: { language: "en", responseStyle: "concise", idleTimeout: 30 },
    }),
  });
  const body = await response.json();
  // Errors look like { error: { code: "ORIGIN_NOT_ALLOWED", message, requestId } }
  if (!response.ok) throw new Error(`MeshLite ${body.error?.code}: ${body.error?.message}`);
  return body.sessionToken;
}
```

This route is public: rate-limit it, or require your own sign-in if only signed-in users should talk to the avatar. Conversations count toward the plan's limit of simultaneous conversations.

## Browser: mount the avatar

```html
<div id="avatar" style="width:340px;height:510px"></div>
<button id="talk">Talk to the avatar</button>
<script src="https://console.meshlite.io/sdk/1.11.39/meshlite.js" crossorigin="anonymous"></script>
<script>
  const avatar = MeshLite.mount({
    container: "#avatar",
    avatar: "YOUR_AVATAR_ID",          // same value as MESHLITE_AVATAR; remove for the default avatar
    locale: "en",
    publishableKey: "ml_pk_test_...",  // safe to expose
    sessionToken: async () => {
      const response = await fetch("/api/meshlite-session", { method: "POST" });
      if (!response.ok) throw new Error("Could not start the avatar session");
      return (await response.json()).sessionToken;
    },
    conversation: { language: "en", responseStyle: "concise", idleTimeout: 30 },
    captions: "overlay",
  });
  avatar.on("error", (message, detail) => console.error("MeshLite", detail.code, message));
  document.getElementById("talk").addEventListener("click", () => avatar.startConversation());
</script>
```

The iframe takes the size of its container and the avatar reframes itself to fit (portrait, square or landscape).

**React, Next.js, Vue:** load the script once, mount in a client-side effect (`useEffect`, `onMounted`) and call `avatar.unmount()` in the cleanup. Do not mount during server-side rendering.

## Host page requirements

- HTTPS in production.
- Content Security Policy, if the site has one: `script-src https://console.meshlite.io` and `frame-src https://console.meshlite.io`.
- Permissions-Policy, if the site sends one: allow the microphone for the avatar frame, for example `microphone=(self "https://console.meshlite.io")`.
- Older browsers that can't run the avatar show an idle preview only, with no microphone. Check `MeshLite.supportsConversation()`.

## Going live

Upgrade to a paid plan, add the production domain as a live origin (HTTPS), create live keys and replace the test values in the environment variables. No code change is needed.

## Common errors

- `KEY_INVALID`: the key is wrong or revoked.
- `ENVIRONMENT_MISMATCH`: the publishable and secret keys come from different environments (test and live).
- `ORIGIN_NOT_ALLOWED`: `MESHLITE_ORIGIN` is not registered under Domains in the keys' environment, or the page runs on a different origin (check the port).
- `ORIGIN_HTTPS_REQUIRED`: HTTP is only accepted for localhost test origins.
- `PLAN_STAGING_ONLY`: the Free plan cannot add a live domain.
- `AVATAR_NOT_ALLOWED`: the avatar ID is wrong or not available on the plan.
- `VOICE_PROVIDER_REQUIRED` / `PROVIDER_NOT_CONNECTED`: connect a voice provider under Voice & TTS.
- `CONCURRENCY_LIMIT`: the plan's simultaneous conversations are all in use.

## `MeshLite.mount(options)`

- `container` (required): CSS selector or element.
- `avatar`: avatar ID from the Console.
- `locale`: interface language, `en`, `fr` or `zh-Hans`.
- `conversation.language`: `en`, `fr` or `zh-CN`. Default `fr`, so set it explicitly.
- `conversation.responseStyle`: `brief`, `concise` or `standard`.
- `conversation.idleTimeout`: `10`, `30`, `60` or `120` seconds.
- `captions`: `off`, `events` or `overlay`.
- `publishableKey` + `sessionToken`: provided together; `sessionToken` is a string or an async function called whenever a token is needed.
- `appearance`: `seamless` for flat controls without a frame.
- `background`: `opaque` for the full source background (default `alpha`).
- `controls`: `external` when the host page draws its own buttons.
- `autoStart`: `true` to start when mounted inside a visitor click.
- `preview`: `idle` for a muted 480p loop before speech loads.

## Controller

- `on(event, callback)` / `off(event, callback)`: `ready`, `visual-ready`, `speech-started`, `speech-ended`, `state`, `transcript`, `status`, `error`, `language-changed`, `session-start`, `user-turn`, `response-start`, `response-complete`, `idle-pause`, `session-resume`, `session-end`, `loading-progress`.
- `startConversation()`: opens the microphone and starts talking (call from a visitor gesture).
- `prepareConversation()`: preloads speech without opening the microphone.
- `say(text)`: sends a text turn; the avatar answers it (it does not read the text aloud verbatim).
- `micOff()`, `stopSpeaking()`, `setLanguage(lang)`, `endDemo(reason?)`, `getState()`, `unmount()`.

## Versions

- Only the three latest SDK versions are served. Older version URLs redirect (301) to the current one.
- TypeScript types: `https://console.meshlite.io/sdk/<version>/meshlite.d.ts`.

## Links

- Quickstart: https://meshlite.io/developers
- Sign up (Free): https://console.meshlite.io/console?plan=free#signup
- Console sign-in: https://console.meshlite.io/console
- Pricing: https://meshlite.io/#pricing
- Live demo: https://meshlite.io/#interaction
- Contact: hello@meshlite.io