AI Features in Your App

This page is about putting AI inside the app you build, not the AI that builds it. You connect a provider once, describe the feature you want, and the agent wires it up: a chat assistant, a "summarise this" button, a "find me notes like this one" search, a "turn this voice memo into text" flow, or a "generate an image from a description" screen.

Three managed connectors cover almost everything people ask for:

• OpenAI - GPT chat, embeddings (the engine behind semantic search), Whisper transcription, and image generation.
• Anthropic - Claude for reasoning, chat, and structured output (when you need the model to return clean JSON your app can rely on).
• Replicate - open image, video, and audio models, paid by the GPU-second.

Connect them in Dashboard → Connectors, or straight from a build with the in-chat connect card. Connect once and every project in your account can use them.

The one thing to understand first: these are secret keys

OpenAI, Anthropic, and Replicate all hand out secret keys - anyone holding the key can spend your money. A key like that can never be bundled into an app people download, because anyone could pull it out of the app and run up your bill.

So the app never calls these providers directly. Instead, the generated app talks to a small server-side relay - a tiny piece of server code that holds the key in its own environment and forwards requests on the app's behalf. The key stays on the server. The app only ever sees the relay.

The AI never sees the decrypted key either. Your key is encrypted with AES-256-GCM the moment you save it, and it is handed to the relay's environment, never to the app bundle and never into a prompt. See Connectors Overview for the full picture of how secrets are handled.

With Supabase connected, the agent can deploy that relay for you as a Supabase Edge Function - you paste the key into your Supabase secrets, following the instructions it generates, and you are done. No Supabase? The agent still writes the relay and gives you clear setup steps to host it yourself. See Environment Variables for where the key lives and why an exported project has no .env file in the app itself.

The short version: publishable keys (Mapbox, the Supabase anon key, and friends) get embedded in the app. AI secret keys never do - they live behind a relay.

What you can build

A chat assistant

The most common request. A conversational screen where the user asks questions and the model answers, often grounded in the user's own data.

"Add an AI chat assistant that answers questions about the user's notes. It should only use what's in their notes, keep the last few messages as context, and stream the reply as it's generated."

The agent builds the chat screen, wires it to the relay, passes the relevant notes as context, and streams the response (more on streaming below). Use OpenAI's GPT models or Anthropic's Claude - both are good at this; Claude is a strong default when the answers need careful reasoning or need to stay strictly on-topic.

Content generation

Drafting, rewriting, expanding, changing tone. Anywhere a blank text box would benefit from a "help me write this" button.

"Add a 'Suggest a reply' button under each message that drafts three short, friendly responses the user can tap to send or edit."

Summarisation

Turn long text into short text. Articles, transcripts, threads, documents.

"Add a 'Summarise' button on each saved article that returns a three-bullet summary and an estimated read time."

Embeddings and semantic search

Embeddings turn text into a list of numbers that captures its meaning, so you can find things by what they mean rather than the exact words. This is what powers "find notes like this one" or "search by describing it." OpenAI generates the embeddings; the vectors get stored in your Supabase database (the agent enables the pgvector extension) and queried by similarity.

"Let the user search their notes by meaning. When they type 'ideas about pricing', return notes that are about pricing even if they never used that word."

For the full search story - keyword search, typo tolerance, and when to reach for Algolia instead of (or alongside) semantic search - see Search.

Transcription (speech to text)

OpenAI's Whisper turns audio into text. Voice notes, recorded meetings, dictation.

"Let the user record a voice memo and save it as text. Transcribe it with Whisper, show the text, and keep the original audio too."

Image generation

OpenAI generates images from a text description. Good for avatars, illustrations, thumbnails, and placeholder art.

"Add a 'Generate cover image' button that creates an illustration from the post's title."

Open image, video, and audio models via Replicate

Replicate hosts thousands of open models and bills by the GPU-second - you pay for the compute a run actually uses. Reach for it when you want something OpenAI does not offer directly: a specific open image model, short video generation, voice cloning, music or sound generation, background removal, upscaling, and so on.

"Use Replicate to remove the background from any photo the user uploads, then let them drop a new colour behind it."

Because each model is different, tell the agent which one you have in mind, or describe the result and let it pick a sensible model.

Streaming responses

For anything that produces text - chat, generation, summarisation - you almost always want the reply to stream: words appear as the model produces them, instead of the user staring at a spinner for ten seconds. It feels far faster even when the total time is the same.

"Stream the assistant's reply token by token, show a typing indicator while it's thinking, and let the user stop generation partway through."

The agent sets the relay up to forward the provider's stream straight through to the app, so the screen fills in live. Image, audio, and video generation are not text, so they show progress differently - a placeholder while the model runs, then the finished result.

Cost awareness (this is billed separately)

This is the part to be clear-eyed about: AI features in your app bill on the provider's own usage, not on your Appsanic credits. Two completely separate meters:

• Appsanic credits pay for the agent that builds your app. See Credits.
• Your OpenAI / Anthropic / Replicate account pays for every call your finished app makes to the model. That bill goes to you, on your provider account, at the provider's prices.

A few habits keep that second bill sane:

• Pick the right model for the job. A small, fast model is plenty for short replies and classification; save the large reasoning models for tasks that genuinely need them. The agent defaults to sensible choices and you can ask it to change them.
• Set usage limits on the provider's side. OpenAI, Anthropic, and Replicate all let you set a hard monthly spend cap. Do this before real users arrive.
• Cap it in your app too. Ask for per-user rate limits so one user cannot run up your bill: "Limit each user to 20 AI requests an hour and show a friendly message when they hit the cap."
• Watch the long inputs. Summarising a whole book costs far more than summarising a paragraph, because you pay by how much text goes in and comes out. Embeddings are cheap per item but add up across a large library, so generate them once and store them.

Which provider for what

You can connect more than one and mix them in a single app - Claude for the chat assistant, OpenAI for embeddings and transcription, Replicate for image work. Connect whichever you need and ask the agent to use them where they fit.

Under the hood (for engineers)

• Relay, not direct calls. The generated app never holds an OpenAI, Anthropic, or Replicate key. It calls a small server-side relay over HTTPS; the relay holds the key in its environment and forwards to the provider. With Supabase connected, that relay is deployed as a Supabase Edge Function and the key lives in your Supabase function secrets. Without Supabase, the agent writes the relay plus host-it-yourself instructions.
• No key in the bundle. An exported project has no .env file in the app and no inlined AI secret - the only credentials in client code are publishable keys (the Supabase anon key and the other public tokens). The AI secret stays in the relay's environment. See Environment Variables.
• Streaming is forwarded from the provider's server-sent event stream through the relay to the app, so generation appears live rather than arriving in one block.
• Embeddings use the OpenAI embeddings API; vectors are stored in Postgres via Supabase's pgvector extension and queried by similarity. See Database & Data.
• Transcription uploads audio to the relay, which calls Whisper and returns the text; the agent keeps the original file in Storage if you ask it to.
• Replicate runs are kicked off through the relay and polled (or webhooked) until the model finishes, then the result URL comes back to the app. Billing is per GPU-second on your Replicate account.
• Billing is the provider's. Every model call in the finished app is metered and charged by OpenAI, Anthropic, or Replicate, entirely separate from Appsanic credits. Set spend caps on the provider side and per-user rate limits in the app.

Next

Read Search for the full picture of finding things in your app - keyword search, typo tolerance, and how semantic search with embeddings fits alongside it.