Embodied Agent Studio

Loading 3D engine

Ira Idle Feeling: neutral

Embodied Agent StudioBody, brain, voice and stage for an AI agent

The body is what people see. Start with a ready-made body, or load your own avatar. Notice how each one changes what you expect the agent to be.

Built-in

Mascot

You can also drag an avatar file onto the stage. Robot model by Tomás Laulhé (CC0), from the three.js project.

Mascot colours

Outfit

Skin

Hair

Eyes

Body behaviour

Switch behaviours off to compare. Does the agent feel more alive, or more uncanny, with its hands moving?

Idle movementBreathing, small sways and glances. Blinking always stays on. Gestures with emotions and wordsOpen palms when happy, a shrug when concerned, a hand to the chin while thinking, pointing at directions and counting numbers. Talking handsSmall hand movements that follow the rhythm of its voice. Let the brain choose gesturesThe model is told which gestures its body can make, and picks one when a person would.

Greeting gesture

How big the gestures are

Try a gesture

Camera and set

Framing

Drag on the stage to orbit. Scroll or pinch to zoom.

Backdrop

Where to get a body

VRoid Studio (free, Windows and macOS): design an anime-style character and use Export as VRM. Full face, blinking and lip movement. vroid.com/en/studio
VRoid Hub: ready-made VRM avatars by other people. Each one has its own licence; check it allows your use.
MetaPerson Creator: a realistic avatar from a selfie. The first avatar is free. Using a real face raises consent questions: whose face is it, and who agreed?
MakeHuman for Blender (MPFB): free, open-source realistic humans. Its viseme pack adds the mouth shapes this studio uses. Export as .glb.
Your own model in Blender, human or not: add a rig and convert it with the free VRM Add-on for Blender, or export a .glb with shape keys named like ARKit (jawOpen, eyeBlinkLeft).
Free character packs (Quaternius, Kenney, KayKit, 100 Avatars): good bodies, but most have no face shapes, so the mouth will not move. That is a design limit worth discussing.

Design questions for the body

The uncanny valley. In 1970 the roboticist Masahiro Mori proposed that as a robot looks more human, people feel more affinity for it, until it is almost but not quite human. At that point affinity drops into eeriness: the uncanny valley. He noted that movement makes the effect stronger, and advised designers to aim for moderate human likeness, or even a deliberately non-human look, rather than risk the valley.

Mori, M. (1970). Bukimi no tani [The uncanny valley]. Energy, 7(4), 33–35.

Mori, M., MacDorman, K. F., & Kageki, N. (2012). The uncanny valley [From the field]. IEEE Robotics & Automation Magazine, 19(2), 98–100. doi.org/10.1109/MRA.2012.2192811

Try the robot, the mascot and your own avatar. Where does each one sit on Mori's curve? Which feels most trustworthy, and why?
Human, animal, object or abstract? Realistic or stylised? What does each choice promise about how smart the agent is?
Does its look match its job and the people it serves? What does its posture say before it speaks?
If it is based on a real person's face, who agreed to that, and what could go wrong?

The brain is a large language model plus your character sheet. Everything you write below becomes the instructions the model reads before every reply.

Character sheet

Name

Design questions for the name

The name is the first thing people learn about the agent. What does it signal: human or machine, gender, age, region, formality?
Does it suit the culture and language of the people it serves? Is it easy for them to say?
A human name can raise expectations of human understanding. Does your agent live up to that, or would a non-human name set fairer expectations?
In "always listening" mode the agent wakes up when it hears its name. Pick one that speech recognition hears clearly, and that people will not say by accident.

Job: what task does it do that a person used to do?

Who it talks to

Personality

How it speaks

What it knows Paste facts, FAQs or rules. Keep it under about 2 pages. Never paste personal data: free tiers may use what you send to improve their models.

Boundaries: what it refuses, and what it does when unsure

Opening line

Reply language

Model

Where the brain runs

API key

Model Model names change often. If you get a "model not found" error, copy the current free model name from the provider's site.

Rehearsal mode answers from your opening line and the "What it knows" box without any AI. Use it to test the body and voice. Switch to a free key to make it think.

See exactly what the brain receives

The voice has two halves: ears that turn speech into text, and a mouth that turns text into speech. The voice is separate from the brain: the brain writes the words, the voice speaks them.

Ears (speech to text)

Language it listens for

How it listens

When it decides you have finished Turn-taking is a design choice: people expect a reply within about a second, but nobody likes being interrupted.

Mouth (text to speech)

Voice engine

Voice The list depends on your browser and computer.

Speed

Pitch

Speak replies out loud

Design questions for the voice

Does the voice match the body and the name? What accent and pace suit the people it serves?
Should it sound human at all, or clearly synthetic? A very human voice on a robot body can feel uncanny too.
How does it show that it is listening, or thinking? People need to know when the microphone is on.
Press to talk gives people control. Always listening feels more natural, but the microphone never rests. Which is right for your setting?

The stage is where people meet your agent. Save your work, share a demo, put it on a website, or bring it into the room on a phone.

Save your work

Open project

The project file holds the character sheet, voice and listening settings, the camera, and your avatar file. It never includes your API key. Open it on any computer to carry on.

Share a demo

How the demo file works

What is inside: the whole studio, your character sheet, voice settings and avatar, in one web page. It opens straight in presentation view.

What is not inside: your API key. Visitors press Add AI key and paste their own free key, or use the script (rehearsal mode). It also needs an internet connection.

To share it as a link:

Rename the file to index.html.
Create a free public GitHub repository and upload the file. GitHub's website accepts files up to 25 MB.
In the repository, open Settings, then Pages. Choose Deploy from a branch, branch main, folder / (root), and save.
After a minute or two the link appears at the top of the Pages settings. Open it on a phone to test.

Larger than 25 MB? Use Netlify Drop (drag the file onto app.netlify.com/drop), or export your VRoid avatar with smaller textures.

Put it on a website

Host your demo first, then paste this code into any web page. Replace the address with your demo's link. Adding ?embed hides the studio controls.

Phones and AR

Camera view (any phone or laptop): press Camera view on the stage. The agent appears over the live camera, but it does not stick to the floor.

View in AR (Android phones with Chrome): open the hosted link on the phone and press View in AR. Move the phone slowly to find the floor, then tap to place the agent. Tap again to move it. iPhones do not support web AR yet, so use camera view there.

Size in AR

Presenting

Coming next: headsets

The same agent can stand in front of you in a VR or mixed reality headset such as Meta Quest, through the browser's WebXR support. That is the next workshop.

Design questions for the stage

Where will people meet this agent: a laptop, a phone, a kiosk, a headset, a website? Standing or sitting? Alone or in public?
How does that change its size, its voice volume and how long it should talk?
Life size in the room feels like a person. Tabletop feels like a toy or a guide. Which fits your agent's job?