j.foucher
dbd61615a9
- DefaultEngine.ini: set GameDefaultMap + EditorStartupMap to TestMap (API key stripped — set locally via Project Settings, not committed) - Content/TestMap.umap: initial test level - Content/test_AI_Actor.uasset: initial test actor - .claude/MEMORY.md: document API key handling, add memory file index, note private git server and TestMap as default map Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
62 lines
3.7 KiB
Markdown
62 lines
3.7 KiB
Markdown
# Project Memory – PS_AI_Agent
|
||
|
||
> This file is committed to the repository so it is available on any machine.
|
||
> Claude Code reads it automatically at session start (via the auto-memory system)
|
||
> when the working directory is inside this repo.
|
||
> **Keep it under ~180 lines** – lines beyond 200 are truncated by the system.
|
||
|
||
---
|
||
|
||
## Project Location
|
||
- Repo root: `<repo_root>/` (wherever this is cloned)
|
||
- UE5 project: `<repo_root>/Unreal/PS_AI_Agent/`
|
||
- `.uproject`: `<repo_root>/Unreal/PS_AI_Agent/PS_AI_Agent.uproject`
|
||
- Engine: **Unreal Engine 5.5** — Win64 primary target
|
||
- Default test map: `/Game/TestMap.TestMap`
|
||
|
||
## Plugins
|
||
| Plugin | Path | Purpose |
|
||
|--------|------|---------|
|
||
| Convai (reference) | `<repo_root>/ConvAI/Convai/` | gRPC + protobuf streaming to Convai API. Has ElevenLabs voice type enum in `ConvaiDefinitions.h`. Used as architectural reference. |
|
||
| **PS_AI_Agent_ElevenLabs** | `<repo_root>/Unreal/PS_AI_Agent/Plugins/PS_AI_Agent_ElevenLabs/` | Our ElevenLabs Conversational AI integration. See `.claude/elevenlabs_plugin.md` for full details. |
|
||
|
||
## User Preferences
|
||
- Plugin naming: `PS_AI_Agent_<Service>` (e.g. `PS_AI_Agent_ElevenLabs`)
|
||
- Save memory frequently during long sessions
|
||
- Goal: ElevenLabs Conversational AI integration — simpler than Convai, no gRPC
|
||
- Full original ask + intent: see `.claude/project_context.md`
|
||
- Git remote is a **private server** — no public exposure risk
|
||
|
||
## Key UE5 Plugin Patterns
|
||
- Settings object: `UCLASS(config=Engine, defaultconfig)` inheriting `UObject`, registered via `ISettingsModule`
|
||
- Module startup: `NewObject<USettings>(..., RF_Standalone)` + `AddToRoot()`
|
||
- WebSocket: `FWebSocketsModule::Get().CreateWebSocket(URL, TEXT(""), Headers)`
|
||
- `WebSockets` is a **module** (Build.cs only) — NOT a plugin, don't put it in `.uplugin`
|
||
- Audio capture: `Audio::FAudioCapture::OpenAudioCaptureStream()` (UE 5.3+, replaces deprecated `OpenCaptureStream`)
|
||
- `AudioCapture` IS a plugin — declare it in `.uplugin` Plugins array
|
||
- Callback type: `FOnAudioCaptureFunction` = `TFunction<void(const void*, int32, int32, int32, double, bool)>`
|
||
- Cast `const void*` to `const float*` inside — device sends float32 interleaved
|
||
- Procedural audio playback: `USoundWaveProcedural` + `OnSoundWaveProceduralUnderflow` delegate
|
||
- Audio capture callbacks arrive on a **background thread** — always marshal to game thread with `AsyncTask(ENamedThreads::GameThread, ...)`
|
||
- Resample mic audio to **16000 Hz mono** before sending to ElevenLabs
|
||
- `TArray::RemoveAt(idx, count, EAllowShrinking::No)` — bool overload deprecated in UE 5.5
|
||
|
||
## Plugin Status
|
||
- **PS_AI_Agent_ElevenLabs**: compiles cleanly on UE 5.5 Win64 (verified 2026-02-19)
|
||
|
||
## API Keys / Secrets
|
||
- ElevenLabs API key is set in **Project Settings → Plugins → ElevenLabs AI Agent** in the Editor
|
||
- UE saves it to `DefaultEngine.ini` under `[/Script/PS_AI_Agent_ElevenLabs.ElevenLabsSettings]`
|
||
- **The key is stripped from `DefaultEngine.ini` before every commit** — do not commit it
|
||
- Each developer sets the key locally; it does not go in git
|
||
|
||
## Claude Memory Files in This Repo
|
||
| File | Contents |
|
||
|------|----------|
|
||
| `.claude/MEMORY.md` | This file — project structure, patterns, status |
|
||
| `.claude/elevenlabs_plugin.md` | Plugin file map, ElevenLabs WS protocol, design decisions |
|
||
| `.claude/elevenlabs_api_reference.md` | Full ElevenLabs API reference (WS messages, REST, signed URL, Agent ID location) |
|
||
| `.claude/project_context.md` | Original ask, intent, short/long-term goals |
|
||
| `.claude/session_log_2026-02-19.md` | Full session record: steps, commits, technical decisions, next steps |
|
||
| `.claude/PS_AI_Agent_ElevenLabs_Documentation.md` | User-facing Markdown reference doc |
|