44 lines
2.5 KiB
Markdown
44 lines
2.5 KiB
Markdown
# Project Memory – PS_AI_Agent
|
||
|
||
> This file is committed to the repository so it is available on any machine.
|
||
> Claude Code reads it automatically at session start (via the auto-memory system)
|
||
> when the working directory is inside this repo.
|
||
> **Keep it under ~180 lines** – lines beyond 200 are truncated by the system.
|
||
|
||
---
|
||
|
||
## Project Location
|
||
- Repo root: `<repo_root>/` (wherever this is cloned)
|
||
- UE5 project: `<repo_root>/Unreal/PS_AI_Agent/`
|
||
- `.uproject`: `<repo_root>/Unreal/PS_AI_Agent/PS_AI_Agent.uproject`
|
||
- Engine: **Unreal Engine 5.5** — Win64 primary target
|
||
|
||
## Plugins
|
||
| Plugin | Path | Purpose |
|
||
|--------|------|---------|
|
||
| Convai (reference) | `<repo_root>/ConvAI/Convai/` | gRPC + protobuf streaming to Convai API. Has ElevenLabs voice type enum in `ConvaiDefinitions.h`. Used as architectural reference. |
|
||
| **PS_AI_Agent_ElevenLabs** | `<repo_root>/Unreal/PS_AI_Agent/Plugins/PS_AI_Agent_ElevenLabs/` | Our ElevenLabs Conversational AI integration. See `.claude/elevenlabs_plugin.md` for full details. |
|
||
|
||
## User Preferences
|
||
- Plugin naming: `PS_AI_Agent_<Service>` (e.g. `PS_AI_Agent_ElevenLabs`)
|
||
- Save memory frequently during long sessions
|
||
- Goal: ElevenLabs Conversational AI integration — simpler than Convai, no gRPC
|
||
- Full original ask + intent: see `.claude/project_context.md`
|
||
|
||
## Key UE5 Plugin Patterns
|
||
- Settings object: `UCLASS(config=Engine, defaultconfig)` inheriting `UObject`, registered via `ISettingsModule`
|
||
- Module startup: `NewObject<USettings>(..., RF_Standalone)` + `AddToRoot()`
|
||
- WebSocket: `FWebSocketsModule::Get().CreateWebSocket(URL, TEXT(""), Headers)`
|
||
- `WebSockets` is a **module** (Build.cs only) — NOT a plugin, don't put it in `.uplugin`
|
||
- Audio capture: `Audio::FAudioCapture::OpenAudioCaptureStream()` (UE 5.3+, replaces deprecated `OpenCaptureStream`)
|
||
- `AudioCapture` IS a plugin — declare it in `.uplugin` Plugins array
|
||
- Callback type: `FOnAudioCaptureFunction` = `TFunction<void(const void*, int32, int32, int32, double, bool)>`
|
||
- Cast `const void*` to `const float*` inside — device sends float32 interleaved
|
||
- Procedural audio playback: `USoundWaveProcedural` + `OnSoundWaveProceduralUnderflow` delegate
|
||
- Audio capture callbacks arrive on a **background thread** — always marshal to game thread with `AsyncTask(ENamedThreads::GameThread, ...)`
|
||
- Resample mic audio to **16000 Hz mono** before sending to ElevenLabs
|
||
- `TArray::RemoveAt(idx, count, EAllowShrinking::No)` — bool overload deprecated in UE 5.5
|
||
|
||
## Plugin Status
|
||
- **PS_AI_Agent_ElevenLabs**: compiles cleanly on UE 5.5 Win64 (verified 2026-02-19)
|