PS_AI_Agent/.claude/MEMORY.md

Project Memory – PS_AI_Agent

This file is committed to the repository so it is available on any machine. Claude Code reads it automatically at session start (via the auto-memory system) when the working directory is inside this repo. Keep it under ~180 lines – lines beyond 200 are truncated by the system.

---

Project Location

• Repo root: <repo_root>/ (wherever this is cloned)
• UE5 project: <repo_root>/Unreal/PS_AI_Agent/
• .uproject: <repo_root>/Unreal/PS_AI_Agent/PS_AI_Agent.uproject
• Engine: Unreal Engine 5.5 — Win64 primary target

Plugins

Plugin	Path	Purpose
Convai (reference)	<repo_root>/ConvAI/Convai/	gRPC + protobuf streaming to Convai API. Has ElevenLabs voice type enum in ConvaiDefinitions.h. Used as architectural reference.
PS_AI_Agent_ElevenLabs	<repo_root>/Unreal/PS_AI_Agent/Plugins/PS_AI_Agent_ElevenLabs/	Our ElevenLabs Conversational AI integration. See .claude/elevenlabs_plugin.md for full details.

User Preferences

• Plugin naming: PS_AI_Agent_<Service> (e.g. PS_AI_Agent_ElevenLabs)
• Save memory frequently during long sessions
• Goal: ElevenLabs Conversational AI integration — simpler than Convai, no gRPC
• Full original ask + intent: see .claude/project_context.md

Key UE5 Plugin Patterns

• Settings object: UCLASS(config=Engine, defaultconfig) inheriting UObject, registered via ISettingsModule
• Module startup: NewObject<USettings>(..., RF_Standalone) + AddToRoot()
• WebSocket: FWebSocketsModule::Get().CreateWebSocket(URL, TEXT(""), Headers)
  • WebSockets is a module (Build.cs only) — NOT a plugin, don't put it in .uplugin
• Audio capture: Audio::FAudioCapture::OpenAudioCaptureStream() (UE 5.3+, replaces deprecated OpenCaptureStream)
  • AudioCapture IS a plugin — declare it in .uplugin Plugins array
  • Callback type: FOnAudioCaptureFunction = TFunction<void(const void*, int32, int32, int32, double, bool)>
  • Cast const void* to const float* inside — device sends float32 interleaved
• Procedural audio playback: USoundWaveProcedural + OnSoundWaveProceduralUnderflow delegate
• Audio capture callbacks arrive on a background thread — always marshal to game thread with AsyncTask(ENamedThreads::GameThread, ...)
• Resample mic audio to 16000 Hz mono before sending to ElevenLabs
• TArray::RemoveAt(idx, count, EAllowShrinking::No) — bool overload deprecated in UE 5.5

Plugin Status

• PS_AI_Agent_ElevenLabs: compiles cleanly on UE 5.5 Win64 (verified 2026-02-19)