Compare commits

..

69 Commits

Author SHA1 Message Date
2f35c134aa ConvAgent: cut BOTH agents' speech when the player interrupts the dialogue
On a player barge-in, the engaged agent stops (its mic feeds the player's voice to ElevenLabs, which interrupts it), but the witness agent never hears the player and kept finishing its line (e.g. '...I'll give you the cash...') mid-scene. InterruptForPlayer now calls InterruptAgent() on both CompA and CompB. Conversations stay open; the witness still receives the contextual_updates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 15:58:32 +02:00
086311da38 ConvReplay: split agent utterances on silence gaps (fix reply glued to greeting)
A first-message greeting opens the agent segment but never fires OnAgentStoppedSpeaking (no agent_response), so the segment stayed open and the later reply was appended to it — inheriting the greeting's early StartTime and playing before the player's question on replay.

RecordAgentAudio now flushes the open segment and starts a fresh one when audio resumes after a >1.5s gap, so the greeting keeps its time and the reply gets its own (correct, later) timestamp. Also flush open agent segments at record stop. Added record-side diagnostic logs behind ps.convreplay.Debug (agent-START/STOP/SPLIT, player-SEG, user-TEXT).

Record-side fix: existing sidecars keep the old segmentation; re-record to benefit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 10:06:40 +02:00
c783613117 ConvReplay: agent-less player voice recording + audio polish
Record every player's mic into the replay even with no agent in the scene (listen-server/standalone). The bridge arms each player pawn's InteractionComponent via a replicated flag; the host captures locally, remote clients relay via a new ServerRecordVoice RPC, and the server broadcasts OnPlayerVoiceForRecording into the existing player-audio pipeline. Agent and agent-less paths are mutually exclusive (no double-record).

Replay routing: match a recorded agent by id, else by nearest position (runtime-spawned NPCs get unstable names on replay), keeping lip-sync.

Player audio: per-utterance bidirectional AGC toward a target RMS with an envelope peak limiter (instant attack, ~9ms release) instead of hard clipping. Live CVars: ps.convreplay.PlayerTargetRms (default 0.08), ps.convreplay.PlayerGain, ps.convreplay.Debug (per-record logs off by default).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 12:21:19 +02:00
d92e402722 fix emotion poses 2026-07-03 10:56:39 +02:00
231ca4145d ConvReplay: record & replay agent + player conversation audio in demos
New PS_AI_ConvReplay bridge plugin records ElevenLabs agent TTS and player mic audio (Opus) into a sidecar next to the .replay, and plays it back during DemoPlay with lip-sync / body-expression / gaze / emotion. Agent audio is re-fed through the agent's own voice; player audio is spatialized at the recorded position.

ConvAgent hooks (generic): PlayExternalAudioPCM, StopExternalAudio, TriggerReplayedAction/TriggerReplayedClientToolCall, and an OnPlayerAudioCaptured delegate broadcast in ServerSendMicAudioFromPlayer.

Agent matching is id-or-nearest-position (runtime-spawned NPCs get unstable names on replay). Player level normalized per-utterance (bidirectional AGC + soft limiter), tunable live via ps.convreplay.PlayerGain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 10:43:02 +02:00
087c83018c ConvAgent: fire the voice-onset interrupt server-side (dedicated-server support)
The agent-to-agent dialogue interrupt was detected only in FeedExternalAudio,
which runs client-side for a remote player. On a DEDICATED server (no host
player) no one could trigger the reliable VAD interrupt -- everyone fell back to
the slower ElevenLabs transcript path.

Also broadcast OnUserVoiceDetected from ServerSendMicAudioFromPlayer, where the
player's mic (the host's OR a remote client's, relayed) always arrives on the
server. Shares the LastUserVoiceOnset debounce so the host does not double-fire.
The orchestrator still attributes the correct player (ResolveSpeakingPlayer
finds the connected client via NetConnectedPawns).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 17:18:10 +02:00
f49a57f892 ConvAgent: agent-to-agent dialogue orchestrator + player 3-way floor mode
New APS_AI_ConvAgent_AgentDialogue actor drives two ElevenLabs agents in a
text-only conversation (they speak aloud and look at each other), and lets a
player barge in and take over.

Orchestrator (AgentDialogue):
- Single-floor agent<->agent turn-taking over text (no STT); gaze override so
  the agents look at each other; TurnDelaySeconds for pacing.
- Voice-onset interrupt: the player is detected via a local mic VAD
  (OnUserVoiceDetected), not the unreliable STT round-trip. On interrupt both
  agents turn to the player; auto-released once the player leaves both agents.
- 3-way "floor" mode: the engaged agent is reframed ("a real person now
  addresses you") while the witness agent overhears the exchange via
  contextual_update, so it can reference it when the player later turns to it.
  Configurable messages: Interrupt/Bystander context + overheard prefix.

Gaze: honour GazeOverrideTarget on every path that could steal the gaze back to
the player -- OnRep_ActiveSpeaker, InteractionComponent AttachGazeTarget /
SetSelectedAgent, and the PS_AI_Behavior proximity gaze (IsConversationActiveOnPawn
now treats an agent-to-agent dialogue as an active conversation).

Conversation lifecycle: a watching player joining/leaving by look no longer
tears down an orchestrated agent's WebSocket (ServerLeaveConversation guarded by
GazeOverrideTarget) -- fixes agents going silent / lip sync deactivating when
the player looks away.

ElevenLabs: add SendContextualUpdate (contextual_update -- inject context with no
reply triggered) on the component and WebSocket proxy; add OnUserVoiceDetected
local voice-onset event fired from FeedExternalAudio.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 17:07:10 +02:00
6d92ec3c26 ConvAgent: add waveform-splice discontinuity debug detector
Diagnostic for the audio "pops": EnqueueAgentAudio compares the last sample of
the previous enqueued block with the first sample of the new one and logs
[AUDIO-DBG] with the delta + a running count when the jump exceeds a threshold
(gated behind ps.ai.ConvAgent.Debug.ElevenLabs; role-tagged). The host's raw-PCM
stream is continuous so its count stays ~0; a client whose count climbs at
message cadence still has a splice problem. Confirms the carry-over fix
(bcb6c44) at runtime.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 16:45:16 +02:00
bcb6c44f57 ConvAgent: carry Opus sub-frame remainder across messages (fix pops)
The remaining pops were per-MESSAGE, not per-sub-chunk: the Opus encoder
processes whole 640-byte frames only, so each ElevenLabs message dropped its
<1-frame (<20ms) tail. The lost ~10-16ms per message left a waveform
discontinuity at each message splice -> audible click (2-3 per sentence).
Confirmed by log (message sizes not 640-aligned; no underflow) and the
standalone-vs-client asymmetry: both use the same playback queue, but the host
plays raw PCM (no drop) and is clean while clients play the Opus-encoded stream
and pop.

Fix (server-only path): carry the sub-frame remainder to the next message.
Prepend the previous message's leftover before encoding, emit only whole frames,
and stash the new tail. No samples dropped -> continuous stream -> no splice
click. Leftover reset per turn (new-turn guard + StopAgentAudio).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 16:41:25 +02:00
cf83a2302d ConvAgent: reassemble agent audio sub-chunks into one enqueue (fix pops)
After sub-chunking each ElevenLabs message into N ~0.8s Opus packets, the
client enqueued each sub-chunk separately, re-running the playback state
machine per sub-chunk (spurious Play() restart / pre-buffer race / underrun
at the seam) -> an audible pop at each ~0.8s boundary (2-3 per sentence).
Confirmed via multi-agent analysis + logs (no generation skips, no decode
truncation -> not a codec issue; the discontinuity is on the enqueue side).

- MulticastReceiveAgentAudio gains a bLastSubChunk flag; the server marks the
  final sub-chunk of each message (encode path frame-aligned so the last
  sub-chunk is well-defined; raw-PCM fallback likewise).
- The client accumulates decoded PCM across a message's sub-chunks and calls
  EnqueueAgentAudio (and lip-sync broadcast) ONCE, on the last sub-chunk.
- ClientReassemblyPCM reset in StopAgentAudio so a half-received message can't
  leak across a turn/interruption.

No added latency: a message's sub-chunks are sent back-to-back in one server
tick and arrive together. Each RPC still <=40 frames (Reliable, under the
encode/decode limits).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 16:23:46 +02:00
5b3a11a733 ConvAgent: agent audio multicast back to Reliable (fix Opus crackle/gaps)
Sending Opus over Unreliable multicast caused missing words and crackling:
Opus is a stateful inter-frame codec, so a dropped or reordered packet
desyncs the decoder and corrupts all following frames until the stream
resets. Unreliable was originally adopted only to stop a reliable-buffer
overflow — but that overflow came from 32-152KB raw-PCM RPCs (pre-Opus).
With Opus + sub-chunking each RPC is ~2KB (2-3 partial bunches) at
~1.25 RPC/s, so the 512-bunch reliable buffer stays in single digits and
cannot overflow. Reliable restores ordered, lossless delivery.

- MulticastReceiveAgentAudio: Unreliable -> Reliable.
- Keep the 40-frame (0.8s, ~2KB) sub-chunking — it is what keeps packets
  small enough for Reliable to be safe.
- Harden the raw-PCM fallback (only reached if Opus init fails): hard-cap
  one message at 320KB and drop the tail so it can never overflow the
  reliable buffer, and warn on every fallback (config bug: [Voice]).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 14:31:02 +02:00
2ca5643f32 ConvAgent: fix agent voice truncation on remote clients (Opus)
Long ElevenLabs audio messages were cut off on remote clients:
- the client Opus decode buffer was only 32000 bytes (~0.88s), so the
  engine decoder aborted with "Decompression buffer too small" and
  dropped the tail of any packet longer than that;
- the server also encoded a whole message as a single Opus packet, which
  the engine encoder caps at 76 frames (~1.52s), silently dropping the rest.
The host/authority plays raw PCM and never decodes, so standalone and
listen-server hosts were unaffected, which masked the bug.

Fix (HandleAudioReceived + MulticastReceiveAgentAudio_Implementation):
- sub-chunk the Opus encode into <=40-frame (25600-byte, 0.8s) blocks,
  one RPC each: stays under both the 76-frame encode cap and the client
  decode buffer, and keeps each Unreliable RPC small;
- raise the client decode buffer 32000 -> 64*1024 as headroom.

Empirically confirmed: networked client logged 16x "Decompression buffer
too small" with Opus active; standalone (Opus equally active) logged zero.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 14:11:28 +02:00
4854a7fa9a ConvAgent: send agent audio multicast Unreliable + add size debug log
- MulticastReceiveAgentAudio: Reliable -> Unreliable. Realtime audio: a lost or
  oversized packet is dropped (minor glitch) instead of backing up the reliable
  buffer and dropping the client connection.
- Raw-PCM fallback chunk size 32000 -> 4000 bytes so an unreliable RPC is not
  auto-promoted to reliable when fragmented into many partial bunches (which
  would reintroduce the overflow).
- Add net-audio size log gated behind ps.ai.ConvAgent.Debug.ElevenLabs; fires on
  Authority even in solo PIE: "[NET-SRV] Opus audio chunk: N bytes (raw M -> Xx)".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 19:52:48 +02:00
41900991ef ConvAgent: enable Voice module so the Opus codec initializes
The project had no [Voice] section, so the engine default bEnabled=false left
FVoiceModule::CreateVoiceEncoder() returning null (Encoder/Decoder NULL).
Networked agent audio then fell back to raw PCM in large reliable multicast
RPCs, overflowing the reliable buffer and dropping VR client connections the
moment the avatar spoke. Enabling Voice lets Opus compress (~13-20x), keeping
RPCs small.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 19:52:48 +02:00
efd9921fde ElevenLabs: guard null gaze target on late join (replicated ref not resolved)
A client that joins while a conversation is already active can run OnRep with
NetConnectedPawns/NetActiveSpeakerPawn entries whose replicated pawn references
haven't resolved yet (actor channel not open). NetConnectedPawns.Num() > 0 did
not guarantee a non-null element, so OnRep_ConversationState dereferenced a null
pawn in its log (crash in Dev/PIE) and ApplyConversationGaze set TargetActor to
null (gaze glitch). Both now skip until the reference resolves — OnRep fires
again once it's patched in. More likely with 3+ clients.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 18:16:35 +02:00
fee7d0c679 Gaze: stateless head/eyes so client matches server (fix ~60° head offset)
Single-player clients sometimes showed the head turned ~60° off while the
server looked correct. Two client-only causes:

1. The client overwrote the actor's rotation every frame (SetActorRotation)
   and then read back its own output as the "replicated" yaw, so SmoothedBodyYaw
   diverged from the true server body yaw.
2. The head/eye cascade was sticky / path-dependent (TargetHeadYaw updated on
   overflow events), so running it independently on a client fed by lagged,
   stepped replication latched into a different state — the head overflowed to
   MaxHeadYaw and stayed there.

Rewrite head/eyes as a STATELESS function of the actual (authoritative on the
server, replicated on the client) body facing + target direction. No sticky
state, FInterpTo only eases the visual. The client no longer overrides the actor
rotation. Body turn stays authority-only (lazy turn beyond head+eyes range).
Server and client now converge to the same pose by construction.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 18:13:53 +02:00
777ffa09a8 ElevenLabs: fix multi-player speaker arbitration with open mic
With an open mic streaming continuously (Server VAD + interruption), every
chunk refreshed the active speaker's LastSpeakTime, so SpeakerSwitchHysteresis
never elapsed and SpeakerIdleTimeout never fired — the first speaker locked the
floor forever and the agent never turned to anyone else.

Now only chunks with real voice energy (RMS >= 0.02) hold/take the floor: the
active speaker is still forwarded continuously (silence tail for VAD), but their
lock only persists while they actually speak. When they pause, another player's
voiced audio can switch in and the agent turns to them (NetActiveSpeakerPawn
replicated -> gaze). Validated with two players on one NPC.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 18:13:44 +02:00
ab2482a209 ElevenLabs: remove temporary [MIC-DBG] mic uplink diagnostics
Client mic uplink confirmed working in a packaged build across two PCs
(networked Server VAD): the agent now replies to the player's voice.
Remove the [MIC-DBG]/[MIC-DBG-SRV] instrumentation added to trace it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 14:27:39 +02:00
c2ca2e3dbb ElevenLabs: fix client mic uplink in networked Server VAD mode
Networked clients could hear the agent but the agent never replied to them
(worked in standalone/listen-server host). Root causes, client-only paths
that never run on the authority:

1. Deferred StartListening was never retried once the conversation went live.
   OnRep_ConversationState now (re)starts listening when bNetIsConversing
   replicates and the local player is in the conversation — closes the
   RPC/replication ordering race so bIsListening reliably opens the uplink.

2. The client stripped ALL silent mic chunks to save bandwidth, but ElevenLabs
   Server VAD needs the trailing silence to detect end-of-turn — so the agent
   heard the words but never the pause and never responded. Replaced the hard
   silence drop with a silence "tail": keep streaming for MicSilenceTailSeconds
   (default 2s, tunable) after the last voiced chunk, then drop during idle.
   The authority path is unchanged (always streams the full mic).

Also: debug HUD now shows "CONNECTED (via server)" on clients instead of a
misleading "DISCONNECTED" (clients have no local WebSocket; IsConnected() is
always false there).

[MIC-DBG] temporary diagnostics are kept in this commit to validate the fix on
another machine; they will be removed in a follow-up once confirmed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 14:07:30 +02:00
8ae73bbacb compil bug fix 2026-06-24 16:58:53 +02:00
6a8c22c77e ElevenLabs: editor-only API key for agent authoring (no leak in builds)
Re-enables agent/tool/action-set authoring in the editor after the API
key was removed from Project Settings. Adds an editor-only key source so
it can never be committed-as-config or baked into a packaged build.

- New UPS_AI_ConvAgent_EditorSettings_ElevenLabs (UDeveloperSettings,
  config=EditorPerProjectUserSettings, Editor category): single global
  field, stored under Saved/Config (never cooked), in the editor module
  (never packaged).
- ResolveEditorApiKey(): editor setting -> ELEVENLABS_API_KEY env -> runtime
  SaveGame fallback. All three editor customizations route through it.
- Fix the now-stale "API Key not set in Project Settings" messages.

Runtime path is untouched: packaged game still uses the client-provided
key via the SaveGame (Set/GetElevenLabsAPIKey).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 15:36:47 +02:00
7d99a89ac7 android 2026-06-12 08:09:24 +02:00
a5b0d5db84 PS_AI_Behavior: freeze movement during conversation + manual lever
Auto pause now triggers as soon as a conversation is genuinely active (agent greeting included) instead of waiting for the user to speak — the NPC no longer walks away during its opening line.

Add DisableBehaviorMovement()/EnableBehaviorMovement() (BlueprintCallable): freeze the NPC's autonomous movement while keeping the Behavior Tree running (perception, threat, gaze, reactions). Movement is paused when EITHER manually disabled OR a conversation is active. Both drive the ConversationPaused Blackboard flag read by the movement-branch decorators; leave the Flee branch un-decorated so fleeing still works.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 09:41:06 +02:00
cfade2a265 PS_AI_Behavior: make spline StopFollowing a firm, BT-respected stop
StopFollowing() now sets a bManuallyStopped flag (cleared only by an explicit StartFollowing/StartFollowingAtDistance). The FindAndFollowSpline and FollowSpline BT tasks bail while it is set, so the BT no longer auto-resumes a manual stop on the next tick. ResumeFollowing() also respects the flag. PauseFollowing() is unchanged (BT may resume).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 09:40:56 +02:00
27d0906d21 ElevenLabs: secure private agents (signed URL) + encrypt key at rest
Builds on the client-provided-key SaveGame work to actually secure usage:

- Private agents: connect via a short-lived Signed URL fetched with the stored key (GET /v1/convai/conversation/get-signed-url), instead of a direct agent_id+header connection which ElevenLabs rejects for auth-enabled agents. Falls back to a direct connection only when no key is set (public agents). Enforces 'valid key required': no key -> no signed URL -> no conversation.

- Force agent creation/update to private (platform_settings.auth.enable_auth=true) so a public, key-less agent can't be created by mistake.

- Encrypt the API key at rest in the SaveGame (AES-256); plaintext kept only in the in-memory cache.

- Redact signed-URL query (auth token) from logs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 17:00:05 +02:00
89d555b734 ElevenLabs: client-provided API key via SaveGame
The API key is no longer stored in project settings / DefaultEngine.ini (it was committed and baked into shipped builds). It is now provided per-install by the client at runtime and persisted in a SaveGame slot.

- Add UPS_AI_ConvAgent_SaveGame_ElevenLabs (slot PS_AI_ConvAgent_ElevenLabs)

- BlueprintLibrary is the single owner of the key: Set/Get/Has/Clear + in-memory cache backed by the SaveGame

- Add async BP node 'Test ElevenLabs API Key' (GET /v1/models)

- Remove API_Key UPROPERTY from UPS_AI_ConvAgent_Settings_ElevenLabs and the key logic from the module

- Route all consumers (WebSocket auth, region probe, editor sync customizations) through GetElevenLabsAPIKey()

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-29 14:27:29 +02:00
7430a18e66 change for UE 5.7 2026-05-28 18:05:06 +02:00
ceaf89f836 bug fix pour 5.7 2026-05-06 17:52:08 +02:00
a1e34e6dfa Decouple ForceDisableConversation from action flow + mic timing fix
- ElevenLabsComponent::ForceDisableConversation() — removed ActionName param. Now a pure conversation master-switch with no knowledge of actions. OnReadyForAction is fired by the action flow itself: PendingActionName is set when OnAgentActionRequested is broadcasted, then OnReadyForAction fires either after blend-out (if conversation was disabled) or on the next tick (if not).
- Rename bConversationDisabledByAction → bConversationForceDisabled — the old name implied coupling with actions which no longer exists.
- Add bPendingStartListening — when StartListening is called before the WebSocket is connected (race between StartConversation and StartListening from InteractionComponent), defer instead of silently failing. Consumed automatically in HandleConnected / ClientStartConversation. Cleared by StopListening / EndConversation.
- PerceptionComponent — demote the "perceived N actors but ALL filtered out" warning to Verbose; it's a normal case for civilians surrounded by non-hostile actors, not a misconfiguration.
- CLAUDE.md — workflow rule: always describe and request confirmation before code changes.
- Martin.uasset — agent config tweaks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-27 12:37:18 +02:00
f338c0586e Add ResetBehavior() and de-escalation on hostile flip
- AIController::ResetBehavior() — immediate reset to initial-tick state: stops movement, clears all BB threat/cover/patrol keys, recomputes TeamId, forces state to Idle (triggers speed update via HandleStateChanged). For civilian calm-down, enemy surrender, mission reset.
- PersonalityComponent::ResetRuntimeState() — clears PerceivedThreatLevel and combat/cover cycle internals. Traits preserved.
- PersonalityComponent::EvaluateReaction() — disguised Enemy (hostile=false) never returns Combat regardless of threat level. Prevents stuck-in-combat bug when hostility flips mid-fight.
- BTService_EvaluateReaction — when hostile flips true→false, soft de-escalation: clears threat so next ApplyReaction transitions to Idle naturally (no abrupt movement stop).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 20:20:22 +02:00
48d1543329 Reorganize Blueprint categories under ASTERION namespace, move bAutoStartBehavior to PersonalityComponent, add user documentation
- Bulk rename Category/ClassGroup strings to ASTERION|PS_AI_ConvAgent|... and ASTERION|PS_AI_Behavior|... (284 occurrences, 24 files) so functions/properties/components group cleanly in Blueprint menus
- Move bAutoStartBehavior from AIController to PersonalityComponent so the same AIController class can drive multiple NPC archetypes with per-instance scripted-mode override
- Add PowerPoint user documentation for both plugins in Docs/ with generator scripts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 16:30:48 +02:00
15871be9e9 Remove plugin binaries from tracking, add Plugins/*/Binaries/ to gitignore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 12:43:56 +02:00
6fcbdb2005 Merge branch 'main' of https://git.polymorph.fr/j.foucher/PS_AI_Agent 2026-04-09 12:40:00 +02:00
e322a400be bin 2026-04-09 12:39:36 +02:00
fda2e4e32f Add .gitattributes to normalize line endings across machines
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 12:42:52 +02:00
e32c53ffd6 Remove Live Coding patch files from tracking, add *.patch_* to gitignore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 12:39:41 +02:00
66171cc0bd Add ElevenLabs Tool system, update ConvAgent binaries and cleanup patches
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 19:27:58 +02:00
c1afddc8b7 Add Scripted state and bAutoStartBehavior for external NPC control
Allow NPCs to start without a Behavior Tree and be controlled externally
via Blueprint, Level Sequences, or triggers. Adds Scripted enum state that
prevents BT services from overriding state, plus StartBehavior/StopBehavior
BlueprintCallable functions to toggle BT execution at runtime.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 19:26:26 +02:00
a31ac1d782 refactor 2026-04-03 08:19:49 +02:00
661e8c66ac Add CVar debug system: ps.ai.Behavior.Debug controls all diagnostic logs
- New CVar ps.ai.Behavior.Debug: -1=use bDebug property, 0=force off, 1=force on
- IsDebugEnabled() static helper checks CVar then PersonalityComponent.bDebug
- All diagnostic logs gated by IsDebugEnabled (no spam when debug off)
- Logs use Log level (visible) when enabled, silent when disabled
- Applied to: UpdateThreat, FindAndFollowSpline, FindCover, PerceptionComponent,
  PersonalityComponent EvaluateReaction

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:44:32 +02:00
b473fcda22 Defer BodyExpression activation to first agent speech, demote spammy logs to Verbose
- BodyExpressionComponent: don't activate on OnConversationConnected, wait for
  OnSpeakingStarted (prevents body anims while NPC still walking)
- Demote recurring logs to Verbose: UpdateThreat per-tick, FindAndFollowSpline
  debug, FindCover distance check, perception target scoring

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:38:41 +02:00
c32aba9902 Add local VAD and conversation-driven NPC pause
MicrophoneCaptureComponent:
- Local Voice Activity Detection (RMS-based, independent of ElevenLabs)
- Configurable threshold, onset time, silence time
- bIsUserSpeaking flag + OnUserVoiceActivityChanged delegate
- Hysteresis prevents flickering between speech/silence

AIController gaze bridge:
- Resolve MicComponent from player Pawn (not NPC) via reflection
- ConversationPaused BB key blocks movement branches via BT decorator
- NPC stops only when user actually speaks (not just on proximity connect)
- NPC resumes when conversation disconnects
- Spline PauseFollowing/ResumeFollowing on conversation start/end

BT setup required:
- Add Blackboard Condition (ConversationPaused Is Not Set, Aborts=Both)
  on spline and patrol branches

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 16:22:21 +02:00
90bee03b44 Gaze bridge cleanup: revert premature conversation pause, clean up diag logs
- Revert conversation movement pause (needs local VAD, not bNetIsConversing)
- Remove temporary gaze proximity debug log
- Remove bConversationPaused flag (will be reimplemented with VAD)
- TODO: pause NPC movement only when local voice activity is detected

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 12:50:38 +02:00
d9fb46bc96 Add gaze bridge: behavior-driven look-at via ConvAgent GazeComponent
New BTService_UpdateGaze bridges PS_AI_Behavior to PS_AI_ConvAgent's
GazeComponent via runtime reflection (zero compile dependency).

Priority system:
- Combat/TakingCover: gaze disabled (aim animation handles it)
- Alerted: look at ThreatActor (head + eyes, no body)
- Conversation: skip (ConvAgent manages)
- Proximity: glance at nearest perceived actor within radius

Proximity gaze features:
- Lock on target until release (no jumping)
- Configurable duration, cooldown, radius on AIController
- Front-facing only (dot product filter)
- Skip spectators and hostile actors

GazeComponent fix:
- Sync SmoothedBodyYaw to actual actor rotation when body tracking
  is disabled, preventing stale head offset during spline movement

Files:
- New: BTService_UpdateGaze.h/.cpp
- Modified: AIController.h/.cpp (gaze bridge, config, bind, cleanup)
- Modified: GazeComponent.cpp (body yaw sync fix)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 12:16:33 +02:00
d471714fbd Fix rival faction detection: include Faction in fallback TeamId resolution
- GetTeamAttitudeTowards fallback (interface path) was calling MakeTeamId
  without Faction, so enemies of different factions had identical TeamIds
  → always Friendly instead of Hostile
- Now reads Faction from PersonalityProfile when resolving via interface

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 18:14:36 +02:00
af93194f48 Multiple fixes: threat decay per profile, state hysteresis, spline pathfinding, attack readiness
- Move ThreatDecayRate from global Settings to PersonalityProfile (per-archetype)
- Add state hysteresis in EvaluateReaction to prevent Fleeing/Combat flickering
- FindCover: verify distance on arrival, retry movement if too far
- FindAndFollowSpline: sample multiple spline points when closest is blocked
- BTTask_Attack: call BehaviorStartAttack immediately (draw weapon before LOS/range)
- CoverShootCycle: call BehaviorStartAttack on entry (weapon ready during cover approach)
- FindOwningPawn: walk AttachParentActor chain for ChildActor weapons without Owner
- Initialize PreferCover BB key to false at setup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 17:42:05 +02:00
f5897b46cc Remove unused Loyalty/Discipline traits, lower Aggressivity combat threshold to 0
- Remove Loyalty and Discipline from EPS_AI_Behavior_TraitAxis enum (never used in gameplay)
- Clean up PersonalityProfile and PersonalityComponent default trait initialization
- Add descriptive tooltips to remaining traits (Courage, Aggressivity, Caution)
- Lower Aggressivity combat gate from 0.3 to 0.0 (only Aggressivity=0 prevents combat)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 07:16:19 +02:00
de3a5310f4 Update tooltips and comments: fix outdated references, clarify personality modulation
- BTTask_Attack.h: remove reference to non-existent GetBehaviorOptimalAttackRange, document PersonalityProfile ranges
- AIController.h: update TeamId comment to reflect actual MakeTeamId encoding (nibble-based)
- CoverShootCycle.h: clarify that Peek/Cover durations are base values modulated by personality traits
- FindCover.h: clarify ManualPointBonus is additive score
- CombatComponent.h: clarify AttackRange/AttackCooldown are for ExecuteAttack, not BTTask_Attack

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 07:04:59 +02:00
69b9844a4b bin 2026-03-31 20:33:30 +02:00
78149fffcd Add personality-driven Combat/Cover cycle with BB PreferCover key
- Add CombatCoverCycleDuration to PersonalityProfile (configurable base duration)
- PersonalityComponent cycles bPreferCover based on Aggressivity/Caution ratio
  - Combat duration = CycleDuration × Aggressivity/(Aggressivity+Caution)
  - Cover duration = CycleDuration × Caution/(Aggressivity+Caution)
  - Min 2s per phase, ±20% jitter
- Write PreferCover bool to Blackboard for BT decorator observer aborts
- IsCoverNeeded decorator checks both target type AND ShouldPreferCover()
- Remove TakingCover state from EvaluateReaction — cover is now a sub-mode of Combat
- BT uses Blackboard Condition on PreferCover with Observer Aborts=Both

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 19:26:37 +02:00
b60086d107 Add server authority checks for networking safety
- PerceptionComponent: omniscient TActorIterator only runs with HasAuthority()
- PersonalityComponent: ApplyReaction and ForceState gate replicated CurrentState writes to server only

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 18:24:55 +02:00
d6325b373d Fix IsCoverNeeded decorator: resolve AimTargetActor to owning Pawn
- IsCoverNeeded used Cast<APawn> on ThreatActor which failed for AimTargetActors
  → always returned true (assume dangerous) → enemies took cover against civilians
- Fix: use FindOwningPawn to walk Owner/Instigator chain to the actual Pawn
- Revert inline civilian check in EvaluateReaction (decorator handles it in BT)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 18:20:04 +02:00
011bfcf62a Add uncrouch on state change and flee: civilians stand up when leaving cover
- AIController: auto uncrouch when leaving Fleeing/TakingCover state
- FleeFrom: uncrouch at start of flee (civilian was still crouched from HidingSpot)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 17:59:22 +02:00
391a35ac2c bin 2026-03-31 16:33:57 +02:00
44f7e860aa Cover system Phase 8: EQS filter fix, LOS lateral spread, omniscient threat awareness, debug improvements
- Fix EQS SetScore filter bug: pass FloatValueMin/Max instead of hardcoded 0/1 (filters were ignored)
- Add LateralSpread to LineOfSight EQS test: multiple traces for wider LOS check
- Add bDrawDebug to CoverQuality and LineOfSight EQS tests
- Ignore ThreatActor collision in EQS traces (AimTargetActor was blocking its own LOS)
- Add navmesh projection in EQSContext_CoverLocation for cover points inside geometry
- Add omniscient awareness: enemies detect top-priority targets (Protectors) within sight radius without perception cone
- Suppress target switching during TakingCover state to prevent cover invalidation
- Fix flanking check: trace at chest height instead of feet
- Add debug visualization for EQS fallback paths (NO REFINE, NO FIRE POS, IN PLACE)
- Clean up diagnostic logs: verbose for per-trace EQS details, log level for summaries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 16:30:40 +02:00
86d3ae118d Cover system Phase 7: flanking detection, EQS tuning, target persistence, MoveTimeout safety
- CoverShootCycle: SetFocus only during AtCover (not shooting), flanking detection every 0.5s abandons compromised cover
- CoverQuality EQS: ground hit filter, MaxNearbyHitDist (300cm), lateral spread traces, capsule-relative heights
- EQSContext_CoverLocation: prefer CoverPoint actor location over refined vector
- UpdateThreat: never abandon target during Combat/TakingCover states
- MoveTimeout (5s) on all movement states to prevent stuck NPCs
- FindCover: detailed refinement result logging
- Debug: Peeking timer log for investigating timer-stops-counting bug (needs rebuild + test)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 21:12:39 +02:00
9aaf16c655 WIP: Cover system firing position, IsCoverNeeded decorator, EQS contexts
- CoverShootCycle: add FiringPositionQuery EQS for peek/shoot positions
  (NPC moves between cover position and firing position with LOS)
- Add BTDecorator_IsCoverNeeded: skip cover when target is Civilian
- Add EQSContext_CoverLocation: provides BB CoverLocation to EQS generators
- FindCover: add debug draw toggle and EQS refinement debug spheres
- Definitions: add ECoverShootSubState and CoverPointType::HidingSpot

NOTE: Has compilation errors to fix (signature mismatches in
CoverShootCycle StartPeeking/ReturnToCover, missing forward-declare)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 17:53:52 +02:00
98f0dbdce5 Add cover system LOS checks, crouch interface, and EQS refinement
- SetBehaviorCrouch() interface function for cover/hiding crouch control
- CoverShootCycle: continuous LOS check during Peeking (stop if target hides)
- CoverShootCycle: crouch/stand transitions at all cover state changes
- CoverShootCycle: fail when no LOS and no advancing cover (falls to Attack)
- FindCover: crouch on arrival, stand up on abort
- FindCover: optional EQS RefinementQuery to refine exact position around CoverPoints

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 07:47:11 +02:00
25abd59512 Fix EQS firing position: invalid BB vector check, AlreadyAtGoal fallback
- LastKnownTargetPosition check now filters FLT_MAX/InvalidLocation (not just zero)
- EQS result with AlreadyAtGoal triggers fallback advance instead of no-op
- bProjectGoal enabled for EQS move to handle NavMesh projection

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 11:14:04 +02:00
59a23e61db Add EQS LineOfSight test, LOS combat plan, and updated binaries
- EQSTest_LineOfSight: new EQS test for firing position queries
- Updated plugin binaries (PS_AI_Behavior + PS_AI_ConvAgent)
- TeamComponent, Settings, Build.cs updates from LOS plan implementation
- PLAN.md updated with LOS combat implementation progress

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 11:07:21 +01:00
65b86e2fbd Fix combat BT stuck on spline, LOS through target capsule, NoTeam targeting
- Attack/CoverShootCycle: de-escalate to Alerted on failure so decorator
  can re-trigger when threat returns (fixes BT stuck on spline branch)
- UpdateThreat: keep BB ThreatActor during brief perception gaps instead
  of clearing immediately (use LOS timeout for graceful degradation)
- HasLineOfSight: ignore actors up the attachment chain so trace doesn't
  hit the target Pawn's capsule when aiming at its AimTarget child actor
- NoTeam actors (spectators, editor pawns) treated as Neutral instead of
  Hostile, plus SpectatorPawn explicit filter in perception
- BB debug key ThreatPawnName shows owning Pawn name (resolved via
  perception's LastThreatOwningPawn) instead of cryptic AimTarget name
- FindOwningPawn promoted to public static on PerceptionComponent

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-28 11:03:27 +01:00
2588883a1c Fix perception to separate Pawn (team checks) from ThreatTarget (aiming)
Split ResolveToPawn into FindOwningPawn + GetThreatTarget so non-Pawn
actors (PS_AimTargetActor) are properly resolved for team/attitude checks
while still being used as BB target. Add attack range hysteresis (10%
buffer), target persistence (80% threshold), melee no-cooldown chase,
ranged midpoint approach. New files: CoverShootCycle task, CheckCombatType
decorator, MinAttackRange/MaxAttackRange in PersonalityProfile.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:20:42 +01:00
e7c3598dce Fix combat cycle, spline resume, team identity, and debug system
- Add TeamComponent for player pawn team identity (Role-based: Civilian/Enemy/Protector)
- Add IsTargetActorValid to interface for dead target filtering
- Fix GetTeamAttitudeTowards to check IGenericTeamAgentInterface + TeamComponent
- Guarantee BehaviorStopAttack via OnTaskFinished (all exit paths)
- Prevent Combat state without ThreatActor (stay Alerted until perception catches up)
- Resume spline at closest point from current position after combat
- Sync CurrentSpline and SplineProgress to Blackboard in FollowSpline tick
- Auto-detect Patrol state when NPC has a spline (fixes Idle speed=0 blocking movement)
- Add per-component debug toggles (Personality + SplineFollower independent)
- Use AddMovementInput instead of RequestDirectMove for reliable post-combat movement
- Add bTickInEditor for Personality debug in Simulate mode

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 12:43:08 +01:00
0c4c409ebc add binaries 2026-03-27 08:11:03 +01:00
137aeb5953 Update CLAUDE.md with PS_AI_Behavior architecture and testing status
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 20:04:55 +01:00
eb5113ea30 Fix TeamId auto-assignment and interface default implementations
- Always recalculate TeamId at OnPossess (BP CDOs may reset to 0)
- Remove duplicate _Implementation definitions (UHT auto-generates them)
- Fixes crash when interface functions not overridden in BP

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 20:03:14 +01:00
1799ba28c0 Delegate combat to Pawn via IPS_AI_Behavior interface
- Add BehaviorStartAttack/BehaviorStopAttack to IPS_AI_Behavior_Interface
- Attack task now calls interface instead of CombatComponent directly
- Task stays InProgress permanently, Decorator Observer Aborts handles exit
- Remove CombatComponent dependency from Attack task
- Pawn handles actual aiming/shooting via its own systems

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:42:30 +01:00
9d054cc46f Fix BT state transitions, attack persistence, and hostile team switching
BT Attack task:
- Stay InProgress permanently instead of returning Succeeded after each attack
- Stay InProgress on MoveToActor failure instead of Failed (retry next tick)
- Add verbose logging for attack state (target, range, distance)

BT EvaluateReaction service:
- Auto-detect hostility changes via IPS_AI_Behavior interface
- Dynamically update TeamId when IsBehaviorHostile() changes (infiltrator reveal)

AIController:
- Remove GetBehaviorTeamId from interface (TeamId is now 100% automatic)
- TeamId derived from NPCType + hostile state, no user implementation needed
- Add BB State change logging for debug
- Use SetValueAsEnum consistently for BehaviorState key

Interface:
- Remove GetBehaviorTeamId — TeamId is computed by the plugin automatically

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 18:02:48 +01:00
2e04cb0334 Fix perception, patrol, and spline following for PS_AI_Behavior
Perception:
- Configure senses in BeginPlay + RequestStimuliListenerUpdate (not constructor)
- Filter threats by team attitude: only Hostile actors generate threat
- Skip Friendly and Neutral actors in CalculateThreatLevel and GetHighestThreatActor
- All Hostile actors are valid threats regardless of TargetPriority list

Blackboard:
- Use SetValueAsEnum/GetValueAsEnum for BehaviorState key (was Int)

Patrol:
- Auto NavMesh patrol when no manual waypoints defined
- Project HomeLocation onto NavMesh before searching random points
- Fallback to NPC current position if HomeLocation not on NavMesh

Spline following:
- Choose direction based on NPC forward vector vs spline tangent (not longest distance)
- Skip re-search if already following a spline (prevent BT re-boucle)
- Reverse at end: wait for NPC to catch up before inverting direction
- Use NPC actual CharacterMovement speed for target point advancement
- Face toward target point (not spline tangent) to prevent crab-walking in turns
- Junction switching: direction continuity check, 70% switch chance, world gap tolerance
- Debug draw: green sphere (target), yellow line (gap), cyan arrow (tangent), text overlay
- Add GetWorldDirectionAtDistance to SplinePath

Editor:
- Add Placeable to SplinePath and CoverPoint UCLASS
- Add PersonalityProfileFactory for Data Asset creation
- Add EnsureBlackboardAsset declaration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 15:44:39 +01:00
133 changed files with 13046 additions and 1572 deletions

View File

@@ -1,6 +1,23 @@
# PS_AI_ConvAgent — Notes pour Claude # PS_AI_Agent — Notes pour Claude
## Naming Convention ## Workflow règles
**TOUJOURS demander avant de lancer un changement de code.** Avant d'appeler Edit/Write/sed sur un fichier source :
1. Expliquer ce qu'on s'apprête à faire (quel fichier, quelle logique, quel impact)
2. Attendre la confirmation explicite de l'utilisateur
3. Seulement après → effectuer le changement
Cela s'applique à : modifications de code (.cpp/.h), refactors, renames bulk, ajouts de nouvelles fonctions.
Cela ne s'applique PAS à : lecture de fichiers, recherches, git status/log/diff, builds, questions diagnostiques.
Exceptions où on peut agir directement :
- L'utilisateur dit explicitement "fais-le" / "go" / "lance" / "commit"
- L'utilisateur demande explicitement un changement précis ("renomme X en Y", "ajoute cette fonction")
- Annulation / revert demandé par l'utilisateur
## Plugins dans ce repo
### PS_AI_ConvAgent
- **Module**: `PS_AI_ConvAgent` / `PS_AI_ConvAgentEditor` - **Module**: `PS_AI_ConvAgent` / `PS_AI_ConvAgentEditor`
- **API macro**: `PS_AI_CONVAGENT_API` - **API macro**: `PS_AI_CONVAGENT_API`
- **Class prefix**: `PS_AI_ConvAgent_` (e.g. `UPS_AI_ConvAgent_PostureComponent`) - **Class prefix**: `PS_AI_ConvAgent_` (e.g. `UPS_AI_ConvAgent_PostureComponent`)
@@ -8,29 +25,71 @@
- **UI display**: `"PS AI ConvAgent"` (categories, DisplayName) - **UI display**: `"PS AI ConvAgent"` (categories, DisplayName)
- **CoreRedirects**: DefaultEngine.ini handles both ElevenLabs* and PS_AI_Agent_* → PS_AI_ConvAgent_* - **CoreRedirects**: DefaultEngine.ini handles both ElevenLabs* and PS_AI_Agent_* → PS_AI_ConvAgent_*
### PS_AI_Behavior
- **Module**: `PS_AI_Behavior` / `PS_AI_BehaviorEditor`
- **API macro**: `PS_AI_BEHAVIOR_API`
- **Class prefix**: `PS_AI_Behavior_` (e.g. `UPS_AI_Behavior_PersonalityComponent`)
- **UI display**: `"PS AI Behavior"` (categories, DisplayName)
- **Interface**: `IPS_AI_Behavior_Interface` — implements on host project Pawns
## PS_AI_Behavior — Architecture
### Core Design
- **Interface-driven**: `IPS_AI_Behavior_Interface` on Pawns decouples plugin from host project
- **Personality profiles**: DataAsset with 5 trait axes, thresholds, target priority, speed per state
- **NPCType enum**: Civilian, Enemy, Protector, Any (unified — no separate SplineCategory)
- **TeamIds**: Civilian=1, Enemy=2, Protector=3, auto-assigned in OnPossess
- **Hostile switch**: Enemy with IsBehaviorHostile=false → TeamId=1 (disguised), flips to 2 at runtime
### Blackboard
- Key `BehaviorState` = Enum type, path: `/Script/PS_AI_Behavior.EPS_AI_Behavior_State`
- Values: Idle(0), Patrol(1), Alerted(2), Combat(3), Fleeing(4), TakingCover(5), Dead(6)
- BB asset created manually in editor (runtime fallback exists but prefer asset)
- SetValueAsEnum / GetValueAsEnum (not Int)
### BT Pattern
- Services on Selector: UpdateThreat + EvaluateReaction
- Decorator Observer Aborts = Both on Combat/Flee branches
- Spline branch as fallback (lowest priority, no condition decorator)
- Attack task stays InProgress permanently, Decorator pulls out on state change
### Spline System
- SplinePath actors placed in level, SplineCategory = NPCType
- SplineNetwork WorldSubsystem auto-detects junctions
- SplineFollowerComponent on Pawns: uses Pawn's real velocity, debug green sphere
- Junction switching: 70% probability, direction filter (dot > -0.5)
### Perception
- Senses configured in BeginPlay (NOT constructor — NewObject crashes in CDO)
- RequestStimuliListenerUpdate() after ConfigureSenses
- Detect ALL affiliations, filter by Hostile attitude in CalculateThreatLevel
- Target priority from PersonalityProfile (combat targeting)
### Known Gotchas (UE 5.5)
- BB Enum picker doesn't list plugin enums — use string path
- NTFS junctions (not symlinks) for UE5 cross-project plugin linking
- SVN doesn't traverse junctions — robocopy for initial import
- Live Coding fails on header changes — full rebuild required
- BlueprintNativeEvent: UHT auto-generates _Implementation defaults, don't duplicate in .cpp
- FGenericTeamId::NoTeam comparison: always recalculate TeamId (BP CDOs may reset to 0)
### Testing Status (2026-03-26)
- ✅ Patrol, Spline following, Perception, Threat, Flee, Combat delegation, Hostile switch
- 🔲 Cover Points, EQS, Editor tools (SplineEdMode, CoverPoint placement)
## Posture System — Diagonal Tilt Bug (à corriger) ## Posture System — Diagonal Tilt Bug (à corriger)
### Problème ### Problème
Le tilt diagonal (ear-to-shoulder) est PLUS VISIBLE avec la neck bone chain qu'avec le mono-bone. Le swing-twist decomposition ne suffit pas. Le tilt diagonal (ear-to-shoulder) est PLUS VISIBLE avec la neck bone chain qu'avec le mono-bone.
### Cause probable
Le swing-twist est fait une seule fois sur le tip bone (head), puis le CleanOffset est extrait via `Swing * Inverse(TipBoneRot)`. Ce CleanOffset est ensuite distribué via Slerp à chaque bone de la chaîne. Mais chaque bone a une orientation locale différente — appliquer le même offset "world-space-like" à des bones avec des axes locaux différents peut réintroduire du tilt.
### Piste de fix ### Piste de fix
Faire le swing-twist **PER-BONE** : pour chaque bone de la chaîne, composer le FractionalRot avec le bone's own rotation, swing-twist autour du bone's own tilt axis, puis appliquer le swing seulement. Faire le swing-twist **PER-BONE** : pour chaque bone de la chaîne, composer le FractionalRot avec le bone's own rotation, swing-twist autour du bone's own tilt axis.
### Fichiers concernés ### Fichiers concernés
- `AnimNode_PS_AI_ConvAgent_Posture.cpp` — Evaluate_AnyThread, section multi-bone chain - `AnimNode_PS_AI_ConvAgent_Posture.cpp` — Evaluate_AnyThread, section multi-bone chain
- Commit actuel: `8df6967` sur main
### Config actuelle neck chain (BP_Taro) ## Architecture Posture (ConvAgent)
neck_01=0.25, neck_02=0.35, head=0.40 - **PS_AI_ConvAgent_PostureComponent** (game thread) : cascade Eyes→Head→Body
## Architecture Posture
- **PS_AI_ConvAgent_PostureComponent** (game thread) : cascade Eyes→Head→Body, produit FQuat + eye curves
- **AnimNode_PS_AI_ConvAgent_Posture** (anim thread) : applique rotation sur bone(s) + injecte curves - **AnimNode_PS_AI_ConvAgent_Posture** (anim thread) : applique rotation sur bone(s) + injecte curves
- Pipeline 100% FQuat (pas de round-trip FRotator) - Pipeline 100% FQuat, Thread safety via FCriticalSection
- Thread safety via FCriticalSection - ARKit eye curves normalisées par range fixe (40°/35°)
- ARKit eye curves normalisées par range fixe (40°/35°), pas par MaxEye threshold
- Body drift compensation: applied ONLY on first chain bone, AFTER swing-twist
- Debug gaze: per-eye lines from Face mesh using Z-axis (GetAxisZ())

30
.gitattributes vendored Normal file
View File

@@ -0,0 +1,30 @@
# Force consistent line endings across all machines
# LF in the repo, native (CRLF on Windows) in working tree
* text=auto
# Source code — always normalize
*.cpp text
*.h text
*.cs text
*.py text
*.ini text
*.json text
*.md text
*.xml text
*.yaml text
*.yml text
# Unreal assets — binary, no normalization
*.uasset binary
*.umap binary
*.uproject text
# Binaries — no normalization
*.dll binary
*.exe binary
*.pdb binary
*.lib binary
*.exp binary
*.so binary
*.dylib binary
*.a binary

5
.gitignore vendored
View File

@@ -5,10 +5,13 @@ Unreal/PS_AI_Agent/Intermediate/
Unreal/PS_AI_Agent/Saved/ Unreal/PS_AI_Agent/Saved/
ConvAI/Convai/Binaries/ ConvAI/Convai/Binaries/
# All plugin build artifacts (Binaries + Intermediate for any plugin) # Plugin build artifacts (regenerated)
Unreal/PS_AI_Agent/Plugins/*/Binaries/ Unreal/PS_AI_Agent/Plugins/*/Binaries/
Unreal/PS_AI_Agent/Plugins/*/Intermediate/ Unreal/PS_AI_Agent/Plugins/*/Intermediate/
# Live Coding patch files (temporary, regenerated each session)
*.patch_*
# UE5 generated solution files # UE5 generated solution files
Unreal/PS_AI_Agent/*.sln Unreal/PS_AI_Agent/*.sln
Unreal/PS_AI_Agent/*.suo Unreal/PS_AI_Agent/*.suo

65
Build_Plugin.bat Normal file
View File

@@ -0,0 +1,65 @@
@echo off
setlocal
:: ─── Configuration ──────────────────────────────────────────────────────────
set UE_ROOT=C:\Program Files\Epic Games\UE_5.5
set UPROJECT=E:\ASTERION\GIT\PS_AI_Agent\Unreal\PS_AI_Agent\PS_AI_Agent.uproject
set PLATFORM=Win64
set CONFIG=Development
set TARGET=PS_AI_AgentEditor
:: ─── Colors ─────────────────────────────────────────────────────────────────
set GREEN=[92m
set RED=[91m
set YELLOW=[93m
set RESET=[0m
:: ─── Banner ─────────────────────────────────────────────────────────────────
echo.
echo %GREEN%============================================%RESET%
echo PS_AI_Agent - Plugin Build
echo Target: %TARGET% %PLATFORM% %CONFIG%
echo %GREEN%============================================%RESET%
echo.
:: ─── Verify paths ───────────────────────────────────────────────────────────
if not exist "%UE_ROOT%\Engine\Binaries\ThirdParty\DotNet\8.0.300\win-x64\dotnet.exe" (
echo %RED%ERROR: UE5.5 dotnet not found at %UE_ROOT%%RESET%
echo Check UE_ROOT path in this script.
pause
exit /b 1
)
if not exist "%UPROJECT%" (
echo %RED%ERROR: .uproject not found: %UPROJECT%%RESET%
pause
exit /b 1
)
:: ─── Build ──────────────────────────────────────────────────────────────────
echo %YELLOW%Building %TARGET% (%CONFIG%)...%RESET%
echo.
"%UE_ROOT%\Engine\Binaries\ThirdParty\DotNet\8.0.300\win-x64\dotnet.exe" ^
"%UE_ROOT%\Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.dll" ^
%TARGET% %PLATFORM% %CONFIG% ^
-Project="%UPROJECT%" ^
-WaitMutex ^
-FromMsBuild
set BUILD_EXIT=%ERRORLEVEL%
echo.
if %BUILD_EXIT% EQU 0 (
echo %GREEN%============================================%RESET%
echo BUILD SUCCEEDED
echo %GREEN%============================================%RESET%
) else (
echo %RED%============================================%RESET%
echo BUILD FAILED (exit code: %BUILD_EXIT%)
echo %RED%============================================%RESET%
)
echo.
pause
exit /b %BUILD_EXIT%

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

691
Docs/gen_behavior_doc.js Normal file
View File

@@ -0,0 +1,691 @@
// PS_AI_Behavior User Documentation Generator
// Palette: Warm Terracotta (terracotta #B85042, sand #E7E8D1, sage #A7BEAE)
const pptxgen = require("pptxgenjs");
const pres = new pptxgen();
pres.layout = "LAYOUT_WIDE"; // 13.333 x 7.5 inch
// ─── Color Palette ─────────────────────────────────
const TERRA = "B85042";
const TERRA_D = "7A3229";
const SAND = "E7E8D1";
const SAND_LT = "F5F5EA";
const SAGE = "A7BEAE";
const SAGE_D = "5F7566";
const WHITE = "FFFFFF";
const CHARCOAL = "2E2A26";
const GOLD = "D4A84B";
// ─── Font ──────────────────────────────────────────
const FONT_H = "Calibri";
const FONT_B = "Calibri";
// ─── Master slide backgrounds ──────────────────────
pres.defineSlideMaster({
title: "CONTENT",
background: { color: SAND_LT },
objects: [
{ rect: { x: 0, y: 0, w: 13.333, h: 0.35, fill: { color: TERRA } } },
{ rect: { x: 0, y: 7.15, w: 13.333, h: 0.35, fill: { color: TERRA } } },
{ text: {
text: "PS_AI_Behavior — User Documentation",
options: { x: 0.3, y: 7.18, w: 8, h: 0.3, fontSize: 10, color: SAND, fontFace: FONT_B }
} },
{ text: {
text: "ASTERION",
options: { x: 11.5, y: 7.18, w: 1.5, h: 0.3, fontSize: 10, color: SAND, fontFace: FONT_B, align: "right", bold: true }
} },
],
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 1 — TITLE
// ═══════════════════════════════════════════════════════════════════
let s = pres.addSlide();
s.background = { color: TERRA };
s.addShape(pres.ShapeType.rect, { x: 0, y: 3.3, w: 13.333, h: 0.05, fill: { color: GOLD } });
s.addText("PS_AI_Behavior", { x: 0.6, y: 2.2, w: 12, h: 1.2, fontSize: 60, bold: true, color: WHITE, fontFace: FONT_H });
s.addText("Personality-Driven NPC AI for Unreal Engine", { x: 0.6, y: 3.5, w: 12, h: 0.6, fontSize: 22, color: SAND, fontFace: FONT_B, italic: true });
s.addText("User Documentation", { x: 0.6, y: 4.4, w: 12, h: 0.5, fontSize: 18, color: SAND, fontFace: FONT_B });
s.addText("Civilians • Enemies • Protectors • Cover System • Splines", { x: 0.6, y: 5.2, w: 12, h: 0.4, fontSize: 14, color: SAND, fontFace: FONT_B });
s.addText("ASTERION • 2026", { x: 0.6, y: 6.6, w: 12, h: 0.4, fontSize: 12, color: SAND, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 2 — OVERVIEW
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Plugin Overview", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("What it does", { x: 0.5, y: 1.7, w: 6, h: 0.4, fontSize: 18, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "Personality-driven AI for NPCs", options: { bullet: true, fontSize: 14 } },
{ text: "Three archetypes: Civilian, Enemy, Protector", options: { bullet: true, fontSize: 14 } },
{ text: "Trait-based reactions: Courage, Aggressivity, Caution", options: { bullet: true, fontSize: 14 } },
{ text: "Tactical combat + cover system + EQS", options: { bullet: true, fontSize: 14 } },
{ text: "Spline-based patrol networks", options: { bullet: true, fontSize: 14 } },
{ text: "Interface-driven — no Pawn class dependency", options: { bullet: true, fontSize: 14 } },
], { x: 0.5, y: 2.1, w: 6, h: 3, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 8 });
s.addShape(pres.ShapeType.roundRect, { x: 7, y: 1.7, w: 5.8, h: 4.8, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("Primary Use Case", { x: 7.2, y: 1.9, w: 5.4, h: 0.4, fontSize: 16, bold: true, color: GOLD, fontFace: FONT_H });
s.addText("Populate your world with intelligent NPCs that perceive threats, evaluate risk based on personality, and respond appropriately — civilians flee from gunfire, enemies engage in tactical combat, protectors defend civilians.", {
x: 7.2, y: 2.4, w: 5.4, h: 1.7, fontSize: 13, color: SAND, fontFace: FONT_B, italic: true
});
s.addText("Key Differentiators", { x: 7.2, y: 4.2, w: 5.4, h: 0.4, fontSize: 16, bold: true, color: GOLD, fontFace: FONT_H });
s.addText([
{ text: "Data-driven personality profiles", options: { bullet: true, fontSize: 12 } },
{ text: "Built-in cover cycle state machine", options: { bullet: true, fontSize: 12 } },
{ text: "Spline network with junction detection", options: { bullet: true, fontSize: 12 } },
{ text: "Gaze bridge to ConvAgent (reflection)", options: { bullet: true, fontSize: 12 } },
], { x: 7.2, y: 4.7, w: 5.4, h: 1.7, color: SAND, fontFace: FONT_B, paraSpaceAfter: 4 });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 3 — CORE ARCHITECTURE
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Core Architecture", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("The plugin is built around four concepts:", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const pillars = [
{ title: "Interface", subtitle: "IPS_AI_Behavior_Interface", desc: "Implemented on host project Pawns. Zero compile dependency — identity, team, combat, movement." },
{ title: "NPC Type", subtitle: "Civilian / Enemy / Protector", desc: "Determines team, spline access, default tree. Enemies can be disguised (hostile=false)." },
{ title: "State", subtitle: "Behavior State enum", desc: "Idle, Patrol, Alerted, Combat, Fleeing, TakingCover, Dead, Scripted. Written to Blackboard." },
{ title: "Profile", subtitle: "PersonalityProfile data asset", desc: "Traits, thresholds, target priority, speed per state, default BT." },
];
pillars.forEach((p, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.5 + col * 6.3;
const y = 2.2 + row * 2.3;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 6.0, h: 2.1, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText(p.title, { x: x + 0.2, y: y + 0.15, w: 5.6, h: 0.45, fontSize: 20, bold: true, color: GOLD, fontFace: FONT_H });
s.addText(p.subtitle, { x: x + 0.2, y: y + 0.6, w: 5.6, h: 0.35, fontSize: 12, color: SAND, italic: true, fontFace: "Consolas" });
s.addText(p.desc, { x: x + 0.2, y: y + 1.0, w: 5.6, h: 1.0, fontSize: 12, color: SAND, fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 4 — NPC TYPES & TEAM ID
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("NPC Types & TeamId", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Identity drives perception, targeting, and access control.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
// Three NPC cards
const npcs = [
{ name: "CIVILIAN", color: SAGE, id: "0x00", desc: "Non-hostile. Flees. Allied with Protector." },
{ name: "ENEMY", color: TERRA, id: "0x10", desc: "Hostile. Tactical combat. Can be disguised." },
{ name: "PROTECTOR", color: SAGE_D, id: "0x20", desc: "Police / guard. Defends Civilians." },
];
npcs.forEach((n, i) => {
const x = 0.5 + i * 4.3;
s.addShape(pres.ShapeType.roundRect, { x: x, y: 2.15, w: 4.0, h: 2.2, fill: { color: n.color }, rectRadius: 0.1 });
s.addText(n.name, { x: x + 0.2, y: 2.3, w: 3.6, h: 0.5, fontSize: 22, bold: true, color: WHITE, align: "center", fontFace: FONT_H });
s.addText(`TeamId = ${n.id}`, { x: x + 0.2, y: 2.85, w: 3.6, h: 0.35, fontSize: 14, color: WHITE, align: "center", italic: true, fontFace: "Consolas" });
s.addText(n.desc, { x: x + 0.2, y: 3.3, w: 3.6, h: 0.95, fontSize: 12, color: WHITE, align: "center", fontFace: FONT_B });
});
// TeamId encoding
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 4.6, w: 12.3, h: 2.3, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("TeamId Encoding", { x: 0.7, y: 4.7, w: 12, h: 0.4, fontSize: 16, bold: true, color: GOLD, fontFace: FONT_H });
s.addText("uint8 = (high nibble: NPCType) | (low nibble: Faction 0-15)", { x: 0.7, y: 5.1, w: 12, h: 0.3, fontSize: 13, color: SAND, italic: true, fontFace: "Consolas" });
s.addText([
{ text: "Same type + same faction → Friendly (allies)", options: { bullet: true, fontSize: 12 } },
{ text: "Same type + different faction → Hostile (rival gangs)", options: { bullet: true, fontSize: 12 } },
{ text: "Civilian ↔ Protector → always Friendly", options: { bullet: true, fontSize: 12 } },
{ text: "Everything else → Hostile", options: { bullet: true, fontSize: 12 } },
{ text: "Disguised Enemy (hostile=false) → TeamId 0x01 (civilian disguise)", options: { bullet: true, fontSize: 12 } },
], { x: 0.7, y: 5.45, w: 12, h: 1.4, color: SAND, fontFace: FONT_B, paraSpaceAfter: 3 });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 5 — BEHAVIOR STATES
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Behavior States", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Enum written to Blackboard, drives BT branching.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const states = [
{ name: "Idle", desc: "Resting / waiting. Proximity gaze active." },
{ name: "Patrol", desc: "Wandering on waypoints or splines." },
{ name: "Alerted", desc: "Suspicious, heightened awareness." },
{ name: "Combat", desc: "Engaged with a threat, attacking." },
{ name: "Fleeing", desc: "Running away from danger." },
{ name: "TakingCover", desc: "Behind tactical cover during combat." },
{ name: "Dead", desc: "NPC deceased, all systems shut down." },
{ name: "Scripted", desc: "Manual override (conversations, cinematics)." },
];
states.forEach((state, i) => {
const col = i % 4;
const row = Math.floor(i / 4);
const x = 0.5 + col * 3.15;
const y = 2.2 + row * 2.3;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 2.95, h: 2.1, fill: { color: SAND }, line: { color: TERRA, width: 1 }, rectRadius: 0.08 });
s.addShape(pres.ShapeType.rect, { x: x, y: y, w: 2.95, h: 0.55, fill: { color: TERRA } });
s.addText(state.name, { x: x, y: y + 0.05, w: 2.95, h: 0.5, fontSize: 16, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
s.addText(state.desc, { x: x + 0.15, y: y + 0.7, w: 2.65, h: 1.3, fontSize: 11, color: CHARCOAL, valign: "top", fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 6 — AI CONTROLLER
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("AI Controller", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("APS_AI_Behavior_AIController — the central orchestrator.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
s.addText("Auto-setup on Possession", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "Discovers PersonalityComponent on Pawn", options: { bullet: true, fontSize: 12 } },
{ text: "Computes TeamId from NPCType + Faction", options: { bullet: true, fontSize: 12 } },
{ text: "Initializes Blackboard with required keys", options: { bullet: true, fontSize: 12 } },
{ text: "Binds gaze bridge to ConvAgent (if present)", options: { bullet: true, fontSize: 12 } },
{ text: "Starts BT (unless bAutoStartBehavior = false)", options: { bullet: true, fontSize: 12 } },
], { x: 0.5, y: 2.55, w: 6, h: 2.5, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 4 });
s.addText("Key Methods", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "StartBehavior() / StopBehavior()", options: { bullet: true, fontSize: 12 } },
{ text: "SetBehaviorState() / GetBehaviorState()", options: { bullet: true, fontSize: 12 } },
{ text: "SetTeamId() — runtime team change", options: { bullet: true, fontSize: 12 } },
{ text: "GetBehaviorPerception()", options: { bullet: true, fontSize: 12 } },
{ text: "GetPersonalityComponent()", options: { bullet: true, fontSize: 12 } },
], { x: 7, y: 2.55, w: 6, h: 2.5, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 4 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.2, w: 12.3, h: 1.8, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("Gaze Bridge (Proximity)", { x: 0.7, y: 5.3, w: 12, h: 0.35, fontSize: 14, bold: true, color: GOLD, fontFace: FONT_H });
s.addText("Reflection-based integration with PS_AI_ConvAgent. During Idle/Patrol, NPC glances at nearby actors within GazeProximityRadius. Configurable duration and cooldown. Auto-pauses during active conversations.", {
x: 0.7, y: 5.7, w: 12, h: 1.25, fontSize: 12, color: SAND, italic: true, fontFace: FONT_B
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 7 — PERSONALITY COMPONENT
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Personality Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Runtime traits & state reactions for the NPC.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
// Traits triangle
s.addText("The Three Traits", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
const traits = [
{ name: "Courage", desc: "0 = coward, 1 = fearless", effect: "Gates flee threshold & cover advancement", color: "C9463A" },
{ name: "Aggressivity", desc: "0 = peaceful, 1 = violent", effect: "Drives attack threshold, combat/cover cycle", color: "D4803A" },
{ name: "Caution", desc: "0 = reckless, 1 = prudent", effect: "Flee threshold, cover duration, spline threat", color: "7C9D7B" },
];
traits.forEach((t, i) => {
const y = 2.6 + i * 0.95;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 1.8, h: 0.8, fill: { color: t.color }, rectRadius: 0.08 });
s.addText(t.name, { x: 0.5, y: y + 0.1, w: 1.8, h: 0.3, fontSize: 14, bold: true, color: WHITE, align: "center", fontFace: FONT_H });
s.addText(t.desc, { x: 0.5, y: y + 0.45, w: 1.8, h: 0.3, fontSize: 10, color: WHITE, align: "center", italic: true, fontFace: FONT_B });
s.addText(t.effect, { x: 2.4, y: y + 0.15, w: 4.2, h: 0.6, fontSize: 12, color: CHARCOAL, valign: "middle", fontFace: FONT_B });
});
s.addText("Properties & Events", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "Profile — PersonalityProfile DataAsset", options: { bullet: true, fontSize: 12 } },
{ text: "bAutoStartBehavior — per-NPC override", options: { bullet: true, fontSize: 12 } },
{ text: "RuntimeTraits — editable at runtime", options: { bullet: true, fontSize: 12 } },
{ text: "PerceivedThreatLevel — replicated", options: { bullet: true, fontSize: 12 } },
{ text: "CurrentState — replicated with OnRep", options: { bullet: true, fontSize: 12 } },
{ text: "EvaluateReaction() / ApplyReaction()", options: { bullet: true, fontSize: 12 } },
{ text: "ModifyTrait() / GetTrait()", options: { bullet: true, fontSize: 12 } },
{ text: "ForceState() — scripting override", options: { bullet: true, fontSize: 12 } },
{ text: "OnBehaviorStateChanged delegate", options: { bullet: true, fontSize: 12 } },
], { x: 7, y: 2.55, w: 6, h: 4.5, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 3 });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 8 — PERSONALITY PROFILE
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("PersonalityProfile DataAsset", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 32, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("The behavioral recipe — one profile per NPC archetype.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const profileCats = [
{ title: "Identity", items: ["ProfileName", "NPCType", "Faction (0-15)"] },
{ title: "Traits", items: ["Courage (0-1)", "Aggressivity (0-1)", "Caution (0-1)"] },
{ title: "Thresholds", items: ["FleeThreshold", "AttackThreshold", "AlertThreshold", "ThreatDecayRate"] },
{ title: "Combat", items: ["TargetPriority[]", "MinAttackRange / MaxAttackRange", "CombatCoverCycleDuration"] },
{ title: "Movement", items: ["SpeedPerState map", "DefaultWalkSpeed"] },
{ title: "Behavior", items: ["DefaultBehaviorTree (soft ref)"] },
];
profileCats.forEach((c, i) => {
const col = i % 3;
const row = Math.floor(i / 3);
const x = 0.5 + col * 4.15;
const y = 2.2 + row * 2.3;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 3.95, h: 2.1, fill: { color: SAND }, line: { color: TERRA, width: 1 }, rectRadius: 0.08 });
s.addText(c.title, { x: x + 0.15, y: y + 0.1, w: 3.65, h: 0.35, fontSize: 14, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: x + 0.15, y: y + 0.45, w: 0.35, h: 0.04, fill: { color: GOLD } });
s.addText(c.items.map(t => ({ text: t, options: { bullet: true, fontSize: 11 } })),
{ x: x + 0.15, y: y + 0.55, w: 3.65, h: 1.4, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 3 });
});
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 6.85, w: 12.3, h: 0.35, fill: { color: TERRA }, rectRadius: 0.04 });
s.addText("Example presets: Cowardly Villager • Aggressive Guard • Rival Gang Member", { x: 0.7, y: 6.88, w: 12, h: 0.3, fontSize: 11, color: SAND, italic: true, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 9 — COMPONENTS OVERVIEW
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Components", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Attach to Pawns as needed — most are optional.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const components = [
{ name: "Personality", desc: "Traits, profile, state reactions", required: true },
{ name: "Perception", desc: "AI Perception wrapper, threat computation", required: "auto" },
{ name: "Team", desc: "Identity for non-AI pawns (player)", required: false },
{ name: "Combat", desc: "Attack events, fire/kill delegates", required: false },
{ name: "Spline Follower", desc: "Spline walking, junction detection", required: false },
];
components.forEach((c, i) => {
const y = 2.2 + i * 0.9;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 3.5, h: 0.8, fill: { color: TERRA }, rectRadius: 0.08 });
s.addText(c.name, { x: 0.5, y: y, w: 3.5, h: 0.8, fontSize: 16, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
s.addShape(pres.ShapeType.roundRect, { x: 4.2, y: y, w: 7.0, h: 0.8, fill: { color: SAND }, line: { color: TERRA, width: 1 }, rectRadius: 0.06 });
s.addText(c.desc, { x: 4.4, y: y + 0.1, w: 6.8, h: 0.6, fontSize: 13, color: CHARCOAL, valign: "middle", fontFace: FONT_B });
let tagText = c.required === true ? "Required" : (c.required === "auto" ? "Auto" : "Optional");
let tagColor = c.required === true ? TERRA : (c.required === "auto" ? SAGE_D : GOLD);
s.addShape(pres.ShapeType.roundRect, { x: 11.4, y: y + 0.2, w: 1.4, h: 0.4, fill: { color: tagColor }, rectRadius: 0.08 });
s.addText(tagText, { x: 11.4, y: y + 0.2, w: 1.4, h: 0.4, fontSize: 11, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 10 — PERCEPTION COMPONENT
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Perception Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Sight, hearing, damage senses — auto-configured.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
s.addText("Senses", { x: 0.5, y: 2.15, w: 4, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "Sight — vision cone + line of sight", options: { bullet: true, fontSize: 13 } },
{ text: "Hearing — noise events with tags", options: { bullet: true, fontSize: 13 } },
{ text: "Damage — damage received tracking", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 4, h: 2, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Threat Scoring", { x: 4.8, y: 2.15, w: 4, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "Target priority (from Profile)", options: { bullet: true, fontSize: 13 } },
{ text: "Recent damage weight", options: { bullet: true, fontSize: 13 } },
{ text: "Proximity weight", options: { bullet: true, fontSize: 13 } },
{ text: "Target persistence (80% score hold)", options: { bullet: true, fontSize: 13 } },
], { x: 4.8, y: 2.55, w: 4.2, h: 2.2, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Key Methods", { x: 9.3, y: 2.15, w: 3.5, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "GetHighestThreatActor()", options: { bullet: true, fontSize: 12 } },
{ text: "CalculateThreatLevel()", options: { bullet: true, fontSize: 12 } },
{ text: "GetThreatLocation()", options: { bullet: true, fontSize: 12 } },
{ text: "ClassifyActor()", options: { bullet: true, fontSize: 12 } },
], { x: 9.3, y: 2.55, w: 3.5, h: 2, color: CHARCOAL, fontFace: "Consolas", paraSpaceAfter: 4 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.15, w: 12.3, h: 1.85, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("Perception / Threat Resolution", { x: 0.7, y: 5.25, w: 12, h: 0.35, fontSize: 14, bold: true, color: GOLD, fontFace: FONT_H });
s.addText("Detects ALL affiliations → filters by Hostile attitude. Supports separate ThreatTarget (e.g. AimTargetActor sphere) vs. owning Pawn. Line-of-sight tracking with last-known-position memory.", {
x: 0.7, y: 5.65, w: 12, h: 1.3, fontSize: 12, color: SAND, italic: true, fontFace: FONT_B
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 11 — COMBAT COMPONENT
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Combat Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Attack execution, cooldown, damage events.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
s.addText("Properties", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "AttackRange — max distance (cm)", options: { bullet: true, fontSize: 13 } },
{ text: "AttackCooldown — minimum between attacks", options: { bullet: true, fontSize: 13 } },
{ text: "AttackDamage — base damage value", options: { bullet: true, fontSize: 13 } },
{ text: "CurrentTarget — replicated", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 2.3, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Methods & Events", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "CanAttack() — cooldown check", options: { bullet: true, fontSize: 13 } },
{ text: "IsInAttackRange() — distance test", options: { bullet: true, fontSize: 13 } },
{ text: "ExecuteAttack() — apply damage + multicast", options: { bullet: true, fontSize: 13 } },
{ text: "OnAttackExecuted (multicast)", options: { bullet: true, fontSize: 13 } },
{ text: "OnDamageReceived (server-only)", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 2.5, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.3, w: 12.3, h: 1.7, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("Combat Type", { x: 0.7, y: 5.4, w: 12, h: 0.35, fontSize: 14, bold: true, color: GOLD, fontFace: FONT_H });
s.addText("NPC's combat type (Melee / Ranged) comes from IPS_AI_Behavior::GetBehaviorCombatType(). Melee = rush the target (no cooldown between re-positions). Ranged = maintain distance, use cover cycle, peek to fire.", {
x: 0.7, y: 5.75, w: 12, h: 1.2, fontSize: 12, color: SAND, italic: true, fontFace: FONT_B
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 12 — WORLD ACTORS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("World Actors", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Placed by level designers — drive navigation and tactics.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const actors = [
{ name: "SplinePath", desc: "Placeable spline defining a navigation corridor. Filtered by NPCType. Bidirectional, with priority for junction selection." },
{ name: "SplineNetwork", desc: "WorldSubsystem — auto-detects intersections between SplinePath actors. Provides navigation queries, threat-aware switching." },
{ name: "CoverPoint", desc: "Manually placed tactical position. Cover or HidingSpot type, quality score, crouch flag, max occupants, type filter." },
];
actors.forEach((a, i) => {
const y = 2.2 + i * 1.5;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 12.3, h: 1.35, fill: { color: SAND }, line: { color: TERRA, width: 1 }, rectRadius: 0.1 });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: y, w: 3.5, h: 1.35, fill: { color: TERRA } });
s.addText(a.name, { x: 0.5, y: y, w: 3.5, h: 1.35, fontSize: 20, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
s.addText(a.desc, { x: 4.2, y: y + 0.15, w: 8.4, h: 1.05, fontSize: 13, color: CHARCOAL, valign: "middle", fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 13 — BEHAVIOR TREE SERVICES & TASKS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Behavior Tree — Services & Tasks", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 30, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Reusable BT nodes — build custom trees per NPC type.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
s.addText("Services (root-level)", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
const services = [
{ name: "UpdateThreat", desc: "Queries perception, writes threat BB keys" },
{ name: "EvaluateReaction", desc: "Computes state from traits + threat" },
{ name: "UpdateGaze", desc: "Manages eye contact, ConvAgent bridge" },
];
services.forEach((svc, i) => {
const y = 2.55 + i * 0.7;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 2.3, h: 0.55, fill: { color: SAGE_D }, rectRadius: 0.06 });
s.addText(svc.name, { x: 0.5, y: y, w: 2.3, h: 0.55, fontSize: 12, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
s.addText(svc.desc, { x: 2.9, y: y, w: 3.7, h: 0.55, fontSize: 11, color: CHARCOAL, valign: "middle", fontFace: FONT_B });
});
s.addText("Tasks (leaves)", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
const tasks = [
{ name: "FollowSpline", desc: "Drives SplineFollower" },
{ name: "FindAndFollowSpline", desc: "Auto-locate + follow" },
{ name: "Patrol", desc: "Cycle PatrolPoints" },
{ name: "FindCover", desc: "Manual + procedural cover" },
{ name: "CoverShootCycle", desc: "Cover → peek → fire cycle" },
{ name: "Attack", desc: "Chase + attack (Melee/Ranged)" },
{ name: "FleeFrom", desc: "Move away from threat" },
];
tasks.forEach((task, i) => {
const y = 2.55 + i * 0.58;
s.addShape(pres.ShapeType.roundRect, { x: 7, y: y, w: 2.3, h: 0.45, fill: { color: TERRA }, rectRadius: 0.06 });
s.addText(task.name, { x: 7, y: y, w: 2.3, h: 0.45, fontSize: 11, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
s.addText(task.desc, { x: 9.4, y: y, w: 3.5, h: 0.45, fontSize: 10, color: CHARCOAL, valign: "middle", fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 14 — COVER SYSTEM
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Cover System", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Manual CoverPoints + procedural raycast fallback + EQS refinement.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const coverCols = [
{
title: "Manual CoverPoints",
items: ["Placed by level designer", "Type: Cover or HidingSpot", "Quality score (01)", "MaxOccupants, Crouch flag", "Type filter (Any / Civ / Enemy / Protector)"]
},
{
title: "Procedural",
items: ["Fallback if no manual points", "Raycast sampling around threat", "LOS blockage test", "Advancement bias toward threat"]
},
{
title: "EQS Refinement",
items: ["OnCircle generator around cover", "LineOfSight filter", "Distance + quality tests", "Optional firing position refine"]
},
];
coverCols.forEach((c, i) => {
const x = 0.5 + i * 4.28;
s.addShape(pres.ShapeType.roundRect, { x: x, y: 2.2, w: 4.08, h: 3.1, fill: { color: SAND }, line: { color: TERRA, width: 1 }, rectRadius: 0.1 });
s.addText(c.title, { x: x + 0.2, y: 2.3, w: 3.78, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: x + 0.2, y: 2.72, w: 0.4, h: 0.04, fill: { color: GOLD } });
s.addText(c.items.map(t => ({ text: t, options: { bullet: true, fontSize: 11 } })),
{ x: x + 0.2, y: 2.85, w: 3.78, h: 2.3, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 3 });
});
// Cover cycle state machine
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.55, w: 12.3, h: 1.45, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("Cover Cycle State Machine (CombatSubState)", { x: 0.7, y: 5.65, w: 12, h: 0.35, fontSize: 14, bold: true, color: GOLD, fontFace: FONT_H });
const subStates = ["Engaging", "AtCover", "MovingToFire", "Peeking", "ReturningToCover", "Advancing"];
subStates.forEach((sub, i) => {
const x = 0.7 + i * 2.0;
s.addShape(pres.ShapeType.roundRect, { x: x, y: 6.15, w: 1.9, h: 0.7, fill: { color: TERRA_D }, rectRadius: 0.05 });
s.addText(sub, { x: x, y: 6.15, w: 1.9, h: 0.7, fontSize: 11, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 15 — INTERFACE (IMPLEMENTERS GUIDE)
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("IPS_AI_Behavior_Interface", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 34, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("What your Pawn must implement (C++ or Blueprint).", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const ifCats = [
{
title: "Identity",
items: ["GetBehaviorNPCType()", "SetBehaviorNPCType()", "IsBehaviorHostile()", "SetBehaviorHostile()"]
},
{
title: "Movement",
items: ["SetBehaviorMovementSpeed()", "GetBehaviorMovementSpeed()", "SetBehaviorCrouch()"]
},
{
title: "State Feedback",
items: ["OnBehaviorStateChanged()", "IsTargetActorValid()", "GetBehaviorThreatActor()"]
},
{
title: "Combat",
items: ["BehaviorStartAttack()", "BehaviorStopAttack()", "CanBehaviorAttack()", "GetBehaviorCombatType()"]
},
];
ifCats.forEach((c, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.5 + col * 6.3;
const y = 2.2 + row * 2.4;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 6.0, h: 2.2, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText(c.title, { x: x + 0.2, y: y + 0.15, w: 5.6, h: 0.4, fontSize: 16, bold: true, color: GOLD, fontFace: FONT_H });
s.addText(c.items.map(t => ({ text: t, options: { bullet: true, fontSize: 12 } })),
{ x: x + 0.2, y: y + 0.6, w: 5.6, h: 1.55, color: SAND, fontFace: "Consolas", paraSpaceAfter: 3 });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 16 — SETUP WORKFLOW
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Setup Workflow", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("From zero to a behaving NPC in 6 steps.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const bsteps = [
{ num: "1", title: "Personality Profile", desc: "Create DataAsset — traits, thresholds, DefaultBehaviorTree." },
{ num: "2", title: "Implement Interface", desc: "Pawn: implement IPS_AI_Behavior_Interface (identity, combat, movement)." },
{ num: "3", title: "Add Components", desc: "Pawn: add PersonalityComponent. Optional: Combat, SplineFollower." },
{ num: "4", title: "Behavior Tree", desc: "Branch on BehaviorState. Add services UpdateThreat + EvaluateReaction at root." },
{ num: "5", title: "Set AIController", desc: "Pawn's AIController class = APS_AI_Behavior_AIController." },
{ num: "6", title: "Place World Actors", desc: "Optional: SplinePaths, CoverPoints, SplineNetwork (auto)." },
];
bsteps.forEach((step, i) => {
const y = 2.2 + i * 0.78;
s.addShape(pres.ShapeType.ellipse, { x: 0.5, y: y, w: 0.7, h: 0.7, fill: { color: GOLD } });
s.addText(step.num, { x: 0.5, y: y, w: 0.7, h: 0.7, fontSize: 24, bold: true, color: TERRA, align: "center", valign: "middle", fontFace: FONT_H });
s.addShape(pres.ShapeType.roundRect, { x: 1.4, y: y, w: 11.4, h: 0.7, fill: { color: SAND }, line: { color: TERRA, width: 1 }, rectRadius: 0.05 });
s.addText(step.title, { x: 1.6, y: y + 0.05, w: 3.5, h: 0.3, fontSize: 13, bold: true, color: TERRA, fontFace: FONT_H });
s.addText(step.desc, { x: 1.6, y: y + 0.35, w: 11.2, h: 0.35, fontSize: 11, color: CHARCOAL, italic: true, fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 17 — BLACKBOARD KEYS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Blackboard Keys", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Auto-created by the AIController — use in your BT decorators and tasks.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const bbKeys = [
{ name: "BehaviorState", type: "Enum", desc: "Current state (Idle, Combat, etc.)" },
{ name: "ThreatActor", type: "Object", desc: "Current highest-scored threat" },
{ name: "ThreatLocation", type: "Vector", desc: "Last known threat position" },
{ name: "ThreatLevel", type: "Float", desc: "Perceived threat (01+)" },
{ name: "CoverLocation", type: "Vector", desc: "Target cover position" },
{ name: "CoverPoint", type: "Object", desc: "Current CoverPoint actor" },
{ name: "PatrolIndex", type: "Int", desc: "Current patrol waypoint index" },
{ name: "HomeLocation", type: "Vector", desc: "Spawn / home point" },
{ name: "CurrentSpline", type: "Object", desc: "Active SplinePath" },
{ name: "SplineProgress", type: "Float", desc: "Distance along current spline" },
{ name: "CombatSubState", type: "Enum", desc: "Cover cycle sub-state (Peeking, etc.)" },
{ name: "LastKnownTargetPosition", type: "Vector", desc: "LOS-lost target memory" },
{ name: "PreferCover", type: "Bool", desc: "Combat/cover cycle toggle" },
{ name: "ConversationPaused", type: "Bool", desc: "Active dialogue pause flag" },
];
bbKeys.forEach((k, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.5 + col * 6.3;
const y = 2.2 + row * 0.62;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 1.8, h: 0.5, fill: { color: TERRA }, rectRadius: 0.05 });
s.addText(k.name, { x: x, y: y, w: 1.8, h: 0.5, fontSize: 10, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: "Consolas" });
s.addShape(pres.ShapeType.roundRect, { x: x + 1.85, y: y + 0.1, w: 0.6, h: 0.3, fill: { color: GOLD }, rectRadius: 0.05 });
s.addText(k.type, { x: x + 1.85, y: y + 0.1, w: 0.6, h: 0.3, fontSize: 9, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
s.addText(k.desc, { x: x + 2.55, y: y, w: 3.7, h: 0.5, fontSize: 10, color: CHARCOAL, valign: "middle", italic: true, fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 18 — NETWORKING
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Networking", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Server-authoritative AI, minimal replication for visuals.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
s.addText("Server-only", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "All AI planning & decision-making", options: { bullet: true, fontSize: 13 } },
{ text: "Threat evaluation, state transitions", options: { bullet: true, fontSize: 13 } },
{ text: "Perception stimuli processing", options: { bullet: true, fontSize: 13 } },
{ text: "Path selection, cover choice", options: { bullet: true, fontSize: 13 } },
{ text: "Combat targeting", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 2.7, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Replicated to clients", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "CurrentState (OnRep fires delegate)", options: { bullet: true, fontSize: 13 } },
{ text: "PerceivedThreatLevel (HUD/debug)", options: { bullet: true, fontSize: 13 } },
{ text: "CurrentTarget (anims/visuals)", options: { bullet: true, fontSize: 13 } },
{ text: "CurrentSpline, bIsFollowing", options: { bullet: true, fontSize: 13 } },
{ text: "OnAttackExecuted (multicast)", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 2.7, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.4, w: 12.3, h: 1.6, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("Character movement syncs via CharacterMovementComponent. Spline follower updates server-side, position replicates naturally. Clients receive enough state for animation and HUD without any AI logic running locally.", {
x: 0.7, y: 5.6, w: 12, h: 1.2, fontSize: 13, color: SAND, italic: true, fontFace: FONT_B, valign: "middle"
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 19 — INTEGRATION WITH CONVAGENT
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Integration with PS_AI_ConvAgent", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 28, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Optional — behavior and conversation plugins talk via reflection.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
const integrations = [
{ title: "Gaze Bridge", desc: "AIController discovers GazeComponent on the Pawn, sets target during Idle/Patrol (proximity gaze) and forwards current threat during Combat." },
{ title: "Conversation Pause", desc: "When ConvAgent is conversing, AI planning pauses — NPC stops moving, waits. Local VAD detects user speech." },
{ title: "Disguise Reveal", desc: "Enemy NPC with IsBehaviorHostile=false starts as Civilian TeamId. Flips to hostile via interface when triggered by dialogue." },
{ title: "Forced State", desc: "ConvAgent LLM can invoke tools that call ForceState(Scripted) or ForceState(Fleeing) for narrative control." },
];
integrations.forEach((item, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.5 + col * 6.3;
const y = 2.2 + row * 2.3;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 6.0, h: 2.1, fill: { color: SAND }, line: { color: TERRA, width: 1 }, rectRadius: 0.1 });
s.addText(item.title, { x: x + 0.2, y: y + 0.15, w: 5.6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: x + 0.2, y: y + 0.58, w: 0.4, h: 0.04, fill: { color: GOLD } });
s.addText(item.desc, { x: x + 0.2, y: y + 0.7, w: 5.6, h: 1.3, fontSize: 12, color: CHARCOAL, fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 20 — DEBUG TOOLS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Debug Tools", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: TERRA, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: GOLD } });
s.addText("Visualize state, perception, and targeting at runtime.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: CHARCOAL, italic: true, fontFace: FONT_B });
s.addText("Per-NPC Debug", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "PersonalityComponent.bDebug — floating text", options: { bullet: true, fontSize: 13 } },
{ text: "Name, NPCType, TeamId, State", options: { bullet: true, fontSize: 13 } },
{ text: "ThreatLevel, ThreatActor", options: { bullet: true, fontSize: 13 } },
{ text: "Hostile, active spline", options: { bullet: true, fontSize: 13 } },
{ text: "SplineFollower debug sphere (green)", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 2.8, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Global Logging", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: TERRA, fontFace: FONT_H });
s.addText([
{ text: "CVar: ps.ai.Behavior.Debug 1", options: { bullet: true, fontSize: 13 } },
{ text: "Toggles verbose logs globally", options: { bullet: true, fontSize: 13 } },
{ text: "LogPS_AI_Behavior category", options: { bullet: true, fontSize: 13 } },
{ text: "BTView / AIDebug (Unreal built-in)", options: { bullet: true, fontSize: 13 } },
{ text: "EQS Visualizer for cover queries", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 2.8, color: CHARCOAL, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.5, w: 12.3, h: 1.5, fill: { color: TERRA }, rectRadius: 0.1 });
s.addText("Pro tip", { x: 0.7, y: 5.6, w: 12, h: 0.35, fontSize: 14, bold: true, color: GOLD, fontFace: FONT_H });
s.addText("Enable bDebug only on specific NPCs (or via a debug HUD key). Global debug logs can be overwhelming in crowded scenes. The floating text is cheap but accumulates.",
{ x: 0.7, y: 5.95, w: 12, h: 1.0, fontSize: 12, color: SAND, italic: true, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 21 — SUMMARY
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide();
s.background = { color: TERRA };
s.addShape(pres.ShapeType.rect, { x: 0, y: 2.5, w: 13.333, h: 0.05, fill: { color: GOLD } });
s.addText("Summary", { x: 0.6, y: 0.8, w: 12, h: 1.2, fontSize: 54, bold: true, color: WHITE, fontFace: FONT_H });
s.addText("Complete NPC behavior stack for tactical game AI.", { x: 0.6, y: 2.0, w: 12, h: 0.5, fontSize: 18, color: SAND, italic: true, fontFace: FONT_B });
const bsummary = [
{ h: "Personality-driven", t: "Three traits modulate every decision" },
{ h: "Cover-aware", t: "Manual CoverPoints + procedural fallback + EQS" },
{ h: "Spline navigation", t: "Network with junction detection, type filtering" },
{ h: "Perception built-in", t: "Sight + hearing + damage, priority-based threat" },
{ h: "Server-authoritative", t: "Minimal replication, client just animates" },
{ h: "Plugin-friendly", t: "Gaze bridge to ConvAgent, interface-only dependency" },
];
bsummary.forEach((item, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.6 + col * 6.2;
const y = 3.0 + row * 1.4;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 5.9, h: 1.2, fill: { color: TERRA_D }, line: { color: GOLD, width: 1 }, rectRadius: 0.1 });
s.addText(item.h, { x: x + 0.2, y: y + 0.1, w: 5.5, h: 0.4, fontSize: 18, bold: true, color: GOLD, fontFace: FONT_H });
s.addText(item.t, { x: x + 0.2, y: y + 0.55, w: 5.5, h: 0.6, fontSize: 12, color: SAND, italic: true, fontFace: FONT_B });
});
s.addText("ASTERION • PS_AI_Behavior • 2026", { x: 0.6, y: 6.9, w: 12, h: 0.4, fontSize: 12, color: SAND, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SAVE
// ═══════════════════════════════════════════════════════════════════
pres.writeFile({ fileName: "PS_AI_Behavior_Documentation.pptx" })
.then(name => console.log(`Generated: ${name}`))
.catch(err => { console.error(err); process.exit(1); });

711
Docs/gen_convagent_doc.js Normal file
View File

@@ -0,0 +1,711 @@
// PS_AI_ConvAgent User Documentation Generator
// Palette: Midnight Executive (navy #1E2761, ice blue #CADCFC, white)
const pptxgen = require("pptxgenjs");
const pres = new pptxgen();
pres.layout = "LAYOUT_WIDE"; // 13.333 x 7.5 inch
// ─── Color Palette ─────────────────────────────────
const NAVY = "1E2761";
const NAVY_DARK = "141B45";
const ICE = "CADCFC";
const ICE_DEEP = "7892D6";
const WHITE = "FFFFFF";
const GREY = "3A3F5C";
const LIGHT_GREY= "F5F7FC";
const ACCENT = "F9A825"; // warm gold accent
// ─── Font ──────────────────────────────────────────
const FONT_H = "Calibri";
const FONT_B = "Calibri";
// ─── Master slide backgrounds ──────────────────────
pres.defineSlideMaster({
title: "CONTENT",
background: { color: LIGHT_GREY },
objects: [
{ rect: { x: 0, y: 0, w: 13.333, h: 0.35, fill: { color: NAVY } } },
{ rect: { x: 0, y: 7.15, w: 13.333, h: 0.35, fill: { color: NAVY } } },
{ text: {
text: "PS_AI_ConvAgent — User Documentation",
options: { x: 0.3, y: 7.18, w: 8, h: 0.3, fontSize: 10, color: ICE, fontFace: FONT_B }
} },
{ text: {
text: "ASTERION",
options: { x: 11.5, y: 7.18, w: 1.5, h: 0.3, fontSize: 10, color: ICE, fontFace: FONT_B, align: "right", bold: true }
} },
],
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 1 — TITLE
// ═══════════════════════════════════════════════════════════════════
let s = pres.addSlide();
s.background = { color: NAVY };
s.addShape(pres.ShapeType.rect, { x: 0, y: 3.3, w: 13.333, h: 0.05, fill: { color: ACCENT } });
s.addText("PS_AI_ConvAgent", { x: 0.6, y: 2.2, w: 12, h: 1.2, fontSize: 60, bold: true, color: WHITE, fontFace: FONT_H });
s.addText("Conversational AI for MetaHumans in Unreal Engine", { x: 0.6, y: 3.5, w: 12, h: 0.6, fontSize: 22, color: ICE, fontFace: FONT_B, italic: true });
s.addText("User Documentation", { x: 0.6, y: 4.4, w: 12, h: 0.5, fontSize: 18, color: ICE_DEEP, fontFace: FONT_B });
s.addText("ElevenLabs • MetaHuman • Listen Server", { x: 0.6, y: 5.2, w: 12, h: 0.4, fontSize: 14, color: ICE_DEEP, fontFace: FONT_B });
s.addText("ASTERION • 2026", { x: 0.6, y: 6.6, w: 12, h: 0.4, fontSize: 12, color: ICE_DEEP, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 2 — OVERVIEW
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Plugin Overview", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("What it does", { x: 0.5, y: 1.7, w: 6, h: 0.4, fontSize: 18, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "Full conversational AI NPC integration for Unreal Engine", options: { bullet: true, fontSize: 14 } },
{ text: "Real-time voice capture, turn-taking, TTS playback", options: { bullet: true, fontSize: 14 } },
{ text: "Emotion-driven MetaHuman animations (face, body, gaze)", options: { bullet: true, fontSize: 14 } },
{ text: "Multiplayer support with speaker arbitration", options: { bullet: true, fontSize: 14 } },
{ text: "Tool system for LLM-triggered game actions", options: { bullet: true, fontSize: 14 } },
], { x: 0.5, y: 2.1, w: 6, h: 2.6, color: GREY, fontFace: FONT_B, paraSpaceAfter: 8 });
s.addShape(pres.ShapeType.roundRect, { x: 7, y: 1.7, w: 5.8, h: 4.8, fill: { color: NAVY }, rectRadius: 0.1 });
s.addText("Primary Use Case", { x: 7.2, y: 1.9, w: 5.4, h: 0.4, fontSize: 16, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText("Interactive, emotionally expressive voice NPCs that listen to players, respond with contextual answers, express emotion through facial animation and body language, and trigger physical actions based on conversation flow.", {
x: 7.2, y: 2.4, w: 5.4, h: 1.7, fontSize: 13, color: ICE, fontFace: FONT_B, italic: true
});
s.addText("Key Differentiators", { x: 7.2, y: 4.2, w: 5.4, h: 0.4, fontSize: 16, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText([
{ text: "Full-duplex Server VAD or push-to-talk", options: { bullet: true, fontSize: 12 } },
{ text: "Adaptive pre-buffering & speculative TTS", options: { bullet: true, fontSize: 12 } },
{ text: "Spectral lip sync + artist-driven poses", options: { bullet: true, fontSize: 12 } },
{ text: "Opus codec + distance LOD for multiplayer", options: { bullet: true, fontSize: 12 } },
], { x: 7.2, y: 4.7, w: 5.4, h: 1.7, color: ICE, fontFace: FONT_B, paraSpaceAfter: 4 });
s.addText("\"Drop components on a MetaHuman, configure a few data assets, and talk.\"", {
x: 0.5, y: 5.3, w: 6, h: 0.6, fontSize: 13, color: NAVY, italic: true, fontFace: FONT_B, bold: true
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 3 — ARCHITECTURE DIAGRAM
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Architecture", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Component chain on a MetaHuman NPC", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
// Input row
const boxY = 2.4;
const boxH = 1.0;
const boxes = [
{ x: 0.5, label: "Microphone\nCapture", sub: "VAD • 16kHz" },
{ x: 2.6, label: "Interaction\nComponent", sub: "Agent select\n(player side)" },
{ x: 4.7, label: "ElevenLabs\nComponent", sub: "WebSocket\nTurn-taking" },
{ x: 6.8, label: "Lip Sync\nComponent", sub: "Spectral\nFFT → visemes" },
{ x: 8.9, label: "Facial Expr.\nComponent", sub: "Emotion\ncurves" },
{ x: 11.0, label: "Body Expr.\nComponent", sub: "Gesture\nanim blend" },
];
boxes.forEach((b, i) => {
s.addShape(pres.ShapeType.roundRect, { x: b.x, y: boxY, w: 2.0, h: boxH, fill: { color: NAVY }, rectRadius: 0.1 });
s.addText(b.label, { x: b.x, y: boxY + 0.1, w: 2.0, h: 0.5, fontSize: 13, bold: true, color: WHITE, align: "center", fontFace: FONT_H });
s.addText(b.sub, { x: b.x, y: boxY + 0.58, w: 2.0, h: 0.4, fontSize: 9, color: ICE, align: "center", fontFace: FONT_B });
});
// Arrows between boxes
for (let i = 0; i < boxes.length - 1; i++) {
const ax = boxes[i].x + 2.0;
s.addShape(pres.ShapeType.line, { x: ax, y: boxY + 0.5, w: 0.1, h: 0, line: { color: ACCENT, width: 2 } });
}
// Gaze row
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 4.0, w: 2.0, h: boxH, fill: { color: ICE_DEEP }, rectRadius: 0.1 });
s.addText("Gaze\nComponent", { x: 0.5, y: 4.1, w: 2.0, h: 0.5, fontSize: 13, bold: true, color: WHITE, align: "center", fontFace: FONT_H });
s.addText("Eyes • Head\n• Body", { x: 0.5, y: 4.58, w: 2.0, h: 0.4, fontSize: 9, color: WHITE, align: "center", fontFace: FONT_B });
// AnimGraph pipeline
s.addText("→ AnimGraph (Face + Body)", { x: 2.8, y: 4.2, w: 4.5, h: 0.5, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText("FacialExpression → LipSync | Gaze → BodyExpression", { x: 2.8, y: 4.65, w: 8, h: 0.4, fontSize: 12, color: GREY, italic: true, fontFace: FONT_B });
// Project settings layer
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.6, w: 12.3, h: 0.9, fill: { color: LIGHT_GREY }, line: { color: NAVY, width: 1 }, rectRadius: 0.08 });
s.addText("PROJECT SETTINGS — API Key • Server Region • Global Context Prompt", { x: 0.7, y: 5.75, w: 12, h: 0.3, fontSize: 13, bold: true, color: NAVY, fontFace: FONT_H });
s.addText("Data Assets: AgentConfig • EmotionPoseMap • LipSyncPoseMap • BodyPoseMap • ActionSet • Tool", { x: 0.7, y: 6.1, w: 12, h: 0.3, fontSize: 11, color: GREY, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 4 — ELEVENLABS COMPONENT
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("ElevenLabs Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("The core conversation engine — WebSocket, turn-taking, audio pipeline.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("Key Properties", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "AgentConfig — identity, voice, tools", options: { bullet: true, fontSize: 12 } },
{ text: "TurnMode — ServerVAD / Client (push-to-talk)", options: { bullet: true, fontSize: 12 } },
{ text: "MicChunkDurationMs (50250 ms)", options: { bullet: true, fontSize: 12 } },
{ text: "AudioPreBufferMs / bAdaptivePreBuffer", options: { bullet: true, fontSize: 12 } },
{ text: "bAllowInterruption, bPersistentSession", options: { bullet: true, fontSize: 12 } },
{ text: "SoundAttenuation (optional 3D audio)", options: { bullet: true, fontSize: 12 } },
{ text: "AudioLODCullDistance / LipSyncLODDistance", options: { bullet: true, fontSize: 12 } },
], { x: 0.5, y: 2.55, w: 6, h: 3.6, color: GREY, fontFace: FONT_B, paraSpaceAfter: 4 });
s.addText("Key Methods", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "StartConversation() / EndConversation()", options: { bullet: true, fontSize: 12 } },
{ text: "StartListening() / StopListening()", options: { bullet: true, fontSize: 12 } },
{ text: "SendTextMessage() — text without voice", options: { bullet: true, fontSize: 12 } },
{ text: "InterruptAgent() — force stop mid-speech", options: { bullet: true, fontSize: 12 } },
{ text: "ForceDisableConversation(Action) — blend to neutral", options: { bullet: true, fontSize: 12 } },
{ text: "ForceEnableConversation() — re-enable", options: { bullet: true, fontSize: 12 } },
{ text: "FeedExternalAudio() — external mic source", options: { bullet: true, fontSize: 12 } },
], { x: 7, y: 2.55, w: 6, h: 3.6, color: GREY, fontFace: FONT_B, paraSpaceAfter: 4 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 6.2, w: 12.3, h: 0.75, fill: { color: NAVY }, rectRadius: 0.08 });
s.addText("Major events: OnAgentConnected • OnAgentTranscript • OnAgentTextResponse • OnAgentStartedSpeaking • OnAgentEmotionChanged • OnAgentActionRequested • OnActiveSpeakerChanged • OnReadyForAction", { x: 0.7, y: 6.3, w: 12, h: 0.55, fontSize: 11, color: ICE, fontFace: FONT_B, italic: true });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 5 — MICROPHONE CAPTURE
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Microphone Capture Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 34, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Captures, resamples, and performs local voice activity detection.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("Audio Pipeline", { x: 0.5, y: 2.2, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "WASAPI 48kHz stereo float32 input", options: { bullet: true, fontSize: 13 } },
{ text: "Resample to 16kHz mono float", options: { bullet: true, fontSize: 13 } },
{ text: "Convert to int16 LE byte stream", options: { bullet: true, fontSize: 13 } },
{ text: "Delivered via multicast delegate", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.6, w: 6, h: 2.4, color: GREY, fontFace: FONT_B, paraSpaceAfter: 6 });
s.addText("Voice Activity Detection", { x: 7, y: 2.2, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "VolumeMultiplier — mic gain", options: { bullet: true, fontSize: 13 } },
{ text: "VoiceActivityThreshold — RMS threshold", options: { bullet: true, fontSize: 13 } },
{ text: "VoiceOnsetTime — debounce on speech start", options: { bullet: true, fontSize: 13 } },
{ text: "VoiceSilenceTime — debounce on end", options: { bullet: true, fontSize: 13 } },
{ text: "OnUserVoiceActivityChanged event", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.6, w: 6, h: 2.4, color: GREY, fontFace: FONT_B, paraSpaceAfter: 6 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.4, w: 12.3, h: 1.4, fill: { color: LIGHT_GREY }, line: { color: NAVY, width: 1 }, rectRadius: 0.08 });
s.addText("Public API", { x: 0.7, y: 5.5, w: 12, h: 0.3, fontSize: 13, bold: true, color: NAVY, fontFace: FONT_H });
s.addText("StartCapture() • StopCapture() • IsCapturing()", { x: 0.7, y: 5.85, w: 12, h: 0.3, fontSize: 13, color: GREY, fontFace: "Consolas" });
s.addText("The component auto-opens the default input device. Output is consumed by ElevenLabsComponent (or InteractionComponent, which relays via Server RPC in networked games).", { x: 0.7, y: 6.2, w: 12, h: 0.55, fontSize: 11, color: GREY, italic: true, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 6 — LIP SYNC
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Lip Sync Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Spectral analysis of agent audio → 15 OVR visemes → ARKit blendshapes.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("How it works", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "FFT on agent audio stream", options: { bullet: true, fontSize: 13 } },
{ text: "Estimates 15 OVR viseme weights", options: { bullet: true, fontSize: 13 } },
{ text: "Maps to ARKit blendshapes (jawOpen, mouthFunnel…)", options: { bullet: true, fontSize: 13 } },
{ text: "Optional: custom PoseMap with artist-crafted poses", options: { bullet: true, fontSize: 13 } },
{ text: "Thread-safe curve output for AnimGraph", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 3.5, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Key Properties", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "LipSyncStrength — overall intensity", options: { bullet: true, fontSize: 13 } },
{ text: "SmoothingSpeed (3565) — viseme interp", options: { bullet: true, fontSize: 13 } },
{ text: "SpeechBlendDuration — fade in/out", options: { bullet: true, fontSize: 13 } },
{ text: "EmotionExpressionBlend (01) — emotion bleed through mouth", options: { bullet: true, fontSize: 13 } },
{ text: "EnvelopeAttackMs / EnvelopeReleaseMs", options: { bullet: true, fontSize: 13 } },
{ text: "PoseMap — optional artist override", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 3.5, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 6.2, w: 12.3, h: 0.75, fill: { color: NAVY }, rectRadius: 0.08 });
s.addText("AnimGraph placement (Face AnimBP): Live Link → FacialExpression → LipSync → mh_arkit_mapping_pose", { x: 0.7, y: 6.3, w: 12, h: 0.55, fontSize: 12, color: ICE, fontFace: FONT_B, italic: true, bold: true });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 7 — FACIAL EXPRESSION
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Facial Expression Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 34, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Emotion-driven face animation via real-time AnimSequence sampling.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("Design", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "Plays AnimSequences in real-time (looping)", options: { bullet: true, fontSize: 13 } },
{ text: "Extracts emotion curves (eyes, brows, cheeks, mouth mood)", options: { bullet: true, fontSize: 13 } },
{ text: "Injects into AnimGraph via FacialExpression AnimNode", options: { bullet: true, fontSize: 13 } },
{ text: "Lip sync overrides mouth area during speech", options: { bullet: true, fontSize: 13 } },
{ text: "3 intensity levels: Normal / Medium / Extreme", options: { bullet: true, fontSize: 13 } },
{ text: "Crossfades between emotions", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 3.5, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Properties", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "EmotionPoseMap — data asset of AnimSequences", options: { bullet: true, fontSize: 13 } },
{ text: "ActivationBlendDuration (default 0.5s)", options: { bullet: true, fontSize: 13 } },
{ text: "EmotionBlendDuration — crossfade time", options: { bullet: true, fontSize: 13 } },
{ text: "bActive — auto-managed by conversation", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 2.4, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addShape(pres.ShapeType.roundRect, { x: 7, y: 5.0, w: 5.8, h: 1.5, fill: { color: LIGHT_GREY }, line: { color: NAVY, width: 1 }, rectRadius: 0.08 });
s.addText("Emotions covered", { x: 7.15, y: 5.1, w: 5.6, h: 0.3, fontSize: 13, bold: true, color: NAVY, fontFace: FONT_H });
s.addText("Joy • Sadness • Anger • Fear • Surprise • Disgust • Neutral", { x: 7.15, y: 5.45, w: 5.6, h: 0.9, fontSize: 12, color: GREY, italic: true, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 8 — BODY EXPRESSION
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Body Expression Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 34, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Upper-body gestures and posture driven by emotion + speaking state.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("Animation lists per emotion", { x: 0.5, y: 2.15, w: 6.3, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
const animLists = [
{ name: "Idle", desc: "Played while listening (not speaking)" },
{ name: "Normal", desc: "Low-intensity speaking anims" },
{ name: "Medium", desc: "Medium-intensity speaking" },
{ name: "Extreme", desc: "High-intensity speaking" },
];
animLists.forEach((a, i) => {
const y = 2.7 + i * 0.75;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 1.5, h: 0.55, fill: { color: NAVY }, rectRadius: 0.05 });
s.addText(a.name, { x: 0.5, y: y, w: 1.5, h: 0.55, fontSize: 13, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
s.addText(a.desc, { x: 2.15, y: y + 0.05, w: 4.6, h: 0.45, fontSize: 12, color: GREY, valign: "middle", fontFace: FONT_B });
});
s.addText("Key Properties", { x: 7.2, y: 2.15, w: 5.8, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "BodyPoseMap — emotion → anim lists", options: { bullet: true, fontSize: 13 } },
{ text: "ActivationBlendDuration (0.5s)", options: { bullet: true, fontSize: 13 } },
{ text: "EmotionBlendDuration (1.0s)", options: { bullet: true, fontSize: 13 } },
{ text: "BlendWeight (01)", options: { bullet: true, fontSize: 13 } },
], { x: 7.2, y: 2.55, w: 5.8, h: 2, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("AnimNode behaviour", { x: 7.2, y: 4.7, w: 5.8, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "Override mode, per-bone mask", options: { bullet: true, fontSize: 12 } },
{ text: "Default BlendRootBone = spine_02", options: { bullet: true, fontSize: 12 } },
{ text: "Excludes neck_01 (Gaze handles it)", options: { bullet: true, fontSize: 12 } },
{ text: "Lower body passes through upstream", options: { bullet: true, fontSize: 12 } },
], { x: 7.2, y: 5.1, w: 5.8, h: 2, color: GREY, fontFace: FONT_B, paraSpaceAfter: 3 });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 9 — GAZE
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Gaze Component", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Multi-layer look-at: body • head • eyes.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
const layers = [
{ name: "BODY", ratio: "full 360°", desc: "Actor yaw — follows target around the character" },
{ name: "HEAD", ratio: "limited", desc: "Head bone yaw + pitch, clamped" },
{ name: "EYES", ratio: "limited", desc: "ARKit eye curves, fine targeting" },
];
layers.forEach((l, i) => {
const y = 2.3 + i * 1.1;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 2.2, h: 0.9, fill: { color: NAVY }, rectRadius: 0.08 });
s.addText(l.name, { x: 0.5, y: y + 0.05, w: 2.2, h: 0.4, fontSize: 18, bold: true, color: WHITE, align: "center", fontFace: FONT_H });
s.addText(l.ratio, { x: 0.5, y: y + 0.48, w: 2.2, h: 0.3, fontSize: 11, color: ACCENT, align: "center", italic: true, fontFace: FONT_B });
s.addText(l.desc, { x: 3.0, y: y + 0.15, w: 5.0, h: 0.7, fontSize: 13, color: GREY, valign: "middle", fontFace: FONT_B });
});
s.addText("Key Properties", { x: 8.5, y: 2.15, w: 4.5, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "TargetActor — who to look at", options: { bullet: true, fontSize: 12 } },
{ text: "MaxHeadYaw / MaxHeadPitch", options: { bullet: true, fontSize: 12 } },
{ text: "MaxEyeHorizontal / MaxEyeVertical", options: { bullet: true, fontSize: 12 } },
{ text: "BodyInterpSpeed / HeadInterpSpeed / EyeInterpSpeed", options: { bullet: true, fontSize: 12 } },
{ text: "HeadAnimationCompensation (0.9)", options: { bullet: true, fontSize: 12 } },
{ text: "EyeAnimationCompensation (0.6)", options: { bullet: true, fontSize: 12 } },
{ text: "BodyDriftCompensation (0.8)", options: { bullet: true, fontSize: 12 } },
{ text: "NeckBoneChain — multi-bone distribution", options: { bullet: true, fontSize: 12 } },
], { x: 8.5, y: 2.55, w: 4.5, h: 4, color: GREY, fontFace: FONT_B, paraSpaceAfter: 3 });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 10 — INTERACTION COMPONENT
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Interaction Component (Player)", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 32, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Placed on the player pawn — selects NPCs, routes mic, manages gaze.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("Selection Logic", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "Proximity + view cone detection", options: { bullet: true, fontSize: 13 } },
{ text: "MaxInteractionDistance (300 cm default)", options: { bullet: true, fontSize: 13 } },
{ text: "ViewConeHalfAngle (45° default)", options: { bullet: true, fontSize: 13 } },
{ text: "SelectionStickyAngle — hysteresis", options: { bullet: true, fontSize: 13 } },
{ text: "ConversationSwitchDelay — anti-flicker", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 3, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Auto-management", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "bAutoStartConversation — open WS on select", options: { bullet: true, fontSize: 13 } },
{ text: "bAutoManageListening — mic open/close", options: { bullet: true, fontSize: 13 } },
{ text: "bAutoManageGaze — set agent gaze to player", options: { bullet: true, fontSize: 13 } },
{ text: "bAutoPassiveGaze — out-of-range passive tracking", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 2.5, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.8, w: 12.3, h: 1.2, fill: { color: NAVY }, rectRadius: 0.08 });
s.addText("Events: OnAgentSelected • OnAgentDeselected • OnNoAgentInRange", { x: 0.7, y: 5.95, w: 12, h: 0.3, fontSize: 13, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText("Methods: GetSelectedAgent() • ForceSelectAgent() • StartConversationWithSelectedAgent() • SendTextToSelectedAgent() • ClearSelection()", { x: 0.7, y: 6.3, w: 12, h: 0.55, fontSize: 11, color: ICE, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 11 — DATA ASSETS OVERVIEW
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Data Assets", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Reusable configuration assets for agent identity, animations, and tools.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
const dataAssets = [
{ name: "AgentConfig", desc: "Agent identity, voice, LLM model, tools", color: NAVY },
{ name: "EmotionPoseMap", desc: "Emotion → facial AnimSequences (3 levels)", color: NAVY },
{ name: "LipSyncPoseMap", desc: "15 OVR visemes → mouth poses", color: NAVY },
{ name: "BodyPoseMap", desc: "Emotion → body gesture anim lists", color: NAVY },
{ name: "ActionSet", desc: "Actions the agent can trigger", color: ICE_DEEP },
{ name: "Tool", desc: "Custom LLM tools with parameters", color: ICE_DEEP },
];
dataAssets.forEach((d, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.5 + col * 6.3;
const y = 2.2 + row * 1.5;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 6.0, h: 1.3, fill: { color: d.color }, rectRadius: 0.1 });
s.addText(d.name, { x: x + 0.2, y: y + 0.2, w: 5.6, h: 0.4, fontSize: 18, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText(d.desc, { x: x + 0.2, y: y + 0.65, w: 5.6, h: 0.6, fontSize: 12, color: ICE, italic: true, fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 12 — AGENT CONFIG DETAILS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("AgentConfig (ElevenLabs)", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 34, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Complete agent definition — editable in Content Browser with API buttons.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
const configCats = [
{ title: "Identity", items: ["AgentID", "AgentName"] },
{ title: "Voice", items: ["VoiceID / VoiceName", "TTSModelID", "Stability • Similarity • Speed"] },
{ title: "LLM", items: ["LLMModel (gemini-2.5-flash default)", "Language (en default)"] },
{ title: "Behavior", items: ["CharacterPrompt (personality)", "FirstMessage (greeting)", "TurnTimeout, MaxTurns"] },
{ title: "Latency", items: ["TurnEagerness (Eager/Normal/Patient)", "bSpeculativeTurn"] },
{ title: "Tools", items: ["Tools[] array of Tool assets", "ActionSet integration"] },
];
configCats.forEach((c, i) => {
const col = i % 3;
const row = Math.floor(i / 3);
const x = 0.5 + col * 4.15;
const y = 2.2 + row * 2.15;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 3.95, h: 1.95, fill: { color: LIGHT_GREY }, line: { color: NAVY, width: 1 }, rectRadius: 0.08 });
s.addText(c.title, { x: x + 0.15, y: y + 0.1, w: 3.65, h: 0.35, fontSize: 14, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: x + 0.15, y: y + 0.45, w: 0.35, h: 0.04, fill: { color: ACCENT } });
s.addText(c.items.map(t => ({ text: t, options: { bullet: true, fontSize: 11 } })),
{ x: x + 0.15, y: y + 0.55, w: 3.65, h: 1.3, color: GREY, fontFace: FONT_B, paraSpaceAfter: 2 });
});
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 6.55, w: 12.3, h: 0.45, fill: { color: NAVY }, rectRadius: 0.05 });
s.addText("Editor: Voice picker • Model picker • LLM picker • Create / Update / Fetch Agent buttons", { x: 0.7, y: 6.6, w: 12, h: 0.35, fontSize: 11, color: ICE, fontFace: FONT_B, italic: true, bold: true });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 13 — POSE MAPS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Pose Maps", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Three pose-map data assets map abstract states → AnimSequences.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
// Emotion
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 2.15, w: 4.0, h: 4.5, fill: { color: NAVY }, rectRadius: 0.1 });
s.addText("EmotionPoseMap", { x: 0.6, y: 2.3, w: 3.8, h: 0.4, fontSize: 17, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText("Emotion → facial AnimSequences\n(3 intensity levels per emotion)", { x: 0.6, y: 2.7, w: 3.8, h: 0.8, fontSize: 11, color: ICE, italic: true, fontFace: FONT_B });
s.addText([
{ text: "Joy (N / M / E)", options: { bullet: true, fontSize: 11 } },
{ text: "Sadness (N / M / E)", options: { bullet: true, fontSize: 11 } },
{ text: "Anger (N / M / E)", options: { bullet: true, fontSize: 11 } },
{ text: "Fear (N / M / E)", options: { bullet: true, fontSize: 11 } },
{ text: "Surprise (N / M / E)", options: { bullet: true, fontSize: 11 } },
{ text: "Disgust (N / M / E)", options: { bullet: true, fontSize: 11 } },
{ text: "Neutral (N / M / E)", options: { bullet: true, fontSize: 11 } },
], { x: 0.7, y: 3.7, w: 3.7, h: 2.8, color: ICE, fontFace: FONT_B, paraSpaceAfter: 3 });
// LipSync
s.addShape(pres.ShapeType.roundRect, { x: 4.7, y: 2.15, w: 4.0, h: 4.5, fill: { color: NAVY }, rectRadius: 0.1 });
s.addText("LipSyncPoseMap", { x: 4.8, y: 2.3, w: 3.8, h: 0.4, fontSize: 17, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText("15 OVR visemes →\nphoneme AnimSequences", { x: 4.8, y: 2.7, w: 3.8, h: 0.7, fontSize: 11, color: ICE, italic: true, fontFace: FONT_B });
s.addText([
{ text: "sil • PP • FF • TH • DD", options: { bullet: true, fontSize: 11 } },
{ text: "kk • CH • SS • nn • RR", options: { bullet: true, fontSize: 11 } },
{ text: "aa • E • ih • oh • ou", options: { bullet: true, fontSize: 11 } },
{ text: "MetaHuman MHF_* recommended", options: { bullet: true, fontSize: 11 } },
{ text: "Shareable across characters", options: { bullet: true, fontSize: 11 } },
], { x: 4.9, y: 3.5, w: 3.7, h: 2.9, color: ICE, fontFace: FONT_B, paraSpaceAfter: 4 });
// Body
s.addShape(pres.ShapeType.roundRect, { x: 8.9, y: 2.15, w: 4.0, h: 4.5, fill: { color: NAVY }, rectRadius: 0.1 });
s.addText("BodyPoseMap", { x: 9.0, y: 2.3, w: 3.8, h: 0.4, fontSize: 17, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText("Emotion → body anim lists\n(Idle / N / M / E per emotion)", { x: 9.0, y: 2.7, w: 3.8, h: 0.8, fontSize: 11, color: ICE, italic: true, fontFace: FONT_B });
s.addText([
{ text: "Idle: listening animations", options: { bullet: true, fontSize: 11 } },
{ text: "Normal: light speaking", options: { bullet: true, fontSize: 11 } },
{ text: "Medium: expressive speaking", options: { bullet: true, fontSize: 11 } },
{ text: "Extreme: emphatic speaking", options: { bullet: true, fontSize: 11 } },
{ text: "Random selection per loop", options: { bullet: true, fontSize: 11 } },
{ text: "Continuous variety", options: { bullet: true, fontSize: 11 } },
], { x: 9.0, y: 3.7, w: 3.8, h: 2.8, color: ICE, fontFace: FONT_B, paraSpaceAfter: 3 });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 14 — TOOLS & ACTIONSETS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Tools & ActionSets", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("LLM-triggered actions that bridge conversation → gameplay.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("Tool asset", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "ToolName — snake_case identifier", options: { bullet: true, fontSize: 13 } },
{ text: "ToolDescription — when LLM should use it", options: { bullet: true, fontSize: 13 } },
{ text: "PromptFragment — appended system prompt", options: { bullet: true, fontSize: 13 } },
{ text: "Parameters[] — typed LLM arguments", options: { bullet: true, fontSize: 13 } },
{ text: "ToolID auto-populated on Create", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 2.8, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Built-in Tools", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "set_emotion(emotion, intensity)", options: { bullet: true, fontSize: 13 } },
{ text: "perform_action(action) — from ActionSet", options: { bullet: true, fontSize: 13 } },
{ text: "Custom tools — any parameters", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 2, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 5.4, w: 12.3, h: 1.6, fill: { color: LIGHT_GREY }, line: { color: NAVY, width: 1 }, rectRadius: 0.08 });
s.addText("Action flow", { x: 0.7, y: 5.5, w: 12, h: 0.3, fontSize: 14, bold: true, color: NAVY, fontFace: FONT_H });
s.addText("1. LLM decides to call a tool (e.g. perform_action(\"flee\"))\n2. Plugin fires OnAgentActionRequested(\"flee\")\n3. Game calls ForceDisableConversation(\"flee\") → animations blend to neutral\n4. OnReadyForAction(\"flee\") fires → play gameplay montage\n5. Game calls ForceEnableConversation() when done", { x: 0.7, y: 5.85, w: 12, h: 1.1, fontSize: 11, color: GREY, fontFace: "Consolas" });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 15 — ANIMGRAPH INTEGRATION
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("AnimGraph Integration", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 34, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Four AnimNodes auto-discover components — no manual wiring.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
// Face AnimBP chain
s.addText("Face AnimBP", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
const faceChain = [
{ label: "Live Link", color: ICE_DEEP },
{ label: "Facial\nExpression", color: "E88B3E" },
{ label: "Gaze\n(eyes only)", color: "5E7AC7" },
{ label: "Lip Sync", color: "3EB086" },
{ label: "mh_arkit\n_mapping", color: GREY },
];
faceChain.forEach((n, i) => {
const x = 0.5 + i * 2.55;
s.addShape(pres.ShapeType.roundRect, { x: x, y: 2.65, w: 2.3, h: 1.0, fill: { color: n.color }, rectRadius: 0.1 });
s.addText(n.label, { x: x, y: 2.75, w: 2.3, h: 0.8, fontSize: 12, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
});
// Body AnimBP chain
s.addText("Body AnimBP", { x: 0.5, y: 4.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
const bodyChain = [
{ label: "Locomotion\n/ Idle", color: ICE_DEEP },
{ label: "Gaze\n(head rot)", color: "5E7AC7" },
{ label: "Body\nExpression", color: "B85042" },
{ label: "Output Pose", color: GREY },
];
bodyChain.forEach((n, i) => {
const x = 0.5 + i * 3.2;
s.addShape(pres.ShapeType.roundRect, { x: x, y: 4.65, w: 2.9, h: 1.0, fill: { color: n.color }, rectRadius: 0.1 });
s.addText(n.label, { x: x, y: 4.75, w: 2.9, h: 0.8, fontSize: 12, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: FONT_H });
});
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: 6.15, w: 12.3, h: 0.85, fill: { color: NAVY }, rectRadius: 0.08 });
s.addText("Why this order?", { x: 0.7, y: 6.2, w: 12, h: 0.3, fontSize: 13, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText("Emotion curves set mood (brows, eyes, cheeks). Lip sync overrides mouth during speech. Gaze head rotation runs before body expression so it doesn't fight arms.", { x: 0.7, y: 6.55, w: 12, h: 0.4, fontSize: 11, color: ICE, italic: true, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 16 — PROJECT SETTINGS
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Project Settings", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Project Settings → Plugins → PS AI ConvAgent - ElevenLabs", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
const settings = [
{ name: "API_Key", desc: "ElevenLabs API key — required for authentication" },
{ name: "ServerRegion", desc: "Global / US / EU / India" },
{ name: "CustomWebSocketURL", desc: "Override WebSocket URL (advanced)" },
{ name: "bVerboseLogging", desc: "Verbose WebSocket logging (development)" },
{ name: "GlobalContextPrompt", desc: "Shared context injected as dynamic variables for all agents" },
];
settings.forEach((setting, i) => {
const y = 2.3 + i * 0.85;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 3.5, h: 0.65, fill: { color: NAVY }, rectRadius: 0.05 });
s.addText(setting.name, { x: 0.5, y: y, w: 3.5, h: 0.65, fontSize: 13, bold: true, color: ACCENT, align: "center", valign: "middle", fontFace: "Consolas" });
s.addText(setting.desc, { x: 4.2, y: y + 0.05, w: 8.6, h: 0.55, fontSize: 12, color: GREY, valign: "middle", fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 17 — SETUP WORKFLOW
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Setup Workflow", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("From zero to a talking MetaHuman in 5 steps.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
const steps = [
{ num: "1", title: "Project Settings", desc: "Enter API_Key, pick ServerRegion." },
{ num: "2", title: "Data Assets", desc: "Create AgentConfig, EmotionPoseMap, LipSyncPoseMap, BodyPoseMap." },
{ num: "3", title: "NPC Components", desc: "Add ElevenLabs + Mic + LipSync + FacialExpr + BodyExpr + Gaze on the MetaHuman." },
{ num: "4", title: "AnimGraphs", desc: "Wire AnimNodes in Face + Body AnimBPs. Components auto-discover." },
{ num: "5", title: "Player Pawn", desc: "Add InteractionComponent. Bind selection events to UI. Play." },
];
steps.forEach((step, i) => {
const y = 2.25 + i * 0.9;
s.addShape(pres.ShapeType.ellipse, { x: 0.5, y: y, w: 0.8, h: 0.8, fill: { color: ACCENT } });
s.addText(step.num, { x: 0.5, y: y, w: 0.8, h: 0.8, fontSize: 28, bold: true, color: NAVY, align: "center", valign: "middle", fontFace: FONT_H });
s.addShape(pres.ShapeType.roundRect, { x: 1.6, y: y, w: 11.2, h: 0.8, fill: { color: LIGHT_GREY }, line: { color: NAVY, width: 1 }, rectRadius: 0.05 });
s.addText(step.title, { x: 1.8, y: y + 0.1, w: 3.5, h: 0.3, fontSize: 14, bold: true, color: NAVY, fontFace: FONT_H });
s.addText(step.desc, { x: 1.8, y: y + 0.4, w: 11, h: 0.35, fontSize: 11, color: GREY, italic: true, fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 18 — NETWORKING
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Networking & Multiplayer", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 34, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Listen Server model — server owns WebSocket, clients relay via RPCs.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
s.addText("Audio Pipeline", { x: 0.5, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "Client mic → Server RPC (Opus-encoded)", options: { bullet: true, fontSize: 13 } },
{ text: "Server forwards to ElevenLabs WebSocket", options: { bullet: true, fontSize: 13 } },
{ text: "Agent audio → Multicast RPC to all clients", options: { bullet: true, fontSize: 13 } },
{ text: "Clients decode Opus → procedural sound wave", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 2.55, w: 6, h: 2.4, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Speaker Arbitration", { x: 7, y: 2.15, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "SpeakerSwitchHysteresis — min silence before switching", options: { bullet: true, fontSize: 13 } },
{ text: "SpeakerIdleTimeout — clear active speaker", options: { bullet: true, fontSize: 13 } },
{ text: "NetActiveSpeakerPawn replicated for gaze", options: { bullet: true, fontSize: 13 } },
{ text: "Only active speaker's audio forwarded to LLM", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 2.55, w: 6, h: 2.4, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Level of Detail (LOD)", { x: 0.5, y: 5.1, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "AudioLODCullDistance — skip audio at distance", options: { bullet: true, fontSize: 13 } },
{ text: "LipSyncLODDistance — skip lip sync at distance", options: { bullet: true, fontSize: 13 } },
{ text: "Active speaker always full quality", options: { bullet: true, fontSize: 13 } },
], { x: 0.5, y: 5.5, w: 6, h: 1.5, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
s.addText("Replicated State", { x: 7, y: 5.1, w: 6, h: 0.4, fontSize: 16, bold: true, color: NAVY, fontFace: FONT_H });
s.addText([
{ text: "bNetIsConversing", options: { bullet: true, fontSize: 13 } },
{ text: "NetConnectedPawns[]", options: { bullet: true, fontSize: 13 } },
{ text: "CurrentEmotion + CurrentEmotionIntensity", options: { bullet: true, fontSize: 13 } },
], { x: 7, y: 5.5, w: 6, h: 1.5, color: GREY, fontFace: FONT_B, paraSpaceAfter: 5 });
// ═══════════════════════════════════════════════════════════════════
// SLIDE 19 — FORCE DISABLE FLOW
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("ForceDisableConversation Flow", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 30, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Clean handoff between conversation and gameplay actions.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
const flow = [
{ step: "LLM Tool Call", desc: "perform_action(\"flee\")" },
{ step: "OnAgentActionRequested", desc: "Event fires on ElevenLabs component" },
{ step: "ForceDisableConversation", desc: "Blend out face/body/gaze animations" },
{ step: "OnReadyForAction", desc: "All components have reached neutral" },
{ step: "Play Gameplay", desc: "Montage / behavior switch" },
{ step: "ForceEnableConversation", desc: "Resume conversation when done" },
];
flow.forEach((f, i) => {
const y = 2.2 + i * 0.75;
s.addShape(pres.ShapeType.roundRect, { x: 0.5, y: y, w: 4, h: 0.6, fill: { color: NAVY }, rectRadius: 0.08 });
s.addText(f.step, { x: 0.5, y: y, w: 4, h: 0.6, fontSize: 13, bold: true, color: WHITE, align: "center", valign: "middle", fontFace: "Consolas" });
s.addText("→", { x: 4.6, y: y + 0.05, w: 0.5, h: 0.5, fontSize: 22, color: ACCENT, align: "center", valign: "middle", bold: true, fontFace: FONT_H });
s.addText(f.desc, { x: 5.2, y: y + 0.05, w: 7.6, h: 0.5, fontSize: 13, color: GREY, italic: true, valign: "middle", fontFace: FONT_B });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 20 — EVENTS CHEAT SHEET
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide({ masterName: "CONTENT" });
s.addText("Events Cheat Sheet", { x: 0.5, y: 0.6, w: 12, h: 0.8, fontSize: 36, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: 0.5, y: 1.45, w: 0.6, h: 0.06, fill: { color: ACCENT } });
s.addText("Blueprint-assignable delegates on ElevenLabs component.", { x: 0.5, y: 1.65, w: 12, h: 0.4, fontSize: 14, color: GREY, italic: true, fontFace: FONT_B });
const eventCols = [
{
title: "Connection",
items: ["OnAgentConnected", "OnAgentDisconnected", "OnAgentError"]
},
{
title: "Dialogue",
items: ["OnAgentTranscript", "OnAgentTextResponse", "OnAgentPartialResponse", "OnAgentStartedGenerating"]
},
{
title: "Audio",
items: ["OnAgentStartedSpeaking", "OnAudioPlaybackStarted", "OnAgentStoppedSpeaking", "OnAgentInterrupted"]
},
{
title: "State / Actions",
items: ["OnAgentEmotionChanged", "OnAgentActionRequested", "OnAgentClientToolCall", "OnActiveSpeakerChanged", "OnReadyForAction", "OnAgentResponseTimeout"]
},
];
eventCols.forEach((c, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.5 + col * 6.3;
const y = 2.15 + row * 2.45;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 6.0, h: 2.25, fill: { color: LIGHT_GREY }, line: { color: NAVY, width: 1 }, rectRadius: 0.08 });
s.addText(c.title, { x: x + 0.2, y: y + 0.1, w: 5.6, h: 0.3, fontSize: 14, bold: true, color: NAVY, fontFace: FONT_H });
s.addShape(pres.ShapeType.rect, { x: x + 0.2, y: y + 0.42, w: 0.35, h: 0.04, fill: { color: ACCENT } });
s.addText(c.items.map(t => ({ text: t, options: { bullet: true, fontSize: 11 } })),
{ x: x + 0.2, y: y + 0.52, w: 5.6, h: 1.7, color: GREY, fontFace: "Consolas", paraSpaceAfter: 2 });
});
// ═══════════════════════════════════════════════════════════════════
// SLIDE 21 — CLOSING / SUMMARY
// ═══════════════════════════════════════════════════════════════════
s = pres.addSlide();
s.background = { color: NAVY };
s.addShape(pres.ShapeType.rect, { x: 0, y: 2.5, w: 13.333, h: 0.05, fill: { color: ACCENT } });
s.addText("Summary", { x: 0.6, y: 0.8, w: 12, h: 1.2, fontSize: 54, bold: true, color: WHITE, fontFace: FONT_H });
s.addText("A complete conversational AI stack for Unreal Engine.", { x: 0.6, y: 2.0, w: 12, h: 0.5, fontSize: 18, color: ICE, italic: true, fontFace: FONT_B });
const summary = [
{ h: "Real-time", t: "Spectral lip sync, speculative TTS, adaptive pre-buffering" },
{ h: "Expressive", t: "Emotion-driven face/body animation via AnimSequences" },
{ h: "Multiplayer", t: "Listen Server + Opus codec + distance LOD + speaker arbitration" },
{ h: "Data-driven", t: "Reusable PoseMaps, AgentConfigs, Tools, ActionSets" },
{ h: "Bridgeable", t: "Behavior plugin gaze bridge, ForceDisable handoff for gameplay" },
{ h: "BP-friendly", t: "All components blueprintable, rich event system" },
];
summary.forEach((item, i) => {
const col = i % 2;
const row = Math.floor(i / 2);
const x = 0.6 + col * 6.2;
const y = 3.0 + row * 1.4;
s.addShape(pres.ShapeType.roundRect, { x: x, y: y, w: 5.9, h: 1.2, fill: { color: NAVY_DARK }, line: { color: ACCENT, width: 1 }, rectRadius: 0.1 });
s.addText(item.h, { x: x + 0.2, y: y + 0.1, w: 5.5, h: 0.4, fontSize: 18, bold: true, color: ACCENT, fontFace: FONT_H });
s.addText(item.t, { x: x + 0.2, y: y + 0.55, w: 5.5, h: 0.6, fontSize: 12, color: ICE, italic: true, fontFace: FONT_B });
});
s.addText("ASTERION • PS_AI_ConvAgent • 2026", { x: 0.6, y: 6.9, w: 12, h: 0.4, fontSize: 12, color: ICE_DEEP, fontFace: FONT_B });
// ═══════════════════════════════════════════════════════════════════
// SAVE
// ═══════════════════════════════════════════════════════════════════
pres.writeFile({ fileName: "PS_AI_ConvAgent_Documentation.pptx" })
.then(name => console.log(`Generated: ${name}`))
.catch(err => { console.error(err); process.exit(1); });

View File

@@ -166,6 +166,14 @@ DefaultPlatformService=Null
[OnlineSubsystemNull] [OnlineSubsystemNull]
bEnabled=true bEnabled=true
[Voice]
; Enable the Voice module so FVoiceModule::CreateVoiceEncoder() returns a valid
; Opus encoder/decoder. The engine default is bEnabled=false, which leaves the
; encoder NULL and makes networked agent audio fall back to raw PCM in large
; reliable multicast RPCs — these overflow the reliable buffer and drop VR
; client connections the moment the avatar speaks. With Opus, chunks are ~1-4KB.
bEnabled=true
[/Script/AndroidFileServerEditor.AndroidFileServerRuntimeSettings] [/Script/AndroidFileServerEditor.AndroidFileServerRuntimeSettings]
bEnablePlugin=True bEnablePlugin=True
bAllowNetworkConnection=True bAllowNetworkConnection=True
@@ -181,6 +189,9 @@ bUseManualIPAddress=False
ManualIPAddress= ManualIPAddress=
[/Script/PS_AI_ConvAgent.PS_AI_ConvAgent_Settings_ElevenLabs] [/Script/PS_AI_ConvAgent.PS_AI_ConvAgent_Settings_ElevenLabs]
API_Key=7b73c4244ccbec394cc010aaab01b0ec59ce0b11fc636ce4828354f675ca14a5 ; NOTE: there is intentionally no API_Key here. The ElevenLabs key is provided
; per-install by the client at runtime (BlueprintLibrary "Set ElevenLabs API Key")
; and stored in a SaveGame slot (Saved/SaveGames/PS_AI_ConvAgent_ElevenLabs.sav),
; so it is never committed or baked into a shipped build.
ServerRegion=Global ServerRegion=Global

View File

@@ -16,7 +16,8 @@
"PlatformAllowList": [ "PlatformAllowList": [
"Win64", "Win64",
"Mac", "Mac",
"Linux" "Linux",
"Android"
] ]
}, },
{ {

View File

@@ -0,0 +1,29 @@
// Copyright Asterion. All Rights Reserved.
#include "BT/PS_AI_Behavior_BTDecorator_CheckCombatType.h"
#include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_Interface.h"
UPS_AI_Behavior_BTDecorator_CheckCombatType::UPS_AI_Behavior_BTDecorator_CheckCombatType()
{
NodeName = TEXT("Check Combat Type");
}
bool UPS_AI_Behavior_BTDecorator_CheckCombatType::CalculateRawConditionValue(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) const
{
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC) return false;
APawn* Pawn = AIC->GetPawn();
if (!Pawn || !Pawn->Implements<UPS_AI_Behavior_Interface>()) return false;
return IPS_AI_Behavior_Interface::Execute_GetBehaviorCombatType(Pawn) == RequiredType;
}
FString UPS_AI_Behavior_BTDecorator_CheckCombatType::GetStaticDescription() const
{
const UEnum* TypeEnum = StaticEnum<EPS_AI_Behavior_CombatType>();
return FString::Printf(TEXT("Combat Type == %s"),
*TypeEnum->GetDisplayNameTextByValue(static_cast<int64>(RequiredType)).ToString());
}

View File

@@ -0,0 +1,67 @@
// Copyright Asterion. All Rights Reserved.
#include "BT/PS_AI_Behavior_BTDecorator_IsCoverNeeded.h"
#include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_PerceptionComponent.h"
#include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_Interface.h"
#include "BehaviorTree/BlackboardComponent.h"
UPS_AI_Behavior_BTDecorator_IsCoverNeeded::UPS_AI_Behavior_BTDecorator_IsCoverNeeded()
{
NodeName = TEXT("Is Cover Needed");
// Default: cover needed against Protector and Enemy, not Civilian
DangerousTargetTypes.Add(EPS_AI_Behavior_NPCType::Protector);
DangerousTargetTypes.Add(EPS_AI_Behavior_NPCType::Enemy);
}
bool UPS_AI_Behavior_BTDecorator_IsCoverNeeded::CalculateRawConditionValue(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) const
{
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) return false;
AActor* ThreatActor = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
if (!ThreatActor) return false;
// Resolve ThreatActor to owning Pawn (AimTargetActor → Character)
APawn* ThreatPawn = UPS_AI_Behavior_PerceptionComponent::FindOwningPawn(ThreatActor);
if (!ThreatPawn || !ThreatPawn->Implements<UPS_AI_Behavior_Interface>())
{
return true; // Can't resolve → assume dangerous (safe default)
}
const EPS_AI_Behavior_NPCType TargetType =
IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(ThreatPawn);
if (!DangerousTargetTypes.Contains(TargetType))
{
return false; // Target is not dangerous → no cover needed
}
// Check personality-driven combat/cover cycle timer
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (AIC)
{
if (const UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent())
{
return Personality->ShouldPreferCover();
}
}
return true; // No personality → default to cover
}
FString UPS_AI_Behavior_BTDecorator_IsCoverNeeded::GetStaticDescription() const
{
const UEnum* TypeEnum = StaticEnum<EPS_AI_Behavior_NPCType>();
FString TypeList;
for (const EPS_AI_Behavior_NPCType& Type : DangerousTargetTypes)
{
if (!TypeList.IsEmpty()) TypeList += TEXT(", ");
TypeList += TypeEnum->GetDisplayNameTextByValue(static_cast<int64>(Type)).ToString();
}
return FString::Printf(TEXT("Cover needed vs: %s"),
TypeList.IsEmpty() ? TEXT("None") : *TypeList);
}

View File

@@ -2,9 +2,13 @@
#include "BT/PS_AI_Behavior_BTService_EvaluateReaction.h" #include "BT/PS_AI_Behavior_BTService_EvaluateReaction.h"
#include "PS_AI_Behavior_AIController.h" #include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_PerceptionComponent.h"
#include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_PersonalityComponent.h" #include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_PersonalityProfile.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "BehaviorTree/BlackboardComponent.h" #include "BehaviorTree/BlackboardComponent.h"
#include "Perception/AISense_Hearing.h"
UPS_AI_Behavior_BTService_EvaluateReaction::UPS_AI_Behavior_BTService_EvaluateReaction() UPS_AI_Behavior_BTService_EvaluateReaction::UPS_AI_Behavior_BTService_EvaluateReaction()
{ {
@@ -19,16 +23,173 @@ void UPS_AI_Behavior_BTService_EvaluateReaction::TickNode(
Super::TickNode(OwnerComp, NodeMemory, DeltaSeconds); Super::TickNode(OwnerComp, NodeMemory, DeltaSeconds);
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner()); APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC) return; if (!AIC) { UE_LOG(LogPS_AI_Behavior, Warning, TEXT("EvaluateReaction: no AIC!")); return; }
// Scripted state: external control — don't touch the state
if (AIC->GetBehaviorState() == EPS_AI_Behavior_State::Scripted) { return; }
UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent(); UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent();
if (!Personality) return; if (!Personality) { UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] EvaluateReaction: no PersonalityComponent!"), *AIC->GetName()); return; }
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) return; if (!BB) { UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] EvaluateReaction: no BB!"), *AIC->GetName()); return; }
// Evaluate and apply the reaction // ─── Check for hostility change → update TeamId dynamically ────────
const EPS_AI_Behavior_State NewState = Personality->ApplyReaction(); APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
const bool bHostile = IPS_AI_Behavior_Interface::Execute_IsBehaviorHostile(Pawn);
const EPS_AI_Behavior_NPCType NPCType = IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(Pawn);
// Get faction from profile
const uint8 Faction = (Personality && Personality->Profile)
? Personality->Profile->Faction : 0;
// Infiltrated Enemy (hostile=false) → disguised as Civilian.
// When hostile flips to true → reveal true Enemy TeamId.
uint8 ExpectedTeamId;
if (NPCType == EPS_AI_Behavior_NPCType::Enemy && !bHostile)
{
ExpectedTeamId = PS_AI_Behavior_Team::DisguisedTeamId;
}
else
{
ExpectedTeamId = PS_AI_Behavior_Team::MakeTeamId(NPCType, Faction);
}
if (AIC->GetGenericTeamId().GetId() != ExpectedTeamId)
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Hostility changed: TeamId 0x%02X -> 0x%02X (hostile=%d)"),
*AIC->GetName(), AIC->GetGenericTeamId().GetId(), ExpectedTeamId, (int32)bHostile);
AIC->SetTeamId(ExpectedTeamId);
// Enemy becoming non-hostile (disguise) — soft de-escalation:
// clear threat so the BT re-evaluates to Idle naturally on the next
// ApplyReaction below. Do NOT stop movement or force speed abruptly —
// the natural state transition will handle that via HandleStateChanged.
if (NPCType == EPS_AI_Behavior_NPCType::Enemy && !bHostile)
{
BB->ClearValue(PS_AI_Behavior_BB::ThreatActor);
BB->ClearValue(PS_AI_Behavior_BB::ThreatLocation);
BB->SetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel, 0.0f);
if (Personality)
{
Personality->PerceivedThreatLevel = 0.0f;
}
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Hostile→false: threat cleared, BT will transition naturally."),
*AIC->GetName());
}
}
}
// ─── Gunshot reaction: flip non-hostile enemies to hostile ──────────
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
const EPS_AI_Behavior_NPCType NPCType = IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(Pawn);
const bool bHostile = IPS_AI_Behavior_Interface::Execute_IsBehaviorHostile(Pawn);
if (NPCType == EPS_AI_Behavior_NPCType::Enemy && !bHostile)
{
// Check if this enemy perceives a gunshot stimulus → become hostile toward shooter
if (UPS_AI_Behavior_PerceptionComponent* Perception = AIC->GetBehaviorPerception())
{
AActor* GunShotInstigator = nullptr;
TArray<AActor*> PerceivedActors;
Perception->GetCurrentlyPerceivedActors(nullptr, PerceivedActors);
for (AActor* RawActor : PerceivedActors)
{
if (!RawActor || GunShotInstigator) continue;
// Resolve weapon/item to owning Pawn (walk Owner/Instigator chain)
AActor* ResolvedActor = RawActor;
if (!Cast<APawn>(RawActor))
{
AActor* Cur = RawActor;
for (int32 D = 0; D < 4; ++D)
{
if (APawn* IP = Cur->GetInstigator()) { ResolvedActor = IP; break; }
AActor* OA = Cur->GetOwner();
if (!OA || OA == Cur) break;
if (APawn* OP = Cast<APawn>(OA)) { ResolvedActor = OP; break; }
Cur = OA;
}
}
// Skip same exact team (same NPCType + same Faction)
// Allied teams still allow gunfire through (e.g. disguised enemy hears Protector fire)
const uint8 MyTeam = AIC->GetGenericTeamId().GetId();
const uint8 TheirTeam = UPS_AI_Behavior_PerceptionComponent::GetActorTeamId(ResolvedActor);
if (MyTeam == TheirTeam)
{
continue;
}
FActorPerceptionBlueprintInfo Info;
if (Perception->GetActorsPerception(RawActor, Info))
{
for (const FAIStimulus& S : Info.LastSensedStimuli)
{
if (S.IsValid() &&
S.Type == UAISense::GetSenseID<UAISense_Hearing>() &&
PS_AI_Behavior_Tags_Internal::IsGunfire(S.Tag))
{
// For VR: check if Pawn has a custom threat actor
if (ResolvedActor->Implements<UPS_AI_Behavior_Interface>())
{
AActor* ThreatActor = IPS_AI_Behavior_Interface::Execute_GetBehaviorThreatActor(ResolvedActor);
if (ThreatActor) ResolvedActor = ThreatActor;
}
GunShotInstigator = ResolvedActor;
break;
}
}
}
}
if (GunShotInstigator)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Gunshot heard from '%s' — becoming hostile!"),
*AIC->GetName(), *GunShotInstigator->GetName());
// 1. Flip hostile → TeamId will update on next block above
IPS_AI_Behavior_Interface::Execute_SetBehaviorHostile(Pawn, true);
// 2. Immediately update TeamId so perception sees the shooter as hostile NOW
const uint8 EnemyFaction = (Personality && Personality->Profile)
? Personality->Profile->Faction : 0;
AIC->SetTeamId(PS_AI_Behavior_Team::MakeTeamId(EPS_AI_Behavior_NPCType::Enemy, EnemyFaction));
// 3. Write the shooter as ThreatActor in BB so combat targets them directly
BB->SetValueAsObject(PS_AI_Behavior_BB::ThreatActor, GunShotInstigator);
BB->SetValueAsVector(PS_AI_Behavior_BB::ThreatLocation, GunShotInstigator->GetActorLocation());
// 4. Set a meaningful threat level so EvaluateReaction enters Combat
if (Personality)
{
Personality->PerceivedThreatLevel = FMath::Max(Personality->PerceivedThreatLevel, 0.6f);
BB->SetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel, Personality->PerceivedThreatLevel);
}
}
}
}
}
// ─── Evaluate and apply the reaction ────────────────────────────────
EPS_AI_Behavior_State NewState = Personality->ApplyReaction();
// Don't enter Combat or TakingCover without a valid ThreatActor in BB
if (NewState == EPS_AI_Behavior_State::Combat || NewState == EPS_AI_Behavior_State::TakingCover)
{
const AActor* ThreatActor = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
if (!ThreatActor)
{
// Threat level says fight, but no target yet — stay Alerted until perception catches up
NewState = EPS_AI_Behavior_State::Alerted;
}
}
// Write to Blackboard // Write to Blackboard
AIC->SetBehaviorState(NewState); AIC->SetBehaviorState(NewState);

View File

@@ -0,0 +1,28 @@
// Copyright Asterion. All Rights Reserved.
#include "BT/PS_AI_Behavior_BTService_UpdateGaze.h"
#include "PS_AI_Behavior_AIController.h"
UPS_AI_Behavior_BTService_UpdateGaze::UPS_AI_Behavior_BTService_UpdateGaze()
{
NodeName = TEXT("Update Gaze");
Interval = 0.3f;
RandomDeviation = 0.05f;
}
void UPS_AI_Behavior_BTService_UpdateGaze::TickNode(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds)
{
Super::TickNode(OwnerComp, NodeMemory, DeltaSeconds);
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (AIC)
{
AIC->UpdateGazeTarget(DeltaSeconds);
}
}
FString UPS_AI_Behavior_BTService_UpdateGaze::GetStaticDescription() const
{
return TEXT("Updates gaze target from behavior state + proximity.\nBridges to PS_AI_ConvAgent GazeComponent (if present).");
}

View File

@@ -4,6 +4,8 @@
#include "PS_AI_Behavior_AIController.h" #include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_PerceptionComponent.h" #include "PS_AI_Behavior_PerceptionComponent.h"
#include "PS_AI_Behavior_PersonalityComponent.h" #include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_PersonalityProfile.h"
#include "PS_AI_Behavior_Statics.h"
#include "PS_AI_Behavior_Settings.h" #include "PS_AI_Behavior_Settings.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "BehaviorTree/BlackboardComponent.h" #include "BehaviorTree/BlackboardComponent.h"
@@ -23,11 +25,42 @@ void UPS_AI_Behavior_BTService_UpdateThreat::TickNode(
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner()); APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC) return; if (!AIC) return;
// Scripted state: external control — don't accumulate threat
if (AIC->GetBehaviorState() == EPS_AI_Behavior_State::Scripted) { return; }
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) return; if (!BB) return;
UPS_AI_Behavior_PerceptionComponent* Perception = AIC->GetBehaviorPerception(); UPS_AI_Behavior_PerceptionComponent* Perception = AIC->GetBehaviorPerception();
if (!Perception) return; if (!Perception)
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] UpdateThreat: No PerceptionComponent!"), *AIC->GetName());
return;
}
UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent();
// Debug: check what perception sees
TArray<AActor*> PerceivedActors;
Perception->GetCurrentlyPerceivedActors(nullptr, PerceivedActors);
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] UpdateThreat: Perceived %d actors"),
*AIC->GetName(), PerceivedActors.Num());
for (AActor* A : PerceivedActors)
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT(" - %s"), *A->GetName());
}
}
// Also check known actors (perceived in the past but maybe lost sight)
TArray<AActor*> KnownActors;
Perception->GetKnownPerceivedActors(nullptr, KnownActors);
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()) && KnownActors.Num() != PerceivedActors.Num())
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] UpdateThreat: Known (past) %d actors"),
*AIC->GetName(), KnownActors.Num());
}
// Calculate current threat // Calculate current threat
const float RawThreat = Perception->CalculateThreatLevel(); const float RawThreat = Perception->CalculateThreatLevel();
@@ -36,28 +69,215 @@ void UPS_AI_Behavior_BTService_UpdateThreat::TickNode(
const float StoredThreat = BB->GetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel); const float StoredThreat = BB->GetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel);
// Apply decay when no threat, or take the max of new vs decayed // Apply decay when no threat, or take the max of new vs decayed
const UPS_AI_Behavior_Settings* Settings = GetDefault<UPS_AI_Behavior_Settings>(); // Decay rate comes from PersonalityProfile (per-archetype), fallback to global Settings
const float DecayedThreat = FMath::Max(0.0f, StoredThreat - Settings->ThreatDecayRate * DeltaSeconds); float DecayRate = GetDefault<UPS_AI_Behavior_Settings>()->ThreatDecayRate;
if (Personality && Personality->Profile)
{
DecayRate = Personality->Profile->ThreatDecayRate;
}
const float DecayedThreat = FMath::Max(0.0f, StoredThreat - DecayRate * DeltaSeconds);
const float FinalThreat = FMath::Max(RawThreat, DecayedThreat); const float FinalThreat = FMath::Max(RawThreat, DecayedThreat);
BB->SetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel, FinalThreat); BB->SetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel, FinalThreat);
// Update threat actor and location if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] UpdateThreat: raw=%.2f, stored=%.2f, final=%.2f"),
*AIC->GetName(), RawThreat, StoredThreat, FinalThreat);
}
// ─── Update threat actor and location with LOS tracking ─────────
FUpdateThreatMemory* Memory = reinterpret_cast<FUpdateThreatMemory*>(NodeMemory);
AActor* ThreatActor = Perception->GetHighestThreatActor(); AActor* ThreatActor = Perception->GetHighestThreatActor();
AActor* CurrentBBTarget = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
if (ThreatActor) if (ThreatActor)
{ {
// Target switched by perception (higher score)
if (ThreatActor != CurrentBBTarget)
{
const EPS_AI_Behavior_State CurrentState = AIC->GetBehaviorState();
// During TakingCover: don't switch target — cover is positioned against current threat.
// The flanking check will handle cover invalidation if needed.
if (CurrentBBTarget && CurrentState == EPS_AI_Behavior_State::TakingCover)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] UpdateThreat: suppressed target switch to '%s' — in TakingCover (keeping '%s')"),
*AIC->GetName(), *ThreatActor->GetName(), *CurrentBBTarget->GetName());
ThreatActor = CurrentBBTarget; // Keep current target
}
else
{
Memory->TimeSinceLOS = 0.0f;
Memory->bHadLOS = true;
Memory->bInvestigating = false;
Memory->LastVisiblePosition = ThreatActor->GetActorLocation();
BB->ClearValue(PS_AI_Behavior_BB::LastKnownTargetPosition);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] UpdateThreat: target switched to '%s', LOS tracking reset"),
*AIC->GetName(), *ThreatActor->GetName());
}
}
BB->SetValueAsObject(PS_AI_Behavior_BB::ThreatActor, ThreatActor); BB->SetValueAsObject(PS_AI_Behavior_BB::ThreatActor, ThreatActor);
// Debug: write the owning Pawn's name so BB shows who we're actually targeting
if (ThreatActor != CurrentBBTarget)
{
APawn* OwningPawn = Perception->LastThreatOwningPawn.Get();
BB->SetValueAsString(PS_AI_Behavior_BB::ThreatPawnName,
OwningPawn ? OwningPawn->GetName() : ThreatActor->GetName());
}
// ─── LOS check ──────────────────────────────────────────────
const APawn* Pawn = AIC->GetPawn();
const bool bHasLOS = Pawn
? UPS_AI_Behavior_Statics::HasLineOfSight(Pawn->GetWorld(), Pawn, ThreatActor, EyeHeightOffset)
: false;
if (bHasLOS)
{
// Can see target → update last visible position, reset timer
Memory->LastVisiblePosition = ThreatActor->GetActorLocation();
Memory->TimeSinceLOS = 0.0f;
Memory->bHadLOS = true;
Memory->bInvestigating = false;
BB->SetValueAsVector(PS_AI_Behavior_BB::ThreatLocation, ThreatActor->GetActorLocation());
BB->ClearValue(PS_AI_Behavior_BB::LastKnownTargetPosition);
}
else
{
// Cannot see target → accumulate time, write last known position
Memory->TimeSinceLOS += DeltaSeconds;
// Write ThreatLocation as the LAST VISIBLE position (not real-time)
if (!Memory->LastVisiblePosition.IsZero())
{
BB->SetValueAsVector(PS_AI_Behavior_BB::ThreatLocation, Memory->LastVisiblePosition);
}
else
{
BB->SetValueAsVector(PS_AI_Behavior_BB::ThreatLocation, ThreatActor->GetActorLocation()); BB->SetValueAsVector(PS_AI_Behavior_BB::ThreatLocation, ThreatActor->GetActorLocation());
} }
else if (FinalThreat <= 0.01f)
// Phase 1: after LOSLostTimeout → write LastKnownTargetPosition for investigation
if (Memory->TimeSinceLOS > LOSLostTimeout && !Memory->bInvestigating)
{ {
// Clear threat data when fully decayed Memory->bInvestigating = true;
BB->SetValueAsVector(PS_AI_Behavior_BB::LastKnownTargetPosition, Memory->LastVisiblePosition);
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] UpdateThreat: LOS lost for %.0fs → investigating last known position %s"),
*AIC->GetName(), Memory->TimeSinceLOS, *Memory->LastVisiblePosition.ToString());
}
// Phase 2: after 2x timeout → give up, clear target so NPC can re-evaluate
// BUT never abandon in active combat states — the NPC knows the threat is there
if (Memory->TimeSinceLOS > LOSLostTimeout * 2.0f)
{
const EPS_AI_Behavior_State CurrentState = AIC->GetBehaviorState();
if (CurrentState == EPS_AI_Behavior_State::Combat ||
CurrentState == EPS_AI_Behavior_State::TakingCover)
{
// In combat: keep target, just update location to last known
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] UpdateThreat: LOS lost for %.0fs but in %s state — keeping target"),
*AIC->GetName(), Memory->TimeSinceLOS,
CurrentState == EPS_AI_Behavior_State::Combat ? TEXT("Combat") : TEXT("TakingCover"));
}
else
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] UpdateThreat: LOS lost for %.0fs → clearing target, de-escalating"),
*AIC->GetName(), Memory->TimeSinceLOS);
BB->ClearValue(PS_AI_Behavior_BB::ThreatActor); BB->ClearValue(PS_AI_Behavior_BB::ThreatActor);
BB->ClearValue(PS_AI_Behavior_BB::LastKnownTargetPosition);
BB->ClearValue(PS_AI_Behavior_BB::ThreatPawnName);
Memory->TimeSinceLOS = 0.0f;
Memory->bInvestigating = false;
Memory->bHadLOS = true;
}
}
}
}
else if (CurrentBBTarget)
{
// Perception lost the actor momentarily, but we had a target in BB.
// Don't clear immediately — use the LOS tracking timeout instead.
// This prevents brief perception gaps (1-2 ticks) from dropping the target.
Memory->TimeSinceLOS += DeltaSeconds;
// Keep ThreatLocation at last visible position
if (!Memory->LastVisiblePosition.IsZero())
{
BB->SetValueAsVector(PS_AI_Behavior_BB::ThreatLocation, Memory->LastVisiblePosition);
}
// Phase 1: after LOSLostTimeout → investigate last known position
if (Memory->TimeSinceLOS > LOSLostTimeout && !Memory->bInvestigating)
{
Memory->bInvestigating = true;
BB->SetValueAsVector(PS_AI_Behavior_BB::LastKnownTargetPosition, Memory->LastVisiblePosition);
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] UpdateThreat: perception lost target for %.0fs → investigating last known position %s"),
*AIC->GetName(), Memory->TimeSinceLOS, *Memory->LastVisiblePosition.ToString());
}
// Phase 2: after 2x timeout → give up, clear target
// BUT never abandon in active combat states
if (Memory->TimeSinceLOS > LOSLostTimeout * 2.0f)
{
const EPS_AI_Behavior_State CurrentState = AIC->GetBehaviorState();
if (CurrentState == EPS_AI_Behavior_State::Combat ||
CurrentState == EPS_AI_Behavior_State::TakingCover)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] UpdateThreat: perception lost target for %.0fs but in %s state — keeping target"),
*AIC->GetName(), Memory->TimeSinceLOS,
CurrentState == EPS_AI_Behavior_State::Combat ? TEXT("Combat") : TEXT("TakingCover"));
}
else
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] UpdateThreat: perception lost target for %.0fs → clearing target, de-escalating"),
*AIC->GetName(), Memory->TimeSinceLOS);
BB->ClearValue(PS_AI_Behavior_BB::ThreatActor);
BB->ClearValue(PS_AI_Behavior_BB::LastKnownTargetPosition);
BB->ClearValue(PS_AI_Behavior_BB::ThreatPawnName);
Memory->TimeSinceLOS = 0.0f;
Memory->bInvestigating = false;
Memory->bHadLOS = true;
}
}
}
else
{
// No threat actor at all (never had one)
Memory->TimeSinceLOS = 0.0f;
Memory->bInvestigating = false;
Memory->bHadLOS = true;
if (FinalThreat > 0.0f)
{
// Threat exists but no specific actor target (e.g. gunshot from friendly).
FVector ThreatLoc;
if (Perception->GetGunShotStimulusLocation(ThreatLoc))
{
BB->SetValueAsVector(PS_AI_Behavior_BB::ThreatLocation, ThreatLoc);
}
}
else
{
BB->ClearValue(PS_AI_Behavior_BB::ThreatLocation); BB->ClearValue(PS_AI_Behavior_BB::ThreatLocation);
} }
}
// Sync to PersonalityComponent // Sync to PersonalityComponent
UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent();
if (Personality) if (Personality)
{ {
Personality->PerceivedThreatLevel = FinalThreat; Personality->PerceivedThreatLevel = FinalThreat;
@@ -66,5 +286,7 @@ void UPS_AI_Behavior_BTService_UpdateThreat::TickNode(
FString UPS_AI_Behavior_BTService_UpdateThreat::GetStaticDescription() const FString UPS_AI_Behavior_BTService_UpdateThreat::GetStaticDescription() const
{ {
return TEXT("Updates Blackboard threat data from perception.\nWrites: ThreatActor, ThreatLocation, ThreatLevel."); return FString::Printf(
TEXT("Updates Blackboard threat data from perception.\nLOS timeout: %.0fs (clear at %.0fs)\nWrites: ThreatActor, ThreatLocation, ThreatLevel, LastKnownTargetPosition."),
LOSLostTimeout, LOSLostTimeout * 2.0f);
} }

View File

@@ -2,10 +2,16 @@
#include "BT/PS_AI_Behavior_BTTask_Attack.h" #include "BT/PS_AI_Behavior_BTTask_Attack.h"
#include "PS_AI_Behavior_AIController.h" #include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_CombatComponent.h" #include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_PersonalityProfile.h"
#include "PS_AI_Behavior_Statics.h"
#include "BehaviorTree/BlackboardComponent.h" #include "BehaviorTree/BlackboardComponent.h"
#include "Navigation/PathFollowingComponent.h" #include "Navigation/PathFollowingComponent.h"
#include "NavigationSystem.h"
#include "EnvironmentQuery/EnvQuery.h"
#include "EnvironmentQuery/EnvQueryManager.h"
UPS_AI_Behavior_BTTask_Attack::UPS_AI_Behavior_BTTask_Attack() UPS_AI_Behavior_BTTask_Attack::UPS_AI_Behavior_BTTask_Attack()
{ {
@@ -18,52 +24,119 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_Attack::ExecuteTask(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory)
{ {
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner()); APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC || !AIC->GetPawn()) return EBTNodeResult::Failed; if (!AIC || !AIC->GetPawn())
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("Attack: FAILED — no AIC or Pawn"));
return EBTNodeResult::Failed;
}
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) return EBTNodeResult::Failed; if (!BB)
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: FAILED — no BB"), *AIC->GetName());
return EBTNodeResult::Failed;
}
// Get threat actor
AActor* Target = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor)); AActor* Target = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
if (!Target) if (!Target)
{ {
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: FAILED — no ThreatActor in BB"), *AIC->GetName());
// De-escalate so decorator can re-trigger when threat returns
AIC->SetBehaviorState(EPS_AI_Behavior_State::Alerted);
return EBTNodeResult::Failed; return EBTNodeResult::Failed;
} }
// Get combat component APawn* Pawn = AIC->GetPawn();
UPS_AI_Behavior_CombatComponent* Combat =
AIC->GetPawn()->FindComponentByClass<UPS_AI_Behavior_CombatComponent>(); // Validate target before starting attack
if (!Combat) if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{ {
UE_LOG(LogPS_AI_Behavior, Warning, if (!IPS_AI_Behavior_Interface::Execute_IsTargetActorValid(Pawn, Target))
TEXT("[%s] Attack task: no CombatComponent on Pawn."), *AIC->GetName()); {
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: FAILED — IsTargetActorValid returned false for '%s'"),
*AIC->GetName(), *Target->GetName());
BB->ClearValue(PS_AI_Behavior_BB::ThreatActor);
// De-escalate so decorator can re-trigger when threat returns
AIC->SetBehaviorState(EPS_AI_Behavior_State::Alerted);
return EBTNodeResult::Failed; return EBTNodeResult::Failed;
} }
// Try to attack immediately if in range
if (Combat->IsInAttackRange(Target))
{
if (Combat->CanAttack())
{
Combat->ExecuteAttack(Target);
return EBTNodeResult::Succeeded;
}
// In range but on cooldown — wait
return EBTNodeResult::InProgress;
}
// Out of range — move toward target
const EPathFollowingRequestResult::Type Result = AIC->MoveToActor(
Target, Combat->AttackRange * 0.8f, /*bUsePathfinding=*/true,
/*bAllowStrafe=*/true);
if (Result == EPathFollowingRequestResult::Failed)
{
return EBTNodeResult::Failed;
} }
// ─── Query combat type and range ─────────────────────────────────
FAttackMemory* Memory = reinterpret_cast<FAttackMemory*>(NodeMemory); FAttackMemory* Memory = reinterpret_cast<FAttackMemory*>(NodeMemory);
Memory->bMovingToTarget = false;
Memory->bAttacking = false;
Memory->bInRange = false;
Memory->bHasLOS = true;
Memory->bSeekingFiringPos = false;
Memory->bEQSQueryRunning = false;
Memory->RepositionTimer = 0.0f;
Memory->LOSCheckTimer = 0.0f;
Memory->CombatType = EPS_AI_Behavior_CombatType::Melee;
Memory->MinRange = 100.0f;
Memory->MaxRange = AttackMoveRadius;
// CombatType from interface
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
Memory->CombatType = IPS_AI_Behavior_Interface::Execute_GetBehaviorCombatType(Pawn);
}
// Min/Max attack range from PersonalityProfile
if (UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent())
{
if (Personality->Profile)
{
Memory->MinRange = Personality->Profile->MinAttackRange;
Memory->MaxRange = Personality->Profile->MaxAttackRange;
}
}
// ─── Initial LOS check (ranged only) ────────────────────────────
if (Memory->CombatType == EPS_AI_Behavior_CombatType::Ranged)
{
Memory->bHasLOS = UPS_AI_Behavior_Statics::HasLineOfSight(
Pawn->GetWorld(), Pawn, Target, EyeHeightOffset);
}
// Melee: approach to half MinRange. Ranged: approach to midpoint of band.
const float ApproachRange = (Memory->CombatType == EPS_AI_Behavior_CombatType::Melee)
? Memory->MinRange * 0.5f : (Memory->MinRange + Memory->MaxRange) * 0.5f;
// Start attack immediately (draw weapon, enter combat stance)
// regardless of range or LOS — the NPC is committed to combat
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStartAttack(Pawn, Target);
Memory->bAttacking = true;
}
// Check if already in range
const float DistToTarget = FVector::Dist(Pawn->GetActorLocation(), Target->GetActorLocation());
if (DistToTarget <= Memory->MaxRange)
{
Memory->bInRange = true;
}
// Initial move toward target if not in range
if (DistToTarget > Memory->MaxRange)
{
const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation(
Target->GetActorLocation(), ApproachRange, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/true, /*bCanStrafe=*/false);
if (Result != EPathFollowingRequestResult::AlreadyAtGoal)
{
Memory->bMovingToTarget = true; Memory->bMovingToTarget = true;
}
}
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: started on '%s' (%s, range=[%.0f-%.0f], dist=%.0f, inRange=%d, LOS=%d, attacking=%d)"),
*AIC->GetName(), *Target->GetName(),
Memory->CombatType == EPS_AI_Behavior_CombatType::Ranged ? TEXT("Ranged") : TEXT("Melee"),
Memory->MinRange, Memory->MaxRange, DistToTarget,
Memory->bInRange ? 1 : 0, Memory->bHasLOS ? 1 : 0, Memory->bAttacking ? 1 : 0);
return EBTNodeResult::InProgress; return EBTNodeResult::InProgress;
} }
@@ -80,54 +153,196 @@ void UPS_AI_Behavior_BTTask_Attack::TickTask(
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
AActor* Target = BB ? Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor)) : nullptr; AActor* Target = BB ? Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor)) : nullptr;
if (!Target) if (!Target)
{
// De-escalate so decorator can re-trigger when threat returns
AIC->SetBehaviorState(EPS_AI_Behavior_State::Alerted);
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
}
APawn* Pawn = AIC->GetPawn();
// Validate target
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
if (!IPS_AI_Behavior_Interface::Execute_IsTargetActorValid(Pawn, Target))
{ {
AIC->StopMovement(); AIC->StopMovement();
BB->ClearValue(PS_AI_Behavior_BB::ThreatActor);
// De-escalate so decorator can re-trigger when threat returns
AIC->SetBehaviorState(EPS_AI_Behavior_State::Alerted);
FinishLatentTask(OwnerComp, EBTNodeResult::Failed); FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return; return;
} }
UPS_AI_Behavior_CombatComponent* Combat =
AIC->GetPawn()->FindComponentByClass<UPS_AI_Behavior_CombatComponent>();
if (!Combat)
{
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
} }
// Check if we can attack now
if (Combat->IsInAttackRange(Target))
{
FAttackMemory* Memory = reinterpret_cast<FAttackMemory*>(NodeMemory); FAttackMemory* Memory = reinterpret_cast<FAttackMemory*>(NodeMemory);
if (Memory->bMovingToTarget)
// Tick cooldowns
if (Memory->RepositionTimer > 0.0f)
{ {
AIC->StopMovement(); Memory->RepositionTimer -= DeltaSeconds;
Memory->bMovingToTarget = false;
} }
if (Combat->CanAttack()) // ─── Periodic LOS check (ranged only) ───────────────────────────
if (Memory->CombatType == EPS_AI_Behavior_CombatType::Ranged)
{ {
Combat->ExecuteAttack(Target); Memory->LOSCheckTimer -= DeltaSeconds;
FinishLatentTask(OwnerComp, EBTNodeResult::Succeeded); if (Memory->LOSCheckTimer <= 0.0f)
{
Memory->LOSCheckTimer = LOSCheckInterval;
Memory->bHasLOS = UPS_AI_Behavior_Statics::HasLineOfSight(
Pawn->GetWorld(), Pawn, Target, EyeHeightOffset);
}
}
// Melee always has "LOS" (they just rush)
const float DistToTarget = FVector::Dist(Pawn->GetActorLocation(), Target->GetActorLocation());
const bool bCanReposition = (Memory->RepositionTimer <= 0.0f);
// ─── MOVEMENT LOGIC ─────────────────────────────────────────────
if (Memory->CombatType == EPS_AI_Behavior_CombatType::Melee)
{
// Melee: continuously chase target (no LOS check — just rush)
if (AIC->GetMoveStatus() == EPathFollowingStatus::Idle && DistToTarget > Memory->MinRange)
{
AIC->MoveToLocation(
Target->GetActorLocation(), Memory->MinRange * 0.5f, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/true, /*bCanStrafe=*/false);
} }
// Else: wait for cooldown (stay InProgress)
} }
else else
{ {
// Still moving — check if movement failed // ─── Ranged: LOS-aware positioning ──────────────────────────
FAttackMemory* Memory = reinterpret_cast<FAttackMemory*>(NodeMemory); if (Memory->bSeekingFiringPos)
if (Memory->bMovingToTarget && AIC->GetMoveStatus() == EPathFollowingStatus::Idle)
{ {
// Movement ended but not in range — try again // Waiting for firing position move to complete
const EPathFollowingRequestResult::Type Result = AIC->MoveToActor( if (AIC->GetMoveStatus() == EPathFollowingStatus::Idle)
Target, Combat->AttackRange * 0.8f, /*bUsePathfinding=*/true, {
/*bAllowStrafe=*/true); Memory->bSeekingFiringPos = false;
// Re-check LOS immediately after arriving
Memory->LOSCheckTimer = 0.0f;
}
}
else if (!Memory->bHasLOS && !Memory->bEQSQueryRunning &&
AIC->GetMoveStatus() == EPathFollowingStatus::Idle)
{
// No LOS and idle → find a firing position
// Check if investigation (last known position) is active
// ClearValue sets vectors to FAISystem::InvalidLocation, not zero — must check both
const FVector LastKnownPos = BB->GetValueAsVector(PS_AI_Behavior_BB::LastKnownTargetPosition);
const bool bHasLastKnown = !LastKnownPos.IsZero() &&
!LastKnownPos.ContainsNaN() &&
LastKnownPos.X < 1e30f; // Filter out FLT_MAX / InvalidLocation
if (bHasLastKnown)
{
// Investigation mode: go to last known position
AIC->MoveToLocation(
LastKnownPos, 100.0f, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/true, /*bCanStrafe=*/false);
Memory->bSeekingFiringPos = true;
if (Result == EPathFollowingRequestResult::Failed) UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Attack: investigating last known position %s"),
*AIC->GetName(), *LastKnownPos.ToString());
}
else
{ {
FinishLatentTask(OwnerComp, EBTNodeResult::Failed); // No investigation yet — try EQS for a flanking position
RunFiringPositionQuery(OwnerComp, NodeMemory, Pawn, Target);
}
}
else if (Memory->bHasLOS && AIC->GetMoveStatus() == EPathFollowingStatus::Idle)
{
// Has LOS — normal range maintenance
const float MidRange = (Memory->MinRange + Memory->MaxRange) * 0.5f;
if (bCanReposition && DistToTarget < Memory->MinRange)
{
// Too close — back away
const FVector AwayDir = (Pawn->GetActorLocation() - Target->GetActorLocation()).GetSafeNormal2D();
const float RetreatDist = MidRange - DistToTarget + 50.0f;
const FVector RetreatPoint = Pawn->GetActorLocation() + AwayDir * RetreatDist;
UNavigationSystemV1* NavSys = FNavigationSystem::GetCurrent<UNavigationSystemV1>(GetWorld());
if (NavSys)
{
FNavLocation NavLoc;
if (NavSys->ProjectPointToNavigation(RetreatPoint, NavLoc, FVector(300.0f, 300.0f, 200.0f)))
{
AIC->MoveToLocation(
NavLoc.Location, 50.0f, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/false, /*bCanStrafe=*/false);
Memory->RepositionTimer = RepositionCooldown * 3.0f;
} }
} }
} }
else if (DistToTarget > Memory->MaxRange)
{
// Too far — advance
AIC->MoveToLocation(
Target->GetActorLocation(), MidRange, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/true, /*bCanStrafe=*/false);
}
}
}
// ─── Toggle attack based on range + LOS ─────────────────────────
const float EnterRange = Memory->MaxRange;
const float LeaveRange = Memory->MaxRange * 1.1f;
const bool bNowInRange = Memory->bInRange
? (DistToTarget <= LeaveRange)
: (DistToTarget <= EnterRange);
// For ranged: require LOS to attack. For melee: always allow.
const bool bCanAttack = (Memory->CombatType == EPS_AI_Behavior_CombatType::Melee) || Memory->bHasLOS;
if (bNowInRange && bCanAttack && !Memory->bInRange)
{
// Entered range with LOS → start attacking
Memory->bInRange = true;
if (Pawn->Implements<UPS_AI_Behavior_Interface>() && !Memory->bAttacking)
{
IPS_AI_Behavior_Interface::Execute_BehaviorStartAttack(Pawn, Target);
Memory->bAttacking = true;
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: in range + LOS → StartAttack on '%s'"),
*AIC->GetName(), *Target->GetName());
}
}
else if (bNowInRange && !bCanAttack && Memory->bAttacking)
{
// In range but lost LOS → stop attacking (stay in range though)
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(Pawn);
Memory->bAttacking = false;
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: in range but NO LOS → StopAttack"),
*AIC->GetName());
}
}
else if (bNowInRange && bCanAttack && !Memory->bAttacking)
{
// In range, regained LOS → restart attacking
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStartAttack(Pawn, Target);
Memory->bAttacking = true;
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: LOS regained → StartAttack on '%s'"),
*AIC->GetName(), *Target->GetName());
}
}
else if (!bNowInRange && Memory->bInRange)
{
// Left range → stop attacking
Memory->bInRange = false;
if (Pawn->Implements<UPS_AI_Behavior_Interface>() && Memory->bAttacking)
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(Pawn);
Memory->bAttacking = false;
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Attack: out of range → StopAttack"),
*AIC->GetName());
}
}
} }
EBTNodeResult::Type UPS_AI_Behavior_BTTask_Attack::AbortTask( EBTNodeResult::Type UPS_AI_Behavior_BTTask_Attack::AbortTask(
@@ -141,7 +356,127 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_Attack::AbortTask(
return EBTNodeResult::Aborted; return EBTNodeResult::Aborted;
} }
void UPS_AI_Behavior_BTTask_Attack::OnTaskFinished(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, EBTNodeResult::Type TaskResult)
{
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (AIC)
{
APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(Pawn);
}
}
Super::OnTaskFinished(OwnerComp, NodeMemory, TaskResult);
}
// ─── EQS Firing Position Query ──────────────────────────────────────────────
void UPS_AI_Behavior_BTTask_Attack::RunFiringPositionQuery(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APawn* Pawn, AActor* Target)
{
FAttackMemory* Memory = reinterpret_cast<FAttackMemory*>(NodeMemory);
if (!FiringPositionQuery)
{
// No EQS query configured → fallback: advance toward target
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (AIC)
{
const float MidRange = (Memory->MinRange + Memory->MaxRange) * 0.5f;
AIC->MoveToLocation(
Target->GetActorLocation(), MidRange, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/true, /*bCanStrafe=*/false);
Memory->bSeekingFiringPos = true;
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] Attack: no EQS query, fallback advance toward target"),
*AIC->GetName());
}
return;
}
UWorld* World = Pawn->GetWorld();
UEnvQueryManager* EQSManager = UEnvQueryManager::GetCurrent(World);
if (!EQSManager)
{
return;
}
Memory->bEQSQueryRunning = true;
TWeakObjectPtr<AActor> WeakTarget = Target;
FEnvQueryRequest Request(FiringPositionQuery, Pawn);
Request.Execute(EEnvQueryRunMode::SingleResult,
FQueryFinishedSignature::CreateUObject(this,
&UPS_AI_Behavior_BTTask_Attack::OnFiringPositionQueryFinished,
&OwnerComp, NodeMemory, WeakTarget));
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] Attack: EQS firing position query launched"),
*Pawn->GetName());
}
void UPS_AI_Behavior_BTTask_Attack::OnFiringPositionQueryFinished(
TSharedPtr<FEnvQueryResult> Result,
UBehaviorTreeComponent* OwnerComp, uint8* NodeMemory,
TWeakObjectPtr<AActor> WeakTarget)
{
if (!OwnerComp || !NodeMemory) return;
FAttackMemory* Memory = reinterpret_cast<FAttackMemory*>(NodeMemory);
Memory->bEQSQueryRunning = false;
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp->GetAIOwner());
if (!AIC || !AIC->GetPawn()) return;
AActor* Target = WeakTarget.Get();
// Fallback lambda: advance directly toward target
auto FallbackAdvance = [&](const TCHAR* Reason)
{
if (Target)
{
const float MidRange = (Memory->MinRange + Memory->MaxRange) * 0.5f;
AIC->MoveToLocation(
Target->GetActorLocation(), MidRange, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/true, /*bCanStrafe=*/false);
Memory->bSeekingFiringPos = true;
}
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Attack: %s, fallback advance toward target"),
*AIC->GetName(), Reason);
};
if (Result.IsValid() && Result->IsSuccessful())
{
const FVector FiringPos = Result->GetItemAsLocation(0);
const EPathFollowingRequestResult::Type MoveResult = AIC->MoveToLocation(
FiringPos, 50.0f, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectGoal=*/true, /*bCanStrafe=*/false);
if (MoveResult == EPathFollowingRequestResult::RequestSuccessful)
{
Memory->bSeekingFiringPos = true;
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Attack: EQS moving to firing position %s"),
*AIC->GetName(), *FiringPos.ToString());
}
else
{
// AlreadyAtGoal or Failed — EQS position is too close or unreachable
FallbackAdvance(TEXT("EQS position unreachable or too close"));
}
}
else
{
FallbackAdvance(TEXT("EQS no result"));
}
}
FString UPS_AI_Behavior_BTTask_Attack::GetStaticDescription() const FString UPS_AI_Behavior_BTTask_Attack::GetStaticDescription() const
{ {
return TEXT("Move to threat and attack via CombatComponent."); return FString::Printf(TEXT("LOS-aware attack.\nFallback radius: %.0fcm\nLOS check: every %.1fs\nEQS query: %s"),
AttackMoveRadius, LOSCheckInterval,
FiringPositionQuery ? *FiringPositionQuery->GetName() : TEXT("None (fallback)"));
} }

View File

@@ -0,0 +1,885 @@
// Copyright Asterion. All Rights Reserved.
#include "BT/PS_AI_Behavior_BTTask_CoverShootCycle.h"
#include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_CoverPoint.h"
#include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_PersonalityProfile.h"
#include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_Statics.h"
#include "BehaviorTree/BlackboardComponent.h"
#include "Navigation/PathFollowingComponent.h"
#include "CollisionQueryParams.h"
#include "EngineUtils.h"
#include "EnvironmentQuery/EnvQuery.h"
#include "EnvironmentQuery/EnvQueryManager.h"
#include "DrawDebugHelpers.h"
// ─── Helper: Crouch at cover if required ────────────────────────────────────
namespace
{
void CrouchAtCoverIfNeeded(APawn* Pawn, UBlackboardComponent* BB)
{
if (!Pawn || !Pawn->Implements<UPS_AI_Behavior_Interface>() || !BB) return;
const APS_AI_Behavior_CoverPoint* CoverPt =
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (CoverPt && CoverPt->bCrouch)
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, true);
}
}
void SetSubState(UBlackboardComponent* BB, EPS_AI_Behavior_CombatSubState State)
{
BB->SetValueAsEnum(PS_AI_Behavior_BB::CombatSubState, static_cast<uint8>(State));
}
}
// ─── Constructor ────────────────────────────────────────────────────────────
UPS_AI_Behavior_BTTask_CoverShootCycle::UPS_AI_Behavior_BTTask_CoverShootCycle()
{
NodeName = TEXT("Cover Shoot Cycle");
bNotifyTick = true;
bNotifyTaskFinished = true;
}
// ─── ExecuteTask ────────────────────────────────────────────────────────────
EBTNodeResult::Type UPS_AI_Behavior_BTTask_CoverShootCycle::ExecuteTask(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory)
{
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC || !AIC->GetPawn()) return EBTNodeResult::Failed;
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) return EBTNodeResult::Failed;
const FVector CoverLoc = BB->GetValueAsVector(PS_AI_Behavior_BB::CoverLocation);
if (CoverLoc.IsZero())
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] CoverShootCycle: no CoverLocation in BB."), *AIC->GetName());
return EBTNodeResult::Failed;
}
AActor* Target = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
if (!Target)
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] CoverShootCycle: no ThreatActor in BB."), *AIC->GetName());
return EBTNodeResult::Failed;
}
// ─── Initialize memory ──────────────────────────────────────────
FCoverShootMemory* Memory = reinterpret_cast<FCoverShootMemory*>(NodeMemory);
Memory->SubState = EPS_AI_Behavior_CombatSubState::Engaging;
Memory->Timer = 0.0f;
Memory->PhaseDuration = 0.0f;
Memory->CycleCount = 0;
Memory->bMoveRequested = false;
Memory->FiringPosition = FVector::ZeroVector;
Memory->bHasFiringPosition = false;
Memory->bEQSRunning = false;
Memory->MoveTimeout = 5.0f;
// Base values
Memory->EffPeekMin = PeekDurationMin;
Memory->EffPeekMax = PeekDurationMax;
Memory->EffCoverMin = CoverDurationMin;
Memory->EffCoverMax = CoverDurationMax;
Memory->EffMaxCycles = MaxCyclesBeforeAdvance;
Memory->bCanAdvance = true;
// Modulate by personality traits
if (UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent())
{
const float Aggressivity = Personality->GetTrait(EPS_AI_Behavior_TraitAxis::Aggressivity);
const float Caution = Personality->GetTrait(EPS_AI_Behavior_TraitAxis::Caution);
const float Courage = Personality->GetTrait(EPS_AI_Behavior_TraitAxis::Courage);
const float AggrFactor = 0.7f + Aggressivity * 0.6f;
Memory->EffPeekMin *= AggrFactor;
Memory->EffPeekMax *= AggrFactor;
Memory->EffCoverMin /= AggrFactor;
Memory->EffCoverMax /= AggrFactor;
Memory->EffMaxCycles = FMath::Max(1, FMath::RoundToInt(MaxCyclesBeforeAdvance * (1.5f - Aggressivity * 0.5f)));
const float CautionFactor = 0.5f + Caution * 1.0f;
Memory->EffCoverMin *= CautionFactor;
Memory->EffCoverMax *= CautionFactor;
Memory->EffPeekMin /= CautionFactor;
Memory->EffPeekMax /= CautionFactor;
Memory->bCanAdvance = (Courage >= 0.3f);
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] CoverShootCycle: peek=[%.1f-%.1f]s, cover=[%.1f-%.1f]s, maxCycles=%d, canAdvance=%d"),
*AIC->GetName(),
Memory->EffPeekMin, Memory->EffPeekMax,
Memory->EffCoverMin, Memory->EffCoverMax,
Memory->EffMaxCycles, (int32)Memory->bCanAdvance);
}
// ─── Move to cover position ─────────────────────────────────────
const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation(
CoverLoc, 30.0f, true, true, true, false);
if (Result == EPathFollowingRequestResult::Failed)
{
return EBTNodeResult::Failed;
}
// Face the threat while in cover
AIC->SetFocus(Target);
// Start attack immediately (draw weapon, enter combat stance)
APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStartAttack(Pawn, Target);
}
if (Result == EPathFollowingRequestResult::AlreadyAtGoal)
{
Memory->SubState = EPS_AI_Behavior_CombatSubState::AtCover;
Memory->PhaseDuration = FMath::RandRange(Memory->EffCoverMin, Memory->EffCoverMax);
Memory->Timer = Memory->PhaseDuration;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::AtCover);
}
else
{
Memory->bMoveRequested = true;
Memory->MoveTimeout = 5.0f;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::Engaging);
}
return EBTNodeResult::InProgress;
}
// ─── TickTask ───────────────────────────────────────────────────────────────
void UPS_AI_Behavior_BTTask_CoverShootCycle::TickTask(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds)
{
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC || !AIC->GetPawn())
{
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
}
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) { FinishLatentTask(OwnerComp, EBTNodeResult::Failed); return; }
AActor* Target = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
if (!Target)
{
AIC->SetBehaviorState(EPS_AI_Behavior_State::Alerted);
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
}
APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
if (!IPS_AI_Behavior_Interface::Execute_IsTargetActorValid(Pawn, Target))
{
AIC->StopMovement();
BB->ClearValue(PS_AI_Behavior_BB::ThreatActor);
AIC->SetBehaviorState(EPS_AI_Behavior_State::Alerted);
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
}
}
FCoverShootMemory* Memory = reinterpret_cast<FCoverShootMemory*>(NodeMemory);
// Wait for EQS to complete
if (Memory->bEQSRunning) return;
switch (Memory->SubState)
{
// ─── ENGAGING: Moving to cover ──────────────────────────────────
case EPS_AI_Behavior_CombatSubState::Engaging:
{
Memory->MoveTimeout -= DeltaSeconds;
if (!Memory->bMoveRequested || AIC->GetMoveStatus() == EPathFollowingStatus::Idle || Memory->MoveTimeout <= 0.0f)
{
Memory->bMoveRequested = false;
Memory->SubState = EPS_AI_Behavior_CombatSubState::AtCover;
Memory->PhaseDuration = FMath::RandRange(Memory->EffCoverMin, Memory->EffCoverMax);
Memory->Timer = Memory->PhaseDuration;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::AtCover);
AIC->SetFocus(Target);
CrouchAtCoverIfNeeded(Pawn, BB);
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] CoverShootCycle: at cover, ducking for %.1fs"),
*AIC->GetName(), Memory->PhaseDuration);
}
break;
}
// ─── AT COVER: Ducked, waiting ──────────────────────────────────
case EPS_AI_Behavior_CombatSubState::AtCover:
{
// Flanking check: if threat has LOS to NPC, cover is compromised → abandon
Memory->LOSCheckTimer -= DeltaSeconds;
if (Memory->LOSCheckTimer <= 0.0f)
{
Memory->LOSCheckTimer = 0.5f;
// Check if threat can see NPC at chest height (not feet)
const FVector ThreatEye = Target->GetActorLocation() + FVector(0, 0, 60.0f);
const FVector NpcChest = Pawn->GetActorLocation() + FVector(0, 0, 60.0f);
FHitResult FlankHit;
FCollisionQueryParams FlankParams(SCENE_QUERY_STAT(FlankLOS), true);
FlankParams.AddIgnoredActor(Target);
FlankParams.AddIgnoredActor(Pawn);
const bool bThreatHasLOS = !Pawn->GetWorld()->LineTraceSingleByChannel(
FlankHit, ThreatEye, NpcChest, ECC_Visibility, FlankParams);
if (bThreatHasLOS)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] CoverShootCycle: FLANKED — threat has LOS, abandoning cover"),
*AIC->GetName());
// Release current cover and fail → BT will re-run FindCover
APS_AI_Behavior_CoverPoint* OldPoint =
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (OldPoint) OldPoint->Release(Pawn);
BB->ClearValue(PS_AI_Behavior_BB::CoverPoint);
BB->ClearValue(PS_AI_Behavior_BB::CoverLocation);
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
}
AIC->ClearFocus(EAIFocusPriority::Gameplay);
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
}
}
Memory->Timer -= DeltaSeconds;
if (Memory->Timer <= 0.0f)
{
// Try to find a firing position via EQS, or peek in place
StartPeeking(OwnerComp, NodeMemory, AIC, Pawn, Target);
}
break;
}
// ─── MOVING TO FIRE: Going to firing position ───────────────────
case EPS_AI_Behavior_CombatSubState::MovingToFire:
{
Memory->MoveTimeout -= DeltaSeconds;
if (!Memory->bMoveRequested || AIC->GetMoveStatus() == EPathFollowingStatus::Idle || Memory->MoveTimeout <= 0.0f)
{
// Arrived at firing position → start shooting
Memory->bMoveRequested = false;
Memory->SubState = EPS_AI_Behavior_CombatSubState::Peeking;
Memory->PhaseDuration = FMath::RandRange(Memory->EffPeekMin, Memory->EffPeekMax);
Memory->Timer = Memory->PhaseDuration;
Memory->LOSCheckTimer = 0.3f;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::Peeking);
AIC->ClearFocus(EAIFocusPriority::Gameplay);
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
IPS_AI_Behavior_Interface::Execute_BehaviorStartAttack(Pawn, Target);
}
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] CoverShootCycle: at firing position, shooting for %.1fs"),
*AIC->GetName(), Memory->PhaseDuration);
}
break;
}
// ─── PEEKING: Shooting at target ────────────────────────────────
case EPS_AI_Behavior_CombatSubState::Peeking:
{
// Continuous LOS check
Memory->LOSCheckTimer -= DeltaSeconds;
if (Memory->LOSCheckTimer <= 0.0f)
{
Memory->LOSCheckTimer = 0.3f;
const bool bStillHasLOS = UPS_AI_Behavior_Statics::HasLineOfSight(
Pawn->GetWorld(), Pawn, Target, 150.0f);
if (!bStillHasLOS)
{
// Lost LOS → stop shooting, return to cover
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(Pawn);
}
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] CoverShootCycle: lost LOS during peek → returning to cover"),
*AIC->GetName());
ReturnToCover(OwnerComp, NodeMemory, AIC, Pawn);
break;
}
}
Memory->Timer -= DeltaSeconds;
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] Peeking: timer=%.2f/%.2f dt=%.4f"),
*AIC->GetName(), Memory->Timer, Memory->PhaseDuration, DeltaSeconds);
if (Memory->Timer <= 0.0f)
{
// Peek timer expired → stop shooting
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(Pawn);
}
Memory->CycleCount++;
// Should we advance to closer cover?
if (Memory->bCanAdvance && Memory->CycleCount >= Memory->EffMaxCycles)
{
// ─── Advance to next cover ──────────────────────
Memory->SubState = EPS_AI_Behavior_CombatSubState::Advancing;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::Advancing);
APS_AI_Behavior_CoverPoint* OldPoint =
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (OldPoint) OldPoint->Release(Pawn);
const FVector NpcLoc = Pawn->GetActorLocation();
const FVector ThreatLoc = Target->GetActorLocation();
EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Any;
if (UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent())
{
NPCType = Personality->GetNPCType();
}
float NewScore = -1.0f;
APS_AI_Behavior_CoverPoint* NewPoint =
FindAdvancingCover(GetWorld(), NpcLoc, ThreatLoc, NPCType, NewScore);
if (NewPoint)
{
NewPoint->Claim(Pawn);
BB->SetValueAsObject(PS_AI_Behavior_BB::CoverPoint, NewPoint);
BB->SetValueAsVector(PS_AI_Behavior_BB::CoverLocation, NewPoint->GetActorLocation());
Memory->bHasFiringPosition = false; // Reset firing position for new cover
AIC->MoveToLocation(
NewPoint->GetActorLocation(), 30.0f, true, true, true, false);
Memory->bMoveRequested = true;
Memory->MoveTimeout = 5.0f;
Memory->CycleCount = 0;
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] CoverShootCycle: advancing to '%s' (score %.2f)"),
*AIC->GetName(), *NewPoint->GetName(), NewScore);
}
else
{
// No better cover → return to current cover, reset cycle
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] CoverShootCycle: no advancing cover found, returning to cover"),
*AIC->GetName());
ReturnToCover(OwnerComp, NodeMemory, AIC, Pawn);
Memory->CycleCount = 0;
}
}
else
{
// ─── Return to cover ────────────────────────────
ReturnToCover(OwnerComp, NodeMemory, AIC, Pawn);
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] CoverShootCycle: returning to cover (cycle %d/%d)"),
*AIC->GetName(), Memory->CycleCount, Memory->EffMaxCycles);
}
}
break;
}
// ─── RETURNING TO COVER: Moving back after shooting ─────────────
case EPS_AI_Behavior_CombatSubState::ReturningToCover:
{
Memory->MoveTimeout -= DeltaSeconds;
if (!Memory->bMoveRequested || AIC->GetMoveStatus() == EPathFollowingStatus::Idle || Memory->MoveTimeout <= 0.0f)
{
Memory->bMoveRequested = false;
Memory->SubState = EPS_AI_Behavior_CombatSubState::AtCover;
Memory->PhaseDuration = FMath::RandRange(Memory->EffCoverMin, Memory->EffCoverMax);
Memory->Timer = Memory->PhaseDuration;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::AtCover);
AIC->SetFocus(Target);
CrouchAtCoverIfNeeded(Pawn, BB);
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] CoverShootCycle: back at cover, ducking for %.1fs"),
*AIC->GetName(), Memory->PhaseDuration);
}
break;
}
// ─── ADVANCING: Moving to next cover ────────────────────────────
case EPS_AI_Behavior_CombatSubState::Advancing:
{
Memory->MoveTimeout -= DeltaSeconds;
if (!Memory->bMoveRequested || AIC->GetMoveStatus() == EPathFollowingStatus::Idle || Memory->MoveTimeout <= 0.0f)
{
Memory->bMoveRequested = false;
Memory->SubState = EPS_AI_Behavior_CombatSubState::AtCover;
Memory->PhaseDuration = FMath::RandRange(Memory->EffCoverMin, Memory->EffCoverMax);
Memory->Timer = Memory->PhaseDuration;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::AtCover);
AIC->SetFocus(Target);
CrouchAtCoverIfNeeded(Pawn, BB);
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] CoverShootCycle: arrived at new cover, ducking"),
*AIC->GetName());
}
break;
}
}
// ─── Debug visualization ────────────────────────────────────────
#if ENABLE_DRAW_DEBUG
if (bDebugDraw && Pawn && Target)
{
UWorld* World = Pawn->GetWorld();
const FVector HeadLoc = Pawn->GetActorLocation() + FVector(0, 0, 65.0f);
const FVector TargetLoc = Target->GetActorLocation();
const FVector CoverLoc = BB->GetValueAsVector(PS_AI_Behavior_BB::CoverLocation);
// Sub-state label
const TCHAR* SubStateStr = TEXT("?");
FColor SubStateColor = FColor::White;
switch (Memory->SubState)
{
case EPS_AI_Behavior_CombatSubState::Engaging: SubStateStr = TEXT("ENGAGING"); SubStateColor = FColor::Yellow; break;
case EPS_AI_Behavior_CombatSubState::AtCover: SubStateStr = TEXT("AT COVER"); SubStateColor = FColor::Cyan; break;
case EPS_AI_Behavior_CombatSubState::MovingToFire: SubStateStr = TEXT("→ FIRE POS"); SubStateColor = FColor::Magenta; break;
case EPS_AI_Behavior_CombatSubState::Peeking: SubStateStr = TEXT("PEEKING"); SubStateColor = FColor::Red; break;
case EPS_AI_Behavior_CombatSubState::ReturningToCover: SubStateStr = TEXT("→ COVER"); SubStateColor = FColor::Blue; break;
case EPS_AI_Behavior_CombatSubState::Advancing: SubStateStr = TEXT("ADVANCING"); SubStateColor = FColor::Orange; break;
}
DrawDebugString(World, HeadLoc + FVector(0, 0, 30.0f),
FString::Printf(TEXT("%s [%.1fs] C:%d/%d"), SubStateStr, Memory->Timer, Memory->CycleCount, Memory->EffMaxCycles),
nullptr, SubStateColor, 0.0f, true);
// LOS line to target (eye height on both ends)
const bool bLOS = UPS_AI_Behavior_Statics::HasLineOfSight(World, Pawn, Target, 150.0f);
const FVector TargetHeadLoc = TargetLoc + FVector(0, 0, 60.0f);
DrawDebugLine(World, HeadLoc, TargetHeadLoc,
bLOS ? FColor::Green : FColor::Red, false, 0.0f, 0, 1.0f);
// Firing position marker
if (Memory->bHasFiringPosition)
{
// Firing position: red solid point
DrawDebugSphere(World, Memory->FiringPosition + FVector(0, 0, 30.0f),
20.0f, 8, FColor::Red, false, 0.0f);
// Line cover → firing position (white)
DrawDebugLine(World, CoverLoc + FVector(0, 0, 20.0f),
Memory->FiringPosition + FVector(0, 0, 20.0f),
FColor::White, false, 0.0f, 0, 1.0f);
// Line firing position → threat (green, head height)
DrawDebugLine(World, Memory->FiringPosition + FVector(0, 0, 65.0f),
TargetHeadLoc,
FColor(0, 200, 0), false, 0.0f, 0, 1.0f);
DrawDebugString(World, Memory->FiringPosition + FVector(0, 0, 50.0f),
TEXT("FIRE"), nullptr, FColor::Red, 0.0f, true);
}
else if (Memory->SubState == EPS_AI_Behavior_CombatSubState::Peeking)
{
// No firing position — shooting in place
DrawDebugSphere(World, Pawn->GetActorLocation() + FVector(0, 0, 30.0f),
15.0f, 6, FColor::Orange, false, 0.0f);
DrawDebugString(World, Pawn->GetActorLocation() + FVector(0, 0, 50.0f),
TEXT("IN PLACE"), nullptr, FColor::Orange, 0.0f, true);
}
}
#endif
}
// ─── StartPeeking ───────────────────────────────────────────────────────────
void UPS_AI_Behavior_BTTask_CoverShootCycle::StartPeeking(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APS_AI_Behavior_AIController* AIC, APawn* Pawn, AActor* Target)
{
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
FCoverShootMemory* Memory = reinterpret_cast<FCoverShootMemory*>(NodeMemory);
// If we have a FiringPositionQuery → find a firing position via EQS
if (FiringPositionQuery)
{
RunFiringPositionQuery(OwnerComp, NodeMemory, Pawn);
return; // EQS callback will handle the transition
}
// No EQS query → legacy behavior: check LOS from cover and shoot in place
const bool bHasLOS = UPS_AI_Behavior_Statics::HasLineOfSight(
Pawn->GetWorld(), Pawn, Target, 150.0f);
if (!bHasLOS)
{
// No LOS and no firing position query → try to advance
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] CoverShootCycle: no LOS from cover and no FiringPositionQuery → advancing"),
*AIC->GetName());
Memory->SubState = EPS_AI_Behavior_CombatSubState::Advancing;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::Advancing);
APS_AI_Behavior_CoverPoint* OldPoint =
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (OldPoint) OldPoint->Release(Pawn);
const FVector NpcLoc = Pawn->GetActorLocation();
const FVector ThreatLoc = Target->GetActorLocation();
EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Any;
if (UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent())
{
NPCType = Personality->GetNPCType();
}
float NewScore = -1.0f;
APS_AI_Behavior_CoverPoint* NewPoint =
FindAdvancingCover(GetWorld(), NpcLoc, ThreatLoc, NPCType, NewScore);
if (NewPoint)
{
NewPoint->Claim(Pawn);
BB->SetValueAsObject(PS_AI_Behavior_BB::CoverPoint, NewPoint);
BB->SetValueAsVector(PS_AI_Behavior_BB::CoverLocation, NewPoint->GetActorLocation());
AIC->MoveToLocation(NewPoint->GetActorLocation(), 30.0f, true, true, true, false);
Memory->bMoveRequested = true;
Memory->MoveTimeout = 5.0f;
Memory->CycleCount = 0;
}
else
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] CoverShootCycle: no LOS and no advancing cover → abandoning cover"),
*AIC->GetName());
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
}
return;
}
// Has LOS → shoot in place (legacy)
Memory->SubState = EPS_AI_Behavior_CombatSubState::Peeking;
Memory->PhaseDuration = FMath::RandRange(Memory->EffPeekMin, Memory->EffPeekMax);
Memory->Timer = Memory->PhaseDuration;
Memory->LOSCheckTimer = 0.3f;
Memory->bHasFiringPosition = false;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::Peeking);
AIC->ClearFocus(EAIFocusPriority::Gameplay);
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
IPS_AI_Behavior_Interface::Execute_BehaviorStartAttack(Pawn, Target);
}
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] CoverShootCycle: peeking in place for %.1fs"),
*AIC->GetName(), Memory->PhaseDuration);
}
// ─── ReturnToCover ──────────────────────────────────────────────────────────
void UPS_AI_Behavior_BTTask_CoverShootCycle::ReturnToCover(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APS_AI_Behavior_AIController* AIC, APawn* Pawn)
{
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
FCoverShootMemory* Memory = reinterpret_cast<FCoverShootMemory*>(NodeMemory);
// If we have a firing position (we moved away from cover), move back
if (Memory->bHasFiringPosition)
{
const FVector CoverLoc = BB->GetValueAsVector(PS_AI_Behavior_BB::CoverLocation);
Memory->SubState = EPS_AI_Behavior_CombatSubState::ReturningToCover;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::ReturningToCover);
AIC->MoveToLocation(CoverLoc, 30.0f, true, true, true, false);
Memory->bMoveRequested = true;
Memory->MoveTimeout = 5.0f;
}
else
{
// No firing position was used → already at cover, duck directly
Memory->SubState = EPS_AI_Behavior_CombatSubState::AtCover;
Memory->PhaseDuration = FMath::RandRange(Memory->EffCoverMin, Memory->EffCoverMax);
Memory->Timer = Memory->PhaseDuration;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::AtCover);
AIC->SetFocus(Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor)));
CrouchAtCoverIfNeeded(Pawn, BB);
}
}
// ─── EQS Firing Position ────────────────────────────────────────────────────
void UPS_AI_Behavior_BTTask_CoverShootCycle::RunFiringPositionQuery(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, APawn* Pawn)
{
FCoverShootMemory* Memory = reinterpret_cast<FCoverShootMemory*>(NodeMemory);
UWorld* World = Pawn->GetWorld();
UEnvQueryManager* EQSManager = UEnvQueryManager::GetCurrent(World);
if (!EQSManager)
{
// Fallback: peek in place
Memory->bHasFiringPosition = false;
return;
}
Memory->bEQSRunning = true;
FEnvQueryRequest Request(FiringPositionQuery, Pawn);
Request.Execute(EEnvQueryRunMode::SingleResult,
FQueryFinishedSignature::CreateUObject(this,
&UPS_AI_Behavior_BTTask_CoverShootCycle::OnFiringPositionQueryFinished,
&OwnerComp, NodeMemory));
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] CoverShootCycle: firing position EQS launched"),
*Pawn->GetName());
}
void UPS_AI_Behavior_BTTask_CoverShootCycle::OnFiringPositionQueryFinished(
TSharedPtr<FEnvQueryResult> Result,
UBehaviorTreeComponent* OwnerComp, uint8* NodeMemory)
{
if (!OwnerComp || !NodeMemory) return;
FCoverShootMemory* Memory = reinterpret_cast<FCoverShootMemory*>(NodeMemory);
Memory->bEQSRunning = false;
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp->GetAIOwner());
if (!AIC || !AIC->GetPawn()) return;
APawn* Pawn = AIC->GetPawn();
UBlackboardComponent* BB = OwnerComp->GetBlackboardComponent();
if (!BB) return;
AActor* Target = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
if (Result.IsValid() && Result->IsSuccessful())
{
// Found a firing position → move to it
Memory->FiringPosition = Result->GetItemAsLocation(0);
Memory->bHasFiringPosition = true;
Memory->SubState = EPS_AI_Behavior_CombatSubState::MovingToFire;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::MovingToFire);
// Stand up to move to firing position
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
}
AIC->MoveToLocation(Memory->FiringPosition, 30.0f, true, true, true, false);
Memory->bMoveRequested = true;
Memory->MoveTimeout = 5.0f;
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] CoverShootCycle: moving to firing position %s"),
*AIC->GetName(), *Memory->FiringPosition.ToString());
#if ENABLE_DRAW_DEBUG
if (bDebugDraw)
{
UWorld* World = Pawn->GetWorld();
// Draw all EQS firing candidates as solid boxes
const int32 NumItems = Result->Items.Num();
for (int32 i = 0; i < NumItems; ++i)
{
if (!Result->Items[i].IsValid()) continue;
const FVector ItemLoc = Result->GetItemAsLocation(i);
const float ItemScore = Result->GetItemScore(i);
const uint8 G = static_cast<uint8>(FMath::Lerp(50.0f, 255.0f, FMath::Clamp(ItemScore, 0.0f, 1.0f)));
DrawDebugBox(World, ItemLoc + FVector(0, 0, 20.0f),
FVector(8.0f), FColor(255, G, 0), false, 5.0f);
}
// Chosen firing position: large red point
DrawDebugSphere(World, Memory->FiringPosition + FVector(0, 0, 30.0f),
25.0f, 8, FColor::Red, false, 5.0f);
DrawDebugString(World, Memory->FiringPosition + FVector(0, 0, 55.0f),
FString::Printf(TEXT("FIRE POS (%d)"), NumItems),
nullptr, FColor::Red, 5.0f, true);
}
#endif
}
else
{
// EQS failed → fallback: peek in place if LOS exists
Memory->bHasFiringPosition = false;
const int32 NumItems = Result.IsValid() ? Result->Items.Num() : 0;
const FVector CoverLoc = BB->GetValueAsVector(PS_AI_Behavior_BB::CoverLocation);
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] CoverShootCycle: firing position EQS FAILED at cover %s — items=%d"),
*AIC->GetName(), *CoverLoc.ToString(), NumItems);
#if ENABLE_DRAW_DEBUG
if (bDebugDraw)
{
UWorld* World = Pawn->GetWorld();
const FVector PawnLoc = Pawn->GetActorLocation();
DrawDebugSphere(World, PawnLoc + FVector(0, 0, 30.0f),
20.0f, 8, FColor::Orange, false, 5.0f);
DrawDebugString(World, PawnLoc + FVector(0, 0, 55.0f),
TEXT("NO FIRE POS"), nullptr, FColor::Orange, 5.0f, true);
}
#endif
if (Target)
{
const bool bHasLOS = UPS_AI_Behavior_Statics::HasLineOfSight(
Pawn->GetWorld(), Pawn, Target, 150.0f);
if (bHasLOS)
{
Memory->SubState = EPS_AI_Behavior_CombatSubState::Peeking;
Memory->PhaseDuration = FMath::RandRange(Memory->EffPeekMin, Memory->EffPeekMax);
Memory->Timer = Memory->PhaseDuration;
Memory->LOSCheckTimer = 0.3f;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::Peeking);
AIC->ClearFocus(EAIFocusPriority::Gameplay);
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
IPS_AI_Behavior_Interface::Execute_BehaviorStartAttack(Pawn, Target);
}
}
else
{
// No LOS, no firing position → stay in cover, wait
Memory->SubState = EPS_AI_Behavior_CombatSubState::AtCover;
Memory->PhaseDuration = FMath::RandRange(Memory->EffCoverMin, Memory->EffCoverMax);
Memory->Timer = Memory->PhaseDuration;
SetSubState(BB, EPS_AI_Behavior_CombatSubState::AtCover);
AIC->SetFocus(Target);
}
}
}
}
// ─── AbortTask ──────────────────────────────────────────────────────────────
EBTNodeResult::Type UPS_AI_Behavior_BTTask_CoverShootCycle::AbortTask(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory)
{
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (AIC)
{
AIC->StopMovement();
AIC->ClearFocus(EAIFocusPriority::Gameplay);
APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(Pawn);
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
}
}
return EBTNodeResult::Aborted;
}
// ─── OnTaskFinished ─────────────────────────────────────────────────────────
void UPS_AI_Behavior_BTTask_CoverShootCycle::OnTaskFinished(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, EBTNodeResult::Type TaskResult)
{
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (AIC)
{
AIC->ClearFocus(EAIFocusPriority::Gameplay);
APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(Pawn);
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
}
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (BB)
{
APS_AI_Behavior_CoverPoint* Point =
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (Point && Pawn)
{
Point->Release(Pawn);
}
}
}
Super::OnTaskFinished(OwnerComp, NodeMemory, TaskResult);
}
// ─── Advancing Cover Search ─────────────────────────────────────────────────
APS_AI_Behavior_CoverPoint* UPS_AI_Behavior_BTTask_CoverShootCycle::FindAdvancingCover(
const UWorld* World, const FVector& NpcLoc, const FVector& ThreatLoc,
EPS_AI_Behavior_NPCType NPCType, float& OutScore) const
{
APS_AI_Behavior_CoverPoint* BestPoint = nullptr;
OutScore = -1.0f;
const float NpcDistToThreat = FVector::Dist(NpcLoc, ThreatLoc);
for (TActorIterator<APS_AI_Behavior_CoverPoint> It(const_cast<UWorld*>(World)); It; ++It)
{
APS_AI_Behavior_CoverPoint* Point = *It;
if (!Point || !Point->bEnabled) continue;
if (Point->PointType != CoverPointType) continue;
if (!Point->IsAccessibleTo(NPCType)) continue;
if (!Point->HasRoom()) continue;
const float DistFromNpc = FVector::Dist(NpcLoc, Point->GetActorLocation());
if (DistFromNpc > AdvanceSearchRadius) continue;
const float CoverDistToThreat = FVector::Dist(Point->GetActorLocation(), ThreatLoc);
if (CoverDistToThreat >= NpcDistToThreat) continue;
float Score = Point->EvaluateAgainstThreat(ThreatLoc);
Score += FMath::GetMappedRangeValueClamped(
FVector2D(0.0f, AdvanceSearchRadius), FVector2D(0.15f, 0.0f), DistFromNpc);
if (NpcDistToThreat > 0.0f)
{
const float AdvanceRatio = (NpcDistToThreat - CoverDistToThreat) / NpcDistToThreat;
Score += AdvancementBias * AdvanceRatio * 0.3f;
}
// LOS bonus — favor covers near positions with LOS to threat
{
const FVector TraceStart = Point->GetActorLocation() + FVector(0, 0, 150.0f);
FHitResult Hit;
FCollisionQueryParams Params(SCENE_QUERY_STAT(CoverLOS), true);
if (!const_cast<UWorld*>(World)->LineTraceSingleByChannel(
Hit, TraceStart, ThreatLoc, ECC_Visibility, Params))
{
Score += 0.3f;
}
}
if (Score > OutScore)
{
OutScore = Score;
BestPoint = Point;
}
}
return BestPoint;
}
// ─── GetStaticDescription ───────────────────────────────────────────────────
FString UPS_AI_Behavior_BTTask_CoverShootCycle::GetStaticDescription() const
{
return FString::Printf(
TEXT("Cover-shoot cycle.\nPeek: %.1f%.1fs | Cover: %.1f%.1fs\nAdvance after %d cycles (radius %.0fcm)\nFiring EQS: %s"),
PeekDurationMin, PeekDurationMax,
CoverDurationMin, CoverDurationMax,
MaxCyclesBeforeAdvance, AdvanceSearchRadius,
FiringPositionQuery ? *FiringPositionQuery->GetName() : TEXT("None (in-place)"));
}

View File

@@ -6,6 +6,7 @@
#include "PS_AI_Behavior_SplineNetwork.h" #include "PS_AI_Behavior_SplineNetwork.h"
#include "PS_AI_Behavior_SplinePath.h" #include "PS_AI_Behavior_SplinePath.h"
#include "PS_AI_Behavior_PersonalityComponent.h" #include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_Statics.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "Navigation/PathFollowingComponent.h" #include "Navigation/PathFollowingComponent.h"
@@ -31,6 +32,101 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindAndFollowSpline::ExecuteTask(
return EBTNodeResult::Failed; return EBTNodeResult::Failed;
} }
// Respect a manual StopFollowing(): do not search for or resume a spline until
// the caller explicitly StartFollowing() again. Without this, the task would
// resume any CurrentSpline and override the stop on the very next tick.
if (Follower->bManuallyStopped)
{
return EBTNodeResult::Failed;
}
// Debug: log state on entry
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] FindAndFollowSpline: bIsFollowing=%d CurrentSpline=%s"),
*AIC->GetName(),
(int32)Follower->bIsFollowing,
Follower->CurrentSpline ? *Follower->CurrentSpline->GetName() : TEXT("null"));
}
// If already following a spline, don't re-search — just succeed immediately
// The Follow Spline task will continue the movement
if (Follower->bIsFollowing && Follower->CurrentSpline)
{
return EBTNodeResult::Succeeded;
}
// If paused (e.g. after combat abort) but still has a valid spline, resume it
// Find the closest point on the spline from NPC's CURRENT position (not old pause point)
if (!Follower->bIsFollowing && Follower->CurrentSpline)
{
const FVector PawnLoc = AIC->GetPawn()->GetActorLocation();
float ClosestDist = 0.0f;
FVector ClosestPoint = FVector::ZeroVector;
const float GapToSpline = Follower->CurrentSpline->GetClosestPointOnSpline(
PawnLoc, ClosestDist, ClosestPoint);
// Determine direction from pawn forward vs spline tangent at closest point
const FVector PawnFwd = AIC->GetPawn()->GetActorForwardVector();
const FVector SplineDir = Follower->CurrentSpline->GetWorldDirectionAtDistance(ClosestDist);
const bool bForward = FVector::DotProduct(PawnFwd, SplineDir) >= 0.0f;
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindAndFollowSpline: resuming spline '%s' at closest point (gap=%.0fcm, dist=%.0f, bWalkToSpline=%d, AcceptanceRadius=%.0f)"),
*AIC->GetName(), *Follower->CurrentSpline->GetName(), GapToSpline, ClosestDist,
(int32)bWalkToSpline, AcceptanceRadius);
}
if (bWalkToSpline && GapToSpline > AcceptanceRadius)
{
// Walk to closest spline point via NavMesh
Follower->CurrentDistance = ClosestDist;
const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation(
ClosestPoint, AcceptanceRadius, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectDestinationToNavigation=*/true,
/*bCanStrafe=*/false);
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindAndFollowSpline: MoveToLocation result=%d (0=Failed, 1=AlreadyAtGoal, 2=RequestSuccessful)"),
*AIC->GetName(), (int32)Result);
}
if (Result == EPathFollowingRequestResult::AlreadyAtGoal)
{
Follower->StartFollowingAtDistance(Follower->CurrentSpline, ClosestDist, bForward);
return EBTNodeResult::Succeeded;
}
if (Result != EPathFollowingRequestResult::Failed)
{
FFindSplineMemory* Memory = reinterpret_cast<FFindSplineMemory*>(NodeMemory);
Memory->bMovingToSpline = true;
return EBTNodeResult::InProgress;
}
// Pathfinding failed — fall through to full re-search below
UE_LOG(LogPS_AI_Behavior, Warning,
TEXT("[%s] FindAndFollowSpline: pathfinding FAILED to reach spline, falling through to re-search"),
*AIC->GetName());
}
else
{
// Close enough — clear any residual movement request and start following
AIC->StopMovement();
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindAndFollowSpline: close enough, StartFollowingAtDistance(dist=%.0f, fwd=%d)"),
*AIC->GetName(), ClosestDist, (int32)bForward);
}
Follower->StartFollowingAtDistance(Follower->CurrentSpline, ClosestDist, bForward);
return EBTNodeResult::Succeeded;
}
}
// Determine NPC type // Determine NPC type
EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Civilian; EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Civilian;
UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent(); UPS_AI_Behavior_PersonalityComponent* Personality = AIC->GetPersonalityComponent();
@@ -56,9 +152,12 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindAndFollowSpline::ExecuteTask(
AIC->GetPawn()->GetActorLocation(), NPCType, MaxSearchDistance, AIC->GetPawn()->GetActorLocation(), NPCType, MaxSearchDistance,
ClosestSpline, DistAlongSpline)) ClosestSpline, DistAlongSpline))
{ {
UE_LOG(LogPS_AI_Behavior, Verbose, if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindAndFollowSpline: no accessible spline within %.0fcm."), TEXT("[%s] FindAndFollowSpline: no accessible spline within %.0fcm."),
*AIC->GetName(), MaxSearchDistance); *AIC->GetName(), MaxSearchDistance);
}
return EBTNodeResult::Failed; return EBTNodeResult::Failed;
} }
@@ -66,30 +165,74 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindAndFollowSpline::ExecuteTask(
const FVector SplinePoint = ClosestSpline->GetWorldLocationAtDistance(DistAlongSpline); const FVector SplinePoint = ClosestSpline->GetWorldLocationAtDistance(DistAlongSpline);
const float GapToSpline = FVector::Dist(AIC->GetPawn()->GetActorLocation(), SplinePoint); const float GapToSpline = FVector::Dist(AIC->GetPawn()->GetActorLocation(), SplinePoint);
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindAndFollowSpline: found spline '%s' at dist=%.0f/%.0f, gap=%.0fcm"),
*AIC->GetName(), *ClosestSpline->GetName(), DistAlongSpline,
ClosestSpline->GetSplineLength(), GapToSpline);
}
if (bWalkToSpline && GapToSpline > AcceptanceRadius) if (bWalkToSpline && GapToSpline > AcceptanceRadius)
{ {
// Walk to spline first via NavMesh // Try to reach the closest spline point. If blocked, sample other points along the spline.
const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation( const float SplineLength = ClosestSpline->GetSplineLength();
SplinePoint, AcceptanceRadius, /*bStopOnOverlap=*/true, const int32 NumSamples = 8;
/*bUsePathfinding=*/true, /*bProjectDestinationToNavigation=*/true, const float SampleStep = SplineLength / NumSamples;
/*bCanStrafe=*/false);
if (Result == EPathFollowingRequestResult::Failed) // Start with the closest point, then try evenly spaced samples
TArray<float> DistancesToTry;
DistancesToTry.Add(DistAlongSpline);
for (int32 i = 1; i <= NumSamples; ++i)
{ {
// Can't reach via NavMesh — try starting anyway (snap) const float D = FMath::Fmod(DistAlongSpline + i * SampleStep, SplineLength);
Follower->StartFollowingAtDistance(ClosestSpline, DistAlongSpline); DistancesToTry.Add(D);
}
float ReachableDist = -1.0f;
FVector ReachablePoint = FVector::ZeroVector;
for (float TestDist : DistancesToTry)
{
const FVector TestPoint = ClosestSpline->GetWorldLocationAtDistance(TestDist);
const EPathFollowingRequestResult::Type TestResult = AIC->MoveToLocation(
TestPoint, AcceptanceRadius, true, true, true, false);
if (TestResult == EPathFollowingRequestResult::AlreadyAtGoal)
{
const FVector Fwd = AIC->GetPawn()->GetActorForwardVector();
const FVector SpDir = ClosestSpline->GetWorldDirectionAtDistance(TestDist);
Follower->StartFollowingAtDistance(ClosestSpline, TestDist,
FVector::DotProduct(Fwd, SpDir) >= 0.0f);
return EBTNodeResult::Succeeded; return EBTNodeResult::Succeeded;
} }
if (Result == EPathFollowingRequestResult::AlreadyAtGoal) if (TestResult != EPathFollowingRequestResult::Failed)
{ {
Follower->StartFollowingAtDistance(ClosestSpline, DistAlongSpline); ReachableDist = TestDist;
return EBTNodeResult::Succeeded; ReachablePoint = TestPoint;
break; // Found a reachable point, use it
} }
// Store the spline to connect to after reaching it // This point is blocked, try next
AIC->StopMovement();
}
if (ReachableDist < 0.0f)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindAndFollowSpline: no reachable point on spline '%s' (%d samples tested)"),
*AIC->GetName(), *ClosestSpline->GetName(), DistancesToTry.Num());
return EBTNodeResult::Failed;
}
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindAndFollowSpline: walking to reachable spline point at dist=%.0f (gap=%.0fcm)"),
*AIC->GetName(), ReachableDist,
FVector::Dist(AIC->GetPawn()->GetActorLocation(), ReachablePoint));
Follower->CurrentSpline = ClosestSpline; Follower->CurrentSpline = ClosestSpline;
Follower->CurrentDistance = DistAlongSpline; Follower->CurrentDistance = ReachableDist;
FFindSplineMemory* Memory = reinterpret_cast<FFindSplineMemory*>(NodeMemory); FFindSplineMemory* Memory = reinterpret_cast<FFindSplineMemory*>(NodeMemory);
Memory->bMovingToSpline = true; Memory->bMovingToSpline = true;
@@ -97,7 +240,26 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindAndFollowSpline::ExecuteTask(
} }
// Close enough — start immediately // Close enough — start immediately
Follower->StartFollowingAtDistance(ClosestSpline, DistAlongSpline); // Choose direction based on NPC's current movement direction vs spline tangent
const FVector NpcForward = AIC->GetPawn()->GetActorForwardVector();
const FVector SplineDirForward = ClosestSpline->GetWorldDirectionAtDistance(DistAlongSpline);
const float DotForward = FVector::DotProduct(NpcForward, SplineDirForward);
const bool bForward = (DotForward >= 0.0f);
// If NPC is very close to an end, ensure we don't start going into a wall
const float SplineLen = ClosestSpline->GetSplineLength();
const float MinEndMargin = 50.0f;
bool bFinalForward = bForward;
if (bFinalForward && DistAlongSpline > SplineLen - MinEndMargin)
{
bFinalForward = false; // Too close to the end, go backward
}
else if (!bFinalForward && DistAlongSpline < MinEndMargin)
{
bFinalForward = true; // Too close to the start, go forward
}
Follower->StartFollowingAtDistance(ClosestSpline, DistAlongSpline, bFinalForward);
return EBTNodeResult::Succeeded; return EBTNodeResult::Succeeded;
} }
@@ -114,6 +276,18 @@ void UPS_AI_Behavior_BTTask_FindAndFollowSpline::TickTask(
return; return;
} }
// Manual StopFollowing() requested mid-approach: cancel the move-to-spline and bail.
if (UPS_AI_Behavior_SplineFollowerComponent* StopCheck =
AIC->GetPawn()->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>())
{
if (StopCheck->bManuallyStopped)
{
AIC->StopMovement();
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
}
}
// Check if we've reached the spline // Check if we've reached the spline
if (AIC->GetMoveStatus() == EPathFollowingStatus::Idle) if (AIC->GetMoveStatus() == EPathFollowingStatus::Idle)
{ {

View File

@@ -2,8 +2,10 @@
#include "BT/PS_AI_Behavior_BTTask_FindCover.h" #include "BT/PS_AI_Behavior_BTTask_FindCover.h"
#include "PS_AI_Behavior_AIController.h" #include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_CoverPoint.h" #include "PS_AI_Behavior_CoverPoint.h"
#include "PS_AI_Behavior_PersonalityComponent.h" #include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_Statics.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "BehaviorTree/BlackboardComponent.h" #include "BehaviorTree/BlackboardComponent.h"
#include "NavigationSystem.h" #include "NavigationSystem.h"
@@ -11,6 +13,21 @@
#include "CollisionQueryParams.h" #include "CollisionQueryParams.h"
#include "Engine/World.h" #include "Engine/World.h"
#include "EngineUtils.h" #include "EngineUtils.h"
#include "EnvironmentQuery/EnvQuery.h"
#include "EnvironmentQuery/EnvQueryManager.h"
#include "DrawDebugHelpers.h"
namespace
{
FColor ScoreToColor(float Score)
{
// 0 = red, 0.5 = yellow, 1 = green
const float Clamped = FMath::Clamp(Score, 0.0f, 1.0f);
const uint8 R = static_cast<uint8>(FMath::Lerp(255.0f, 0.0f, Clamped));
const uint8 G = static_cast<uint8>(FMath::Lerp(0.0f, 255.0f, Clamped));
return FColor(R, G, 0);
}
}
UPS_AI_Behavior_BTTask_FindCover::UPS_AI_Behavior_BTTask_FindCover() UPS_AI_Behavior_BTTask_FindCover::UPS_AI_Behavior_BTTask_FindCover()
{ {
@@ -83,6 +100,17 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindCover::ExecuteTask(
} }
const float Score = EvaluateCoverQuality(World, NavLoc.Location, ThreatLoc, NpcLoc); const float Score = EvaluateCoverQuality(World, NavLoc.Location, ThreatLoc, NpcLoc);
#if ENABLE_DRAW_DEBUG
if (bDebugDraw)
{
DrawDebugSphere(World, NavLoc.Location + FVector(0, 0, 30.0f),
15.0f, 6, ScoreToColor(Score), false, 5.0f);
DrawDebugString(World, NavLoc.Location + FVector(0, 0, 55.0f),
FString::Printf(TEXT("%.2f"), Score), nullptr, FColor::White, 5.0f, true);
}
#endif
if (Score > BestScore) if (Score > BestScore)
{ {
BestScore = Score; BestScore = Score;
@@ -95,7 +123,10 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindCover::ExecuteTask(
if (BestScore < 0.1f) if (BestScore < 0.1f)
{ {
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] FindCover: no suitable cover found."), *AIC->GetName()); if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] FindCover: no suitable cover found."), *AIC->GetName());
}
return EBTNodeResult::Failed; return EBTNodeResult::Failed;
} }
@@ -105,22 +136,56 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindCover::ExecuteTask(
ChosenPoint->Claim(AIC->GetPawn()); ChosenPoint->Claim(AIC->GetPawn());
BB->SetValueAsObject(PS_AI_Behavior_BB::CoverPoint, ChosenPoint); BB->SetValueAsObject(PS_AI_Behavior_BB::CoverPoint, ChosenPoint);
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] FindCover: using manual %s '%s' (score %.2f)"), if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] FindCover: using manual %s '%s' (score %.2f)"),
*AIC->GetName(), *AIC->GetName(),
ChosenPoint->PointType == EPS_AI_Behavior_CoverPointType::Cover ? TEXT("Cover") : TEXT("HidingSpot"), ChosenPoint->PointType == EPS_AI_Behavior_CoverPointType::Cover ? TEXT("Cover") : TEXT("HidingSpot"),
*ChosenPoint->GetName(), BestScore); *ChosenPoint->GetName(), BestScore);
} }
}
else else
{ {
BB->ClearValue(PS_AI_Behavior_BB::CoverPoint); BB->ClearValue(PS_AI_Behavior_BB::CoverPoint);
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] FindCover: using procedural cover (score %.2f)"), if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] FindCover: using procedural cover (score %.2f)"),
*AIC->GetName(), BestScore); *AIC->GetName(), BestScore);
} }
}
BB->SetValueAsVector(PS_AI_Behavior_BB::CoverLocation, BestCoverPos); BB->SetValueAsVector(PS_AI_Behavior_BB::CoverLocation, BestCoverPos);
// Navigate to cover #if ENABLE_DRAW_DEBUG
if (bDebugDraw)
{
// Search radius circle
DrawDebugCircle(World, NpcLoc, SearchRadius, 32, FColor(80, 80, 80),
false, 5.0f, 0, 1.0f, FVector::RightVector, FVector::ForwardVector);
// Chosen cover: large sphere + line from NPC
DrawDebugSphere(World, BestCoverPos + FVector(0, 0, 40.0f),
30.0f, 10, FColor::Cyan, false, 5.0f);
DrawDebugLine(World, NpcLoc, BestCoverPos, FColor::Cyan, false, 5.0f, 0, 1.5f);
DrawDebugString(World, BestCoverPos + FVector(0, 0, 65.0f),
FString::Printf(TEXT("COVER %.2f %s"), BestScore,
ChosenPoint ? TEXT("(Manual)") : TEXT("(Procedural)")),
nullptr, FColor::Cyan, 5.0f, true);
// Line to threat
DrawDebugLine(World, BestCoverPos, ThreatLoc, FColor(255, 100, 0), false, 5.0f, 0, 1.0f);
}
#endif
FCoverMemory* Memory = reinterpret_cast<FCoverMemory*>(NodeMemory);
// If we have a refinement EQS query and a manual CoverPoint, refine the position
if (RefinementQuery && ChosenPoint)
{
RunRefinementQuery(OwnerComp, NodeMemory, AIC->GetPawn(), BestCoverPos);
return EBTNodeResult::InProgress;
}
// Navigate to cover directly (no refinement)
const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation( const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation(
BestCoverPos, AcceptanceRadius, /*bStopOnOverlap=*/true, BestCoverPos, AcceptanceRadius, /*bStopOnOverlap=*/true,
/*bUsePathfinding=*/true, /*bProjectDestinationToNavigation=*/true, /*bUsePathfinding=*/true, /*bProjectDestinationToNavigation=*/true,
@@ -136,7 +201,6 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindCover::ExecuteTask(
return EBTNodeResult::Succeeded; return EBTNodeResult::Succeeded;
} }
FCoverMemory* Memory = reinterpret_cast<FCoverMemory*>(NodeMemory);
Memory->bMoveRequested = true; Memory->bMoveRequested = true;
return EBTNodeResult::InProgress; return EBTNodeResult::InProgress;
} }
@@ -145,6 +209,7 @@ void UPS_AI_Behavior_BTTask_FindCover::TickTask(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds)
{ {
FCoverMemory* Memory = reinterpret_cast<FCoverMemory*>(NodeMemory); FCoverMemory* Memory = reinterpret_cast<FCoverMemory*>(NodeMemory);
if (Memory->bEQSRunning) return; // Waiting for EQS callback
if (!Memory->bMoveRequested) return; if (!Memory->bMoveRequested) return;
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner()); APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
@@ -152,7 +217,57 @@ void UPS_AI_Behavior_BTTask_FindCover::TickTask(
if (AIC->GetMoveStatus() == EPathFollowingStatus::Idle) if (AIC->GetMoveStatus() == EPathFollowingStatus::Idle)
{ {
// Verify we actually reached the cover — pathfinding can stop early (blocked by other NPCs, etc.)
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
APawn* Pawn = AIC->GetPawn();
if (BB && Pawn)
{
const FVector CoverLoc = BB->GetValueAsVector(PS_AI_Behavior_BB::CoverLocation);
const float DistToCover = FVector::Dist2D(Pawn->GetActorLocation(), CoverLoc);
if (UPS_AI_Behavior_Statics::IsDebugEnabled(AIC->GetPawn()))
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindCover: MoveStatus=Idle, dist2D=%.0fcm, acceptance=%.0fcm, coverLoc=%s, pawnLoc=%s"),
*AIC->GetName(), DistToCover, AcceptanceRadius,
*CoverLoc.ToString(), *Pawn->GetActorLocation().ToString());
}
if (DistToCover > AcceptanceRadius * 2.0f)
{
// Still too far — retry movement
const EPathFollowingRequestResult::Type Retry = AIC->MoveToLocation(
CoverLoc, AcceptanceRadius, true, true, true, false);
if (Retry == EPathFollowingRequestResult::Failed)
{
// Can't reach at all — give up
Memory->bMoveRequested = false; Memory->bMoveRequested = false;
FinishLatentTask(OwnerComp, EBTNodeResult::Failed);
return;
}
if (Retry != EPathFollowingRequestResult::AlreadyAtGoal)
{
return; // Keep waiting for retry movement
}
// AlreadyAtGoal — fall through to success
}
}
Memory->bMoveRequested = false;
// Crouch at cover if the point requires it
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>() && BB)
{
const APS_AI_Behavior_CoverPoint* CoverPt =
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (CoverPt && CoverPt->bCrouch)
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, true);
}
}
FinishLatentTask(OwnerComp, EBTNodeResult::Succeeded); FinishLatentTask(OwnerComp, EBTNodeResult::Succeeded);
} }
} }
@@ -165,6 +280,13 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindCover::AbortTask(
{ {
AIC->StopMovement(); AIC->StopMovement();
// Stand up if crouching
APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, false);
}
// Release any claimed cover point // Release any claimed cover point
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (BB) if (BB)
@@ -173,7 +295,7 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FindCover::AbortTask(
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint)); Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (Point) if (Point)
{ {
Point->Release(AIC->GetPawn()); if (Pawn) Point->Release(Pawn);
BB->ClearValue(PS_AI_Behavior_BB::CoverPoint); BB->ClearValue(PS_AI_Behavior_BB::CoverPoint);
} }
} }
@@ -215,6 +337,30 @@ APS_AI_Behavior_CoverPoint* UPS_AI_Behavior_BTTask_FindCover::FindBestManualCove
Score += FMath::GetMappedRangeValueClamped( Score += FMath::GetMappedRangeValueClamped(
FVector2D(0.0f, SearchRadius), FVector2D(0.15f, 0.0f), Dist); FVector2D(0.0f, SearchRadius), FVector2D(0.15f, 0.0f), Dist);
// Advancement bias — prefer covers closer to threat than NPC is
if (AdvancementBias > 0.0f)
{
const float NpcDistToThreat = FVector::Dist(NpcLoc, ThreatLoc);
const float CoverDistToThreat = FVector::Dist(Point->GetActorLocation(), ThreatLoc);
if (NpcDistToThreat > 0.0f && CoverDistToThreat < NpcDistToThreat)
{
const float AdvanceRatio = (NpcDistToThreat - CoverDistToThreat) / NpcDistToThreat;
Score += AdvancementBias * AdvanceRatio * 0.3f;
}
}
// Debug: sphere colored by score (cover candidates)
#if ENABLE_DRAW_DEBUG
if (bDebugDraw)
{
const FVector Loc = Point->GetActorLocation();
DrawDebugSphere(const_cast<UWorld*>(World), Loc + FVector(0, 0, 30.0f),
15.0f, 6, ScoreToColor(Score), false, 5.0f);
DrawDebugString(const_cast<UWorld*>(World), Loc + FVector(0, 0, 55.0f),
FString::Printf(TEXT("%.2f"), Score), nullptr, FColor::White, 5.0f, true);
}
#endif
if (Score > OutScore) if (Score > OutScore)
{ {
OutScore = Score; OutScore = Score;
@@ -258,14 +404,180 @@ float UPS_AI_Behavior_BTTask_FindCover::EvaluateCoverQuality(
Score += FMath::GetMappedRangeValueClamped( Score += FMath::GetMappedRangeValueClamped(
FVector2D(0.0f, SearchRadius), FVector2D(0.15f, 0.0f), DistFromNpc); FVector2D(0.0f, SearchRadius), FVector2D(0.15f, 0.0f), DistFromNpc);
// Advancement bias — prefer candidates closer to threat than NPC is
if (AdvancementBias > 0.0f)
{
const float NpcDistToThreat = FVector::Dist(NpcLoc, ThreatLoc);
if (NpcDistToThreat > 0.0f && DistFromThreat < NpcDistToThreat)
{
const float AdvanceRatio = (NpcDistToThreat - DistFromThreat) / NpcDistToThreat;
Score += AdvancementBias * AdvanceRatio * 0.3f;
}
}
return Score; return Score;
} }
FString UPS_AI_Behavior_BTTask_FindCover::GetStaticDescription() const FString UPS_AI_Behavior_BTTask_FindCover::GetStaticDescription() const
{ {
return FString::Printf(TEXT("Find cover within %.0fcm\nManual %s + Procedural (%d candidates)\nBonus: +%.0f%%"), return FString::Printf(TEXT("Find cover within %.0fcm\nManual %s + Procedural (%d candidates)\nBonus: +%.0f%%\nRefinement: %s"),
SearchRadius, SearchRadius,
CoverPointType == EPS_AI_Behavior_CoverPointType::Cover ? TEXT("Cover") : TEXT("Hiding"), CoverPointType == EPS_AI_Behavior_CoverPointType::Cover ? TEXT("Cover") : TEXT("Hiding"),
NumCandidates, NumCandidates,
ManualPointBonus * 100.0f); ManualPointBonus * 100.0f,
RefinementQuery ? *RefinementQuery->GetName() : TEXT("None"));
}
// ─── EQS Refinement ────────────────────────────────────────────────────────
void UPS_AI_Behavior_BTTask_FindCover::RunRefinementQuery(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APawn* Pawn, const FVector& CoverCenter)
{
FCoverMemory* Memory = reinterpret_cast<FCoverMemory*>(NodeMemory);
UWorld* World = Pawn->GetWorld();
UEnvQueryManager* EQSManager = UEnvQueryManager::GetCurrent(World);
if (!EQSManager)
{
// Fallback: move directly to the cover center
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (AIC)
{
AIC->MoveToLocation(CoverCenter, AcceptanceRadius, true, true, true, false);
Memory->bMoveRequested = true;
}
return;
}
Memory->bEQSRunning = true;
FEnvQueryRequest Request(RefinementQuery, Pawn);
Request.Execute(EEnvQueryRunMode::SingleResult,
FQueryFinishedSignature::CreateUObject(this,
&UPS_AI_Behavior_BTTask_FindCover::OnRefinementQueryFinished,
&OwnerComp, NodeMemory, CoverCenter));
if (UPS_AI_Behavior_Statics::IsDebugEnabled(Pawn))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] FindCover: EQS refinement query launched around %s"),
*Pawn->GetName(), *CoverCenter.ToString());
}
}
void UPS_AI_Behavior_BTTask_FindCover::OnRefinementQueryFinished(
TSharedPtr<FEnvQueryResult> Result,
UBehaviorTreeComponent* OwnerComp, uint8* NodeMemory,
FVector OriginalCoverPos)
{
if (!OwnerComp || !NodeMemory) return;
FCoverMemory* Memory = reinterpret_cast<FCoverMemory*>(NodeMemory);
Memory->bEQSRunning = false;
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp->GetAIOwner());
if (!AIC || !AIC->GetPawn()) return;
FVector FinalPos = OriginalCoverPos; // Fallback to original position
if (Result.IsValid() && Result->IsSuccessful())
{
FinalPos = Result->GetItemAsLocation(0);
// Log all items with their scores for debugging
const int32 NumSurvived = Result->Items.Num();
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] FindCover: EQS refinement — %d items survived, refined %s → %s"),
*AIC->GetName(), NumSurvived, *OriginalCoverPos.ToString(), *FinalPos.ToString());
#if ENABLE_DRAW_DEBUG
if (bDebugDraw)
{
UWorld* World = AIC->GetWorld();
if (World)
{
// Draw EQS refinement items as BOXES (cover = spheres, refinement = boxes)
const int32 NumItems = Result->Items.Num();
for (int32 i = 0; i < NumItems; ++i)
{
if (!Result->Items[i].IsValid()) continue;
const FVector ItemLoc = Result->GetItemAsLocation(i);
const float ItemScore = Result->GetItemScore(i);
DrawDebugBox(World, ItemLoc + FVector(0, 0, 20.0f),
FVector(8.0f), ScoreToColor(ItemScore), false, 5.0f);
}
// Original position (yellow box)
DrawDebugBox(World, OriginalCoverPos + FVector(0, 0, 50.0f),
FVector(15.0f), FColor::Yellow, false, 5.0f, 0, 2.0f);
// Refined position (green box, bigger)
DrawDebugBox(World, FinalPos + FVector(0, 0, 50.0f),
FVector(20.0f), FColor::Green, false, 5.0f, 0, 2.5f);
// Arrow from original to refined
DrawDebugDirectionalArrow(World, OriginalCoverPos + FVector(0, 0, 50.0f),
FinalPos + FVector(0, 0, 50.0f), 10.0f, FColor::Green, false, 5.0f, 0, 1.5f);
DrawDebugString(World, FinalPos + FVector(0, 0, 75.0f),
FString::Printf(TEXT("REFINED (%d)"), NumItems),
nullptr, FColor::Green, 5.0f, true);
}
}
#endif
}
else
{
const int32 NumItems = Result.IsValid() ? Result->Items.Num() : 0;
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] FindCover: EQS refinement FAILED — items=%d, valid=%s"),
*AIC->GetName(), NumItems,
Result.IsValid() ? TEXT("yes") : TEXT("no"));
#if ENABLE_DRAW_DEBUG
if (bDebugDraw)
{
UWorld* World = AIC->GetWorld();
if (World)
{
// Show that refinement failed — orange box at original position
DrawDebugBox(World, OriginalCoverPos + FVector(0, 0, 50.0f),
FVector(20.0f), FColor::Orange, false, 5.0f, 0, 2.5f);
DrawDebugString(World, OriginalCoverPos + FVector(0, 0, 75.0f),
TEXT("NO REFINE"), nullptr, FColor::Orange, 5.0f, true);
}
}
#endif
}
// Update BB with refined position
UBlackboardComponent* BB = OwnerComp->GetBlackboardComponent();
if (BB)
{
BB->SetValueAsVector(PS_AI_Behavior_BB::CoverLocation, FinalPos);
}
// Navigate to the refined (or original) position
const EPathFollowingRequestResult::Type MoveResult = AIC->MoveToLocation(
FinalPos, AcceptanceRadius, true, true, true, false);
if (MoveResult == EPathFollowingRequestResult::Failed)
{
FinishLatentTask(*OwnerComp, EBTNodeResult::Failed);
return;
}
if (MoveResult == EPathFollowingRequestResult::AlreadyAtGoal)
{
// Crouch at cover if needed
APawn* Pawn = AIC->GetPawn();
if (Pawn && Pawn->Implements<UPS_AI_Behavior_Interface>())
{
const APS_AI_Behavior_CoverPoint* CoverPt =
Cast<APS_AI_Behavior_CoverPoint>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (CoverPt && CoverPt->bCrouch)
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(Pawn, true);
}
}
FinishLatentTask(*OwnerComp, EBTNodeResult::Succeeded);
return;
}
Memory->bMoveRequested = true;
} }

View File

@@ -2,6 +2,7 @@
#include "BT/PS_AI_Behavior_BTTask_FleeFrom.h" #include "BT/PS_AI_Behavior_BTTask_FleeFrom.h"
#include "PS_AI_Behavior_AIController.h" #include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "BehaviorTree/BlackboardComponent.h" #include "BehaviorTree/BlackboardComponent.h"
#include "NavigationSystem.h" #include "NavigationSystem.h"
@@ -20,6 +21,13 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FleeFrom::ExecuteTask(
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner()); APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC || !AIC->GetPawn()) return EBTNodeResult::Failed; if (!AIC || !AIC->GetPawn()) return EBTNodeResult::Failed;
// Stand up if crouching (e.g. was hiding at a cover point before fleeing)
APawn* FleePawn = AIC->GetPawn();
if (FleePawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(FleePawn, false);
}
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) return EBTNodeResult::Failed; if (!BB) return EBTNodeResult::Failed;

View File

@@ -5,6 +5,7 @@
#include "PS_AI_Behavior_SplineFollowerComponent.h" #include "PS_AI_Behavior_SplineFollowerComponent.h"
#include "PS_AI_Behavior_SplinePath.h" #include "PS_AI_Behavior_SplinePath.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "BehaviorTree/BlackboardComponent.h"
UPS_AI_Behavior_BTTask_FollowSpline::UPS_AI_Behavior_BTTask_FollowSpline() UPS_AI_Behavior_BTTask_FollowSpline::UPS_AI_Behavior_BTTask_FollowSpline()
{ {
@@ -23,6 +24,12 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FollowSpline::ExecuteTask(
AIC->GetPawn()->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>(); AIC->GetPawn()->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>();
if (!Follower) return EBTNodeResult::Failed; if (!Follower) return EBTNodeResult::Failed;
// Respect a manual StopFollowing(): refuse to resume until an explicit restart.
if (Follower->bManuallyStopped)
{
return EBTNodeResult::Failed;
}
if (!Follower->CurrentSpline) if (!Follower->CurrentSpline)
{ {
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] FollowSpline: no current spline set."), UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] FollowSpline: no current spline set."),
@@ -61,6 +68,19 @@ void UPS_AI_Behavior_BTTask_FollowSpline::TickTask(
{ {
UPS_AI_Behavior_SplineFollowerComponent* FollowerCheck = UPS_AI_Behavior_SplineFollowerComponent* FollowerCheck =
AICCheck->GetPawn()->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>(); AICCheck->GetPawn()->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>();
// Sync spline info to Blackboard
if (FollowerCheck)
{
if (UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent())
{
BB->SetValueAsObject(PS_AI_Behavior_BB::CurrentSpline,
FollowerCheck->CurrentSpline);
BB->SetValueAsFloat(PS_AI_Behavior_BB::SplineProgress,
FollowerCheck->GetProgress());
}
}
if (FollowerCheck && !FollowerCheck->bIsFollowing) if (FollowerCheck && !FollowerCheck->bIsFollowing)
{ {
FinishLatentTask(OwnerComp, EBTNodeResult::Succeeded); FinishLatentTask(OwnerComp, EBTNodeResult::Succeeded);
@@ -112,6 +132,14 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_FollowSpline::AbortTask(
Follower->PauseFollowing(); Follower->PauseFollowing();
} }
} }
// Clear spline info from BB
if (UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent())
{
BB->ClearValue(PS_AI_Behavior_BB::CurrentSpline);
BB->SetValueAsFloat(PS_AI_Behavior_BB::SplineProgress, 0.0f);
}
return EBTNodeResult::Aborted; return EBTNodeResult::Aborted;
} }

View File

@@ -4,6 +4,7 @@
#include "PS_AI_Behavior_AIController.h" #include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "BehaviorTree/BlackboardComponent.h" #include "BehaviorTree/BlackboardComponent.h"
#include "NavigationSystem.h"
#include "Navigation/PathFollowingComponent.h" #include "Navigation/PathFollowingComponent.h"
UPS_AI_Behavior_BTTask_Patrol::UPS_AI_Behavior_BTTask_Patrol() UPS_AI_Behavior_BTTask_Patrol::UPS_AI_Behavior_BTTask_Patrol()
@@ -17,27 +18,39 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_Patrol::ExecuteTask(
UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory)
{ {
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner()); APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC) return EBTNodeResult::Failed; if (!AIC || !AIC->GetPawn()) return EBTNodeResult::Failed;
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (!BB) return EBTNodeResult::Failed; if (!BB) return EBTNodeResult::Failed;
// Check we have patrol points
if (AIC->PatrolPoints.Num() == 0)
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Patrol task: no patrol points defined."), *AIC->GetName());
return EBTNodeResult::Failed;
}
FPatrolMemory* Memory = reinterpret_cast<FPatrolMemory*>(NodeMemory); FPatrolMemory* Memory = reinterpret_cast<FPatrolMemory*>(NodeMemory);
Memory->bIsWaiting = false; Memory->bIsWaiting = false;
Memory->bMoveRequested = false; Memory->bMoveRequested = false;
Memory->WaitRemaining = 0.0f; Memory->WaitRemaining = 0.0f;
// Get current patrol index FVector Destination;
if (AIC->PatrolPoints.Num() > 0)
{
// ─── Mode manuel : waypoints définis ────────────────────────────
const int32 PatrolIdx = BB->GetValueAsInt(PS_AI_Behavior_BB::PatrolIndex); const int32 PatrolIdx = BB->GetValueAsInt(PS_AI_Behavior_BB::PatrolIndex);
const int32 SafeIdx = PatrolIdx % AIC->PatrolPoints.Num(); const int32 SafeIdx = PatrolIdx % AIC->PatrolPoints.Num();
const FVector Destination = AIC->PatrolPoints[SafeIdx]; Destination = AIC->PatrolPoints[SafeIdx];
}
else
{
// ─── Mode auto : point aléatoire sur NavMesh ────────────────────
const FVector HomeLoc = BB->GetValueAsVector(PS_AI_Behavior_BB::HomeLocation);
const FVector CurrentLoc = AIC->GetPawn()->GetActorLocation();
if (!FindRandomPatrolPoint(GetWorld(), HomeLoc, CurrentLoc, Destination))
{
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] Patrol: no valid NavMesh point found within %.0fcm."),
*AIC->GetName(), PatrolRadius);
return EBTNodeResult::Failed;
}
}
// Issue move request // Issue move request
const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation( const EPathFollowingRequestResult::Type Result = AIC->MoveToLocation(
@@ -47,19 +60,22 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_Patrol::ExecuteTask(
if (Result == EPathFollowingRequestResult::Failed) if (Result == EPathFollowingRequestResult::Failed)
{ {
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] Patrol: MoveTo failed for point %d."), UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] Patrol: MoveTo failed."), *AIC->GetName());
*AIC->GetName(), SafeIdx);
return EBTNodeResult::Failed; return EBTNodeResult::Failed;
} }
if (Result == EPathFollowingRequestResult::AlreadyAtGoal) if (Result == EPathFollowingRequestResult::AlreadyAtGoal)
{ {
// Already there — start wait
Memory->bIsWaiting = true; Memory->bIsWaiting = true;
Memory->WaitRemaining = FMath::RandRange(MinWaitTime, MaxWaitTime); Memory->WaitRemaining = FMath::RandRange(MinWaitTime, MaxWaitTime);
// Advance patrol index // Advance patrol index (for manual mode)
BB->SetValueAsInt(PS_AI_Behavior_BB::PatrolIndex, (SafeIdx + 1) % AIC->PatrolPoints.Num()); if (AIC->PatrolPoints.Num() > 0)
{
const int32 PatrolIdx = BB->GetValueAsInt(PS_AI_Behavior_BB::PatrolIndex);
BB->SetValueAsInt(PS_AI_Behavior_BB::PatrolIndex,
(PatrolIdx + 1) % AIC->PatrolPoints.Num());
}
return EBTNodeResult::InProgress; return EBTNodeResult::InProgress;
} }
@@ -88,24 +104,22 @@ void UPS_AI_Behavior_BTTask_Patrol::TickTask(
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner()); APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(OwnerComp.GetAIOwner());
if (!AIC) { FinishLatentTask(OwnerComp, EBTNodeResult::Failed); return; } if (!AIC) { FinishLatentTask(OwnerComp, EBTNodeResult::Failed); return; }
const EPathFollowingStatus::Type MoveStatus = AIC->GetMoveStatus(); if (AIC->GetMoveStatus() == EPathFollowingStatus::Idle)
if (MoveStatus == EPathFollowingStatus::Idle)
{ {
// Move completed — start wait at waypoint // Move completed — start wait
Memory->bMoveRequested = false; Memory->bMoveRequested = false;
Memory->bIsWaiting = true; Memory->bIsWaiting = true;
Memory->WaitRemaining = FMath::RandRange(MinWaitTime, MaxWaitTime); Memory->WaitRemaining = FMath::RandRange(MinWaitTime, MaxWaitTime);
// Advance patrol index // Advance patrol index (for manual mode)
if (AIC->PatrolPoints.Num() > 0)
{
UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent(); UBlackboardComponent* BB = OwnerComp.GetBlackboardComponent();
if (BB) if (BB)
{ {
const int32 PatrolIdx = BB->GetValueAsInt(PS_AI_Behavior_BB::PatrolIndex); const int32 PatrolIdx = BB->GetValueAsInt(PS_AI_Behavior_BB::PatrolIndex);
const int32 NumPoints = AIC->PatrolPoints.Num(); BB->SetValueAsInt(PS_AI_Behavior_BB::PatrolIndex,
if (NumPoints > 0) (PatrolIdx + 1) % AIC->PatrolPoints.Num());
{
BB->SetValueAsInt(PS_AI_Behavior_BB::PatrolIndex, (PatrolIdx + 1) % NumPoints);
} }
} }
} }
@@ -123,8 +137,58 @@ EBTNodeResult::Type UPS_AI_Behavior_BTTask_Patrol::AbortTask(
return EBTNodeResult::Aborted; return EBTNodeResult::Aborted;
} }
bool UPS_AI_Behavior_BTTask_Patrol::FindRandomPatrolPoint(
const UWorld* World, const FVector& HomeLoc,
const FVector& CurrentLoc, FVector& OutPoint) const
{
UNavigationSystemV1* NavSys = FNavigationSystem::GetCurrent<UNavigationSystemV1>(
const_cast<UWorld*>(World));
if (!NavSys) return false;
// Project HomeLoc onto NavMesh first (spawn position might not be exactly on it)
FNavLocation ProjectedHome;
const FVector ProjectionExtent(500.0f, 500.0f, 500.0f);
FVector SearchOrigin = HomeLoc;
if (NavSys->ProjectPointToNavigation(HomeLoc, ProjectedHome, ProjectionExtent))
{
SearchOrigin = ProjectedHome.Location;
}
else
{
// HomeLoc not on NavMesh — try from current position instead
FNavLocation ProjectedCurrent;
if (NavSys->ProjectPointToNavigation(CurrentLoc, ProjectedCurrent, ProjectionExtent))
{
SearchOrigin = ProjectedCurrent.Location;
}
else
{
return false; // Neither home nor current pos are on NavMesh
}
}
// Try multiple times to find a valid point far enough from current position
for (int32 Attempt = 0; Attempt < 10; ++Attempt)
{
FNavLocation NavLoc;
if (NavSys->GetRandomReachablePointInRadius(SearchOrigin, PatrolRadius, NavLoc))
{
const float DistFromCurrent = FVector::Dist2D(NavLoc.Location, CurrentLoc);
if (DistFromCurrent >= MinPatrolDistance)
{
OutPoint = NavLoc.Location;
return true;
}
}
}
return false;
}
FString UPS_AI_Behavior_BTTask_Patrol::GetStaticDescription() const FString UPS_AI_Behavior_BTTask_Patrol::GetStaticDescription() const
{ {
return FString::Printf(TEXT("Patrol (wait %.1f-%.1fs, radius %.0fcm)"), return FString::Printf(
MinWaitTime, MaxWaitTime, AcceptanceRadius); TEXT("Patrol (wait %.1f-%.1fs)\nManual waypoints OR auto NavMesh (radius %.0fcm, min dist %.0fcm)"),
MinWaitTime, MaxWaitTime, PatrolRadius, MinPatrolDistance);
} }

View File

@@ -0,0 +1,61 @@
// Copyright Asterion. All Rights Reserved.
#include "EQS/PS_AI_Behavior_EQSContext_CoverLocation.h"
#include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_Definitions.h"
#include "EnvironmentQuery/EnvQueryTypes.h"
#include "EnvironmentQuery/Items/EnvQueryItemType_Point.h"
#include "BehaviorTree/BlackboardComponent.h"
#include "NavigationSystem.h"
void UPS_AI_Behavior_EQSContext_CoverLocation::ProvideContext(
FEnvQueryInstance& QueryInstance, FEnvQueryContextData& ContextData) const
{
const AActor* QuerierActor = Cast<AActor>(QueryInstance.Owner.Get());
if (!QuerierActor) return;
const APawn* QuerierPawn = Cast<APawn>(QuerierActor);
if (!QuerierPawn) return;
APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(QuerierPawn->GetController());
if (!AIC) return;
UBlackboardComponent* BB = AIC->GetBlackboardComponent();
if (!BB) return;
// Use CoverPoint actor location (center of cover geometry) for circle generation.
// Fallback to CoverLocation vector if no actor.
FVector RawLoc = FVector::ZeroVector;
const AActor* CoverPointActor = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::CoverPoint));
if (CoverPointActor)
{
RawLoc = CoverPointActor->GetActorLocation();
}
else
{
RawLoc = BB->GetValueAsVector(PS_AI_Behavior_BB::CoverLocation);
}
if (RawLoc.IsZero()) return;
// Project to navmesh so the EQS generator works correctly
// (cover points inside geometry or above navmesh need projection)
FVector FinalLoc = RawLoc;
UNavigationSystemV1* NavSys = FNavigationSystem::GetCurrent<UNavigationSystemV1>(QuerierPawn->GetWorld());
if (NavSys)
{
FNavLocation NavLoc;
if (NavSys->ProjectPointToNavigation(RawLoc, NavLoc, FVector(200.0f, 200.0f, 200.0f)))
{
FinalLoc = NavLoc.Location;
}
}
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[EQSContext_CoverLocation] %s '%s' raw=%s → nav=%s"),
CoverPointActor ? TEXT("CoverPoint") : TEXT("CoverLocation"),
CoverPointActor ? *CoverPointActor->GetName() : TEXT("(vector)"),
*RawLoc.ToString(), *FinalLoc.ToString());
UEnvQueryItemType_Point::SetContextHelper(ContextData, FinalLoc);
}

View File

@@ -2,10 +2,12 @@
#include "EQS/PS_AI_Behavior_EQSTest_CoverQuality.h" #include "EQS/PS_AI_Behavior_EQSTest_CoverQuality.h"
#include "EQS/PS_AI_Behavior_EQSContext_Threat.h" #include "EQS/PS_AI_Behavior_EQSContext_Threat.h"
#include "PS_AI_Behavior_Definitions.h"
#include "EnvironmentQuery/EnvQueryTypes.h" #include "EnvironmentQuery/EnvQueryTypes.h"
#include "EnvironmentQuery/Items/EnvQueryItemType_VectorBase.h" #include "EnvironmentQuery/Items/EnvQueryItemType_VectorBase.h"
#include "CollisionQueryParams.h" #include "CollisionQueryParams.h"
#include "Engine/World.h" #include "Engine/World.h"
#include "DrawDebugHelpers.h"
UPS_AI_Behavior_EQSTest_CoverQuality::UPS_AI_Behavior_EQSTest_CoverQuality() UPS_AI_Behavior_EQSTest_CoverQuality::UPS_AI_Behavior_EQSTest_CoverQuality()
{ {
@@ -37,6 +39,20 @@ void UPS_AI_Behavior_EQSTest_CoverQuality::RunTest(FEnvQueryInstance& QueryInsta
FCollisionQueryParams TraceParams(SCENE_QUERY_STAT(CoverQualityEQS), true); FCollisionQueryParams TraceParams(SCENE_QUERY_STAT(CoverQualityEQS), true);
// Ignore the threat actor itself (e.g. AimTargetActor sphere blocks its own traces)
TArray<AActor*> ThreatActors;
if (QueryInstance.PrepareContext(UPS_AI_Behavior_EQSContext_Threat::StaticClass(), ThreatActors))
{
for (AActor* A : ThreatActors)
{
TraceParams.AddIgnoredActor(A);
for (AActor* Parent = A->GetAttachParentActor(); Parent; Parent = Parent->GetAttachParentActor())
{
TraceParams.AddIgnoredActor(Parent);
}
}
}
// Compute height steps // Compute height steps
TArray<float> TraceHeights; TArray<float> TraceHeights;
if (NumTraceHeights == 1) if (NumTraceHeights == 1)
@@ -52,27 +68,82 @@ void UPS_AI_Behavior_EQSTest_CoverQuality::RunTest(FEnvQueryInstance& QueryInsta
} }
} }
// Compute lateral offsets: center + left + right perpendicular to threat direction
// This helps evaluate cover around narrow objects (poles, pillars)
TArray<float> LateralOffsets;
LateralOffsets.Add(0.0f); // Center trace
if (LateralSpread > 0.0f)
{
LateralOffsets.Add(-LateralSpread);
LateralOffsets.Add(LateralSpread);
}
const int32 TotalTraces = TraceHeights.Num() * LateralOffsets.Num();
for (FEnvQueryInstance::ItemIterator It(this, QueryInstance); It; ++It) for (FEnvQueryInstance::ItemIterator It(this, QueryInstance); It; ++It)
{ {
const FVector CandidatePos = GetItemLocation(QueryInstance, It.GetIndex()); const FVector CandidatePos = GetItemLocation(QueryInstance, It.GetIndex());
float BlockedCount = 0.0f; float BlockedCount = 0.0f;
// Direction from candidate to threat (2D), and its perpendicular
const FVector DirToThreat = (ThreatLoc - CandidatePos).GetSafeNormal2D();
const FVector LateralDir = FVector::CrossProduct(FVector::UpVector, DirToThreat);
for (float Height : TraceHeights) for (float Height : TraceHeights)
{ {
const FVector TraceStart = CandidatePos + FVector(0, 0, Height); for (float Lateral : LateralOffsets)
const FVector TraceEnd = ThreatLoc + FVector(0, 0, 150.0f); // Approx eye height {
const FVector TraceStart = CandidatePos + FVector(0, 0, Height) + LateralDir * Lateral;
const FVector TraceEnd = ThreatLoc + FVector(0, 0, 60.0f) + LateralDir * Lateral;
FHitResult Hit; FHitResult Hit;
if (World->LineTraceSingleByChannel(Hit, TraceStart, TraceEnd, if (World->LineTraceSingleByChannel(Hit, TraceStart, TraceEnd,
ECC_Visibility, TraceParams)) ECC_Visibility, TraceParams))
{
const float HitDist = FVector::Dist(CandidatePos, Hit.ImpactPoint);
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("CoverQuality[%d] h=%.0f lat=%.0f: HIT '%s' dist=%.0fcm"),
It.GetIndex(), Height, Lateral,
Hit.GetActor() ? *Hit.GetActor()->GetName() : TEXT("null"),
HitDist);
const bool bGroundHit = Hit.ImpactPoint.Z < TraceStart.Z - 10.0f;
if (!bGroundHit && HitDist <= MaxNearbyHitDist)
{ {
BlockedCount += 1.0f; BlockedCount += 1.0f;
} }
} }
else
{
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("CoverQuality[%d] h=%.0f lat=%.0f: NO HIT"),
It.GetIndex(), Height, Lateral);
}
}
}
// Score: ratio of blocked traces (0.0 = fully exposed, 1.0 = fully covered) const float Score = BlockedCount / static_cast<float>(TotalTraces);
const float Score = BlockedCount / static_cast<float>(TraceHeights.Num());
It.SetScore(TestPurpose, FilterType, Score, 0.0f, 1.0f); UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("CoverQuality[%d] at %s → score=%.2f (blocked=%d/%d)"),
It.GetIndex(), *CandidatePos.ToString(),
Score, (int32)BlockedCount, TotalTraces);
#if ENABLE_DRAW_DEBUG
if (bDrawDebug)
{
// Color: red(0) → yellow(0.5) → green(1)
const uint8 R = static_cast<uint8>(FMath::Lerp(255.0f, 0.0f, FMath::Clamp(Score, 0.0f, 1.0f)));
const uint8 G = static_cast<uint8>(FMath::Lerp(0.0f, 255.0f, FMath::Clamp(Score, 0.0f, 1.0f)));
DrawDebugBox(const_cast<UWorld*>(World), CandidatePos + FVector(0, 0, 20.0f),
FVector(8.0f), FColor(R, G, 0), false, 5.0f);
DrawDebugString(const_cast<UWorld*>(World), CandidatePos + FVector(0, 0, 35.0f),
FString::Printf(TEXT("%.0f%%"), Score * 100.0f), nullptr,
FColor(R, G, 0), 5.0f, true);
}
#endif
It.SetScore(TestPurpose, FilterType, Score, FloatValueMin.GetValue(), FloatValueMax.GetValue());
} }
} }
@@ -84,6 +155,8 @@ FText UPS_AI_Behavior_EQSTest_CoverQuality::GetDescriptionTitle() const
FText UPS_AI_Behavior_EQSTest_CoverQuality::GetDescriptionDetails() const FText UPS_AI_Behavior_EQSTest_CoverQuality::GetDescriptionDetails() const
{ {
return FText::FromString(FString::Printf( return FText::FromString(FString::Printf(
TEXT("%d traces from %.0f to %.0fcm height"), TEXT("%d heights (%.0f%.0fcm) × %s, maxHitDist=%.0f"),
NumTraceHeights, MinTraceHeight, MaxTraceHeight)); NumTraceHeights, MinTraceHeight, MaxTraceHeight,
LateralSpread > 0.0f ? *FString::Printf(TEXT("3 lateral (±%.0fcm)"), LateralSpread) : TEXT("center only"),
MaxNearbyHitDist));
} }

View File

@@ -0,0 +1,139 @@
// Copyright Asterion. All Rights Reserved.
#include "EQS/PS_AI_Behavior_EQSTest_LineOfSight.h"
#include "EQS/PS_AI_Behavior_EQSContext_Threat.h"
#include "PS_AI_Behavior_Definitions.h"
#include "EnvironmentQuery/EnvQueryTypes.h"
#include "EnvironmentQuery/Items/EnvQueryItemType_VectorBase.h"
#include "CollisionQueryParams.h"
#include "Engine/World.h"
#include "DrawDebugHelpers.h"
UPS_AI_Behavior_EQSTest_LineOfSight::UPS_AI_Behavior_EQSTest_LineOfSight()
{
Cost = EEnvTestCost::High; // Uses raycasts
ValidItemType = UEnvQueryItemType_VectorBase::StaticClass();
SetWorkOnFloatValues(true);
}
void UPS_AI_Behavior_EQSTest_LineOfSight::RunTest(FEnvQueryInstance& QueryInstance) const
{
UObject* QueryOwner = QueryInstance.Owner.Get();
if (!QueryOwner) return;
// Get threat locations from context
TArray<FVector> ThreatLocations;
if (!QueryInstance.PrepareContext(UPS_AI_Behavior_EQSContext_Threat::StaticClass(), ThreatLocations))
{
return;
}
if (ThreatLocations.Num() == 0)
{
return;
}
const FVector ThreatLoc = ThreatLocations[0];
const UWorld* World = GEngine->GetWorldFromContextObject(QueryOwner, EGetWorldErrorMode::LogAndReturnNull);
if (!World) return;
FCollisionQueryParams TraceParams(SCENE_QUERY_STAT(LOSTestEQS), true);
// Ignore the threat actor itself (e.g. AimTargetActor sphere blocks its own traces)
TArray<AActor*> ThreatActors;
if (QueryInstance.PrepareContext(UPS_AI_Behavior_EQSContext_Threat::StaticClass(), ThreatActors))
{
for (AActor* A : ThreatActors)
{
TraceParams.AddIgnoredActor(A);
// Also ignore parent chain (AimTarget → Character capsule)
for (AActor* Parent = A->GetAttachParentActor(); Parent; Parent = Parent->GetAttachParentActor())
{
TraceParams.AddIgnoredActor(Parent);
}
}
}
// Lateral offsets: center + optional left/right perpendicular to threat direction
TArray<float> LateralOffsets;
LateralOffsets.Add(0.0f);
if (LateralSpread > 0.0f)
{
LateralOffsets.Add(-LateralSpread);
LateralOffsets.Add(LateralSpread);
}
const int32 TotalTraces = LateralOffsets.Num();
for (FEnvQueryInstance::ItemIterator It(this, QueryInstance); It; ++It)
{
const FVector CandidatePos = GetItemLocation(QueryInstance, It.GetIndex());
// Direction from candidate to threat (2D) and its perpendicular
const FVector DirToThreat = (ThreatLoc - CandidatePos).GetSafeNormal2D();
const FVector LateralDir = FVector::CrossProduct(FVector::UpVector, DirToThreat);
int32 ClearCount = 0;
for (float Lateral : LateralOffsets)
{
const FVector TraceStart = CandidatePos + FVector(0, 0, TraceHeight) + LateralDir * Lateral;
const FVector TraceEnd = ThreatLoc + FVector(0, 0, TargetHeightOffset) + LateralDir * Lateral;
FHitResult Hit;
const bool bBlocked = World->LineTraceSingleByChannel(
Hit, TraceStart, TraceEnd, ECC_Visibility, TraceParams);
if (!bBlocked)
{
ClearCount++;
}
else
{
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("LOS[%d] lat=%.0f: BLOCKED by '%s' at %s (dist=%.0fcm)"),
It.GetIndex(), Lateral,
Hit.GetActor() ? *Hit.GetActor()->GetName() : TEXT("null"),
*Hit.ImpactPoint.ToString(),
FVector::Dist(CandidatePos, Hit.ImpactPoint));
}
#if ENABLE_DRAW_DEBUG
if (bDrawDebug)
{
const FColor Color = bBlocked ? FColor::Red : FColor::Green;
DrawDebugLine(const_cast<UWorld*>(World), TraceStart, TraceEnd,
Color, false, 5.0f, 0, 0.5f);
}
#endif
}
// Score: ratio of clear traces (1.0 = all clear, 0.0 = all blocked)
const float Score = static_cast<float>(ClearCount) / static_cast<float>(TotalTraces);
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("LOS[%d] at %s → clear=%d/%d score=%.2f"),
It.GetIndex(), *CandidatePos.ToString(),
ClearCount, TotalTraces, Score);
#if ENABLE_DRAW_DEBUG
if (bDrawDebug)
{
const uint8 G = static_cast<uint8>(Score * 255.0f);
DrawDebugSphere(const_cast<UWorld*>(World), CandidatePos + FVector(0, 0, 20.0f),
10.0f, 6, FColor(255 - G, G, 0), false, 5.0f);
}
#endif
It.SetScore(TestPurpose, FilterType, Score, FloatValueMin.GetValue(), FloatValueMax.GetValue());
}
}
FText UPS_AI_Behavior_EQSTest_LineOfSight::GetDescriptionTitle() const
{
return FText::FromString(TEXT("Line of Sight (to Threat)"));
}
FText UPS_AI_Behavior_EQSTest_LineOfSight::GetDescriptionDetails() const
{
return FText::FromString(FString::Printf(
TEXT("Trace from %.0fcm to threat +%.0fcm"), TraceHeight, TargetHeightOffset));
}

View File

@@ -4,8 +4,11 @@
#include "PS_AI_Behavior_Interface.h" #include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_PerceptionComponent.h" #include "PS_AI_Behavior_PerceptionComponent.h"
#include "PS_AI_Behavior_PersonalityComponent.h" #include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_SplineFollowerComponent.h"
#include "PS_AI_Behavior_TeamComponent.h"
#include "PS_AI_Behavior_PersonalityProfile.h" #include "PS_AI_Behavior_PersonalityProfile.h"
#include "BehaviorTree/BehaviorTree.h" #include "BehaviorTree/BehaviorTree.h"
#include "BehaviorTree/BehaviorTreeComponent.h"
#include "BehaviorTree/BlackboardComponent.h" #include "BehaviorTree/BlackboardComponent.h"
#include "BehaviorTree/BlackboardData.h" #include "BehaviorTree/BlackboardData.h"
#include "BehaviorTree/Blackboard/BlackboardKeyType_Enum.h" #include "BehaviorTree/Blackboard/BlackboardKeyType_Enum.h"
@@ -13,6 +16,10 @@
#include "BehaviorTree/Blackboard/BlackboardKeyType_Int.h" #include "BehaviorTree/Blackboard/BlackboardKeyType_Int.h"
#include "BehaviorTree/Blackboard/BlackboardKeyType_Object.h" #include "BehaviorTree/Blackboard/BlackboardKeyType_Object.h"
#include "BehaviorTree/Blackboard/BlackboardKeyType_Vector.h" #include "BehaviorTree/Blackboard/BlackboardKeyType_Vector.h"
#include "BehaviorTree/Blackboard/BlackboardKeyType_String.h"
#include "BehaviorTree/Blackboard/BlackboardKeyType_Bool.h"
#include "DrawDebugHelpers.h"
#include "GameFramework/SpectatorPawn.h"
APS_AI_Behavior_AIController::APS_AI_Behavior_AIController() APS_AI_Behavior_AIController::APS_AI_Behavior_AIController()
{ {
@@ -39,68 +46,63 @@ void APS_AI_Behavior_AIController::OnPossess(APawn* InPawn)
*GetName(), *InPawn->GetName()); *GetName(), *InPawn->GetName());
} }
// Auto-assign Team ID from IPS_AI_Behavior interface (preferred) or PersonalityComponent // Assign Team ID from NPCType + Faction
if (TeamId == FGenericTeamId::NoTeam)
{ {
EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Any; EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Any;
if (InPawn->Implements<UPS_AI_Behavior_Interface>()) if (InPawn->Implements<UPS_AI_Behavior_Interface>())
{ {
// Use the interface — the host project controls the storage
NPCType = IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(InPawn); NPCType = IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(InPawn);
// Also check if the interface provides a specific TeamId
const uint8 InterfaceTeamId = IPS_AI_Behavior_Interface::Execute_GetBehaviorTeamId(InPawn);
if (InterfaceTeamId != FGenericTeamId::NoTeam)
{
TeamId = InterfaceTeamId;
}
} }
else if (PersonalityComp) else if (PersonalityComp)
{ {
// Fallback: get from PersonalityProfile
NPCType = PersonalityComp->GetNPCType(); NPCType = PersonalityComp->GetNPCType();
} }
// If interface didn't set a specific TeamId, derive from NPCType // Get faction from profile (0 = default)
if (TeamId == FGenericTeamId::NoTeam) const uint8 Faction = (PersonalityComp && PersonalityComp->Profile)
{ ? PersonalityComp->Profile->Faction : 0;
switch (NPCType)
{ // Infiltrated enemy: disguised as Civilian until hostile
case EPS_AI_Behavior_NPCType::Civilian: if (NPCType == EPS_AI_Behavior_NPCType::Enemy &&
TeamId = 1; InPawn->Implements<UPS_AI_Behavior_Interface>() &&
break;
case EPS_AI_Behavior_NPCType::Enemy:
// Check if infiltrated (hostile=false → disguised as civilian)
if (InPawn->Implements<UPS_AI_Behavior_Interface>() &&
!IPS_AI_Behavior_Interface::Execute_IsBehaviorHostile(InPawn)) !IPS_AI_Behavior_Interface::Execute_IsBehaviorHostile(InPawn))
{ {
TeamId = 1; // Disguised as Civilian TeamId = PS_AI_Behavior_Team::DisguisedTeamId;
} }
else else
{ {
TeamId = 2; TeamId = PS_AI_Behavior_Team::MakeTeamId(NPCType, Faction);
}
break;
case EPS_AI_Behavior_NPCType::Protector:
TeamId = 3;
break;
default:
TeamId = FGenericTeamId::NoTeam; // 255 → Neutral to everyone
break;
}
} }
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Auto-assigned TeamId=%d from NPCType=%s"), UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] TeamId=0x%02X (%s, faction %d)"),
*GetName(), TeamId, *UEnum::GetValueAsString(NPCType)); *GetName(), TeamId, *UEnum::GetValueAsString(NPCType), Faction);
} }
SetupBlackboard(); SetupBlackboard();
StartBehavior();
TryBindConversationAgent();
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Possessed Pawn '%s' — BT started, TeamId=%d."), // Read auto-start from PersonalityComponent (defaults to true if no component)
*GetName(), *InPawn->GetName(), TeamId); const bool bAutoStart = PersonalityComp
? PersonalityComp->bAutoStartBehavior
: true;
if (bAutoStart)
{
StartBehavior();
}
else
{
// Stay in Scripted state — no BT, no services, controlled externally
SetBehaviorState(EPS_AI_Behavior_State::Scripted);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] bAutoStartBehavior=false (from Profile) — entering Scripted state."),
*GetName());
}
TryBindConversationAgent();
TryBindGazeComponent();
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Possessed Pawn '%s' — TeamId=%d, AutoStart=%d."),
*GetName(), *InPawn->GetName(), TeamId, (int32)bAutoStart);
} }
void APS_AI_Behavior_AIController::OnUnPossess() void APS_AI_Behavior_AIController::OnUnPossess()
@@ -112,6 +114,14 @@ void APS_AI_Behavior_AIController::OnUnPossess()
Brain->StopLogic(TEXT("Unpossessed")); Brain->StopLogic(TEXT("Unpossessed"));
} }
ClearGazeTarget();
CachedGazeComponent = nullptr;
CachedConvAgentComponent = nullptr;
CachedMicComponent = nullptr;
bConversationPaused = false;
bUserHasSpokenInConversation = false;
ProximityGazeTarget = nullptr;
PersonalityComp = nullptr; PersonalityComp = nullptr;
Super::OnUnPossess(); Super::OnUnPossess();
} }
@@ -185,6 +195,36 @@ void APS_AI_Behavior_AIController::SetupBlackboard()
SplineProgressEntry.EntryName = PS_AI_Behavior_BB::SplineProgress; SplineProgressEntry.EntryName = PS_AI_Behavior_BB::SplineProgress;
SplineProgressEntry.KeyType = NewObject<UBlackboardKeyType_Float>(BlackboardAsset); SplineProgressEntry.KeyType = NewObject<UBlackboardKeyType_Float>(BlackboardAsset);
BlackboardAsset->Keys.Add(SplineProgressEntry); BlackboardAsset->Keys.Add(SplineProgressEntry);
// CombatSubState (stored as uint8 enum)
FBlackboardEntry CombatSubStateEntry;
CombatSubStateEntry.EntryName = PS_AI_Behavior_BB::CombatSubState;
CombatSubStateEntry.KeyType = NewObject<UBlackboardKeyType_Enum>(BlackboardAsset);
BlackboardAsset->Keys.Add(CombatSubStateEntry);
// LastKnownTargetPosition (vector — for LOS investigation)
FBlackboardEntry LastKnownEntry;
LastKnownEntry.EntryName = PS_AI_Behavior_BB::LastKnownTargetPosition;
LastKnownEntry.KeyType = NewObject<UBlackboardKeyType_Vector>(BlackboardAsset);
BlackboardAsset->Keys.Add(LastKnownEntry);
// ThreatPawnName (debug: name of the Pawn behind ThreatActor)
FBlackboardEntry ThreatPawnNameEntry;
ThreatPawnNameEntry.EntryName = PS_AI_Behavior_BB::ThreatPawnName;
ThreatPawnNameEntry.KeyType = NewObject<UBlackboardKeyType_String>(BlackboardAsset);
BlackboardAsset->Keys.Add(ThreatPawnNameEntry);
// PreferCover (bool: personality-driven combat/cover cycle)
FBlackboardEntry PreferCoverEntry;
PreferCoverEntry.EntryName = PS_AI_Behavior_BB::PreferCover;
PreferCoverEntry.KeyType = NewObject<UBlackboardKeyType_Bool>(BlackboardAsset);
BlackboardAsset->Keys.Add(PreferCoverEntry);
// ConversationPaused (bool: NPC paused during active conversation)
FBlackboardEntry ConvPausedEntry;
ConvPausedEntry.EntryName = PS_AI_Behavior_BB::ConversationPaused;
ConvPausedEntry.KeyType = NewObject<UBlackboardKeyType_Bool>(BlackboardAsset);
BlackboardAsset->Keys.Add(ConvPausedEntry);
} }
UBlackboardComponent* RawBBComp = nullptr; UBlackboardComponent* RawBBComp = nullptr;
@@ -198,36 +238,282 @@ void APS_AI_Behavior_AIController::SetupBlackboard()
Blackboard->SetValueAsInt(PS_AI_Behavior_BB::PatrolIndex, 0); Blackboard->SetValueAsInt(PS_AI_Behavior_BB::PatrolIndex, 0);
Blackboard->SetValueAsEnum(PS_AI_Behavior_BB::State, Blackboard->SetValueAsEnum(PS_AI_Behavior_BB::State,
static_cast<uint8>(EPS_AI_Behavior_State::Idle)); static_cast<uint8>(EPS_AI_Behavior_State::Idle));
Blackboard->SetValueAsBool(PS_AI_Behavior_BB::PreferCover, false);
} }
} }
void APS_AI_Behavior_AIController::StartBehavior() void APS_AI_Behavior_AIController::StartBehavior()
{ {
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] StartBehavior called. Pawn=%s, HasBB=%d"),
*GetName(),
GetPawn() ? *GetPawn()->GetName() : TEXT("null"),
Blackboard != nullptr);
// Check if a BT was previously loaded and stopped (StopBehavior case)
UBehaviorTreeComponent* BTComp = Cast<UBehaviorTreeComponent>(GetBrainComponent());
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] BrainComponent=%s, BTComp=%s, CurrentTree=%s, IsRunning=%d"),
*GetName(),
GetBrainComponent() ? *GetBrainComponent()->GetName() : TEXT("null"),
BTComp ? *BTComp->GetName() : TEXT("null"),
(BTComp && BTComp->GetCurrentTree()) ? *BTComp->GetCurrentTree()->GetName() : TEXT("null"),
BTComp ? (int32)BTComp->IsRunning() : -1);
if (BTComp && BTComp->GetCurrentTree() && !BTComp->IsRunning())
{
BTComp->RestartLogic();
if (GetBehaviorState() == EPS_AI_Behavior_State::Scripted)
{
SetBehaviorState(EPS_AI_Behavior_State::Idle);
}
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] StartBehavior — BT restarted (RestartLogic). IsRunning=%d"),
*GetName(), (int32)BTComp->IsRunning());
return;
}
// First-time start: load and run the BT
UBehaviorTree* BTToRun = BehaviorTreeAsset; UBehaviorTree* BTToRun = BehaviorTreeAsset;
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] BehaviorTreeAsset=%s"),
*GetName(), BTToRun ? *BTToRun->GetName() : TEXT("null"));
// Fallback: get from personality profile // Fallback: get from personality profile
if (!BTToRun && PersonalityComp && PersonalityComp->Profile) if (!BTToRun && PersonalityComp && PersonalityComp->Profile)
{ {
BTToRun = PersonalityComp->Profile->DefaultBehaviorTree.LoadSynchronous(); BTToRun = PersonalityComp->Profile->DefaultBehaviorTree.LoadSynchronous();
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Loaded from Profile: %s"),
*GetName(), BTToRun ? *BTToRun->GetName() : TEXT("null"));
} }
if (BTToRun) if (!BTToRun)
{ {
RunBehaviorTree(BTToRun); UE_LOG(LogPS_AI_Behavior, Warning,
TEXT("[%s] StartBehavior FAILED — No BehaviorTree assigned and none in PersonalityProfile."),
*GetName());
return;
}
const bool bSuccess = RunBehaviorTree(BTToRun);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] RunBehaviorTree('%s') returned %d"),
*GetName(), *BTToRun->GetName(), (int32)bSuccess);
// Check post-run state
BTComp = Cast<UBehaviorTreeComponent>(GetBrainComponent());
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Post-run: BTComp=%s, IsRunning=%d, BB=%s"),
*GetName(),
BTComp ? *BTComp->GetName() : TEXT("null"),
BTComp ? (int32)BTComp->IsRunning() : -1,
Blackboard ? TEXT("valid") : TEXT("null"));
if (bSuccess && GetBehaviorState() == EPS_AI_Behavior_State::Scripted)
{
SetBehaviorState(EPS_AI_Behavior_State::Idle);
}
}
void APS_AI_Behavior_AIController::StopBehavior()
{
if (UBrainComponent* Brain = GetBrainComponent())
{
Brain->StopLogic(TEXT("Scripted"));
}
StopMovement();
SetBehaviorState(EPS_AI_Behavior_State::Scripted);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] StopBehavior — BT stopped, entering Scripted state."), *GetName());
}
void APS_AI_Behavior_AIController::ResetBehavior()
{
APawn* MyPawn = GetPawn();
if (!MyPawn)
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] ResetBehavior: no Pawn."), *GetName());
return;
}
// ─── 1. Stop any active movement ────────────────────────────────────
StopMovement();
if (auto* Spline = MyPawn->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>())
{
Spline->PauseFollowing();
}
// ─── 2. Reset PersonalityComponent runtime state ────────────────────
if (PersonalityComp)
{
PersonalityComp->ResetRuntimeState();
}
// ─── 3. Recompute TeamId from current NPCType + Faction + Hostile ──
{
EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Any;
if (MyPawn->Implements<UPS_AI_Behavior_Interface>())
{
NPCType = IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(MyPawn);
}
else if (PersonalityComp)
{
NPCType = PersonalityComp->GetNPCType();
}
const uint8 Faction = (PersonalityComp && PersonalityComp->Profile)
? PersonalityComp->Profile->Faction : 0;
uint8 NewTeamId;
if (NPCType == EPS_AI_Behavior_NPCType::Enemy &&
MyPawn->Implements<UPS_AI_Behavior_Interface>() &&
!IPS_AI_Behavior_Interface::Execute_IsBehaviorHostile(MyPawn))
{
NewTeamId = PS_AI_Behavior_Team::DisguisedTeamId;
} }
else else
{ {
UE_LOG(LogPS_AI_Behavior, Warning, NewTeamId = PS_AI_Behavior_Team::MakeTeamId(NPCType, Faction);
TEXT("[%s] No BehaviorTree assigned and none in PersonalityProfile — NPC will be inert."),
*GetName());
} }
SetTeamId(NewTeamId);
}
// ─── 4. Reset Blackboard keys to initial-tick defaults ──────────────
if (Blackboard)
{
// Threat
Blackboard->ClearValue(PS_AI_Behavior_BB::ThreatActor);
Blackboard->ClearValue(PS_AI_Behavior_BB::ThreatLocation);
Blackboard->SetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel, 0.0f);
Blackboard->ClearValue(PS_AI_Behavior_BB::LastKnownTargetPosition);
Blackboard->ClearValue(PS_AI_Behavior_BB::ThreatPawnName);
// Cover
Blackboard->ClearValue(PS_AI_Behavior_BB::CoverLocation);
Blackboard->ClearValue(PS_AI_Behavior_BB::CoverPoint);
Blackboard->SetValueAsBool(PS_AI_Behavior_BB::PreferCover, false);
Blackboard->SetValueAsEnum(PS_AI_Behavior_BB::CombatSubState,
static_cast<uint8>(EPS_AI_Behavior_CombatSubState::Engaging));
// Patrol / Spline
Blackboard->SetValueAsInt(PS_AI_Behavior_BB::PatrolIndex, 0);
Blackboard->ClearValue(PS_AI_Behavior_BB::CurrentSpline);
Blackboard->SetValueAsFloat(PS_AI_Behavior_BB::SplineProgress, 0.0f);
// Re-capture HomeLocation to current actor location (initial-tick semantics)
Blackboard->SetValueAsVector(PS_AI_Behavior_BB::HomeLocation,
MyPawn->GetActorLocation());
// Conversation pause
Blackboard->SetValueAsBool(PS_AI_Behavior_BB::ConversationPaused, false);
}
// ─── 5. Clear local gaze / conversation caches ──────────────────────
bConversationPaused = false;
bUserHasSpokenInConversation = false;
ProximityGazeTarget = nullptr;
ProximityGazeDuration = 0.0f;
ProximityGazeCooldownTimer = 0.0f;
ProximityGazeCooldownActor = nullptr;
ClearGazeTarget();
// ─── 6. Force state back to Idle ────────────────────────────────────
// ForceState on PersonalityComp updates CurrentState + triggers HandleStateChanged
// which calls SetBehaviorMovementSpeed on the Pawn (so walk speed is reset).
// SetBehaviorState syncs the Blackboard for the BT.
if (PersonalityComp)
{
PersonalityComp->ForceState(EPS_AI_Behavior_State::Idle);
}
SetBehaviorState(EPS_AI_Behavior_State::Idle);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] ResetBehavior — back to initial-tick state (Idle)."), *GetName());
} }
void APS_AI_Behavior_AIController::SetBehaviorState(EPS_AI_Behavior_State NewState) void APS_AI_Behavior_AIController::SetBehaviorState(EPS_AI_Behavior_State NewState)
{ {
if (Blackboard) if (Blackboard)
{ {
Blackboard->SetValueAsEnum(PS_AI_Behavior_BB::State, static_cast<uint8>(NewState)); const uint8 OldVal = Blackboard->GetValueAsEnum(PS_AI_Behavior_BB::State);
const uint8 NewVal = static_cast<uint8>(NewState);
if (OldVal != NewVal)
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] BB State: %s -> %s"),
*GetName(),
*UEnum::GetValueAsString(static_cast<EPS_AI_Behavior_State>(OldVal)),
*UEnum::GetValueAsString(NewState));
}
Blackboard->SetValueAsEnum(PS_AI_Behavior_BB::State, NewVal);
// ─── Leaving cover/hiding: stand up ─────────────────────────
const EPS_AI_Behavior_State OldState = static_cast<EPS_AI_Behavior_State>(OldVal);
if ((OldState == EPS_AI_Behavior_State::Fleeing || OldState == EPS_AI_Behavior_State::TakingCover)
&& NewState != EPS_AI_Behavior_State::Fleeing && NewState != EPS_AI_Behavior_State::TakingCover)
{
APawn* MyPawn = GetPawn();
if (MyPawn && MyPawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_SetBehaviorCrouch(MyPawn, false);
}
}
// ─── Scripted: stop movement, NPC stays alive ──────────────
if (NewState == EPS_AI_Behavior_State::Scripted)
{
StopMovement();
}
// ─── Dead: shut down all AI systems ─────────────────────────
if (NewState == EPS_AI_Behavior_State::Dead)
{
HandleDeath();
}
}
}
void APS_AI_Behavior_AIController::HandleDeath()
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] HandleDeath — shutting down AI systems."), *GetName());
ClearGazeTarget();
ProximityGazeTarget = nullptr;
// 1. Stop the Behavior Tree (no more services, tasks, or decorators)
if (UBrainComponent* Brain = GetBrainComponent())
{
Brain->StopLogic(TEXT("Dead"));
}
// 2. Stop any active movement
StopMovement();
// 3. Stop spline following
if (APawn* MyPawn = GetPawn())
{
if (UPS_AI_Behavior_SplineFollowerComponent* Spline =
MyPawn->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>())
{
Spline->StopFollowing();
}
// 4. Stop attack if in combat
if (MyPawn->Implements<UPS_AI_Behavior_Interface>())
{
IPS_AI_Behavior_Interface::Execute_BehaviorStopAttack(MyPawn);
}
}
// 5. Disable perception (stop detecting / being source of stimuli updates)
if (BehaviorPerception)
{
BehaviorPerception->Deactivate();
}
// 6. Clear Blackboard threat data
if (Blackboard)
{
Blackboard->ClearValue(PS_AI_Behavior_BB::ThreatActor);
Blackboard->ClearValue(PS_AI_Behavior_BB::ThreatLocation);
Blackboard->SetValueAsFloat(PS_AI_Behavior_BB::ThreatLevel, 0.0f);
} }
} }
@@ -261,8 +547,6 @@ void APS_AI_Behavior_AIController::SetGenericTeamId(const FGenericTeamId& InTeam
ETeamAttitude::Type APS_AI_Behavior_AIController::GetTeamAttitudeTowards(const AActor& Other) const ETeamAttitude::Type APS_AI_Behavior_AIController::GetTeamAttitudeTowards(const AActor& Other) const
{ {
const uint8 OtherTeamId = FGenericTeamId::NoTeam;
// Try to get the other actor's team ID // Try to get the other actor's team ID
const APawn* OtherPawn = Cast<APawn>(&Other); const APawn* OtherPawn = Cast<APawn>(&Other);
if (!OtherPawn) if (!OtherPawn)
@@ -274,39 +558,85 @@ ETeamAttitude::Type APS_AI_Behavior_AIController::GetTeamAttitudeTowards(const A
if (OtherPawn) if (OtherPawn)
{ {
// Check via AIController first // 1) Check controller's IGenericTeamAgentInterface (AI controllers, custom player controllers)
if (const AAIController* OtherAIC = Cast<AAIController>(OtherPawn->GetController())) if (const AController* OtherController = OtherPawn->GetController())
{ {
OtherTeam = OtherAIC->GetGenericTeamId().GetId(); if (const IGenericTeamAgentInterface* TeamAgent = Cast<IGenericTeamAgentInterface>(OtherController))
}
// Check via IPS_AI_Behavior interface
else if (OtherPawn->Implements<UPS_AI_Behavior_Interface>())
{ {
OtherTeam = IPS_AI_Behavior_Interface::Execute_GetBehaviorTeamId(const_cast<APawn*>(OtherPawn)); OtherTeam = TeamAgent->GetGenericTeamId().GetId();
} }
} }
// NoTeam (255) → Neutral // 2) Fallback: check for our TeamComponent on the Pawn (player characters, etc.)
if (TeamId == FGenericTeamId::NoTeam || OtherTeam == FGenericTeamId::NoTeam) if (OtherTeam == FGenericTeamId::NoTeam)
{ {
if (const UPS_AI_Behavior_TeamComponent* TeamComp = OtherPawn->FindComponentByClass<UPS_AI_Behavior_TeamComponent>())
{
OtherTeam = TeamComp->GetGenericTeamId().GetId();
}
}
// 3) Fallback: check if Pawn implements IPS_AI_Behavior_Interface → derive TeamId from NPCType + Faction
if (OtherTeam == FGenericTeamId::NoTeam && OtherPawn->Implements<UPS_AI_Behavior_Interface>())
{
const EPS_AI_Behavior_NPCType OtherNPCType =
IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(const_cast<APawn*>(OtherPawn));
if (OtherNPCType != EPS_AI_Behavior_NPCType::Any)
{
uint8 OtherFaction = 0;
if (const auto* OtherPersonality = OtherPawn->FindComponentByClass<UPS_AI_Behavior_PersonalityComponent>())
{
if (OtherPersonality->Profile)
{
OtherFaction = OtherPersonality->Profile->Faction;
}
}
OtherTeam = PS_AI_Behavior_Team::MakeTeamId(OtherNPCType, OtherFaction);
}
}
}
// ─── NoTeam handling ────────────────────────────────────────────────
if (TeamId == FGenericTeamId::NoTeam)
{
// We have no team → can't determine attitude
return ETeamAttitude::Neutral; return ETeamAttitude::Neutral;
} }
// Same team → Friendly if (OtherTeam == FGenericTeamId::NoTeam)
{
// NoTeam = not part of the behavior system (spectators, editor pawns, etc.) → always Neutral
static bool bLoggedNoTeamWarning = false;
if (!bLoggedNoTeamWarning)
{
bLoggedNoTeamWarning = true;
UE_LOG(LogPS_AI_Behavior, Warning,
TEXT("GetTeamAttitudeTowards: '%s' has NoTeam (no AIController team, no TeamComponent, no Interface). "
"Add UPS_AI_Behavior_TeamComponent to fix team detection. Treating as Neutral."),
OtherPawn ? *OtherPawn->GetName() : *Other.GetName());
}
return ETeamAttitude::Neutral;
}
// Same TeamId = same NPCType + same Faction → always Friendly
if (TeamId == OtherTeam) if (TeamId == OtherTeam)
{ {
return ETeamAttitude::Friendly; return ETeamAttitude::Friendly;
} }
// ─── Custom cross-team attitudes ──────────────────────────────────── // ─── NPCType-based attitude ─────────────────────────────────────────
const EPS_AI_Behavior_NPCType MyType = PS_AI_Behavior_Team::GetNPCType(TeamId);
const EPS_AI_Behavior_NPCType TheirType = PS_AI_Behavior_Team::GetNPCType(OtherTeam);
// Civilian (1) ↔ Protector (3) → Friendly // Civilian ↔ Protector → always Friendly (regardless of faction)
if ((TeamId == 1 && OtherTeam == 3) || (TeamId == 3 && OtherTeam == 1)) if ((MyType == EPS_AI_Behavior_NPCType::Civilian && TheirType == EPS_AI_Behavior_NPCType::Protector) ||
(MyType == EPS_AI_Behavior_NPCType::Protector && TheirType == EPS_AI_Behavior_NPCType::Civilian))
{ {
return ETeamAttitude::Friendly; return ETeamAttitude::Friendly;
} }
// Everything else → Hostile // Same NPCType but different faction (e.g. rival enemy gangs) → Hostile
// Different NPCType (e.g. Enemy vs Civilian) → Hostile
return ETeamAttitude::Hostile; return ETeamAttitude::Hostile;
} }
@@ -355,3 +685,365 @@ void APS_AI_Behavior_AIController::TryBindConversationAgent()
*GetName()); *GetName());
} }
} }
// ─── Gaze Bridge ────────────────────────────────────────────────────────────
void APS_AI_Behavior_AIController::TryBindGazeComponent()
{
APawn* MyPawn = GetPawn();
if (!MyPawn) return;
// Soft lookup via reflection — no compile dependency on PS_AI_ConvAgent
static UClass* GazeClass = nullptr;
if (!GazeClass)
{
GazeClass = LoadClass<UActorComponent>(nullptr,
TEXT("/Script/PS_AI_ConvAgent.PS_AI_ConvAgent_GazeComponent"));
}
if (!GazeClass) return; // PS_AI_ConvAgent plugin not loaded
UActorComponent* GazeComp = MyPawn->FindComponentByClass(GazeClass);
if (!GazeComp) return;
CachedGazeComponent = GazeComp;
// Cache property pointers
GazeProp_TargetActor = CastField<FObjectProperty>(
GazeClass->FindPropertyByName(TEXT("TargetActor")));
GazeProp_bActive = CastField<FBoolProperty>(
GazeClass->FindPropertyByName(TEXT("bActive")));
GazeProp_bEnableBodyTracking = CastField<FBoolProperty>(
GazeClass->FindPropertyByName(TEXT("bEnableBodyTracking")));
// Also cache the ElevenLabsComponent for conversation detection
static UClass* ConvClass = nullptr;
if (!ConvClass)
{
ConvClass = LoadClass<UActorComponent>(nullptr,
TEXT("/Script/PS_AI_ConvAgent.PS_AI_ConvAgent_ElevenLabsComponent"));
}
if (ConvClass)
{
UActorComponent* ConvComp = MyPawn->FindComponentByClass(ConvClass);
if (ConvComp)
{
CachedConvAgentComponent = ConvComp;
ConvProp_bNetIsConversing = CastField<FBoolProperty>(
ConvClass->FindPropertyByName(TEXT("bNetIsConversing")));
// Agent-to-agent dialogue: the orchestrator pins this agent's gaze on
// the other agent via GazeOverrideTarget. Cache it so we can treat that
// as an active conversation (see IsConversationActiveOnPawn).
ConvProp_GazeOverrideTarget = CastField<FObjectProperty>(
ConvClass->FindPropertyByName(TEXT("GazeOverrideTarget")));
}
}
// Cache MicrophoneCaptureComponent for local VAD
// The mic lives on the PLAYER's Pawn, not the NPC — find it on the first player
static UClass* MicClass = nullptr;
if (!MicClass)
{
MicClass = LoadClass<UActorComponent>(nullptr,
TEXT("/Script/PS_AI_ConvAgent.PS_AI_ConvAgent_MicrophoneCaptureComponent"));
}
if (MicClass)
{
MicProp_bIsUserSpeaking = CastField<FBoolProperty>(
MicClass->FindPropertyByName(TEXT("bIsUserSpeaking")));
// Don't cache the component here — resolve dynamically in UpdateGazeTarget
// because the player Pawn may not exist yet at OnPossess time
}
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Gaze bridge bound: TargetActor=%s, BodyTracking=%s, Conversation=%s, VAD=%s"),
*GetName(),
GazeProp_TargetActor ? TEXT("OK") : TEXT("MISS"),
GazeProp_bEnableBodyTracking ? TEXT("OK") : TEXT("MISS"),
ConvProp_bNetIsConversing ? TEXT("OK") : TEXT("N/A"),
MicProp_bIsUserSpeaking ? TEXT("OK") : TEXT("N/A"));
}
void APS_AI_Behavior_AIController::SetGazeTarget(AActor* Target, bool bEnableBody)
{
UActorComponent* GazeComp = CachedGazeComponent.Get();
if (!GazeComp) return;
if (GazeProp_TargetActor)
{
GazeProp_TargetActor->SetObjectPropertyValue_InContainer(GazeComp, Target);
}
if (GazeProp_bActive)
{
GazeProp_bActive->SetPropertyValue_InContainer(GazeComp, true);
}
if (GazeProp_bEnableBodyTracking)
{
GazeProp_bEnableBodyTracking->SetPropertyValue_InContainer(GazeComp, bEnableBody);
}
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] Gaze → '%s' (body=%d)"),
*GetName(), Target ? *Target->GetName() : TEXT("null"), bEnableBody ? 1 : 0);
}
void APS_AI_Behavior_AIController::ClearGazeTarget()
{
UActorComponent* GazeComp = CachedGazeComponent.Get();
if (!GazeComp || !GazeProp_TargetActor) return;
GazeProp_TargetActor->SetObjectPropertyValue_InContainer(GazeComp, static_cast<UObject*>(nullptr));
if (GazeProp_bEnableBodyTracking)
{
GazeProp_bEnableBodyTracking->SetPropertyValue_InContainer(GazeComp, false);
}
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] Gaze → cleared"), *GetName());
}
bool APS_AI_Behavior_AIController::IsConversationActiveOnPawn() const
{
UActorComponent* ConvComp = CachedConvAgentComponent.Get();
if (!ConvComp) return false;
// Player-driven conversation (a human joined this agent).
if (ConvProp_bNetIsConversing &&
ConvProp_bNetIsConversing->GetPropertyValue_InContainer(ConvComp))
{
return true;
}
// Agent-to-agent dialogue: the orchestrator sets GazeOverrideTarget on this
// agent. Treat it as an active conversation so proximity gaze / wandering back
// off and the ConvAgent owns the gaze (agents look at each other, not a passing
// player). Clearing the override (StopDialogue) restores normal behavior.
if (ConvProp_GazeOverrideTarget &&
ConvProp_GazeOverrideTarget->GetObjectPropertyValue_InContainer(ConvComp))
{
return true;
}
return false;
}
void APS_AI_Behavior_AIController::ApplyMovementPaused(bool bPaused)
{
if (bPaused == bConversationPaused)
{
return; // no change — idempotent
}
bConversationPaused = bPaused;
// Blackboard flag read by the movement-branch decorators (Patrol / Spline).
if (Blackboard)
{
Blackboard->SetValueAsBool(PS_AI_Behavior_BB::ConversationPaused, bPaused);
}
if (bPaused)
{
StopMovement(); // cancel any in-flight MoveTo immediately
}
if (APawn* MyPawn = GetPawn())
{
if (auto* Spline = MyPawn->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>())
{
if (bPaused) { Spline->PauseFollowing(); }
else { Spline->ResumeFollowing(); }
}
}
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Movement %s."),
*GetName(), bPaused ? TEXT("paused") : TEXT("resumed"));
}
void APS_AI_Behavior_AIController::DisableBehaviorMovement()
{
bMovementManuallyDisabled = true;
ApplyMovementPaused(true);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] DisableBehaviorMovement — manual freeze (BT keeps running)."), *GetName());
}
void APS_AI_Behavior_AIController::EnableBehaviorMovement()
{
bMovementManuallyDisabled = false;
// Stay paused if a conversation is still active; otherwise resume.
ApplyMovementPaused(IsConversationActiveOnPawn());
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] EnableBehaviorMovement — manual freeze released."), *GetName());
}
void APS_AI_Behavior_AIController::UpdateGazeTarget(float DeltaSeconds)
{
if (!CachedGazeComponent.IsValid()) return;
// Tick cooldown
if (ProximityGazeCooldownTimer > 0.0f)
{
ProximityGazeCooldownTimer -= DeltaSeconds;
}
const EPS_AI_Behavior_State CurrentState = GetBehaviorState();
// ── Priority 1: Combat → disable gaze (aim animation handles it) ───
if (CurrentState == EPS_AI_Behavior_State::Combat ||
CurrentState == EPS_AI_Behavior_State::TakingCover)
{
ClearGazeTarget();
ProximityGazeTarget = nullptr;
ProximityGazeDuration = 0.0f;
return;
}
// ── Priority 1b: Alerted → look at ThreatActor (not shooting yet) ──
if (CurrentState == EPS_AI_Behavior_State::Alerted)
{
AActor* ThreatActor = Blackboard
? Cast<AActor>(Blackboard->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor))
: nullptr;
if (ThreatActor)
{
APawn* ThreatPawn = UPS_AI_Behavior_PerceptionComponent::FindOwningPawn(ThreatActor);
AActor* GazeActor = ThreatPawn ? static_cast<AActor*>(ThreatPawn) : ThreatActor;
SetGazeTarget(GazeActor, /*bEnableBody=*/ false);
ProximityGazeTarget = nullptr;
ProximityGazeDuration = 0.0f;
return;
}
}
// ── Priority 2: Conversation active OR manual freeze → pause movement ──
// The BT keeps running; a "ConversationPaused" Blackboard decorator blocks the
// movement branches (Patrol / Spline). Leave the Flee branch un-decorated so
// fleeing still works. Movement pauses as soon as the conversation is genuinely
// active (agent greeting included) — it no longer waits for the user to speak —
// or whenever DisableBehaviorMovement() was called.
{
const bool bConversing = IsConversationActiveOnPawn();
ApplyMovementPaused(bMovementManuallyDisabled || bConversing);
if (bConversing)
{
// Gaze is driven by the ConvAgent during conversation.
ProximityGazeTarget = nullptr;
ProximityGazeDuration = 0.0f;
return;
}
// Manual-only freeze (no active conversation): movement stays paused above,
// but proximity gaze below still runs so the NPC can look around.
}
// ── Priority 3/4: Proximity gaze ────────────────────────────────
if (!bEnableProximityGaze || CurrentState == EPS_AI_Behavior_State::Dead)
{
ClearGazeTarget();
return;
}
APawn* MyPawn = GetPawn();
if (!MyPawn) return;
// Check current locked target
AActor* LockedTarget = ProximityGazeTarget.Get();
if (LockedTarget)
{
const float Dist = FVector::Dist(MyPawn->GetActorLocation(), LockedTarget->GetActorLocation());
ProximityGazeDuration += DeltaSeconds;
bool bRelease = (Dist > GazeProximityRadius);
bRelease |= (ProximityGazeDuration >= GazeMaxDuration);
bRelease |= !IsValid(LockedTarget);
if (bRelease)
{
ProximityGazeCooldownActor = LockedTarget;
ProximityGazeCooldownTimer = GazeCooldown;
ProximityGazeTarget = nullptr;
ProximityGazeDuration = 0.0f;
ClearGazeTarget();
return;
}
return; // Still locked
}
// Scan for new proximity target from perceived actors (within sight cone)
if (!BehaviorPerception)
{
ClearGazeTarget();
return;
}
TArray<AActor*> PerceivedActors;
BehaviorPerception->GetCurrentlyPerceivedActors(nullptr, PerceivedActors);
const FVector MyLoc = MyPawn->GetActorLocation();
AActor* BestTarget = nullptr;
float BestDistSq = GazeProximityRadius * GazeProximityRadius;
for (AActor* RawActor : PerceivedActors)
{
APawn* CandidatePawn = UPS_AI_Behavior_PerceptionComponent::FindOwningPawn(RawActor);
if (!CandidatePawn || CandidatePawn == MyPawn) continue;
if (CandidatePawn->IsA<ASpectatorPawn>()) continue;
// Skip actor on cooldown
if (CandidatePawn == ProximityGazeCooldownActor.Get() && ProximityGazeCooldownTimer > 0.0f) continue;
const FVector CandidateLoc = CandidatePawn->GetActorLocation();
const float DistSq = FVector::DistSquared(MyLoc, CandidateLoc);
if (DistSq > BestDistSq) continue;
// Skip actors behind the NPC (only look at actors in front)
const FVector DirToCandidate = (CandidateLoc - MyLoc).GetSafeNormal2D();
const FVector MyForward = MyPawn->GetActorForwardVector().GetSafeNormal2D();
if (FVector::DotProduct(MyForward, DirToCandidate) < 0.0f) continue;
// Skip hostile actors (handled by Priority 1)
const ETeamAttitude::Type Attitude = GetTeamAttitudeTowards(*CandidatePawn);
if (Attitude == ETeamAttitude::Hostile) continue;
BestDistSq = DistSq;
BestTarget = CandidatePawn;
}
if (BestTarget)
{
ProximityGazeTarget = BestTarget;
ProximityGazeDuration = 0.0f;
SetGazeTarget(BestTarget, /*bEnableBody=*/ false); // Head + eyes only
}
else
{
ClearGazeTarget();
}
#if ENABLE_DRAW_DEBUG
// Draw gaze debug line if PersonalityComponent has bDebug enabled
if (PersonalityComp && PersonalityComp->bDebug)
{
AActor* GazeTarget = ProximityGazeTarget.Get();
// In combat, the gaze target is the ThreatActor
if (!GazeTarget && Blackboard &&
(CurrentState == EPS_AI_Behavior_State::Combat ||
CurrentState == EPS_AI_Behavior_State::Alerted ||
CurrentState == EPS_AI_Behavior_State::TakingCover))
{
GazeTarget = Cast<AActor>(Blackboard->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
}
if (GazeTarget)
{
const FVector HeadLoc = MyPawn->GetActorLocation() + FVector(0, 0, 160.0f);
const FVector TargetLoc = GazeTarget->GetActorLocation() + FVector(0, 0, 60.0f);
DrawDebugLine(GetWorld(), HeadLoc, TargetLoc,
FColor::Magenta, false, 0.0f, 0, 0.5f);
DrawDebugString(GetWorld(), HeadLoc + FVector(0, 0, 20.0f),
FString::Printf(TEXT("GAZE: %s [%.1fs]"),
*GazeTarget->GetName(), ProximityGazeDuration),
nullptr, FColor::Magenta, 0.0f, true);
}
}
#endif
}

View File

@@ -4,7 +4,9 @@
#include "Components/ArrowComponent.h" #include "Components/ArrowComponent.h"
#include "Components/BillboardComponent.h" #include "Components/BillboardComponent.h"
#include "Engine/World.h" #include "Engine/World.h"
#include "Engine/Texture2D.h"
#include "CollisionQueryParams.h" #include "CollisionQueryParams.h"
#include "UObject/ConstructorHelpers.h"
APS_AI_Behavior_CoverPoint::APS_AI_Behavior_CoverPoint() APS_AI_Behavior_CoverPoint::APS_AI_Behavior_CoverPoint()
{ {
@@ -27,6 +29,13 @@ APS_AI_Behavior_CoverPoint::APS_AI_Behavior_CoverPoint()
SpriteComp->SetRelativeLocation(FVector(0, 0, 80.0f)); SpriteComp->SetRelativeLocation(FVector(0, 0, 80.0f));
SpriteComp->bIsScreenSizeScaled = true; SpriteComp->bIsScreenSizeScaled = true;
SpriteComp->ScreenSize = 0.0025f; SpriteComp->ScreenSize = 0.0025f;
// Load editor sprites — Cover: default (no change), HidingSpot: fog icon
static ConstructorHelpers::FObjectFinder<UTexture2D> HidingSpotSpriteFinder(
TEXT("/Engine/EditorResources/S_AtmosphericHeightFog"));
CoverSpriteTexture = nullptr; // Keep default billboard sprite for Cover
HidingSpotSpriteTexture = HidingSpotSpriteFinder.Succeeded() ? HidingSpotSpriteFinder.Object : nullptr;
#endif #endif
} }
@@ -164,13 +173,17 @@ void APS_AI_Behavior_CoverPoint::UpdateVisualization()
if (!ArrowComp) return; if (!ArrowComp) return;
FLinearColor Color = FLinearColor::White; FLinearColor Color = FLinearColor::White;
UTexture2D* SpriteTexture = nullptr;
switch (PointType) switch (PointType)
{ {
case EPS_AI_Behavior_CoverPointType::Cover: case EPS_AI_Behavior_CoverPointType::Cover:
Color = FLinearColor(0.2f, 0.5f, 1.0f); // Blue Color = FLinearColor(0.2f, 0.5f, 1.0f); // Blue
SpriteTexture = CoverSpriteTexture;
break; break;
case EPS_AI_Behavior_CoverPointType::HidingSpot: case EPS_AI_Behavior_CoverPointType::HidingSpot:
Color = FLinearColor(1.0f, 0.85f, 0.0f); // Yellow Color = FLinearColor(1.0f, 0.85f, 0.0f); // Yellow
SpriteTexture = HidingSpotSpriteTexture;
break; break;
default: default:
break; break;
@@ -182,6 +195,11 @@ void APS_AI_Behavior_CoverPoint::UpdateVisualization()
} }
ArrowComp->SetArrowColor(Color); ArrowComp->SetArrowColor(Color);
if (SpriteComp && SpriteTexture)
{
SpriteComp->SetSprite(SpriteTexture);
}
#endif #endif
} }

View File

@@ -2,6 +2,6 @@
#include "PS_AI_Behavior_Interface.h" #include "PS_AI_Behavior_Interface.h"
// UInterface boilerplate — no default implementation needed. // Default implementations for BlueprintNativeEvent functions are auto-generated
// All functions are BlueprintNativeEvent and must be implemented // by UHT via GENERATED_BODY(). No manual _Implementation needed.
// by the class that declares "implements IPS_AI_Behavior". // If a BP class doesn't override a function, the UHT-generated default is used.

View File

@@ -4,6 +4,8 @@
#include "PS_AI_Behavior_Interface.h" #include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_PersonalityComponent.h" #include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_PersonalityProfile.h" #include "PS_AI_Behavior_PersonalityProfile.h"
#include "PS_AI_Behavior_TeamComponent.h"
#include "PS_AI_Behavior_Statics.h"
#include "PS_AI_Behavior_Settings.h" #include "PS_AI_Behavior_Settings.h"
#include "Perception/AISenseConfig_Sight.h" #include "Perception/AISenseConfig_Sight.h"
#include "Perception/AISenseConfig_Hearing.h" #include "Perception/AISenseConfig_Hearing.h"
@@ -12,11 +14,15 @@
#include "Perception/AISense_Hearing.h" #include "Perception/AISense_Hearing.h"
#include "Perception/AISense_Damage.h" #include "Perception/AISense_Damage.h"
#include "GameFramework/Pawn.h" #include "GameFramework/Pawn.h"
#include "GameFramework/SpectatorPawn.h"
#include "AIController.h" #include "AIController.h"
#include "BehaviorTree/BlackboardComponent.h"
#include "EngineUtils.h"
UPS_AI_Behavior_PerceptionComponent::UPS_AI_Behavior_PerceptionComponent() UPS_AI_Behavior_PerceptionComponent::UPS_AI_Behavior_PerceptionComponent()
{ {
// Senses are configured in BeginPlay after settings are available // Senses must be configured after construction (NewObject not allowed in CDO constructors).
// We configure in BeginPlay and force a perception system update.
} }
void UPS_AI_Behavior_PerceptionComponent::BeginPlay() void UPS_AI_Behavior_PerceptionComponent::BeginPlay()
@@ -24,6 +30,9 @@ void UPS_AI_Behavior_PerceptionComponent::BeginPlay()
ConfigureSenses(); ConfigureSenses();
Super::BeginPlay(); Super::BeginPlay();
// Force the perception system to re-register our senses now that they're configured
RequestStimuliListenerUpdate();
OnPerceptionUpdated.AddDynamic(this, &UPS_AI_Behavior_PerceptionComponent::HandlePerceptionUpdated); OnPerceptionUpdated.AddDynamic(this, &UPS_AI_Behavior_PerceptionComponent::HandlePerceptionUpdated);
} }
@@ -49,7 +58,7 @@ void UPS_AI_Behavior_PerceptionComponent::ConfigureSenses()
// ─── Hearing ──────────────────────────────────────────────────────── // ─── Hearing ────────────────────────────────────────────────────────
UAISenseConfig_Hearing* HearingConfig = NewObject<UAISenseConfig_Hearing>(this); UAISenseConfig_Hearing* HearingConfig = NewObject<UAISenseConfig_Hearing>(this);
HearingConfig->HearingRange = Settings->DefaultHearingRange; HearingConfig->HearingRange = Settings->DefaultHearingRange;
HearingConfig->SetMaxAge(Settings->PerceptionMaxAge); HearingConfig->SetMaxAge(Settings->HearingMaxAge);
HearingConfig->DetectionByAffiliation.bDetectEnemies = true; HearingConfig->DetectionByAffiliation.bDetectEnemies = true;
HearingConfig->DetectionByAffiliation.bDetectNeutrals = true; HearingConfig->DetectionByAffiliation.bDetectNeutrals = true;
HearingConfig->DetectionByAffiliation.bDetectFriendlies = true; HearingConfig->DetectionByAffiliation.bDetectFriendlies = true;
@@ -70,20 +79,128 @@ void UPS_AI_Behavior_PerceptionComponent::HandlePerceptionUpdated(const TArray<A
// This callback can be used for immediate alert reactions. // This callback can be used for immediate alert reactions.
} }
// ─── Helpers ────────────────────────────────────────────────────────────────
/**
* Find the owning Pawn for a perceived actor.
* Walks up the Owner/Instigator chain until a Pawn is found.
* Returns the Pawn itself — used for team/attitude checks.
* Returns nullptr if no Pawn can be found.
*/
APawn* UPS_AI_Behavior_PerceptionComponent::FindOwningPawn(AActor* Actor)
{
if (!Actor) return nullptr;
// If already a Pawn, return it
if (APawn* ActorAsPawn = Cast<APawn>(Actor))
{
return ActorAsPawn;
}
// Not a Pawn — walk up Owner/Instigator/Attachment chain to find the owning Pawn
AActor* Current = Actor;
for (int32 Depth = 0; Depth < 4; ++Depth) // Safety limit
{
// Try Instigator first (most direct for weapons)
if (APawn* InstigatorPawn = Current->GetInstigator())
{
return InstigatorPawn;
}
// Try Owner
AActor* OwnerActor = Current->GetOwner();
if (OwnerActor && OwnerActor != Current)
{
if (APawn* OwnerPawn = Cast<APawn>(OwnerActor))
{
return OwnerPawn;
}
Current = OwnerActor;
continue;
}
// Try attachment parent (ChildActorComponent → parent Character)
AActor* ParentActor = Current->GetAttachParentActor();
if (ParentActor && ParentActor != Current)
{
if (APawn* ParentPawn = Cast<APawn>(ParentActor))
{
return ParentPawn;
}
Current = ParentActor;
continue;
}
break; // No more chain to walk
}
// Fallback: could not resolve to a Pawn.
// Log once per actor class to avoid spam.
static TSet<FName> WarnedClasses;
const FName ClassName = Actor->GetClass()->GetFName();
if (!WarnedClasses.Contains(ClassName))
{
WarnedClasses.Add(ClassName);
UE_LOG(LogPS_AI_Behavior, Warning,
TEXT("FindOwningPawn: '%s' (class=%s) could not be resolved to a Pawn. Set Owner or Instigator. Instigator=%s, Owner=%s"),
*Actor->GetName(), *Actor->GetClass()->GetName(),
Actor->GetInstigator() ? *Actor->GetInstigator()->GetName() : TEXT("null"),
Actor->GetOwner() ? *Actor->GetOwner()->GetName() : TEXT("null"));
}
return nullptr;
}
/**
* Get the threat target actor for a Pawn.
* Calls GetBehaviorThreatActor() if the Pawn implements the interface,
* otherwise returns the Pawn itself.
* This is what goes into the Blackboard (e.g. PS_AimTargetActor for aiming).
*/
static AActor* GetThreatTarget(APawn* Pawn)
{
if (!Pawn) return nullptr;
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
AActor* Resolved = IPS_AI_Behavior_Interface::Execute_GetBehaviorThreatActor(Pawn);
return Resolved ? Resolved : Pawn;
}
return Pawn;
}
/** Extract an actor's TeamId (checking controller, then TeamComponent). */
uint8 UPS_AI_Behavior_PerceptionComponent::GetActorTeamId(const AActor* Actor)
{
const APawn* ActorPawn = Cast<APawn>(Actor);
if (!ActorPawn) return FGenericTeamId::NoTeam;
if (const AController* C = ActorPawn->GetController())
{
if (const IGenericTeamAgentInterface* T = Cast<IGenericTeamAgentInterface>(C))
{
return T->GetGenericTeamId().GetId();
}
}
if (const UPS_AI_Behavior_TeamComponent* TC = ActorPawn->FindComponentByClass<UPS_AI_Behavior_TeamComponent>())
{
return TC->GetGenericTeamId().GetId();
}
return FGenericTeamId::NoTeam;
}
// ─── Actor Classification ─────────────────────────────────────────────────── // ─── Actor Classification ───────────────────────────────────────────────────
EPS_AI_Behavior_TargetType UPS_AI_Behavior_PerceptionComponent::ClassifyActor(const AActor* Actor) EPS_AI_Behavior_TargetType UPS_AI_Behavior_PerceptionComponent::ClassifyActor(const AActor* Actor)
{ {
if (!Actor) return EPS_AI_Behavior_TargetType::Civilian; // Safe default if (!Actor) return EPS_AI_Behavior_TargetType::Civilian; // Safe default
// Check if player-controlled
const APawn* Pawn = Cast<APawn>(Actor); const APawn* Pawn = Cast<APawn>(Actor);
if (Pawn && Pawn->IsPlayerControlled())
{
return EPS_AI_Behavior_TargetType::Player;
}
// Check via IPS_AI_Behavior interface // 1) Check via IPS_AI_Behavior interface (NPCs and any Pawn implementing it)
if (Actor->Implements<UPS_AI_Behavior_Interface>()) if (Actor->Implements<UPS_AI_Behavior_Interface>())
{ {
const EPS_AI_Behavior_NPCType NPCType = const EPS_AI_Behavior_NPCType NPCType =
@@ -98,9 +215,21 @@ EPS_AI_Behavior_TargetType UPS_AI_Behavior_PerceptionComponent::ClassifyActor(co
} }
} }
// Fallback: check PersonalityComponent // 2) Check TeamComponent (player characters, non-AI pawns)
if (Pawn) if (Pawn)
{ {
if (const UPS_AI_Behavior_TeamComponent* TeamComp = Pawn->FindComponentByClass<UPS_AI_Behavior_TeamComponent>())
{
switch (TeamComp->Role)
{
case EPS_AI_Behavior_NPCType::Civilian: return EPS_AI_Behavior_TargetType::Civilian;
case EPS_AI_Behavior_NPCType::Enemy: return EPS_AI_Behavior_TargetType::Enemy;
case EPS_AI_Behavior_NPCType::Protector: return EPS_AI_Behavior_TargetType::Protector;
default: break;
}
}
// 3) Fallback: PersonalityComponent
if (const auto* PersonalityComp = Pawn->FindComponentByClass<UPS_AI_Behavior_PersonalityComponent>()) if (const auto* PersonalityComp = Pawn->FindComponentByClass<UPS_AI_Behavior_PersonalityComponent>())
{ {
switch (PersonalityComp->GetNPCType()) switch (PersonalityComp->GetNPCType())
@@ -111,6 +240,12 @@ EPS_AI_Behavior_TargetType UPS_AI_Behavior_PerceptionComponent::ClassifyActor(co
default: break; default: break;
} }
} }
// 4) Player-controlled but no role defined → Player type
if (Pawn->IsPlayerControlled())
{
return EPS_AI_Behavior_TargetType::Player;
}
} }
return EPS_AI_Behavior_TargetType::Civilian; return EPS_AI_Behavior_TargetType::Civilian;
@@ -128,7 +263,7 @@ AActor* UPS_AI_Behavior_PerceptionComponent::GetHighestThreatActor()
{ {
// Owner is the AIController, get the Pawn // Owner is the AIController, get the Pawn
const AAIController* AIC = Cast<AAIController>(Owner); const AAIController* AIC = Cast<AAIController>(Owner);
const APawn* MyPawn = AIC ? AIC->GetPawn() : Cast<APawn>(const_cast<AActor*>(Owner)); const APawn* MyPawn = AIC ? ToRawPtr(AIC->GetPawn()) : Cast<APawn>(const_cast<AActor*>(Owner));
if (MyPawn) if (MyPawn)
{ {
@@ -152,6 +287,62 @@ AActor* UPS_AI_Behavior_PerceptionComponent::GetHighestThreatActor(
TArray<AActor*> PerceivedActors; TArray<AActor*> PerceivedActors;
GetCurrentlyPerceivedActors(nullptr, PerceivedActors); // All senses at once GetCurrentlyPerceivedActors(nullptr, PerceivedActors); // All senses at once
// Resolve debug flag once for all logs in this function
const AAIController* DebugAIC = Cast<AAIController>(GetOwner());
const APawn* DebugPawn = DebugAIC ? DebugAIC->GetPawn() : nullptr;
const bool bDebugLog = UPS_AI_Behavior_Statics::IsDebugEnabled(DebugPawn);
// ─── Omniscient awareness for high-priority target types ────────
// Protectors (police, player) are always known if within sight radius,
// even outside the perception cone. This ensures enemies prioritize
// dangerous threats they're aware of (e.g. a cop standing next to them).
if (TargetPriority.Num() > 0)
{
const AAIController* OwnerAIC = Cast<AAIController>(GetOwner());
const APawn* OwnerPawn = OwnerAIC ? OwnerAIC->GetPawn() : nullptr;
// Only run on server — TActorIterator results differ on clients
if (OwnerPawn && OwnerPawn->HasAuthority())
{
const EPS_AI_Behavior_TargetType TopPriority = TargetPriority[0];
const UPS_AI_Behavior_Settings* Settings = GetDefault<UPS_AI_Behavior_Settings>();
const float AwarenessRadius = Settings->DefaultSightRadius;
const FVector OwnerLoc = OwnerPawn->GetActorLocation();
for (TActorIterator<APawn> It(GetWorld()); It; ++It)
{
APawn* CandidatePawn = *It;
if (!CandidatePawn || CandidatePawn == OwnerPawn) continue;
// Already perceived → skip
if (PerceivedActors.Contains(CandidatePawn)) continue;
// Check distance
const float Dist = FVector::Dist(OwnerLoc, CandidatePawn->GetActorLocation());
if (Dist > AwarenessRadius) continue;
// Check if this actor matches the top priority type
const EPS_AI_Behavior_TargetType CandidateType = ClassifyActor(CandidatePawn);
const EPS_AI_Behavior_TargetType MappedType =
(CandidateType == EPS_AI_Behavior_TargetType::Player)
? EPS_AI_Behavior_TargetType::Protector
: CandidateType;
if (MappedType == TopPriority)
{
PerceivedActors.Add(CandidatePawn);
if (bDebugLog)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Omniscient awareness: added '%s' (type=%d, dist=%.0f) — top priority target"),
*OwnerPawn->GetName(), *CandidatePawn->GetName(),
static_cast<int32>(CandidateType), Dist);
}
}
}
}
}
if (PerceivedActors.Num() == 0) if (PerceivedActors.Num() == 0)
{ {
return nullptr; return nullptr;
@@ -176,36 +367,130 @@ AActor* UPS_AI_Behavior_PerceptionComponent::GetHighestThreatActor(
AActor* BestThreat = nullptr; AActor* BestThreat = nullptr;
float BestScore = -1.0f; float BestScore = -1.0f;
for (AActor* Actor : PerceivedActors) // Get our Pawn for IsTargetActorValid checks
{
if (!Actor || Actor == Owner) continue;
// Skip self (when owner is AIController, also skip own pawn)
const AAIController* AIC = Cast<AAIController>(Owner); const AAIController* AIC = Cast<AAIController>(Owner);
if (AIC && Actor == AIC->GetPawn()) continue; APawn* MyPawn = AIC ? ToRawPtr(AIC->GetPawn()) : Cast<APawn>(const_cast<AActor*>(Owner));
// ─── Classify this actor ──────────────────────────────────────── // ─── Score accumulation per resolved actor ───────────────────────
const EPS_AI_Behavior_TargetType ActorType = ClassifyActor(Actor); // Multiple RawActors can resolve to the same Pawn (e.g. Pawn seen + weapon heard).
// Accumulate their sense scores so a single actor gets all stimuli credit.
// Check if this target type is in our priority list at all struct FActorScore
const int32 PriorityIndex = ActivePriority.IndexOfByKey(ActorType);
if (PriorityIndex == INDEX_NONE)
{ {
// Not a valid target for this NPC AActor* Actor = nullptr;
float Score = 0.0f;
EPS_AI_Behavior_TargetType ActorType = EPS_AI_Behavior_TargetType::Civilian;
bool bIsHostile = false;
};
TMap<AActor*, FActorScore> ScoreMap;
for (AActor* RawActor : PerceivedActors)
{
// Find the owning Pawn (for team/attitude checks)
APawn* OwningPawn = FindOwningPawn(RawActor);
if (!OwningPawn) continue; // Can't resolve → skip
// Skip self
if (OwningPawn == Owner) continue;
if (AIC && OwningPawn == AIC->GetPawn()) continue;
// Skip spectator pawns (editor camera, spectating players, etc.)
if (OwningPawn->IsA<ASpectatorPawn>()) continue;
// Get the threat target (PS_AimTargetActor or Pawn itself — for BB targeting)
AActor* ThreatTarget = GetThreatTarget(OwningPawn);
if (!ThreatTarget) ThreatTarget = OwningPawn;
// Skip non-hostile actors UNLESS they have a gunfire hearing stimulus
bool bActorIsHostile = false;
bool bActorHasGunshot = false;
if (AIC)
{
// Attitude check against the PAWN (has TeamId), not the ThreatTarget
const ETeamAttitude::Type Attitude = AIC->GetTeamAttitudeTowards(*OwningPawn);
bActorIsHostile = (Attitude == ETeamAttitude::Hostile);
if (!bActorIsHostile)
{
// Same exact team (same NPCType + same Faction) → always skip
// Allied teams (Civilian ↔ Protector) → allow gunfire through
if (AIC->GetGenericTeamId().GetId() == GetActorTeamId(OwningPawn))
{
if (bDebugLog)
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Target '%s': SKIPPED (same team 0x%02X)"),
*Owner->GetName(), *OwningPawn->GetName(), GetActorTeamId(OwningPawn));
}
continue; continue;
} }
// ─── Score calculation ────────────────────────────────────────── // Allied or Neutral: check for gunfire tag — gunshot source is a valid threat
float Score = 0.0f; FActorPerceptionBlueprintInfo GunInfo;
if (GetActorsPerception(RawActor, GunInfo))
{
for (const FAIStimulus& S : GunInfo.LastSensedStimuli)
{
if (S.IsValid() &&
S.Type == UAISense::GetSenseID<UAISense_Hearing>() &&
PS_AI_Behavior_Tags_Internal::IsGunfire(S.Tag))
{
bActorHasGunshot = true;
break;
}
}
}
// Priority rank bonus: higher priority = much higher score if (!bActorHasGunshot)
// Max priority entries = ~4, so (4 - index) * 100 gives clear separation {
Score += (ActivePriority.Num() - PriorityIndex) * 100.0f; if (bDebugLog)
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Target '%s': SKIPPED (attitude=%d, not hostile, no gunshot, theirTeam=0x%02X, myTeam=0x%02X)"),
*Owner->GetName(), *OwningPawn->GetName(), static_cast<int32>(Attitude),
GetActorTeamId(OwningPawn), AIC->GetGenericTeamId().GetId());
}
continue; // Not hostile, no gunshot — skip
}
}
}
// Damage sense override: actor that hit us gets a massive bonus // Skip invalid targets (dead, despawning, etc.) via interface
// (bypasses priority — self-defense) if (MyPawn && MyPawn->Implements<UPS_AI_Behavior_Interface>())
{
if (!IPS_AI_Behavior_Interface::Execute_IsTargetActorValid(MyPawn, ThreatTarget))
{
continue;
}
}
// ─── Classify & score per owning Pawn, but store ThreatTarget for BB ───
FActorScore& Entry = ScoreMap.FindOrAdd(OwningPawn);
if (!Entry.Actor)
{
// First time seeing this Pawn — initialize with its ThreatTarget
Entry.Actor = ThreatTarget;
Entry.ActorType = ClassifyActor(OwningPawn);
Entry.bIsHostile = bActorIsHostile;
// Priority rank bonus (applied once per actor)
const int32 PriorityIndex = ActivePriority.IndexOfByKey(Entry.ActorType);
if (PriorityIndex != INDEX_NONE)
{
Entry.Score += (ActivePriority.Num() - PriorityIndex) * 100.0f;
}
else
{
Entry.Score += 10.0f;
}
// Distance bonus (applied once per actor)
const float Dist = FVector::Dist(OwnerLoc, ThreatTarget->GetActorLocation());
Entry.Score += FMath::GetMappedRangeValueClamped(
FVector2D(0.0f, 6000.0f), FVector2D(20.0f, 0.0f), Dist);
}
// Accumulate sense scores from this RawActor (weapon heard + pawn seen → both count)
FActorPerceptionBlueprintInfo Info; FActorPerceptionBlueprintInfo Info;
if (GetActorsPerception(Actor, Info)) if (GetActorsPerception(RawActor, Info))
{ {
for (const FAIStimulus& Stimulus : Info.LastSensedStimuli) for (const FAIStimulus& Stimulus : Info.LastSensedStimuli)
{ {
@@ -213,30 +498,102 @@ AActor* UPS_AI_Behavior_PerceptionComponent::GetHighestThreatActor(
if (Stimulus.Type == UAISense::GetSenseID<UAISense_Damage>()) if (Stimulus.Type == UAISense::GetSenseID<UAISense_Damage>())
{ {
Score += 500.0f; // Self-defense: always prioritize attacker Entry.Score += 500.0f;
} }
else if (Stimulus.Type == UAISense::GetSenseID<UAISense_Sight>()) else if (Stimulus.Type == UAISense::GetSenseID<UAISense_Hearing>())
{ {
Score += 10.0f; if (PS_AI_Behavior_Tags_Internal::IsGunfire(Stimulus.Tag))
{
Entry.Score += 400.0f;
} }
else else
{ {
Score += 5.0f; // Hearing Entry.Score += 5.0f;
} }
} }
} else if (Stimulus.Type == UAISense::GetSenseID<UAISense_Sight>())
// Distance: closer targets score higher (0-20 range)
const float Dist = FVector::Dist(OwnerLoc, Actor->GetActorLocation());
Score += FMath::GetMappedRangeValueClamped(
FVector2D(0.0f, 6000.0f), FVector2D(20.0f, 0.0f), Dist);
if (Score > BestScore)
{ {
BestScore = Score; Entry.Score += 10.0f;
BestThreat = Actor;
} }
} }
}
}
// ─── Pick best actor from accumulated scores ─────────────────────
// Target persistence: get current target from BB to avoid flickering
AActor* CurrentTarget = nullptr;
float CurrentTargetScore = -1.0f;
if (AIC)
{
if (const UBlackboardComponent* BB = AIC->GetBlackboardComponent())
{
CurrentTarget = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor));
}
}
APawn* BestOwningPawn = nullptr;
for (const auto& Pair : ScoreMap)
{
const FActorScore& Entry = Pair.Value;
if (bDebugLog)
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Target '%s' (pawn='%s'): type=%d, hostile=%d, score=%.0f"),
*Owner->GetName(), *Entry.Actor->GetName(), *Pair.Key->GetName(),
static_cast<int32>(Entry.ActorType), Entry.bIsHostile ? 1 : 0, Entry.Score);
}
if (Entry.Score > BestScore)
{
BestScore = Entry.Score;
BestThreat = Entry.Actor;
BestOwningPawn = Cast<APawn>(Pair.Key);
}
// Track current target's score for persistence check
if (Entry.Actor == CurrentTarget)
{
CurrentTargetScore = Entry.Score;
}
}
// Target persistence: keep current target if its score is within 20% of the best
// This prevents flickering between targets with nearly identical scores
if (CurrentTarget && CurrentTargetScore > 0.0f && BestThreat != CurrentTarget)
{
if (CurrentTargetScore >= BestScore * 0.8f)
{
BestThreat = CurrentTarget;
BestScore = CurrentTargetScore;
// Find the owning Pawn for the kept target
for (const auto& Pair : ScoreMap)
{
if (Pair.Value.Actor == CurrentTarget) { BestOwningPawn = Cast<APawn>(Pair.Key); break; }
}
}
}
// Store the owning Pawn for callers (UpdateThreat reads this for debug display)
LastThreatOwningPawn = BestOwningPawn;
if (BestThreat)
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] → Best target: '%s' (pawn='%s', score=%.0f%s)"),
*Owner->GetName(), *BestThreat->GetName(),
BestOwningPawn ? *BestOwningPawn->GetName() : TEXT("?"),
BestScore,
(BestThreat == CurrentTarget) ? TEXT(" [kept]") : TEXT(""));
}
else if (PerceivedActors.Num() > 0)
{
// Normal case: civilian surrounded by non-hostile actors, or disguised enemy
// seeing allies. Only log at Verbose — not an issue.
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] GetHighestThreatActor: perceived %d actors, none hostile (myTeam=0x%02X)."),
*Owner->GetName(), PerceivedActors.Num(),
AIC ? AIC->GetGenericTeamId().GetId() : 255);
}
return BestThreat; return BestThreat;
} }
@@ -252,20 +609,88 @@ float UPS_AI_Behavior_PerceptionComponent::CalculateThreatLevel()
TArray<AActor*> PerceivedActors; TArray<AActor*> PerceivedActors;
GetCurrentlyPerceivedActors(nullptr, PerceivedActors); // All senses GetCurrentlyPerceivedActors(nullptr, PerceivedActors); // All senses
for (AActor* Actor : PerceivedActors) // Get our AIController and Pawn for checks
const AAIController* AIC = Cast<AAIController>(Owner);
APawn* MyPawn = AIC ? ToRawPtr(AIC->GetPawn()) : Cast<APawn>(const_cast<AActor*>(Owner));
for (AActor* RawActor : PerceivedActors)
{ {
if (!Actor) continue; if (!RawActor) continue;
// Find the owning Pawn (for team/attitude checks)
APawn* OwningPawn = FindOwningPawn(RawActor);
if (!OwningPawn) continue;
// Skip spectator pawns
if (OwningPawn->IsA<ASpectatorPawn>()) continue;
// Get the threat target for position-based calculations
AActor* ThreatTarget = GetThreatTarget(OwningPawn);
if (!ThreatTarget) ThreatTarget = OwningPawn;
// Determine hostility and check for gunshot stimuli
bool bIsHostile = false;
bool bHasGunshot = false;
if (AIC)
{
// Attitude check against the PAWN (has TeamId)
const ETeamAttitude::Type Attitude = AIC->GetTeamAttitudeTowards(*OwningPawn);
bIsHostile = (Attitude == ETeamAttitude::Hostile);
// Same exact team (same NPCType + same Faction) → always skip
// Allied teams (Civilian ↔ Protector) → allow gunfire through
if (AIC->GetGenericTeamId().GetId() == GetActorTeamId(OwningPawn))
{
continue;
}
}
// For non-hostile actors, check if they have a gunfire hearing stimulus
if (!bIsHostile)
{
FActorPerceptionBlueprintInfo GunInfo;
if (GetActorsPerception(RawActor, GunInfo))
{
for (const FAIStimulus& S : GunInfo.LastSensedStimuli)
{
if (S.IsValid() &&
S.Type == UAISense::GetSenseID<UAISense_Hearing>() &&
PS_AI_Behavior_Tags_Internal::IsGunfire(S.Tag))
{
bHasGunshot = true;
break;
}
}
}
}
// Skip actors that are neither hostile nor gunshot sources
if (!bIsHostile && !bHasGunshot)
{
continue;
}
// Skip invalid targets (dead, despawning, etc.) via interface
if (MyPawn && MyPawn->Implements<UPS_AI_Behavior_Interface>())
{
if (!IPS_AI_Behavior_Interface::Execute_IsTargetActorValid(MyPawn, ThreatTarget))
{
continue;
}
}
float ActorThreat = 0.0f; float ActorThreat = 0.0f;
const float Dist = FVector::Dist(OwnerLoc, Actor->GetActorLocation()); const float Dist = FVector::Dist(OwnerLoc, ThreatTarget->GetActorLocation());
// Closer = more threatening if (bIsHostile)
{
// ─── Normal hostile threat calculation ──────────────────────
ActorThreat += FMath::GetMappedRangeValueClamped( ActorThreat += FMath::GetMappedRangeValueClamped(
FVector2D(0.0f, 5000.0f), FVector2D(0.5f, 0.05f), Dist); FVector2D(0.0f, 5000.0f), FVector2D(0.5f, 0.05f), Dist);
// Sense-based multiplier
FActorPerceptionBlueprintInfo Info; FActorPerceptionBlueprintInfo Info;
if (GetActorsPerception(Actor, Info)) if (GetActorsPerception(RawActor, Info))
{ {
for (const FAIStimulus& Stimulus : Info.LastSensedStimuli) for (const FAIStimulus& Stimulus : Info.LastSensedStimuli)
{ {
@@ -273,7 +698,7 @@ float UPS_AI_Behavior_PerceptionComponent::CalculateThreatLevel()
if (Stimulus.Type == UAISense::GetSenseID<UAISense_Damage>()) if (Stimulus.Type == UAISense::GetSenseID<UAISense_Damage>())
{ {
ActorThreat += 0.6f; // Being hit = big threat spike ActorThreat += 0.6f;
} }
else if (Stimulus.Type == UAISense::GetSenseID<UAISense_Sight>()) else if (Stimulus.Type == UAISense::GetSenseID<UAISense_Sight>())
{ {
@@ -285,6 +710,15 @@ float UPS_AI_Behavior_PerceptionComponent::CalculateThreatLevel()
} }
} }
} }
}
else if (bHasGunshot)
{
// ─── Gunshot from non-hostile actor ────────────────────────
// Generates significant threat regardless of team affiliation
ActorThreat += FMath::GetMappedRangeValueClamped(
FVector2D(0.0f, 5000.0f), FVector2D(0.4f, 0.05f), Dist);
ActorThreat += 0.3f; // Gunshot hearing boost
}
TotalThreat += ActorThreat; TotalThreat += ActorThreat;
} }
@@ -324,3 +758,31 @@ bool UPS_AI_Behavior_PerceptionComponent::GetThreatLocation(FVector& OutLocation
return false; return false;
} }
bool UPS_AI_Behavior_PerceptionComponent::GetGunShotStimulusLocation(FVector& OutLocation)
{
TArray<AActor*> PerceivedActors;
GetCurrentlyPerceivedActors(nullptr, PerceivedActors);
for (AActor* Actor : PerceivedActors)
{
if (!Actor) continue;
FActorPerceptionBlueprintInfo Info;
if (GetActorsPerception(Actor, Info))
{
for (const FAIStimulus& Stimulus : Info.LastSensedStimuli)
{
if (Stimulus.IsValid() &&
Stimulus.Type == UAISense::GetSenseID<UAISense_Hearing>() &&
PS_AI_Behavior_Tags_Internal::IsGunfire(Stimulus.Tag))
{
OutLocation = Stimulus.StimulusLocation;
return true;
}
}
}
}
return false;
}

View File

@@ -3,11 +3,20 @@
#include "PS_AI_Behavior_PersonalityComponent.h" #include "PS_AI_Behavior_PersonalityComponent.h"
#include "PS_AI_Behavior_Interface.h" #include "PS_AI_Behavior_Interface.h"
#include "PS_AI_Behavior_PersonalityProfile.h" #include "PS_AI_Behavior_PersonalityProfile.h"
#include "PS_AI_Behavior_Statics.h"
#include "PS_AI_Behavior_AIController.h"
#include "PS_AI_Behavior_SplineFollowerComponent.h"
#include "PS_AI_Behavior_SplinePath.h"
#include "BehaviorTree/BlackboardComponent.h"
#include "Net/UnrealNetwork.h" #include "Net/UnrealNetwork.h"
#include "DrawDebugHelpers.h"
#include "Kismet/KismetSystemLibrary.h"
#include "AIController.h"
UPS_AI_Behavior_PersonalityComponent::UPS_AI_Behavior_PersonalityComponent() UPS_AI_Behavior_PersonalityComponent::UPS_AI_Behavior_PersonalityComponent()
{ {
PrimaryComponentTick.bCanEverTick = false; PrimaryComponentTick.bCanEverTick = true;
bTickInEditor = true; // Required for Simulate In Editor
SetIsReplicatedByDefault(true); SetIsReplicatedByDefault(true);
} }
@@ -36,10 +45,9 @@ void UPS_AI_Behavior_PersonalityComponent::BeginPlay()
else else
{ {
// Defaults — all traits at 0.5 // Defaults — all traits at 0.5
for (uint8 i = 0; i <= static_cast<uint8>(EPS_AI_Behavior_TraitAxis::Discipline); ++i) RuntimeTraits.Add(EPS_AI_Behavior_TraitAxis::Courage, 0.5f);
{ RuntimeTraits.Add(EPS_AI_Behavior_TraitAxis::Aggressivity, 0.5f);
RuntimeTraits.Add(static_cast<EPS_AI_Behavior_TraitAxis>(i), 0.5f); RuntimeTraits.Add(EPS_AI_Behavior_TraitAxis::Caution, 0.5f);
}
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] No PersonalityProfile assigned — using default traits."), UE_LOG(LogPS_AI_Behavior, Warning, TEXT("[%s] No PersonalityProfile assigned — using default traits."),
*GetOwner()->GetName()); *GetOwner()->GetName());
} }
@@ -64,6 +72,26 @@ EPS_AI_Behavior_State UPS_AI_Behavior_PersonalityComponent::EvaluateReaction() c
return EPS_AI_Behavior_State::Dead; return EPS_AI_Behavior_State::Dead;
} }
if (CurrentState == EPS_AI_Behavior_State::Scripted)
{
return EPS_AI_Behavior_State::Scripted;
}
// Disguised Enemy (NPCType=Enemy but IsBehaviorHostile=false) — cannot enter
// active combat states. Behave like a civilian (Alerted max, or Fleeing if terrified).
bool bDisguisedEnemy = false;
if (APawn* OwnerPawn = Cast<APawn>(GetOwner()))
{
if (OwnerPawn->Implements<UPS_AI_Behavior_Interface>())
{
const EPS_AI_Behavior_NPCType NPCT =
IPS_AI_Behavior_Interface::Execute_GetBehaviorNPCType(OwnerPawn);
const bool bHostile =
IPS_AI_Behavior_Interface::Execute_IsBehaviorHostile(OwnerPawn);
bDisguisedEnemy = (NPCT == EPS_AI_Behavior_NPCType::Enemy && !bHostile);
}
}
const float Courage = GetTrait(EPS_AI_Behavior_TraitAxis::Courage); const float Courage = GetTrait(EPS_AI_Behavior_TraitAxis::Courage);
const float Aggressivity = GetTrait(EPS_AI_Behavior_TraitAxis::Aggressivity); const float Aggressivity = GetTrait(EPS_AI_Behavior_TraitAxis::Aggressivity);
const float Caution = GetTrait(EPS_AI_Behavior_TraitAxis::Caution); const float Caution = GetTrait(EPS_AI_Behavior_TraitAxis::Caution);
@@ -80,54 +108,221 @@ EPS_AI_Behavior_State UPS_AI_Behavior_PersonalityComponent::EvaluateReaction() c
const float EffectiveFleeThresh = FleeThresh * (0.5f + Courage * 0.5f) * (1.5f - Caution * 0.5f); const float EffectiveFleeThresh = FleeThresh * (0.5f + Courage * 0.5f) * (1.5f - Caution * 0.5f);
const float EffectiveAttackThresh = AttackThresh * (1.5f - Aggressivity * 0.5f); const float EffectiveAttackThresh = AttackThresh * (1.5f - Aggressivity * 0.5f);
// Decision cascade if (UPS_AI_Behavior_Statics::IsDebugEnabled(Cast<APawn>(GetOwner())))
if (PerceivedThreatLevel >= EffectiveFleeThresh && Courage < 0.7f) {
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] EvaluateReaction: threat=%.2f, aggr=%.2f, courage=%.2f, caution=%.2f | attackThresh=%.2f, fleeThresh=%.2f, alertThresh=%.2f"),
GetOwner() ? *GetOwner()->GetName() : TEXT("?"),
PerceivedThreatLevel, Aggressivity, Courage, Caution,
EffectiveAttackThresh, EffectiveFleeThresh, AlertThresh);
}
// Decision cascade — with hysteresis to prevent state flickering.
// Once in a state, require a larger threshold drop to leave it.
const float Hysteresis = 0.15f;
if (CurrentState == EPS_AI_Behavior_State::Fleeing)
{
// Stay fleeing until threat drops well below the threshold
if (PerceivedThreatLevel >= (EffectiveFleeThresh - Hysteresis))
{
return EPS_AI_Behavior_State::Fleeing;
}
}
else if (PerceivedThreatLevel >= EffectiveFleeThresh && Courage < 0.7f)
{ {
return EPS_AI_Behavior_State::Fleeing; return EPS_AI_Behavior_State::Fleeing;
} }
if (PerceivedThreatLevel >= EffectiveAttackThresh && Aggressivity > 0.3f) // Disguised enemies never enter active combat — skip the Combat branch entirely.
if (!bDisguisedEnemy)
{ {
// Cautious NPCs prefer cover over direct combat if (CurrentState == EPS_AI_Behavior_State::Combat)
if (Caution > 0.6f) {
// Stay in combat until threat drops well below the threshold
if (PerceivedThreatLevel >= (EffectiveAttackThresh - Hysteresis))
{ {
return EPS_AI_Behavior_State::TakingCover;
}
return EPS_AI_Behavior_State::Combat; return EPS_AI_Behavior_State::Combat;
} }
}
else if (PerceivedThreatLevel >= EffectiveAttackThresh && Aggressivity > 0.0f)
{
return EPS_AI_Behavior_State::Combat;
}
}
if (PerceivedThreatLevel >= AlertThresh) if (PerceivedThreatLevel >= AlertThresh)
{ {
return EPS_AI_Behavior_State::Alerted; return EPS_AI_Behavior_State::Alerted;
} }
// No threat — maintain patrol or idle // No threat — patrol if following a spline, otherwise idle
return (CurrentState == EPS_AI_Behavior_State::Patrol) if (const AActor* Owner = GetOwner())
? EPS_AI_Behavior_State::Patrol {
: EPS_AI_Behavior_State::Idle; if (const UPS_AI_Behavior_SplineFollowerComponent* Spline =
Owner->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>())
{
if (Spline->bIsFollowing || Spline->CurrentSpline)
{
return EPS_AI_Behavior_State::Patrol;
}
}
}
return EPS_AI_Behavior_State::Idle;
} }
EPS_AI_Behavior_State UPS_AI_Behavior_PersonalityComponent::ApplyReaction() EPS_AI_Behavior_State UPS_AI_Behavior_PersonalityComponent::ApplyReaction()
{ {
const EPS_AI_Behavior_State NewState = EvaluateReaction(); // Only server can change replicated state
if (!GetOwner() || !GetOwner()->HasAuthority())
{
return CurrentState;
}
EPS_AI_Behavior_State NewState = EvaluateReaction();
// ─── Combat/Cover cycle timer ──────────────────────────────────
// While in Combat, alternate bPreferCover flag based on personality ratio.
// Writes PreferCover bool to Blackboard so BT decorators can react via observer aborts.
if (NewState == EPS_AI_Behavior_State::Combat)
{
if (!bCombatCoverCycleActive)
{
// Start the cycle — begin with cover if cautious, attack if aggressive
bCombatCoverCycleActive = true;
const float Aggressivity = GetTrait(EPS_AI_Behavior_TraitAxis::Aggressivity);
const float Caution = GetTrait(EPS_AI_Behavior_TraitAxis::Caution);
const float CombatRatio = (Aggressivity + Caution > 0.0f)
? Aggressivity / (Aggressivity + Caution) : 0.5f;
bPreferCover = (CombatRatio <= 0.5f);
CombatCoverTimer = CalculatePhaseDuration(
bPreferCover ? EPS_AI_Behavior_State::TakingCover : EPS_AI_Behavior_State::Combat);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Combat/Cover cycle started: %s for %.1fs"),
*GetOwner()->GetName(), bPreferCover ? TEXT("Cover") : TEXT("Attack"), CombatCoverTimer);
}
else
{
// Tick down the timer (ApplyReaction is called every ~0.5s by the BT service)
CombatCoverTimer -= 0.5f;
if (CombatCoverTimer <= 0.0f)
{
// Switch phase
bPreferCover = !bPreferCover;
CombatCoverTimer = CalculatePhaseDuration(
bPreferCover ? EPS_AI_Behavior_State::TakingCover : EPS_AI_Behavior_State::Combat);
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] Combat/Cover cycle switch: → %s for %.1fs"),
*GetOwner()->GetName(), bPreferCover ? TEXT("Cover") : TEXT("Attack"), CombatCoverTimer);
}
}
// Write to BB so Blackboard-based decorators can observe the change
if (AActor* Owner = GetOwner())
{
if (AAIController* AIC = Cast<AAIController>(Cast<APawn>(Owner) ?
Cast<APawn>(Owner)->GetController() : nullptr))
{
if (UBlackboardComponent* BB = AIC->GetBlackboardComponent())
{
BB->SetValueAsBool(PS_AI_Behavior_BB::PreferCover, bPreferCover);
}
}
}
}
else
{
// Left combat → reset cycle
if (bCombatCoverCycleActive)
{
bCombatCoverCycleActive = false;
bPreferCover = false;
CombatCoverTimer = 0.0f;
if (AActor* Owner = GetOwner())
{
if (AAIController* AIC = Cast<AAIController>(Cast<APawn>(Owner) ?
Cast<APawn>(Owner)->GetController() : nullptr))
{
if (UBlackboardComponent* BB = AIC->GetBlackboardComponent())
{
BB->SetValueAsBool(PS_AI_Behavior_BB::PreferCover, false);
}
}
}
}
}
if (NewState != CurrentState) if (NewState != CurrentState)
{ {
const EPS_AI_Behavior_State OldState = CurrentState; const EPS_AI_Behavior_State OldState = CurrentState;
CurrentState = NewState; // Replicated → OnRep fires on clients CurrentState = NewState; // Replicated → OnRep fires on clients
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] State: %s -> %s (Threat: %.2f)"), if (UPS_AI_Behavior_Statics::IsDebugEnabled(Cast<APawn>(GetOwner())))
{
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] State: %s -> %s (Threat: %.2f)"),
*GetOwner()->GetName(), *GetOwner()->GetName(),
*UEnum::GetValueAsString(OldState), *UEnum::GetValueAsString(OldState),
*UEnum::GetValueAsString(NewState), *UEnum::GetValueAsString(NewState),
PerceivedThreatLevel); PerceivedThreatLevel);
}
HandleStateChanged(OldState, NewState); HandleStateChanged(OldState, NewState);
} }
return CurrentState; return CurrentState;
} }
float UPS_AI_Behavior_PersonalityComponent::CalculatePhaseDuration(EPS_AI_Behavior_State Phase) const
{
const float Aggressivity = GetTrait(EPS_AI_Behavior_TraitAxis::Aggressivity);
const float Caution = GetTrait(EPS_AI_Behavior_TraitAxis::Caution);
const float CycleDuration = Profile ? Profile->CombatCoverCycleDuration : 15.0f;
const float Sum = Aggressivity + Caution;
const float CombatRatio = (Sum > 0.0f) ? Aggressivity / Sum : 0.5f;
float Duration;
if (Phase == EPS_AI_Behavior_State::Combat)
{
Duration = CycleDuration * CombatRatio;
}
else
{
Duration = CycleDuration * (1.0f - CombatRatio);
}
// Clamp minimum 2s, add ±20% jitter
Duration = FMath::Max(Duration, 2.0f);
Duration *= FMath::RandRange(0.8f, 1.2f);
return Duration;
}
void UPS_AI_Behavior_PersonalityComponent::ResetRuntimeState()
{
if (!GetOwner() || !GetOwner()->HasAuthority())
{
return;
}
PerceivedThreatLevel = 0.0f;
CombatCoverTimer = 0.0f;
bCombatCoverCycleActive = false;
bPreferCover = false;
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] PersonalityComponent runtime state reset."),
*GetOwner()->GetName());
}
void UPS_AI_Behavior_PersonalityComponent::ForceState(EPS_AI_Behavior_State NewState) void UPS_AI_Behavior_PersonalityComponent::ForceState(EPS_AI_Behavior_State NewState)
{ {
// Only server can change replicated state
if (!GetOwner() || !GetOwner()->HasAuthority())
{
return;
}
if (NewState != CurrentState) if (NewState != CurrentState)
{ {
const EPS_AI_Behavior_State OldState = CurrentState; const EPS_AI_Behavior_State OldState = CurrentState;
@@ -175,6 +370,112 @@ void UPS_AI_Behavior_PersonalityComponent::HandleStateChanged(
} }
} }
void UPS_AI_Behavior_PersonalityComponent::TickComponent(
float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction)
{
Super::TickComponent(DeltaTime, TickType, ThisTickFunction);
if (bDebug)
{
DrawDebugInfo();
}
}
void UPS_AI_Behavior_PersonalityComponent::DrawDebugInfo() const
{
const AActor* Owner = GetOwner();
if (!Owner) return;
const UWorld* World = GetWorld();
if (!World) return;
// Position: above the NPC's head
const FVector HeadPos = Owner->GetActorLocation() + FVector(0.f, 0.f, 120.f);
// ─── Gather info ────────────────────────────────────────────────────
const FString NPCName = Owner->GetName();
const EPS_AI_Behavior_NPCType NPCType = GetNPCType();
// TeamId + Hostile from AIController
uint8 DebugTeamId = 255;
FString ThreatActorName = TEXT("none");
float ThreatLevel = PerceivedThreatLevel;
bool bHostile = false;
const APawn* Pawn = Cast<APawn>(Owner);
if (Pawn)
{
if (const APS_AI_Behavior_AIController* AIC = Cast<APS_AI_Behavior_AIController>(Pawn->GetController()))
{
DebugTeamId = AIC->GetGenericTeamId().GetId();
// Get threat target name from Blackboard (prefer Pawn name for readability)
if (const UBlackboardComponent* BB = AIC->GetBlackboardComponent())
{
const FString PawnName = BB->GetValueAsString(PS_AI_Behavior_BB::ThreatPawnName);
if (!PawnName.IsEmpty())
{
ThreatActorName = PawnName;
}
else if (AActor* ThreatActor = Cast<AActor>(BB->GetValueAsObject(PS_AI_Behavior_BB::ThreatActor)))
{
ThreatActorName = ThreatActor->GetName();
}
}
}
if (Pawn->Implements<UPS_AI_Behavior_Interface>())
{
bHostile = IPS_AI_Behavior_Interface::Execute_IsBehaviorHostile(const_cast<APawn*>(Pawn));
}
}
// Spline info
FString SplineInfo = TEXT("");
if (const UPS_AI_Behavior_SplineFollowerComponent* Spline =
Owner->FindComponentByClass<UPS_AI_Behavior_SplineFollowerComponent>())
{
if (Spline->bIsFollowing && Spline->CurrentSpline)
{
SplineInfo = FString::Printf(TEXT("\nSpline: %s (%.0f%%)"),
*Spline->CurrentSpline->GetName(), Spline->GetProgress() * 100.f);
}
}
// ─── Color by state ─────────────────────────────────────────────────
FColor TextColor;
switch (CurrentState)
{
case EPS_AI_Behavior_State::Idle: TextColor = FColor::White; break;
case EPS_AI_Behavior_State::Patrol: TextColor = FColor::Green; break;
case EPS_AI_Behavior_State::Alerted: TextColor = FColor::Yellow; break;
case EPS_AI_Behavior_State::Combat: TextColor = FColor::Red; break;
case EPS_AI_Behavior_State::Fleeing: TextColor = FColor::Orange; break;
case EPS_AI_Behavior_State::TakingCover: TextColor = FColor::Cyan; break;
case EPS_AI_Behavior_State::Dead: TextColor = FColor(80, 80, 80); break;
default: TextColor = FColor::White; break;
}
// ─── Build text ─────────────────────────────────────────────────────
// Decode faction from TeamId
const uint8 DebugFaction = PS_AI_Behavior_Team::GetFaction(DebugTeamId);
const FString FactionStr = DebugFaction > 0 ? FString::Printf(TEXT(" F%d"), DebugFaction) : TEXT("");
const FString DebugText = FString::Printf(
TEXT("%s [%s%s] %s\n%s Threat:%.2f → %s%s"),
*NPCName,
*UEnum::GetDisplayValueAsText(NPCType).ToString(),
*FactionStr,
bHostile ? TEXT("HOSTILE") : TEXT(""),
*UEnum::GetDisplayValueAsText(CurrentState).ToString(),
ThreatLevel,
*ThreatActorName,
*SplineInfo);
UKismetSystemLibrary::DrawDebugString(
GetOwner(), HeadPos, DebugText, nullptr, FLinearColor(TextColor), 0.f);
}
EPS_AI_Behavior_NPCType UPS_AI_Behavior_PersonalityComponent::GetNPCType() const EPS_AI_Behavior_NPCType UPS_AI_Behavior_PersonalityComponent::GetNPCType() const
{ {
// Prefer the IPS_AI_Behavior interface on the owning actor // Prefer the IPS_AI_Behavior interface on the owning actor

View File

@@ -7,9 +7,7 @@ UPS_AI_Behavior_PersonalityProfile::UPS_AI_Behavior_PersonalityProfile()
// Initialize all trait axes to a neutral 0.5 // Initialize all trait axes to a neutral 0.5
TraitScores.Add(EPS_AI_Behavior_TraitAxis::Courage, 0.5f); TraitScores.Add(EPS_AI_Behavior_TraitAxis::Courage, 0.5f);
TraitScores.Add(EPS_AI_Behavior_TraitAxis::Aggressivity, 0.5f); TraitScores.Add(EPS_AI_Behavior_TraitAxis::Aggressivity, 0.5f);
TraitScores.Add(EPS_AI_Behavior_TraitAxis::Loyalty, 0.5f);
TraitScores.Add(EPS_AI_Behavior_TraitAxis::Caution, 0.5f); TraitScores.Add(EPS_AI_Behavior_TraitAxis::Caution, 0.5f);
TraitScores.Add(EPS_AI_Behavior_TraitAxis::Discipline, 0.5f);
// Default target priority: Protector first, then Player, then Civilian // Default target priority: Protector first, then Player, then Civilian
TargetPriority.Add(EPS_AI_Behavior_TargetType::Protector); TargetPriority.Add(EPS_AI_Behavior_TargetType::Protector);

View File

@@ -8,6 +8,7 @@
#include "GameFramework/Character.h" #include "GameFramework/Character.h"
#include "GameFramework/CharacterMovementComponent.h" #include "GameFramework/CharacterMovementComponent.h"
#include "Net/UnrealNetwork.h" #include "Net/UnrealNetwork.h"
#include "DrawDebugHelpers.h"
UPS_AI_Behavior_SplineFollowerComponent::UPS_AI_Behavior_SplineFollowerComponent() UPS_AI_Behavior_SplineFollowerComponent::UPS_AI_Behavior_SplineFollowerComponent()
{ {
@@ -57,8 +58,16 @@ bool UPS_AI_Behavior_SplineFollowerComponent::StartFollowingAtDistance(
CurrentDistance = FMath::Clamp(StartDistance, 0.0f, Spline->GetSplineLength()); CurrentDistance = FMath::Clamp(StartDistance, 0.0f, Spline->GetSplineLength());
bMovingForward = bForward; bMovingForward = bForward;
bIsFollowing = true; bIsFollowing = true;
bManuallyStopped = false; // an explicit Start lifts any previous manual stop
LastHandledJunctionIndex = -1; LastHandledJunctionIndex = -1;
UE_LOG(LogPS_AI_Behavior, Warning,
TEXT("[%s] StartFollowingAtDistance: spline='%s' dist=%.0f/%.0f dir=%s (was spline='%s')"),
GetOwner() ? *GetOwner()->GetName() : TEXT("?"),
*Spline->GetName(), CurrentDistance, Spline->GetSplineLength(),
bForward ? TEXT("FWD") : TEXT("BWD"),
OldSpline ? *OldSpline->GetName() : TEXT("none"));
if (OldSpline && OldSpline != Spline) if (OldSpline && OldSpline != Spline)
{ {
OnSplineChanged.Broadcast(OldSpline, Spline); OnSplineChanged.Broadcast(OldSpline, Spline);
@@ -74,18 +83,23 @@ bool UPS_AI_Behavior_SplineFollowerComponent::StartFollowingAtDistance(
void UPS_AI_Behavior_SplineFollowerComponent::StopFollowing() void UPS_AI_Behavior_SplineFollowerComponent::StopFollowing()
{ {
bIsFollowing = false; bIsFollowing = false;
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("[%s] Stopped following spline."), // Firm, manual stop: the Behavior Tree spline tasks will refuse to re-start
// following until StartFollowing()/StartFollowingAtDistance() is called again.
bManuallyStopped = true;
UE_LOG(LogPS_AI_Behavior, Log, TEXT("[%s] StopFollowing: manual stop (BT will not auto-resume)."),
GetOwner() ? *GetOwner()->GetName() : TEXT("?")); GetOwner() ? *GetOwner()->GetName() : TEXT("?"));
} }
void UPS_AI_Behavior_SplineFollowerComponent::PauseFollowing() void UPS_AI_Behavior_SplineFollowerComponent::PauseFollowing()
{ {
// Pause is NOT a manual stop — the BT is allowed to resume it.
bIsFollowing = false; bIsFollowing = false;
} }
void UPS_AI_Behavior_SplineFollowerComponent::ResumeFollowing() void UPS_AI_Behavior_SplineFollowerComponent::ResumeFollowing()
{ {
if (CurrentSpline) // Do not resume over a manual stop — only an explicit Start lifts that.
if (CurrentSpline && !bManuallyStopped)
{ {
bIsFollowing = true; bIsFollowing = true;
} }
@@ -146,82 +160,158 @@ void UPS_AI_Behavior_SplineFollowerComponent::TickComponent(
return; return;
} }
// ─── Advance along spline ─────────────────────────────────────────── // ─── Pure Pursuit: project NPC onto spline, target = lookahead ahead ──
const float Speed = GetEffectiveSpeed(); // This eliminates gap accumulation in turns. The NPC always steers toward
const float Delta = Speed * DeltaTime; // a point that follows the curve naturally at a fixed distance ahead.
if (bMovingForward)
{
CurrentDistance += Delta;
}
else
{
CurrentDistance -= Delta;
}
// ─── End of spline handling ─────────────────────────────────────────
if (CurrentDistance >= SplineLen)
{
if (CurrentSpline->SplineComp && CurrentSpline->SplineComp->IsClosedLoop())
{
CurrentDistance = FMath::Fmod(CurrentDistance, SplineLen);
}
else if (bReverseAtEnd)
{
CurrentDistance = SplineLen - (CurrentDistance - SplineLen);
bMovingForward = false;
}
else
{
CurrentDistance = SplineLen;
bIsFollowing = false;
OnSplineEndReached.Broadcast(CurrentSpline);
return;
}
}
else if (CurrentDistance <= 0.0f)
{
if (CurrentSpline->SplineComp && CurrentSpline->SplineComp->IsClosedLoop())
{
CurrentDistance = SplineLen + CurrentDistance;
}
else if (bReverseAtEnd)
{
CurrentDistance = -CurrentDistance;
bMovingForward = true;
}
else
{
CurrentDistance = 0.0f;
bIsFollowing = false;
OnSplineEndReached.Broadcast(CurrentSpline);
return;
}
}
// ─── Move the pawn ──────────────────────────────────────────────────
const FVector TargetLocation = CurrentSpline->GetWorldLocationAtDistance(CurrentDistance);
const FRotator TargetRotation = CurrentSpline->GetWorldRotationAtDistance(CurrentDistance);
AActor* Owner = GetOwner(); AActor* Owner = GetOwner();
const FVector CurrentLocation = Owner->GetActorLocation(); const FVector CurrentLocation = Owner->GetActorLocation();
const FRotator CurrentRotation = Owner->GetActorRotation(); const FRotator CurrentRotation = Owner->GetActorRotation();
// Use Character movement if available for proper physics/collision // Project NPC position onto the spline to find where we actually are
float ProjectedDistance = 0.0f;
FVector ProjectedPoint;
CurrentSpline->GetClosestPointOnSpline(CurrentLocation, ProjectedDistance, ProjectedPoint);
// Ensure monotonic progress: don't let projection jump backward past current
// (can happen at tight S-curves where two segments are close in world space)
const float ProgressDir = bMovingForward ? 1.0f : -1.0f;
const float ProgressDelta = (ProjectedDistance - CurrentDistance) * ProgressDir;
if (ProgressDelta < -LookaheadDistance)
{
// Projection jumped too far backward — keep current distance and advance normally
float Speed = GetEffectiveSpeed();
ACharacter* TmpChar = Cast<ACharacter>(Owner);
if (TmpChar && TmpChar->GetCharacterMovement())
{
Speed = TmpChar->GetCharacterMovement()->GetMaxSpeed();
}
if (bMovingForward)
{
CurrentDistance += Speed * DeltaTime;
}
else
{
CurrentDistance -= Speed * DeltaTime;
}
}
else
{
CurrentDistance = ProjectedDistance;
}
// ─── Compute target (lookahead) distance ────────────────────────────
float TargetDistance;
if (bMovingForward)
{
TargetDistance = CurrentDistance + LookaheadDistance;
}
else
{
TargetDistance = CurrentDistance - LookaheadDistance;
}
// ─── End of spline handling ─────────────────────────────────────────
if (TargetDistance >= SplineLen)
{
if (CurrentSpline->SplineComp && CurrentSpline->SplineComp->IsClosedLoop())
{
TargetDistance = FMath::Fmod(TargetDistance, SplineLen);
}
else if (bReverseAtEnd)
{
// Clamp target to end, and reverse when NPC is close enough
TargetDistance = SplineLen;
const FVector EndPoint = CurrentSpline->GetWorldLocationAtDistance(SplineLen);
const float GapToEnd = FVector::Dist2D(CurrentLocation, EndPoint);
if (GapToEnd < 80.0f)
{
bMovingForward = false;
TargetDistance = CurrentDistance - LookaheadDistance;
TargetDistance = FMath::Max(TargetDistance, 0.0f);
}
}
else
{
TargetDistance = SplineLen;
const FVector EndPoint = CurrentSpline->GetWorldLocationAtDistance(SplineLen);
const float GapToEnd = FVector::Dist2D(CurrentLocation, EndPoint);
if (GapToEnd < 80.0f)
{
bIsFollowing = false;
OnSplineEndReached.Broadcast(CurrentSpline);
return;
}
}
}
else if (TargetDistance <= 0.0f)
{
if (CurrentSpline->SplineComp && CurrentSpline->SplineComp->IsClosedLoop())
{
TargetDistance = SplineLen + TargetDistance;
}
else if (bReverseAtEnd)
{
TargetDistance = 0.0f;
const FVector StartPoint = CurrentSpline->GetWorldLocationAtDistance(0.0f);
const float GapToStart = FVector::Dist2D(CurrentLocation, StartPoint);
if (GapToStart < 80.0f)
{
bMovingForward = true;
TargetDistance = CurrentDistance + LookaheadDistance;
TargetDistance = FMath::Min(TargetDistance, SplineLen);
}
}
else
{
TargetDistance = 0.0f;
const FVector StartPoint = CurrentSpline->GetWorldLocationAtDistance(0.0f);
const float GapToStart = FVector::Dist2D(CurrentLocation, StartPoint);
if (GapToStart < 80.0f)
{
bIsFollowing = false;
OnSplineEndReached.Broadcast(CurrentSpline);
return;
}
}
}
// ─── Move the pawn ──────────────────────────────────────────────────
const FVector TargetLocation = CurrentSpline->GetWorldLocationAtDistance(TargetDistance);
float Speed = GetEffectiveSpeed();
ACharacter* Character = Cast<ACharacter>(Owner); ACharacter* Character = Cast<ACharacter>(Owner);
if (Character && Character->GetCharacterMovement()) if (Character && Character->GetCharacterMovement())
{ {
// Compute velocity to reach the spline point Speed = Character->GetCharacterMovement()->GetMaxSpeed();
FVector DesiredVelocity = (TargetLocation - CurrentLocation) / FMath::Max(DeltaTime, 0.001f);
// Clamp to avoid teleporting on large frame spikes const FVector ToTarget = TargetLocation - CurrentLocation;
const float MaxVel = Speed * 3.0f; const float GapToTarget = ToTarget.Size2D();
if (DesiredVelocity.SizeSquared() > MaxVel * MaxVel)
FVector MoveDirection;
if (GapToTarget < 1.0f)
{ {
DesiredVelocity = DesiredVelocity.GetSafeNormal() * MaxVel; // On top of the target — use spline tangent direction
MoveDirection = CurrentSpline->GetWorldDirectionAtDistance(CurrentDistance)
* (bMovingForward ? 1.0f : -1.0f);
}
else
{
MoveDirection = ToTarget.GetSafeNormal();
} }
Character->GetCharacterMovement()->RequestDirectMove(DesiredVelocity, /*bForceMaxSpeed=*/false); // Use AddMovementInput — works reliably regardless of AIController movement state
Character->AddMovementInput(MoveDirection, 1.0f);
// Debug: log velocity to verify movement is happening
const FVector Vel = Character->GetVelocity();
if (Vel.Size() < 1.0f)
{
UE_LOG(LogPS_AI_Behavior, Warning,
TEXT("[%s] SplineFollower: AddMovementInput but velocity=%.1f! MaxSpeed=%.0f Gap=%.0f CMC_MovementMode=%d"),
*GetOwner()->GetName(), Vel.Size(), Speed, GapToTarget,
(int32)Character->GetCharacterMovement()->MovementMode.GetValue());
}
} }
else else
{ {
@@ -229,16 +319,57 @@ void UPS_AI_Behavior_SplineFollowerComponent::TickComponent(
Owner->SetActorLocation(TargetLocation); Owner->SetActorLocation(TargetLocation);
} }
// Smooth rotation — flip if going backward // Smooth rotation — face toward the target point (follows curve naturally)
FRotator FinalTargetRot = TargetRotation; const FVector ToTargetDir = (TargetLocation - CurrentLocation).GetSafeNormal2D();
FRotator FinalTargetRot;
if (!ToTargetDir.IsNearlyZero())
{
FinalTargetRot = ToTargetDir.Rotation();
}
else
{
// Fallback to spline tangent
FinalTargetRot = CurrentSpline->GetWorldRotationAtDistance(CurrentDistance);
if (!bMovingForward) if (!bMovingForward)
{ {
FinalTargetRot.Yaw += 180.0f; FinalTargetRot.Yaw += 180.0f;
} }
}
const FRotator SmoothedRot = FMath::RInterpConstantTo( const FRotator SmoothedRot = FMath::RInterpConstantTo(
CurrentRotation, FinalTargetRot, DeltaTime, RotationInterpSpeed); CurrentRotation, FinalTargetRot, DeltaTime, RotationInterpSpeed);
Owner->SetActorRotation(FRotator(0.0f, SmoothedRot.Yaw, 0.0f)); // Only yaw Owner->SetActorRotation(FRotator(0.0f, SmoothedRot.Yaw, 0.0f));
// ─── Debug drawing ─────────────────────────────────────────────────
if (bDebug)
{
const UWorld* World = GetWorld();
if (World)
{
// Projected point on spline (small blue sphere = where NPC actually is on spline)
DrawDebugSphere(World, ProjectedPoint, 10.0f, 6, FColor::Blue, false, -1.0f, 0, 1.5f);
// Lookahead target point (green sphere = where NPC steers toward)
DrawDebugSphere(World, TargetLocation, 15.0f, 8, FColor::Green, false, -1.0f, 0, 2.0f);
// Line from NPC to lookahead target (yellow)
DrawDebugLine(World, CurrentLocation, TargetLocation, FColor::Yellow, false, -1.0f, 0, 1.5f);
// Spline tangent direction at NPC position (cyan arrow)
const FVector Tangent = CurrentSpline->GetWorldDirectionAtDistance(CurrentDistance)
* (bMovingForward ? 1.0f : -1.0f);
DrawDebugDirectionalArrow(World, ProjectedPoint, ProjectedPoint + Tangent * 150.0f,
20.0f, FColor::Cyan, false, -1.0f, 0, 2.0f);
// Info text
const float DebugGap = FVector::Dist2D(CurrentLocation, TargetLocation);
const float SplineGap = FVector::Dist2D(CurrentLocation, ProjectedPoint);
DrawDebugString(World, CurrentLocation + FVector(0, 0, 100.0f),
FString::Printf(TEXT("Look: %.0fcm Off: %.0fcm D: %.0f/%.0f"),
DebugGap, SplineGap, CurrentDistance, SplineLen),
nullptr, FColor::White, -1.0f, true);
}
}
// ─── Junction handling ────────────────────────────────────────────── // ─── Junction handling ──────────────────────────────────────────────
HandleJunctions(); HandleJunctions();
@@ -277,8 +408,31 @@ void UPS_AI_Behavior_SplineFollowerComponent::HandleJunctions()
LastHandledJunctionIndex = i; LastHandledJunctionIndex = i;
OnApproachingJunction.Broadcast(CurrentSpline, i, DistToJunction); OnApproachingJunction.Broadcast(CurrentSpline, i, DistToJunction);
if (bDebug)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Junction detected: idx=%d, dist along=%.0f, dist to=%.0fcm, worldPos=(%.0f,%.0f,%.0f)"),
GetOwner() ? *GetOwner()->GetName() : TEXT("?"),
i, J.DistanceOnThisSpline, DistToJunction,
J.WorldLocation.X, J.WorldLocation.Y, J.WorldLocation.Z);
}
if (bAutoChooseAtJunction) if (bAutoChooseAtJunction)
{ {
// Only switch with a probability — don't always change at every junction
const float SwitchChance = 0.7f; // 70% chance to switch
const float Roll = FMath::FRand();
if (Roll > SwitchChance)
{
if (bDebug)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Junction %d: skipped (roll=%.2f > chance=%.2f)"),
GetOwner() ? *GetOwner()->GetName() : TEXT("?"), i, Roll, SwitchChance);
}
break; // Stay on current spline
}
// Use SplineNetwork subsystem to choose // Use SplineNetwork subsystem to choose
UPS_AI_Behavior_SplineNetwork* Network = UPS_AI_Behavior_SplineNetwork* Network =
GetWorld()->GetSubsystem<UPS_AI_Behavior_SplineNetwork>(); GetWorld()->GetSubsystem<UPS_AI_Behavior_SplineNetwork>();
@@ -301,7 +455,67 @@ void UPS_AI_Behavior_SplineFollowerComponent::HandleJunctions()
if (ChosenSpline && ChosenSpline != CurrentSpline) if (ChosenSpline && ChosenSpline != CurrentSpline)
{ {
SwitchToSpline(ChosenSpline, J.DistanceOnOtherSpline, bMovingForward); // Only switch if the junction points are actually close in world space
const FVector CurrentPos = GetOwner()->GetActorLocation();
const FVector JunctionPos = J.WorldLocation;
const float WorldGap = FVector::Dist(CurrentPos, JunctionPos);
if (WorldGap > JunctionDetectionDistance * 2.0f)
{
if (bDebug)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Junction %d: rejected (worldGap=%.0f > detect=%.0f)"),
GetOwner() ? *GetOwner()->GetName() : TEXT("?"),
i, WorldGap, JunctionDetectionDistance);
}
break; // Too far in world space
}
// Check direction continuity: avoid U-turns
// Compare our current movement direction with the new spline's direction at the junction
const FVector CurrentDir = CurrentSpline->GetWorldDirectionAtDistance(CurrentDistance)
* (bMovingForward ? 1.0f : -1.0f);
const FVector NewDirForward = ChosenSpline->GetWorldDirectionAtDistance(J.DistanceOnOtherSpline);
const FVector NewDirBackward = -NewDirForward;
const float DotForward = FVector::DotProduct(CurrentDir, NewDirForward);
const float DotBackward = FVector::DotProduct(CurrentDir, NewDirBackward);
// Choose the direction on the new spline that best continues our current heading
bool bNewForward;
float BestDot;
if (DotForward >= DotBackward)
{
bNewForward = true;
BestDot = DotForward;
}
else
{
bNewForward = false;
BestDot = DotBackward;
}
// Only switch if the direction change is less than ~135° (dot > -0.7)
// This prevents full U-turns but allows wide turns at crossings
if (BestDot < -0.7f)
{
if (bDebug)
{
UE_LOG(LogPS_AI_Behavior, Log,
TEXT("[%s] Junction %d: rejected U-turn (dot=%.2f)"),
GetOwner() ? *GetOwner()->GetName() : TEXT("?"), i, BestDot);
}
break; // Would cause a near-180° U-turn, skip
}
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] Junction switch: %s -> %s (dot=%.2f, dir=%s, worldGap=%.0fcm)"),
GetOwner() ? *GetOwner()->GetName() : TEXT("?"),
*CurrentSpline->GetName(), *ChosenSpline->GetName(),
BestDot, bNewForward ? TEXT("fwd") : TEXT("bwd"),
FVector::Dist(CurrentPos, JunctionPos));
SwitchToSpline(ChosenSpline, J.DistanceOnOtherSpline, bNewForward);
} }
} }

View File

@@ -19,6 +19,12 @@ void APS_AI_Behavior_SplinePath::BeginPlay()
{ {
Super::BeginPlay(); Super::BeginPlay();
UpdateSplineVisualization(); UpdateSplineVisualization();
// Hide spline debug at runtime — only visible in editor
if (SplineComp)
{
SplineComp->SetDrawDebug(false);
}
} }
bool APS_AI_Behavior_SplinePath::IsAccessibleTo(EPS_AI_Behavior_NPCType NPCType) const bool APS_AI_Behavior_SplinePath::IsAccessibleTo(EPS_AI_Behavior_NPCType NPCType) const
@@ -102,6 +108,12 @@ FRotator APS_AI_Behavior_SplinePath::GetWorldRotationAtDistance(float Distance)
return SplineComp->GetRotationAtDistanceAlongSpline(Distance, ESplineCoordinateSpace::World); return SplineComp->GetRotationAtDistanceAlongSpline(Distance, ESplineCoordinateSpace::World);
} }
FVector APS_AI_Behavior_SplinePath::GetWorldDirectionAtDistance(float Distance) const
{
if (!SplineComp) return FVector::ForwardVector;
return SplineComp->GetDirectionAtDistanceAlongSpline(Distance, ESplineCoordinateSpace::World);
}
void APS_AI_Behavior_SplinePath::UpdateSplineVisualization() void APS_AI_Behavior_SplinePath::UpdateSplineVisualization()
{ {
if (!SplineComp) return; if (!SplineComp) return;

View File

@@ -0,0 +1,80 @@
// Copyright Asterion. All Rights Reserved.
#include "PS_AI_Behavior_Statics.h"
#include "PS_AI_Behavior_PersonalityComponent.h"
#include "Perception/AISense_Hearing.h"
#include "CollisionQueryParams.h"
#include "Engine/World.h"
static TAutoConsoleVariable<int32> CVarBehaviorDebug(
TEXT("ps.ai.Behavior.Debug"),
-1,
TEXT("Global debug override for PS_AI_Behavior. -1=use per-NPC bDebug property, 0=force off, 1=force on."),
ECVF_Default);
void UPS_AI_Behavior_Statics::ReportGunfire(UObject* WorldContext, FVector Location,
AActor* Shooter, bool bIsEnemyFire, float Loudness, float MaxRange)
{
UWorld* World = GEngine ? GEngine->GetWorldFromContextObject(WorldContext, EGetWorldErrorMode::LogAndReturnNull) : nullptr;
if (!World)
{
UE_LOG(LogPS_AI_Behavior, Warning, TEXT("ReportGunfire: invalid WorldContext."));
return;
}
const FName Tag = bIsEnemyFire ? PS_AI_Behavior_Tags::EnemyFire : PS_AI_Behavior_Tags::PlayerFire;
UAISense_Hearing::ReportNoiseEvent(
World,
Location,
Loudness,
Shooter,
MaxRange,
Tag);
UE_LOG(LogPS_AI_Behavior, Verbose, TEXT("ReportGunfire: at %s by '%s' (tag=%s, loudness=%.1f)"),
*Location.ToString(),
Shooter ? *Shooter->GetName() : TEXT("null"),
*Tag.ToString(),
Loudness);
}
bool UPS_AI_Behavior_Statics::HasLineOfSight(const UWorld* World, const AActor* Source,
const AActor* Target, float EyeHeightOffset)
{
if (!World || !Source || !Target) return false;
const FVector TraceStart = Source->GetActorLocation() + FVector(0, 0, EyeHeightOffset);
const FVector TraceEnd = Target->GetActorLocation();
FHitResult Hit;
FCollisionQueryParams Params(SCENE_QUERY_STAT(BehaviorLOS), true);
Params.AddIgnoredActor(Source);
Params.AddIgnoredActor(Target);
// Also ignore actors up the attachment chain (e.g. AimTarget → ChildActor → Character capsule)
// This handles cases where Owner/Instigator aren't set but the actor is physically attached.
for (const AActor* Cur = Target->GetAttachParentActor(); Cur; Cur = Cur->GetAttachParentActor())
{
Params.AddIgnoredActor(Cur);
}
for (const AActor* Cur = Source->GetAttachParentActor(); Cur; Cur = Cur->GetAttachParentActor())
{
Params.AddIgnoredActor(Cur);
}
return !World->LineTraceSingleByChannel(Hit, TraceStart, TraceEnd, ECC_Visibility, Params);
}
bool UPS_AI_Behavior_Statics::IsDebugEnabled(const APawn* Pawn)
{
// CVar override: 0=off, 1=on, -1=use property
const int32 CVar = CVarBehaviorDebug.GetValueOnGameThread();
if (CVar == 0) return false;
if (CVar == 1) return true;
// Fall through to per-NPC property
if (!Pawn) return false;
const auto* Personality = Pawn->FindComponentByClass<UPS_AI_Behavior_PersonalityComponent>();
return Personality && Personality->bDebug;
}

View File

@@ -0,0 +1,13 @@
// Copyright Asterion. All Rights Reserved.
#include "PS_AI_Behavior_TeamComponent.h"
UPS_AI_Behavior_TeamComponent::UPS_AI_Behavior_TeamComponent()
{
PrimaryComponentTick.bCanEverTick = false;
}
uint8 UPS_AI_Behavior_TeamComponent::GetTeamId() const
{
return PS_AI_Behavior_Team::MakeTeamId(Role, Faction);
}

View File

@@ -0,0 +1,29 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "BehaviorTree/BTDecorator.h"
#include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_BTDecorator_CheckCombatType.generated.h"
/**
* BT Decorator: Checks the Pawn's combat type via IPS_AI_Behavior_Interface.
* Use to route: ranged NPCs → cover-shoot cycle, melee NPCs → rush attack.
*/
UCLASS(meta = (DisplayName = "PS AI: Check Combat Type"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTDecorator_CheckCombatType : public UBTDecorator
{
GENERATED_BODY()
public:
UPS_AI_Behavior_BTDecorator_CheckCombatType();
/** The combat type this decorator requires to pass. */
UPROPERTY(EditAnywhere, Category = "Combat Type")
EPS_AI_Behavior_CombatType RequiredType = EPS_AI_Behavior_CombatType::Ranged;
protected:
virtual bool CalculateRawConditionValue(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) const override;
virtual FString GetStaticDescription() const override;
};

View File

@@ -0,0 +1,37 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "BehaviorTree/BTDecorator.h"
#include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_BTDecorator_IsCoverNeeded.generated.h"
/**
* BT Decorator: Checks whether the current threat target warrants taking cover.
*
* Reads the ThreatActor from Blackboard and inspects its NPCType.
* Cover is considered necessary against armed/dangerous targets (Protector, Enemy)
* but not against unarmed targets (Civilian).
*
* Place on the cover-shoot sequence so that enemies skip cover when chasing civilians.
*/
UCLASS(meta = (DisplayName = "PS AI: Is Cover Needed"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTDecorator_IsCoverNeeded : public UBTDecorator
{
GENERATED_BODY()
public:
UPS_AI_Behavior_BTDecorator_IsCoverNeeded();
/**
* NPC types that are considered dangerous enough to require cover.
* If the threat's NPCType is NOT in this set, the decorator fails → skip cover.
*/
UPROPERTY(EditAnywhere, Category = "Cover", meta = (Bitmask, BitmaskEnum = "/Script/PS_AI_Behavior.EPS_AI_Behavior_NPCType"))
TArray<EPS_AI_Behavior_NPCType> DangerousTargetTypes;
protected:
virtual bool CalculateRawConditionValue(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) const override;
virtual FString GetStaticDescription() const override;
};

View File

@@ -0,0 +1,33 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "BehaviorTree/BTService.h"
#include "PS_AI_Behavior_BTService_UpdateGaze.generated.h"
/**
* BT Service: Updates the NPC's gaze target based on behavior state and proximity.
*
* Bridges PS_AI_Behavior to PS_AI_ConvAgent's GazeComponent via runtime reflection.
* No compile-time dependency on PS_AI_ConvAgent — silently skips if not present.
*
* Priority order:
* 1. Combat/Alerted → look at ThreatActor (head + eyes + body)
* 2. Conversation active → skip (managed by ConvAgent)
* 3. Proximity → closest perceived actor within radius (head + eyes only)
*
* Place on the root Selector alongside UpdateThreat and EvaluateReaction.
*/
UCLASS(meta = (DisplayName = "PS AI: Update Gaze"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTService_UpdateGaze : public UBTService
{
GENERATED_BODY()
public:
UPS_AI_Behavior_BTService_UpdateGaze();
protected:
virtual void TickNode(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override;
virtual FString GetStaticDescription() const override;
};

View File

@@ -22,7 +22,26 @@ class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTService_UpdateThreat : public UBTServ
public: public:
UPS_AI_Behavior_BTService_UpdateThreat(); UPS_AI_Behavior_BTService_UpdateThreat();
/** Seconds without LOS before writing LastKnownTargetPosition and starting investigation. */
UPROPERTY(EditAnywhere, Category = "Threat|LOS", meta = (ClampMin = "5.0"))
float LOSLostTimeout = 20.0f;
/** Eye height offset for LOS traces (cm above pawn origin). */
UPROPERTY(EditAnywhere, Category = "Threat|LOS", meta = (ClampMin = "50.0", ClampMax = "200.0"))
float EyeHeightOffset = 150.0f;
protected: protected:
virtual void TickNode(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override; virtual void TickNode(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override;
virtual FString GetStaticDescription() const override; virtual FString GetStaticDescription() const override;
private:
struct FUpdateThreatMemory
{
float TimeSinceLOS = 0.0f;
FVector LastVisiblePosition = FVector::ZeroVector;
bool bHadLOS = true;
bool bInvestigating = false;
};
virtual uint16 GetInstanceMemorySize() const override { return sizeof(FUpdateThreatMemory); }
}; };

View File

@@ -4,12 +4,27 @@
#include "CoreMinimal.h" #include "CoreMinimal.h"
#include "BehaviorTree/BTTaskNode.h" #include "BehaviorTree/BTTaskNode.h"
#include "EnvironmentQuery/EnvQueryTypes.h"
#include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_BTTask_Attack.generated.h" #include "PS_AI_Behavior_BTTask_Attack.generated.h"
class UEnvQuery;
/** /**
* BT Task: Move to and attack the threat actor. * BT Task: Move toward the threat actor and delegate combat to the Pawn.
* If out of range, moves toward the target. If in range, executes attack via CombatComponent. *
* Succeeds after one attack, fails if target is lost or unreachable. * Queries IPS_AI_Behavior_Interface for CombatType (Melee/Ranged).
* Attack ranges come from PersonalityProfile (MinAttackRange/MaxAttackRange),
* with fallback to AttackMoveRadius if no profile is available.
* - Melee: rush toward target, continuous pursuit within MinRange.
* - Ranged: maintain distance — back away if closer than MinRange, advance if farther than MaxRange.
*
* Calls IPS_AI_Behavior_Interface::BehaviorStartAttack() on enter and
* BehaviorStopAttack() on abort. The Pawn handles the actual combat
* (aim, fire, melee, etc.) via its own systems.
*
* Stays InProgress permanently — the Decorator Observer Aborts pulls it out
* when the BehaviorState changes away from Combat.
*/ */
UCLASS(meta = (DisplayName = "PS AI: Attack")) UCLASS(meta = (DisplayName = "PS AI: Attack"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTTask_Attack : public UBTTaskNode class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTTask_Attack : public UBTTaskNode
@@ -19,17 +34,62 @@ class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTTask_Attack : public UBTTaskNode
public: public:
UPS_AI_Behavior_BTTask_Attack(); UPS_AI_Behavior_BTTask_Attack();
/** Fallback attack range (cm) used when PersonalityProfile is unavailable. */
UPROPERTY(EditAnywhere, Category = "Attack", meta = (ClampMin = "50.0"))
float AttackMoveRadius = 300.0f;
/** Cooldown between reposition attempts (seconds). Prevents constant re-pathing. */
UPROPERTY(EditAnywhere, Category = "Attack", meta = (ClampMin = "0.5", ClampMax = "5.0"))
float RepositionCooldown = 1.5f;
/** Eye height offset for line-of-sight traces (cm above actor origin). */
UPROPERTY(EditAnywhere, Category = "Attack|LOS", meta = (ClampMin = "50.0", ClampMax = "200.0"))
float EyeHeightOffset = 150.0f;
/** How often to check LOS (seconds). Avoids per-frame traces. */
UPROPERTY(EditAnywhere, Category = "Attack|LOS", meta = (ClampMin = "0.1", ClampMax = "1.0"))
float LOSCheckInterval = 0.2f;
/**
* EQS query asset for finding a firing position with clear LOS.
* Compose in editor: OnCircle generator + LineOfSight filter + Distance tests.
* If null, fallback: advance directly toward the target.
*/
UPROPERTY(EditAnywhere, Category = "Attack|LOS")
TObjectPtr<UEnvQuery> FiringPositionQuery;
protected: protected:
virtual EBTNodeResult::Type ExecuteTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override; virtual EBTNodeResult::Type ExecuteTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override;
virtual void TickTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override; virtual void TickTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override;
virtual EBTNodeResult::Type AbortTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override; virtual EBTNodeResult::Type AbortTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override;
virtual void OnTaskFinished(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, EBTNodeResult::Type TaskResult) override;
virtual FString GetStaticDescription() const override; virtual FString GetStaticDescription() const override;
private: private:
struct FAttackMemory struct FAttackMemory
{ {
bool bMovingToTarget = false; bool bMovingToTarget = false;
bool bAttacking = false; // true when BehaviorStartAttack is active
bool bInRange = false; // true when within MaxRange of target
bool bHasLOS = true; // true when line-of-sight to target is clear
bool bSeekingFiringPos = false; // true when moving to a flanking/firing position
EPS_AI_Behavior_CombatType CombatType = EPS_AI_Behavior_CombatType::Melee;
float MinRange = 100.0f; // backs away if closer
float MaxRange = 300.0f; // advances if farther
float RepositionTimer = 0.0f;
float LOSCheckTimer = 0.0f; // cooldown for LOS trace checks
bool bEQSQueryRunning = false; // true while an EQS query is in flight
}; };
virtual uint16 GetInstanceMemorySize() const override { return sizeof(FAttackMemory); } virtual uint16 GetInstanceMemorySize() const override { return sizeof(FAttackMemory); }
/** Run the EQS firing position query. Falls back to advancing toward target if no query asset. */
void RunFiringPositionQuery(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APawn* Pawn, AActor* Target);
/** Callback when the EQS query completes. */
void OnFiringPositionQueryFinished(TSharedPtr<FEnvQueryResult> Result,
UBehaviorTreeComponent* OwnerComp, uint8* NodeMemory,
TWeakObjectPtr<AActor> WeakTarget);
}; };

View File

@@ -0,0 +1,150 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "BehaviorTree/BTTaskNode.h"
#include "EnvironmentQuery/EnvQueryTypes.h"
#include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_BTTask_CoverShootCycle.generated.h"
class APS_AI_Behavior_AIController;
class APS_AI_Behavior_CoverPoint;
class UEnvQuery;
/**
* BT Task: Cover-shoot cycle for ranged combat.
*
* State machine:
* Engaging → AtCover (crouch) → [EQS] → MovingToFire → Peeking (shoot)
* → ReturningToCover → AtCover ... → Advancing → AtCover ...
*
* If FiringPositionQuery is set, the NPC physically moves between the cover
* position (protected) and a nearby firing position (with LOS to threat).
* If null, the NPC shoots from cover (legacy behavior: stand up and fire in place).
*
* Personality traits modulate timing:
* - Aggressivity → shorter cover duration, advances sooner
* - Caution → longer cover duration, shorter peek
* - Courage < 0.3 → never advances (stays in cover)
*
* Writes CombatSubState to BB for animation sync.
* Stays InProgress — Decorator Observer Aborts pulls out on state change.
*
* Prerequisites: BTTask_FindCover must run first to write CoverLocation/CoverPoint to BB.
*/
UCLASS(meta = (DisplayName = "PS AI: Cover Shoot Cycle"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTTask_CoverShootCycle : public UBTTaskNode
{
GENERATED_BODY()
public:
UPS_AI_Behavior_BTTask_CoverShootCycle();
// ─── Timing ─────────────────────────────────────────────────────────
/** Base minimum time (seconds) spent peeking/shooting. Modulated at runtime: Aggressivity increases, Caution decreases. */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Timing", meta = (ClampMin = "0.5"))
float PeekDurationMin = 2.0f;
/** Base maximum time (seconds) spent peeking/shooting. Modulated at runtime: Aggressivity increases, Caution decreases. */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Timing", meta = (ClampMin = "0.5"))
float PeekDurationMax = 5.0f;
/** Base minimum time (seconds) spent ducked behind cover. Modulated at runtime: Caution increases, Aggressivity decreases. */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Timing", meta = (ClampMin = "0.5"))
float CoverDurationMin = 1.0f;
/** Base maximum time (seconds) spent ducked behind cover. Modulated at runtime: Caution increases, Aggressivity decreases. */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Timing", meta = (ClampMin = "0.5"))
float CoverDurationMax = 3.0f;
// ─── Firing Position ────────────────────────────────────────────────
/**
* Optional EQS query to find a firing position near the cover.
* Should return positions with LOS to threat, close to CoverLocation.
* Use OnCircle generator around PS AI: Cover Location + LineOfSight filter + Distance score.
* If null, the NPC shoots from cover (stand up in place — legacy fallback).
*/
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Firing Position")
TObjectPtr<UEnvQuery> FiringPositionQuery;
// ─── Advancement ────────────────────────────────────────────────────
/** Number of peek/duck cycles before attempting to advance to a closer cover point. Modulated by Aggressivity. */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Advancement", meta = (ClampMin = "1", ClampMax = "10"))
int32 MaxCyclesBeforeAdvance = 3;
/** Search radius for the next cover point when advancing (cm). */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Advancement", meta = (ClampMin = "200.0"))
float AdvanceSearchRadius = 1200.0f;
/** Advancement bias when searching for the next cover (0=neutral, 1=strongly toward threat). */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Advancement", meta = (ClampMin = "0.0", ClampMax = "1.0"))
float AdvancementBias = 0.7f;
/** Cover point type to search when advancing. */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Advancement")
EPS_AI_Behavior_CoverPointType CoverPointType = EPS_AI_Behavior_CoverPointType::Cover;
// ─── Debug ─────────────────────────────────────────────────────────
/** Draw debug info: sub-state label, LOS line, cycle counter, firing position. */
UPROPERTY(EditAnywhere, Category = "Cover Shoot|Debug")
bool bDebugDraw = false;
protected:
virtual EBTNodeResult::Type ExecuteTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override;
virtual void TickTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override;
virtual EBTNodeResult::Type AbortTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override;
virtual void OnTaskFinished(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, EBTNodeResult::Type TaskResult) override;
virtual FString GetStaticDescription() const override;
private:
struct FCoverShootMemory
{
EPS_AI_Behavior_CombatSubState SubState = EPS_AI_Behavior_CombatSubState::Engaging;
float Timer = 0.0f;
float PhaseDuration = 0.0f;
int32 CycleCount = 0;
bool bMoveRequested = false;
float LOSCheckTimer = 0.0f;
// Firing position (found via EQS)
FVector FiringPosition = FVector::ZeroVector;
bool bHasFiringPosition = false;
bool bEQSRunning = false;
float MoveTimeout = 0.0f;
// Effective durations (modulated by personality)
float EffPeekMin = 2.0f;
float EffPeekMax = 5.0f;
float EffCoverMin = 1.0f;
float EffCoverMax = 3.0f;
int32 EffMaxCycles = 3;
bool bCanAdvance = true;
};
virtual uint16 GetInstanceMemorySize() const override { return sizeof(FCoverShootMemory); }
/** Find an advancing cover point closer to the threat. */
APS_AI_Behavior_CoverPoint* FindAdvancingCover(
const UWorld* World, const FVector& NpcLoc, const FVector& ThreatLoc,
EPS_AI_Behavior_NPCType NPCType, float& OutScore) const;
/** Run EQS to find the best firing position near the cover. */
void RunFiringPositionQuery(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, APawn* Pawn);
/** Callback when the firing position EQS query completes. */
void OnFiringPositionQueryFinished(TSharedPtr<FEnvQueryResult> Result,
UBehaviorTreeComponent* OwnerComp, uint8* NodeMemory);
/** Transition to peeking: move to firing position or shoot in place. */
void StartPeeking(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APS_AI_Behavior_AIController* AIC, APawn* Pawn, AActor* Target);
/** Transition back to cover after shooting. */
void ReturnToCover(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APS_AI_Behavior_AIController* AIC, APawn* Pawn);
};

View File

@@ -4,10 +4,12 @@
#include "CoreMinimal.h" #include "CoreMinimal.h"
#include "BehaviorTree/BTTaskNode.h" #include "BehaviorTree/BTTaskNode.h"
#include "EnvironmentQuery/EnvQueryTypes.h"
#include "PS_AI_Behavior_Definitions.h" #include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_BTTask_FindCover.generated.h" #include "PS_AI_Behavior_BTTask_FindCover.generated.h"
class APS_AI_Behavior_CoverPoint; class APS_AI_Behavior_CoverPoint;
class UEnvQuery;
/** /**
* BT Task: Find a cover position and navigate to it. * BT Task: Find a cover position and navigate to it.
@@ -49,8 +51,9 @@ public:
EPS_AI_Behavior_CoverPointType CoverPointType = EPS_AI_Behavior_CoverPointType::Cover; EPS_AI_Behavior_CoverPointType CoverPointType = EPS_AI_Behavior_CoverPointType::Cover;
/** /**
* Bonus score added to manual CoverPoints over procedural candidates. * Score bonus added to manual CoverPoints (additive, 0-1 range).
* Higher = manual points are strongly preferred. * A manual point with score 0.5 + bonus 0.3 = 0.8 total.
* Higher = manual points strongly preferred over procedural candidates.
*/ */
UPROPERTY(EditAnywhere, Category = "Cover|Manual Points", meta = (ClampMin = "0.0", ClampMax = "1.0")) UPROPERTY(EditAnywhere, Category = "Cover|Manual Points", meta = (ClampMin = "0.0", ClampMax = "1.0"))
float ManualPointBonus = 0.3f; float ManualPointBonus = 0.3f;
@@ -62,6 +65,33 @@ public:
UPROPERTY(EditAnywhere, Category = "Cover|Manual Points") UPROPERTY(EditAnywhere, Category = "Cover|Manual Points")
bool bUseManualPointsOnly = false; bool bUseManualPointsOnly = false;
/**
* Bias toward covers that advance toward the threat (0 = none, 1 = strong).
* Used for cover-to-cover progression during combat.
* Covers closer to the threat than the NPC's current position score higher.
*/
UPROPERTY(EditAnywhere, Category = "Cover|Advancement", meta = (ClampMin = "0.0", ClampMax = "1.0"))
float AdvancementBias = 0.0f;
/**
* Optional EQS query to refine the exact cover position around a selected CoverPoint.
* The query runs centered on the chosen CoverPoint and picks the best nearby spot.
* Use OnCircle generator (small radius ~200cm) + CoverQuality test + Distance tests.
* If null, the NPC goes directly to the CoverPoint's location (no refinement).
*/
UPROPERTY(EditAnywhere, Category = "Cover|EQS Refinement")
TObjectPtr<UEnvQuery> RefinementQuery;
// ─── Debug ─────────────────────────────────────────────────────────
/** Draw debug spheres showing cover candidates and their scores at runtime. */
UPROPERTY(EditAnywhere, Category = "Cover|Debug")
bool bDebugDraw = false;
/** Radius around the CoverPoint for EQS refinement search (cm). */
UPROPERTY(EditAnywhere, Category = "Cover|EQS Refinement", meta = (ClampMin = "100.0", ClampMax = "500.0"))
float RefinementRadius = 200.0f;
protected: protected:
virtual EBTNodeResult::Type ExecuteTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override; virtual EBTNodeResult::Type ExecuteTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override;
virtual void TickTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override; virtual void TickTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override;
@@ -72,6 +102,7 @@ private:
struct FCoverMemory struct FCoverMemory
{ {
bool bMoveRequested = false; bool bMoveRequested = false;
bool bEQSRunning = false;
}; };
virtual uint16 GetInstanceMemorySize() const override { return sizeof(FCoverMemory); } virtual uint16 GetInstanceMemorySize() const override { return sizeof(FCoverMemory); }
@@ -90,4 +121,13 @@ private:
APS_AI_Behavior_CoverPoint* FindBestManualCoverPoint( APS_AI_Behavior_CoverPoint* FindBestManualCoverPoint(
const UWorld* World, const FVector& NpcLoc, const FVector& ThreatLoc, const UWorld* World, const FVector& NpcLoc, const FVector& ThreatLoc,
EPS_AI_Behavior_NPCType NPCType, float& OutScore) const; EPS_AI_Behavior_NPCType NPCType, float& OutScore) const;
/** Run EQS refinement query around the chosen CoverPoint. */
void RunRefinementQuery(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory,
APawn* Pawn, const FVector& CoverCenter);
/** Callback when the refinement EQS query completes. */
void OnRefinementQueryFinished(TSharedPtr<FEnvQueryResult> Result,
UBehaviorTreeComponent* OwnerComp, uint8* NodeMemory,
FVector OriginalCoverPos);
}; };

View File

@@ -7,11 +7,14 @@
#include "PS_AI_Behavior_BTTask_Patrol.generated.h" #include "PS_AI_Behavior_BTTask_Patrol.generated.h"
/** /**
* BT Task: Navigate to the next patrol waypoint. * BT Task: Patrol around the NPC's home location.
* Reads PatrolIndex from Blackboard, navigates to the corresponding point in
* the AIController's PatrolPoints array, then increments the index (cyclic).
* *
* Optional random wait at each waypoint. * Two modes (automatic):
* - If AIController has PatrolPoints → cycles through them.
* - If no PatrolPoints → picks a random NavMesh point within PatrolRadius
* around the HomeLocation (from Blackboard).
*
* Waits a random time at each waypoint before moving to the next.
*/ */
UCLASS(meta = (DisplayName = "PS AI: Patrol")) UCLASS(meta = (DisplayName = "PS AI: Patrol"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTTask_Patrol : public UBTTaskNode class PS_AI_BEHAVIOR_API UPS_AI_Behavior_BTTask_Patrol : public UBTTaskNode
@@ -33,6 +36,20 @@ public:
UPROPERTY(EditAnywhere, Category = "Patrol", meta = (ClampMin = "0.0")) UPROPERTY(EditAnywhere, Category = "Patrol", meta = (ClampMin = "0.0"))
float MaxWaitTime = 4.0f; float MaxWaitTime = 4.0f;
/**
* Radius around HomeLocation to pick random patrol points (cm).
* Only used when no manual PatrolPoints are defined.
*/
UPROPERTY(EditAnywhere, Category = "Patrol|Auto", meta = (ClampMin = "200.0"))
float PatrolRadius = 1500.0f;
/**
* Minimum distance from current position for a random patrol point (cm).
* Prevents picking a point right next to the NPC.
*/
UPROPERTY(EditAnywhere, Category = "Patrol|Auto", meta = (ClampMin = "100.0"))
float MinPatrolDistance = 300.0f;
protected: protected:
virtual EBTNodeResult::Type ExecuteTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override; virtual EBTNodeResult::Type ExecuteTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory) override;
virtual void TickTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override; virtual void TickTask(UBehaviorTreeComponent& OwnerComp, uint8* NodeMemory, float DeltaSeconds) override;
@@ -40,7 +57,6 @@ protected:
virtual FString GetStaticDescription() const override; virtual FString GetStaticDescription() const override;
private: private:
/** Per-instance memory. */
struct FPatrolMemory struct FPatrolMemory
{ {
float WaitRemaining = 0.0f; float WaitRemaining = 0.0f;
@@ -49,4 +65,11 @@ private:
}; };
virtual uint16 GetInstanceMemorySize() const override { return sizeof(FPatrolMemory); } virtual uint16 GetInstanceMemorySize() const override { return sizeof(FPatrolMemory); }
/**
* Find a random navigable point within PatrolRadius of HomeLocation.
* Ensures the point is at least MinPatrolDistance from the NPC.
*/
bool FindRandomPatrolPoint(const UWorld* World, const FVector& HomeLoc,
const FVector& CurrentLoc, FVector& OutPoint) const;
}; };

View File

@@ -0,0 +1,22 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "EnvironmentQuery/EnvQueryContext.h"
#include "PS_AI_Behavior_EQSContext_CoverLocation.generated.h"
/**
* EQS Context: Returns the current CoverLocation from the Blackboard.
* Use as the "Generate Around" context in EQS queries (e.g., OnCircle generator)
* to generate refinement candidates around the chosen CoverPoint instead of around the Querier.
*/
UCLASS(meta = (DisplayName = "PS AI: Cover Location"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_EQSContext_CoverLocation : public UEnvQueryContext
{
GENERATED_BODY()
public:
virtual void ProvideContext(FEnvQueryInstance& QueryInstance,
FEnvQueryContextData& ContextData) const override;
};

View File

@@ -22,16 +22,35 @@ public:
UPS_AI_Behavior_EQSTest_CoverQuality(); UPS_AI_Behavior_EQSTest_CoverQuality();
/** Number of raycasts from candidate to threat at different heights. */ /** Number of raycasts from candidate to threat at different heights. */
UPROPERTY(EditDefaultsOnly, Category = "Cover", meta = (ClampMin = "1", ClampMax = "5")) UPROPERTY(EditAnywhere, Category = "Cover", meta = (ClampMin = "1", ClampMax = "5"))
int32 NumTraceHeights = 3; int32 NumTraceHeights = 3;
/** Minimum height for the lowest trace (cm above ground). */ /** Minimum height for the lowest trace (cm relative to item — items are at navmesh level). */
UPROPERTY(EditDefaultsOnly, Category = "Cover", meta = (ClampMin = "10.0")) UPROPERTY(EditAnywhere, Category = "Cover")
float MinTraceHeight = 50.0f; float MinTraceHeight = 0.0f;
/** Maximum height for the highest trace (cm above ground). */ /** Maximum height for the highest trace (cm relative to item — 70 ≈ knee-to-waist from navmesh). */
UPROPERTY(EditDefaultsOnly, Category = "Cover", meta = (ClampMin = "50.0")) UPROPERTY(EditAnywhere, Category = "Cover")
float MaxTraceHeight = 180.0f; float MaxTraceHeight = 70.0f;
/**
* Maximum distance (cm) between candidate and the blocking hit point for it to count as cover.
* Hits farther than this are likely distant obstacles, not actual nearby cover.
*/
UPROPERTY(EditAnywhere, Category = "Cover", meta = (ClampMin = "50.0"))
float MaxNearbyHitDist = 300.0f;
/**
* Lateral offset (cm) for side traces. Traces are fired from center, left, and right.
* Helps evaluate cover around narrow objects like poles/pillars.
* 0 = center trace only, 30 = adds traces at ±30cm to each side.
*/
UPROPERTY(EditAnywhere, Category = "Cover", meta = (ClampMin = "0.0", ClampMax = "100.0"))
float LateralSpread = 30.0f;
/** Draw debug boxes for each candidate with color-coded score. */
UPROPERTY(EditAnywhere, Category = "Debug")
bool bDrawDebug = false;
protected: protected:
virtual void RunTest(FEnvQueryInstance& QueryInstance) const override; virtual void RunTest(FEnvQueryInstance& QueryInstance) const override;

View File

@@ -0,0 +1,50 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "EnvironmentQuery/EnvQueryTest.h"
#include "PS_AI_Behavior_EQSTest_LineOfSight.generated.h"
/**
* EQS Test: Evaluates whether a candidate position has clear line-of-sight to the threat.
* Inverse of EQSTest_CoverQuality — high score means the candidate CAN see the threat.
*
* Use with EQSContext_Threat as the context for the threat location.
* Combine with a ring/grid generator and distance tests to find firing positions.
*
* Score: 1.0 = clear LOS, 0.0 = fully blocked.
*/
UCLASS(meta = (DisplayName = "PS AI: Line of Sight"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_EQSTest_LineOfSight : public UEnvQueryTest
{
GENERATED_BODY()
public:
UPS_AI_Behavior_EQSTest_LineOfSight();
/** Height offset for the trace origin at the candidate position (cm above ground). Approximates NPC eye height. */
UPROPERTY(EditDefaultsOnly, Category = "LOS", meta = (ClampMin = "10.0"))
float TraceHeight = 150.0f;
/** Height offset for the trace end at the threat position (cm above ground). Approximates target center mass. */
UPROPERTY(EditDefaultsOnly, Category = "LOS", meta = (ClampMin = "0.0"))
float TargetHeightOffset = 100.0f;
/**
* Lateral offset (cm) for side traces perpendicular to threat direction.
* 0 = center trace only, 30 = adds traces at ±30cm.
* Requires wider clear LOS, not just a narrow slit past cover edge.
*/
UPROPERTY(EditAnywhere, Category = "LOS", meta = (ClampMin = "0.0", ClampMax = "100.0"))
float LateralSpread = 0.0f;
/** Draw debug spheres and LOS lines for each candidate. */
UPROPERTY(EditAnywhere, Category = "Debug")
bool bDrawDebug = false;
protected:
virtual void RunTest(FEnvQueryInstance& QueryInstance) const override;
virtual FText GetDescriptionTitle() const override;
virtual FText GetDescriptionDetails() const override;
};

View File

@@ -33,22 +33,22 @@ public:
/** /**
* Team ID — determines perception affiliation (Enemy/Friendly/Neutral). * Team ID — determines perception affiliation (Enemy/Friendly/Neutral).
* Auto-assigned from NPCType at possession if left at 255 (NoTeam): * Auto-assigned from NPCType + Faction at possession via MakeTeamId():
* - Civilian = Team 1 * - Encoding: high nibble = NPCType, low nibble = Faction
* - Enemy = Team 2 * - Civilian F0 = 0x00, Enemy F0 = 0x10, Protector F0 = 0x20
* - Neutral = 255 (NoTeam → perceived as Neutral by everyone) * - Same NPCType + same Faction → Friendly
* * - Same NPCType + different Faction → Hostile (rival gangs)
* Two NPCs with the SAME Team ID → Friendly (ignored by perception). * - Civilian ↔ Protector → always Friendly
* Two NPCs with DIFFERENT Team IDs → Enemy (detected by perception). * - Everything else → Hostile
* A NPC with Team ID 255 → Neutral to everyone.
* *
* Disguised enemies (hostile=false) use DisguisedTeamId (0x01).
* You can override this in Blueprint or per-instance in the editor. * You can override this in Blueprint or per-instance in the editor.
*/ */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Team") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Team")
uint8 TeamId = FGenericTeamId::NoTeam; uint8 TeamId = FGenericTeamId::NoTeam;
/** Set the team ID at runtime. */ /** Set the team ID at runtime. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Team") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Team")
void SetTeamId(uint8 NewTeamId); void SetTeamId(uint8 NewTeamId);
// ─── IGenericTeamAgentInterface (inherited from AAIController) ──── // ─── IGenericTeamAgentInterface (inherited from AAIController) ────
@@ -71,24 +71,85 @@ public:
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Patrol") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Patrol")
TArray<FVector> PatrolPoints; TArray<FVector> PatrolPoints;
// ─── Gaze Configuration ────────────────────────────────────────────
/** Detection radius for proximity gaze (cm). NPC glances at nearby actors within this range. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_Behavior|Gaze", meta = (ClampMin = "100.0", ClampMax = "1500.0"))
float GazeProximityRadius = 400.0f;
/** Maximum duration (seconds) the NPC looks at a proximity target before releasing. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_Behavior|Gaze", meta = (ClampMin = "0.5", ClampMax = "30.0"))
float GazeMaxDuration = 3.0f;
/** Cooldown (seconds) before re-targeting the same actor after releasing. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_Behavior|Gaze", meta = (ClampMin = "0.0", ClampMax = "30.0"))
float GazeCooldown = 5.0f;
/** Enable/disable proximity gaze (NPC glances at nearby actors in Idle/Patrol). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_Behavior|Gaze")
bool bEnableProximityGaze = true;
// ─── Component Access ─────────────────────────────────────────────── // ─── Component Access ───────────────────────────────────────────────
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
UPS_AI_Behavior_PerceptionComponent* GetBehaviorPerception() const { return BehaviorPerception; } UPS_AI_Behavior_PerceptionComponent* GetBehaviorPerception() const { return BehaviorPerception; }
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
UPS_AI_Behavior_PersonalityComponent* GetPersonalityComponent() const { return PersonalityComp; } UPS_AI_Behavior_PersonalityComponent* GetPersonalityComponent() const { return PersonalityComp; }
// ─── Blackboard Helpers ───────────────────────────────────────────── // ─── Blackboard Helpers ─────────────────────────────────────────────
/** Write the current behavior state to the Blackboard. */ /** Write the current behavior state to the Blackboard. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Blackboard") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Blackboard")
void SetBehaviorState(EPS_AI_Behavior_State NewState); void SetBehaviorState(EPS_AI_Behavior_State NewState);
/** Read the current behavior state from the Blackboard. */ /** Read the current behavior state from the Blackboard. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Blackboard") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Blackboard")
EPS_AI_Behavior_State GetBehaviorState() const; EPS_AI_Behavior_State GetBehaviorState() const;
/** Start (or restart) the Behavior Tree. If state is Scripted, transitions to Idle. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void StartBehavior();
/** Stop the Behavior Tree and enter Scripted state. NPC stays alive and perceptible. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void StopBehavior();
/**
* Reset the NPC to its initial post-possession state (as if OnPossess had just run).
* Works for any NPC type (civilian, enemy, protector).
*
* Resets:
* - Blackboard: all threat, cover, patrol, spline keys back to defaults
* - Blackboard: HomeLocation re-captured to current actor location
* - PersonalityComponent: PerceivedThreatLevel = 0, combat/cover cycle cleared
* - TeamId: recomputed from current NPCType + Faction + Hostile flag
* - BehaviorState: forced to Idle (BT transitions to Patrol if on a spline)
* - Movement: any pending MoveTo cancelled, spline follower paused/reset
*
* Does NOT reset: runtime traits (ModifyTrait changes are kept), BT (stays running).
*
* Use cases: civilian calmed down by dialogue, enemy surrendering, mission reset.
*/
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void ResetBehavior();
/**
* Freeze the NPC's autonomous movement WITHOUT stopping the Behavior Tree.
* The BT keeps running (perception, threat, gaze, reactions) but movement
* branches are blocked via the ConversationPaused Blackboard flag, and any
* in-flight move is cancelled. Fleeing is NOT blocked (leave the Flee branch
* un-decorated). Use this to keep an NPC in place during a scripted moment
* (e.g. dialogue) regardless of the automatic conversation pause.
*/
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void DisableBehaviorMovement();
/** Release the manual movement freeze set by DisableBehaviorMovement().
* Movement stays paused if a conversation is still active. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void EnableBehaviorMovement();
protected: protected:
virtual void OnPossess(APawn* InPawn) override; virtual void OnPossess(APawn* InPawn) override;
virtual void OnUnPossess() override; virtual void OnUnPossess() override;
@@ -101,16 +162,71 @@ protected:
UPROPERTY(Transient) UPROPERTY(Transient)
TObjectPtr<UPS_AI_Behavior_PersonalityComponent> PersonalityComp; TObjectPtr<UPS_AI_Behavior_PersonalityComponent> PersonalityComp;
/** Shut down all AI systems when the NPC dies. Called automatically by SetBehaviorState(Dead). */
void HandleDeath();
private: private:
/** Initialize Blackboard with required keys. */ /** Initialize Blackboard with required keys. */
void SetupBlackboard(); void SetupBlackboard();
/** Start the Behavior Tree (from asset or profile). */
void StartBehavior();
/** /**
* Attempt to bind to PS_AI_ConvAgent_ElevenLabsComponent if present on the Pawn. * Attempt to bind to PS_AI_ConvAgent_ElevenLabsComponent if present on the Pawn.
* Uses UObject reflection — no compile-time dependency on PS_AI_ConvAgent. * Uses UObject reflection — no compile-time dependency on PS_AI_ConvAgent.
*/ */
void TryBindConversationAgent(); void TryBindConversationAgent();
// ─── Gaze Bridge (runtime reflection to PS_AI_ConvAgent) ────────
/** Discover GazeComponent on the Pawn and cache property pointers. */
void TryBindGazeComponent();
/** Update the gaze target based on behavior state and proximity. Called by BTService_UpdateGaze. */
void UpdateGazeTarget(float DeltaSeconds);
/** Set the GazeComponent's target actor via reflection. */
void SetGazeTarget(AActor* Target, bool bEnableBody);
/** Clear the GazeComponent's target (smooth return to neutral). */
void ClearGazeTarget();
/** Check if a conversation is active on the Pawn (via reflection on ElevenLabsComponent). */
bool IsConversationActiveOnPawn() const;
/** Apply (or release) the movement pause: syncs the ConversationPaused Blackboard
* flag, stops in-flight movement, and pauses/resumes the spline follower.
* No-op if the state is unchanged. */
void ApplyMovementPaused(bool bPaused);
// Cached reflection pointers (null if PS_AI_ConvAgent not loaded)
TWeakObjectPtr<UActorComponent> CachedGazeComponent;
FObjectProperty* GazeProp_TargetActor = nullptr;
FBoolProperty* GazeProp_bActive = nullptr;
FBoolProperty* GazeProp_bEnableBodyTracking = nullptr;
// Conversation detection cache
TWeakObjectPtr<UActorComponent> CachedConvAgentComponent;
FBoolProperty* ConvProp_bNetIsConversing = nullptr;
// Set by the ConvAgent agent-to-agent orchestrator while two agents converse.
FObjectProperty* ConvProp_GazeOverrideTarget = nullptr;
// Mic VAD cache (local voice activity detection)
TWeakObjectPtr<UActorComponent> CachedMicComponent;
FBoolProperty* MicProp_bIsUserSpeaking = nullptr;
// Conversation pause state
bool bConversationPaused = false;
bool bUserHasSpokenInConversation = false;
// True while movement is frozen by an explicit DisableBehaviorMovement() call.
// Combined with the automatic conversation pause: movement is paused when
// either this is set OR a conversation is active.
bool bMovementManuallyDisabled = false;
// Proximity gaze state
TWeakObjectPtr<AActor> ProximityGazeTarget;
float ProximityGazeDuration = 0.0f;
float ProximityGazeCooldownTimer = 0.0f;
TWeakObjectPtr<AActor> ProximityGazeCooldownActor;
friend class UPS_AI_Behavior_BTService_UpdateGaze;
}; };

View File

@@ -17,7 +17,7 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE_TwoParams(FOnDamageReceived, float, Damage, A
* *
* Attach to the NPC Pawn alongside PersonalityComponent. * Attach to the NPC Pawn alongside PersonalityComponent.
*/ */
UCLASS(ClassGroup = "PS AI Behavior", meta = (BlueprintSpawnableComponent, DisplayName = "PS AI Behavior - Combat")) UCLASS(ClassGroup = "ASTERION|PS_AI_Behavior", meta = (BlueprintSpawnableComponent, DisplayName = "PS AI Behavior - Combat"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_CombatComponent : public UActorComponent class PS_AI_BEHAVIOR_API UPS_AI_Behavior_CombatComponent : public UActorComponent
{ {
GENERATED_BODY() GENERATED_BODY()
@@ -27,11 +27,11 @@ public:
// ─── Configuration ────────────────────────────────────────────────── // ─── Configuration ──────────────────────────────────────────────────
/** Maximum distance at which the NPC can attack (cm). */ /** Maximum distance at which ExecuteAttack() succeeds (cm). Independent from PersonalityProfile ranges used by BTTask_Attack. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Combat", meta = (ClampMin = "50.0")) UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Combat", meta = (ClampMin = "50.0"))
float AttackRange = 200.0f; float AttackRange = 200.0f;
/** Cooldown between attacks (seconds). */ /** Minimum time between consecutive ExecuteAttack() calls (seconds). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Combat", meta = (ClampMin = "0.1")) UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Combat", meta = (ClampMin = "0.1"))
float AttackCooldown = 1.5f; float AttackCooldown = 1.5f;
@@ -52,21 +52,21 @@ public:
// ─── Delegates ────────────────────────────────────────────────────── // ─── Delegates ──────────────────────────────────────────────────────
/** Fired on ALL machines (server + clients) when an attack is executed. */ /** Fired on ALL machines (server + clients) when an attack is executed. */
UPROPERTY(BlueprintAssignable, Category = "PS AI Behavior|Combat") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_Behavior|Combat")
FOnAttackExecuted OnAttackExecuted; FOnAttackExecuted OnAttackExecuted;
/** Fired on server when damage is received. */ /** Fired on server when damage is received. */
UPROPERTY(BlueprintAssignable, Category = "PS AI Behavior|Combat") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_Behavior|Combat")
FOnDamageReceived OnDamageReceived; FOnDamageReceived OnDamageReceived;
// ─── API ──────────────────────────────────────────────────────────── // ─── API ────────────────────────────────────────────────────────────
/** Whether the NPC can currently attack (cooldown elapsed). Server-only. */ /** Whether the NPC can currently attack (cooldown elapsed). Server-only. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Combat") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Combat")
bool CanAttack() const; bool CanAttack() const;
/** Whether the target is within attack range. */ /** Whether the target is within attack range. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Combat") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Combat")
bool IsInAttackRange(AActor* Target) const; bool IsInAttackRange(AActor* Target) const;
/** /**
@@ -74,14 +74,14 @@ public:
* cooldown, and multicasts cosmetic event to all clients. * cooldown, and multicasts cosmetic event to all clients.
* @return True if the attack was executed. * @return True if the attack was executed.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Combat") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Combat")
bool ExecuteAttack(AActor* Target); bool ExecuteAttack(AActor* Target);
/** /**
* Called when this NPC takes damage. Updates threat perception. * Called when this NPC takes damage. Updates threat perception.
* Hook this into the owning Actor's OnTakeAnyDamage. * Hook this into the owning Actor's OnTakeAnyDamage.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Combat") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Combat")
void NotifyDamageReceived(float Damage, AActor* DamageInstigator); void NotifyDamageReceived(float Damage, AActor* DamageInstigator);
// ─── Replication ──────────────────────────────────────────────────── // ─── Replication ────────────────────────────────────────────────────

View File

@@ -9,6 +9,7 @@
class UArrowComponent; class UArrowComponent;
class UBillboardComponent; class UBillboardComponent;
class UTexture2D;
/** /**
* A manually placed strategic point in the level. * A manually placed strategic point in the level.
@@ -24,7 +25,7 @@ class UBillboardComponent;
* - Crouch flag: NPC should crouch at this cover. * - Crouch flag: NPC should crouch at this cover.
* - Editor: color-coded (blue=Cover, yellow=HidingSpot), arrow shows facing. * - Editor: color-coded (blue=Cover, yellow=HidingSpot), arrow shows facing.
*/ */
UCLASS(BlueprintType, Blueprintable, meta = (DisplayName = "PS AI Cover Point")) UCLASS(BlueprintType, Blueprintable, Placeable, meta = (DisplayName = "PS AI Cover Point"))
class PS_AI_BEHAVIOR_API APS_AI_Behavior_CoverPoint : public AActor class PS_AI_BEHAVIOR_API APS_AI_Behavior_CoverPoint : public AActor
{ {
GENERATED_BODY() GENERATED_BODY()
@@ -87,23 +88,23 @@ public:
// ─── API ──────────────────────────────────────────────────────────── // ─── API ────────────────────────────────────────────────────────────
/** Can this point be used by the given NPC type? */ /** Can this point be used by the given NPC type? */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Cover") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Cover")
bool IsAccessibleTo(EPS_AI_Behavior_NPCType NPCType) const; bool IsAccessibleTo(EPS_AI_Behavior_NPCType NPCType) const;
/** Is there room for one more occupant? */ /** Is there room for one more occupant? */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Cover") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Cover")
bool HasRoom() const; bool HasRoom() const;
/** Try to claim this point for an NPC. Returns true if successful. */ /** Try to claim this point for an NPC. Returns true if successful. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Cover") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Cover")
bool Claim(AActor* Occupant); bool Claim(AActor* Occupant);
/** Release this point (NPC leaves cover). */ /** Release this point (NPC leaves cover). */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Cover") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Cover")
void Release(AActor* Occupant); void Release(AActor* Occupant);
/** Get the facing direction (forward vector of the actor). */ /** Get the facing direction (forward vector of the actor). */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Cover") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Cover")
FVector GetCoverDirection() const; FVector GetCoverDirection() const;
/** /**
@@ -112,11 +113,11 @@ public:
* @param ThreatLocation Where the threat is. * @param ThreatLocation Where the threat is.
* @return Combined score 0.0 to 1.0. * @return Combined score 0.0 to 1.0.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Cover") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Cover")
float EvaluateAgainstThreat(const FVector& ThreatLocation) const; float EvaluateAgainstThreat(const FVector& ThreatLocation) const;
/** Enable/disable at runtime (e.g. barricade destroyed). */ /** Enable/disable at runtime (e.g. barricade destroyed). */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Cover") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Cover")
void SetEnabled(bool bNewEnabled); void SetEnabled(bool bNewEnabled);
#if WITH_EDITOR #if WITH_EDITOR
@@ -128,4 +129,13 @@ protected:
private: private:
void UpdateVisualization(); void UpdateVisualization();
#if WITH_EDITORONLY_DATA
/** Cached editor sprite textures (loaded once in constructor). */
UPROPERTY(Transient)
TObjectPtr<UTexture2D> CoverSpriteTexture;
UPROPERTY(Transient)
TObjectPtr<UTexture2D> HidingSpotSpriteTexture;
#endif
}; };

View File

@@ -43,6 +43,7 @@ enum class EPS_AI_Behavior_State : uint8
Fleeing UMETA(DisplayName = "Fleeing"), Fleeing UMETA(DisplayName = "Fleeing"),
TakingCover UMETA(DisplayName = "Taking Cover"), TakingCover UMETA(DisplayName = "Taking Cover"),
Dead UMETA(DisplayName = "Dead"), Dead UMETA(DisplayName = "Dead"),
Scripted UMETA(DisplayName = "Scripted"),
}; };
/** /**
@@ -70,17 +71,106 @@ enum class EPS_AI_Behavior_CoverPointType : uint8
HidingSpot UMETA(DisplayName = "Hiding Spot", ToolTip = "Civilian hiding place (under desks, in closets, behind cars)"), HidingSpot UMETA(DisplayName = "Hiding Spot", ToolTip = "Civilian hiding place (under desks, in closets, behind cars)"),
}; };
/** Combat style — determines engagement behavior (cover cycle vs rush). */
UENUM(BlueprintType)
enum class EPS_AI_Behavior_CombatType : uint8
{
Melee UMETA(DisplayName = "Melee", ToolTip = "Close-range: rush the target"),
Ranged UMETA(DisplayName = "Ranged", ToolTip = "Long-range: use cover, maintain distance"),
};
/** Sub-state within Combat for the cover-shoot cycle. Written to BB for animation sync. */
UENUM(BlueprintType)
enum class EPS_AI_Behavior_CombatSubState : uint8
{
Engaging UMETA(DisplayName = "Engaging", ToolTip = "Moving to combat position"),
AtCover UMETA(DisplayName = "At Cover", ToolTip = "Ducked behind cover"),
MovingToFire UMETA(DisplayName = "Moving To Fire", ToolTip = "Moving to firing position"),
Peeking UMETA(DisplayName = "Peeking", ToolTip = "At firing position, shooting"),
ReturningToCover UMETA(DisplayName = "Returning To Cover", ToolTip = "Moving back to cover after shooting"),
Advancing UMETA(DisplayName = "Advancing", ToolTip = "Moving to next cover"),
};
/** Personality trait axes — each scored 0.0 (low) to 1.0 (high). */ /** Personality trait axes — each scored 0.0 (low) to 1.0 (high). */
UENUM(BlueprintType) UENUM(BlueprintType)
enum class EPS_AI_Behavior_TraitAxis : uint8 enum class EPS_AI_Behavior_TraitAxis : uint8
{ {
Courage UMETA(DisplayName = "Courage", ToolTip = "0 = coward, 1 = fearless"), Courage UMETA(DisplayName = "Courage", ToolTip = "0 = coward, 1 = fearless. Gates flee threshold and cover advancement."),
Aggressivity UMETA(DisplayName = "Aggressivity", ToolTip = "0 = peaceful, 1 = violent"), Aggressivity UMETA(DisplayName = "Aggressivity", ToolTip = "0 = peaceful, 1 = violent. Drives attack threshold and Combat/Cover time ratio."),
Loyalty UMETA(DisplayName = "Loyalty", ToolTip = "0 = selfish, 1 = devoted"), Caution UMETA(DisplayName = "Caution", ToolTip = "0 = reckless, 1 = prudent. Drives flee threshold, cover duration, and spline threat avoidance."),
Caution UMETA(DisplayName = "Caution", ToolTip = "0 = reckless, 1 = prudent"),
Discipline UMETA(DisplayName = "Discipline", ToolTip = "0 = undisciplined, 1 = disciplined"),
}; };
// ─── TeamId Encoding ────────────────────────────────────────────────────────
//
// TeamId encodes NPCType + Faction in a single uint8:
// High nibble = NPCType (0=Civilian, 1=Enemy, 2=Protector)
// Low nibble = Faction (0-15)
//
// Examples:
// Civilian → 0x00 (TeamId 0)
// Enemy faction 0 → 0x10 (TeamId 16)
// Enemy faction 1 → 0x11 (TeamId 17)
// Protector → 0x20 (TeamId 32)
//
namespace PS_AI_Behavior_Team
{
/** Build a TeamId from NPCType + Faction. */
inline uint8 MakeTeamId(EPS_AI_Behavior_NPCType Type, uint8 Faction = 0)
{
uint8 Base;
switch (Type)
{
case EPS_AI_Behavior_NPCType::Civilian: Base = 0x00; break;
case EPS_AI_Behavior_NPCType::Enemy: Base = 0x10; break;
case EPS_AI_Behavior_NPCType::Protector: Base = 0x20; break;
default: return 255; // NoTeam
}
return Base | (Faction & 0x0F);
}
/** Extract the NPCType from a TeamId. */
inline EPS_AI_Behavior_NPCType GetNPCType(uint8 InTeamId)
{
switch (InTeamId & 0xF0)
{
case 0x00: return EPS_AI_Behavior_NPCType::Civilian;
case 0x10: return EPS_AI_Behavior_NPCType::Enemy;
case 0x20: return EPS_AI_Behavior_NPCType::Protector;
default: return EPS_AI_Behavior_NPCType::Any;
}
}
/** Extract the Faction from a TeamId. */
inline uint8 GetFaction(uint8 InTeamId)
{
return InTeamId & 0x0F;
}
/** TeamId used for disguised enemies (same as Civilian faction 0). */
inline constexpr uint8 DisguisedTeamId = 0x00;
}
// ─── Stimulus Tags ──────────────────────────────────────────────────────────
namespace PS_AI_Behavior_Tags
{
/** Tag for enemy gunfire noise events. */
inline const FName EnemyFire = TEXT("EnemyFire");
/** Tag for player/protector gunfire noise events. */
inline const FName PlayerFire = TEXT("PlayerFire");
}
namespace PS_AI_Behavior_Tags_Internal
{
/** Returns true if the tag is any gunfire tag (EnemyFire or PlayerFire). */
inline bool IsGunfire(const FName& Tag)
{
return Tag == PS_AI_Behavior_Tags::EnemyFire || Tag == PS_AI_Behavior_Tags::PlayerFire;
}
}
// ─── Blackboard Key Names ─────────────────────────────────────────────────── // ─── Blackboard Key Names ───────────────────────────────────────────────────
namespace PS_AI_Behavior_BB namespace PS_AI_Behavior_BB
@@ -95,4 +185,9 @@ namespace PS_AI_Behavior_BB
inline const FName HomeLocation = TEXT("HomeLocation"); inline const FName HomeLocation = TEXT("HomeLocation");
inline const FName CurrentSpline = TEXT("CurrentSpline"); inline const FName CurrentSpline = TEXT("CurrentSpline");
inline const FName SplineProgress = TEXT("SplineProgress"); inline const FName SplineProgress = TEXT("SplineProgress");
inline const FName CombatSubState = TEXT("CombatSubState");
inline const FName LastKnownTargetPosition = TEXT("LastKnownTargetPosition");
inline const FName ThreatPawnName = TEXT("ThreatPawnName"); // Debug: name of the owning Pawn behind ThreatActor
inline const FName PreferCover = TEXT("PreferCover"); // Bool: personality-driven cover preference cycle
inline const FName ConversationPaused = TEXT("ConversationPaused"); // Bool: NPC paused during active conversation
} }

View File

@@ -27,7 +27,6 @@
* virtual void SetBehaviorNPCType_Implementation(EPS_AI_Behavior_NPCType T) override { MyType = T; } * virtual void SetBehaviorNPCType_Implementation(EPS_AI_Behavior_NPCType T) override { MyType = T; }
* virtual bool IsBehaviorHostile_Implementation() const override { return bHostile; } * virtual bool IsBehaviorHostile_Implementation() const override { return bHostile; }
* virtual void SetBehaviorHostile_Implementation(bool b) override { bHostile = b; } * virtual void SetBehaviorHostile_Implementation(bool b) override { bHostile = b; }
* virtual uint8 GetBehaviorTeamId_Implementation() const override { return bHostile ? 2 : 1; }
* }; * };
*/ */
UINTERFACE(BlueprintType, Blueprintable, meta = (DisplayName = "PS AI Behavior Interface")) UINTERFACE(BlueprintType, Blueprintable, meta = (DisplayName = "PS AI Behavior Interface"))
@@ -45,11 +44,11 @@ public:
// ─── NPC Type ─────────────────────────────────────────────────────── // ─── NPC Type ───────────────────────────────────────────────────────
/** Get this NPC's type (Civilian, Enemy, Protector). */ /** Get this NPC's type (Civilian, Enemy, Protector). */
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
EPS_AI_Behavior_NPCType GetBehaviorNPCType() const; EPS_AI_Behavior_NPCType GetBehaviorNPCType() const;
/** Set this NPC's type. Called by gameplay logic or plugin actions. */ /** Set this NPC's type. Called by gameplay logic or plugin actions. */
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void SetBehaviorNPCType(EPS_AI_Behavior_NPCType NewType); void SetBehaviorNPCType(EPS_AI_Behavior_NPCType NewType);
// ─── Hostility ────────────────────────────────────────────────────── // ─── Hostility ──────────────────────────────────────────────────────
@@ -59,26 +58,16 @@ public:
* An infiltrated Enemy with IsHostile=false appears as Civilian to the perception system. * An infiltrated Enemy with IsHostile=false appears as Civilian to the perception system.
* When SetHostile(true) is called, the NPC reveals itself and TeamId changes. * When SetHostile(true) is called, the NPC reveals itself and TeamId changes.
*/ */
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
bool IsBehaviorHostile() const; bool IsBehaviorHostile() const;
/** /**
* Set hostility state. Typically called by gameplay scripts or ConvAgent actions. * Set hostility state. Typically called by gameplay scripts or ConvAgent actions.
* Implementors should update their TeamId accordingly. * Implementors should update their TeamId accordingly.
*/ */
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void SetBehaviorHostile(bool bNewHostile); void SetBehaviorHostile(bool bNewHostile);
// ─── Team ───────────────────────────────────────────────────────────
/**
* Get the Team ID for perception affiliation.
* Convention: Civilian=1, Enemy=2, Protector=3, NoTeam=255.
* Infiltrated enemies return 1 (Civilian) until SetHostile(true).
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior")
uint8 GetBehaviorTeamId() const;
// ─── Movement ─────────────────────────────────────────────────────── // ─── Movement ───────────────────────────────────────────────────────
/** /**
@@ -91,13 +80,13 @@ public:
* *
* @param NewSpeed Desired walk speed in cm/s. * @param NewSpeed Desired walk speed in cm/s.
*/ */
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void SetBehaviorMovementSpeed(float NewSpeed); void SetBehaviorMovementSpeed(float NewSpeed);
/** /**
* Get the Pawn's current movement speed (cm/s). * Get the Pawn's current movement speed (cm/s).
*/ */
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
float GetBehaviorMovementSpeed() const; float GetBehaviorMovementSpeed() const;
/** /**
@@ -109,6 +98,85 @@ public:
* @param NewState The new behavioral state. * @param NewState The new behavioral state.
* @param OldState The previous behavioral state. * @param OldState The previous behavioral state.
*/ */
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "PS AI Behavior") UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void OnBehaviorStateChanged(EPS_AI_Behavior_State NewState, EPS_AI_Behavior_State OldState); void OnBehaviorStateChanged(EPS_AI_Behavior_State NewState, EPS_AI_Behavior_State OldState);
// ─── Target Validation ──────────────────────────────────────────────
/**
* Check whether a perceived actor is still a valid target (alive, not despawning, etc.).
* Called by the perception and combat systems to discard dead or invalid targets.
*
* The host project implements this to check its own health/death system.
* Default returns true (all actors are valid unless the Pawn says otherwise).
*
* @param TargetActor The actor to validate.
* @return True if the actor can still be targeted.
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
bool IsTargetActorValid(AActor* TargetActor) const;
// ─── Combat ─────────────────────────────────────────────────────────
/**
* Order the Pawn to start attacking a target.
* The Pawn implements this with its own combat system (aim, fire, melee, etc.).
* Called when the BT enters the Attack task.
*
* @param Target The actor to attack.
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void BehaviorStartAttack(AActor* Target);
/**
* Order the Pawn to stop attacking.
* Called when the BT exits the Attack task (state changed, target lost, etc.).
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void BehaviorStopAttack();
/**
* Query whether the Pawn can currently attack the target (has ammo, weapon ready, etc.).
* If false, the BT will keep the NPC in combat stance but won't call BehaviorStartAttack.
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
bool CanBehaviorAttack(AActor* Target) const;
// ─── Stance ─────────────────────────────────────────────────────────
/**
* Order the Pawn to crouch or stand up.
* Called by the cover system when entering/leaving a cover point with bCrouch set.
* The Pawn implements this however it wants (CharacterMovement->Crouch, animation, etc.).
*
* @param bCrouch True = crouch, False = stand up.
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
void SetBehaviorCrouch(bool bCrouch);
// ─── Combat Style ───────────────────────────────────────────────────
/**
* Get this NPC's combat type (Melee or Ranged).
* Depends on the Pawn's current weapon — implement on your Character.
* Melee NPCs rush the target; Ranged NPCs use cover and maintain distance.
* Default: Melee.
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
EPS_AI_Behavior_CombatType GetBehaviorCombatType() const;
// ─── Actor Resolution ──────────────────────────────────────────────
/**
* Get the actor that represents the threat source for this Pawn.
* Called by the perception system to resolve weapons, VR tracked actors, etc.
* to the actual actor that NPCs should target and flee from.
*
* Default: returns Self (the Pawn itself).
* Override in VR to return the tracked body actor instead of the static Pawn root.
*
* @return The actor to use as threat source / target.
*/
UFUNCTION(BlueprintNativeEvent, BlueprintCallable, Category = "ASTERION|PS_AI_Behavior")
AActor* GetBehaviorThreatActor() const;
}; };

View File

@@ -14,7 +14,7 @@
* *
* Automatically added by PS_AI_Behavior_AIController — you don't need to add it manually. * Automatically added by PS_AI_Behavior_AIController — you don't need to add it manually.
*/ */
UCLASS(ClassGroup = "PS AI Behavior", meta = (DisplayName = "PS AI Behavior - Perception")) UCLASS(ClassGroup = "ASTERION|PS_AI_Behavior", meta = (DisplayName = "PS AI Behavior - Perception"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_PerceptionComponent : public UAIPerceptionComponent class PS_AI_BEHAVIOR_API UPS_AI_Behavior_PerceptionComponent : public UAIPerceptionComponent
{ {
GENERATED_BODY() GENERATED_BODY()
@@ -32,7 +32,7 @@ public:
* If empty, uses default [Protector, Player, Civilian]. * If empty, uses default [Protector, Player, Civilian].
* @return The most threatening actor, or nullptr if none perceived. * @return The most threatening actor, or nullptr if none perceived.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Perception") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Perception")
AActor* GetHighestThreatActor(const TArray<EPS_AI_Behavior_TargetType>& TargetPriority); AActor* GetHighestThreatActor(const TArray<EPS_AI_Behavior_TargetType>& TargetPriority);
/** Convenience overload — reads priority from the Pawn's PersonalityProfile. */ /** Convenience overload — reads priority from the Pawn's PersonalityProfile. */
@@ -42,7 +42,7 @@ public:
* Compute an aggregate threat level from all currently perceived hostile stimuli. * Compute an aggregate threat level from all currently perceived hostile stimuli.
* Returns 0.0 (no threat) to 1.0+ (extreme danger). * Returns 0.0 (no threat) to 1.0+ (extreme danger).
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Perception") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Perception")
float CalculateThreatLevel(); float CalculateThreatLevel();
/** /**
@@ -50,9 +50,27 @@ public:
* @param OutLocation Filled with the threat location if any threat exists. * @param OutLocation Filled with the threat location if any threat exists.
* @return True if a threat was found. * @return True if a threat was found.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Perception") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Perception")
bool GetThreatLocation(FVector& OutLocation); bool GetThreatLocation(FVector& OutLocation);
/**
* Get the location of a perceived gunshot stimulus (any team affiliation).
* Used to set ThreatLocation when a gunshot is heard but the shooter isn't hostile.
* @param OutLocation Filled with the gunshot location if found.
* @return True if a gunshot stimulus was found.
*/
bool GetGunShotStimulusLocation(FVector& OutLocation);
/** Extract an actor's TeamId (checking controller, then TeamComponent). Returns NoTeam if unresolvable. */
static uint8 GetActorTeamId(const AActor* Actor);
/** Walk Owner/Instigator chain to find the Pawn that owns a perceived actor (weapon, AimTarget, etc.). */
static APawn* FindOwningPawn(AActor* Actor);
/** The owning Pawn of the last selected ThreatActor (set by GetHighestThreatActor). */
UPROPERTY(Transient)
TWeakObjectPtr<APawn> LastThreatOwningPawn;
protected: protected:
virtual void BeginPlay() override; virtual void BeginPlay() override;
@@ -64,7 +82,7 @@ protected:
* Uses IsPlayerControlled() for Player, IPS_AI_Behavior interface or * Uses IsPlayerControlled() for Player, IPS_AI_Behavior interface or
* PersonalityComponent for NPC type. * PersonalityComponent for NPC type.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Perception") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Perception")
static EPS_AI_Behavior_TargetType ClassifyActor(const AActor* Actor); static EPS_AI_Behavior_TargetType ClassifyActor(const AActor* Actor);
private: private:

View File

@@ -22,7 +22,7 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE_TwoParams(FOnBehaviorStateChanged, EPS_AI_Beh
* *
* Attach to the NPC Pawn/Character. * Attach to the NPC Pawn/Character.
*/ */
UCLASS(ClassGroup = "PS AI Behavior", meta = (BlueprintSpawnableComponent, DisplayName = "PS AI Behavior - Personality")) UCLASS(ClassGroup = "ASTERION|PS_AI_Behavior", meta = (BlueprintSpawnableComponent, DisplayName = "PS AI Behavior - Personality"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_PersonalityComponent : public UActorComponent class PS_AI_BEHAVIOR_API UPS_AI_Behavior_PersonalityComponent : public UActorComponent
{ {
GENERATED_BODY() GENERATED_BODY()
@@ -36,6 +36,23 @@ public:
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Personality") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Personality")
TObjectPtr<UPS_AI_Behavior_PersonalityProfile> Profile; TObjectPtr<UPS_AI_Behavior_PersonalityProfile> Profile;
/**
* If false, the Behavior Tree does NOT start on possess.
* Use this for NPCs controlled externally (conversations, cinematics).
* Call StartBehavior() on the AIController to activate manually.
*/
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Personality")
bool bAutoStartBehavior = true;
/**
* Master debug toggle for this NPC.
* When enabled, draws floating text above the NPC's head with:
* Name, NPCType, TeamId, State, ThreatLevel, ThreatActor, Hostile, Spline.
* Also enables debug visuals on SplineFollowerComponent.
*/
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Personality|Debug")
bool bDebug = false;
// ─── Runtime State ────────────────────────────────────────────────── // ─── Runtime State ──────────────────────────────────────────────────
/** /**
@@ -64,7 +81,7 @@ public:
// ─── Delegates ────────────────────────────────────────────────────── // ─── Delegates ──────────────────────────────────────────────────────
/** Fired when the behavioral state changes (on server AND clients via OnRep). */ /** Fired when the behavioral state changes (on server AND clients via OnRep). */
UPROPERTY(BlueprintAssignable, Category = "PS AI Behavior|Personality") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_Behavior|Personality")
FOnBehaviorStateChanged OnBehaviorStateChanged; FOnBehaviorStateChanged OnBehaviorStateChanged;
// ─── API ──────────────────────────────────────────────────────────── // ─── API ────────────────────────────────────────────────────────────
@@ -73,7 +90,7 @@ public:
* Evaluate the NPC's reaction based on current traits and perceived threat. * Evaluate the NPC's reaction based on current traits and perceived threat.
* Returns the recommended behavioral state. Server-only. * Returns the recommended behavioral state. Server-only.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
EPS_AI_Behavior_State EvaluateReaction() const; EPS_AI_Behavior_State EvaluateReaction() const;
/** /**
@@ -81,31 +98,49 @@ public:
* Server-only: state is replicated to clients via OnRep. * Server-only: state is replicated to clients via OnRep.
* @return The new behavioral state. * @return The new behavioral state.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
EPS_AI_Behavior_State ApplyReaction(); EPS_AI_Behavior_State ApplyReaction();
/** Get a runtime trait value (returns 0.5 if undefined). */ /** Get a runtime trait value (returns 0.5 if undefined). */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
float GetTrait(EPS_AI_Behavior_TraitAxis Axis) const; float GetTrait(EPS_AI_Behavior_TraitAxis Axis) const;
/** Modify a runtime trait by delta, clamped to [0, 1]. Server-only. */ /** Modify a runtime trait by delta, clamped to [0, 1]. Server-only. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
void ModifyTrait(EPS_AI_Behavior_TraitAxis Axis, float Delta); void ModifyTrait(EPS_AI_Behavior_TraitAxis Axis, float Delta);
/** Force a specific state (e.g. from conversation agent action). Server-only. */ /** Force a specific state (e.g. from conversation agent action). Server-only. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
void ForceState(EPS_AI_Behavior_State NewState); void ForceState(EPS_AI_Behavior_State NewState);
/**
* Reset runtime state — perceived threat level, combat/cover cycle.
* Does NOT reset traits (ModifyTrait changes are preserved).
* Called by AIController::ResetBehavior().
*/
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
void ResetRuntimeState();
/** Get the NPC type from the interface or profile. */ /** Get the NPC type from the interface or profile. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
EPS_AI_Behavior_NPCType GetNPCType() const; EPS_AI_Behavior_NPCType GetNPCType() const;
/**
* Whether the NPC currently prefers cover over direct attack.
* Driven by the Combat/Cover cycle timer based on Aggressivity vs Caution.
* Used by the IsCoverNeeded BT decorator.
*/
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
bool ShouldPreferCover() const { return bPreferCover; }
// ─── Replication ──────────────────────────────────────────────────── // ─── Replication ────────────────────────────────────────────────────
virtual void GetLifetimeReplicatedProps(TArray<FLifetimeProperty>& OutLifetimeProps) const override; virtual void GetLifetimeReplicatedProps(TArray<FLifetimeProperty>& OutLifetimeProps) const override;
protected: protected:
virtual void BeginPlay() override; virtual void BeginPlay() override;
virtual void TickComponent(float DeltaTime, ELevelTick TickType,
FActorComponentTickFunction* ThisTickFunction) override;
UFUNCTION() UFUNCTION()
void OnRep_CurrentState(EPS_AI_Behavior_State OldState); void OnRep_CurrentState(EPS_AI_Behavior_State OldState);
@@ -118,4 +153,21 @@ private:
* - Calls IPS_AI_Behavior::OnBehaviorStateChanged on the Pawn * - Calls IPS_AI_Behavior::OnBehaviorStateChanged on the Pawn
*/ */
void HandleStateChanged(EPS_AI_Behavior_State OldState, EPS_AI_Behavior_State NewState); void HandleStateChanged(EPS_AI_Behavior_State OldState, EPS_AI_Behavior_State NewState);
// ─── Combat/Cover Cycle Timer ──────────────────────────────────────
/** Countdown timer for the current Combat or TakingCover phase. */
float CombatCoverTimer = 0.0f;
/** Whether the combat/cover cycle timer is active. */
bool bCombatCoverCycleActive = false;
/** Current cover preference — toggled by the cycle timer. */
bool bPreferCover = false;
/** Calculate the duration for a Combat or TakingCover phase based on personality. */
float CalculatePhaseDuration(EPS_AI_Behavior_State Phase) const;
/** Draw floating debug text above the NPC's head. */
void DrawDebugInfo() const;
}; };

View File

@@ -32,6 +32,21 @@ public:
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality") UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality")
EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Civilian; EPS_AI_Behavior_NPCType NPCType = EPS_AI_Behavior_NPCType::Civilian;
/**
* Faction index within the same NPCType.
* Used to create rival groups of the same type (e.g. two enemy gangs).
*
* Same NPCType + same Faction → Friendly (allies)
* Same NPCType + different Faction → Hostile (rivals)
* Civilian ↔ Protector → always Friendly (regardless of faction)
* Everything else → Hostile
*
* Example: "Gang A" Enemy profile → Faction 0, "Gang B" Enemy profile → Faction 1.
*/
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality",
meta = (ClampMin = "0", ClampMax = "15"))
uint8 Faction = 0;
// ─── Trait Scores ─────────────────────────────────────────────────── // ─── Trait Scores ───────────────────────────────────────────────────
/** Personality trait scores. Each axis ranges from 0.0 to 1.0. */ /** Personality trait scores. Each axis ranges from 0.0 to 1.0. */
@@ -64,6 +79,15 @@ public:
meta = (ClampMin = "0.0", ClampMax = "1.0")) meta = (ClampMin = "0.0", ClampMax = "1.0"))
float AlertThreshold = 0.15f; float AlertThreshold = 0.15f;
/**
* How fast the perceived threat level decays per second when no active threat is perceived.
* Low values = NPC stays scared/alert longer (civilians).
* High values = NPC calms down quickly (trained soldiers).
*/
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality|Reaction",
meta = (ClampMin = "0.001", ClampMax = "1.0"))
float ThreatDecayRate = 0.05f;
// ─── Target Priority (Combat) ─────────────────────────────────────── // ─── Target Priority (Combat) ───────────────────────────────────────
/** /**
@@ -80,6 +104,33 @@ public:
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality|Combat") UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality|Combat")
TArray<EPS_AI_Behavior_TargetType> TargetPriority; TArray<EPS_AI_Behavior_TargetType> TargetPriority;
/**
* Minimum attack range (cm). NPC backs away if target is closer than this.
* Melee: ~100cm. Ranged: ~600cm.
*/
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality|Combat", meta = (ClampMin = "0.0"))
float MinAttackRange = 100.0f;
/**
* Maximum attack range (cm). NPC advances if target is farther than this.
* Between Min and Max, the NPC holds position and attacks.
* Melee: ~300cm. Ranged: ~1500cm.
*/
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality|Combat", meta = (ClampMin = "50.0"))
float MaxAttackRange = 300.0f;
// ─── Combat/Cover Cycle ────────────────────────────────────────────
/**
* Base duration (seconds) of the Combat↔TakingCover cycle.
* The actual time in each state is proportional to Aggressivity vs Caution:
* Combat duration = CombatCoverCycleDuration × Aggressivity / (Aggressivity + Caution)
* TakingCover duration = CombatCoverCycleDuration × Caution / (Aggressivity + Caution)
*/
UPROPERTY(EditAnywhere, BlueprintReadOnly, Category = "Personality|Combat",
meta = (ClampMin = "4.0", ClampMax = "60.0"))
float CombatCoverCycleDuration = 15.0f;
// ─── Movement Speed per State ────────────────────────────────────── // ─── Movement Speed per State ──────────────────────────────────────
/** /**
@@ -109,11 +160,11 @@ public:
/** /**
* Get the score for a given trait axis. Returns 0.5 if the axis is not defined. * Get the score for a given trait axis. Returns 0.5 if the axis is not defined.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
float GetTrait(EPS_AI_Behavior_TraitAxis Axis) const; float GetTrait(EPS_AI_Behavior_TraitAxis Axis) const;
/** Get the speed for a given state. Returns DefaultWalkSpeed if state not in SpeedPerState. */ /** Get the speed for a given state. Returns DefaultWalkSpeed if state not in SpeedPerState. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Personality") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Personality")
float GetSpeedForState(EPS_AI_Behavior_State State) const; float GetSpeedForState(EPS_AI_Behavior_State State) const;
/** UPrimaryDataAsset interface */ /** UPrimaryDataAsset interface */

View File

@@ -10,7 +10,7 @@
* Project-wide settings for the PS AI Behavior plugin. * Project-wide settings for the PS AI Behavior plugin.
* Accessible via Project Settings -> Plugins -> PS AI Behavior. * Accessible via Project Settings -> Plugins -> PS AI Behavior.
*/ */
UCLASS(config = Game, defaultconfig, meta = (DisplayName = "PS AI Behavior")) UCLASS(config = Game, defaultconfig, meta = (DisplayName = "ASTERION|PS_AI_Behavior"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_Settings : public UDeveloperSettings class PS_AI_BEHAVIOR_API UPS_AI_Behavior_Settings : public UDeveloperSettings
{ {
GENERATED_BODY() GENERATED_BODY()
@@ -38,15 +38,19 @@ public:
UPROPERTY(config, EditAnywhere, Category = "Perception", meta = (ClampMin = 100.0, ClampMax = 10000.0)) UPROPERTY(config, EditAnywhere, Category = "Perception", meta = (ClampMin = 100.0, ClampMax = 10000.0))
float DefaultHearingRange = 3000.0f; float DefaultHearingRange = 3000.0f;
/** Seconds before a perceived stimulus is forgotten. */ /** Seconds before a sight stimulus is forgotten. */
UPROPERTY(config, EditAnywhere, Category = "Perception", meta = (ClampMin = 1.0, ClampMax = 60.0)) UPROPERTY(config, EditAnywhere, Category = "Perception", meta = (ClampMin = 1.0, ClampMax = 60.0))
float PerceptionMaxAge = 10.0f; float PerceptionMaxAge = 10.0f;
/** Seconds before a hearing stimulus is forgotten (gunshots, etc.). Higher = NPCs stay scared longer. */
UPROPERTY(config, EditAnywhere, Category = "Perception", meta = (ClampMin = 1.0, ClampMax = 120.0))
float HearingMaxAge = 30.0f;
// ─── Threat ───────────────────────────────────────────────────────── // ─── Threat ─────────────────────────────────────────────────────────
/** Threat level decay rate per second when no threat is visible. */ /** Threat level decay rate per second when no active threat stimulus exists. */
UPROPERTY(config, EditAnywhere, Category = "Threat", meta = (ClampMin = 0.0, ClampMax = 2.0)) UPROPERTY(config, EditAnywhere, Category = "Threat", meta = (ClampMin = 0.0, ClampMax = 2.0))
float ThreatDecayRate = 0.15f; float ThreatDecayRate = 0.05f;
// ─── Section Name ─────────────────────────────────────────────────── // ─── Section Name ───────────────────────────────────────────────────

View File

@@ -32,7 +32,7 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam(FOnSplineEndReached,
* *
* Attach to the NPC Pawn. Works with or without the AI Controller. * Attach to the NPC Pawn. Works with or without the AI Controller.
*/ */
UCLASS(ClassGroup = "PS AI Behavior", meta = (BlueprintSpawnableComponent, DisplayName = "PS AI Behavior - Spline Follower")) UCLASS(ClassGroup = "ASTERION|PS_AI_Behavior", meta = (BlueprintSpawnableComponent, DisplayName = "PS AI Behavior - Spline Follower"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_SplineFollowerComponent : public UActorComponent class PS_AI_BEHAVIOR_API UPS_AI_Behavior_SplineFollowerComponent : public UActorComponent
{ {
GENERATED_BODY() GENERATED_BODY()
@@ -58,6 +58,14 @@ public:
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Spline Follower", meta = (ClampMin = "30.0")) UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Spline Follower", meta = (ClampMin = "30.0"))
float RotationInterpSpeed = 360.0f; float RotationInterpSpeed = 360.0f;
/**
* How far ahead on the spline to place the target point (cm).
* Larger = smoother wider turns, smaller = tighter cornering.
* The NPC always steers toward this point, which follows the spline curve naturally.
*/
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Spline Follower", meta = (ClampMin = "50.0"))
float LookaheadDistance = 200.0f;
/** /**
* Whether to auto-choose a spline at junctions. * Whether to auto-choose a spline at junctions.
* If false, OnApproachingJunction fires and you must call SwitchToSpline manually. * If false, OnApproachingJunction fires and you must call SwitchToSpline manually.
@@ -71,7 +79,11 @@ public:
* If false, stop and fire OnSplineEndReached. * If false, stop and fire OnSplineEndReached.
*/ */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Spline Follower") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Spline Follower")
bool bReverseAtEnd = false; bool bReverseAtEnd = true;
/** Draw debug info: target point on spline, direction, gap distance. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Spline Follower|Debug")
bool bDebug = false;
// ─── Runtime State ────────────────────────────────────────────────── // ─── Runtime State ──────────────────────────────────────────────────
@@ -94,18 +106,25 @@ public:
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Replicated, Category = "Spline Follower|Runtime") UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Replicated, Category = "Spline Follower|Runtime")
bool bIsFollowing = false; bool bIsFollowing = false;
/** Set by StopFollowing(): a firm, manual stop. While true, the Behavior Tree
* spline tasks refuse to (re)start following — only an explicit StartFollowing()
* / StartFollowingAtDistance() clears it. This is what distinguishes a deliberate
* stop from PauseFollowing() (which the BT is allowed to auto-resume). */
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, Category = "Spline Follower|Runtime")
bool bManuallyStopped = false;
// ─── Delegates ────────────────────────────────────────────────────── // ─── Delegates ──────────────────────────────────────────────────────
/** Fired when approaching a junction. Use to make custom spline selection. */ /** Fired when approaching a junction. Use to make custom spline selection. */
UPROPERTY(BlueprintAssignable, Category = "PS AI Behavior|Spline Follower") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
FOnApproachingJunction OnApproachingJunction; FOnApproachingJunction OnApproachingJunction;
/** Fired when the NPC switches to a different spline. */ /** Fired when the NPC switches to a different spline. */
UPROPERTY(BlueprintAssignable, Category = "PS AI Behavior|Spline Follower") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
FOnSplineChanged OnSplineChanged; FOnSplineChanged OnSplineChanged;
/** Fired when the NPC reaches the end of a spline (if bReverseAtEnd is false). */ /** Fired when the NPC reaches the end of a spline (if bReverseAtEnd is false). */
UPROPERTY(BlueprintAssignable, Category = "PS AI Behavior|Spline Follower") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
FOnSplineEndReached OnSplineEndReached; FOnSplineEndReached OnSplineEndReached;
// ─── API ──────────────────────────────────────────────────────────── // ─── API ────────────────────────────────────────────────────────────
@@ -116,24 +135,24 @@ public:
* @param bForward Direction of travel. * @param bForward Direction of travel.
* @return True if successfully started. * @return True if successfully started.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
bool StartFollowing(APS_AI_Behavior_SplinePath* Spline, bool bForward = true); bool StartFollowing(APS_AI_Behavior_SplinePath* Spline, bool bForward = true);
/** /**
* Start following the given spline from a specific distance. * Start following the given spline from a specific distance.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
bool StartFollowingAtDistance(APS_AI_Behavior_SplinePath* Spline, float StartDistance, bool bForward = true); bool StartFollowingAtDistance(APS_AI_Behavior_SplinePath* Spline, float StartDistance, bool bForward = true);
/** Stop following the current spline. */ /** Stop following the current spline. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
void StopFollowing(); void StopFollowing();
/** Pause/resume without losing state. */ /** Pause/resume without losing state. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
void PauseFollowing(); void PauseFollowing();
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
void ResumeFollowing(); void ResumeFollowing();
/** /**
@@ -142,15 +161,15 @@ public:
* @param DistanceOnNew Distance along the new spline to start from. * @param DistanceOnNew Distance along the new spline to start from.
* @param bNewForward Direction on the new spline. * @param bNewForward Direction on the new spline.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
void SwitchToSpline(APS_AI_Behavior_SplinePath* NewSpline, float DistanceOnNew, bool bNewForward = true); void SwitchToSpline(APS_AI_Behavior_SplinePath* NewSpline, float DistanceOnNew, bool bNewForward = true);
/** Get the effective walk speed (considering spline override). */ /** Get the effective walk speed (considering spline override). */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
float GetEffectiveSpeed() const; float GetEffectiveSpeed() const;
/** Get progress as a 0-1 ratio. */ /** Get progress as a 0-1 ratio. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline Follower") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline Follower")
float GetProgress() const; float GetProgress() const;
protected: protected:

View File

@@ -35,7 +35,7 @@ public:
* Called automatically after world initialization. Can be called again * Called automatically after world initialization. Can be called again
* if splines are added/removed at runtime. * if splines are added/removed at runtime.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|SplineNetwork") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|SplineNetwork")
void RebuildNetwork(); void RebuildNetwork();
// ─── Queries ──────────────────────────────────────────────────────── // ─── Queries ────────────────────────────────────────────────────────
@@ -49,7 +49,7 @@ public:
* @param OutDistance Distance along the spline to the closest point. * @param OutDistance Distance along the spline to the closest point.
* @return True if a spline was found within MaxDistance. * @return True if a spline was found within MaxDistance.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|SplineNetwork") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|SplineNetwork")
bool FindClosestSpline(const FVector& WorldLocation, bool FindClosestSpline(const FVector& WorldLocation,
EPS_AI_Behavior_NPCType NPCType, float MaxDistance, EPS_AI_Behavior_NPCType NPCType, float MaxDistance,
APS_AI_Behavior_SplinePath*& OutSpline, float& OutDistance) const; APS_AI_Behavior_SplinePath*& OutSpline, float& OutDistance) const;
@@ -57,7 +57,7 @@ public:
/** /**
* Get all splines of a given category. * Get all splines of a given category.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|SplineNetwork") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|SplineNetwork")
TArray<APS_AI_Behavior_SplinePath*> GetSplinesForCategory( TArray<APS_AI_Behavior_SplinePath*> GetSplinesForCategory(
EPS_AI_Behavior_NPCType Category) const; EPS_AI_Behavior_NPCType Category) const;
@@ -73,7 +73,7 @@ public:
* @param CautionScore Optional: NPC's caution trait (0-1). Higher = prefer quieter paths. * @param CautionScore Optional: NPC's caution trait (0-1). Higher = prefer quieter paths.
* @return The chosen spline (could be the same if staying is best). * @return The chosen spline (could be the same if staying is best).
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|SplineNetwork") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|SplineNetwork")
APS_AI_Behavior_SplinePath* ChooseSplineAtJunction( APS_AI_Behavior_SplinePath* ChooseSplineAtJunction(
APS_AI_Behavior_SplinePath* CurrentSpline, int32 JunctionIndex, APS_AI_Behavior_SplinePath* CurrentSpline, int32 JunctionIndex,
EPS_AI_Behavior_NPCType NPCType, EPS_AI_Behavior_NPCType NPCType,
@@ -81,11 +81,11 @@ public:
float CautionScore = 0.5f) const; float CautionScore = 0.5f) const;
/** Total number of splines in the network. */ /** Total number of splines in the network. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|SplineNetwork") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|SplineNetwork")
int32 GetSplineCount() const { return AllSplines.Num(); } int32 GetSplineCount() const { return AllSplines.Num(); }
/** Total number of junctions detected. */ /** Total number of junctions detected. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|SplineNetwork") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|SplineNetwork")
int32 GetJunctionCount() const { return TotalJunctions; } int32 GetJunctionCount() const { return TotalJunctions; }
private: private:

View File

@@ -44,7 +44,7 @@ struct FPS_AI_Behavior_SplineJunction
* and lets NPCs switch between paths at those points. * and lets NPCs switch between paths at those points.
* - Supports bidirectional travel by default. * - Supports bidirectional travel by default.
*/ */
UCLASS(BlueprintType, Blueprintable, meta = (DisplayName = "PS AI Spline Path")) UCLASS(BlueprintType, Blueprintable, Placeable, meta = (DisplayName = "PS AI Spline Path"))
class PS_AI_BEHAVIOR_API APS_AI_Behavior_SplinePath : public AActor class PS_AI_BEHAVIOR_API APS_AI_Behavior_SplinePath : public AActor
{ {
GENERATED_BODY() GENERATED_BODY()
@@ -94,7 +94,7 @@ public:
// ─── API ──────────────────────────────────────────────────────────── // ─── API ────────────────────────────────────────────────────────────
/** Can the given NPC type use this spline? */ /** Can the given NPC type use this spline? */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline")
bool IsAccessibleTo(EPS_AI_Behavior_NPCType NPCType) const; bool IsAccessibleTo(EPS_AI_Behavior_NPCType NPCType) const;
/** /**
@@ -104,7 +104,7 @@ public:
* @param OutWorldPoint World location of the closest point on the spline. * @param OutWorldPoint World location of the closest point on the spline.
* @return Distance from WorldLocation to the closest spline point. * @return Distance from WorldLocation to the closest spline point.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline")
float GetClosestPointOnSpline(const FVector& WorldLocation, float GetClosestPointOnSpline(const FVector& WorldLocation,
float& OutDistance, FVector& OutWorldPoint) const; float& OutDistance, FVector& OutWorldPoint) const;
@@ -115,22 +115,26 @@ public:
* @param bForward Travel direction (true = increasing distance). * @param bForward Travel direction (true = increasing distance).
* @return Array of upcoming junctions. * @return Array of upcoming junctions.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline")
TArray<FPS_AI_Behavior_SplineJunction> GetUpcomingJunctions( TArray<FPS_AI_Behavior_SplineJunction> GetUpcomingJunctions(
float CurrentDistance, float LookAheadDist, bool bForward) const; float CurrentDistance, float LookAheadDist, bool bForward) const;
/** Total spline length. */ /** Total spline length. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline")
float GetSplineLength() const; float GetSplineLength() const;
/** Get world location at a distance along the spline. */ /** Get world location at a distance along the spline. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline")
FVector GetWorldLocationAtDistance(float Distance) const; FVector GetWorldLocationAtDistance(float Distance) const;
/** Get world rotation at a distance along the spline. */ /** Get world rotation at a distance along the spline. */
UFUNCTION(BlueprintCallable, Category = "PS AI Behavior|Spline") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline")
FRotator GetWorldRotationAtDistance(float Distance) const; FRotator GetWorldRotationAtDistance(float Distance) const;
/** Get world-space tangent direction at a distance along the spline. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Spline")
FVector GetWorldDirectionAtDistance(float Distance) const;
#if WITH_EDITOR #if WITH_EDITOR
virtual void PostEditChangeProperty(FPropertyChangedEvent& PropertyChangedEvent) override; virtual void PostEditChangeProperty(FPropertyChangedEvent& PropertyChangedEvent) override;
#endif #endif

View File

@@ -0,0 +1,59 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "Kismet/BlueprintFunctionLibrary.h"
#include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_Statics.generated.h"
/**
* Static helpers for the PS AI Behavior plugin.
* Call from Blueprint or C++ to interact with the behavior system.
*/
UCLASS()
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_Statics : public UBlueprintFunctionLibrary
{
GENERATED_BODY()
public:
/**
* Report a gunfire noise event so all nearby NPCs react.
* Uses tag "EnemyFire" or "PlayerFire" depending on bIsEnemyFire.
*
* Compatible with existing ReportNoiseEvent calls using those tags.
*
* - Enemies (non-hostile) hearing this will become hostile toward the shooter.
* - Civilians hearing this will flee based on their personality traits.
* - Already-hostile enemies treat this as additional threat.
*
* @param WorldContext Any world-context object (self, weapon, etc.)
* @param Location World location of the gunshot.
* @param Shooter The actor who fired (Instigator for perception).
* @param bIsEnemyFire True = tag "EnemyFire", False = tag "PlayerFire".
* @param Loudness Noise loudness (default 1.0). Scales with HearingRange.
* @param MaxRange Override max hearing range (0 = use sense default).
*/
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Perception",
meta = (WorldContext = "WorldContext", DefaultToSelf = "Shooter"))
static void ReportGunfire(UObject* WorldContext, FVector Location,
AActor* Shooter, bool bIsEnemyFire = false,
float Loudness = 1.0f, float MaxRange = 0.0f);
/**
* Check if there is a clear line-of-sight between two actors.
* Traces from the source actor's eye height to the target actor's location.
*
* @param World The world to trace in.
* @param Source The actor looking (trace starts at Source + EyeHeightOffset).
* @param Target The actor being looked at (trace ends at Target location).
* @param EyeHeightOffset Height offset above Source's origin for the trace start (cm).
* @return True if line-of-sight is clear (no blocking geometry).
*/
static bool HasLineOfSight(const UWorld* World, const AActor* Source, const AActor* Target,
float EyeHeightOffset = 150.0f);
/** Check if debug is enabled for a given Pawn (checks CVar override first, then PersonalityComponent.bDebug). */
static bool IsDebugEnabled(const APawn* Pawn);
};

View File

@@ -0,0 +1,51 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "Components/ActorComponent.h"
#include "GenericTeamAgentInterface.h"
#include "PS_AI_Behavior_Definitions.h"
#include "PS_AI_Behavior_TeamComponent.generated.h"
/**
* Drop this component on any Pawn (player or non-AI character) to give it
* a role that the PS AI Behavior perception system will recognize.
*
* NPCs controlled by PS_AI_Behavior_AIController already have their role
* set automatically via the Interface — this component is meant for Pawns
* WITHOUT that controller (typically the player character).
*
* Role determines team affiliation:
* Civilian (Team 1) — Enemies attack, Protectors defend
* Enemy (Team 2) — Protectors and Civilians react
* Protector (Team 3) — Allied with Civilians, hostile to Enemies
*/
UCLASS(ClassGroup = "ASTERION|PS_AI_Behavior", meta = (BlueprintSpawnableComponent,
DisplayName = "PS AI Behavior Pawn Identity"))
class PS_AI_BEHAVIOR_API UPS_AI_Behavior_TeamComponent : public UActorComponent
{
GENERATED_BODY()
public:
UPS_AI_Behavior_TeamComponent();
/** Role of this actor in the behavior system. Determines team affiliation. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_Behavior|Team")
EPS_AI_Behavior_NPCType Role = EPS_AI_Behavior_NPCType::Civilian;
/** Faction within the same role. Same role + different faction = rivals. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_Behavior|Team",
meta = (ClampMin = "0", ClampMax = "15"))
uint8 Faction = 0;
/** Change role at runtime. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_Behavior|Team")
void SetRole(EPS_AI_Behavior_NPCType NewRole) { Role = NewRole; }
/** Returns the TeamId derived from the current Role. */
uint8 GetTeamId() const;
/** Returns the TeamId as FGenericTeamId. */
FGenericTeamId GetGenericTeamId() const { return FGenericTeamId(GetTeamId()); }
};

View File

@@ -0,0 +1,29 @@
// Copyright Asterion. All Rights Reserved.
#include "PS_AI_Behavior_PersonalityProfileFactory.h"
#include "PS_AI_Behavior_PersonalityProfile.h"
UPS_AI_Behavior_PersonalityProfileFactory::UPS_AI_Behavior_PersonalityProfileFactory()
{
bCreateNew = true;
bEditAfterNew = true;
SupportedClass = UPS_AI_Behavior_PersonalityProfile::StaticClass();
}
UObject* UPS_AI_Behavior_PersonalityProfileFactory::FactoryCreateNew(
UClass* InClass, UObject* InParent, FName InName, EObjectFlags Flags,
UObject* Context, FFeedbackContext* Warn)
{
return NewObject<UPS_AI_Behavior_PersonalityProfile>(InParent, InClass, InName, Flags);
}
FText UPS_AI_Behavior_PersonalityProfileFactory::GetDisplayName() const
{
return FText::FromString(TEXT("Personality Profile"));
}
uint32 UPS_AI_Behavior_PersonalityProfileFactory::GetMenuCategories() const
{
// EAssetTypeCategories::Misc = 1
return 1;
}

View File

@@ -0,0 +1,27 @@
// Copyright Asterion. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "Factories/Factory.h"
#include "PS_AI_Behavior_PersonalityProfileFactory.generated.h"
/**
* Factory to create PersonalityProfile Data Assets from the Content Browser.
* Right-click → PS AI Behavior → Personality Profile
*/
UCLASS()
class UPS_AI_Behavior_PersonalityProfileFactory : public UFactory
{
GENERATED_BODY()
public:
UPS_AI_Behavior_PersonalityProfileFactory();
virtual UObject* FactoryCreateNew(UClass* InClass, UObject* InParent,
FName InName, EObjectFlags Flags, UObject* Context,
FFeedbackContext* Warn) override;
virtual FText GetDisplayName() const override;
virtual uint32 GetMenuCategories() const override;
};

View File

@@ -94,12 +94,23 @@ bool FPS_AI_Behavior_SplineEdMode::HandleClick(
return false; // No ground hit return false; // No ground hit
} }
FVector ClickLocation = Hit.ImpactPoint; // Use the ray trace impact point directly — no re-snap needed, this IS the ground
FVector ClickLocation = Hit.ImpactPoint + FVector(0, 0, 5.0f); // Slight offset above ground
// Ctrl+Click on existing spline → select for extension // ─── Without Ctrl: pass click through to editor (select/move actors) ─
if (Click.IsControlDown()) if (!Click.IsControlDown())
{
return false; // Let the editor handle selection/dragging
}
// ─── Ctrl+Click: place or extend ─────────────────────────────────────
switch (ActiveTool)
{
case EPS_AI_Behavior_EdModeTool::Spline:
{
// Ctrl+Click near an existing spline → select for extension
if (!ActiveSpline)
{ {
// Check if we hit a SplinePath
AActor* HitActor = Hit.GetActor(); AActor* HitActor = Hit.GetActor();
APS_AI_Behavior_SplinePath* HitSpline = Cast<APS_AI_Behavior_SplinePath>(HitActor); APS_AI_Behavior_SplinePath* HitSpline = Cast<APS_AI_Behavior_SplinePath>(HitActor);
@@ -123,25 +134,16 @@ bool FPS_AI_Behavior_SplineEdMode::HandleClick(
SelectSplineForExtension(HitSpline); SelectSplineForExtension(HitSpline);
return true; return true;
} }
return false;
} }
// Snap to ground // Ctrl+Click elsewhere → add spline point
if (bSnapToGround)
{
SnapToGround(ClickLocation);
}
// ─── Route to active tool ───────────────────────────────────────────
switch (ActiveTool)
{
case EPS_AI_Behavior_EdModeTool::Spline:
AddPointToSpline(ClickLocation); AddPointToSpline(ClickLocation);
}
break; break;
case EPS_AI_Behavior_EdModeTool::CoverPoint: case EPS_AI_Behavior_EdModeTool::CoverPoint:
{ {
// Ctrl+Click → place cover point
// Cover point faces toward the camera (typical workflow) // Cover point faces toward the camera (typical workflow)
const FVector CamLoc = InViewportClient->GetViewLocation(); const FVector CamLoc = InViewportClient->GetViewLocation();
const FVector DirToCamera = (CamLoc - ClickLocation).GetSafeNormal2D(); const FVector DirToCamera = (CamLoc - ClickLocation).GetSafeNormal2D();

View File

@@ -22,4 +22,7 @@ private:
void RegisterSplinePanel(); void RegisterSplinePanel();
void UnregisterSplinePanel(); void UnregisterSplinePanel();
/** Create or update the BB_Behavior Blackboard asset with proper enum keys. */
void EnsureBlackboardAsset();
}; };

View File

@@ -19,6 +19,10 @@ void FPS_AI_ConvAgentModule::StartupModule()
Settings = NewObject<UPS_AI_ConvAgent_Settings_ElevenLabs>(GetTransientPackage(), "PS_AI_ConvAgent_Settings_ElevenLabs", RF_Standalone); Settings = NewObject<UPS_AI_ConvAgent_Settings_ElevenLabs>(GetTransientPackage(), "PS_AI_ConvAgent_Settings_ElevenLabs", RF_Standalone);
Settings->AddToRoot(); Settings->AddToRoot();
// NOTE: the client-provided API key (if any) is loaded lazily on the first
// GetSettings() call rather than here — see GetSettings(). Loading at module
// startup would be too early: the SaveGame system may not be ready yet.
if (ISettingsModule* SettingsModule = FModuleManager::GetModulePtr<ISettingsModule>("Settings")) if (ISettingsModule* SettingsModule = FModuleManager::GetModulePtr<ISettingsModule>("Settings"))
{ {
SettingsModule->RegisterSettings( SettingsModule->RegisterSettings(

View File

@@ -0,0 +1,561 @@
// Copyright ASTERION. All Rights Reserved.
#include "PS_AI_ConvAgent_AgentDialogue.h"
#include "PS_AI_ConvAgent_ElevenLabsComponent.h"
#include "GameFramework/Actor.h"
#include "GameFramework/Pawn.h"
#include "GameFramework/PlayerController.h"
#include "TimerManager.h"
#include "Engine/World.h"
DEFINE_LOG_CATEGORY_STATIC(LogPS_AI_ConvAgent_Dialogue, Log, All);
static const TCHAR* FloorName(uint8 F)
{
switch (F) { case 1: return TEXT("A"); case 2: return TEXT("B"); default: return TEXT("None"); }
}
APS_AI_ConvAgent_AgentDialogue::APS_AI_ConvAgent_AgentDialogue()
{
// Ticks only while "addressing player" (auto-release watcher); disabled otherwise.
PrimaryActorTick.bCanEverTick = true;
PrimaryActorTick.bStartWithTickEnabled = false;
bReplicates = false; // server-side orchestration; agents replicate their own audio
}
void APS_AI_ConvAgent_AgentDialogue::BeginPlay()
{
Super::BeginPlay();
if (bAutoStartOnBeginPlay && HasAuthority())
{
StartDialogue();
}
}
void APS_AI_ConvAgent_AgentDialogue::EndPlay(const EEndPlayReason::Type Reason)
{
StopDialogue(false);
UnbindAll();
Super::EndPlay(Reason);
}
UPS_AI_ConvAgent_ElevenLabsComponent* APS_AI_ConvAgent_AgentDialogue::ResolveAgentComp(AActor* Actor) const
{
return Actor ? Actor->FindComponentByClass<UPS_AI_ConvAgent_ElevenLabsComponent>() : nullptr;
}
void APS_AI_ConvAgent_AgentDialogue::StartDialogue()
{
if (!HasAuthority())
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Warning,
TEXT("StartDialogue ignored — must run on the server (authority)."));
return;
}
if (bRunning)
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log, TEXT("StartDialogue: already running."));
return;
}
CompA = ResolveAgentComp(AgentA);
CompB = ResolveAgentComp(AgentB);
if (!CompA || !CompB)
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Warning,
TEXT("StartDialogue: no ElevenLabs component on %s and/or %s."),
*GetNameSafe(AgentA), *GetNameSafe(AgentB));
return;
}
if (CompA == CompB)
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Warning, TEXT("StartDialogue: AgentA and AgentB are the same agent."));
return;
}
UnbindAll();
BindAgent(CompA, /*bIsA=*/true);
BindAgent(CompB, /*bIsA=*/false);
// CRITICAL: this is a TEXT-only loop. The agents must NEVER open their mics,
// otherwise each one HEARS the other's TTS and self-replies (via STT) — that
// produces the overlap / "already has the answer before the other spoke" chaos.
CompA->bAutoStartListening = false;
CompB->bAutoStartListening = false;
// The dialogue is interrupted when the PLAYER speaks — detected via a local mic
// voice-onset (primary) and OnAgentTranscript (backup). Enable transcript, and
// force Server turn mode + interruption so per-agent editor config can't silently
// block the mic path / the player barging in.
CompA->bEnableUserTranscript = true;
CompB->bEnableUserTranscript = true;
CompA->TurnMode = EPS_AI_ConvAgent_TurnMode_ElevenLabs::Server;
CompB->TurnMode = EPS_AI_ConvAgent_TurnMode_ElevenLabs::Server;
CompA->bAllowInterruption = true;
CompB->bAllowInterruption = true;
// Make the two agents look at EACH OTHER (not at a bystanding player).
// Replicated, so the audience sees it correctly on every machine.
CompA->SetGazeOverrideTarget(AgentB);
CompB->SetGazeOverrideTarget(AgentA);
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Warning,
TEXT("[Dialogue] Reminder: both agents must have an EMPTY FirstMessage in their AgentConfig "
"(a configured first message auto-greets on connect and collides with the opening)."));
// Reset state.
Floor = EFloor::None;
TurnId = 0;
bTextReady = false;
bSpeechDone = false;
bForwardedThisTurn = false;
PendingLine.Reset();
bOpeningSent = false;
bRunning = true;
CompA->StartConversation();
CompB->StartConversation();
TryKickoff();
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log,
TEXT("[Dialogue] started: %s <-> %s (starter=%s, maxTurns=%d, waitForSpeech=%d)."),
*GetNameSafe(AgentA), *GetNameSafe(AgentB),
bAgentAStarts ? *GetNameSafe(AgentA) : *GetNameSafe(AgentB),
MaxTurns, bWaitForSpeechToFinish ? 1 : 0);
}
void APS_AI_ConvAgent_AgentDialogue::StopDialogue(bool bEndConversations)
{
if (!bRunning && !bAddressingPlayer) { return; }
bRunning = false;
bAddressingPlayer = false;
AddressedByPlayer = nullptr;
EngagedFloor = EFloor::None;
SetActorTickEnabled(false);
if (UWorld* W = GetWorld())
{
W->GetTimerManager().ClearTimer(TurnTimer);
}
// Restore normal (player-tracking) gaze.
if (CompA) { CompA->SetGazeOverrideTarget(nullptr); }
if (CompB) { CompB->SetGazeOverrideTarget(nullptr); }
if (bEndConversations)
{
if (CompA) { CompA->EndConversation(); }
if (CompB) { CompB->EndConversation(); }
}
UnbindAll();
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log, TEXT("[Dialogue] stopped after %d turns."), TurnId);
OnDialogueFinished.Broadcast(TurnId);
}
void APS_AI_ConvAgent_AgentDialogue::BindAgent(UPS_AI_ConvAgent_ElevenLabsComponent* Comp, bool bIsA)
{
if (!Comp) { return; }
if (bIsA)
{
Comp->OnAgentTextResponse.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTextA);
Comp->OnAgentStoppedSpeaking.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleStoppedA);
Comp->OnAgentConnected.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleConnectedA);
// Per-agent interrupt/relay triggers (voice-onset = primary, transcript = words).
Comp->OnUserVoiceDetected.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleVoiceOnsetA);
Comp->OnAgentTranscript.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTranscriptA);
}
else
{
Comp->OnAgentTextResponse.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTextB);
Comp->OnAgentStoppedSpeaking.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleStoppedB);
Comp->OnAgentConnected.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleConnectedB);
Comp->OnUserVoiceDetected.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleVoiceOnsetB);
Comp->OnAgentTranscript.AddDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTranscriptB);
}
}
void APS_AI_ConvAgent_AgentDialogue::UnbindAll()
{
if (CompA)
{
CompA->OnAgentTextResponse.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTextA);
CompA->OnAgentStoppedSpeaking.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleStoppedA);
CompA->OnAgentConnected.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleConnectedA);
CompA->OnAgentTranscript.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTranscriptA);
CompA->OnUserVoiceDetected.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleVoiceOnsetA);
}
if (CompB)
{
CompB->OnAgentTextResponse.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTextB);
CompB->OnAgentStoppedSpeaking.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleStoppedB);
CompB->OnAgentConnected.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleConnectedB);
CompB->OnAgentTranscript.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleTranscriptB);
CompB->OnUserVoiceDetected.RemoveDynamic(this, &APS_AI_ConvAgent_AgentDialogue::HandleVoiceOnsetB);
}
}
void APS_AI_ConvAgent_AgentDialogue::HandleVoiceOnsetA() { OnPlayerSpoke(EFloor::A, FString()); }
void APS_AI_ConvAgent_AgentDialogue::HandleVoiceOnsetB() { OnPlayerSpoke(EFloor::B, FString()); }
void APS_AI_ConvAgent_AgentDialogue::HandleTranscriptA(const FPS_AI_ConvAgent_TranscriptSegment_ElevenLabs& Segment)
{
if (Segment.Speaker == TEXT("user")) { OnPlayerSpoke(EFloor::A, Segment.Text); }
}
void APS_AI_ConvAgent_AgentDialogue::HandleTranscriptB(const FPS_AI_ConvAgent_TranscriptSegment_ElevenLabs& Segment)
{
if (Segment.Speaker == TEXT("user")) { OnPlayerSpoke(EFloor::B, Segment.Text); }
}
void APS_AI_ConvAgent_AgentDialogue::OnPlayerSpoke(EFloor Which, const FString& PlayerWords)
{
if (bRunning)
{
// The player barged into the agent-to-agent dialogue: enter 3-way floor mode.
InterruptForPlayer(Which);
}
else if (bAddressingPlayer)
{
// Already talking to the player — did they turn to the OTHER agent?
if (Which != EngagedFloor) { SetEngaged(Which); }
}
else
{
return; // orchestrator idle
}
// Relay the player's actual words (transcript path) to the WITNESS agent, so it
// "overhears" the exchange and can reference it later. Only while in floor mode.
if (bAddressingPlayer && !PlayerWords.IsEmpty() && !OverheardPrefix.IsEmpty())
{
RelayToBystander(Which, OverheardPrefix + TEXT("(la personne) ") + PlayerWords);
}
}
void APS_AI_ConvAgent_AgentDialogue::InterruptForPlayer(EFloor Engaged)
{
APawn* Player = ResolveSpeakingPlayer();
if (!Player)
{
// Couldn't identify the speaker — fall back to a plain stop.
StopDialogue(/*bEndConversations=*/false);
return;
}
// Enter 3-way "floor" mode: stop the agent↔agent forwarding, turn BOTH agents to the
// player, and keep relaying the exchange to the witness agent. Agents stay connected;
// the orchestrator ticks to auto-release once the player leaves both (see Tick).
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log,
TEXT("[Dialogue] player '%s' spoke to %s — entering 3-way floor mode."),
*GetNameSafe(Player), FloorName((uint8)Engaged));
bRunning = false;
if (UWorld* W = GetWorld())
{
W->GetTimerManager().ClearTimer(TurnTimer);
}
// The player barged in — cut BOTH agents' current speech now. The engaged agent
// is already interrupted by the player's mic (ElevenLabs), but the WITNESS agent
// never hears the player, so without this it keeps finishing its dealer<->client
// line ("...I'll give you the cash...") while the player is talking. Interrupting
// leaves the conversations open; the witness still receives the contextual_updates.
if (CompA) { CompA->InterruptAgent(); }
if (CompB) { CompB->InterruptAgent(); }
// Keep the agent bindings: in floor mode the text/transcript handlers RELAY to the
// witness instead of forwarding (their dialogue logic no-ops while !bRunning).
if (CompA) { CompA->SetGazeOverrideTarget(Player); }
if (CompB) { CompB->SetGazeOverrideTarget(Player); }
bAddressingPlayer = true;
AddressedByPlayer = Player;
EngagedFloor = EFloor::None; // force SetEngaged to send the reframe messages
SetEngaged(Engaged);
SetActorTickEnabled(true);
}
void APS_AI_ConvAgent_AgentDialogue::SetEngaged(EFloor Which)
{
if (Which == EFloor::None || Which == EngagedFloor) { return; }
EngagedFloor = Which;
UPS_AI_ConvAgent_ElevenLabsComponent* Engaged = (Which == EFloor::A) ? CompA.Get() : CompB.Get();
UPS_AI_ConvAgent_ElevenLabsComponent* Bystander = (Which == EFloor::A) ? CompB.Get() : CompA.Get();
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log,
TEXT("[Dialogue] player now addressing %s (the other agent becomes witness)."),
FloorName((uint8)Which));
// Silent reframe (contextual_update = no reply triggered).
if (Engaged && !InterruptContextMessage.IsEmpty())
{
Engaged->SendContextualUpdate(InterruptContextMessage);
}
if (Bystander && !BystanderContextMessage.IsEmpty())
{
Bystander->SendContextualUpdate(BystanderContextMessage);
}
}
void APS_AI_ConvAgent_AgentDialogue::RelayToBystander(EFloor EngagedSide, const FString& FormattedLine)
{
UPS_AI_ConvAgent_ElevenLabsComponent* Bystander = (EngagedSide == EFloor::A) ? CompB.Get() : CompA.Get();
if (Bystander)
{
Bystander->SendContextualUpdate(FormattedLine);
}
}
void APS_AI_ConvAgent_AgentDialogue::Tick(float DeltaSeconds)
{
Super::Tick(DeltaSeconds);
if (!bAddressingPlayer) { return; }
APawn* Player = AddressedByPlayer.Get();
if (!Player)
{
EndPlayerAddress();
return;
}
// Release once the player has walked farther than the threshold from BOTH agents.
const FVector P = Player->GetActorLocation();
const float Threshold = PlayerAddressReleaseDistance;
const bool bFarFromA = !AgentA || FVector::Dist(P, AgentA->GetActorLocation()) > Threshold;
const bool bFarFromB = !AgentB || FVector::Dist(P, AgentB->GetActorLocation()) > Threshold;
if (bFarFromA && bFarFromB)
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log,
TEXT("[Dialogue] player left both agents — releasing."));
EndPlayerAddress();
}
}
void APS_AI_ConvAgent_AgentDialogue::EndPlayerAddress()
{
bAddressingPlayer = false;
AddressedByPlayer = nullptr;
EngagedFloor = EFloor::None;
SetActorTickEnabled(false);
// Hand the agents back to normal gaze and end their (now idle) conversations.
if (CompA) { CompA->SetGazeOverrideTarget(nullptr); CompA->EndConversation(); }
if (CompB) { CompB->SetGazeOverrideTarget(nullptr); CompB->EndConversation(); }
UnbindAll(); // floor mode kept the bindings active — release them now.
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log,
TEXT("[Dialogue] player address ended — agents released (after %d turns)."), TurnId);
OnDialogueFinished.Broadcast(TurnId);
}
APawn* APS_AI_ConvAgent_AgentDialogue::ResolveSpeakingPlayer() const
{
// The player who spoke is the active speaker on one of the agents (a pawn that is
// neither AgentA nor AgentB), else a connected non-agent pawn, else the local player.
AActor* A = AgentA.Get();
AActor* B = AgentB.Get();
auto IsPlayer = [A, B](APawn* P) -> bool { return P && P != A && P != B; };
if (CompA && IsPlayer(CompA->NetActiveSpeakerPawn)) { return CompA->NetActiveSpeakerPawn; }
if (CompB && IsPlayer(CompB->NetActiveSpeakerPawn)) { return CompB->NetActiveSpeakerPawn; }
if (CompA) { for (APawn* P : CompA->NetConnectedPawns) { if (IsPlayer(P)) { return P; } } }
if (CompB) { for (APawn* P : CompB->NetConnectedPawns) { if (IsPlayer(P)) { return P; } } }
// Voice-onset path: NetActiveSpeakerPawn is usually unset (the player never formally
// joined the agent). Pick the player pawn NEAREST to either agent.
UWorld* W = GetWorld();
if (!W) { return nullptr; }
APawn* Nearest = nullptr;
double BestDistSq = TNumericLimits<double>::Max();
for (FConstPlayerControllerIterator It = W->GetPlayerControllerIterator(); It; ++It)
{
APlayerController* PC = It->Get();
APawn* P = PC ? PC->GetPawn() : nullptr;
if (!IsPlayer(P)) { continue; }
const FVector L = P->GetActorLocation();
double D = TNumericLimits<double>::Max();
if (A) { D = FMath::Min(D, FVector::DistSquared(L, A->GetActorLocation())); }
if (B) { D = FMath::Min(D, FVector::DistSquared(L, B->GetActorLocation())); }
if (D < BestDistSq) { BestDistSq = D; Nearest = P; }
}
if (Nearest) { return Nearest; }
if (APlayerController* PC = W->GetFirstPlayerController())
{
return PC->GetPawn();
}
return nullptr;
}
void APS_AI_ConvAgent_AgentDialogue::BeginTurn(EFloor NewFloor)
{
Floor = NewFloor;
bTextReady = false;
bSpeechDone = false;
bForwardedThisTurn = false;
PendingLine.Reset();
}
bool APS_AI_ConvAgent_AgentDialogue::IsFloor(bool bFromA) const
{
return (bFromA && Floor == EFloor::A) || (!bFromA && Floor == EFloor::B);
}
void APS_AI_ConvAgent_AgentDialogue::TryKickoff()
{
if (!bRunning || bOpeningSent) { return; }
const bool bBothReady = CompA && CompB && CompA->IsConnected() && CompB->IsConnected();
if (!bBothReady)
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log, TEXT("[Dialogue] Kickoff waiting: A.connected=%d B.connected=%d"),
(CompA && CompA->IsConnected()) ? 1 : 0, (CompB && CompB->IsConnected()) ? 1 : 0);
return; // wait for the other OnAgentConnected
}
bOpeningSent = true;
BeginTurn(bAgentAStarts ? EFloor::A : EFloor::B);
UPS_AI_ConvAgent_ElevenLabsComponent* Starter = (Floor == EFloor::A) ? CompA : CompB;
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log, TEXT("[Dialogue] FLOOR=%s SEND-OPENING '%s'"),
FloorName((uint8)Floor), *OpeningMessage);
Starter->SendTextMessage(OpeningMessage);
}
void APS_AI_ConvAgent_AgentDialogue::TryForward()
{
if (!bRunning || bForwardedThisTurn) { return; }
// Need the floor-holder's full text; and, unless we're on a headless server,
// also require it to have finished speaking so the voices never overlap.
if (!bTextReady) { return; }
if (bWaitForSpeechToFinish && !bSpeechDone) { return; }
UPS_AI_ConvAgent_ElevenLabsComponent* Listener = (Floor == EFloor::A) ? CompB : CompA;
AActor* SpeakerActor = (Floor == EFloor::A) ? AgentA : AgentB;
if (!Listener) { return; }
if (MaxTurns > 0 && TurnId >= MaxTurns)
{
StopDialogue(false);
return;
}
bForwardedThisTurn = true;
++TurnId;
const FString Line = PendingLine;
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Log, TEXT("[Dialogue] turn=%d FORWARD %s->%s : '%.60s'"),
TurnId, FloorName((uint8)Floor), FloorName((uint8)(Floor == EFloor::A ? EFloor::B : EFloor::A)), *Line);
OnDialogueTurn.Broadcast(TurnId, SpeakerActor, Line);
// Hand the floor to the listener BEFORE sending, so the listener's upcoming
// text/speech events are attributed to it (and the old speaker's late/duplicate
// events are now non-floor and get ignored).
BeginTurn(Floor == EFloor::A ? EFloor::B : EFloor::A);
auto DoSend = [this, Line, Listener]()
{
if (bRunning && Listener && Listener->IsConnected())
{
Listener->SendTextMessage(Line);
}
};
if (TurnDelaySeconds > 0.0f)
{
if (UWorld* W = GetWorld())
{
W->GetTimerManager().SetTimer(TurnTimer, FTimerDelegate::CreateLambda(DoSend), TurnDelaySeconds, false);
}
else { DoSend(); }
}
else { DoSend(); }
}
// ── Delegate handlers ─────────────────────────────────────────────────────────
// Every handler ignores events from the agent that does NOT hold the floor.
void APS_AI_ConvAgent_AgentDialogue::HandleTextA(const FString& Text)
{
if (bAddressingPlayer)
{
// Floor mode: the ENGAGED agent's spoken line is overheard by the witness.
if (EngagedFloor == EFloor::A && !Text.IsEmpty() && !OverheardPrefix.IsEmpty())
{
RelayToBystander(EFloor::A, OverheardPrefix + TEXT("(l'autre personnage) ") + Text);
}
return;
}
if (!bRunning) { return; }
if (!IsFloor(/*bFromA=*/true))
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Verbose, TEXT("[Dialogue] IGNORED text from NON-FLOOR A (floor=%s)"), FloorName((uint8)Floor));
return;
}
if (Text.IsEmpty()) { return; }
PendingLine = Text;
bTextReady = true;
TryForward();
}
void APS_AI_ConvAgent_AgentDialogue::HandleTextB(const FString& Text)
{
if (bAddressingPlayer)
{
// Floor mode: the ENGAGED agent's spoken line is overheard by the witness.
if (EngagedFloor == EFloor::B && !Text.IsEmpty() && !OverheardPrefix.IsEmpty())
{
RelayToBystander(EFloor::B, OverheardPrefix + TEXT("(l'autre personnage) ") + Text);
}
return;
}
if (!bRunning) { return; }
if (!IsFloor(/*bFromA=*/false))
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Verbose, TEXT("[Dialogue] IGNORED text from NON-FLOOR B (floor=%s)"), FloorName((uint8)Floor));
return;
}
if (Text.IsEmpty()) { return; }
PendingLine = Text;
bTextReady = true;
TryForward();
}
void APS_AI_ConvAgent_AgentDialogue::HandleStoppedA()
{
if (!bRunning) { return; }
if (!IsFloor(/*bFromA=*/true))
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Verbose, TEXT("[Dialogue] IGNORED stopped from NON-FLOOR A (floor=%s)"), FloorName((uint8)Floor));
return;
}
bSpeechDone = true;
TryForward();
}
void APS_AI_ConvAgent_AgentDialogue::HandleStoppedB()
{
if (!bRunning) { return; }
if (!IsFloor(/*bFromA=*/false))
{
UE_LOG(LogPS_AI_ConvAgent_Dialogue, Verbose, TEXT("[Dialogue] IGNORED stopped from NON-FLOOR B (floor=%s)"), FloorName((uint8)Floor));
return;
}
bSpeechDone = true;
TryForward();
}
void APS_AI_ConvAgent_AgentDialogue::HandleConnectedA(const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& Info)
{
TryKickoff();
}
void APS_AI_ConvAgent_AgentDialogue::HandleConnectedB(const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& Info)
{
TryKickoff();
}

View File

@@ -1,7 +1,98 @@
// Copyright ASTERION. All Rights Reserved. // Copyright ASTERION. All Rights Reserved.
#include "PS_AI_ConvAgent_BlueprintLibrary.h" #include "PS_AI_ConvAgent_BlueprintLibrary.h"
#include "PS_AI_ConvAgent.h"
#include "PS_AI_ConvAgent_SaveGame_ElevenLabs.h"
#include "Components/SkeletalMeshComponent.h" #include "Components/SkeletalMeshComponent.h"
#include "Kismet/GameplayStatics.h"
#include "Misc/AES.h"
namespace
{
// In-memory cache of the client API key (PLAINTEXT). This BlueprintLibrary is
// the single owner of the key — every consumer (WebSocket auth, region probe,
// editor sync tools) reads it via GetElevenLabsAPIKey(). The SaveGame on disk
// is the persistent backing store (AES-encrypted); this cache avoids a disk
// read + decrypt on every access. All access is on the game thread, so no
// synchronization is needed.
bool GElevenLabsKeyLoaded = false;
FString GElevenLabsKeyCache;
// 32-byte AES key used to obfuscate the API key at rest in the .sav file.
// IMPORTANT: this is embedded in the binary, so it is NOT real security — a
// determined attacker can extract it. It only raises the bar against casual
// inspection of the save file (agreed "trusted distribution" threat model).
void GetObfuscationAESKey(FAES::FAESKey& OutKey)
{
static const uint8 KeyBytes[FAES::FAESKey::KeySize] = {
0x9A, 0x3C, 0x71, 0x05, 0xE2, 0x4F, 0xB8, 0x16,
0x2D, 0xC4, 0x88, 0x5B, 0x37, 0xF1, 0x60, 0xAA,
0x14, 0x7E, 0xD9, 0x42, 0x6B, 0x90, 0x0C, 0xF5,
0x53, 0x21, 0xBE, 0x8D, 0x47, 0xA0, 0x39, 0xCF
};
FMemory::Memcpy(OutKey.Key, KeyBytes, FAES::FAESKey::KeySize);
}
/** Encrypt a plaintext key to AES block-padded bytes. Sets OutPlainLen to the
* original UTF-8 byte length so padding can be stripped on decrypt. */
TArray<uint8> EncryptElevenLabsKey(const FString& Plain, int32& OutPlainLen)
{
FTCHARToUTF8 Utf8(*Plain);
OutPlainLen = Utf8.Length();
TArray<uint8> Buffer;
Buffer.Append(reinterpret_cast<const uint8*>(Utf8.Get()), Utf8.Length());
if (Buffer.Num() == 0)
{
return Buffer; // empty key → empty blob
}
// Pad up to a multiple of the AES block size (required by FAES).
const int32 Remainder = Buffer.Num() % FAES::AESBlockSize;
if (Remainder != 0)
{
Buffer.AddZeroed(FAES::AESBlockSize - Remainder);
}
FAES::FAESKey Key;
GetObfuscationAESKey(Key);
FAES::EncryptData(Buffer.GetData(), Buffer.Num(), Key);
return Buffer;
}
/** Decrypt AES block-padded bytes back to the original plaintext key. */
FString DecryptElevenLabsKey(const TArray<uint8>& Encrypted, int32 PlainLen)
{
if (Encrypted.Num() == 0 || PlainLen <= 0 || (Encrypted.Num() % FAES::AESBlockSize) != 0)
{
return FString();
}
TArray<uint8> Buffer = Encrypted;
FAES::FAESKey Key;
GetObfuscationAESKey(Key);
FAES::DecryptData(Buffer.GetData(), Buffer.Num(), Key);
// Strip padding back to the original UTF-8 length, then null-terminate.
const int32 Len = FMath::Clamp(PlainLen, 0, Buffer.Num());
Buffer.SetNum(Len);
Buffer.Add(0);
return FString(FUTF8ToTCHAR(reinterpret_cast<const ANSICHAR*>(Buffer.GetData())).Get());
}
/** Load the API-key SaveGame from its slot, or null if none exists yet. */
UPS_AI_ConvAgent_SaveGame_ElevenLabs* LoadElevenLabsKeySaveGame()
{
const TCHAR* Slot = UPS_AI_ConvAgent_SaveGame_ElevenLabs::GetSlotName();
const int32 UserIndex = UPS_AI_ConvAgent_SaveGame_ElevenLabs::GetUserIndex();
if (UGameplayStatics::DoesSaveGameExist(Slot, UserIndex))
{
return Cast<UPS_AI_ConvAgent_SaveGame_ElevenLabs>(
UGameplayStatics::LoadGameFromSlot(Slot, UserIndex));
}
return nullptr;
}
}
void UPS_AI_ConvAgent_BlueprintLibrary::SetPostProcessAnimBlueprint( void UPS_AI_ConvAgent_BlueprintLibrary::SetPostProcessAnimBlueprint(
USkeletalMeshComponent* SkelMeshComp, USkeletalMeshComponent* SkelMeshComp,
@@ -15,3 +106,73 @@ void UPS_AI_ConvAgent_BlueprintLibrary::SetPostProcessAnimBlueprint(
SkelMeshComp->SetOverridePostProcessAnimBP(AnimBPClass); SkelMeshComp->SetOverridePostProcessAnimBP(AnimBPClass);
} }
// ── ElevenLabs API key (per-user, client-provided) ──────────────────────────
void UPS_AI_ConvAgent_BlueprintLibrary::SetElevenLabsAPIKey(const FString& Key)
{
// 1) Persist to the SaveGame slot (writes to disk immediately).
UPS_AI_ConvAgent_SaveGame_ElevenLabs* Save = LoadElevenLabsKeySaveGame();
if (!Save)
{
Save = Cast<UPS_AI_ConvAgent_SaveGame_ElevenLabs>(
UGameplayStatics::CreateSaveGameObject(UPS_AI_ConvAgent_SaveGame_ElevenLabs::StaticClass()));
}
if (!Save)
{
UE_LOG(LogTemp, Warning, TEXT("[PS_AI_ConvAgent] SetElevenLabsAPIKey: failed to create SaveGame object."));
return;
}
int32 PlainLen = 0;
Save->EncryptedAPIKey = EncryptElevenLabsKey(Key, PlainLen);
Save->PlainKeyLength = PlainLen;
const bool bSaved = UGameplayStatics::SaveGameToSlot(
Save,
UPS_AI_ConvAgent_SaveGame_ElevenLabs::GetSlotName(),
UPS_AI_ConvAgent_SaveGame_ElevenLabs::GetUserIndex());
if (!bSaved)
{
UE_LOG(LogTemp, Warning, TEXT("[PS_AI_ConvAgent] SetElevenLabsAPIKey: SaveGameToSlot failed "
"(check write permissions at the install location)."));
}
// 2) Update the in-memory cache so the new key takes effect immediately
// (next conversation / region probe / editor sync) without a restart.
GElevenLabsKeyCache = Key;
GElevenLabsKeyLoaded = true;
}
FString UPS_AI_ConvAgent_BlueprintLibrary::GetElevenLabsAPIKey()
{
// Lazy-load once from the SaveGame (decrypting), then serve from the cache.
if (!GElevenLabsKeyLoaded)
{
GElevenLabsKeyLoaded = true;
if (const UPS_AI_ConvAgent_SaveGame_ElevenLabs* Save = LoadElevenLabsKeySaveGame())
{
GElevenLabsKeyCache = DecryptElevenLabsKey(Save->EncryptedAPIKey, Save->PlainKeyLength);
}
}
return GElevenLabsKeyCache;
}
bool UPS_AI_ConvAgent_BlueprintLibrary::HasElevenLabsAPIKey()
{
return !GetElevenLabsAPIKey().IsEmpty();
}
void UPS_AI_ConvAgent_BlueprintLibrary::ClearElevenLabsAPIKey()
{
const TCHAR* Slot = UPS_AI_ConvAgent_SaveGame_ElevenLabs::GetSlotName();
const int32 UserIndex = UPS_AI_ConvAgent_SaveGame_ElevenLabs::GetUserIndex();
if (UGameplayStatics::DoesSaveGameExist(Slot, UserIndex))
{
UGameplayStatics::DeleteGameInSlot(Slot, UserIndex);
}
// Clear the cache too so the running session stops using the old key.
GElevenLabsKeyCache.Empty();
GElevenLabsKeyLoaded = true;
}

View File

@@ -232,18 +232,17 @@ void UPS_AI_ConvAgent_BodyExpressionComponent::PickAndSwitchAnim()
void UPS_AI_ConvAgent_BodyExpressionComponent::OnConversationConnected( void UPS_AI_ConvAgent_BodyExpressionComponent::OnConversationConnected(
const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& Info) const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& Info)
{ {
bActive = true; // Don't activate yet — wait for the agent to actually speak (OnSpeakingStarted).
// This prevents body expressions from playing while the NPC is still walking
// and the user hasn't said anything yet.
bIsSpeaking = false; bIsSpeaking = false;
LastEventName = TEXT("Connected"); LastEventName = TEXT("Connected");
LastEventWorldTime = GetWorld() ? GetWorld()->GetTimeSeconds() : 0.0f; LastEventWorldTime = GetWorld() ? GetWorld()->GetTimeSeconds() : 0.0f;
// Start with an idle anim
PickAndSwitchAnim();
if (bDebug) if (bDebug)
{ {
UE_LOG(LogPS_AI_ConvAgent_BodyExpr, Log, UE_LOG(LogPS_AI_ConvAgent_BodyExpr, Log,
TEXT("Conversation connected — body expression activating (idle).")); TEXT("Conversation connected — waiting for agent speech to activate body expressions."));
} }
} }
@@ -268,6 +267,12 @@ void UPS_AI_ConvAgent_BodyExpressionComponent::OnConversationDisconnected(
void UPS_AI_ConvAgent_BodyExpressionComponent::OnSpeakingStarted() void UPS_AI_ConvAgent_BodyExpressionComponent::OnSpeakingStarted()
{ {
// Activate on first speech if not already active (deferred from OnConversationConnected)
if (!bActive)
{
bActive = true;
}
bIsSpeaking = true; bIsSpeaking = true;
LastEventName = TEXT("SpeakStart"); LastEventName = TEXT("SpeakStart");
LastEventWorldTime = GetWorld() ? GetWorld()->GetTimeSeconds() : 0.0f; LastEventWorldTime = GetWorld() ? GetWorld()->GetTimeSeconds() : 0.0f;
@@ -278,7 +283,7 @@ void UPS_AI_ConvAgent_BodyExpressionComponent::OnSpeakingStarted()
if (bDebug) if (bDebug)
{ {
UE_LOG(LogPS_AI_ConvAgent_BodyExpr, Log, UE_LOG(LogPS_AI_ConvAgent_BodyExpr, Log,
TEXT("Agent started speaking — switching to speaking body anim.")); TEXT("Agent started speaking — body expression activating + speaking anim."));
} }
} }

View File

@@ -10,6 +10,7 @@
#include "PS_AI_ConvAgent_InteractionSubsystem.h" #include "PS_AI_ConvAgent_InteractionSubsystem.h"
#include "PS_AI_ConvAgent_InteractionComponent.h" #include "PS_AI_ConvAgent_InteractionComponent.h"
#include "PS_AI_ConvAgent.h" #include "PS_AI_ConvAgent.h"
#include "PS_AI_ConvAgent_BlueprintLibrary.h"
#include "Components/AudioComponent.h" #include "Components/AudioComponent.h"
#include "Sound/SoundAttenuation.h" #include "Sound/SoundAttenuation.h"
@@ -101,10 +102,9 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::TickComponent(float DeltaTime, ELevel
{ {
Super::TickComponent(DeltaTime, TickType, ThisTickFunction); Super::TickComponent(DeltaTime, TickType, ThisTickFunction);
// ── ForceDisableConversation blend-out monitoring ───────────────────── // ── Conversation disable: blend-out monitoring ─────────────────────────
// After ForceDisableConversation(), sub-components are blending their // After ForceDisableConversation(), sub-components blend their CurrentActiveAlpha
// CurrentActiveAlpha to 0. Once all are at (near) zero, fire OnReadyForAction // to 0. Once all near zero (or timeout), clear the wait flag.
// so the game can start the physical action.
if (bWaitingForBlendOut) if (bWaitingForBlendOut)
{ {
BlendOutElapsedTime += DeltaTime; BlendOutElapsedTime += DeltaTime;
@@ -139,23 +139,33 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::TickComponent(float DeltaTime, ELevel
bWaitingForBlendOut = false; bWaitingForBlendOut = false;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("ForceDisableConversation: all components blended to neutral — firing OnReadyForAction for action '%s'."), TEXT("ForceDisableConversation: all components blended to neutral."));
*PendingActionName);
OnReadyForAction.Broadcast(PendingActionName);
} }
else if (bTimedOut) else if (bTimedOut)
{ {
// Safety timeout — some component didn't reach alpha 0 in time. // Safety timeout — some component didn't reach alpha 0 in time.
// Fire OnReadyForAction anyway to avoid blocking the game action forever.
bWaitingForBlendOut = false; bWaitingForBlendOut = false;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("ForceDisableConversation: blend-out timed out after %.1fs — not all components at neutral. Firing OnReadyForAction for action '%s' anyway."), TEXT("ForceDisableConversation: blend-out timed out after %.1fs — not all components at neutral."),
BlendOutTimeoutSeconds, *PendingActionName); BlendOutTimeoutSeconds);
OnReadyForAction.Broadcast(PendingActionName);
} }
} }
// ── Action flow: fire OnReadyForAction when ready ──────────────────────
// An action was requested via OnAgentActionRequested. Fire OnReadyForAction
// as soon as the agent is ready:
// - If ForceDisableConversation was called, wait for blend-out to complete.
// - Otherwise, fire on the next tick (quasi-immediate).
if (!PendingActionName.IsEmpty() && !bWaitingForBlendOut)
{
const FString ActionToFire = PendingActionName;
PendingActionName.Empty();
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("OnReadyForAction: firing for action '%s'."), *ActionToFire);
OnReadyForAction.Broadcast(ActionToFire);
}
// Response timeout: if the server hasn't started generating within ResponseTimeoutSeconds // Response timeout: if the server hasn't started generating within ResponseTimeoutSeconds
// after the user stopped speaking, notify Blueprint so it can react (e.g. show "try again"). // after the user stopped speaking, notify Blueprint so it can react (e.g. show "try again").
if (bWaitingForAgentResponse && ResponseTimeoutSeconds > 0.0f && TurnEndTime > 0.0) if (bWaitingForAgentResponse && ResponseTimeoutSeconds > 0.0f && TurnEndTime > 0.0)
@@ -400,7 +410,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::TickComponent(float DeltaTime, ELevel
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_ElevenLabsComponent::StartConversation() void UPS_AI_ConvAgent_ElevenLabsComponent::StartConversation()
{ {
if (bConversationDisabledByAction) if (bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("StartConversation: blocked — conversation disabled by ForceDisableConversation(). Call ForceEnableConversation() first.")); TEXT("StartConversation: blocked — conversation disabled by ForceDisableConversation(). Call ForceEnableConversation() first."));
@@ -492,6 +502,9 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StartConversation_Internal()
void UPS_AI_ConvAgent_ElevenLabsComponent::EndConversation() void UPS_AI_ConvAgent_ElevenLabsComponent::EndConversation()
{ {
// Clear any pending listen request so it doesn't leak across sessions.
bPendingStartListening = false;
if (GetOwnerRole() == ROLE_Authority) if (GetOwnerRole() == ROLE_Authority)
{ {
// Standalone / listen-server: leave via the local player controller. // Standalone / listen-server: leave via the local player controller.
@@ -525,7 +538,12 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StartListening()
const bool bEffectivelyConnected = IsConnected() || (GetOwnerRole() != ROLE_Authority && bNetIsConversing); const bool bEffectivelyConnected = IsConnected() || (GetOwnerRole() != ROLE_Authority && bNetIsConversing);
if (!bEffectivelyConnected) if (!bEffectivelyConnected)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, TEXT("StartListening: not connected (IsConnected=%s bNetIsConversing=%s Role=%d)."), // Not ready yet — defer until HandleConnected fires.
// Prevents silent audio loss when StartListening is called synchronously
// after StartConversation (async WebSocket handshake).
bPendingStartListening = true;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("StartListening deferred: WebSocket not connected yet (IsConnected=%s bNetIsConversing=%s Role=%d). Will open mic on connect."),
IsConnected() ? TEXT("true") : TEXT("false"), IsConnected() ? TEXT("true") : TEXT("false"),
bNetIsConversing ? TEXT("true") : TEXT("false"), bNetIsConversing ? TEXT("true") : TEXT("false"),
static_cast<int32>(GetOwnerRole())); static_cast<int32>(GetOwnerRole()));
@@ -621,6 +639,10 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StartListening()
void UPS_AI_ConvAgent_ElevenLabsComponent::StopListening() void UPS_AI_ConvAgent_ElevenLabsComponent::StopListening()
{ {
// Also cancel any pending-start request (e.g. StartListening was deferred
// before WS connect and the user now wants to cancel).
bPendingStartListening = false;
if (!bIsListening) return; if (!bIsListening) return;
bIsListening = false; bIsListening = false;
@@ -717,6 +739,23 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StopListening()
} }
} }
void UPS_AI_ConvAgent_ElevenLabsComponent::SendContextualUpdate(const FString& Text)
{
// Server-authoritative: the WebSocket lives on the server. The agent-to-agent
// orchestrator (server) uses this to reframe the agents when the player takes over.
if (GetOwnerRole() == ROLE_Authority)
{
if (IsConnected() && WebSocketProxy)
{
WebSocketProxy->SendContextualUpdate(Text);
}
else
{
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, TEXT("SendContextualUpdate: not connected."));
}
}
}
void UPS_AI_ConvAgent_ElevenLabsComponent::SendTextMessage(const FString& Text) void UPS_AI_ConvAgent_ElevenLabsComponent::SendTextMessage(const FString& Text)
{ {
if (GetOwnerRole() == ROLE_Authority) if (GetOwnerRole() == ROLE_Authority)
@@ -781,21 +820,19 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::InterruptAgent()
// ForceDisableConversation / ForceEnableConversation // ForceDisableConversation / ForceEnableConversation
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_ElevenLabsComponent::ForceDisableConversation(const FString& ActionName) void UPS_AI_ConvAgent_ElevenLabsComponent::ForceDisableConversation()
{ {
if (bConversationDisabledByAction) if (bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("ForceDisableConversation: already disabled (pending action: %s)."), *PendingActionName); TEXT("ForceDisableConversation: already disabled."));
return; return;
} }
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("ForceDisableConversation: shutting down conversation for action '%s', blending to neutral."), *ActionName); TEXT("ForceDisableConversation: shutting down conversation, blending to neutral."));
PendingActionName = ActionName; bConversationForceDisabled = true;
bConversationDisabledByAction = true;
bWaitingForBlendOut = true; bWaitingForBlendOut = true;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
@@ -856,7 +893,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ForceDisableConversation(const FStrin
void UPS_AI_ConvAgent_ElevenLabsComponent::ForceEnableConversation() void UPS_AI_ConvAgent_ElevenLabsComponent::ForceEnableConversation()
{ {
if (!bConversationDisabledByAction) if (!bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("ForceEnableConversation: not currently disabled.")); TEXT("ForceEnableConversation: not currently disabled."));
@@ -866,7 +903,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ForceEnableConversation()
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("ForceEnableConversation: conversation re-enabled.")); TEXT("ForceEnableConversation: conversation re-enabled."));
bConversationDisabledByAction = false; bConversationForceDisabled = false;
bWaitingForBlendOut = false; bWaitingForBlendOut = false;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
@@ -880,6 +917,27 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::FeedExternalAudio(const TArray<float>
// Same logic as OnMicrophoneDataCaptured but called from an external source // Same logic as OnMicrophoneDataCaptured but called from an external source
// (e.g. InteractionComponent on the pawn) instead of a local mic component. // (e.g. InteractionComponent on the pawn) instead of a local mic component.
// On server: check local WebSocket. On client: check replicated conversation state. // On server: check local WebSocket. On client: check replicated conversation state.
// Local voice-onset detection: broadcast on the FIRST voiced frame BEFORE the
// listening/echo gates below, so an agent-to-agent dialogue can be interrupted the
// instant the player speaks — no "listening" state and no ElevenLabs STT round-trip
// needed. Debounced to fire once per utterance. (OnAudioCaptured is marshaled to the
// game thread by the mic component, so this runs on the game thread — safe to broadcast.)
{
float OnsetSumSq = 0.0f;
for (const float S : FloatPCM) { OnsetSumSq += S * S; }
const float OnsetRms = FMath::Sqrt(OnsetSumSq / FMath::Max(1, FloatPCM.Num()));
if (OnsetRms >= 0.02f)
{
const double NowSec = FPlatformTime::Seconds();
if (NowSec - LastUserVoiceOnset > 0.5)
{
LastUserVoiceOnset = NowSec;
OnUserVoiceDetected.Broadcast();
}
}
}
const bool bCanSend = (GetOwnerRole() == ROLE_Authority) ? IsConnected() : bNetIsConversing; const bool bCanSend = (GetOwnerRole() == ROLE_Authority) ? IsConnected() : bNetIsConversing;
if (!bCanSend || !bIsListening) return; if (!bCanSend || !bIsListening) return;
@@ -889,13 +947,15 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::FeedExternalAudio(const TArray<float>
return; return;
} }
// ── Network silence gate ──────────────────────────────────────────────── // ── Network silence gate (clients only) ─────────────────────────────────
// On clients the mic audio is sent via unreliable RPCs (~3200 bytes every // On clients the mic audio is sent via unreliable RPCs (~256 Kbits/s).
// 100ms = ~256 Kbits/s). Sending silence saturates the connection and // Streaming silence the whole session saturates the connection and starves
// starves movement replication, causing chaotic teleporting. // movement replication (teleporting). But ElevenLabs Server VAD needs the
// Skip silent chunks on clients only — the server path uses a local // silence AFTER speech to detect end-of-turn — dropping it entirely means the
// WebSocket and doesn't touch the network, so it keeps the full stream // agent hears the words but never the pause, so it never replies.
// for proper ElevenLabs VAD (voice-activity detection). // Compromise: keep streaming a short silence "tail" after the last voiced
// chunk (MicSilenceTailSeconds), then drop silence during long idle gaps.
// The authority path is untouched — it always streams the full mic.
if (GetOwnerRole() != ROLE_Authority) if (GetOwnerRole() != ROLE_Authority)
{ {
float SumSq = 0.0f; float SumSq = 0.0f;
@@ -904,11 +964,24 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::FeedExternalAudio(const TArray<float>
SumSq += Sample * Sample; SumSq += Sample * Sample;
} }
const float Rms = FMath::Sqrt(SumSq / FMath::Max(1, FloatPCM.Num())); const float Rms = FMath::Sqrt(SumSq / FMath::Max(1, FloatPCM.Num()));
if (Rms < 0.005f) // ~-46 dBFS — well below any speech level
const double NowSec = FPlatformTime::Seconds();
if (Rms >= 0.005f) // ~-46 dBFS — above this is voiced speech.
{
LastMicVoicedTime = NowSec; // Voiced: always send, remember when.
}
else
{
// Silence: send only within the tail window after the last voiced
// chunk (so VAD sees end-of-turn); drop once the tail has elapsed.
const bool bWithinTail = (LastMicVoicedTime > 0.0)
&& (NowSec - LastMicVoicedTime) <= MicSilenceTailSeconds;
if (!bWithinTail)
{ {
return; return;
} }
} }
}
TArray<uint8> PCMBytes = FloatPCMToInt16Bytes(FloatPCM); TArray<uint8> PCMBytes = FloatPCMToInt16Bytes(FloatPCM);
@@ -1069,7 +1142,14 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::HandleConnected(const FPS_AI_ConvAgen
{ {
StartListening(); StartListening();
} }
// Consume any pending listen request deferred before WebSocket was ready.
else if (bPendingStartListening)
{
bPendingStartListening = false;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("HandleConnected: consuming pending StartListening request."));
StartListening();
}
} }
void UPS_AI_ConvAgent_ElevenLabsComponent::HandleDisconnected(int32 StatusCode, const FString& Reason) void UPS_AI_ConvAgent_ElevenLabsComponent::HandleDisconnected(int32 StatusCode, const FString& Reason)
@@ -1182,41 +1262,123 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::HandleAudioReceived(const TArray<uint
// Network: broadcast audio to all clients. // Network: broadcast audio to all clients.
if (GetOwnerRole() == ROLE_Authority) if (GetOwnerRole() == ROLE_Authority)
{ {
// Net-audio size debug: toggle at runtime with
// ps.ai.ConvAgent.Debug.ElevenLabs 1 (or set bDebug on the component).
const int32 NetDbgCVar = CVarDebugElevenLabs.GetValueOnGameThread();
const bool bNetAudioDebug = (NetDbgCVar >= 0) ? (NetDbgCVar > 0) : bDebug;
if (OpusEncoder.IsValid()) if (OpusEncoder.IsValid())
{ {
// Opus path: compress then send. // Opus path: compress then send, in sub-blocks.
//
// A whole ElevenLabs message can be several seconds. Two hard limits
// force us to split it before each Encode()/RPC:
// 1. UE's Opus encoder caps ONE packet at 76 frames (~1.52s) — frames
// beyond that are silently dropped at encode (VoiceCodecOpus.cpp).
// 2. The client decode buffer only holds ~44 frames (~0.88s) before
// the engine decoder aborts with "Decompression buffer too small"
// and drops the packet tail (MulticastReceiveAgentAudio_Implementation).
// 40 frames (25600 bytes, 0.8s) stays safely under BOTH, so nothing is
// truncated on server or client. It also keeps each Unreliable RPC small.
static constexpr int32 BytesPerFrame =
(PS_AI_ConvAgent_Audio_ElevenLabs::SampleRate / 50)
* PS_AI_ConvAgent_Audio_ElevenLabs::Channels
* static_cast<int32>(sizeof(int16)); // 640 at 16kHz mono
static constexpr int32 MaxOpusChunkBytes = 40 * BytesPerFrame; // 25600 = 0.8s
// Carry the sub-frame remainder from the PREVIOUS message so no samples
// are dropped at message boundaries. Opus encodes whole 640-byte frames
// only; dropping each message's <1-frame tail left a ~10-16ms hole and a
// waveform discontinuity at every message splice → audible pops on clients
// (the host plays raw PCM directly, so standalone/host never had it).
// Prepend the leftover, encode only whole frames, stash the new tail.
if (!bAgentSpeaking)
{
// First message of a new turn — don't glue on the previous turn's tail.
AgentAudioEncodeLeftover.Reset();
}
TArray<uint8> ContinuousPCM = MoveTemp(AgentAudioEncodeLeftover);
ContinuousPCM.Append(PCMData);
const int32 UsableBytes = (ContinuousPCM.Num() / BytesPerFrame) * BytesPerFrame;
if (ContinuousPCM.Num() > UsableBytes)
{
AgentAudioEncodeLeftover.Append(
ContinuousPCM.GetData() + UsableBytes, ContinuousPCM.Num() - UsableBytes);
}
int32 Offset = 0;
while (Offset < UsableBytes)
{
const int32 BlockBytes = FMath::Min(MaxOpusChunkBytes, UsableBytes - Offset);
const bool bLast = (Offset + BlockBytes >= UsableBytes);
uint32 CompressedSize = static_cast<uint32>(OpusWorkBuffer.Num()); uint32 CompressedSize = static_cast<uint32>(OpusWorkBuffer.Num());
OpusEncoder->Encode(PCMData.GetData(), PCMData.Num(), OpusEncoder->Encode(ContinuousPCM.GetData() + Offset, BlockBytes,
OpusWorkBuffer.GetData(), CompressedSize); OpusWorkBuffer.GetData(), CompressedSize);
if (CompressedSize > 0) if (CompressedSize > 0)
{ {
TArray<uint8> CompressedData; TArray<uint8> CompressedData;
CompressedData.Append(OpusWorkBuffer.GetData(), CompressedSize); CompressedData.Append(OpusWorkBuffer.GetData(), CompressedSize);
MulticastReceiveAgentAudio(CompressedData);
if (bNetAudioDebug)
{
const float Ratio = static_cast<float>(BlockBytes) / static_cast<float>(CompressedSize);
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[NET-SRV] Opus audio chunk: %u bytes sent (raw %d → %.1fx compression, last=%d)."),
CompressedSize, BlockBytes, Ratio, bLast ? 1 : 0);
}
MulticastReceiveAgentAudio(CompressedData, bLast);
}
Offset += BlockBytes;
} }
} }
else else
{ {
// Fallback: send raw PCM (no compression). ~32 KB/s at 16kHz 16-bit mono. // Fallback: Opus init failed → send raw PCM (no compression, ~32 KB/s).
// Fine for LAN; revisit with proper Opus if internet play is needed. // This should not happen in the working config — it means [Voice]
// UE5 limits replicated TArrays to 65535 elements, so we must chunk. // bEnabled=true is missing in DefaultEngine.ini. Warn on every message
static bool bWarnedOnce = false; // so it is loud, not hidden behind a one-shot flag.
if (!bWarnedOnce) // The RPC is now Reliable, so we must protect the reliable buffer (512
{ // bunches): each ~4 KB chunk ≈ ~4 partial bunches. Hard-cap one message
bWarnedOnce = true; // at MaxFallbackBytes and drop the tail so a single message can never
// approach the buffer and trigger a reliable-buffer overflow (the exact
// cause of the old VR disconnect).
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("[NET-SRV] Opus encoder unavailable — sending raw PCM in chunks (no compression).")); TEXT("[NET-SRV] Opus encoder unavailable — sending raw PCM (no compression). "
"Set [Voice] bEnabled=true in DefaultEngine.ini to restore Opus."));
static constexpr int32 MaxChunkBytes = 4000;
static constexpr int32 MaxFallbackBytes = 320000; // ~80 reliable chunks/message, well under the 512-bunch buffer
const int32 SendableBytes = FMath::Min(PCMData.Num(), MaxFallbackBytes);
if (PCMData.Num() > MaxFallbackBytes)
{
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("[NET-SRV] Raw-PCM fallback capped at %d bytes — dropping %d-byte tail "
"to protect the reliable buffer."),
MaxFallbackBytes, PCMData.Num() - MaxFallbackBytes);
} }
static constexpr int32 MaxChunkBytes = 32000; // 1s of 16kHz 16-bit mono, well under 65535 limit
int32 Offset = 0; int32 Offset = 0;
while (Offset < PCMData.Num()) while (Offset < SendableBytes)
{ {
const int32 ChunkSize = FMath::Min(MaxChunkBytes, PCMData.Num() - Offset); const int32 ChunkSize = FMath::Min(MaxChunkBytes, SendableBytes - Offset);
const bool bLast = (Offset + ChunkSize >= SendableBytes);
TArray<uint8> Chunk; TArray<uint8> Chunk;
Chunk.Append(PCMData.GetData() + Offset, ChunkSize); Chunk.Append(PCMData.GetData() + Offset, ChunkSize);
MulticastReceiveAgentAudio(Chunk);
if (bNetAudioDebug)
{
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("[NET-SRV] RAW PCM chunk: %d bytes sent (NO compression — Opus disabled, last=%d)."),
ChunkSize, bLast ? 1 : 0);
}
MulticastReceiveAgentAudio(Chunk, bLast);
Offset += ChunkSize; Offset += ChunkSize;
} }
} }
@@ -1451,6 +1613,10 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::HandleClientToolCall(const FPS_AI_Con
TEXT("[T+%.2fs] Agent action requested: %s"), T, *ActionName); TEXT("[T+%.2fs] Agent action requested: %s"), T, *ActionName);
} }
// Track the pending action so OnReadyForAction can fire with the same name.
// If the user synchronously calls ForceDisableConversation() in their handler,
// OnReadyForAction will wait for blend-out. Otherwise it fires on next tick.
PendingActionName = ActionName;
OnAgentActionRequested.Broadcast(ActionName); OnAgentActionRequested.Broadcast(ActionName);
// Auto-respond so the agent can continue speaking. // Auto-respond so the agent can continue speaking.
@@ -1604,6 +1770,42 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::OnProceduralUnderflow(
void UPS_AI_ConvAgent_ElevenLabsComponent::EnqueueAgentAudio(const TArray<uint8>& PCMData) void UPS_AI_ConvAgent_ElevenLabsComponent::EnqueueAgentAudio(const TArray<uint8>& PCMData)
{ {
// Debug: flag a waveform discontinuity at the splice between the previous
// enqueued block and this one — the signature of the audio pops. The host's
// raw-PCM stream is continuous so its count should stay ~0; if a client's
// count climbs at message cadence, a splice problem remains.
{
const int32 DbgVal = CVarDebugElevenLabs.GetValueOnGameThread();
const bool bAudioDbg = (DbgVal >= 0) ? (DbgVal > 0) : bDebug;
if (bAudioDbg && PCMData.Num() >= static_cast<int32>(sizeof(int16)))
{
const int16* Samples = reinterpret_cast<const int16*>(PCMData.GetData());
const int32 NumSamples = PCMData.Num() / static_cast<int32>(sizeof(int16));
if (bHasLastEnqueuedSample)
{
const int32 Delta = FMath::Abs(static_cast<int32>(Samples[0]) - static_cast<int32>(LastEnqueuedSample));
static constexpr int32 DiscontinuityThreshold = 3000; // |delta| on the int16 scale
if (Delta > DiscontinuityThreshold)
{
++AudioSpliceDiscontinuityCount;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("[AUDIO-DBG] Role=%d splice discontinuity: |delta|=%d (prev=%d first=%d) count=%d — pop likely here."),
static_cast<int32>(GetOwnerRole()), Delta,
static_cast<int32>(LastEnqueuedSample), static_cast<int32>(Samples[0]),
AudioSpliceDiscontinuityCount);
}
else if (DbgVal >= 2)
{
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[AUDIO-DBG] Role=%d splice ok: |delta|=%d"),
static_cast<int32>(GetOwnerRole()), Delta);
}
}
LastEnqueuedSample = Samples[NumSamples - 1];
bHasLastEnqueuedSample = true;
}
}
{ {
FScopeLock Lock(&AudioQueueLock); FScopeLock Lock(&AudioQueueLock);
AudioQueue.Append(PCMData); AudioQueue.Append(PCMData);
@@ -1778,6 +1980,9 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StopAgentAudio()
bool bWasSpeaking = false; bool bWasSpeaking = false;
double Now = 0.0; double Now = 0.0;
bPreBuffering = false; // Clear pre-buffer state on stop. bPreBuffering = false; // Clear pre-buffer state on stop.
ClientReassemblyPCM.Reset(); // Client: drop any half-received message so it can't leak into the next turn.
AgentAudioEncodeLeftover.Reset(); // Server: drop the carried sub-frame tail so it can't glue onto the next turn.
bHasLastEnqueuedSample = false; // Debug: don't count the inter-turn splice as a discontinuity.
{ {
FScopeLock Lock(&AudioQueueLock); FScopeLock Lock(&AudioQueueLock);
AudioQueue.Empty(); AudioQueue.Empty();
@@ -1987,6 +2192,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::GetLifetimeReplicatedProps(
DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, bNetIsConversing); DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, bNetIsConversing);
DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, NetConnectedPawns); DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, NetConnectedPawns);
DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, NetActiveSpeakerPawn); DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, NetActiveSpeakerPawn);
DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, GazeOverrideTarget);
DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, CurrentEmotion); DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, CurrentEmotion);
DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, CurrentEmotionIntensity); DOREPLIFETIME(UPS_AI_ConvAgent_ElevenLabsComponent, CurrentEmotionIntensity);
} }
@@ -2010,17 +2216,34 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::OnRep_ConversationState()
// on the local pawn, but remote clients never run that code path. // on the local pawn, but remote clients never run that code path.
if (UPS_AI_ConvAgent_GazeComponent* Gaze = Owner->FindComponentByClass<UPS_AI_ConvAgent_GazeComponent>()) if (UPS_AI_ConvAgent_GazeComponent* Gaze = Owner->FindComponentByClass<UPS_AI_ConvAgent_GazeComponent>())
{ {
if (bNetIsConversing && NetConnectedPawns.Num() > 0) if (GazeOverrideTarget)
{
// Override (agent-to-agent dialogue): look at the fixed actor.
Gaze->bActive = true;
if (Gaze->TargetActor != GazeOverrideTarget)
{
Gaze->TargetActor = GazeOverrideTarget;
Gaze->ResetBodyTarget();
}
Gaze->bEnableBodyTracking = true;
}
else if (bNetIsConversing && NetConnectedPawns.Num() > 0)
{ {
Gaze->bActive = true; Gaze->bActive = true;
// Use active speaker if set, otherwise first connected pawn. // Use active speaker if set, otherwise first connected pawn.
// GazeTarget can be null on a late joiner if the replicated pawn
// reference hasn't resolved yet (actor channel not open). Skip until
// it resolves — OnRep fires again once the reference is patched in.
APawn* GazeTarget = NetActiveSpeakerPawn ? NetActiveSpeakerPawn : NetConnectedPawns.Last(); APawn* GazeTarget = NetActiveSpeakerPawn ? NetActiveSpeakerPawn : NetConnectedPawns.Last();
if (GazeTarget)
{
Gaze->TargetActor = GazeTarget; Gaze->TargetActor = GazeTarget;
Gaze->ResetBodyTarget(); Gaze->ResetBodyTarget();
Gaze->bEnableBodyTracking = true; Gaze->bEnableBodyTracking = true;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[NET-REP] Gaze ACTIVATED, TargetActor set to %s"), *GazeTarget->GetName()); TEXT("[NET-REP] Gaze ACTIVATED, TargetActor set to %s"), *GazeTarget->GetName());
} }
}
else else
{ {
Gaze->bActive = false; Gaze->bActive = false;
@@ -2053,6 +2276,25 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::OnRep_ConversationState()
} }
} }
// ── Client: (re)start listening once the conversation is confirmed live ──
// StartListening() called early on the client defers (bPendingStartListening)
// because bNetIsConversing wasn't replicated yet. The ClientConversationStarted
// relay RPC may also race ahead of this replication. OnRep is the reliable
// "conversation is live" signal, so consume the deferred listen here — this
// closes the race regardless of RPC/replication ordering. Gated on
// IsLocalPlayerConversating() so only the joining client opens its mic path.
if (GetOwnerRole() != ROLE_Authority && bNetIsConversing && IsLocalPlayerConversating())
{
if (bPendingStartListening
|| (bAutoStartListening && TurnMode == EPS_AI_ConvAgent_TurnMode_ElevenLabs::Server))
{
bPendingStartListening = false;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[NET-REP] Conversation live on client — starting listening (mic uplink)."));
StartListening();
}
}
if (!bNetIsConversing) if (!bNetIsConversing)
{ {
// Conversation ended on server — clean up local playback. // Conversation ended on server — clean up local playback.
@@ -2069,6 +2311,11 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::OnRep_ActiveSpeaker()
AActor* Owner = GetOwner(); AActor* Owner = GetOwner();
if (!Owner) return; if (!Owner) return;
// Guard: during an agent-to-agent dialogue the gaze is pinned to the other
// agent (GazeOverrideTarget) — a replicated active-speaker change (e.g. a
// player joining/speaking) must NOT yank the eyes back onto the player.
if (!GazeOverrideTarget)
{
if (auto* Gaze = Owner->FindComponentByClass<UPS_AI_ConvAgent_GazeComponent>()) if (auto* Gaze = Owner->FindComponentByClass<UPS_AI_ConvAgent_GazeComponent>())
{ {
if (NetActiveSpeakerPawn) if (NetActiveSpeakerPawn)
@@ -2080,6 +2327,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::OnRep_ActiveSpeaker()
} }
// Don't clear gaze when null — keep looking at last speaker. // Don't clear gaze when null — keep looking at last speaker.
} }
}
OnActiveSpeakerChanged.Broadcast(NetActiveSpeakerPawn, nullptr); OnActiveSpeakerChanged.Broadcast(NetActiveSpeakerPawn, nullptr);
} }
@@ -2100,7 +2348,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ServerJoinConversation_Implementation
if (!Pawn) return; if (!Pawn) return;
// Block join while force-disabled (ForceDisableConversation active). // Block join while force-disabled (ForceDisableConversation active).
if (bConversationDisabledByAction) if (bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[NET] ServerJoinConversation: blocked — conversation disabled by ForceDisableConversation().")); TEXT("[NET] ServerJoinConversation: blocked — conversation disabled by ForceDisableConversation()."));
@@ -2192,8 +2440,11 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ServerLeaveConversation_Implementatio
SetActiveSpeaker(NetConnectedPawns.Num() > 0 ? NetConnectedPawns[0] : nullptr); SetActiveSpeaker(NetConnectedPawns.Num() > 0 ? NetConnectedPawns[0] : nullptr);
} }
// If no players left, fully end the conversation. // If no players left, fully end the conversation — UNLESS the agent-to-agent
if (NetConnectedPawns.Num() == 0) // orchestrator owns it (GazeOverrideTarget set). A player merely WATCHING the
// dialogue joins/leaves just by looking around; that must NOT close the agent's
// WebSocket, stop its audio, or deactivate its lip sync. Keep the dialogue alive.
if (NetConnectedPawns.Num() == 0 && !GazeOverrideTarget)
{ {
// Cancel any pending reconnection (ephemeral mode only). // Cancel any pending reconnection (ephemeral mode only).
if (!bPersistentSession) if (!bPersistentSession)
@@ -2229,7 +2480,8 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ServerLeaveConversation_Implementatio
} }
else else
{ {
// Other players still connected — update gaze. // Players still connected, OR an orchestrated agent-to-agent dialogue owns
// this conversation (0 players but GazeOverrideTarget set) — keep it running.
ApplyConversationGaze(); ApplyConversationGaze();
} }
} }
@@ -2273,6 +2525,38 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ServerSendMicAudioFromPlayer(
{ {
if (!WebSocketProxy || !WebSocketProxy->IsConnected()) return; if (!WebSocketProxy || !WebSocketProxy->IsConnected()) return;
// Server-side voice-onset. This runs on the SERVER (only the server has a live
// WebSocket — clients returned above), and the player's mic — the host's OR a
// remote client's, relayed here — always passes through this function. Broadcasting
// OnUserVoiceDetected here is what makes the agent-to-agent dialogue interrupt work
// on a DEDICATED server (no host player). Debounced (shares LastUserVoiceOnset with
// the FeedExternalAudio path, so the host doesn't double-fire).
{
const int32 OnsetSamples = PCMBytes.Num() / (int32)sizeof(int16);
if (OnsetSamples > 0)
{
const int16* OS = reinterpret_cast<const int16*>(PCMBytes.GetData());
double OnsetSumSq = 0.0;
for (int32 i = 0; i < OnsetSamples; ++i) { const double v = OS[i] / 32768.0; OnsetSumSq += v * v; }
const double OnsetRms = FMath::Sqrt(OnsetSumSq / OnsetSamples);
if (OnsetRms >= 0.02)
{
const double NowSec = FPlatformTime::Seconds();
if (NowSec - LastUserVoiceOnset > 0.5)
{
LastUserVoiceOnset = NowSec;
OnUserVoiceDetected.Broadcast();
}
}
}
}
// Replay capture: expose every relayed player mic chunk (with its pawn) so an
// external recorder (PS_AI_ConvReplay) can save player speech. Server-only — clients
// have no live WebSocket and returned above. Fired before arbitration so audio from
// every player is captured, even a non-active speaker.
OnPlayerAudioCaptured.Broadcast(SpeakerPawn, PCMBytes);
// Standalone / single-player: bypass speaker arbitration entirely. // Standalone / single-player: bypass speaker arbitration entirely.
// There's only one player — no need for Contains check or speaker switching. // There's only one player — no need for Contains check or speaker switching.
if (NetConnectedPawns.Num() <= 1) if (NetConnectedPawns.Num() <= 1)
@@ -2304,16 +2588,44 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ServerSendMicAudioFromPlayer(
} }
const double Now = FPlatformTime::Seconds(); const double Now = FPlatformTime::Seconds();
LastSpeakTime.FindOrAdd(SpeakerPawn) = Now;
// If this player IS the active speaker, forward immediately. // Real voice energy of this chunk (int16 mono PCM). Only ACTUAL speech may
// hold or take the floor — not the silence-tail or ambient that an open mic
// streams continuously. Otherwise the first speaker's continuous stream keeps
// refreshing LastSpeakTime, so no one else can ever switch in and the agent
// stays locked on them (multi-player gaze/turn-taking broken).
const int32 NumSamples = PCMBytes.Num() / (int32)sizeof(int16);
double SumSq = 0.0;
if (NumSamples > 0)
{
const int16* S = reinterpret_cast<const int16*>(PCMBytes.GetData());
for (int32 i = 0; i < NumSamples; ++i) { const double v = S[i] / 32768.0; SumSq += v * v; }
}
const double Rms = (NumSamples > 0) ? FMath::Sqrt(SumSq / NumSamples) : 0.0;
const bool bVoiced = (Rms >= 0.02); // speech vs ambient/breath; tune if needed.
// Only real voice asserts holding/taking the floor.
if (bVoiced)
{
LastSpeakTime.FindOrAdd(SpeakerPawn) = Now;
}
// The active speaker is always forwarded (keeps the silence tail flowing to
// ElevenLabs VAD for end-of-turn detection).
if (NetActiveSpeakerPawn == SpeakerPawn) if (NetActiveSpeakerPawn == SpeakerPawn)
{ {
WebSocketProxy->SendAudioChunk(PCMBytes); WebSocketProxy->SendAudioChunk(PCMBytes);
return; return;
} }
// Speaker switch: only switch if current speaker has been silent // A different player can only TAKE the floor with actual voice — their
// silence/tail must not steal it from the current speaker.
if (!bVoiced)
{
return;
}
// Speaker switch: only switch if current speaker has been voice-silent
// for at least SpeakerSwitchHysteresis seconds. // for at least SpeakerSwitchHysteresis seconds.
bool bCanSwitch = true; bool bCanSwitch = true;
if (NetActiveSpeakerPawn) if (NetActiveSpeakerPawn)
@@ -2367,6 +2679,14 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ClientConversationStarted_Implementat
{ {
StartListening(); StartListening();
} }
// Consume any pending listen request deferred before conversation was ready.
else if (bPendingStartListening)
{
bPendingStartListening = false;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[Client] Consuming pending StartListening request."));
StartListening();
}
} }
void UPS_AI_ConvAgent_ElevenLabsComponent::ClientConversationFailed_Implementation( void UPS_AI_ConvAgent_ElevenLabsComponent::ClientConversationFailed_Implementation(
@@ -2381,7 +2701,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ClientConversationFailed_Implementati
// Network: Multicast RPCs // Network: Multicast RPCs
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_ElevenLabsComponent::MulticastReceiveAgentAudio_Implementation( void UPS_AI_ConvAgent_ElevenLabsComponent::MulticastReceiveAgentAudio_Implementation(
const TArray<uint8>& AudioData) const TArray<uint8>& AudioData, bool bLastSubChunk)
{ {
// Server already handled playback in HandleAudioReceived. // Server already handled playback in HandleAudioReceived.
if (GetOwnerRole() == ROLE_Authority) return; if (GetOwnerRole() == ROLE_Authority) return;
@@ -2399,13 +2719,17 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::MulticastReceiveAgentAudio_Implementa
if (OpusDecoder.IsValid()) if (OpusDecoder.IsValid())
{ {
// Opus path: decode compressed audio. // Opus path: decode compressed audio.
const uint32 MaxDecompressedSize = 16000 * 2; // Buffer must hold a full packet's decoded PCM. The engine decoder aborts
// with "Decompression buffer too small to decode voice" (dropping the
// packet tail) if this is smaller than the packet needs. The server caps
// packets at 40 frames (25600 bytes); 64KB gives ample headroom even up to
// the engine's 76-frame (~48640 bytes) maximum, so no packet is truncated.
const uint32 MaxDecompressedSize = 64 * 1024;
PCMBuffer.SetNumUninitialized(MaxDecompressedSize); PCMBuffer.SetNumUninitialized(MaxDecompressedSize);
uint32 DecompressedSize = MaxDecompressedSize; uint32 DecompressedSize = MaxDecompressedSize;
OpusDecoder->Decode(AudioData.GetData(), AudioData.Num(), OpusDecoder->Decode(AudioData.GetData(), AudioData.Num(),
PCMBuffer.GetData(), DecompressedSize); PCMBuffer.GetData(), DecompressedSize);
if (DecompressedSize == 0) return; PCMBuffer.SetNum(DecompressedSize); // may be 0 — handled by reassembly below
PCMBuffer.SetNum(DecompressedSize);
} }
else else
{ {
@@ -2413,14 +2737,73 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::MulticastReceiveAgentAudio_Implementa
PCMBuffer = AudioData; PCMBuffer = AudioData;
} }
// Local playback. // Reassemble one ElevenLabs message from its N sub-chunks, then enqueue ONCE.
EnqueueAgentAudio(PCMBuffer); // The server splits a message into sub-chunks (to respect the Opus 76-frame /
// client decode-buffer limits), sent back-to-back in the same tick. Enqueuing
// each sub-chunk separately re-ran the playback state machine → a pop at each
// ~0.8s seam. Accumulating and enqueuing once per message plays it contiguously.
if (PCMBuffer.Num() > 0)
{
ClientReassemblyPCM.Append(PCMBuffer);
}
if (!bLastSubChunk)
{
return; // wait for the remaining sub-chunks of this message
}
if (ClientReassemblyPCM.Num() > 0)
{
// Local playback (one contiguous block per message).
EnqueueAgentAudio(ClientReassemblyPCM);
// Feed lip-sync (within LOD or speaker). // Feed lip-sync (within LOD or speaker).
if (bIsSpeaker || LipSyncLODDistance <= 0.f || Dist <= LipSyncLODDistance) if (bIsSpeaker || LipSyncLODDistance <= 0.f || Dist <= LipSyncLODDistance)
{ {
OnAgentAudioData.Broadcast(PCMBuffer); OnAgentAudioData.Broadcast(ClientReassemblyPCM);
} }
ClientReassemblyPCM.Reset();
}
}
// ─────────────────────────────────────────────────────────────────────────────
// Replay support (PS_AI_ConvReplay bridge)
// ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_ElevenLabsComponent::PlayExternalAudioPCM(const TArray<uint8>& PCM16)
{
if (PCM16.Num() == 0) return;
// Same tail as MulticastReceiveAgentAudio_Implementation, minus the LOD culls:
// enqueue for playback (drives body expression + start/stop via bAgentSpeaking) and
// broadcast the raw PCM so the LipSync component animates visemes.
EnqueueAgentAudio(PCM16);
// During replay there is no live agent_response message, so the normal
// silence-detection stop path (TickComponent) would otherwise wait for the long
// hard-timeout. Mark the response as received so OnAgentStoppedSpeaking fires
// shortly after the buffer drains and body expression returns to idle.
// (Set AFTER EnqueueAgentAudio, which resets this flag on the first chunk.)
bAgentResponseReceived = true;
OnAgentAudioData.Broadcast(PCM16);
}
void UPS_AI_ConvAgent_ElevenLabsComponent::StopExternalAudio()
{
StopAgentAudio();
}
void UPS_AI_ConvAgent_ElevenLabsComponent::TriggerReplayedAction(const FString& ActionName)
{
PendingActionName = ActionName;
OnAgentActionRequested.Broadcast(ActionName);
}
void UPS_AI_ConvAgent_ElevenLabsComponent::TriggerReplayedClientToolCall(
const FPS_AI_ConvAgent_ClientToolCall_ElevenLabs& ToolCall)
{
OnAgentClientToolCall.Broadcast(ToolCall);
} }
void UPS_AI_ConvAgent_ElevenLabsComponent::MulticastAgentStartedSpeaking_Implementation() void UPS_AI_ConvAgent_ElevenLabsComponent::MulticastAgentStartedSpeaking_Implementation()
@@ -2599,14 +2982,27 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ApplyConversationGaze()
auto* Gaze = Owner->FindComponentByClass<UPS_AI_ConvAgent_GazeComponent>(); auto* Gaze = Owner->FindComponentByClass<UPS_AI_ConvAgent_GazeComponent>();
if (!Gaze) return; if (!Gaze) return;
if (bNetIsConversing && NetConnectedPawns.Num() > 0) // Override (agent-to-agent dialogue): look at a fixed actor, ignore players.
if (GazeOverrideTarget)
{
Gaze->bActive = true;
if (Gaze->TargetActor != GazeOverrideTarget)
{
Gaze->TargetActor = GazeOverrideTarget;
Gaze->ResetBodyTarget();
}
Gaze->bEnableBodyTracking = true;
}
else if (bNetIsConversing && NetConnectedPawns.Num() > 0)
{ {
Gaze->bActive = true; Gaze->bActive = true;
// Look at active speaker if set, otherwise last connected pawn. // Look at active speaker if set, otherwise last connected pawn.
// GazeTarget can be null if a replicated pawn ref hasn't resolved yet
// (late joiner). Don't overwrite the target with null in that case.
AActor* GazeTarget = NetActiveSpeakerPawn AActor* GazeTarget = NetActiveSpeakerPawn
? static_cast<AActor*>(NetActiveSpeakerPawn) ? static_cast<AActor*>(NetActiveSpeakerPawn)
: static_cast<AActor*>(NetConnectedPawns.Last()); : static_cast<AActor*>(NetConnectedPawns.Last());
if (Gaze->TargetActor != GazeTarget) if (GazeTarget && Gaze->TargetActor != GazeTarget)
{ {
Gaze->TargetActor = GazeTarget; Gaze->TargetActor = GazeTarget;
Gaze->ResetBodyTarget(); Gaze->ResetBodyTarget();
@@ -2637,6 +3033,20 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ApplyConversationGaze()
} }
} }
void UPS_AI_ConvAgent_ElevenLabsComponent::SetGazeOverrideTarget(AActor* Target)
{
if (GazeOverrideTarget == Target) return;
GazeOverrideTarget = Target; // replicates → OnRep_GazeOverrideTarget on clients
ApplyConversationGaze(); // apply immediately on the server/authority
}
void UPS_AI_ConvAgent_ElevenLabsComponent::OnRep_GazeOverrideTarget()
{
// Re-evaluate the gaze on clients (honours the override, or restores normal
// player-tracking when it was cleared to null).
ApplyConversationGaze();
}
void UPS_AI_ConvAgent_ElevenLabsComponent::SetActiveSpeaker(APawn* NewSpeaker) void UPS_AI_ConvAgent_ElevenLabsComponent::SetActiveSpeaker(APawn* NewSpeaker)
{ {
if (NetActiveSpeakerPawn == NewSpeaker) return; if (NetActiveSpeakerPawn == NewSpeaker) return;
@@ -2675,12 +3085,18 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::DrawDebugHUD() const
const FColor GoodColor = FColor::Green; const FColor GoodColor = FColor::Green;
const FString OwnerName = GetOwner()->GetName(); const FString OwnerName = GetOwner()->GetName();
const bool bConnected = IsConnected(); // On a client there is no local WebSocket (it lives on the server), so
// IsConnected() is always false. Report the effective networked state via the
// replicated bNetIsConversing flag instead, to avoid a misleading "DISCONNECTED".
const bool bIsAuthority = (GetOwnerRole() == ROLE_Authority);
const bool bConnected = bIsAuthority ? IsConnected() : bNetIsConversing;
const TCHAR* ConnStr = bConnected
? (bIsAuthority ? TEXT("CONNECTED") : TEXT("CONNECTED (via server)"))
: TEXT("DISCONNECTED");
GEngine->AddOnScreenDebugMessage(BaseKey, DisplayTime, GEngine->AddOnScreenDebugMessage(BaseKey, DisplayTime,
bConnected ? GoodColor : FColor::Red, bConnected ? GoodColor : FColor::Red,
FString::Printf(TEXT("=== ELEVENLABS [%s]: %s ==="), *OwnerName, FString::Printf(TEXT("=== ELEVENLABS [%s]: %s ==="), *OwnerName, ConnStr));
bConnected ? TEXT("CONNECTED") : TEXT("DISCONNECTED")));
// Session info // Session info
FString ModeStr = (TurnMode == EPS_AI_ConvAgent_TurnMode_ElevenLabs::Server) FString ModeStr = (TurnMode == EPS_AI_ConvAgent_TurnMode_ElevenLabs::Server)
@@ -2745,12 +3161,13 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::DrawDebugHUD() const
bWantsReconnect ? TEXT(" (ACTIVE)") : TEXT(""))); bWantsReconnect ? TEXT(" (ACTIVE)") : TEXT("")));
// ForceDisable state // ForceDisable state
if (bConversationDisabledByAction || bWaitingForBlendOut) if (bConversationForceDisabled || bWaitingForBlendOut)
{ {
const FColor DisableColor = FColor::Orange; const FColor DisableColor = FColor::Orange;
const FString PendingStr = PendingActionName.IsEmpty() ? TEXT("<none>") : PendingActionName;
GEngine->AddOnScreenDebugMessage(BaseKey + 9, DisplayTime, DisableColor, GEngine->AddOnScreenDebugMessage(BaseKey + 9, DisplayTime, DisableColor,
FString::Printf(TEXT(" FORCE DISABLED — action: '%s' blendOut: %s (%.1fs)"), FString::Printf(TEXT(" FORCE DISABLED — pending action: '%s' blendOut: %s (%.1fs)"),
*PendingActionName, *PendingStr,
bWaitingForBlendOut ? TEXT("WAITING") : TEXT("DONE"), bWaitingForBlendOut ? TEXT("WAITING") : TEXT("DONE"),
BlendOutElapsedTime)); BlendOutElapsedTime));
@@ -2783,12 +3200,13 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::DrawDebugHUD() const
void UPS_AI_ConvAgent_ElevenLabsComponent::FetchServerRegion() void UPS_AI_ConvAgent_ElevenLabsComponent::FetchServerRegion()
{ {
const UPS_AI_ConvAgent_Settings_ElevenLabs* Settings = FPS_AI_ConvAgentModule::Get().GetSettings(); const UPS_AI_ConvAgent_Settings_ElevenLabs* Settings = FPS_AI_ConvAgentModule::Get().GetSettings();
if (!Settings || Settings->API_Key.IsEmpty()) return; const FString APIKey = UPS_AI_ConvAgent_BlueprintLibrary::GetElevenLabsAPIKey();
if (!Settings || APIKey.IsEmpty()) return;
auto Request = FHttpModule::Get().CreateRequest(); auto Request = FHttpModule::Get().CreateRequest();
Request->SetURL(Settings->GetAPIBaseURL() + TEXT("/v1/models")); Request->SetURL(Settings->GetAPIBaseURL() + TEXT("/v1/models"));
Request->SetVerb(TEXT("GET")); Request->SetVerb(TEXT("GET"));
Request->SetHeader(TEXT("xi-api-key"), Settings->API_Key); Request->SetHeader(TEXT("xi-api-key"), APIKey);
TWeakObjectPtr<UPS_AI_ConvAgent_ElevenLabsComponent> WeakThis(this); TWeakObjectPtr<UPS_AI_ConvAgent_ElevenLabsComponent> WeakThis(this);
Request->OnProcessRequestComplete().BindLambda( Request->OnProcessRequestComplete().BindLambda(

View File

@@ -405,11 +405,7 @@ void UPS_AI_ConvAgent_GazeComponent::TickComponent(
const FVector ToTarget = TargetPos - EyeOrigin; const FVector ToTarget = TargetPos - EyeOrigin;
// ── 2. Body: smooth interp toward sticky target ──────────────────── // ── 1b. Horizontal direction / world yaw to the target ─────────────
//
// TargetBodyWorldYaw is persistent — only updated when head+eyes
// can't reach the target. Same sticky pattern as TargetHeadYaw.
const FVector HorizontalDir = FVector(ToTarget.X, ToTarget.Y, 0.0f); const FVector HorizontalDir = FVector(ToTarget.X, ToTarget.Y, 0.0f);
float TargetWorldYaw = 0.0f; float TargetWorldYaw = 0.0f;
if (!HorizontalDir.IsNearlyZero(1.0f)) if (!HorizontalDir.IsNearlyZero(1.0f))
@@ -417,11 +413,21 @@ void UPS_AI_ConvAgent_GazeComponent::TickComponent(
TargetWorldYaw = HorizontalDir.Rotation().Yaw; TargetWorldYaw = HorizontalDir.Rotation().Yaw;
} }
// Body smoothly interpolates toward its persistent target. // ── 2. BODY (authority only): lazy turn ────────────────────────────
// Server/standalone: directly rotates the actor. // Rotate the actor to face the target only once head+eyes can no longer
// Client: does NOT rotate — accepts replicated rotation from server. // cover it. Clients NEVER rotate the body — they receive it replicated.
if (bEnableBodyTracking && Owner->HasAuthority()) // Body turn state (TargetBodyWorldYaw) lives on the authority only.
if (bEnableBodyTracking && Owner->HasAuthority() && !HorizontalDir.IsNearlyZero(1.0f))
{ {
const float BodyFacingYaw = Owner->GetActorRotation().Yaw + MeshForwardYawOffset;
const float DeltaToTarget = FMath::FindDeltaAngleDegrees(BodyFacingYaw, TargetWorldYaw);
// Commit to facing the target once it leaves the head+eyes range.
if (FMath::Abs(DeltaToTarget) > MaxHeadYaw + MaxEyeHorizontal)
{
TargetBodyWorldYaw = TargetWorldYaw - MeshForwardYawOffset;
}
const float BodyDelta = FMath::FindDeltaAngleDegrees( const float BodyDelta = FMath::FindDeltaAngleDegrees(
Owner->GetActorRotation().Yaw, TargetBodyWorldYaw); Owner->GetActorRotation().Yaw, TargetBodyWorldYaw);
if (FMath::Abs(BodyDelta) > 0.1f) if (FMath::Abs(BodyDelta) > 0.1f)
@@ -431,113 +437,49 @@ void UPS_AI_ConvAgent_GazeComponent::TickComponent(
} }
} }
// ── Smoothed body yaw for cascade computations ────────────────── // Smoothed mirror of the (authoritative/replicated) body yaw — DEBUG
// Server: direct copy (no lag). Client: interpolate toward replicated // only. NEVER written back to the actor: the old client override
// rotation to avoid step artifacts from discrete ~30Hz network updates. // (SetActorRotation) corrupted the head input and made the client head
// The client also overrides the actor's rotation with the smoothed value // diverge by up to MaxHeadYaw. The head reads the real actor yaw below.
// so the body mesh doesn't visually jump at network tick rate.
if (Owner->HasAuthority())
{ {
SmoothedBodyYaw = Owner->GetActorRotation().Yaw; const float ActorYaw = Owner->GetActorRotation().Yaw;
} const float YawDelta = FMath::FindDeltaAngleDegrees(SmoothedBodyYaw, ActorYaw);
else SmoothedBodyYaw += FMath::FInterpTo(0.0f, YawDelta, SafeDeltaTime, BodyInterpSpeed * 3.0f);
{
const float ReplicatedYaw = Owner->GetActorRotation().Yaw;
const float ClientDelta = FMath::FindDeltaAngleDegrees(SmoothedBodyYaw, ReplicatedYaw);
SmoothedBodyYaw += FMath::FInterpTo(0.0f, ClientDelta, SafeDeltaTime, BodyInterpSpeed * 3.0f);
// Override actor rotation with smoothed value so the body mesh
// shows the interpolated rotation instead of the raw replicated steps.
// This is local-only — replication will overwrite on next network update,
// and SmoothedBodyYaw will absorb the correction smoothly.
FRotator SmoothedRot = Owner->GetActorRotation();
SmoothedRot.Yaw = SmoothedBodyYaw;
Owner->SetActorRotation(SmoothedRot);
} }
// ── 3. Compute DeltaYaw after body interp ────────────────────────── // When body tracking is off, keep the body target synced (debug only).
if (!bEnableBodyTracking)
{
TargetBodyWorldYaw = Owner->GetActorRotation().Yaw;
}
// ── 3. HEAD + EYES — STATELESS cascade ─────────────────────────────
// Pure function of the ACTUAL body facing (authoritative on the server,
// replicated on the client) and the target direction. No sticky /
// path-dependent state, so server and client converge to the same pose
// regardless of how the body rotation arrived (smooth vs ~30Hz steps).
// FInterpTo only eases the visual; it never latches.
const float BodyFacingYaw = Owner->GetActorRotation().Yaw + MeshForwardYawOffset;
float DeltaYaw = 0.0f; float DeltaYaw = 0.0f;
if (!HorizontalDir.IsNearlyZero(1.0f)) if (!HorizontalDir.IsNearlyZero(1.0f))
{ {
const float CurrentFacingYaw = SmoothedBodyYaw + MeshForwardYawOffset; DeltaYaw = FMath::FindDeltaAngleDegrees(BodyFacingYaw, TargetWorldYaw);
DeltaYaw = FMath::FindDeltaAngleDegrees(CurrentFacingYaw, TargetWorldYaw);
} }
// ── 4. Pitch from 3D direction ─────────────────────────────────────
const float HorizontalDist = HorizontalDir.Size(); const float HorizontalDist = HorizontalDir.Size();
const float TargetPitch = (HorizontalDist > 1.0f) const float TargetPitch = (HorizontalDist > 1.0f)
? FMath::RadiansToDegrees(FMath::Atan2(ToTarget.Z, HorizontalDist)) ? FMath::RadiansToDegrees(FMath::Atan2(ToTarget.Z, HorizontalDist))
: 0.0f; : 0.0f;
// ── 5. BODY overflow: check against persistent TargetBodyWorldYaw ── // Head takes as much of the yaw gap as its range allows; eyes the rest.
// // (TargetHeadYaw/Pitch are kept only as debug mirrors — no longer sticky.)
// Same pattern as head: check from body's TARGET position (not current). TargetHeadYaw = FMath::Clamp(DeltaYaw, -MaxHeadYaw, MaxHeadYaw);
// Body triggers when head+eyes combined range is exceeded.
const float BodyTargetFacing = TargetBodyWorldYaw + MeshForwardYawOffset;
const float DeltaFromBodyTarget = FMath::FindDeltaAngleDegrees(
BodyTargetFacing, TargetWorldYaw);
bool bBodyOverflowed = false;
if (bEnableBodyTracking && FMath::Abs(DeltaFromBodyTarget) > MaxHeadYaw + MaxEyeHorizontal)
{
// Body realigns to face target
TargetBodyWorldYaw = TargetWorldYaw - MeshForwardYawOffset;
// Head returns to ~0° since body will face target directly
TargetHeadYaw = 0.0f;
bBodyOverflowed = true;
}
// ── 6. HEAD: realign when eyes overflow (check against body TARGET) ──
//
// Head overflow is checked relative to where the BODY IS GOING
// (TargetBodyWorldYaw), not where it currently is. This prevents
// the head from overcompensating during body interpolation —
// otherwise the head turns to track while body catches up, then
// snaps back when body arrives (two-step animation artifact).
//
// GUARD: Skip eye overflow check when body just overflowed in this
// same frame. Body overflow already set TargetHeadYaw=0 (head will
// recenter as body turns). Allowing eye overflow to also fire would
// snap TargetHeadYaw to a second value in the same frame, causing
// a visible 1-2 frame jerk as FInterpTo chases two targets.
if (!bBodyOverflowed)
{
const float HeadDeltaYaw = FMath::FindDeltaAngleDegrees(
TargetBodyWorldYaw + MeshForwardYawOffset, TargetWorldYaw);
const float EyeDeltaYaw = HeadDeltaYaw - TargetHeadYaw;
if (FMath::Abs(EyeDeltaYaw) > MaxEyeHorizontal)
{
TargetHeadYaw = FMath::Clamp(HeadDeltaYaw, -MaxHeadYaw, MaxHeadYaw);
}
}
// Head smoothly interpolates toward its persistent target
CurrentHeadYaw = FMath::FInterpTo(CurrentHeadYaw, TargetHeadYaw, SafeDeltaTime, HeadInterpSpeed); CurrentHeadYaw = FMath::FInterpTo(CurrentHeadYaw, TargetHeadYaw, SafeDeltaTime, HeadInterpSpeed);
// Eyes = remaining gap (during transients, eyes may sit at MaxEye while
// the head catches up — ARKit normalization keeps visual deflection small)
CurrentEyeYaw = FMath::Clamp(DeltaYaw - CurrentHeadYaw, -MaxEyeHorizontal, MaxEyeHorizontal); CurrentEyeYaw = FMath::Clamp(DeltaYaw - CurrentHeadYaw, -MaxEyeHorizontal, MaxEyeHorizontal);
// ── 5. PITCH: relative cascade (Eyes → Head, no body pitch) ────────
// Same pattern: check against persistent TargetHeadPitch
const float EyeDeltaPitch = TargetPitch - TargetHeadPitch;
if (FMath::Abs(EyeDeltaPitch) > MaxEyeVertical)
{
// Eyes overflow → head realigns toward target pitch
TargetHeadPitch = FMath::Clamp(TargetPitch, -MaxHeadPitch, MaxHeadPitch); TargetHeadPitch = FMath::Clamp(TargetPitch, -MaxHeadPitch, MaxHeadPitch);
}
CurrentHeadPitch = FMath::FInterpTo(CurrentHeadPitch, TargetHeadPitch, SafeDeltaTime, HeadInterpSpeed); CurrentHeadPitch = FMath::FInterpTo(CurrentHeadPitch, TargetHeadPitch, SafeDeltaTime, HeadInterpSpeed);
// Eyes = remaining pitch gap
CurrentEyePitch = FMath::Clamp(TargetPitch - CurrentHeadPitch, -MaxEyeVertical, MaxEyeVertical); CurrentEyePitch = FMath::Clamp(TargetPitch - CurrentHeadPitch, -MaxEyeVertical, MaxEyeVertical);
} }
else else

View File

@@ -21,6 +21,17 @@ static TAutoConsoleVariable<int32> CVarDebugInteraction(
TEXT("Debug HUD for Interaction. -1=use property, 0=off, 1-3=verbosity."), TEXT("Debug HUD for Interaction. -1=use property, 0=off, 1-3=verbosity."),
ECVF_Default); ECVF_Default);
// Convert normalized float mic samples to int16 little-endian bytes (16kHz mono).
static void ConvReplay_FloatToInt16Bytes(const TArray<float>& In, TArray<uint8>& Out)
{
Out.SetNumUninitialized(In.Num() * static_cast<int32>(sizeof(int16)));
int16* D = reinterpret_cast<int16*>(Out.GetData());
for (int32 i = 0; i < In.Num(); ++i)
{
D[i] = static_cast<int16>(FMath::Clamp<int32>(FMath::RoundToInt(In[i] * 32767.0f), -32768, 32767));
}
}
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
// Constructor // Constructor
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
@@ -132,6 +143,9 @@ void UPS_AI_ConvAgent_InteractionComponent::TickComponent(float DeltaTime, ELeve
bInitialized = true; bInitialized = true;
// If a replay recording started before the mic existed, begin capturing now.
UpdateReplayCapture();
if (bDebug) if (bDebug)
{ {
UE_LOG(LogPS_AI_ConvAgent_Select, Log, UE_LOG(LogPS_AI_ConvAgent_Select, Log,
@@ -191,7 +205,8 @@ void UPS_AI_ConvAgent_InteractionComponent::TickComponent(float DeltaTime, ELeve
// Re-activate passive gaze using configured checkboxes. // Re-activate passive gaze using configured checkboxes.
// ExecuteLeave killed the gaze — re-enable passively. // ExecuteLeave killed the gaze — re-enable passively.
if (bAutoManageGaze) // (Skip if the agent is in an agent-to-agent dialogue.)
if (bAutoManageGaze && !Pending->GazeOverrideTarget)
{ {
if (UPS_AI_ConvAgent_GazeComponent* Gaze = FindGazeOnAgent(Pending)) if (UPS_AI_ConvAgent_GazeComponent* Gaze = FindGazeOnAgent(Pending))
{ {
@@ -301,6 +316,9 @@ void UPS_AI_ConvAgent_InteractionComponent::UpdatePassiveGaze()
if (Agent == GazeRetainedAgent.Get()) continue; if (Agent == GazeRetainedAgent.Get()) continue;
if (Agent == PendingLeaveAgent.Get()) continue; if (Agent == PendingLeaveAgent.Get()) continue;
if (Agent->IsConversationDisabled()) continue; if (Agent->IsConversationDisabled()) continue;
// Skip agents driven by an agent-to-agent dialogue (they look at each
// other via GazeOverrideTarget — don't pull their gaze toward the player).
if (Agent->GazeOverrideTarget) continue;
AActor* AgentActor = Agent->GetOwner(); AActor* AgentActor = Agent->GetOwner();
if (!AgentActor) continue; if (!AgentActor) continue;
@@ -600,7 +618,8 @@ void UPS_AI_ConvAgent_InteractionComponent::SetSelectedAgent(UPS_AI_ConvAgent_El
// ExecuteLeave → ServerLeaveConversation → ApplyConversationGaze() // ExecuteLeave → ServerLeaveConversation → ApplyConversationGaze()
// already killed bActive and cleared TargetActor. Override that: // already killed bActive and cleared TargetActor. Override that:
// the agent should keep looking at the player passively. // the agent should keep looking at the player passively.
if (bAutoManageGaze) // (Skip while an agent-to-agent dialogue owns the gaze.)
if (bAutoManageGaze && !OldAgent->GazeOverrideTarget)
{ {
if (UPS_AI_ConvAgent_GazeComponent* Gaze = FindGazeOnAgent(OldAgent)) if (UPS_AI_ConvAgent_GazeComponent* Gaze = FindGazeOnAgent(OldAgent))
{ {
@@ -919,6 +938,11 @@ void UPS_AI_ConvAgent_InteractionComponent::AttachGazeTarget(
// from ever firing. // from ever firing.
if (AgentPtr->IsConversationDisabled()) return; if (AgentPtr->IsConversationDisabled()) return;
// Agent-to-agent dialogue: this agent's gaze is pinned on the OTHER agent via
// GazeOverrideTarget. Don't attach it to the player just because the player is
// looking at / selecting it — let the override hold (the dialogue owns the gaze).
if (AgentPtr->GazeOverrideTarget) return;
if (UPS_AI_ConvAgent_GazeComponent* Gaze = FindGazeOnAgent(AgentPtr)) if (UPS_AI_ConvAgent_GazeComponent* Gaze = FindGazeOnAgent(AgentPtr))
{ {
Gaze->TargetActor = GetOwner(); Gaze->TargetActor = GetOwner();
@@ -1052,10 +1076,34 @@ void UPS_AI_ConvAgent_InteractionComponent::CleanupRetainedGaze(UPS_AI_ConvAgent
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_InteractionComponent::OnMicAudioCaptured(const TArray<float>& FloatPCM) void UPS_AI_ConvAgent_InteractionComponent::OnMicAudioCaptured(const TArray<float>& FloatPCM)
{ {
UPS_AI_ConvAgent_ElevenLabsComponent* Agent = SelectedAgent.Get(); if (UPS_AI_ConvAgent_ElevenLabsComponent* Agent = SelectedAgent.Get())
if (!Agent) return; {
Agent->FeedExternalAudio(FloatPCM); Agent->FeedExternalAudio(FloatPCM);
return;
}
// No agent selected: if a replay is being recorded, still capture this pawn's
// voice for the recorder. Convert to int16, accumulate, and relay in chunks —
// the host broadcasts locally; a remote client sends a Server RPC.
if (!bReplayRecordActive) return;
TArray<uint8> Bytes;
ConvReplay_FloatToInt16Bytes(FloatPCM, Bytes);
RecordMicBuffer.Append(Bytes);
static constexpr int32 RecordChunkMinBytes = 3200; // ~100ms @ 16kHz mono int16
if (RecordMicBuffer.Num() >= RecordChunkMinBytes)
{
if (GetOwnerRole() == ROLE_Authority)
{
OnPlayerVoiceForRecording.Broadcast(Cast<APawn>(GetOwner()), RecordMicBuffer);
}
else
{
ServerRecordVoice(RecordMicBuffer);
}
RecordMicBuffer.Reset();
}
} }
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
@@ -1144,7 +1192,7 @@ void UPS_AI_ConvAgent_InteractionComponent::GetLifetimeReplicatedProps(
TArray<FLifetimeProperty>& OutLifetimeProps) const TArray<FLifetimeProperty>& OutLifetimeProps) const
{ {
Super::GetLifetimeReplicatedProps(OutLifetimeProps); Super::GetLifetimeReplicatedProps(OutLifetimeProps);
// No replicated properties — but the component must be replicated for RPCs. DOREPLIFETIME(UPS_AI_ConvAgent_InteractionComponent, bReplayRecordActive);
} }
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
@@ -1221,6 +1269,50 @@ void UPS_AI_ConvAgent_InteractionComponent::ServerRelayMicAudio_Implementation(
Agent->ServerSendMicAudioFromPlayer(SenderPawn, PCMToSend); Agent->ServerSendMicAudioFromPlayer(SenderPawn, PCMToSend);
} }
// ─────────────────────────────────────────────────────────────────────────────
// Agent-less replay recording (driven by PS_AI_ConvReplay)
// ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_InteractionComponent::SetReplayRecordActive(bool bActive)
{
if (GetOwnerRole() != ROLE_Authority) return;
if (bReplayRecordActive == bActive) return;
bReplayRecordActive = bActive;
// Authority (host player) receives no OnRep — apply locally now.
UpdateReplayCapture();
}
void UPS_AI_ConvAgent_InteractionComponent::OnRep_ReplayRecordActive()
{
UpdateReplayCapture();
}
void UPS_AI_ConvAgent_InteractionComponent::UpdateReplayCapture()
{
if (!bReplayRecordActive)
{
RecordMicBuffer.Reset();
// Only stop the mic if no agent conversation still needs it.
if (MicComponent && MicComponent->IsCapturing() && !SelectedAgent.IsValid())
{
MicComponent->StopCapture();
}
return;
}
// Recording: make sure the mic is capturing. On a remote pawn's server-side
// replica MicComponent is null (tick disabled) and this is a harmless no-op —
// that client captures on its own machine and relays via ServerRecordVoice.
if (MicComponent && !MicComponent->IsCapturing())
{
MicComponent->StartCapture();
}
}
void UPS_AI_ConvAgent_InteractionComponent::ServerRecordVoice_Implementation(const TArray<uint8>& PCMBytes)
{
OnPlayerVoiceForRecording.Broadcast(Cast<APawn>(GetOwner()), PCMBytes);
}
void UPS_AI_ConvAgent_InteractionComponent::ServerRelaySendText_Implementation( void UPS_AI_ConvAgent_InteractionComponent::ServerRelaySendText_Implementation(
AActor* AgentActor, const FString& Text) AActor* AgentActor, const FString& Text)
{ {

View File

@@ -198,6 +198,54 @@ void UPS_AI_ConvAgent_MicrophoneCaptureComponent::OnAudioGenerate(
PeakLevel.store(Peak, std::memory_order_relaxed); PeakLevel.store(Peak, std::memory_order_relaxed);
} }
// ── Local Voice Activity Detection ──────────────────────────────────
// Runs on the audio thread for minimal latency. Uses hysteresis to
// prevent flickering. Dispatches state change to game thread.
{
const float Rms = CurrentRMS.load(std::memory_order_relaxed);
const float BufferDuration = static_cast<float>(NumFrames) / FMath::Max(1, InSampleRate);
const bool bAboveThreshold = (Rms > VoiceActivityThreshold);
if (bAboveThreshold)
{
VoiceSilenceAccumulator.store(0.0f, std::memory_order_relaxed);
const float NewOnset = VoiceOnsetAccumulator.load(std::memory_order_relaxed) + BufferDuration;
VoiceOnsetAccumulator.store(NewOnset, std::memory_order_relaxed);
if (!bIsUserSpeaking && NewOnset >= VoiceOnsetTime)
{
bIsUserSpeaking = true;
TWeakObjectPtr<UPS_AI_ConvAgent_MicrophoneCaptureComponent> WeakSelf(this);
AsyncTask(ENamedThreads::GameThread, [WeakSelf]()
{
if (auto* Self = WeakSelf.Get())
{
Self->OnUserVoiceActivityChanged.Broadcast(true);
}
});
}
}
else
{
VoiceOnsetAccumulator.store(0.0f, std::memory_order_relaxed);
const float NewSilence = VoiceSilenceAccumulator.load(std::memory_order_relaxed) + BufferDuration;
VoiceSilenceAccumulator.store(NewSilence, std::memory_order_relaxed);
if (bIsUserSpeaking && NewSilence >= VoiceSilenceTime)
{
bIsUserSpeaking = false;
TWeakObjectPtr<UPS_AI_ConvAgent_MicrophoneCaptureComponent> WeakSelf(this);
AsyncTask(ENamedThreads::GameThread, [WeakSelf]()
{
if (auto* Self = WeakSelf.Get())
{
Self->OnUserVoiceActivityChanged.Broadcast(false);
}
});
}
}
}
// Resample + downmix to 16000 Hz mono. // Resample + downmix to 16000 Hz mono.
TArray<float> Resampled = ResampleTo16000(FloatAudio, NumFrames, InNumChannels, InSampleRate); TArray<float> Resampled = ResampleTo16000(FloatAudio, NumFrames, InNumChannels, InSampleRate);

View File

@@ -0,0 +1,78 @@
// Copyright ASTERION. All Rights Reserved.
#include "PS_AI_ConvAgent_TestAPIKey_AsyncAction.h"
#include "PS_AI_ConvAgent.h"
#include "HttpModule.h"
#include "Interfaces/IHttpRequest.h"
#include "Interfaces/IHttpResponse.h"
UPS_AI_ConvAgent_TestAPIKey_AsyncAction* UPS_AI_ConvAgent_TestAPIKey_AsyncAction::TestElevenLabsAPIKey(const FString& Key)
{
UPS_AI_ConvAgent_TestAPIKey_AsyncAction* Action = NewObject<UPS_AI_ConvAgent_TestAPIKey_AsyncAction>();
Action->KeyToTest = Key;
return Action;
}
void UPS_AI_ConvAgent_TestAPIKey_AsyncAction::Activate()
{
// Keep this action alive across the async HTTP round-trip. Removed right
// before SetReadyToDestroy() once a result has been broadcast.
AddToRoot();
if (KeyToTest.IsEmpty())
{
OnInvalid.Broadcast(TEXT("API key is empty."));
RemoveFromRoot();
SetReadyToDestroy();
return;
}
// Respect the configured server region for the probe URL; fall back to global.
FString BaseURL = TEXT("https://api.elevenlabs.io");
if (FPS_AI_ConvAgentModule::IsAvailable())
{
if (const UPS_AI_ConvAgent_Settings_ElevenLabs* Settings = FPS_AI_ConvAgentModule::Get().GetSettings())
{
BaseURL = Settings->GetAPIBaseURL();
}
}
const TSharedRef<IHttpRequest, ESPMode::ThreadSafe> Request = FHttpModule::Get().CreateRequest();
Request->SetURL(BaseURL + TEXT("/v1/models"));
Request->SetVerb(TEXT("GET"));
Request->SetHeader(TEXT("xi-api-key"), KeyToTest);
TWeakObjectPtr<UPS_AI_ConvAgent_TestAPIKey_AsyncAction> WeakThis(this);
Request->OnProcessRequestComplete().BindLambda(
[WeakThis](FHttpRequestPtr /*Req*/, FHttpResponsePtr Resp, bool bSuccess)
{
if (!WeakThis.IsValid())
{
return;
}
UPS_AI_ConvAgent_TestAPIKey_AsyncAction* Self = WeakThis.Get();
const int32 Code = Resp.IsValid() ? Resp->GetResponseCode() : 0;
if (bSuccess && Code == 200)
{
Self->OnValid.Broadcast(TEXT("API key is valid."));
}
else if (Code == 401 || Code == 403)
{
Self->OnInvalid.Broadcast(FString::Printf(TEXT("Invalid API key (HTTP %d)."), Code));
}
else if (Code == 0)
{
Self->OnInvalid.Broadcast(TEXT("Could not reach ElevenLabs (network error)."));
}
else
{
Self->OnInvalid.Broadcast(FString::Printf(TEXT("Key test failed (HTTP %d)."), Code));
}
Self->RemoveFromRoot();
Self->SetReadyToDestroy();
});
Request->ProcessRequest();
}

View File

@@ -2,6 +2,7 @@
#include "PS_AI_ConvAgent_WebSocket_ElevenLabsProxy.h" #include "PS_AI_ConvAgent_WebSocket_ElevenLabsProxy.h"
#include "PS_AI_ConvAgent.h" #include "PS_AI_ConvAgent.h"
#include "PS_AI_ConvAgent_BlueprintLibrary.h"
#include "WebSocketsModule.h" #include "WebSocketsModule.h"
#include "IWebSocket.h" #include "IWebSocket.h"
@@ -9,6 +10,10 @@
#include "Json.h" #include "Json.h"
#include "Misc/Base64.h" #include "Misc/Base64.h"
#include "HttpModule.h"
#include "Interfaces/IHttpRequest.h"
#include "Interfaces/IHttpResponse.h"
DEFINE_LOG_CATEGORY_STATIC(LogPS_AI_ConvAgent_WS_ElevenLabs, Log, All); DEFINE_LOG_CATEGORY_STATIC(LogPS_AI_ConvAgent_WS_ElevenLabs, Log, All);
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
@@ -28,7 +33,36 @@ void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::Connect(const FString& AgentIDO
FModuleManager::LoadModuleChecked<FWebSocketsModule>("WebSockets"); FModuleManager::LoadModuleChecked<FWebSocketsModule>("WebSockets");
} }
const FString URL = BuildWebSocketURL(AgentIDOverride, APIKeyOverride); const UPS_AI_ConvAgent_Settings_ElevenLabs* Settings = FPS_AI_ConvAgentModule::Get().GetSettings();
const FString ResolvedKey = APIKeyOverride.IsEmpty()
? UPS_AI_ConvAgent_BlueprintLibrary::GetElevenLabsAPIKey()
: APIKeyOverride;
// Private agents (authentication enabled) require a Signed URL: a direct
// ?agent_id=... connection is rejected by ElevenLabs even with the xi-api-key
// header. So whenever we have a key (and no manual URL override), fetch a
// short-lived signed URL with that key and connect to it. This also works for
// public agents, and enforces "valid key required" — no valid key, no signed
// URL, no conversation.
if (!ResolvedKey.IsEmpty() && Settings->CustomWebSocketURL.IsEmpty())
{
if (AgentIDOverride.IsEmpty())
{
const FString Msg = TEXT("Cannot connect: no Agent ID configured. Set it in Project Settings or pass it to Connect().");
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Error, TEXT("%s"), *Msg);
ConnectionState = EPS_AI_ConvAgent_ConnectionState_ElevenLabs::Error;
OnError.Broadcast(Msg);
return;
}
ConnectionState = EPS_AI_ConvAgent_ConnectionState_ElevenLabs::Connecting;
RequestSignedURLAndConnect(AgentIDOverride, ResolvedKey);
return;
}
// Fallback: direct connection. Used for the advanced custom-URL override, or
// for a public agent when no key is available (no authentication).
const FString URL = BuildWebSocketURL(AgentIDOverride, ResolvedKey);
if (URL.IsEmpty()) if (URL.IsEmpty())
{ {
const FString Msg = TEXT("Cannot connect: no Agent ID configured. Set it in Project Settings or pass it to Connect()."); const FString Msg = TEXT("Cannot connect: no Agent ID configured. Set it in Project Settings or pass it to Connect().");
@@ -38,18 +72,105 @@ void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::Connect(const FString& AgentIDO
return; return;
} }
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Log, TEXT("Connecting to ElevenLabs: %s"), *URL);
ConnectionState = EPS_AI_ConvAgent_ConnectionState_ElevenLabs::Connecting; ConnectionState = EPS_AI_ConvAgent_ConnectionState_ElevenLabs::Connecting;
// Headers: the ElevenLabs Conversational AI WS endpoint accepts the
// xi-api-key header on the initial HTTP upgrade request.
TMap<FString, FString> UpgradeHeaders; TMap<FString, FString> UpgradeHeaders;
const UPS_AI_ConvAgent_Settings_ElevenLabs* Settings = FPS_AI_ConvAgentModule::Get().GetSettings();
const FString ResolvedKey = APIKeyOverride.IsEmpty() ? Settings->API_Key : APIKeyOverride;
if (!ResolvedKey.IsEmpty()) if (!ResolvedKey.IsEmpty())
{ {
UpgradeHeaders.Add(TEXT("xi-api-key"), ResolvedKey); UpgradeHeaders.Add(TEXT("xi-api-key"), ResolvedKey);
} }
OpenWebSocket(URL, UpgradeHeaders);
}
void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::RequestSignedURLAndConnect(const FString& AgentID, const FString& APIKey)
{
const UPS_AI_ConvAgent_Settings_ElevenLabs* Settings = FPS_AI_ConvAgentModule::Get().GetSettings();
const FString RequestURL = FString::Printf(
TEXT("%s/v1/convai/conversation/get-signed-url?agent_id=%s"),
*Settings->GetAPIBaseURL(), *AgentID);
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Log, TEXT("Requesting signed URL for agent %s"), *AgentID);
TSharedRef<IHttpRequest, ESPMode::ThreadSafe> Request = FHttpModule::Get().CreateRequest();
Request->SetURL(RequestURL);
Request->SetVerb(TEXT("GET"));
Request->SetHeader(TEXT("xi-api-key"), APIKey);
TWeakObjectPtr<UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy> WeakThis(this);
Request->OnProcessRequestComplete().BindLambda(
[WeakThis](FHttpRequestPtr /*Req*/, FHttpResponsePtr Resp, bool bSuccess)
{
if (!WeakThis.IsValid())
{
return;
}
UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy* Self = WeakThis.Get();
const int32 Code = Resp.IsValid() ? Resp->GetResponseCode() : 0;
if (bSuccess && Code == 200 && Resp.IsValid())
{
FString SignedURL;
TSharedPtr<FJsonObject> Root;
const TSharedRef<TJsonReader<>> Reader = TJsonReaderFactory<>::Create(Resp->GetContentAsString());
if (FJsonSerializer::Deserialize(Reader, Root) && Root.IsValid())
{
Root->TryGetStringField(TEXT("signed_url"), SignedURL);
}
if (!SignedURL.IsEmpty())
{
// Token is embedded in the signed URL — no auth header needed.
Self->OpenWebSocket(SignedURL, TMap<FString, FString>());
return;
}
const FString Msg = TEXT("Signed URL request succeeded but the response had no 'signed_url'.");
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Error, TEXT("%s"), *Msg);
Self->ConnectionState = EPS_AI_ConvAgent_ConnectionState_ElevenLabs::Error;
Self->OnError.Broadcast(Msg);
return;
}
FString Msg;
if (Code == 401 || Code == 403)
{
Msg = FString::Printf(TEXT("Authentication failed (HTTP %d): invalid or unauthorized ElevenLabs API key."), Code);
}
else if (Code == 0)
{
Msg = TEXT("Could not reach ElevenLabs to obtain a signed URL (network error).");
}
else
{
Msg = FString::Printf(TEXT("Failed to obtain a signed URL (HTTP %d)."), Code);
}
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Error, TEXT("%s"), *Msg);
Self->ConnectionState = EPS_AI_ConvAgent_ConnectionState_ElevenLabs::Error;
Self->OnError.Broadcast(Msg);
});
Request->ProcessRequest();
}
void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::OpenWebSocket(const FString& URL, const TMap<FString, FString>& UpgradeHeaders)
{
if (URL.IsEmpty())
{
const FString Msg = TEXT("Cannot open WebSocket: empty URL.");
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Error, TEXT("%s"), *Msg);
ConnectionState = EPS_AI_ConvAgent_ConnectionState_ElevenLabs::Error;
OnError.Broadcast(Msg);
return;
}
// Redact the query string (signed URLs carry an auth token there) before logging.
FString LogURL = URL;
int32 QueryIdx = INDEX_NONE;
if (URL.FindChar(TEXT('?'), QueryIdx))
{
LogURL = URL.Left(QueryIdx) + TEXT("?<params hidden>");
}
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Log, TEXT("Connecting to ElevenLabs: %s"), *LogURL);
WebSocket = FWebSocketsModule::Get().CreateWebSocket(URL, TEXT(""), UpgradeHeaders); WebSocket = FWebSocketsModule::Get().CreateWebSocket(URL, TEXT(""), UpgradeHeaders);
@@ -164,6 +285,23 @@ void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::SendTextMessage(const FString&
SendJsonMessage(Msg); SendJsonMessage(Msg);
} }
void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::SendContextualUpdate(const FString& Text)
{
if (!IsConnected())
{
UE_LOG(LogPS_AI_ConvAgent_WS_ElevenLabs, Warning, TEXT("SendContextualUpdate: not connected."));
return;
}
if (Text.IsEmpty()) return;
// API: { "type": "contextual_update", "text": "..." } — injects context WITHOUT
// triggering an agent response (unlike user_message).
TSharedPtr<FJsonObject> Msg = MakeShareable(new FJsonObject());
Msg->SetStringField(TEXT("type"), TEXT("contextual_update"));
Msg->SetStringField(TEXT("text"), Text);
SendJsonMessage(Msg);
}
void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::SendInterrupt() void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::SendInterrupt()
{ {
if (!IsConnected()) return; if (!IsConnected()) return;
@@ -685,15 +823,18 @@ void UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy::HandleClientToolCall(const TSha
{ {
for (const auto& Pair : (*ParamsObj)->Values) for (const auto& Pair : (*ParamsObj)->Values)
{ {
// UE 5.8 changed FJsonObject::Values from TMap<FString,...> to TMap<FSharedString,...>.
// FSharedString has no implicit FString conversion; deref (*) yields const TCHAR*.
FString Key(*Pair.Key);
FString Value; FString Value;
if (Pair.Value->TryGetString(Value)) if (Pair.Value->TryGetString(Value))
{ {
ToolCall.Parameters.Add(Pair.Key, Value); ToolCall.Parameters.Add(Key, Value);
} }
else else
{ {
// For non-string values, serialize to string // For non-string values, serialize to string
ToolCall.Parameters.Add(Pair.Key, Pair.Value->AsString()); ToolCall.Parameters.Add(Key, Pair.Value->AsString());
} }
} }
} }

View File

@@ -31,13 +31,12 @@ class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_Settings_ElevenLabs : public UObject
GENERATED_BODY() GENERATED_BODY()
public: public:
/** // NOTE: The ElevenLabs API key is NOT stored here on purpose.
* ElevenLabs API key. // It is provided per-install by the client at runtime and persisted in a
* Obtain from https://elevenlabs.io used to authenticate WebSocket connections. // SaveGame slot (see UPS_AI_ConvAgent_SaveGame_ElevenLabs). Read/write it via
* Keep this secret; do not ship with the key hard-coded in a shipping build. // UPS_AI_ConvAgent_BlueprintLibrary::Get/SetElevenLabsAPIKey(). This keeps the
*/ // key out of Project Settings / DefaultEngine.ini so it can never be committed
UPROPERTY(Config, EditAnywhere, Category = "PS AI ConvAgent|ElevenLabs API") // or baked into a shipped build.
FString API_Key;
/** /**
* Server region for ElevenLabs API. * Server region for ElevenLabs API.
@@ -45,18 +44,18 @@ public:
* - US: force US servers (api.us.elevenlabs.io). * - US: force US servers (api.us.elevenlabs.io).
* - EU / India: Enterprise-only data residency endpoints. * - EU / India: Enterprise-only data residency endpoints.
*/ */
UPROPERTY(Config, EditAnywhere, Category = "PS AI ConvAgent|ElevenLabs API") UPROPERTY(Config, EditAnywhere, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs API")
EPS_AI_ConvAgent_ElevenLabsRegion ServerRegion = EPS_AI_ConvAgent_ElevenLabsRegion::Global; EPS_AI_ConvAgent_ElevenLabsRegion ServerRegion = EPS_AI_ConvAgent_ElevenLabsRegion::Global;
/** /**
* Override the ElevenLabs WebSocket URL entirely. Leave empty to use ServerRegion setting. * Override the ElevenLabs WebSocket URL entirely. Leave empty to use ServerRegion setting.
* Example: wss://custom-proxy.example.com/v1/convai/conversation?agent_id=YOUR_ID * Example: wss://custom-proxy.example.com/v1/convai/conversation?agent_id=YOUR_ID
*/ */
UPROPERTY(Config, EditAnywhere, AdvancedDisplay, Category = "PS AI ConvAgent|ElevenLabs API") UPROPERTY(Config, EditAnywhere, AdvancedDisplay, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs API")
FString CustomWebSocketURL; FString CustomWebSocketURL;
/** Log verbose WebSocket messages to the Output Log (useful during development). */ /** Log verbose WebSocket messages to the Output Log (useful during development). */
UPROPERTY(Config, EditAnywhere, AdvancedDisplay, Category = "PS AI ConvAgent|ElevenLabs API") UPROPERTY(Config, EditAnywhere, AdvancedDisplay, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs API")
bool bVerboseLogging = false; bool bVerboseLogging = false;
/** /**
@@ -65,7 +64,7 @@ public:
* Injected at conversation start via dynamic variables — changes take effect * Injected at conversation start via dynamic variables — changes take effect
* on the next conversation without re-syncing agents. * on the next conversation without re-syncing agents.
*/ */
UPROPERTY(Config, EditAnywhere, Category = "PS AI ConvAgent|Global Prompt", UPROPERTY(Config, EditAnywhere, Category = "ASTERION|PS_AI_ConvAgent|Global Prompt",
meta = (MultiLine = "true", meta = (MultiLine = "true",
ToolTip = "Context prompt shared by all agents.\nDescribe the world, era, common rules.\nChanges take effect on next conversation start (no re-sync needed).")) ToolTip = "Context prompt shared by all agents.\nDescribe the world, era, common rules.\nChanges take effect on next conversation start (no re-sync needed)."))
FString GlobalContextPrompt; FString GlobalContextPrompt;
@@ -119,7 +118,8 @@ public:
return FModuleManager::Get().IsModuleLoaded("PS_AI_ConvAgent"); return FModuleManager::Get().IsModuleLoaded("PS_AI_ConvAgent");
} }
/** Access the settings object at runtime */ /** Access the settings object at runtime (region, custom URL, logging, global
* prompt). The API key is NOT here — see UPS_AI_ConvAgent_BlueprintLibrary. */
UPS_AI_ConvAgent_Settings_ElevenLabs* GetSettings() const; UPS_AI_ConvAgent_Settings_ElevenLabs* GetSettings() const;
private: private:

View File

@@ -5,7 +5,7 @@
#include "CoreMinimal.h" #include "CoreMinimal.h"
#include "Engine/DataAsset.h" #include "Engine/DataAsset.h"
#include "PS_AI_ConvAgent_Definitions.h" #include "PS_AI_ConvAgent_Definitions.h"
#include "PS_AI_ConvAgent_ActionSet_ElevenLabs.h" #include "PS_AI_ConvAgent_Tool_ElevenLabs.h"
#include "PS_AI_ConvAgent_AgentConfig_ElevenLabs.generated.h" #include "PS_AI_ConvAgent_AgentConfig_ElevenLabs.generated.h"
/** /**
@@ -206,36 +206,18 @@ public:
meta = (ToolTip = "Start generating a response before confirming end-of-speech.\nReduces latency but may cause occasional false starts.\nDisable if the agent interrupts the user too often.")) meta = (ToolTip = "Start generating a response before confirming end-of-speech.\nReduces latency but may cause occasional false starts.\nDisable if the agent interrupts the user too often."))
bool bSpeculativeTurn = false; bool bSpeculativeTurn = false;
// ── Emotion Tool ───────────────────────────────────────────────────────── // ── Tools ────────────────────────────────────────────────────────────────
/** Include the built-in "set_emotion" client tool in the agent configuration. /** Standalone tools assigned to this agent.
* Allows the LLM to set facial expressions (Joy, Sadness, Anger, etc.) * Each tool is a global resource on ElevenLabs (managed via its own Data Asset).
* that drive the FacialExpression component in real-time. */ * The tool's PromptFragment is appended to CharacterPrompt on Create/Update.
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Emotion Tool", * The tool's ToolID is sent in prompt.tool_ids to the ElevenLabs API.
meta = (ToolTip = "Include the set_emotion client tool.\nAllows the LLM to drive facial expressions.")) *
bool bIncludeEmotionTool = true; * Drag Tool Data Assets here (e.g. set_emotion, perform_action, custom tools).
* Create tools in Content Browser: Miscellaneous > Data Asset > PS AI ConvAgent Tool. */
/** System prompt fragment appended to CharacterPrompt when bIncludeEmotionTool is true. UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Tools",
* Pre-filled with the standard emotion instruction. Editable for customization. */ meta = (ToolTip = "Standalone tools assigned to this agent.\nDrag Tool Data Assets here.\nEach tool's PromptFragment is appended to the system prompt."))
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Emotion Tool", TArray<TObjectPtr<UPS_AI_ConvAgent_Tool_ElevenLabs>> Tools;
meta = (MultiLine = "true", EditCondition = "bIncludeEmotionTool",
ToolTip = "Prompt instructions for the emotion tool.\nAppended to CharacterPrompt when creating/updating the agent."))
FString EmotionToolPromptFragment = TEXT(
"## Facial Expressions\n"
"You have a set_emotion tool to control your facial expression. "
"Use it whenever the emotional context changes:\n"
"- Call set_emotion with emotion=\"joy\" when happy, laughing, or excited\n"
"- Call set_emotion with emotion=\"sadness\" when empathetic or discussing sad topics\n"
"- Call set_emotion with emotion=\"anger\" when frustrated or discussing injustice\n"
"- Call set_emotion with emotion=\"surprise\" when reacting to unexpected information\n"
"- Call set_emotion with emotion=\"fear\" when discussing scary or worrying topics\n"
"- Call set_emotion with emotion=\"disgust\" when reacting to unpleasant things\n"
"- Call set_emotion with emotion=\"neutral\" to return to a calm expression\n\n"
"Use intensity to match the strength of the emotion:\n"
"- \"low\" for subtle hints (slight smile, mild concern)\n"
"- \"medium\" for normal expression (default)\n"
"- \"high\" for strong reactions (big laugh, deep sadness, shock)\n\n"
"Always return to neutral when the emotional moment passes.");
// ── Expressive Mode (V3 Conversational) ───────────────────────────────── // ── Expressive Mode (V3 Conversational) ─────────────────────────────────
@@ -266,24 +248,6 @@ public:
"Example: \"That's great to hear! [laughs] I'm glad we could sort that out for you.\"\n\n" "Example: \"That's great to hear! [laughs] I'm glad we could sort that out for you.\"\n\n"
"Use these tags naturally and sparingly to enhance expressiveness without overusing them."); "Use these tags naturally and sparingly to enhance expressiveness without overusing them.");
// ── Action Tool ─────────────────────────────────────────────────────────
/** Include a configurable "perform_action" client tool in the agent configuration.
* Allows the LLM to trigger physical actions defined in the referenced ActionSet.
* Actions are handled via the OnAgentActionRequested event in Blueprint. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Action Tool",
meta = (ToolTip = "Include the perform_action client tool.\nRequires an ActionSet Data Asset with at least one action."))
bool bIncludeActionTool = false;
/** Reference to a reusable Action Set Data Asset.
* Create one in Content Browser (Miscellaneous > Data Asset > PS AI ConvAgent Action Set),
* define your actions there, then drag it here.
* The same ActionSet can be shared by multiple agents. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Action Tool",
meta = (EditCondition = "bIncludeActionTool",
ToolTip = "Drag an ActionSet Data Asset here.\nDefines which actions the agent can trigger."))
TObjectPtr<UPS_AI_ConvAgent_ActionSet_ElevenLabs> ActionSet;
// ── Dynamic Variables ──────────────────────────────────────────────────── // ── Dynamic Variables ────────────────────────────────────────────────────
/** Key-value pairs sent as dynamic_variables at conversation start. /** Key-value pairs sent as dynamic_variables at conversation start.

View File

@@ -0,0 +1,181 @@
// Copyright ASTERION. All Rights Reserved.
#pragma once
#include "CoreMinimal.h"
#include "GameFramework/Actor.h"
#include "PS_AI_ConvAgent_ElevenLabsComponent.h" // component type + delegates + ConversationInfo struct
#include "PS_AI_ConvAgent_AgentDialogue.generated.h"
class UPS_AI_ConvAgent_ElevenLabsComponent;
/** Fired each time a speaker's line is forwarded to the other agent (subtitles / logging). */
DECLARE_DYNAMIC_MULTICAST_DELEGATE_ThreeParams(FPS_AI_ConvAgent_OnDialogueTurn,
int32, TurnIndex, AActor*, Speaker, const FString&, Text);
/** Fired when the dialogue stops (MaxTurns reached or StopDialogue). */
DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam(FPS_AI_ConvAgent_OnDialogueFinished,
int32, TotalTurns);
/**
* Orchestrator actor that makes TWO ElevenLabs agents converse with each other
* over TEXT (no STT): each agent's completed text response is fed as a user
* message to the OTHER agent, which then replies. Both agents still speak aloud
* (TTS), so humans in the level hear the whole conversation.
*
* Setup: place this actor in the level, assign AgentA / AgentB to two actors that
* each already carry a PS_AI_ConvAgent_ElevenLabsComponent, set an OpeningMessage,
* then call StartDialogue() (or enable bAutoStartOnBeginPlay).
*
* Runs on the SERVER (authority) — the agents' WebSockets are server-owned.
* By default forwarding happens when a speaker FINISHES speaking so the two
* voices never overlap (needs audio playback → listen-server/host). On a
* dedicated server with no audio device, set bWaitForSpeechToFinish=false to
* drive turns from the text event instead.
*/
UCLASS(Blueprintable)
class PS_AI_CONVAGENT_API APS_AI_ConvAgent_AgentDialogue : public AActor
{
GENERATED_BODY()
public:
APS_AI_ConvAgent_AgentDialogue();
/** Actor carrying the FIRST agent's ElevenLabs component. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue")
TObjectPtr<AActor> AgentA;
/** Actor carrying the SECOND agent's ElevenLabs component. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue")
TObjectPtr<AActor> AgentB;
/** Opening line sent to the first speaker to kick off the dialogue. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue")
FString OpeningMessage = TEXT("Bonjour.");
/** If true the opening message goes to AgentA (A replies first); else to AgentB. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue")
bool bAgentAStarts = true;
/** Max forwarded turns before auto-stopping. 0 = unlimited. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue", meta = (ClampMin = "0"))
int32 MaxTurns = 20;
/** Pause (s) after a speaker's turn before the other is prompted. Adds breathing room. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue", meta = (ClampMin = "0.0"))
float TurnDelaySeconds = 0.0f;
/** True (default): forward a line when the speaker FINISHES speaking (clean, non-overlapping voices; needs audio playback).
* False: forward as soon as the text is ready (robust on dedicated servers, but voices may overlap). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue")
bool bWaitForSpeechToFinish = true;
/** Start the dialogue automatically on BeginPlay (authority only). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue")
bool bAutoStartOnBeginPlay = false;
/** After the player interrupts (by speaking), ALL dialogue agents look at the
* player until the player moves farther than this (cm) from BOTH agents, then
* the agents are released back to normal behavior and their conversations end. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue", meta = (ClampMin = "0.0"))
float PlayerAddressReleaseDistance = 500.0f;
/** Sent to BOTH agents as a SILENT context update (ElevenLabs contextual_update — no
* reply triggered) the instant the player interrupts, so the agents know a new/real
* person is now addressing them while KEEPING the prior conversation context.
* Leave empty to inject nothing. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue", meta = (MultiLine = "true"))
FString InterruptContextMessage = TEXT("Une nouvelle personne, bien reelle, s'adresse maintenant directement a toi, ce n'est plus ton interlocuteur precedent. Garde en memoire la conversation qui vient d'avoir lieu, mais reponds desormais a cette personne.");
/** Sent (contextual_update, no reply) to the agent the player is NOT talking to: it
* is a witness. Empty = don't inform the bystander. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue", meta = (MultiLine = "true"))
FString BystanderContextMessage = TEXT("Une nouvelle personne, bien reelle, vient d'entrer et s'adresse a ton interlocuteur, pas a toi pour l'instant. Ecoute ce qui se dit, tu pourras y faire reference plus tard quand elle se tournera vers toi.");
/** Prefix for the "overheard" lines relayed to the witness agent during a 3-way
* exchange (contextual_update, no reply). Empty = disable passive overhearing. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Dialogue")
FString OverheardPrefix = TEXT("[Tu entends la conversation a cote] ");
/** Fired each time a line is forwarded (TurnIndex, Speaker actor, the text). */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|Dialogue")
FPS_AI_ConvAgent_OnDialogueTurn OnDialogueTurn;
/** Fired when the dialogue ends. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|Dialogue")
FPS_AI_ConvAgent_OnDialogueFinished OnDialogueFinished;
/** Open both conversations and kick off the exchange. Authority only. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Dialogue")
void StartDialogue();
/** Stop forwarding turns. Agents stay connected unless bEndConversations. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Dialogue")
void StopDialogue(bool bEndConversations = false);
/** True while the loop is active. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|Dialogue")
bool IsRunning() const { return bRunning; }
protected:
virtual void BeginPlay() override;
virtual void EndPlay(const EEndPlayReason::Type Reason) override;
virtual void Tick(float DeltaSeconds) override; // only active in "addressing player" mode
private:
UPROPERTY(Transient) TObjectPtr<UPS_AI_ConvAgent_ElevenLabsComponent> CompA;
UPROPERTY(Transient) TObjectPtr<UPS_AI_ConvAgent_ElevenLabsComponent> CompB;
// Single-floor turn state: exactly ONE agent holds the floor at a time; every
// event from the OTHER agent is ignored. A turn forwards exactly once, when the
// floor-holder's text AND its speech are both complete.
enum class EFloor : uint8 { None, A, B };
EFloor Floor = EFloor::None;
int32 TurnId = 0; // number of forwarded turns
bool bTextReady = false; // floor-holder's full text arrived this turn
bool bSpeechDone = false; // floor-holder finished speaking this turn
bool bForwardedThisTurn = false; // exactly-one-forward guard
FString PendingLine; // floor-holder's line for this turn
bool bRunning = false;
bool bOpeningSent = false;
FTimerHandle TurnTimer;
// "Addressing player" mode: after the player interrupts by speaking, ALL dialogue
// agents are pinned to look at the player (fast, via the gaze override) and the
// orchestrator ticks to auto-release them once the player leaves both agents.
bool bAddressingPlayer = false;
UPROPERTY(Transient) TObjectPtr<APawn> AddressedByPlayer;
EFloor EngagedFloor = EFloor::None; // which agent the player is currently addressing
APawn* ResolveSpeakingPlayer() const; // the player whose voice was just transcribed
void EndPlayerAddress(); // release agents → normal + end conversations
UPS_AI_ConvAgent_ElevenLabsComponent* ResolveAgentComp(AActor* Actor) const;
void BindAgent(UPS_AI_ConvAgent_ElevenLabsComponent* Comp, bool bIsA);
void UnbindAll();
void TryKickoff();
void BeginTurn(EFloor NewFloor); // set floor + reset per-turn flags
bool IsFloor(bool bFromA) const; // is the given agent the current floor-holder?
void TryForward(); // forward once, when text AND speech done
UFUNCTION() void HandleTextA(const FString& Text);
UFUNCTION() void HandleTextB(const FString& Text);
UFUNCTION() void HandleStoppedA();
UFUNCTION() void HandleStoppedB();
UFUNCTION() void HandleConnectedA(const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& Info);
UFUNCTION() void HandleConnectedB(const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& Info);
// Per-agent interrupt/relay triggers. Knowing WHICH agent the player addressed lets
// us track the engaged agent and relay the exchange to the other ("witness") agent.
// Voice-onset = primary immediate trigger; transcript = backup + carries the words.
// Merely LOOKING at an agent never triggers these — only actually speaking does.
UFUNCTION() void HandleVoiceOnsetA();
UFUNCTION() void HandleVoiceOnsetB();
UFUNCTION() void HandleTranscriptA(const FPS_AI_ConvAgent_TranscriptSegment_ElevenLabs& Segment);
UFUNCTION() void HandleTranscriptB(const FPS_AI_ConvAgent_TranscriptSegment_ElevenLabs& Segment);
void OnPlayerSpoke(EFloor Which, const FString& PlayerWords); // player addressed agent Which
void InterruptForPlayer(EFloor Engaged); // enter 3-way floor mode (Engaged = addressee)
void SetEngaged(EFloor Which); // reframe both agents when the addressee changes
void RelayToBystander(EFloor EngagedSide, const FString& FormattedLine); // "overheard" context
};

View File

@@ -24,8 +24,34 @@ public:
* This allows per-instance override without modifying the Skeletal Mesh asset. * This allows per-instance override without modifying the Skeletal Mesh asset.
* @param SkelMeshComp The target Skeletal Mesh Component. * @param SkelMeshComp The target Skeletal Mesh Component.
* @param AnimBPClass The AnimBlueprint class to use as post-process (nullptr to clear). */ * @param AnimBPClass The AnimBlueprint class to use as post-process (nullptr to clear). */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Utilities", UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Utilities",
meta = (DisplayName = "Set Post Process Anim Blueprint")) meta = (DisplayName = "Set Post Process Anim Blueprint"))
static void SetPostProcessAnimBlueprint(USkeletalMeshComponent* SkelMeshComp, static void SetPostProcessAnimBlueprint(USkeletalMeshComponent* SkelMeshComp,
TSubclassOf<UAnimInstance> AnimBPClass); TSubclassOf<UAnimInstance> AnimBPClass);
// ── ElevenLabs API key (per-user, client-provided) ────────────────────────
/** Store the client's ElevenLabs API key. Persists it per-user in
* Saved/Config/.../Game.ini AND applies it immediately to the running plugin
* (no restart needed). Call this from your settings UI once the user has
* entered — and ideally validated via "Test ElevenLabs API Key" — their key. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs API Key",
meta = (DisplayName = "Set ElevenLabs API Key"))
static void SetElevenLabsAPIKey(const FString& Key);
/** Return the currently stored client ElevenLabs API key (empty if none). */
UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs API Key",
meta = (DisplayName = "Get ElevenLabs API Key"))
static FString GetElevenLabsAPIKey();
/** True if a non-empty client ElevenLabs API key has been stored.
* Use this on startup to decide whether to force the user to the key entry screen. */
UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs API Key",
meta = (DisplayName = "Has ElevenLabs API Key"))
static bool HasElevenLabsAPIKey();
/** Clear the stored client ElevenLabs API key (both on disk and in memory). */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs API Key",
meta = (DisplayName = "Clear ElevenLabs API Key"))
static void ClearElevenLabsAPIKey();
}; };

View File

@@ -55,7 +55,7 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE_TwoParams(FOnBodyExpressionChanged,
// 4. The component listens to conversation lifecycle + emotion events. // 4. The component listens to conversation lifecycle + emotion events.
// 5. The AnimNode reads GetSnapshot() and blends the upper body pose. // 5. The AnimNode reads GetSnapshot() and blends the upper body pose.
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
UCLASS(ClassGroup = "PS AI ConvAgent", meta = (BlueprintSpawnableComponent), UCLASS(ClassGroup = "ASTERION|PS_AI_ConvAgent", meta = (BlueprintSpawnableComponent),
DisplayName = "PS AI ConvAgent Body Expression") DisplayName = "PS AI ConvAgent Body Expression")
class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_BodyExpressionComponent : public UActorComponent class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_BodyExpressionComponent : public UActorComponent
{ {
@@ -68,28 +68,28 @@ public:
/** Body pose map asset containing body animation lists per emotion. /** Body pose map asset containing body animation lists per emotion.
* Create a dedicated PS_AI_ConvAgent_BodyPoseMap asset in the Content Browser. */ * Create a dedicated PS_AI_ConvAgent_BodyPoseMap asset in the Content Browser. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|BodyExpression", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression",
meta = (ToolTip = "Body Pose Map asset with animation lists per emotion.\nRight-click Content Browser > Miscellaneous > PS AI ConvAgent Body Pose Map.")) meta = (ToolTip = "Body Pose Map asset with animation lists per emotion.\nRight-click Content Browser > Miscellaneous > PS AI ConvAgent Body Pose Map."))
TObjectPtr<UPS_AI_ConvAgent_BodyPoseMap> BodyPoseMap; TObjectPtr<UPS_AI_ConvAgent_BodyPoseMap> BodyPoseMap;
/** Whether body expressions are currently active (read-only). /** Whether body expressions are currently active (read-only).
* Managed automatically: activates on conversation connect, deactivates on disconnect. */ * Managed automatically: activates on conversation connect, deactivates on disconnect. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|BodyExpression") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression")
bool bActive = false; bool bActive = false;
/** How long (seconds) to blend in/out when conversation starts/ends. */ /** How long (seconds) to blend in/out when conversation starts/ends. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|BodyExpression", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression",
meta = (ClampMin = "0.05", ClampMax = "3.0")) meta = (ClampMin = "0.05", ClampMax = "3.0"))
float ActivationBlendDuration = 0.5f; float ActivationBlendDuration = 0.5f;
/** Crossfade duration (seconds) when switching between animations. */ /** Crossfade duration (seconds) when switching between animations. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|BodyExpression", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression",
meta = (ClampMin = "0.1", ClampMax = "3.0", meta = (ClampMin = "0.1", ClampMax = "3.0",
ToolTip = "How long (seconds) to crossfade between animations.\n0.5 = snappy, 1.5 = smooth.")) ToolTip = "How long (seconds) to crossfade between animations.\n0.5 = snappy, 1.5 = smooth."))
float EmotionBlendDuration = 1.0f; float EmotionBlendDuration = 1.0f;
/** Overall blend weight for body expressions. 1.0 = full, 0.5 = subtle. */ /** Overall blend weight for body expressions. 1.0 = full, 0.5 = subtle. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|BodyExpression", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression",
meta = (ClampMin = "0.0", ClampMax = "1.0", meta = (ClampMin = "0.0", ClampMax = "1.0",
ToolTip = "Overall blend weight for body expressions.\n1.0 = full, 0.5 = subtle.")) ToolTip = "Overall blend weight for body expressions.\n1.0 = full, 0.5 = subtle."))
float BlendWeight = 1.0f; float BlendWeight = 1.0f;
@@ -97,12 +97,12 @@ public:
// ── Debug ──────────────────────────────────────────────────────────────── // ── Debug ────────────────────────────────────────────────────────────────
/** Enable debug logging for this component. */ /** Enable debug logging for this component. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug")
bool bDebug = false; bool bDebug = false;
/** Verbosity level when bDebug is true. /** Verbosity level when bDebug is true.
* 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */ * 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug",
meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug")) meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug"))
int32 DebugVerbosity = 1; int32 DebugVerbosity = 1;
@@ -117,22 +117,22 @@ public:
} }
/** Get the active emotion. */ /** Get the active emotion. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|BodyExpression") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression")
EPS_AI_ConvAgent_Emotion GetActiveEmotion() const { return ActiveEmotion; } EPS_AI_ConvAgent_Emotion GetActiveEmotion() const { return ActiveEmotion; }
/** Get the active emotion intensity. */ /** Get the active emotion intensity. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|BodyExpression") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression")
EPS_AI_ConvAgent_EmotionIntensity GetActiveIntensity() const { return ActiveEmotionIntensity; } EPS_AI_ConvAgent_EmotionIntensity GetActiveIntensity() const { return ActiveEmotionIntensity; }
/** True when the agent is currently speaking (body uses N/M/E lists). */ /** True when the agent is currently speaking (body uses N/M/E lists). */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|BodyExpression") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression")
bool IsSpeaking() const { return bIsSpeaking; } bool IsSpeaking() const { return bIsSpeaking; }
// ── Events ─────────────────────────────────────────────────────────────── // ── Events ───────────────────────────────────────────────────────────────
/** Fired when the body expression changes (emotion + intensity). /** Fired when the body expression changes (emotion + intensity).
* Only fires when the emotion actually differs from the previous one. */ * Only fires when the emotion actually differs from the previous one. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|BodyExpression|Events") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|BodyExpression|Events")
FOnBodyExpressionChanged OnBodyExpressionChanged; FOnBodyExpressionChanged OnBodyExpressionChanged;
/** Current activation blend alpha (0 = fully inactive, 1 = fully active). /** Current activation blend alpha (0 = fully inactive, 1 = fully active).

View File

@@ -91,11 +91,11 @@ struct PS_AI_CONVAGENT_API FPS_AI_ConvAgent_ConversationInfo_ElevenLabs
GENERATED_BODY() GENERATED_BODY()
/** Unique ID of this conversation session assigned by ElevenLabs. */ /** Unique ID of this conversation session assigned by ElevenLabs. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString ConversationID; FString ConversationID;
/** Agent ID that is responding. */ /** Agent ID that is responding. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString AgentID; FString AgentID;
}; };
@@ -108,15 +108,15 @@ struct PS_AI_CONVAGENT_API FPS_AI_ConvAgent_TranscriptSegment_ElevenLabs
GENERATED_BODY() GENERATED_BODY()
/** Transcribed text. */ /** Transcribed text. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString Text; FString Text;
/** "user" or "agent". */ /** "user" or "agent". */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString Speaker; FString Speaker;
/** Whether this is a final transcript or a tentative (in-progress) one. */ /** Whether this is a final transcript or a tentative (in-progress) one. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
bool bIsFinal = false; bool bIsFinal = false;
}; };
@@ -155,20 +155,20 @@ struct PS_AI_CONVAGENT_API FPS_AI_ConvAgent_ClientToolCall_ElevenLabs
GENERATED_BODY() GENERATED_BODY()
/** Name of the tool the agent wants to invoke (e.g. "set_emotion"). */ /** Name of the tool the agent wants to invoke (e.g. "set_emotion"). */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString ToolName; FString ToolName;
/** Unique ID for this tool invocation — must be echoed back in client_tool_result. */ /** Unique ID for this tool invocation — must be echoed back in client_tool_result. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString ToolCallId; FString ToolCallId;
/** Raw JSON parameters as key-value string pairs. */ /** Raw JSON parameters as key-value string pairs. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
TMap<FString, FString> Parameters; TMap<FString, FString> Parameters;
}; };
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
// Agent action definition (used by ActionSet Data Asset) // Agent action definition (used by ActionSet Data Asset — deprecated, kept for compat)
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
/** Defines a single action that an agent can perform during conversation. */ /** Defines a single action that an agent can perform during conversation. */
USTRUCT(BlueprintType) USTRUCT(BlueprintType)
@@ -177,10 +177,64 @@ struct PS_AI_CONVAGENT_API FPS_AI_ConvAgent_AgentAction_ElevenLabs
GENERATED_BODY() GENERATED_BODY()
/** Action identifier sent by the LLM (e.g. "flee", "draw_weapon"). Keep short, snake_case. */ /** Action identifier sent by the LLM (e.g. "flee", "draw_weapon"). Keep short, snake_case. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString Name; FString Name;
/** Human-readable description — helps the LLM understand when to use this action. */ /** Human-readable description — helps the LLM understand when to use this action. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
FString Description; FString Description;
}; };
// ─────────────────────────────────────────────────────────────────────────────
// Tool parameter data type (maps to JSON Schema "type")
// ─────────────────────────────────────────────────────────────────────────────
UENUM(BlueprintType)
enum class EPS_AI_ConvAgent_ToolParamType : uint8
{
String UMETA(DisplayName = "String"),
Integer UMETA(DisplayName = "Integer"),
Number UMETA(DisplayName = "Number"),
Boolean UMETA(DisplayName = "Boolean"),
};
// ─────────────────────────────────────────────────────────────────────────────
// Tool parameter definition (used by Tool Data Asset)
// ─────────────────────────────────────────────────────────────────────────────
/** Defines a single parameter for a standalone ElevenLabs tool.
* Maps directly to the ElevenLabs tool parameter UI:
* Identifier, Data type, Required, Description (LLM prompt), Enum Values. */
USTRUCT(BlueprintType)
struct PS_AI_CONVAGENT_API FPS_AI_ConvAgent_ToolParameter_ElevenLabs
{
GENERATED_BODY()
/** Parameter identifier (snake_case). Must match the name expected by the client code.
* Examples: "emotion", "intensity", "action", "target_name" */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Parameter name (snake_case).\nThe key sent in the tool call JSON."))
FString Name;
/** Data type for this parameter. Maps to JSON Schema "type". */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Data type: String, Integer, Number, or Boolean."))
EPS_AI_ConvAgent_ToolParamType Type = EPS_AI_ConvAgent_ToolParamType::String;
/** Whether this parameter is required in the tool call. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "If true, the LLM must provide this parameter."))
bool bRequired = true;
/** Description passed to the LLM explaining what this parameter is
* and how to extract or determine its value from the conversation. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (MultiLine = "true",
ToolTip = "LLM prompt describing this parameter.\nGuides the LLM on what value to provide."))
FString Description;
/** Optional predefined values the LLM can choose from.
* If empty, the LLM can use any value of the specified type.
* Examples: ["joy","sadness","anger"] for emotion, ["low","medium","high"] for intensity. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Predefined enum values (optional).\nIf set, the LLM must choose from these."))
TArray<FString> EnumValues;
};

View File

@@ -122,10 +122,21 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnAgentPassiveGazeStarted);
/** Fired when this agent stops passive gaze (left range or became selected). */ /** Fired when this agent stops passive gaze (left range or became selected). */
DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnAgentPassiveGazeStopped); DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnAgentPassiveGazeStopped);
/** Fired on the FIRST voiced frame of the local user's mic (via FeedExternalAudio),
* independent of the ElevenLabs STT round-trip — used to interrupt an agent-to-agent
* dialogue on voice onset. Debounced (~once per utterance). */
DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnUserVoiceDetected);
// Non-dynamic delegate for raw agent audio (high-frequency, C++ consumers only). // Non-dynamic delegate for raw agent audio (high-frequency, C++ consumers only).
// Delivers PCM chunks as int16, 16kHz mono, little-endian. // Delivers PCM chunks as int16, 16kHz mono, little-endian.
DECLARE_MULTICAST_DELEGATE_OneParam(FOnAgentAudioData, const TArray<uint8>& /*PCMData*/); DECLARE_MULTICAST_DELEGATE_OneParam(FOnAgentAudioData, const TArray<uint8>& /*PCMData*/);
/** Fired on the SERVER for each player mic chunk relayed via ServerSendMicAudioFromPlayer
* (int16, 16kHz mono). Carries the speaking pawn so an external recorder (e.g. the
* PS_AI_ConvReplay bridge) can capture player speech for replay. */
DECLARE_DYNAMIC_MULTICAST_DELEGATE_TwoParams(FOnPlayerAudioCaptured,
APawn*, SpeakerPawn, const TArray<uint8>&, PCMData);
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
// UPS_AI_ConvAgent_ElevenLabsComponent // UPS_AI_ConvAgent_ElevenLabsComponent
// //
@@ -139,7 +150,7 @@ DECLARE_MULTICAST_DELEGATE_OneParam(FOnAgentAudioData, const TArray<uint8>& /*PC
// 4. React to events (OnAgentTranscript, OnAgentTextResponse, etc.) in Blueprint. // 4. React to events (OnAgentTranscript, OnAgentTextResponse, etc.) in Blueprint.
// 5. Call EndConversation() when done. // 5. Call EndConversation() when done.
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
UCLASS(ClassGroup = "PS AI ConvAgent", meta = (BlueprintSpawnableComponent), UCLASS(ClassGroup = "ASTERION|PS_AI_ConvAgent", meta = (BlueprintSpawnableComponent),
DisplayName = "PS AI ConvAgent - ElevenLabs Conv") DisplayName = "PS AI ConvAgent - ElevenLabs Conv")
class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_ElevenLabsComponent : public UActorComponent class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_ElevenLabsComponent : public UActorComponent
{ {
@@ -153,18 +164,18 @@ public:
/** Agent configuration data asset. /** Agent configuration data asset.
* When set, the AgentID is resolved from this asset at conversation start. * When set, the AgentID is resolved from this asset at conversation start.
* Create one via Content Browser → right-click → Miscellaneous → PS AI ConvAgent Agent Config. */ * Create one via Content Browser → right-click → Miscellaneous → PS AI ConvAgent Agent Config. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Agent configuration data asset.\nOverrides the AgentID string below when set.")) meta = (ToolTip = "Agent configuration data asset.\nOverrides the AgentID string below when set."))
TObjectPtr<UPS_AI_ConvAgent_AgentConfig_ElevenLabs> AgentConfig; TObjectPtr<UPS_AI_ConvAgent_AgentConfig_ElevenLabs> AgentConfig;
/** ElevenLabs Agent ID used for this conversation. Leave empty to use the default from Project Settings > PS AI ConvAgent - ElevenLabs. /** ElevenLabs Agent ID used for this conversation. Leave empty to use the default from Project Settings > PS AI ConvAgent - ElevenLabs.
* Overridden by AgentConfig if set. */ * Overridden by AgentConfig if set. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "ElevenLabs Agent ID. Leave empty to use the project default from Project Settings.\nOverridden by AgentConfig if set.")) meta = (ToolTip = "ElevenLabs Agent ID. Leave empty to use the project default from Project Settings.\nOverridden by AgentConfig if set."))
FString AgentID; FString AgentID;
/** How turn-taking is managed between the user and the agent.\n- Server VAD (recommended): ElevenLabs automatically detects when the user stops speaking.\n- Client Controlled: You manually call StartListening/StopListening (push-to-talk with a key). */ /** How turn-taking is managed between the user and the agent.\n- Server VAD (recommended): ElevenLabs automatically detects when the user stops speaking.\n- Client Controlled: You manually call StartListening/StopListening (push-to-talk with a key). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Turn-taking mode.\n- Server VAD: ElevenLabs detects end-of-speech automatically (hands-free).\n- Client Controlled: You call StartListening/StopListening manually (push-to-talk).")) meta = (ToolTip = "Turn-taking mode.\n- Server VAD: ElevenLabs detects end-of-speech automatically (hands-free).\n- Client Controlled: You call StartListening/StopListening manually (push-to-talk)."))
EPS_AI_ConvAgent_TurnMode_ElevenLabs TurnMode = EPS_AI_ConvAgent_TurnMode_ElevenLabs::Server; EPS_AI_ConvAgent_TurnMode_ElevenLabs TurnMode = EPS_AI_ConvAgent_TurnMode_ElevenLabs::Server;
@@ -174,38 +185,38 @@ public:
* The agent remembers the full conversation context between interactions. * The agent remembers the full conversation context between interactions.
* When false (ephemeral), each StartConversation opens a fresh WebSocket session * When false (ephemeral), each StartConversation opens a fresh WebSocket session
* and EndConversation closes it. */ * and EndConversation closes it. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Keep the WebSocket open across conversation cycles.\n- True: agent remembers the full conversation between interactions.\n- False: each StartConversation opens a new session.")) meta = (ToolTip = "Keep the WebSocket open across conversation cycles.\n- True: agent remembers the full conversation between interactions.\n- False: each StartConversation opens a new session."))
bool bPersistentSession = true; bool bPersistentSession = true;
/** Automatically open the microphone as soon as the WebSocket connection is established. Only applies in Server VAD mode. In Client (push-to-talk) mode, you must call StartListening manually. */ /** Automatically open the microphone as soon as the WebSocket connection is established. Only applies in Server VAD mode. In Client (push-to-talk) mode, you must call StartListening manually. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Auto-open the microphone when the conversation starts.\nOnly applies in Server VAD mode. In push-to-talk mode, call StartListening() manually.")) meta = (ToolTip = "Auto-open the microphone when the conversation starts.\nOnly applies in Server VAD mode. In push-to-talk mode, call StartListening() manually."))
bool bAutoStartListening = true; bool bAutoStartListening = true;
/** How many milliseconds of microphone audio to accumulate before sending a chunk to ElevenLabs. Lower values reduce latency but may degrade voice detection accuracy. Higher values are more reliable but add delay. */ /** How many milliseconds of microphone audio to accumulate before sending a chunk to ElevenLabs. Lower values reduce latency but may degrade voice detection accuracy. Higher values are more reliable but add delay. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs|Latency", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Latency",
meta = (ClampMin = "20", ClampMax = "500", meta = (ClampMin = "20", ClampMax = "500",
ToolTip = "Mic audio chunk duration sent to ElevenLabs.\n- 50-80ms: lower latency, less reliable voice detection.\n- 100ms (default): good balance.\n- 150-250ms: more reliable, higher latency.")) ToolTip = "Mic audio chunk duration sent to ElevenLabs.\n- 50-80ms: lower latency, less reliable voice detection.\n- 100ms (default): good balance.\n- 150-250ms: more reliable, higher latency."))
int32 MicChunkDurationMs = 100; int32 MicChunkDurationMs = 100;
/** Allow the user to interrupt the agent while it is speaking.\n- In Server VAD mode: the microphone stays open during agent speech and the server detects interruptions automatically.\n- In Client (push-to-talk) mode: pressing the talk key while the agent speaks sends an interrupt signal.\n- When disabled: the user must wait for the agent to finish speaking before talking. */ /** Allow the user to interrupt the agent while it is speaking.\n- In Server VAD mode: the microphone stays open during agent speech and the server detects interruptions automatically.\n- In Client (push-to-talk) mode: pressing the talk key while the agent speaks sends an interrupt signal.\n- When disabled: the user must wait for the agent to finish speaking before talking. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ToolTip = "Allow the user to interrupt the agent while it speaks.\n- Server VAD: mic stays open, server detects user voice automatically.\n- Push-to-talk: pressing the talk key interrupts the agent.\n- Disabled: user must wait for the agent to finish.")) meta = (ToolTip = "Allow the user to interrupt the agent while it speaks.\n- Server VAD: mic stays open, server detects user voice automatically.\n- Push-to-talk: pressing the talk key interrupts the agent.\n- Disabled: user must wait for the agent to finish."))
bool bAllowInterruption = true; bool bAllowInterruption = true;
/** Enable the OnAgentTranscript event, which provides real-time speech-to-text of what the user is saying. Disable if you don't need to display user speech to reduce processing overhead. */ /** Enable the OnAgentTranscript event, which provides real-time speech-to-text of what the user is saying. Disable if you don't need to display user speech to reduce processing overhead. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fire OnAgentTranscript with real-time speech-to-text of user speech.\nDisable if you don't need to display what the user said.")) meta = (ToolTip = "Fire OnAgentTranscript with real-time speech-to-text of user speech.\nDisable if you don't need to display what the user said."))
bool bEnableUserTranscript = true; bool bEnableUserTranscript = true;
/** Enable the OnAgentTextResponse event, which provides the agent's complete text response once fully generated. Disable if you only need the audio output. */ /** Enable the OnAgentTextResponse event, which provides the agent's complete text response once fully generated. Disable if you only need the audio output. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fire OnAgentTextResponse with the agent's complete text once fully generated.\nDisable if you only need the audio output.")) meta = (ToolTip = "Fire OnAgentTextResponse with the agent's complete text once fully generated.\nDisable if you only need the audio output."))
bool bEnableAgentTextResponse = true; bool bEnableAgentTextResponse = true;
/** Enable the OnAgentPartialResponse event, which streams the agent's text word-by-word as the LLM generates it. Use this for real-time subtitles that appear while the agent speaks, rather than waiting for the full response. */ /** Enable the OnAgentPartialResponse event, which streams the agent's text word-by-word as the LLM generates it. Use this for real-time subtitles that appear while the agent speaks, rather than waiting for the full response. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fire OnAgentPartialResponse with streaming text fragments as the LLM generates them.\nIdeal for real-time subtitles. Each event gives one text chunk, not the accumulated text.")) meta = (ToolTip = "Fire OnAgentPartialResponse with streaming text fragments as the LLM generates them.\nIdeal for real-time subtitles. Each event gives one text chunk, not the accumulated text."))
bool bEnableAgentPartialResponse = false; bool bEnableAgentPartialResponse = false;
@@ -214,7 +225,7 @@ public:
* When adaptive mode is on, the system starts here and can only decrease * When adaptive mode is on, the system starts here and can only decrease
* (never increase) as it measures that chunks arrive fast enough. * (never increase) as it measures that chunks arrive fast enough.
* Set to 0 to disable pre-buffering entirely. */ * Set to 0 to disable pre-buffering entirely. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs|Latency", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Latency",
meta = (ClampMin = "0", ClampMax = "4000", meta = (ClampMin = "0", ClampMax = "4000",
ToolTip = "Pre-buffer delay (ms) — your safe 'worst case' value.\nAdaptive mode can only decrease from here, never increase.\nSet 0 to disable pre-buffering entirely.")) ToolTip = "Pre-buffer delay (ms) — your safe 'worst case' value.\nAdaptive mode can only decrease from here, never increase.\nSet 0 to disable pre-buffering entirely."))
int32 AudioPreBufferMs = 300; int32 AudioPreBufferMs = 300;
@@ -223,13 +234,13 @@ public:
* lowers the pre-buffer when TTS chunks arrive fast enough. * lowers the pre-buffer when TTS chunks arrive fast enough.
* The system can only decrease from AudioPreBufferMs — never increase. * The system can only decrease from AudioPreBufferMs — never increase.
* Resets to AudioPreBufferMs at the start of each conversation. */ * Resets to AudioPreBufferMs at the start of each conversation. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs|Latency", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Latency",
meta = (EditCondition = "AudioPreBufferMs > 0", meta = (EditCondition = "AudioPreBufferMs > 0",
ToolTip = "Automatically lower pre-buffer when connection is good.\nCan only decrease, never increase beyond AudioPreBufferMs.\nResets each conversation.")) ToolTip = "Automatically lower pre-buffer when connection is good.\nCan only decrease, never increase beyond AudioPreBufferMs.\nResets each conversation."))
bool bAdaptivePreBuffer = true; bool bAdaptivePreBuffer = true;
/** Safety timeout: if the server does not start generating a response within this many seconds after the user stops speaking, fire OnAgentResponseTimeout. Set to 0 to disable. A normal response starts within 0.1-0.8s. */ /** Safety timeout: if the server does not start generating a response within this many seconds after the user stops speaking, fire OnAgentResponseTimeout. Set to 0 to disable. A normal response starts within 0.1-0.8s. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ClampMin = "0.0", meta = (ClampMin = "0.0",
ToolTip = "Seconds to wait for a server response after the user stops speaking.\nFires OnAgentResponseTimeout if exceeded. Normal latency is 0.1-0.8s.\nSet to 0 to disable. Default: 10s.")) ToolTip = "Seconds to wait for a server response after the user stops speaking.\nFires OnAgentResponseTimeout if exceeded. Normal latency is 0.1-0.8s.\nSet to 0 to disable. Default: 10s."))
float ResponseTimeoutSeconds = 10.0f; float ResponseTimeoutSeconds = 10.0f;
@@ -237,7 +248,7 @@ public:
/** Maximum auto-reconnection attempts on unexpected WebSocket disconnect. /** Maximum auto-reconnection attempts on unexpected WebSocket disconnect.
* Uses exponential backoff (1s, 2s, 4s, ... up to 30s). * Uses exponential backoff (1s, 2s, 4s, ... up to 30s).
* Set to 0 to disable auto-reconnection. */ * Set to 0 to disable auto-reconnection. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs",
meta = (ClampMin = "0", meta = (ClampMin = "0",
ToolTip = "Auto-reconnect attempts on unexpected disconnect.\n0 = disabled. Uses exponential backoff.")) ToolTip = "Auto-reconnect attempts on unexpected disconnect.\n0 = disabled. Uses exponential backoff."))
int32 MaxReconnectAttempts = 5; int32 MaxReconnectAttempts = 5;
@@ -248,7 +259,7 @@ public:
* When set, the agent's audio playback will be spatialized in 3D — volume * When set, the agent's audio playback will be spatialized in 3D — volume
* and panning change based on distance and direction from the listener. * and panning change based on distance and direction from the listener.
* Leave null for non-spatialized (2D) audio (default, same as before). */ * Leave null for non-spatialized (2D) audio (default, same as before). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|ElevenLabs|Audio", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Audio",
meta = (ToolTip = "Sound attenuation for 3D spatialized audio.\nLeave null for non-spatialized (2D) playback.")) meta = (ToolTip = "Sound attenuation for 3D spatialized audio.\nLeave null for non-spatialized (2D) playback."))
USoundAttenuation* SoundAttenuation = nullptr; USoundAttenuation* SoundAttenuation = nullptr;
@@ -256,104 +267,110 @@ public:
/** Enable debug logging for this component. /** Enable debug logging for this component.
* When false, only key events (init, errors, state changes) are logged. */ * When false, only key events (init, errors, state changes) are logged. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug")
bool bDebug = false; bool bDebug = false;
/** Verbosity level when bDebug is true. /** Verbosity level when bDebug is true.
* 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */ * 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug",
meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug")) meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug"))
int32 DebugVerbosity = 1; int32 DebugVerbosity = 1;
/** Show a separate latency debug HUD with detailed per-turn timing breakdown. /** Show a separate latency debug HUD with detailed per-turn timing breakdown.
* Independent from bDebug — can be enabled alone via CVar ps.ai.ConvAgent.Debug.Latency. */ * Independent from bDebug — can be enabled alone via CVar ps.ai.ConvAgent.Debug.Latency. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug")
bool bDebugLatency = false; bool bDebugLatency = false;
// ── Events ──────────────────────────────────────────────────────────────── // ── Events ────────────────────────────────────────────────────────────────
/** Fired when the WebSocket connection is established and the conversation session is ready. Provides the ConversationID and AgentID. */ /** Fired when the WebSocket connection is established and the conversation session is ready. Provides the ConversationID and AgentID. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the connection to ElevenLabs is established and the conversation is ready to begin.")) meta = (ToolTip = "Fires when the connection to ElevenLabs is established and the conversation is ready to begin."))
FOnAgentConnected OnAgentConnected; FOnAgentConnected OnAgentConnected;
/** Fired when the WebSocket connection is closed (gracefully or due to an error). Provides the status code and reason. */ /** Fired when the WebSocket connection is closed (gracefully or due to an error). Provides the status code and reason. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the connection to ElevenLabs is closed. Check StatusCode and Reason for details.")) meta = (ToolTip = "Fires when the connection to ElevenLabs is closed. Check StatusCode and Reason for details."))
FOnAgentDisconnected OnAgentDisconnected; FOnAgentDisconnected OnAgentDisconnected;
/** Fired on any connection or protocol error. The error message describes what went wrong. */ /** Fired on any connection or protocol error. The error message describes what went wrong. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires on connection or protocol errors. The ErrorMessage describes the issue.")) meta = (ToolTip = "Fires on connection or protocol errors. The ErrorMessage describes the issue."))
FOnAgentError OnAgentError; FOnAgentError OnAgentError;
/** Fired with real-time speech-to-text of the user's voice. Includes both tentative (in-progress) and final transcripts. Requires bEnableUserTranscript to be true. */ /** Fired with real-time speech-to-text of the user's voice. Includes both tentative (in-progress) and final transcripts. Requires bEnableUserTranscript to be true. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Real-time speech-to-text of the user's voice.\nIncludes tentative and final transcripts. Enable with bEnableUserTranscript.")) meta = (ToolTip = "Real-time speech-to-text of the user's voice.\nIncludes tentative and final transcripts. Enable with bEnableUserTranscript."))
FOnAgentTranscript OnAgentTranscript; FOnAgentTranscript OnAgentTranscript;
/** Fired once when the agent's complete text response is available. This is the full text that corresponds to the audio the agent speaks. Requires bEnableAgentTextResponse to be true. */ /** Fired once when the agent's complete text response is available. This is the full text that corresponds to the audio the agent speaks. Requires bEnableAgentTextResponse to be true. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "The agent's complete text response (matches the spoken audio).\nFires once when the full text is ready. Enable with bEnableAgentTextResponse.")) meta = (ToolTip = "The agent's complete text response (matches the spoken audio).\nFires once when the full text is ready. Enable with bEnableAgentTextResponse."))
FOnAgentTextResponse OnAgentTextResponse; FOnAgentTextResponse OnAgentTextResponse;
/** Fired repeatedly as the LLM generates text, providing one word/fragment at a time. Use for real-time subtitles. Each call gives a new fragment, NOT the accumulated text. Requires bEnableAgentPartialResponse to be true. */ /** Fired repeatedly as the LLM generates text, providing one word/fragment at a time. Use for real-time subtitles. Each call gives a new fragment, NOT the accumulated text. Requires bEnableAgentPartialResponse to be true. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Streaming text fragments as the LLM generates them (word by word).\nIdeal for real-time subtitles. Enable with bEnableAgentPartialResponse.")) meta = (ToolTip = "Streaming text fragments as the LLM generates them (word by word).\nIdeal for real-time subtitles. Enable with bEnableAgentPartialResponse."))
FOnAgentPartialResponse OnAgentPartialResponse; FOnAgentPartialResponse OnAgentPartialResponse;
/** Fired when the agent begins playing audio (first audio chunk received). Use this to trigger speech animations or UI indicators. */ /** Fired when the agent begins playing audio (first audio chunk received). Use this to trigger speech animations or UI indicators. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the agent starts speaking (first audio chunk). Use for lip-sync or UI feedback.")) meta = (ToolTip = "Fires when the agent starts speaking (first audio chunk). Use for lip-sync or UI feedback."))
FOnAgentStartedSpeaking OnAgentStartedSpeaking; FOnAgentStartedSpeaking OnAgentStartedSpeaking;
/** Fired when audio playback actually starts — AFTER any pre-buffering delay. /** Fired when audio playback actually starts — AFTER any pre-buffering delay.
* Unlike OnAgentStartedSpeaking (first chunk arrival), this fires when audio is audible. * Unlike OnAgentStartedSpeaking (first chunk arrival), this fires when audio is audible.
* Use this for body/gesture animations that should be synced with audible speech. */ * Use this for body/gesture animations that should be synced with audible speech. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when audio playback actually starts (after pre-buffering).\nSynced with audible speech. Use for body animations.")) meta = (ToolTip = "Fires when audio playback actually starts (after pre-buffering).\nSynced with audible speech. Use for body animations."))
FOnAudioPlaybackStarted OnAudioPlaybackStarted; FOnAudioPlaybackStarted OnAudioPlaybackStarted;
/** Fired when the agent finishes playing all audio. Use this to re-open the microphone (in Server VAD mode without interruption) or update UI. */ /** Fired when the agent finishes playing all audio. Use this to re-open the microphone (in Server VAD mode without interruption) or update UI. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the agent finishes speaking. Use to re-open the mic or update UI.")) meta = (ToolTip = "Fires when the agent finishes speaking. Use to re-open the mic or update UI."))
FOnAgentStoppedSpeaking OnAgentStoppedSpeaking; FOnAgentStoppedSpeaking OnAgentStoppedSpeaking;
/** Fired when the agent's speech is interrupted (either by the user speaking over it, or by a manual InterruptAgent call). The audio playback is automatically stopped. */ /** Fired when the agent's speech is interrupted (either by the user speaking over it, or by a manual InterruptAgent call). The audio playback is automatically stopped. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the agent is interrupted mid-speech. Audio is automatically stopped.")) meta = (ToolTip = "Fires when the agent is interrupted mid-speech. Audio is automatically stopped."))
FOnAgentInterrupted OnAgentInterrupted; FOnAgentInterrupted OnAgentInterrupted;
/** Fires on local user voice onset (first voiced mic frame via FeedExternalAudio),
* without waiting for the ElevenLabs STT round-trip. Used to interrupt an
* agent-to-agent dialogue the instant the player speaks. */
UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events")
FOnUserVoiceDetected OnUserVoiceDetected;
/** Fired when the server starts generating a response (before any audio arrives). Use this for "thinking..." UI feedback. In push-to-talk mode, the microphone is automatically closed when this fires. */ /** Fired when the server starts generating a response (before any audio arrives). Use this for "thinking..." UI feedback. In push-to-talk mode, the microphone is automatically closed when this fires. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the server starts generating (before audio arrives).\nUse for 'thinking...' UI. Mic is auto-closed in push-to-talk mode.")) meta = (ToolTip = "Fires when the server starts generating (before audio arrives).\nUse for 'thinking...' UI. Mic is auto-closed in push-to-talk mode."))
FOnAgentStartedGenerating OnAgentStartedGenerating; FOnAgentStartedGenerating OnAgentStartedGenerating;
/** Fired if the server does not start generating a response within ResponseTimeoutSeconds after the user stops speaking. Use this to show a "try again" message or automatically re-open the microphone. */ /** Fired if the server does not start generating a response within ResponseTimeoutSeconds after the user stops speaking. Use this to show a "try again" message or automatically re-open the microphone. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires if the server doesn't respond within ResponseTimeoutSeconds.\nUse to show 'try again' or re-open the mic automatically.")) meta = (ToolTip = "Fires if the server doesn't respond within ResponseTimeoutSeconds.\nUse to show 'try again' or re-open the mic automatically."))
FOnAgentResponseTimeout OnAgentResponseTimeout; FOnAgentResponseTimeout OnAgentResponseTimeout;
/** Fired when the agent changes emotion via the "set_emotion" client tool. The emotion is set BEFORE the corresponding audio arrives, giving you time to smoothly blend facial expressions. */ /** Fired when the agent changes emotion via the "set_emotion" client tool. The emotion is set BEFORE the corresponding audio arrives, giving you time to smoothly blend facial expressions. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the agent sets an emotion (joy, sadness, surprise, fear, anger, disgust).\nDriven by the 'set_emotion' client tool. Arrives before the audio.")) meta = (ToolTip = "Fires when the agent sets an emotion (joy, sadness, surprise, fear, anger, disgust).\nDriven by the 'set_emotion' client tool. Arrives before the audio."))
FOnAgentEmotionChanged OnAgentEmotionChanged; FOnAgentEmotionChanged OnAgentEmotionChanged;
/** Fired when the LLM triggers a perform_action tool call. /** Fired when the LLM triggers a perform_action tool call.
* ActionName is the identifier from the ActionSet (e.g. "flee", "draw_weapon"). * ActionName is the identifier from the ActionSet (e.g. "flee", "draw_weapon").
* Handle this in Blueprint to play animations, trigger AI behavior, etc. */ * Handle this in Blueprint to play animations, trigger AI behavior, etc. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the agent triggers a physical action.\nActionName matches the Name field from the ActionSet Data Asset.")) meta = (ToolTip = "Fires when the agent triggers a physical action.\nActionName matches the Name field from the ActionSet Data Asset."))
FOnAgentActionRequested OnAgentActionRequested; FOnAgentActionRequested OnAgentActionRequested;
/** Fired for client tool calls that are NOT automatically handled (i.e. not "set_emotion", not "perform_action"). You must call GetWebSocketProxy()->SendClientToolResult() to respond. */ /** Fired for client tool calls that are NOT automatically handled (i.e. not "set_emotion", not "perform_action"). You must call GetWebSocketProxy()->SendClientToolResult() to respond. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires for custom client tool calls (not set_emotion, not perform_action).\nYou must respond via GetWebSocketProxy()->SendClientToolResult().")) meta = (ToolTip = "Fires for custom client tool calls (not set_emotion, not perform_action).\nYou must respond via GetWebSocketProxy()->SendClientToolResult()."))
FOnAgentClientToolCall OnAgentClientToolCall; FOnAgentClientToolCall OnAgentClientToolCall;
/** Fired when the active speaker changes in a multi-player shared conversation. /** Fired when the active speaker changes in a multi-player shared conversation.
* NewSpeaker is null when no one is speaking (idle timeout). */ * NewSpeaker is null when no one is speaking (idle timeout). */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when the speaking player changes.\nNewSpeaker is null when no one is speaking.")) meta = (ToolTip = "Fires when the speaking player changes.\nNewSpeaker is null when no one is speaking."))
FOnActiveSpeakerChanged OnActiveSpeakerChanged; FOnActiveSpeakerChanged OnActiveSpeakerChanged;
@@ -361,26 +378,26 @@ public:
* components back to neutral. The NPC is fully idle and the game can start * components back to neutral. The NPC is fully idle and the game can start
* the physical action (flee, draw weapon, etc.). * the physical action (flee, draw weapon, etc.).
* Call ForceEnableConversation() later to allow conversation again. */ * Call ForceEnableConversation() later to allow conversation again. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when all animation components have returned to neutral after ForceDisableConversation().\nThe agent is ready for a game action.")) meta = (ToolTip = "Fires when all animation components have returned to neutral after ForceDisableConversation().\nThe agent is ready for a game action."))
FOnReadyForAction OnReadyForAction; FOnReadyForAction OnReadyForAction;
/** Fired when this agent starts passively tracking a nearby player (head/eyes gaze). */ /** Fired when this agent starts passively tracking a nearby player (head/eyes gaze). */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when this agent starts passively tracking a nearby player.\nTriggered by proximity awareness — the agent is not selected for conversation.")) meta = (ToolTip = "Fires when this agent starts passively tracking a nearby player.\nTriggered by proximity awareness — the agent is not selected for conversation."))
FOnAgentPassiveGazeStarted OnPassiveGazeStarted; FOnAgentPassiveGazeStarted OnPassiveGazeStarted;
/** Fired when this agent stops passively tracking a player (left range or selected). */ /** Fired when this agent stops passively tracking a player (left range or selected). */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|ElevenLabs|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events",
meta = (ToolTip = "Fires when this agent stops passively tracking a player.\nThe player left range or selected this agent for active conversation.")) meta = (ToolTip = "Fires when this agent stops passively tracking a player.\nThe player left range or selected this agent for active conversation."))
FOnAgentPassiveGazeStopped OnPassiveGazeStopped; FOnAgentPassiveGazeStopped OnPassiveGazeStopped;
/** The current emotion of the agent, as set by the "set_emotion" client tool. Defaults to Neutral. */ /** The current emotion of the agent, as set by the "set_emotion" client tool. Defaults to Neutral. */
UPROPERTY(ReplicatedUsing = OnRep_Emotion, BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(ReplicatedUsing = OnRep_Emotion, BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
EPS_AI_ConvAgent_Emotion CurrentEmotion = EPS_AI_ConvAgent_Emotion::Neutral; EPS_AI_ConvAgent_Emotion CurrentEmotion = EPS_AI_ConvAgent_Emotion::Neutral;
/** The current emotion intensity. Defaults to Medium. */ /** The current emotion intensity. Defaults to Medium. */
UPROPERTY(ReplicatedUsing = OnRep_Emotion, BlueprintReadOnly, Category = "PS AI ConvAgent|ElevenLabs") UPROPERTY(ReplicatedUsing = OnRep_Emotion, BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
EPS_AI_ConvAgent_EmotionIntensity CurrentEmotionIntensity = EPS_AI_ConvAgent_EmotionIntensity::Medium; EPS_AI_ConvAgent_EmotionIntensity CurrentEmotionIntensity = EPS_AI_ConvAgent_EmotionIntensity::Medium;
// ── Raw audio data (C++ only, used by LipSync component) ──────────────── // ── Raw audio data (C++ only, used by LipSync component) ────────────────
@@ -388,30 +405,50 @@ public:
* Used internally by UPS_AI_ConvAgent_LipSyncComponent for spectral analysis. */ * Used internally by UPS_AI_ConvAgent_LipSyncComponent for spectral analysis. */
FOnAgentAudioData OnAgentAudioData; FOnAgentAudioData OnAgentAudioData;
/** Fired on the SERVER for each player mic chunk relayed via ServerSendMicAudioFromPlayer
* (int16, 16kHz mono). Lets an external recorder capture player speech for replay.
* Not fired on clients — only the server holds the relayed mic stream. */
UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs|Events")
FOnPlayerAudioCaptured OnPlayerAudioCaptured;
// ── Network state (replicated) ─────────────────────────────────────────── // ── Network state (replicated) ───────────────────────────────────────────
/** True when one or more players are in conversation with this NPC. /** True when one or more players are in conversation with this NPC.
* Replicated to all clients for UI feedback, gaze, LOD, etc. */ * Replicated to all clients for UI feedback, gaze, LOD, etc. */
UPROPERTY(ReplicatedUsing = OnRep_ConversationState, BlueprintReadOnly, Category = "PS AI ConvAgent|Network") UPROPERTY(ReplicatedUsing = OnRep_ConversationState, BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|Network")
bool bNetIsConversing = false; bool bNetIsConversing = false;
/** All player pawns currently in conversation with this NPC. /** All player pawns currently in conversation with this NPC.
* Multiple players can share the same agent — audio is routed via speaker arbitration. * Multiple players can share the same agent — audio is routed via speaker arbitration.
* Replicated to ALL clients for gaze target and LOD distance checks. */ * Replicated to ALL clients for gaze target and LOD distance checks. */
UPROPERTY(ReplicatedUsing = OnRep_ConversationState, BlueprintReadOnly, Category = "PS AI ConvAgent|Network") UPROPERTY(ReplicatedUsing = OnRep_ConversationState, BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|Network")
TArray<TObjectPtr<APawn>> NetConnectedPawns; TArray<TObjectPtr<APawn>> NetConnectedPawns;
/** The player currently speaking (active audio sender). Null if no one is speaking. /** The player currently speaking (active audio sender). Null if no one is speaking.
* Used by GazeComponent for target switching — the NPC looks at whoever is talking. * Used by GazeComponent for target switching — the NPC looks at whoever is talking.
* Replicated to ALL clients so gaze updates everywhere. */ * Replicated to ALL clients so gaze updates everywhere. */
UPROPERTY(ReplicatedUsing = OnRep_ActiveSpeaker, BlueprintReadOnly, Category = "PS AI ConvAgent|Network") UPROPERTY(ReplicatedUsing = OnRep_ActiveSpeaker, BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|Network")
TObjectPtr<APawn> NetActiveSpeakerPawn = nullptr; TObjectPtr<APawn> NetActiveSpeakerPawn = nullptr;
/** When set, the conversation gaze looks at THIS actor instead of any connected
* player pawn — used by the agent-to-agent dialogue so the two agents look at
* EACH OTHER rather than at a bystander. Replicated so clients honor it too. */
UPROPERTY(ReplicatedUsing = OnRep_GazeOverrideTarget, BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|Gaze")
TObjectPtr<AActor> GazeOverrideTarget = nullptr;
UFUNCTION()
void OnRep_GazeOverrideTarget();
/** Force the conversation gaze to look at Target (pass null to clear and restore
* the normal player-tracking behaviour). Call on the server; replicates to clients. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Gaze")
void SetGazeOverrideTarget(AActor* Target);
// ── Multi-player speaker arbitration (server only) ────────────────────── // ── Multi-player speaker arbitration (server only) ──────────────────────
/** Minimum seconds of silence from the current speaker before allowing a speaker switch. /** Minimum seconds of silence from the current speaker before allowing a speaker switch.
* Prevents rapid ping-pong switching when both players make brief sounds. */ * Prevents rapid ping-pong switching when both players make brief sounds. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Network|MultiPlayer", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Network|MultiPlayer",
meta = (ClampMin = "0.0", ClampMax = "5.0", meta = (ClampMin = "0.0", ClampMax = "5.0",
ToolTip = "Minimum silence from current speaker before switching to another.\nPrevents rapid gaze flip-flop.")) ToolTip = "Minimum silence from current speaker before switching to another.\nPrevents rapid gaze flip-flop."))
float SpeakerSwitchHysteresis = 0.3f; float SpeakerSwitchHysteresis = 0.3f;
@@ -419,7 +456,7 @@ public:
/** Seconds after last speech before the active speaker is cleared. /** Seconds after last speech before the active speaker is cleared.
* When cleared, gaze returns to the last speaker position (or closest connected player). * When cleared, gaze returns to the last speaker position (or closest connected player).
* Set to 0 to never clear (last speaker stays active indefinitely). */ * Set to 0 to never clear (last speaker stays active indefinitely). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Network|MultiPlayer", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Network|MultiPlayer",
meta = (ClampMin = "0.0", meta = (ClampMin = "0.0",
ToolTip = "Seconds of silence before clearing the active speaker.\n0 = last speaker stays active indefinitely.")) ToolTip = "Seconds of silence before clearing the active speaker.\n0 = last speaker stays active indefinitely."))
float SpeakerIdleTimeout = 3.0f; float SpeakerIdleTimeout = 3.0f;
@@ -428,17 +465,29 @@ public:
/** Distance (cm) beyond which remote clients stop receiving agent audio entirely. /** Distance (cm) beyond which remote clients stop receiving agent audio entirely.
* The speaking player always receives full audio regardless of distance. */ * The speaking player always receives full audio regardless of distance. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Network|LOD", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Network|LOD",
meta = (ClampMin = "0", ToolTip = "Distance beyond which audio is culled for non-speaking players. 0 = no cull.")) meta = (ClampMin = "0", ToolTip = "Distance beyond which audio is culled for non-speaking players. 0 = no cull."))
float AudioLODCullDistance = 3000.f; float AudioLODCullDistance = 3000.f;
/** Distance (cm) beyond which remote clients skip lip-sync / emotion processing. /** Distance (cm) beyond which remote clients skip lip-sync / emotion processing.
* Audio still plays (if within AudioLODCullDistance) but without facial animation. * Audio still plays (if within AudioLODCullDistance) but without facial animation.
* The speaking player always gets full lip-sync regardless of distance. */ * The speaking player always gets full lip-sync regardless of distance. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Network|LOD", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Network|LOD",
meta = (ClampMin = "0", ToolTip = "Distance beyond which lip-sync is skipped for non-speaking players. 0 = no LOD.")) meta = (ClampMin = "0", ToolTip = "Distance beyond which lip-sync is skipped for non-speaking players. 0 = no LOD."))
float LipSyncLODDistance = 1500.f; float LipSyncLODDistance = 1500.f;
// ── Network mic uplink (clients) ─────────────────────────────────────────
/** Seconds of silence to keep streaming to the server AFTER the player's last
* voiced chunk, so ElevenLabs Server VAD can detect end-of-turn. Beyond this
* tail, silent chunks are dropped to save bandwidth (clients only — the
* authority path always streams the full mic). Set >= the agent's end-of-turn
* silence threshold (+ margin). 0 = drop all silence (breaks Server VAD). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Network|MultiPlayer",
meta = (ClampMin = "0.0", ClampMax = "5.0",
ToolTip = "Silence streamed after last voice so ElevenLabs detects end-of-turn (Server VAD).\nMust be >= the agent's end-of-turn silence. 0 = drop all silence (breaks Server VAD on clients)."))
float MicSilenceTailSeconds = 2.0f;
// ── Network RPCs ───────────────────────────────────────────────────────── // ── Network RPCs ─────────────────────────────────────────────────────────
/** Join a shared conversation with this NPC. Multiple players can join. /** Join a shared conversation with this NPC. Multiple players can join.
@@ -472,9 +521,22 @@ public:
UFUNCTION(Server, Reliable) UFUNCTION(Server, Reliable)
void ServerRequestInterrupt(); void ServerRequestInterrupt();
/** Broadcast agent audio to all clients (Opus-compressed or raw PCM fallback). */ /** Broadcast agent audio to all clients (Opus-compressed or raw PCM fallback).
* Reliable: agent TTS must arrive complete and IN ORDER. Opus is a stateful
* (inter-frame-predictive) codec, so a dropped or reordered packet on an
* Unreliable channel desyncs the decoder → missing words + crackle. Reliable
* guarantees ordered, lossless delivery, keeping the decoder in sync.
* This does NOT reintroduce the old reliable-buffer overflow (which dropped
* VR connections): that came from 32-152 KB raw-PCM RPCs. With Opus each
* sub-chunk is ~2 KB (2-3 partial bunches, below the 8-bunch reliable-
* promotion threshold) at ~1.25 RPC/s, so in-flight bunches stay in single
* digits vs the 512-bunch buffer. Sub-chunking in HandleAudioReceived is what
* keeps packets this small — it must stay.
* bLastSubChunk marks the final sub-chunk of one ElevenLabs message so the
* client can reassemble all sub-chunks and enqueue ONCE per message (enqueuing
* each sub-chunk separately re-ran the playback state machine → pops). */
UFUNCTION(NetMulticast, Reliable) UFUNCTION(NetMulticast, Reliable)
void MulticastReceiveAgentAudio(const TArray<uint8>& AudioData); void MulticastReceiveAgentAudio(const TArray<uint8>& AudioData, bool bLastSubChunk);
/** Notify all clients that the agent started speaking (first audio chunk). */ /** Notify all clients that the agent started speaking (first audio chunk). */
UFUNCTION(NetMulticast, Reliable) UFUNCTION(NetMulticast, Reliable)
@@ -514,25 +576,25 @@ public:
* Open the WebSocket connection and start the conversation. * Open the WebSocket connection and start the conversation.
* If bAutoStartListening is true, microphone capture also starts once connected. * If bAutoStartListening is true, microphone capture also starts once connected.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void StartConversation(); void StartConversation();
/** Close the WebSocket and stop all audio. */ /** Close the WebSocket and stop all audio. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void EndConversation(); void EndConversation();
/** /**
* Start capturing microphone audio and streaming it to ElevenLabs. * Start capturing microphone audio and streaming it to ElevenLabs.
* In Client turn mode, also sends a UserTurnStart signal. * In Client turn mode, also sends a UserTurnStart signal.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void StartListening(); void StartListening();
/** /**
* Stop capturing microphone audio. * Stop capturing microphone audio.
* In Client turn mode, also sends a UserTurnEnd signal. * In Client turn mode, also sends a UserTurnEnd signal.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void StopListening(); void StopListening();
/** /**
@@ -540,40 +602,66 @@ public:
* The agent will respond with audio and text just as if it heard you speak. * The agent will respond with audio and text just as if it heard you speak.
* Useful for testing in the Editor or for text-based interaction. * Useful for testing in the Editor or for text-based interaction.
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void SendTextMessage(const FString& Text); void SendTextMessage(const FString& Text);
/**
* Inject a "silent" context update: the agent incorporates it as context but does
* NOT reply to it (unlike SendTextMessage). Used to tell an agent that a new/real
* interlocutor is now speaking while keeping the prior conversation. Authority only.
*/
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void SendContextualUpdate(const FString& Text);
/** Interrupt the agent's current utterance. */ /** Interrupt the agent's current utterance. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void InterruptAgent(); void InterruptAgent();
// ── Replay support (PS_AI_ConvReplay bridge) ─────────────────────────────
/** /**
* Cleanly disable conversation on this agent so the NPC can switch to game AI. * Play an external PCM buffer through this agent's voice, exactly as if it were
* live TTS: drives audio playback, lip-sync, body expression and start/stop events.
* Role-agnostic (works on a replay/simulated proxy); no LOD culling.
* @param PCM16 int16 samples, 16000 Hz mono, little-endian.
*/
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Replay")
void PlayExternalAudioPCM(const TArray<uint8>& PCM16);
/** Stop/flush any audio currently playing (used when scrubbing a replay). */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Replay")
void StopExternalAudio();
/** Re-broadcast a recorded perform_action reaction during replay (fires OnAgentActionRequested). */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Replay")
void TriggerReplayedAction(const FString& ActionName);
/** Re-broadcast a recorded custom client tool call during replay (fires OnAgentClientToolCall). */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Replay")
void TriggerReplayedClientToolCall(const FPS_AI_ConvAgent_ClientToolCall_ElevenLabs& ToolCall);
/**
* Cleanly disable conversation on this agent.
* Ends the WebSocket connection, stops all audio, and begins blending all * Ends the WebSocket connection, stops all audio, and begins blending all
* visual components (gaze, lip sync, facial expression, body expression) * visual components (gaze, lip sync, facial expression, body expression)
* back to their neutral state. While disabled, the InteractionComponent * back to their neutral state. While disabled, the InteractionComponent
* cannot auto-restart a conversation with this agent. * cannot auto-restart a conversation with this agent.
* *
* When all components have blended to neutral, fires OnReadyForAction. * This function ONLY handles the conversation master-switch. It has no
* knowledge of action/reaction flow — if an action was pending via
* OnAgentActionRequested, OnReadyForAction will fire automatically once
* blend-out completes. If no action is pending, nothing else happens.
*
* Call ForceEnableConversation() later to allow conversation again. * Call ForceEnableConversation() later to allow conversation again.
*
* Typical usage from Blueprint (OnAgentActionRequested):
* 1. ForceDisableConversation("flee")
* 2. Wait for OnReadyForAction — receives the same ActionName
* 3. Play action montage based on ActionName
* 4. When done, call ForceEnableConversation()
*
* @param ActionName The action to perform once neutral (passed through to OnReadyForAction).
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void ForceDisableConversation(const FString& ActionName); void ForceDisableConversation();
/** /**
* Re-enable conversation after a ForceDisableConversation() call. * Re-enable conversation after a ForceDisableConversation() call.
* Clears the disabled flag so the InteractionComponent can auto-start * Clears the disabled flag so the InteractionComponent can auto-start
* a new conversation on the next tick (if the player is still looking). * a new conversation on the next tick (if the player is still looking).
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void ForceEnableConversation(); void ForceEnableConversation();
/** /**
@@ -583,7 +671,7 @@ public:
* The component must be connected and listening (StartListening called) for audio to be sent. * The component must be connected and listening (StartListening called) for audio to be sent.
* @param FloatPCM Float32 samples, 16000 Hz mono (same format as MicrophoneCaptureComponent output). * @param FloatPCM Float32 samples, 16000 Hz mono (same format as MicrophoneCaptureComponent output).
*/ */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void FeedExternalAudio(const TArray<float>& FloatPCM); void FeedExternalAudio(const TArray<float>& FloatPCM);
/** Receive mic audio from a specific player with speaker arbitration. /** Receive mic audio from a specific player with speaker arbitration.
@@ -592,13 +680,13 @@ public:
// ── State queries ───────────────────────────────────────────────────────── // ── State queries ─────────────────────────────────────────────────────────
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
bool IsConnected() const; bool IsConnected() const;
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
bool IsListening() const { return bIsListening; } bool IsListening() const { return bIsListening; }
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
bool IsAgentSpeaking() const { return bAgentSpeaking; } bool IsAgentSpeaking() const { return bAgentSpeaking; }
/** True when the agent is speaking OR the server hasn't confirmed the full response yet. /** True when the agent is speaking OR the server hasn't confirmed the full response yet.
@@ -606,21 +694,21 @@ public:
* produce more audio (e.g. during TTS inter-batch gaps). */ * produce more audio (e.g. during TTS inter-batch gaps). */
bool IsAgentSpeakingOrPending() const { return bAgentSpeaking || (bAgentGenerating && !bAgentResponseReceived); } bool IsAgentSpeakingOrPending() const { return bAgentSpeaking || (bAgentGenerating && !bAgentResponseReceived); }
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& GetConversationInfo() const; const FPS_AI_ConvAgent_ConversationInfo_ElevenLabs& GetConversationInfo() const;
/** True while conversation is force-disabled (ForceDisableConversation was called). /** True while conversation is force-disabled (ForceDisableConversation was called).
* While disabled, StartConversation and ServerJoinConversation are blocked. */ * While disabled, StartConversation and ServerJoinConversation are blocked. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
bool IsConversationDisabled() const { return bConversationDisabledByAction; } bool IsConversationDisabled() const { return bConversationForceDisabled; }
/** True while audio is being pre-buffered (playback hasn't started yet). /** True while audio is being pre-buffered (playback hasn't started yet).
* Used by the LipSync component to pause viseme queue consumption. */ * Used by the LipSync component to pause viseme queue consumption. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
bool IsPreBuffering() const { return bPreBuffering; } bool IsPreBuffering() const { return bPreBuffering; }
/** Access the underlying WebSocket proxy (advanced use). */ /** Access the underlying WebSocket proxy (advanced use). */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy* GetWebSocketProxy() const { return WebSocketProxy; } UPS_AI_ConvAgent_WebSocket_ElevenLabsProxy* GetWebSocketProxy() const { return WebSocketProxy; }
// ───────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────
@@ -643,6 +731,9 @@ public:
int32 GetMicChunkMinBytesPublic() const; int32 GetMicChunkMinBytesPublic() const;
private: private:
// Debounce timestamp (seconds) for the OnUserVoiceDetected voice-onset broadcast.
double LastUserVoiceOnset = 0.0;
// ── Network OnRep handlers ─────────────────────────────────────────────── // ── Network OnRep handlers ───────────────────────────────────────────────
UFUNCTION() UFUNCTION()
void OnRep_ConversationState(); void OnRep_ConversationState();
@@ -706,24 +797,33 @@ private:
UPROPERTY() UPROPERTY()
USoundWaveProcedural* ProceduralSoundWave = nullptr; USoundWaveProcedural* ProceduralSoundWave = nullptr;
// ── ForceDisableConversation state ─────────────────────────────────────── // ── Conversation disable state ───────────────────────────────────────────
// Set by ForceDisableConversation(), cleared by ForceEnableConversation(). // Set by ForceDisableConversation(), cleared by ForceEnableConversation().
// While true, StartConversation() and ServerJoinConversation() are blocked. // While true, StartConversation() and ServerJoinConversation() are blocked.
bool bConversationDisabledByAction = false; bool bConversationForceDisabled = false;
// True while waiting for all visual components to blend back to neutral. // True while waiting for all visual components to blend back to neutral
// Monitored in TickComponent — fires OnReadyForAction when complete. // after ForceDisableConversation(). Monitored in TickComponent.
bool bWaitingForBlendOut = false; bool bWaitingForBlendOut = false;
// The action name passed to ForceDisableConversation(), forwarded to OnReadyForAction. // Accumulated time waiting for blend-out. Safety timeout to unblock
FString PendingActionName;
// Accumulated time waiting for blend-out. Safety timeout fires OnReadyForAction
// if components haven't reached neutral within this limit. // if components haven't reached neutral within this limit.
float BlendOutElapsedTime = 0.0f; float BlendOutElapsedTime = 0.0f;
static constexpr float BlendOutTimeoutSeconds = 5.0f; static constexpr float BlendOutTimeoutSeconds = 5.0f;
// ── Action flow state (independent from disable state) ───────────────────
// Set when OnAgentActionRequested is broadcasted (LLM tool call).
// Cleared when OnReadyForAction is broadcasted.
// - If conversation is being disabled, OnReadyForAction waits for blend-out.
// - If not, OnReadyForAction fires on the next tick (quasi-immediate).
FString PendingActionName;
// ── State ───────────────────────────────────────────────────────────────── // ── State ─────────────────────────────────────────────────────────────────
// Atomic: read from WASAPI background thread (OnMicrophoneDataCaptured), written from game thread. // Atomic: read from WASAPI background thread (OnMicrophoneDataCaptured), written from game thread.
std::atomic<bool> bIsListening{false}; std::atomic<bool> bIsListening{false};
std::atomic<bool> bAgentSpeaking{false}; std::atomic<bool> bAgentSpeaking{false};
// Set when StartListening() is called before the WebSocket is connected.
// Consumed in HandleConnected() to open the microphone automatically once ready.
// Cleared by StopListening() or EndConversation() to cancel the pending request.
bool bPendingStartListening = false;
// True from the first agent_chat_response_part until the first audio chunk arrives. // True from the first agent_chat_response_part until the first audio chunk arrives.
// Used to block StartListening() while the server is processing the previous turn. // Used to block StartListening() while the server is processing the previous turn.
// Atomic: defensive — currently game-thread only, but documents thread-safety contract. // Atomic: defensive — currently game-thread only, but documents thread-safety contract.
@@ -777,6 +877,26 @@ private:
int32 AudioQueueReadOffset = 0; int32 AudioQueueReadOffset = 0;
FCriticalSection AudioQueueLock; FCriticalSection AudioQueueLock;
// Client-only: accumulates decoded PCM across the N sub-chunks of one
// ElevenLabs message; flushed to EnqueueAgentAudio() once, on the last
// sub-chunk (bLastSubChunk). Game-thread only. Reset in StopAgentAudio().
TArray<uint8> ClientReassemblyPCM;
// Server-only: sub-frame (<640 byte) tail carried from the previous agent
// audio message. Opus encodes whole frames only; without carry-over each
// message dropped its <1-frame tail, leaving a ~10-16ms hole + waveform
// discontinuity at every message splice → audible pops on clients. Reset
// per turn (new-turn guard in HandleAudioReceived + StopAgentAudio()).
TArray<uint8> AgentAudioEncodeLeftover;
// Debug: detects waveform discontinuities at enqueue/message splices (the
// signature of the audio "pops"). Tracks the last enqueued sample; a large
// jump to the next block's first sample is flagged and counted. Toggle with
// ps.ai.ConvAgent.Debug.ElevenLabs. Reset in StopAgentAudio().
int16 LastEnqueuedSample = 0;
bool bHasLastEnqueuedSample = false;
int32 AudioSpliceDiscontinuityCount = 0;
// Pre-buffer state: delay playback start to absorb TTS inter-chunk gaps. // Pre-buffer state: delay playback start to absorb TTS inter-chunk gaps.
bool bPreBuffering = false; bool bPreBuffering = false;
double PreBufferStartTime = 0.0; double PreBufferStartTime = 0.0;
@@ -838,6 +958,11 @@ private:
TArray<uint8> MicAccumulationBuffer; TArray<uint8> MicAccumulationBuffer;
FCriticalSection MicSendLock; FCriticalSection MicSendLock;
// Client mic uplink: timestamp (FPlatformTime::Seconds) of the last voiced chunk.
// Used to keep streaming a short silence tail after speech so ElevenLabs Server
// VAD can detect end-of-turn (see MicSilenceTailSeconds). 0 = never voiced yet.
double LastMicVoicedTime = 0.0;
/** Compute the minimum bytes from the user-facing MicChunkDurationMs. /** Compute the minimum bytes from the user-facing MicChunkDurationMs.
* Formula: bytes = SampleRate * (ms / 1000) * BytesPerSample = 16000 * ms / 1000 * 2 = 32 * ms */ * Formula: bytes = SampleRate * (ms / 1000) * BytesPerSample = 16000 * ms / 1000 * 2 = 32 * ms */
int32 GetMicChunkMinBytes() const { return MicChunkDurationMs * 32; } int32 GetMicChunkMinBytes() const { return MicChunkDurationMs * 32; }

View File

@@ -35,7 +35,7 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE_TwoParams(FOnExpressionChanged,
// 4. Emotion animations crossfade smoothly (configurable duration). // 4. Emotion animations crossfade smoothly (configurable duration).
// 5. The AnimNode reads GetCurrentEmotionCurves() and injects them into the pose. // 5. The AnimNode reads GetCurrentEmotionCurves() and injects them into the pose.
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
UCLASS(ClassGroup = "PS AI ConvAgent", meta = (BlueprintSpawnableComponent), UCLASS(ClassGroup = "ASTERION|PS_AI_ConvAgent", meta = (BlueprintSpawnableComponent),
DisplayName = "PS AI ConvAgent Facial Expression") DisplayName = "PS AI ConvAgent Facial Expression")
class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_FacialExpressionComponent : public UActorComponent class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_FacialExpressionComponent : public UActorComponent
{ {
@@ -48,23 +48,23 @@ public:
/** Emotion pose map asset containing emotion AnimSequences (Normal / Medium / Extreme per emotion). /** Emotion pose map asset containing emotion AnimSequences (Normal / Medium / Extreme per emotion).
* Create a dedicated PS_AI_ConvAgent_EmotionPoseMap asset in the Content Browser. */ * Create a dedicated PS_AI_ConvAgent_EmotionPoseMap asset in the Content Browser. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|FacialExpression", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression",
meta = (ToolTip = "Dedicated Emotion Pose Map asset.\nRight-click Content Browser → Miscellaneous → PS AI ConvAgent Emotion Pose Map.")) meta = (ToolTip = "Dedicated Emotion Pose Map asset.\nRight-click Content Browser → Miscellaneous → PS AI ConvAgent Emotion Pose Map."))
TObjectPtr<UPS_AI_ConvAgent_EmotionPoseMap> EmotionPoseMap; TObjectPtr<UPS_AI_ConvAgent_EmotionPoseMap> EmotionPoseMap;
/** Whether facial expressions are currently active (read-only). /** Whether facial expressions are currently active (read-only).
* Managed automatically: activates on conversation connect, deactivates on disconnect. * Managed automatically: activates on conversation connect, deactivates on disconnect.
* The underlying emotion playback keeps running so reactivation is seamless. */ * The underlying emotion playback keeps running so reactivation is seamless. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|FacialExpression") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression")
bool bActive = true; bool bActive = true;
/** How long (seconds) to blend in/out when bActive changes. */ /** How long (seconds) to blend in/out when bActive changes. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|FacialExpression", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression",
meta = (ClampMin = "0.05", ClampMax = "3.0")) meta = (ClampMin = "0.05", ClampMax = "3.0"))
float ActivationBlendDuration = 0.5f; float ActivationBlendDuration = 0.5f;
/** Emotion crossfade duration in seconds. */ /** Emotion crossfade duration in seconds. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|FacialExpression", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression",
meta = (ClampMin = "0.1", ClampMax = "3.0", meta = (ClampMin = "0.1", ClampMax = "3.0",
ToolTip = "How long (seconds) to crossfade between emotions.\n0.5 = snappy, 1.5 = smooth.")) ToolTip = "How long (seconds) to crossfade between emotions.\n0.5 = snappy, 1.5 = smooth."))
float EmotionBlendDuration = 0.5f; float EmotionBlendDuration = 0.5f;
@@ -73,12 +73,12 @@ public:
/** Enable debug logging for this component. /** Enable debug logging for this component.
* When false, only key events (init, errors, state changes) are logged. */ * When false, only key events (init, errors, state changes) are logged. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug")
bool bDebug = false; bool bDebug = false;
/** Verbosity level when bDebug is true. /** Verbosity level when bDebug is true.
* 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */ * 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug",
meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug")) meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug"))
int32 DebugVerbosity = 1; int32 DebugVerbosity = 1;
@@ -86,7 +86,7 @@ public:
/** Get the current emotion curves evaluated from the playing AnimSequence. /** Get the current emotion curves evaluated from the playing AnimSequence.
* Returns a COPY — safe to call from any thread (AnimNode worker thread). */ * Returns a COPY — safe to call from any thread (AnimNode worker thread). */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|FacialExpression") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression")
TMap<FName, float> GetCurrentEmotionCurves() const TMap<FName, float> GetCurrentEmotionCurves() const
{ {
FScopeLock Lock(&EmotionCurveLock); FScopeLock Lock(&EmotionCurveLock);
@@ -94,15 +94,15 @@ public:
} }
/** Get the active emotion. */ /** Get the active emotion. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|FacialExpression") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression")
EPS_AI_ConvAgent_Emotion GetActiveEmotion() const { return ActiveEmotion; } EPS_AI_ConvAgent_Emotion GetActiveEmotion() const { return ActiveEmotion; }
/** Get the active emotion intensity. */ /** Get the active emotion intensity. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|FacialExpression") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression")
EPS_AI_ConvAgent_EmotionIntensity GetActiveIntensity() const { return ActiveEmotionIntensity; } EPS_AI_ConvAgent_EmotionIntensity GetActiveIntensity() const { return ActiveEmotionIntensity; }
/** Check if a curve name belongs to the mouth area (overridden by lip sync). */ /** Check if a curve name belongs to the mouth area (overridden by lip sync). */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|FacialExpression") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression")
static bool IsMouthCurve(const FName& CurveName); static bool IsMouthCurve(const FName& CurveName);
// ── Events ─────────────────────────────────────────────────────────────── // ── Events ───────────────────────────────────────────────────────────────
@@ -111,7 +111,7 @@ public:
* Subscribe in Blueprints to react to expression changes (trigger animations, * Subscribe in Blueprints to react to expression changes (trigger animations,
* particles, sounds, etc.). Only fires when the emotion actually differs * particles, sounds, etc.). Only fires when the emotion actually differs
* from the previous one. */ * from the previous one. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|FacialExpression|Events") UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|FacialExpression|Events")
FOnExpressionChanged OnExpressionChanged; FOnExpressionChanged OnExpressionChanged;
/** Current activation blend alpha (0 = fully inactive, 1 = fully active). /** Current activation blend alpha (0 = fully inactive, 1 = fully active).

View File

@@ -58,7 +58,7 @@ struct PS_AI_CONVAGENT_API FPS_AI_ConvAgent_NeckBoneEntry
// 5. Set TargetOffset for actors without a skeleton (e.g. (0,0,160) for // 5. Set TargetOffset for actors without a skeleton (e.g. (0,0,160) for
// eye-level on a simple actor). // eye-level on a simple actor).
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
UCLASS(ClassGroup = "PS AI ConvAgent", meta = (BlueprintSpawnableComponent), UCLASS(ClassGroup = "ASTERION|PS_AI_ConvAgent", meta = (BlueprintSpawnableComponent),
DisplayName = "PS AI ConvAgent Gaze") DisplayName = "PS AI ConvAgent Gaze")
class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_GazeComponent : public UActorComponent class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_GazeComponent : public UActorComponent
{ {
@@ -71,24 +71,24 @@ public:
/** The actor to look at. Can be any actor (player, prop, etc.). /** The actor to look at. Can be any actor (player, prop, etc.).
* Set to null to smoothly return to neutral. */ * Set to null to smoothly return to neutral. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ToolTip = "Target actor to look at.\nSet to null to return to neutral.")) meta = (ToolTip = "Target actor to look at.\nSet to null to return to neutral."))
TObjectPtr<AActor> TargetActor; TObjectPtr<AActor> TargetActor;
/** When false, all gaze outputs smoothly blend to neutral (passthrough). /** When false, all gaze outputs smoothly blend to neutral (passthrough).
* The underlying tracking keeps running so reactivation is seamless. * The underlying tracking keeps running so reactivation is seamless.
* Managed automatically: activates on conversation connect, deactivates on disconnect. */ * Managed automatically: activates on conversation connect, deactivates on disconnect. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|Gaze") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|Gaze")
bool bActive = true; bool bActive = true;
/** How long (seconds) to blend in/out when bActive changes. */ /** How long (seconds) to blend in/out when bActive changes. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0.05", ClampMax = "3.0")) meta = (ClampMin = "0.05", ClampMax = "3.0"))
float ActivationBlendDuration = 0.5f; float ActivationBlendDuration = 0.5f;
/** When false, body rotation is frozen — only head and eyes track the target. /** When false, body rotation is frozen — only head and eyes track the target.
* Useful to have the agent notice the player (eyes+head) before fully engaging (body). */ * Useful to have the agent notice the player (eyes+head) before fully engaging (body). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ToolTip = "Enable body rotation toward the target.\nWhen false, only head and eyes track.")) meta = (ToolTip = "Enable body rotation toward the target.\nWhen false, only head and eyes track."))
bool bEnableBodyTracking = true; bool bEnableBodyTracking = true;
@@ -96,7 +96,7 @@ public:
* When enabled, TargetOffset is ignored and the agent looks at the midpoint * When enabled, TargetOffset is ignored and the agent looks at the midpoint
* between the target pawn's eye bones. * between the target pawn's eye bones.
* Fallback chain: eye bones → head bone → PawnViewLocation → FallbackEyeHeight. */ * Fallback chain: eye bones → head bone → PawnViewLocation → FallbackEyeHeight. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ToolTip = "Auto-target the pawn's eye bones for eye contact.\nFallback: eye bones > head bone > PawnViewLocation > FallbackEyeHeight.")) meta = (ToolTip = "Auto-target the pawn's eye bones for eye contact.\nFallback: eye bones > head bone > PawnViewLocation > FallbackEyeHeight."))
bool bAutoTargetEyes = true; bool bAutoTargetEyes = true;
@@ -104,7 +104,7 @@ public:
* AND the target is not a Pawn. For Pawns without skeleton, GetPawnViewLocation() * AND the target is not a Pawn. For Pawns without skeleton, GetPawnViewLocation()
* is used instead (accounts for BaseEyeHeight automatically). * is used instead (accounts for BaseEyeHeight automatically).
* Only applies to non-Pawn actors (props, triggers, etc.). */ * Only applies to non-Pawn actors (props, triggers, etc.). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (EditCondition = "bAutoTargetEyes", ClampMin = "0", meta = (EditCondition = "bAutoTargetEyes", ClampMin = "0",
ToolTip = "Height offset (cm) for non-Pawn targets without skeleton.\nPawns use BaseEyeHeight automatically.\nOnly used as last-resort fallback for non-Pawn actors.")) ToolTip = "Height offset (cm) for non-Pawn targets without skeleton.\nPawns use BaseEyeHeight automatically.\nOnly used as last-resort fallback for non-Pawn actors."))
float FallbackEyeHeight = 160.0f; float FallbackEyeHeight = 160.0f;
@@ -112,7 +112,7 @@ public:
/** Offset from the target actor's origin to aim at. /** Offset from the target actor's origin to aim at.
* Only used when bAutoTargetEyes is false. * Only used when bAutoTargetEyes is false.
* E.g. (0,0,160) for eye-level on a simple actor. */ * E.g. (0,0,160) for eye-level on a simple actor. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (EditCondition = "!bAutoTargetEyes", meta = (EditCondition = "!bAutoTargetEyes",
ToolTip = "Offset from target actor origin.\nE.g. (0,0,160) for eye-level.\nOnly used when Auto Target Eyes is disabled.")) ToolTip = "Offset from target actor origin.\nE.g. (0,0,160) for eye-level.\nOnly used when Auto Target Eyes is disabled."))
FVector TargetOffset = FVector(0.0f, 0.0f, 0.0f); FVector TargetOffset = FVector(0.0f, 0.0f, 0.0f);
@@ -127,44 +127,44 @@ public:
// so small movements around the target don't re-trigger higher layers. // so small movements around the target don't re-trigger higher layers.
/** Maximum head yaw rotation in degrees. */ /** Maximum head yaw rotation in degrees. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0", ClampMax = "90")) meta = (ClampMin = "0", ClampMax = "90"))
float MaxHeadYaw = 40.0f; float MaxHeadYaw = 40.0f;
/** Maximum head pitch rotation in degrees. */ /** Maximum head pitch rotation in degrees. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0", ClampMax = "90")) meta = (ClampMin = "0", ClampMax = "90"))
float MaxHeadPitch = 30.0f; float MaxHeadPitch = 30.0f;
/** Maximum horizontal eye angle in degrees. */ /** Maximum horizontal eye angle in degrees. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0", ClampMax = "90")) meta = (ClampMin = "0", ClampMax = "90"))
float MaxEyeHorizontal = 15.0f; float MaxEyeHorizontal = 15.0f;
/** Maximum vertical eye angle in degrees. */ /** Maximum vertical eye angle in degrees. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0", ClampMax = "90")) meta = (ClampMin = "0", ClampMax = "90"))
float MaxEyeVertical = 10.0f; float MaxEyeVertical = 10.0f;
// ── Smoothing speeds ───────────────────────────────────────────────────── // ── Smoothing speeds ─────────────────────────────────────────────────────
/** Body rotation interpolation speed (lower = slower, more natural). */ /** Body rotation interpolation speed (lower = slower, more natural). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0.1", ClampMax = "20")) meta = (ClampMin = "0.1", ClampMax = "20"))
float BodyInterpSpeed = 4.0f; float BodyInterpSpeed = 4.0f;
/** Head rotation interpolation speed. */ /** Head rotation interpolation speed. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0.1", ClampMax = "20")) meta = (ClampMin = "0.1", ClampMax = "20"))
float HeadInterpSpeed = 4.0f; float HeadInterpSpeed = 4.0f;
/** Eye movement interpolation speed (higher = snappier). */ /** Eye movement interpolation speed (higher = snappier). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0.1", ClampMax = "20")) meta = (ClampMin = "0.1", ClampMax = "20"))
float EyeInterpSpeed = 5.0f; float EyeInterpSpeed = 5.0f;
/** Interpolation speed when returning to neutral (TargetActor is null). */ /** Interpolation speed when returning to neutral (TargetActor is null). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0.1", ClampMax = "20")) meta = (ClampMin = "0.1", ClampMax = "20"))
float ReturnToNeutralSpeed = 3.0f; float ReturnToNeutralSpeed = 3.0f;
@@ -178,7 +178,7 @@ public:
* 1.0 = head always points at target regardless of animation. * 1.0 = head always points at target regardless of animation.
* 0.0 = gaze is additive on top of animation (old behavior). * 0.0 = gaze is additive on top of animation (old behavior).
* Default: 1.0 for conversational AI (always look at who you talk to). */ * Default: 1.0 for conversational AI (always look at who you talk to). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0", ClampMax = "1")) meta = (ClampMin = "0", ClampMax = "1"))
float HeadAnimationCompensation = 0.9f; float HeadAnimationCompensation = 0.9f;
@@ -186,7 +186,7 @@ public:
* 1.0 = eyes frozen on gaze target, animation's eye movement removed. * 1.0 = eyes frozen on gaze target, animation's eye movement removed.
* 0.0 = animation's eyes play through, gaze is additive. * 0.0 = animation's eyes play through, gaze is additive.
* Intermediate (e.g. 0.5) = smooth 50/50 blend. */ * Intermediate (e.g. 0.5) = smooth 50/50 blend. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0", ClampMax = "1")) meta = (ClampMin = "0", ClampMax = "1"))
float EyeAnimationCompensation = 0.6f; float EyeAnimationCompensation = 0.6f;
@@ -195,7 +195,7 @@ public:
* counter-rotates the gaze to keep the head pointing at the target. * counter-rotates the gaze to keep the head pointing at the target.
* 1.0 = full compensation — head stays locked on target even during bows. * 1.0 = full compensation — head stays locked on target even during bows.
* 0.0 = no compensation — head follows body movement naturally. */ * 0.0 = no compensation — head follows body movement naturally. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "0", ClampMax = "1")) meta = (ClampMin = "0", ClampMax = "1"))
float BodyDriftCompensation = 0.8f; float BodyDriftCompensation = 0.8f;
@@ -204,12 +204,12 @@ public:
/** Enable debug logging for this component. /** Enable debug logging for this component.
* When false, only key events (init, errors, state changes) are logged. * When false, only key events (init, errors, state changes) are logged.
* When true, additional diagnostic info is logged based on DebugVerbosity. */ * When true, additional diagnostic info is logged based on DebugVerbosity. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug")
bool bDebug = false; bool bDebug = false;
/** Verbosity level when bDebug is true. /** Verbosity level when bDebug is true.
* 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */ * 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug",
meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug")) meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug"))
int32 DebugVerbosity = 1; int32 DebugVerbosity = 1;
@@ -217,7 +217,7 @@ public:
* Green = head bone → target (desired). * Green = head bone → target (desired).
* Cyan = computed gaze direction (body yaw + head + eyes). * Cyan = computed gaze direction (body yaw + head + eyes).
* Useful for verifying that eye contact is accurate. */ * Useful for verifying that eye contact is accurate. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug",
meta = (EditCondition = "bDebug")) meta = (EditCondition = "bDebug"))
bool bDrawDebugGaze = false; bool bDrawDebugGaze = false;
@@ -228,14 +228,14 @@ public:
* 0 = mesh faces +X (default UE convention) * 0 = mesh faces +X (default UE convention)
* 90 = mesh faces +Y * 90 = mesh faces +Y
* -90 = mesh faces -Y */ * -90 = mesh faces -Y */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze",
meta = (ClampMin = "-180", ClampMax = "180")) meta = (ClampMin = "-180", ClampMax = "180"))
float MeshForwardYawOffset = 90.0f; float MeshForwardYawOffset = 90.0f;
// ── Head bone ──────────────────────────────────────────────────────────── // ── Head bone ────────────────────────────────────────────────────────────
/** Name of the head bone on the skeletal mesh (used for eye origin calculation). */ /** Name of the head bone on the skeletal mesh (used for eye origin calculation). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze")
FName HeadBoneName = FName(TEXT("head")); FName HeadBoneName = FName(TEXT("head"));
// ── Neck bone chain ───────────────────────────────────────────────────── // ── Neck bone chain ─────────────────────────────────────────────────────
@@ -244,7 +244,7 @@ public:
* Order: root-to-tip (e.g. neck_01 → neck_02 → head). * Order: root-to-tip (e.g. neck_01 → neck_02 → head).
* Weights should sum to ~1.0. * Weights should sum to ~1.0.
* If empty, falls back to single-bone behavior (HeadBoneName, weight 1.0). */ * If empty, falls back to single-bone behavior (HeadBoneName, weight 1.0). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Gaze") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Gaze")
TArray<FPS_AI_ConvAgent_NeckBoneEntry> NeckBoneChain; TArray<FPS_AI_ConvAgent_NeckBoneEntry> NeckBoneChain;
// ── Getters (read by AnimNode) ─────────────────────────────────────────── // ── Getters (read by AnimNode) ───────────────────────────────────────────
@@ -252,7 +252,7 @@ public:
/** Get current eye gaze curves (8 ARKit eye look curves). /** Get current eye gaze curves (8 ARKit eye look curves).
* Returns a COPY scaled by activation alpha — safe to call from any thread. * Returns a COPY scaled by activation alpha — safe to call from any thread.
* NOTE: Prefer FillCurrentEyeCurves() for hot paths to avoid per-frame allocation. */ * NOTE: Prefer FillCurrentEyeCurves() for hot paths to avoid per-frame allocation. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Gaze") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Gaze")
TMap<FName, float> GetCurrentEyeCurves() const TMap<FName, float> GetCurrentEyeCurves() const
{ {
FScopeLock Lock(&GazeDataLock); FScopeLock Lock(&GazeDataLock);

View File

@@ -26,6 +26,12 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam(FOnConvAgentDeselected,
/** Fired when no agent is within interaction range/view. */ /** Fired when no agent is within interaction range/view. */
DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnNoConvAgentInRange); DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnNoConvAgentInRange);
/** Fired on the SERVER for each player mic chunk captured for agent-less replay
* recording (int16, 16kHz mono). Lets an external recorder (PS_AI_ConvReplay)
* save player speech even when no agent is present in the scene. */
DECLARE_DYNAMIC_MULTICAST_DELEGATE_TwoParams(FOnConvAgentPlayerVoiceForRecording,
APawn*, SpeakerPawn, const TArray<uint8>&, PCMData);
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
// UPS_AI_ConvAgent_InteractionComponent // UPS_AI_ConvAgent_InteractionComponent
// //
@@ -45,7 +51,7 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnNoConvAgentInRange);
// 4. Mic management is auto-detected (no manual flag needed on agents). // 4. Mic management is auto-detected (no manual flag needed on agents).
// 5. Agents manage their own WebSocket connections independently. // 5. Agents manage their own WebSocket connections independently.
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
UCLASS(ClassGroup = "PS AI ConvAgent", meta = (BlueprintSpawnableComponent), UCLASS(ClassGroup = "ASTERION|PS_AI_ConvAgent", meta = (BlueprintSpawnableComponent),
DisplayName = "PS AI ConvAgent Interaction") DisplayName = "PS AI ConvAgent Interaction")
class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_InteractionComponent : public UActorComponent class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_InteractionComponent : public UActorComponent
{ {
@@ -57,7 +63,7 @@ public:
// ── Configuration ───────────────────────────────────────────────────────── // ── Configuration ─────────────────────────────────────────────────────────
/** Maximum distance (cm) at which an agent can be selected. Agents beyond this are ignored. */ /** Maximum distance (cm) at which an agent can be selected. Agents beyond this are ignored. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction",
meta = (ClampMin = "0", meta = (ClampMin = "0",
ToolTip = "Max distance in cm to interact with an agent.\nAgents beyond this distance are never selected.")) ToolTip = "Max distance in cm to interact with an agent.\nAgents beyond this distance are never selected."))
float MaxInteractionDistance = 300.0f; float MaxInteractionDistance = 300.0f;
@@ -65,7 +71,7 @@ public:
/** Half-angle (degrees) of the view cone used for agent selection. /** Half-angle (degrees) of the view cone used for agent selection.
* The cone is centered on the pawn's view direction (camera or control rotation). * The cone is centered on the pawn's view direction (camera or control rotation).
* Agents outside this cone are not considered unless they are the current sticky selection. */ * Agents outside this cone are not considered unless they are the current sticky selection. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction",
meta = (ClampMin = "0", ClampMax = "180", meta = (ClampMin = "0", ClampMax = "180",
ToolTip = "Half-angle of the view cone (degrees).\nAgents outside this cone are not selected.\n45 = 90-degree total FOV.")) ToolTip = "Half-angle of the view cone (degrees).\nAgents outside this cone are not selected.\n45 = 90-degree total FOV."))
float ViewConeHalfAngle = 45.0f; float ViewConeHalfAngle = 45.0f;
@@ -73,7 +79,7 @@ public:
/** Wider half-angle (degrees) used for the currently selected agent. /** Wider half-angle (degrees) used for the currently selected agent.
* This provides hysteresis so the player can look slightly away from * This provides hysteresis so the player can look slightly away from
* the current agent without losing selection (prevents flickering). */ * the current agent without losing selection (prevents flickering). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction",
meta = (ClampMin = "0", ClampMax = "180", meta = (ClampMin = "0", ClampMax = "180",
ToolTip = "Sticky cone half-angle for the current agent (degrees).\nMust be >= ViewConeHalfAngle for hysteresis to work.\n60 = agent stays selected until 120-degree total cone is exceeded.")) ToolTip = "Sticky cone half-angle for the current agent (degrees).\nMust be >= ViewConeHalfAngle for hysteresis to work.\n60 = agent stays selected until 120-degree total cone is exceeded."))
float SelectionStickyAngle = 60.0f; float SelectionStickyAngle = 60.0f;
@@ -81,14 +87,14 @@ public:
/** Height offset (cm) added to the agent's origin for the view cone check. /** Height offset (cm) added to the agent's origin for the view cone check.
* Agent origins are typically at foot level; this shifts the target point * Agent origins are typically at foot level; this shifts the target point
* up to chest/head height so close-range look-at checks don't fail. */ * up to chest/head height so close-range look-at checks don't fail. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction",
meta = (ClampMin = "0", meta = (ClampMin = "0",
ToolTip = "Height offset (cm) above the agent origin for the view cone check.\nPrevents losing selection at close range because the camera looks above the agent's feet.\n150 = roughly chest height for a MetaHuman.")) ToolTip = "Height offset (cm) above the agent origin for the view cone check.\nPrevents losing selection at close range because the camera looks above the agent's feet.\n150 = roughly chest height for a MetaHuman."))
float AgentEyeLevelOffset = 150.0f; float AgentEyeLevelOffset = 150.0f;
/** When false, only distance matters for selection (no view cone check). /** When false, only distance matters for selection (no view cone check).
* The closest agent within MaxInteractionDistance is always selected. */ * The closest agent within MaxInteractionDistance is always selected. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction",
meta = (ToolTip = "Require the player to look at an agent to select it.\nWhen false, the closest agent within range is always selected.")) meta = (ToolTip = "Require the player to look at an agent to select it.\nWhen false, the closest agent within range is always selected."))
bool bRequireLookAt = true; bool bRequireLookAt = true;
@@ -97,20 +103,20 @@ public:
/** Automatically set/clear the agent's GazeComponent TargetActor /** Automatically set/clear the agent's GazeComponent TargetActor
* when the agent is selected/deselected. When false, gaze must * when the agent is selected/deselected. When false, gaze must
* be managed from Blueprint (e.g. on conversation start). */ * be managed from Blueprint (e.g. on conversation start). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Gaze",
meta = (ToolTip = "Automatically point the agent's gaze at the pawn on selection.\nDisable for manual control (e.g. set target only when conversation starts).")) meta = (ToolTip = "Automatically point the agent's gaze at the pawn on selection.\nDisable for manual control (e.g. set target only when conversation starts)."))
bool bAutoManageGaze = true; bool bAutoManageGaze = true;
/** Delay (seconds) before setting the agent's gaze target after selection. /** Delay (seconds) before setting the agent's gaze target after selection.
* 0 = immediate. Useful to let the agent "notice" the player with a beat. */ * 0 = immediate. Useful to let the agent "notice" the player with a beat. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Gaze",
meta = (EditCondition = "bAutoManageGaze", ClampMin = "0", meta = (EditCondition = "bAutoManageGaze", ClampMin = "0",
ToolTip = "Seconds to wait before the agent looks at the pawn.\n0 = immediate.")) ToolTip = "Seconds to wait before the agent looks at the pawn.\n0 = immediate."))
float GazeAttachDelay = 0.0f; float GazeAttachDelay = 0.0f;
/** Delay (seconds) before clearing the agent's gaze target after deselection. /** Delay (seconds) before clearing the agent's gaze target after deselection.
* 0 = immediate. Useful to have the agent keep looking briefly as the player leaves. */ * 0 = immediate. Useful to have the agent keep looking briefly as the player leaves. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Gaze",
meta = (EditCondition = "bAutoManageGaze", ClampMin = "0", meta = (EditCondition = "bAutoManageGaze", ClampMin = "0",
ToolTip = "Seconds to wait before the agent stops looking at the pawn.\n0 = immediate.")) ToolTip = "Seconds to wait before the agent stops looking at the pawn.\n0 = immediate."))
float GazeDetachDelay = 0.0f; float GazeDetachDelay = 0.0f;
@@ -119,20 +125,20 @@ public:
* with head+eyes (and optionally body). This creates natural "awareness" * with head+eyes (and optionally body). This creates natural "awareness"
* without requiring an active conversation. * without requiring an active conversation.
* Also controls the passive gaze retained after switching conversations. */ * Also controls the passive gaze retained after switching conversations. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Gaze",
meta = (EditCondition = "bAutoManageGaze", meta = (EditCondition = "bAutoManageGaze",
ToolTip = "Agents in range but not selected will passively track the player.\nAlso applies to retained gaze after switching conversations.")) ToolTip = "Agents in range but not selected will passively track the player.\nAlso applies to retained gaze after switching conversations."))
bool bAutoPassiveGaze = true; bool bAutoPassiveGaze = true;
/** In passive gaze mode, the agent's head and eyes follow the player. */ /** In passive gaze mode, the agent's head and eyes follow the player. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Gaze",
meta = (EditCondition = "bAutoManageGaze && bAutoPassiveGaze", meta = (EditCondition = "bAutoManageGaze && bAutoPassiveGaze",
ToolTip = "Enable head+eyes tracking in passive gaze mode.")) ToolTip = "Enable head+eyes tracking in passive gaze mode."))
bool bPassiveGazeHeadEyes = true; bool bPassiveGazeHeadEyes = true;
/** In passive gaze mode, the agent's body also turns toward the player. /** In passive gaze mode, the agent's body also turns toward the player.
* When false, only head and eyes track — the body stays in its default animation. */ * When false, only head and eyes track — the body stays in its default animation. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction|Gaze", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Gaze",
meta = (EditCondition = "bAutoManageGaze && bAutoPassiveGaze", meta = (EditCondition = "bAutoManageGaze && bAutoPassiveGaze",
ToolTip = "Enable body rotation in passive gaze mode.\nWhen false, only head and eyes track.")) ToolTip = "Enable body rotation in passive gaze mode.\nWhen false, only head and eyes track."))
bool bPassiveGazeBody = false; bool bPassiveGazeBody = false;
@@ -142,7 +148,7 @@ public:
/** How long (seconds) the player must look at a different agent before switching /** How long (seconds) the player must look at a different agent before switching
* during an active conversation. Prevents accidental switches when glancing around. * during an active conversation. Prevents accidental switches when glancing around.
* Only applies when a conversation is active (bNetIsConversing). */ * Only applies when a conversation is active (bNetIsConversing). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction",
meta = (ClampMin = "0.0", ClampMax = "5.0", meta = (ClampMin = "0.0", ClampMax = "5.0",
ToolTip = "Seconds the player must look at another agent before switching mid-conversation.\nPrevents accidental switches when glancing around.\n0 = instant switch.")) ToolTip = "Seconds the player must look at another agent before switching mid-conversation.\nPrevents accidental switches when glancing around.\n0 = instant switch."))
float ConversationSwitchDelay = 1.0f; float ConversationSwitchDelay = 1.0f;
@@ -153,7 +159,7 @@ public:
* via StartConversationWithSelectedAgent() (e.g. on a key press). * via StartConversationWithSelectedAgent() (e.g. on a key press).
* Set to false when you have multiple agents in a scene to prevent them * Set to false when you have multiple agents in a scene to prevent them
* all from greeting the player simultaneously. */ * all from greeting the player simultaneously. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction",
meta = (ToolTip = "Auto-start the WebSocket when an agent is selected by proximity.\nSet to false to require explicit interaction (call StartConversationWithSelectedAgent).\nUseful with multiple agents to prevent simultaneous greetings.")) meta = (ToolTip = "Auto-start the WebSocket when an agent is selected by proximity.\nSet to false to require explicit interaction (call StartConversationWithSelectedAgent).\nUseful with multiple agents to prevent simultaneous greetings."))
bool bAutoStartConversation = true; bool bAutoStartConversation = true;
@@ -162,48 +168,53 @@ public:
/** Automatically call StartListening/StopListening on the agent's /** Automatically call StartListening/StopListening on the agent's
* ElevenLabsComponent when selected/deselected. When false, listening * ElevenLabsComponent when selected/deselected. When false, listening
* must be managed from Blueprint. */ * must be managed from Blueprint. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Interaction|Listening", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Listening",
meta = (ToolTip = "Automatically open/close the agent's mic turn on selection.\nDisable for manual control (e.g. push-to-talk).")) meta = (ToolTip = "Automatically open/close the agent's mic turn on selection.\nDisable for manual control (e.g. push-to-talk)."))
bool bAutoManageListening = true; bool bAutoManageListening = true;
// ── Debug ──────────────────────────────────────────────────────────────── // ── Debug ────────────────────────────────────────────────────────────────
/** Enable debug logging for this component. */ /** Enable debug logging for this component. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug")
bool bDebug = false; bool bDebug = false;
/** Verbosity level when bDebug is true. /** Verbosity level when bDebug is true.
* 0 = selection changes only, 1 = candidate evaluation, 2 = per-frame data. */ * 0 = selection changes only, 1 = candidate evaluation, 2 = per-frame data. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug",
meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug")) meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug"))
int32 DebugVerbosity = 1; int32 DebugVerbosity = 1;
// ── Events ──────────────────────────────────────────────────────────────── // ── Events ────────────────────────────────────────────────────────────────
/** Fired when a new agent enters selection. Use this to set gaze targets, show UI, etc. */ /** Fired when a new agent enters selection. Use this to set gaze targets, show UI, etc. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|Interaction|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Events",
meta = (ToolTip = "Fires when a new agent is selected.\nSet gaze targets, update UI, etc.")) meta = (ToolTip = "Fires when a new agent is selected.\nSet gaze targets, update UI, etc."))
FOnConvAgentSelected OnAgentSelected; FOnConvAgentSelected OnAgentSelected;
/** Fired when the previously selected agent loses selection. */ /** Fired when the previously selected agent loses selection. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|Interaction|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Events",
meta = (ToolTip = "Fires when the current agent is deselected.\nClear gaze targets, hide UI, etc.")) meta = (ToolTip = "Fires when the current agent is deselected.\nClear gaze targets, hide UI, etc."))
FOnConvAgentDeselected OnAgentDeselected; FOnConvAgentDeselected OnAgentDeselected;
/** Fired when no agent is within range or view cone. */ /** Fired when no agent is within range or view cone. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|Interaction|Events", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Events",
meta = (ToolTip = "Fires when no agent meets the distance/view criteria.")) meta = (ToolTip = "Fires when no agent meets the distance/view criteria."))
FOnNoConvAgentInRange OnNoAgentInRange; FOnNoConvAgentInRange OnNoAgentInRange;
/** Fired on the SERVER for each player mic chunk captured for agent-less replay
* recording. An external recorder (PS_AI_ConvReplay) subscribes to save player voice. */
UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|Interaction|Events")
FOnConvAgentPlayerVoiceForRecording OnPlayerVoiceForRecording;
// ── Blueprint API ───────────────────────────────────────────────────────── // ── Blueprint API ─────────────────────────────────────────────────────────
/** Get the currently selected agent (null if none). */ /** Get the currently selected agent (null if none). */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|Interaction") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
UPS_AI_ConvAgent_ElevenLabsComponent* GetSelectedAgent() const; UPS_AI_ConvAgent_ElevenLabsComponent* GetSelectedAgent() const;
/** Manually select a specific agent, bypassing distance/view checks. /** Manually select a specific agent, bypassing distance/view checks.
* Pass null to clear the selection. Automatic selection resumes next tick. */ * Pass null to clear the selection. Automatic selection resumes next tick. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Interaction") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
void ForceSelectAgent(UPS_AI_ConvAgent_ElevenLabsComponent* Agent); void ForceSelectAgent(UPS_AI_ConvAgent_ElevenLabsComponent* Agent);
/** Start the WebSocket conversation with the currently selected agent. /** Start the WebSocket conversation with the currently selected agent.
@@ -211,20 +222,26 @@ public:
* interacts (e.g. presses a key, enters a trigger zone). * interacts (e.g. presses a key, enters a trigger zone).
* Does nothing if no agent is selected or the agent is already connected. * Does nothing if no agent is selected or the agent is already connected.
* Also starts mic capture and listening automatically. */ * Also starts mic capture and listening automatically. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Interaction") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
void StartConversationWithSelectedAgent(); void StartConversationWithSelectedAgent();
/** Send a text message to the currently selected agent. /** Send a text message to the currently selected agent.
* The agent responds with audio and text, just as if it heard the player speak. * The agent responds with audio and text, just as if it heard the player speak.
* Handles network relay automatically (works on both server and client). * Handles network relay automatically (works on both server and client).
* Does nothing if no agent is selected or not connected. */ * Does nothing if no agent is selected or not connected. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Interaction") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
void SendTextToSelectedAgent(const FString& Text); void SendTextToSelectedAgent(const FString& Text);
/** Clear the current selection. Automatic selection resumes next tick. */ /** Clear the current selection. Automatic selection resumes next tick. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Interaction") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
void ClearSelection(); void ClearSelection();
/** Enable/disable capturing this pawn's mic for replay recording even when no
* agent is selected. Server authority only; the flag replicates so the owning
* client starts capturing + relaying its mic. Driven by PS_AI_ConvReplay. */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
void SetReplayRecordActive(bool bActive);
// ── Network relay RPCs ─────────────────────────────────────────────────── // ── Network relay RPCs ───────────────────────────────────────────────────
// UE5 clients can only call Server RPCs on actors they own. NPC actors are // UE5 clients can only call Server RPCs on actors they own. NPC actors are
// server-owned, so clients can't call RPCs on them directly. // server-owned, so clients can't call RPCs on them directly.
@@ -253,6 +270,11 @@ public:
UFUNCTION(Server, Unreliable) UFUNCTION(Server, Unreliable)
void ServerRelayMicAudio(AActor* AgentActor, const TArray<uint8>& PCMBytes); void ServerRelayMicAudio(AActor* AgentActor, const TArray<uint8>& PCMBytes);
/** Relay: stream this pawn's mic to the server purely for replay recording
* (no agent). The server broadcasts OnPlayerVoiceForRecording for the recorder. */
UFUNCTION(Server, Unreliable)
void ServerRecordVoice(const TArray<uint8>& PCMBytes);
/** Relay: send text message to an NPC agent. */ /** Relay: send text message to an NPC agent. */
UFUNCTION(Server, Reliable) UFUNCTION(Server, Reliable)
void ServerRelaySendText(AActor* AgentActor, const FString& Text); void ServerRelaySendText(AActor* AgentActor, const FString& Text);
@@ -317,6 +339,22 @@ private:
UPROPERTY() UPROPERTY()
UPS_AI_ConvAgent_MicrophoneCaptureComponent* MicComponent = nullptr; UPS_AI_ConvAgent_MicrophoneCaptureComponent* MicComponent = nullptr;
// ── Agent-less replay recording ──────────────────────────────────────────
/** Replicated: the server sets this while a replay is recording so the owning
* client captures + relays its mic even without a selected agent. */
UPROPERTY(ReplicatedUsing = OnRep_ReplayRecordActive)
bool bReplayRecordActive = false;
UFUNCTION()
void OnRep_ReplayRecordActive();
/** Start/stop mic capture to honour bReplayRecordActive (never stops the mic
* while an agent conversation still needs it). */
void UpdateReplayCapture();
/** Accumulates int16 mic bytes for the recording relay, chunked to limit RPC count. */
TArray<uint8> RecordMicBuffer;
/** True once the one-time lazy init in TickComponent has completed. /** True once the one-time lazy init in TickComponent has completed.
* Deferred from BeginPlay because IsLocallyControlled() may return false * Deferred from BeginPlay because IsLocallyControlled() may return false
* before the PlayerController has been replicated/possessed. */ * before the PlayerController has been replicated/possessed. */

View File

@@ -29,11 +29,11 @@ public:
void UnregisterAgent(UPS_AI_ConvAgent_ElevenLabsComponent* Agent); void UnregisterAgent(UPS_AI_ConvAgent_ElevenLabsComponent* Agent);
/** Get all currently registered agent components (weak pointers, may contain stale entries that are cleaned on access). */ /** Get all currently registered agent components (weak pointers, may contain stale entries that are cleaned on access). */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|Interaction") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
TArray<UPS_AI_ConvAgent_ElevenLabsComponent*> GetRegisteredAgents() const; TArray<UPS_AI_ConvAgent_ElevenLabsComponent*> GetRegisteredAgents() const;
/** Get the number of registered agents. */ /** Get the number of registered agents. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|Interaction") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|Interaction")
int32 GetNumRegisteredAgents() const; int32 GetNumRegisteredAgents() const;
private: private:

View File

@@ -39,7 +39,7 @@ DECLARE_DYNAMIC_MULTICAST_DELEGATE(FOnPS_AI_ConvAgent_VisemesReady);
* 3. Conversation starts → lip sync works automatically. * 3. Conversation starts → lip sync works automatically.
* 4. (Optional) Bind OnVisemesReady for custom Blueprint handling. * 4. (Optional) Bind OnVisemesReady for custom Blueprint handling.
*/ */
UCLASS(ClassGroup = "PS AI ConvAgent", meta = (BlueprintSpawnableComponent), UCLASS(ClassGroup = "ASTERION|PS_AI_ConvAgent", meta = (BlueprintSpawnableComponent),
DisplayName = "PS AI ConvAgent Lip Sync") DisplayName = "PS AI ConvAgent Lip Sync")
class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_LipSyncComponent : public UActorComponent class PS_AI_CONVAGENT_API UPS_AI_ConvAgent_LipSyncComponent : public UActorComponent
{ {
@@ -61,16 +61,16 @@ public:
/** Whether lip sync is currently active (read-only). /** Whether lip sync is currently active (read-only).
* Managed automatically: activates on conversation connect, deactivates on disconnect. * Managed automatically: activates on conversation connect, deactivates on disconnect.
* The underlying audio analysis keeps running so reactivation is seamless. */ * The underlying audio analysis keeps running so reactivation is seamless. */
UPROPERTY(BlueprintReadOnly, Category = "PS AI ConvAgent|LipSync") UPROPERTY(BlueprintReadOnly, Category = "ASTERION|PS_AI_ConvAgent|LipSync")
bool bActive = true; bool bActive = true;
/** How long (seconds) to blend in/out when bActive changes. */ /** How long (seconds) to blend in/out when bActive changes. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "0.05", ClampMax = "3.0")) meta = (ClampMin = "0.05", ClampMax = "3.0"))
float ActivationBlendDuration = 0.5f; float ActivationBlendDuration = 0.5f;
/** Overall mouth movement intensity multiplier. */ /** Overall mouth movement intensity multiplier. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "0.0", ClampMax = "3.0", meta = (ClampMin = "0.0", ClampMax = "3.0",
ToolTip = "Lip sync intensity.\n1.0 = normal, higher = more expressive, lower = subtler.")) ToolTip = "Lip sync intensity.\n1.0 = normal, higher = more expressive, lower = subtler."))
float LipSyncStrength = 1.0f; float LipSyncStrength = 1.0f;
@@ -78,13 +78,13 @@ public:
/** Scales the audio amplitude driving mouth movement. /** Scales the audio amplitude driving mouth movement.
* Lower values produce subtler animation, higher values are more pronounced. * Lower values produce subtler animation, higher values are more pronounced.
* Use this to tone down overly strong lip movement without changing the shape. */ * Use this to tone down overly strong lip movement without changing the shape. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "0.5", ClampMax = "1.0", meta = (ClampMin = "0.5", ClampMax = "1.0",
ToolTip = "Audio amplitude scale.\n0.5 = subtle, 0.75 = balanced, 1.0 = full.\nReduces overall mouth movement without affecting viseme shape.")) ToolTip = "Audio amplitude scale.\n0.5 = subtle, 0.75 = balanced, 1.0 = full.\nReduces overall mouth movement without affecting viseme shape."))
float AmplitudeScale = 1.0f; float AmplitudeScale = 1.0f;
/** How quickly viseme weights interpolate towards new values each frame. */ /** How quickly viseme weights interpolate towards new values each frame. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "35.0", ClampMax = "65.0", meta = (ClampMin = "35.0", ClampMax = "65.0",
ToolTip = "Smoothing speed for viseme transitions.\n35 = smooth and soft, 50 = balanced, 65 = sharp and responsive.")) ToolTip = "Smoothing speed for viseme transitions.\n35 = smooth and soft, 50 = balanced, 65 = sharp and responsive."))
float SmoothingSpeed = 50.0f; float SmoothingSpeed = 50.0f;
@@ -95,7 +95,7 @@ public:
* When the agent is NOT speaking, lip sync releases all mouth curves so the * When the agent is NOT speaking, lip sync releases all mouth curves so the
* FacialExpressionComponent's emotion curves (including mouth) can show through. * FacialExpressionComponent's emotion curves (including mouth) can show through.
* When the agent starts speaking, lip sync smoothly takes over mouth curves. */ * When the agent starts speaking, lip sync smoothly takes over mouth curves. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "0.05", ClampMax = "1.0", meta = (ClampMin = "0.05", ClampMax = "1.0",
ToolTip = "Blend duration when speech starts/stops.\nDuring silence, emotion facial curves control the mouth.\nDuring speech, lip sync takes over.")) ToolTip = "Blend duration when speech starts/stops.\nDuring silence, emotion facial curves control the mouth.\nDuring speech, lip sync takes over."))
float SpeechBlendDuration = 0.15f; float SpeechBlendDuration = 0.15f;
@@ -108,7 +108,7 @@ public:
* 0.0 = lip sync completely overrides expression curves (old behavior). * 0.0 = lip sync completely overrides expression curves (old behavior).
* 1.0 = emotion fully additive on expression curves during speech. * 1.0 = emotion fully additive on expression curves during speech.
* 0.5 = balanced blend (recommended). */ * 0.5 = balanced blend (recommended). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "0.0", ClampMax = "1.0", meta = (ClampMin = "0.0", ClampMax = "1.0",
ToolTip = "Emotion bleed-through during speech.\n0 = pure lip sync, 0.5 = balanced, 1 = full emotion on expression curves.\nOnly affects expression curves (smile, frown, etc.), not lip shape.")) ToolTip = "Emotion bleed-through during speech.\n0 = pure lip sync, 0.5 = balanced, 1 = full emotion on expression curves.\nOnly affects expression curves (smile, frown, etc.), not lip shape."))
float EmotionExpressionBlend = 0.5f; float EmotionExpressionBlend = 0.5f;
@@ -118,7 +118,7 @@ public:
/** Envelope attack time in milliseconds. /** Envelope attack time in milliseconds.
* Controls how fast the mouth opens when speech starts or gets louder. * Controls how fast the mouth opens when speech starts or gets louder.
* Lower = snappier onset, higher = gentler opening. */ * Lower = snappier onset, higher = gentler opening. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "1.0", ClampMax = "50.0", meta = (ClampMin = "1.0", ClampMax = "50.0",
ToolTip = "Envelope attack (ms).\n5 = snappy, 10 = balanced, 25 = gentle.\nHow fast the mouth opens on speech onset.")) ToolTip = "Envelope attack (ms).\n5 = snappy, 10 = balanced, 25 = gentle.\nHow fast the mouth opens on speech onset."))
float EnvelopeAttackMs = 10.0f; float EnvelopeAttackMs = 10.0f;
@@ -126,7 +126,7 @@ public:
/** Envelope release time in milliseconds. /** Envelope release time in milliseconds.
* Controls how slowly the mouth closes when speech gets quieter. * Controls how slowly the mouth closes when speech gets quieter.
* Higher = smoother, more natural decay. */ * Higher = smoother, more natural decay. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ClampMin = "10.0", ClampMax = "200.0", meta = (ClampMin = "10.0", ClampMax = "200.0",
ToolTip = "Envelope release (ms).\n25 = responsive, 50 = balanced, 120 = smooth/cinematic.\nHow slowly the mouth closes between syllables.")) ToolTip = "Envelope release (ms).\n25 = responsive, 50 = balanced, 120 = smooth/cinematic.\nHow slowly the mouth closes between syllables."))
float EnvelopeReleaseMs = 50.0f; float EnvelopeReleaseMs = 50.0f;
@@ -135,12 +135,12 @@ public:
/** Enable debug logging for this component. /** Enable debug logging for this component.
* When false, only key events (init, errors, state changes) are logged. */ * When false, only key events (init, errors, state changes) are logged. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug") UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug")
bool bDebug = false; bool bDebug = false;
/** Verbosity level when bDebug is true. /** Verbosity level when bDebug is true.
* 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */ * 0 = minimal extras, 1 = normal debug, 2 = detailed, 3 = per-frame data. */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|Debug", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|Debug",
meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug")) meta = (ClampMin = "0", ClampMax = "3", EditCondition = "bDebug"))
int32 DebugVerbosity = 1; int32 DebugVerbosity = 1;
@@ -152,7 +152,7 @@ public:
* When assigned, curve data is extracted at BeginPlay and replaces the * When assigned, curve data is extracted at BeginPlay and replaces the
* hardcoded ARKit blendshape mapping with artist-crafted poses. * hardcoded ARKit blendshape mapping with artist-crafted poses.
* Leave empty to use the built-in ARKit mapping (backward compatible). */ * Leave empty to use the built-in ARKit mapping (backward compatible). */
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PS AI ConvAgent|LipSync", UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ToolTip = "Reusable pose map asset.\nAssign your MHF_* AnimSequences once, reuse on all MetaHumans.\nLeave empty for built-in ARKit mapping.")) meta = (ToolTip = "Reusable pose map asset.\nAssign your MHF_* AnimSequences once, reuse on all MetaHumans.\nLeave empty for built-in ARKit mapping."))
TObjectPtr<UPS_AI_ConvAgent_LipSyncPoseMap> PoseMap; TObjectPtr<UPS_AI_ConvAgent_LipSyncPoseMap> PoseMap;
@@ -160,19 +160,19 @@ public:
/** Fires every tick when viseme data has been updated. /** Fires every tick when viseme data has been updated.
* Use GetCurrentVisemes() or GetCurrentBlendshapes() to read values. */ * Use GetCurrentVisemes() or GetCurrentBlendshapes() to read values. */
UPROPERTY(BlueprintAssignable, Category = "PS AI ConvAgent|LipSync", UPROPERTY(BlueprintAssignable, Category = "ASTERION|PS_AI_ConvAgent|LipSync",
meta = (ToolTip = "Fires each frame with updated viseme data.\nCall GetCurrentVisemes() or GetCurrentBlendshapes() to read values.")) meta = (ToolTip = "Fires each frame with updated viseme data.\nCall GetCurrentVisemes() or GetCurrentBlendshapes() to read values."))
FOnPS_AI_ConvAgent_VisemesReady OnVisemesReady; FOnPS_AI_ConvAgent_VisemesReady OnVisemesReady;
// ── Getters ─────────────────────────────────────────────────────────────── // ── Getters ───────────────────────────────────────────────────────────────
/** Get current OVR viseme weights (15 values: sil, PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou). */ /** Get current OVR viseme weights (15 values: sil, PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou). */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|LipSync") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|LipSync")
TMap<FName, float> GetCurrentVisemes() const { return SmoothedVisemes; } TMap<FName, float> GetCurrentVisemes() const { return SmoothedVisemes; }
/** Get current ARKit blendshape weights (MetaHuman compatible: jawOpen, mouthFunnel, mouthClose, etc.). /** Get current ARKit blendshape weights (MetaHuman compatible: jawOpen, mouthFunnel, mouthClose, etc.).
* Thread-safe: returns a snapshot updated each tick. Safe to call from anim worker threads. */ * Thread-safe: returns a snapshot updated each tick. Safe to call from anim worker threads. */
UFUNCTION(BlueprintCallable, Category = "PS AI ConvAgent|LipSync") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|LipSync")
TMap<FName, float> GetCurrentBlendshapes() const TMap<FName, float> GetCurrentBlendshapes() const
{ {
FScopeLock Lock(&BlendshapeLock); FScopeLock Lock(&BlendshapeLock);
@@ -181,7 +181,7 @@ public:
/** True when the agent is currently producing speech audio. /** True when the agent is currently producing speech audio.
* When false, lip sync releases mouth curves to let emotion curves through. */ * When false, lip sync releases mouth curves to let emotion curves through. */
UFUNCTION(BlueprintPure, Category = "PS AI ConvAgent|LipSync") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|LipSync")
bool IsSpeaking() const { return bIsSpeaking; } bool IsSpeaking() const { return bIsSpeaking; }
/** Current speech blend alpha (0 = silent/emotion mouth, 1 = lip sync mouth). /** Current speech blend alpha (0 = silent/emotion mouth, 1 = lip sync mouth).

Some files were not shown because too many files have changed in this diff Show More