Decouple ForceDisableConversation from action flow + mic timing fix

- ElevenLabsComponent::ForceDisableConversation() — removed ActionName param. Now a pure conversation master-switch with no knowledge of actions. OnReadyForAction is fired by the action flow itself: PendingActionName is set when OnAgentActionRequested is broadcasted, then OnReadyForAction fires either after blend-out (if conversation was disabled) or on the next tick (if not).
- Rename bConversationDisabledByAction → bConversationForceDisabled — the old name implied coupling with actions which no longer exists.
- Add bPendingStartListening — when StartListening is called before the WebSocket is connected (race between StartConversation and StartListening from InteractionComponent), defer instead of silently failing. Consumed automatically in HandleConnected / ClientStartConversation. Cleared by StopListening / EndConversation.
- PerceptionComponent — demote the "perceived N actors but ALL filtered out" warning to Verbose; it's a normal case for civilians surrounded by non-hostile actors, not a misconfiguration.
- CLAUDE.md — workflow rule: always describe and request confirmation before code changes.
- Martin.uasset — agent config tweaks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-27 12:37:18 +02:00
parent f338c0586e
commit a1e34e6dfa
5 changed files with 109 additions and 48 deletions

View File

@@ -1,5 +1,20 @@
# PS_AI_Agent — Notes pour Claude # PS_AI_Agent — Notes pour Claude
## Workflow règles
**TOUJOURS demander avant de lancer un changement de code.** Avant d'appeler Edit/Write/sed sur un fichier source :
1. Expliquer ce qu'on s'apprête à faire (quel fichier, quelle logique, quel impact)
2. Attendre la confirmation explicite de l'utilisateur
3. Seulement après → effectuer le changement
Cela s'applique à : modifications de code (.cpp/.h), refactors, renames bulk, ajouts de nouvelles fonctions.
Cela ne s'applique PAS à : lecture de fichiers, recherches, git status/log/diff, builds, questions diagnostiques.
Exceptions où on peut agir directement :
- L'utilisateur dit explicitement "fais-le" / "go" / "lance" / "commit"
- L'utilisateur demande explicitement un changement précis ("renomme X en Y", "ajoute cette fonction")
- Annulation / revert demandé par l'utilisateur
## Plugins dans ce repo ## Plugins dans ce repo
### PS_AI_ConvAgent ### PS_AI_ConvAgent

View File

@@ -587,8 +587,10 @@ AActor* UPS_AI_Behavior_PerceptionComponent::GetHighestThreatActor(
} }
else if (PerceivedActors.Num() > 0) else if (PerceivedActors.Num() > 0)
{ {
UE_LOG(LogPS_AI_Behavior, Warning, // Normal case: civilian surrounded by non-hostile actors, or disguised enemy
TEXT("[%s] GetHighestThreatActor: perceived %d actors but ALL were filtered out (myTeam=0x%02X). Check TeamIds and attitude."), // seeing allies. Only log at Verbose — not an issue.
UE_LOG(LogPS_AI_Behavior, Verbose,
TEXT("[%s] GetHighestThreatActor: perceived %d actors, none hostile (myTeam=0x%02X)."),
*Owner->GetName(), PerceivedActors.Num(), *Owner->GetName(), PerceivedActors.Num(),
AIC ? AIC->GetGenericTeamId().GetId() : 255); AIC ? AIC->GetGenericTeamId().GetId() : 255);
} }

View File

@@ -101,10 +101,9 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::TickComponent(float DeltaTime, ELevel
{ {
Super::TickComponent(DeltaTime, TickType, ThisTickFunction); Super::TickComponent(DeltaTime, TickType, ThisTickFunction);
// ── ForceDisableConversation blend-out monitoring ───────────────────── // ── Conversation disable: blend-out monitoring ─────────────────────────
// After ForceDisableConversation(), sub-components are blending their // After ForceDisableConversation(), sub-components blend their CurrentActiveAlpha
// CurrentActiveAlpha to 0. Once all are at (near) zero, fire OnReadyForAction // to 0. Once all near zero (or timeout), clear the wait flag.
// so the game can start the physical action.
if (bWaitingForBlendOut) if (bWaitingForBlendOut)
{ {
BlendOutElapsedTime += DeltaTime; BlendOutElapsedTime += DeltaTime;
@@ -139,23 +138,33 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::TickComponent(float DeltaTime, ELevel
bWaitingForBlendOut = false; bWaitingForBlendOut = false;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("ForceDisableConversation: all components blended to neutral — firing OnReadyForAction for action '%s'."), TEXT("ForceDisableConversation: all components blended to neutral."));
*PendingActionName);
OnReadyForAction.Broadcast(PendingActionName);
} }
else if (bTimedOut) else if (bTimedOut)
{ {
// Safety timeout — some component didn't reach alpha 0 in time. // Safety timeout — some component didn't reach alpha 0 in time.
// Fire OnReadyForAction anyway to avoid blocking the game action forever.
bWaitingForBlendOut = false; bWaitingForBlendOut = false;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("ForceDisableConversation: blend-out timed out after %.1fs — not all components at neutral. Firing OnReadyForAction for action '%s' anyway."), TEXT("ForceDisableConversation: blend-out timed out after %.1fs — not all components at neutral."),
BlendOutTimeoutSeconds, *PendingActionName); BlendOutTimeoutSeconds);
OnReadyForAction.Broadcast(PendingActionName);
} }
} }
// ── Action flow: fire OnReadyForAction when ready ──────────────────────
// An action was requested via OnAgentActionRequested. Fire OnReadyForAction
// as soon as the agent is ready:
// - If ForceDisableConversation was called, wait for blend-out to complete.
// - Otherwise, fire on the next tick (quasi-immediate).
if (!PendingActionName.IsEmpty() && !bWaitingForBlendOut)
{
const FString ActionToFire = PendingActionName;
PendingActionName.Empty();
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("OnReadyForAction: firing for action '%s'."), *ActionToFire);
OnReadyForAction.Broadcast(ActionToFire);
}
// Response timeout: if the server hasn't started generating within ResponseTimeoutSeconds // Response timeout: if the server hasn't started generating within ResponseTimeoutSeconds
// after the user stopped speaking, notify Blueprint so it can react (e.g. show "try again"). // after the user stopped speaking, notify Blueprint so it can react (e.g. show "try again").
if (bWaitingForAgentResponse && ResponseTimeoutSeconds > 0.0f && TurnEndTime > 0.0) if (bWaitingForAgentResponse && ResponseTimeoutSeconds > 0.0f && TurnEndTime > 0.0)
@@ -400,7 +409,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::TickComponent(float DeltaTime, ELevel
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_ElevenLabsComponent::StartConversation() void UPS_AI_ConvAgent_ElevenLabsComponent::StartConversation()
{ {
if (bConversationDisabledByAction) if (bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("StartConversation: blocked — conversation disabled by ForceDisableConversation(). Call ForceEnableConversation() first.")); TEXT("StartConversation: blocked — conversation disabled by ForceDisableConversation(). Call ForceEnableConversation() first."));
@@ -492,6 +501,9 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StartConversation_Internal()
void UPS_AI_ConvAgent_ElevenLabsComponent::EndConversation() void UPS_AI_ConvAgent_ElevenLabsComponent::EndConversation()
{ {
// Clear any pending listen request so it doesn't leak across sessions.
bPendingStartListening = false;
if (GetOwnerRole() == ROLE_Authority) if (GetOwnerRole() == ROLE_Authority)
{ {
// Standalone / listen-server: leave via the local player controller. // Standalone / listen-server: leave via the local player controller.
@@ -525,7 +537,12 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StartListening()
const bool bEffectivelyConnected = IsConnected() || (GetOwnerRole() != ROLE_Authority && bNetIsConversing); const bool bEffectivelyConnected = IsConnected() || (GetOwnerRole() != ROLE_Authority && bNetIsConversing);
if (!bEffectivelyConnected) if (!bEffectivelyConnected)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, TEXT("StartListening: not connected (IsConnected=%s bNetIsConversing=%s Role=%d)."), // Not ready yet — defer until HandleConnected fires.
// Prevents silent audio loss when StartListening is called synchronously
// after StartConversation (async WebSocket handshake).
bPendingStartListening = true;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("StartListening deferred: WebSocket not connected yet (IsConnected=%s bNetIsConversing=%s Role=%d). Will open mic on connect."),
IsConnected() ? TEXT("true") : TEXT("false"), IsConnected() ? TEXT("true") : TEXT("false"),
bNetIsConversing ? TEXT("true") : TEXT("false"), bNetIsConversing ? TEXT("true") : TEXT("false"),
static_cast<int32>(GetOwnerRole())); static_cast<int32>(GetOwnerRole()));
@@ -621,6 +638,10 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StartListening()
void UPS_AI_ConvAgent_ElevenLabsComponent::StopListening() void UPS_AI_ConvAgent_ElevenLabsComponent::StopListening()
{ {
// Also cancel any pending-start request (e.g. StartListening was deferred
// before WS connect and the user now wants to cancel).
bPendingStartListening = false;
if (!bIsListening) return; if (!bIsListening) return;
bIsListening = false; bIsListening = false;
@@ -781,21 +802,19 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::InterruptAgent()
// ForceDisableConversation / ForceEnableConversation // ForceDisableConversation / ForceEnableConversation
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────
void UPS_AI_ConvAgent_ElevenLabsComponent::ForceDisableConversation(const FString& ActionName) void UPS_AI_ConvAgent_ElevenLabsComponent::ForceDisableConversation()
{ {
if (bConversationDisabledByAction) if (bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("ForceDisableConversation: already disabled (pending action: %s)."), *PendingActionName); TEXT("ForceDisableConversation: already disabled."));
return; return;
} }
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("ForceDisableConversation: shutting down conversation for action '%s', blending to neutral."), *ActionName); TEXT("ForceDisableConversation: shutting down conversation, blending to neutral."));
PendingActionName = ActionName; bConversationForceDisabled = true;
bConversationDisabledByAction = true;
bWaitingForBlendOut = true; bWaitingForBlendOut = true;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
@@ -856,7 +875,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ForceDisableConversation(const FStrin
void UPS_AI_ConvAgent_ElevenLabsComponent::ForceEnableConversation() void UPS_AI_ConvAgent_ElevenLabsComponent::ForceEnableConversation()
{ {
if (!bConversationDisabledByAction) if (!bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Warning,
TEXT("ForceEnableConversation: not currently disabled.")); TEXT("ForceEnableConversation: not currently disabled."));
@@ -866,7 +885,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ForceEnableConversation()
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("ForceEnableConversation: conversation re-enabled.")); TEXT("ForceEnableConversation: conversation re-enabled."));
bConversationDisabledByAction = false; bConversationForceDisabled = false;
bWaitingForBlendOut = false; bWaitingForBlendOut = false;
BlendOutElapsedTime = 0.0f; BlendOutElapsedTime = 0.0f;
@@ -1069,7 +1088,14 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::HandleConnected(const FPS_AI_ConvAgen
{ {
StartListening(); StartListening();
} }
// Consume any pending listen request deferred before WebSocket was ready.
else if (bPendingStartListening)
{
bPendingStartListening = false;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("HandleConnected: consuming pending StartListening request."));
StartListening();
}
} }
void UPS_AI_ConvAgent_ElevenLabsComponent::HandleDisconnected(int32 StatusCode, const FString& Reason) void UPS_AI_ConvAgent_ElevenLabsComponent::HandleDisconnected(int32 StatusCode, const FString& Reason)
@@ -1451,6 +1477,10 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::HandleClientToolCall(const FPS_AI_Con
TEXT("[T+%.2fs] Agent action requested: %s"), T, *ActionName); TEXT("[T+%.2fs] Agent action requested: %s"), T, *ActionName);
} }
// Track the pending action so OnReadyForAction can fire with the same name.
// If the user synchronously calls ForceDisableConversation() in their handler,
// OnReadyForAction will wait for blend-out. Otherwise it fires on next tick.
PendingActionName = ActionName;
OnAgentActionRequested.Broadcast(ActionName); OnAgentActionRequested.Broadcast(ActionName);
// Auto-respond so the agent can continue speaking. // Auto-respond so the agent can continue speaking.
@@ -2100,7 +2130,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ServerJoinConversation_Implementation
if (!Pawn) return; if (!Pawn) return;
// Block join while force-disabled (ForceDisableConversation active). // Block join while force-disabled (ForceDisableConversation active).
if (bConversationDisabledByAction) if (bConversationForceDisabled)
{ {
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log, UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[NET] ServerJoinConversation: blocked — conversation disabled by ForceDisableConversation().")); TEXT("[NET] ServerJoinConversation: blocked — conversation disabled by ForceDisableConversation()."));
@@ -2367,6 +2397,14 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::ClientConversationStarted_Implementat
{ {
StartListening(); StartListening();
} }
// Consume any pending listen request deferred before conversation was ready.
else if (bPendingStartListening)
{
bPendingStartListening = false;
UE_LOG(LogPS_AI_ConvAgent_ElevenLabs, Log,
TEXT("[Client] Consuming pending StartListening request."));
StartListening();
}
} }
void UPS_AI_ConvAgent_ElevenLabsComponent::ClientConversationFailed_Implementation( void UPS_AI_ConvAgent_ElevenLabsComponent::ClientConversationFailed_Implementation(
@@ -2745,12 +2783,13 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::DrawDebugHUD() const
bWantsReconnect ? TEXT(" (ACTIVE)") : TEXT(""))); bWantsReconnect ? TEXT(" (ACTIVE)") : TEXT("")));
// ForceDisable state // ForceDisable state
if (bConversationDisabledByAction || bWaitingForBlendOut) if (bConversationForceDisabled || bWaitingForBlendOut)
{ {
const FColor DisableColor = FColor::Orange; const FColor DisableColor = FColor::Orange;
const FString PendingStr = PendingActionName.IsEmpty() ? TEXT("<none>") : PendingActionName;
GEngine->AddOnScreenDebugMessage(BaseKey + 9, DisplayTime, DisableColor, GEngine->AddOnScreenDebugMessage(BaseKey + 9, DisplayTime, DisableColor,
FString::Printf(TEXT(" FORCE DISABLED — action: '%s' blendOut: %s (%.1fs)"), FString::Printf(TEXT(" FORCE DISABLED — pending action: '%s' blendOut: %s (%.1fs)"),
*PendingActionName, *PendingStr,
bWaitingForBlendOut ? TEXT("WAITING") : TEXT("DONE"), bWaitingForBlendOut ? TEXT("WAITING") : TEXT("DONE"),
BlendOutElapsedTime)); BlendOutElapsedTime));

View File

@@ -548,25 +548,21 @@ public:
void InterruptAgent(); void InterruptAgent();
/** /**
* Cleanly disable conversation on this agent so the NPC can switch to game AI. * Cleanly disable conversation on this agent.
* Ends the WebSocket connection, stops all audio, and begins blending all * Ends the WebSocket connection, stops all audio, and begins blending all
* visual components (gaze, lip sync, facial expression, body expression) * visual components (gaze, lip sync, facial expression, body expression)
* back to their neutral state. While disabled, the InteractionComponent * back to their neutral state. While disabled, the InteractionComponent
* cannot auto-restart a conversation with this agent. * cannot auto-restart a conversation with this agent.
* *
* When all components have blended to neutral, fires OnReadyForAction. * This function ONLY handles the conversation master-switch. It has no
* knowledge of action/reaction flow — if an action was pending via
* OnAgentActionRequested, OnReadyForAction will fire automatically once
* blend-out completes. If no action is pending, nothing else happens.
*
* Call ForceEnableConversation() later to allow conversation again. * Call ForceEnableConversation() later to allow conversation again.
*
* Typical usage from Blueprint (OnAgentActionRequested):
* 1. ForceDisableConversation("flee")
* 2. Wait for OnReadyForAction — receives the same ActionName
* 3. Play action montage based on ActionName
* 4. When done, call ForceEnableConversation()
*
* @param ActionName The action to perform once neutral (passed through to OnReadyForAction).
*/ */
UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs") UFUNCTION(BlueprintCallable, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
void ForceDisableConversation(const FString& ActionName); void ForceDisableConversation();
/** /**
* Re-enable conversation after a ForceDisableConversation() call. * Re-enable conversation after a ForceDisableConversation() call.
@@ -612,7 +608,7 @@ public:
/** True while conversation is force-disabled (ForceDisableConversation was called). /** True while conversation is force-disabled (ForceDisableConversation was called).
* While disabled, StartConversation and ServerJoinConversation are blocked. */ * While disabled, StartConversation and ServerJoinConversation are blocked. */
UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs") UFUNCTION(BlueprintPure, Category = "ASTERION|PS_AI_ConvAgent|ElevenLabs")
bool IsConversationDisabled() const { return bConversationDisabledByAction; } bool IsConversationDisabled() const { return bConversationForceDisabled; }
/** True while audio is being pre-buffered (playback hasn't started yet). /** True while audio is being pre-buffered (playback hasn't started yet).
* Used by the LipSync component to pause viseme queue consumption. */ * Used by the LipSync component to pause viseme queue consumption. */
@@ -706,24 +702,33 @@ private:
UPROPERTY() UPROPERTY()
USoundWaveProcedural* ProceduralSoundWave = nullptr; USoundWaveProcedural* ProceduralSoundWave = nullptr;
// ── ForceDisableConversation state ─────────────────────────────────────── // ── Conversation disable state ───────────────────────────────────────────
// Set by ForceDisableConversation(), cleared by ForceEnableConversation(). // Set by ForceDisableConversation(), cleared by ForceEnableConversation().
// While true, StartConversation() and ServerJoinConversation() are blocked. // While true, StartConversation() and ServerJoinConversation() are blocked.
bool bConversationDisabledByAction = false; bool bConversationForceDisabled = false;
// True while waiting for all visual components to blend back to neutral. // True while waiting for all visual components to blend back to neutral
// Monitored in TickComponent — fires OnReadyForAction when complete. // after ForceDisableConversation(). Monitored in TickComponent.
bool bWaitingForBlendOut = false; bool bWaitingForBlendOut = false;
// The action name passed to ForceDisableConversation(), forwarded to OnReadyForAction. // Accumulated time waiting for blend-out. Safety timeout to unblock
FString PendingActionName;
// Accumulated time waiting for blend-out. Safety timeout fires OnReadyForAction
// if components haven't reached neutral within this limit. // if components haven't reached neutral within this limit.
float BlendOutElapsedTime = 0.0f; float BlendOutElapsedTime = 0.0f;
static constexpr float BlendOutTimeoutSeconds = 5.0f; static constexpr float BlendOutTimeoutSeconds = 5.0f;
// ── Action flow state (independent from disable state) ───────────────────
// Set when OnAgentActionRequested is broadcasted (LLM tool call).
// Cleared when OnReadyForAction is broadcasted.
// - If conversation is being disabled, OnReadyForAction waits for blend-out.
// - If not, OnReadyForAction fires on the next tick (quasi-immediate).
FString PendingActionName;
// ── State ───────────────────────────────────────────────────────────────── // ── State ─────────────────────────────────────────────────────────────────
// Atomic: read from WASAPI background thread (OnMicrophoneDataCaptured), written from game thread. // Atomic: read from WASAPI background thread (OnMicrophoneDataCaptured), written from game thread.
std::atomic<bool> bIsListening{false}; std::atomic<bool> bIsListening{false};
std::atomic<bool> bAgentSpeaking{false}; std::atomic<bool> bAgentSpeaking{false};
// Set when StartListening() is called before the WebSocket is connected.
// Consumed in HandleConnected() to open the microphone automatically once ready.
// Cleared by StopListening() or EndConversation() to cancel the pending request.
bool bPendingStartListening = false;
// True from the first agent_chat_response_part until the first audio chunk arrives. // True from the first agent_chat_response_part until the first audio chunk arrives.
// Used to block StartListening() while the server is processing the previous turn. // Used to block StartListening() while the server is processing the previous turn.
// Atomic: defensive — currently game-thread only, but documents thread-safety contract. // Atomic: defensive — currently game-thread only, but documents thread-safety contract.