j.foucher/PS_AI_Agent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix latency HUD showing --- in Server VAD mode

Browse Source 
Move latency reset from StopListening to HandleAgentResponseStarted.
In Server VAD + interruption mode, StopListening is never called so
TurnEndTime stayed at 0 and all dependent metrics showed ---. Now
HandleAgentResponseStarted detects whether StopListening provided a
fresh TurnEndTime; if not (Server VAD), it uses Now as approximation.
Also fix DisplayTime from 0 to 1s to prevent HUD flicker.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
j.foucher 2026-03-05 18:22:28 +01:00
parent 955c97e0dd
commit a8dd5a022f
1 changed files with 21 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
Unreal/PS_AI_Agent/Plugins/PS_AI_ConvAgent/Source/PS_AI_ConvAgent/Private/PS_AI_ConvAgent_ElevenLabsComponent.cpp
View File

@ -595,13 +595,6 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::StopListening()
TurnEndTime = FPlatformTime::Seconds();
// Reset latency measurements for this new response cycle.
CurrentLatencies = FDebugLatencies();
if (TurnStartTime > 0.0)
{
CurrentLatencies.UserSpeechMs = static_cast<float>((TurnEndTime - TurnStartTime) * 1000.0);
}
// Start the response timeout clock — but only when the server hasn't already started
// generating. When StopListening() is called from HandleAgentResponseStarted() as part
// of collision avoidance, bAgentGenerating is already true, meaning the server IS already
@ -1083,14 +1076,30 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::HandleAgentResponseStarted()
}
const double Now = FPlatformTime::Seconds();
GenerationStartTime = Now;
if (TurnEndTime > 0.0)
// --- Latency reset for this new response cycle ---
// In Server VAD mode, StopListening() is not called — the server detects
// end of user speech and immediately starts generating. If TurnEndTime was
// not set by StopListening since the last generation (i.e. it's stale or 0),
// use Now as the best client-side approximation.
const bool bFreshTurnEnd = (TurnEndTime > GenerationStartTime) && (GenerationStartTime > 0.0);
if (!bFreshTurnEnd)
{
CurrentLatencies.STTToGenMs = static_cast<float>((Now - TurnEndTime) * 1000.0);
TurnEndTime = Now;
}
// Reset all latency measurements — new response cycle starts here.
CurrentLatencies = FDebugLatencies();
if (TurnStartTime > 0.0 && TurnEndTime > TurnStartTime)
{
CurrentLatencies.UserSpeechMs = static_cast<float>((TurnEndTime - TurnStartTime) * 1000.0);
}
GenerationStartTime = Now;
CurrentLatencies.STTToGenMs = static_cast<float>((Now - TurnEndTime) * 1000.0);
const double T = Now - SessionStartTime;
const double LatencyFromTurnEnd = TurnEndTime > 0.0 ? Now - TurnEndTime : 0.0;
const double LatencyFromTurnEnd = Now - TurnEndTime;
if (bIsListening)
{
// In Server VAD + interruption mode, keep the mic open so the server can
@ -2407,7 +2416,7 @@ void UPS_AI_ConvAgent_ElevenLabsComponent::DrawLatencyHUD() const
// Separate BaseKey range so it never collides with DrawDebugHUD
const int32 BaseKey = 93700;
const float DisplayTime = 0.0f; // refreshed every tick
const float DisplayTime = 1.0f; // long enough to avoid flicker between ticks
const FColor TitleColor = FColor::Cyan;
const FColor ValueColor = FColor::White;