j.foucher/PS_AI_Agent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix: handle binary WebSocket frames from ElevenLabs

Browse Source 
ElevenLabs sends all JSON messages as binary WS frames, not text frames.
The OnRawMessage callback receives them; we were logging them as warnings
and discarding the data entirely — causing no events to fire at all.

Fix: accumulate binary frame fragments (BytesRemaining > 0 = more coming),
reassemble into a complete buffer, decode as UTF-8 JSON string, then route
through the existing OnWsMessage text handler unchanged.

Added BinaryFrameBuffer (TArray<uint8>) to proxy header for accumulation.
Compiles cleanly on UE 5.5 Win64.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
j.foucher 2026-02-19 13:51:47 +01:00
parent b489d1174c
commit 669c503d06
2 changed files with 24 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
Unreal/PS_AI_Agent/Plugins/PS_AI_Agent_ElevenLabs/Source/PS_AI_Agent_ElevenLabs/Private/ElevenLabsWebSocketProxy.cpp
View File

@ -237,9 +237,26 @@ void UElevenLabsWebSocketProxy::OnWsMessage(const FString& Message)
void UElevenLabsWebSocketProxy::OnWsBinaryMessage(const void* Data, SIZE_T Size, SIZE_T BytesRemaining) void UElevenLabsWebSocketProxy::OnWsBinaryMessage(const void* Data, SIZE_T Size, SIZE_T BytesRemaining)
{ {
// ElevenLabs Conversational AI uses text (JSON) frames only. // ElevenLabs sends its JSON messages as binary WebSocket frames (not text frames).
// If binary frames arrive in future API versions, handle here. // Accumulate fragments until BytesRemaining == 0, then parse the complete message.
UE_LOG(LogElevenLabsWS, Warning, TEXT("Received unexpected binary WebSocket frame (%llu bytes)."), (uint64)Size);
const uint8* Bytes = static_cast<const uint8*>(Data);
BinaryFrameBuffer.Append(Bytes, Size);
if (BytesRemaining > 0)
{
// More fragments coming — wait for the rest
return;
}
// Full message received — interpret as UTF-8 JSON
const FString JsonString = FString(UTF8_TO_TCHAR(
reinterpret_cast<const char*>(BinaryFrameBuffer.GetData())));
BinaryFrameBuffer.Reset();
// Route through the existing text message handler
OnWsMessage(JsonString);
} }
// ───────────────────────────────────────────────────────────────────────────── // ─────────────────────────────────────────────────────────────────────────────

4
Unreal/PS_AI_Agent/Plugins/PS_AI_Agent_ElevenLabs/Source/PS_AI_Agent_ElevenLabs/Public/ElevenLabsWebSocketProxy.h
View File

@ -179,4 +179,8 @@ private:
TSharedPtr<IWebSocket> WebSocket; TSharedPtr<IWebSocket> WebSocket;
EElevenLabsConnectionState ConnectionState = EElevenLabsConnectionState::Disconnected; EElevenLabsConnectionState ConnectionState = EElevenLabsConnectionState::Disconnected;
FElevenLabsConversationInfo ConversationInfo; FElevenLabsConversationInfo ConversationInfo;
// Accumulation buffer for multi-fragment binary WebSocket frames.
// ElevenLabs sends JSON as binary frames; large messages arrive in fragments.
TArray<uint8> BinaryFrameBuffer;
}; };