1. 聊天模型可觀(guān)測(cè)性Chat Model Observability部分ChatModel/StreamingChatModel實(shí)現(xiàn)見(jiàn)「Observability」列支持配置ChatModelListener監(jiān)聽(tīng)三類(lèi)事件事件回調(diào)方法時(shí)機(jī)請(qǐng)求onRequest(ChatModelRequestContext)調(diào)用 LLM 提供商 API之前響應(yīng)onResponse(ChatModelResponseContext)成功收到響應(yīng)后立即調(diào)用錯(cuò)誤onError(ChatModelErrorContext)出錯(cuò)時(shí)僅一次事件屬性遵循 OpenTelemetry Generative AI 語(yǔ)義約定。事件包含的屬性請(qǐng)求Request消息messages模型model溫度TemperatureTop P最大 tokens 數(shù)工具Tools響應(yīng)格式responseFormat等等響應(yīng)Response助手消息AiMessageID模型Token 使用情況TokenUsage結(jié)束原因finishReason等等2. ChatModelListener 完整示例ChatModelListenerlistenernewChatModelListener(){OverridepublicvoidonRequest(ChatModelRequestContextrequestContext){ChatRequestchatRequestrequestContext.chatRequest();ListChatMessagemessageschatRequest.messages();System.out.println(messages);ChatRequestParametersparameterschatRequest.parameters();System.out.println(parameters.modelName());System.out.println(parameters.temperature());System.out.println(parameters.topP());System.out.println(parameters.topK());System.out.println(parameters.frequencyPenalty());System.out.println(parameters.presencePenalty());System.out.println(parameters.maxOutputTokens());System.out.println(parameters.stopSequences());System.out.println(parameters.toolSpecifications());System.out.println(parameters.toolChoice());System.out.println(parameters.responseFormat());// 提供商特有參數(shù)以 OpenAI 為例需 instanceof 判斷if(parametersinstanceofOpenAiChatRequestParametersopenAiParameters){System.out.println(openAiParameters.maxCompletionTokens());System.out.println(openAiParameters.logitBias());System.out.println(openAiParameters.parallelToolCalls());System.out.println(openAiParameters.seed());System.out.println(openAiParameters.user());System.out.println(openAiParameters.store());System.out.println(openAiParameters.metadata());System.out.println(openAiParameters.serviceTier());System.out.println(openAiParameters.reasoningEffort());}System.out.println(requestContext.modelProvider());// 在 onRequest / onResponse / onError 之間傳遞數(shù)據(jù)MapObject,ObjectattributesrequestContext.attributes();attributes.put(my-attribute,my-value);}OverridepublicvoidonResponse(ChatModelResponseContextresponseContext){ChatResponsechatResponseresponseContext.chatResponse();AiMessageaiMessagechatResponse.aiMessage();System.out.println(aiMessage);ChatResponseMetadatametadatachatResponse.metadata();System.out.println(metadata.id());System.out.println(metadata.modelName());System.out.println(metadata.finishReason());if(metadatainstanceofOpenAiChatResponseMetadataopenAiMetadata){System.out.println(openAiMetadata.created());System.out.println(openAiMetadata.serviceTier());System.out.println(openAiMetadata.systemFingerprint());}TokenUsagetokenUsagemetadata.tokenUsage();System.out.println(tokenUsage.inputTokenCount());System.out.println(tokenUsage.outputTokenCount());System.out.println(tokenUsage.totalTokenCount());if(tokenUsageinstanceofOpenAiTokenUsageopenAiTokenUsage){System.out.println(openAiTokenUsage.inputTokensDetails().cachedTokens());System.out.println(openAiTokenUsage.outputTokensDetails().reasoningTokens());}ChatRequestchatRequestresponseContext.chatRequest();System.out.println(chatRequest);System.out.println(responseContext.modelProvider());MapObject,ObjectattributesresponseContext.attributes();System.out.println(attributes.get(my-attribute));}OverridepublicvoidonError(ChatModelErrorContexterrorContext){ThrowableerrorerrorContext.error();error.printStackTrace();ChatRequestchatRequesterrorContext.chatRequest();System.out.println(chatRequest);System.out.println(errorContext.modelProvider());MapObject,ObjectattributeserrorContext.attributes();System.out.println(attributes.get(my-attribute));}};ChatModelmodelOpenAiChatModel.builder().apiKey(System.getenv(OPENAI_API_KEY)).modelName(GPT_4_O_MINI).listeners(List.of(listener)).build();model.chat(Tell me a joke about Java);attributes 映射的作用attributesMapObject, Object用于在同一個(gè)ChatModelListener的onRequest→onResponse/onError之間傳遞信息如請(qǐng)求開(kāi)始時(shí)間、traceId。在多個(gè)ChatModelListener之間傳遞信息。3. 監(jiān)聽(tīng)器的工作方式重要規(guī)則調(diào)用順序與線(xiàn)程監(jiān)聽(tīng)器以L(fǎng)istChatModelListener形式配置按迭代順序依次調(diào)用。同步調(diào)用運(yùn)行在同一個(gè)線(xiàn)程流式場(chǎng)景例外見(jiàn)下。第一個(gè)監(jiān)聽(tīng)器返回之前第二個(gè)不會(huì)被調(diào)用。調(diào)用次數(shù)onRequest()每次請(qǐng)求只調(diào)用一次LLM API 出錯(cuò)重試時(shí)不會(huì)為每次重試重復(fù)調(diào)用。onResponse()只調(diào)用一次成功收到響應(yīng)后立即調(diào)用。onError()只調(diào)用一次重試時(shí)不會(huì)為每次重試重復(fù)調(diào)用。異常處理監(jiān)聽(tīng)器方法拋出的異常以WARN級(jí)別記錄日志。后續(xù)監(jiān)聽(tīng)器照常繼續(xù)執(zhí)行不會(huì)中斷。請(qǐng)求內(nèi)容是“最終請(qǐng)求”三個(gè) Context 中拿到的ChatRequest是合并后的最終請(qǐng)求ChatModel上配置的默認(rèn)ChatRequestParameters 請(qǐng)求特定的ChatRequestParameters。StreamingChatModel 的特殊點(diǎn)onResponse()/onError()與onRequest()在不同線(xiàn)程中執(zhí)行。線(xiàn)程上下文如 MDC、ThreadLocal不會(huì)自動(dòng)傳播→ 需要用attributes把數(shù)據(jù)從onRequest()傳到onResponse()/onError()。時(shí)序onResponse()早于StreamingChatResponseHandler.onCompleteResponse()。onError()早于StreamingChatResponseHandler.onError()。4. 在 Spring Boot 應(yīng)用中的可觀(guān)測(cè)性原文此處為小節(jié)標(biāo)題展開(kāi)內(nèi)容待補(bǔ)充。實(shí)踐要點(diǎn)結(jié)合上文規(guī)則的落地建議把ChatModelListener注冊(cè)為 Spring Bean如Component或Bean在構(gòu)建ChatModel時(shí)通過(guò).listeners(...)注入便于統(tǒng)一收集指標(biāo)/日志。在onRequest()中把traceId、開(kāi)始時(shí)間等放入attributes在onResponse()/onError()中取出配合 Micrometer / OpenTelemetry 上報(bào)耗時(shí)onResponse/onError時(shí)間戳 ?onRequest時(shí)間戳Token 消耗tokenUsage.inputTokenCount()/outputTokenCount()/totalTokenCount()錯(cuò)誤率onError計(jì)數(shù) errorContext.error()監(jiān)聽(tīng)器內(nèi)不要拋異常異常只會(huì)被 WARN 吞掉應(yīng)自行 try-catch 并落日志避免影響監(jiān)控鏈路完整性。速查小結(jié)三個(gè)回調(diào)onRequest請(qǐng)求前一次、onResponse成功響應(yīng)一次、onError出錯(cuò)一次重試不會(huì)重復(fù)觸發(fā)。一個(gè)橋梁attributes映射 —— 跨回調(diào)、跨監(jiān)聽(tīng)器傳值流式場(chǎng)景線(xiàn)程不同必須靠它傳遞上下文。兩類(lèi)參數(shù)通用ChatRequestParameters/ChatResponseMetadata 提供商特有如OpenAiChatRequestParameters、OpenAiChatResponseMetadata、OpenAiTokenUsage用instanceof取。線(xiàn)程非流式同線(xiàn)程同步流式onResponse/onError在另一線(xiàn)程且早于StreamingChatResponseHandler對(duì)應(yīng)回調(diào)。容錯(cuò)監(jiān)聽(tīng)器拋異常只記 WARN不影響后續(xù)監(jiān)聽(tīng)器與主流程。語(yǔ)義約定屬性命名對(duì)齊 OpenTelemetry Generative AI 語(yǔ)義約定便于接入現(xiàn)有 APM。