You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
See discussion here. Currently, all the evaluators included in the Microsoft.Extensions.AI.Evaluation.Quality package will render tool call messages as strings with prefix ToolCall and empty content.
Ideally, tool call messages should be excluded from the rendering by default since the Quality evaluators are designed to assess the quality / efficacy of the LLM responses in the user visible portion of the chat conversation. At the same time, the base classes in the Quality package can be used to author other (custom) derived evaluators that may want to take tool calls into account - so we should leave the door open for derived implementations to override the default rendering of messages.
See discussion here. Currently, all the evaluators included in the Microsoft.Extensions.AI.Evaluation.Quality package will render tool call messages as strings with prefix
ToolCall
and empty content.Ideally, tool call messages should be excluded from the rendering by default since the Quality evaluators are designed to assess the quality / efficacy of the LLM responses in the user visible portion of the chat conversation. At the same time, the base classes in the Quality package can be used to author other (custom) derived evaluators that may want to take tool calls into account - so we should leave the door open for derived implementations to override the default rendering of messages.
FYI @peterwald
The text was updated successfully, but these errors were encountered: