Gemini fails to decode

[gtokman]

Describe the bug

When using Gemini flash models the stream is failing to decode. Looks like Gemini ommits a few fields from the response. Should these fields be optional?

  "id": "chatcmpl-B9Yydp5DMEQ0z5Hr7ujy0DqaBGZ2E",
  "service_tier": "default",
  "system_fingerprint": "fp_eb9dce56a8",

{
  "id": "chatcmpl-B9Yydp5DMEQ0z5Hr7ujy0DqaBGZ2E",
  "object": "chat.completion.chunk",
  "created": 1741619103,
  "model": "gpt-4o-2024-08-06",
  "service_tier": "default",
  "system_fingerprint": "fp_eb9dce56a8",
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": "",
        "refusal": null
      },
      "logprobs": null,
      "finish_reason": null
    }
  ]
}

{
  "choices": [
    {
      "delta": {
        "content": "Hey",
        "role": "assistant"
      },
      "index": 0
    }
  ],
  "created": 1741619073,
  "model": "gemini-2.0-flash-lite",
  "object": "chat.completion.chunk"
}

[gtokman]

Also, I'm not sure if the goal of the library is to be agnostic across many different models. If not, happy to close this!

[nezhyborets]

@gtokman current decision is to support other providers without a harm to the main one - OpenAI. In this case (and there are other similar cases) making fields optional would somewhat drift the Swift structs away from official API Reference of OpenAI, because the reference would point out that a field is non-optional, but it would be optional in our SDK, which may be strange. But let's not close this, maybe we'll make a workaround for such a case without downgrading the experience of working with OpenAI

[gtokman]

@nezhyborets thanks for the context! What about in this case where the ChatCompletionMessageParam doesn't include the reasoning prop?

{
  "id": "chatcmpl-260011a1-d778-45bf-877c-46291ddeb5c3",
  "object": "chat.completion",
  "created": 1741629033,
  "model": "qwen-qwq-32b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning": "Okay, the user is asking...",
        "tool_calls": [
          {
            "id": "call_zgfy",
            "type": "function",
            "function": {
              "name": "get_linked_accounts",
              "arguments": "{}"
            }
          }
        ]
      },
      "logprobs": null,
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "queue_time": 0.21565806099999998,
    "prompt_tokens": 282,
    "prompt_time": 0.019009824,
    "completion_tokens": 181,
    "completion_time": 0.418294019,
    "total_tokens": 463,
    "total_time": 0.437303843
  },
  "system_fingerprint": "fp_c81bd6c71c",
  "x_groq": {
    "id": "req_01jp0jw2qwf6ar1kf7jbza1m7h"
  }
}

[nezhyborets]

@gtokman interesting! Sometime ago someone asked for the same field for DeepSeek. But it has reasoning_content key 😅 anyway, this addition doesn't interfere with OpenAI API spec and can be just added as an optional field to main Result structs set (ChatResult in this case). We'll just need to think about how to deal with different keys for the same thing. But maybe just add two optional fields.

[gtokman]

@nezhyborets also noticed the naming descrepencys across different APIs.

The node OpenAI client supports all of these from my testing:

- https://api.fireworks.ai/inference/v1
- https://api.together.xyz/v1
- https://api.perplexity.ai
- https://generativelanguage.googleapis.com/v1beta/openai
- https://api.x.ai/v1
- https://api.anthropic.com/v1
- https://api.mistral.ai/v1
- https://api.groq.com/openai/v1

[nezhyborets]

@gtokman you could use host and basePath fields of Configuration to build a path. As far as I can see it would work with any of the paths above. Let me know if it doesn't work for any

[gtokman]

@nezhyborets no I meant these providers all are OpenClient compatible. Although some of them ommit certain fields like the Gemini example above.