Model Troubleshooting
FastGPT Self-Hosting Model Troubleshooting
(1) How to check model availability issues
- For a self-hosted model, confirm that its service is running.
- Send a
curlrequest directly to the upstream model API. Test this for both cloud and self-hosted models. - Send the same type of request through OneAPI to check the gateway connection.
- Test the model in FastGPT.
Use the following curl examples, substituting your service URL and credentials:
curl https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "gpt-4o",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "Hello!"
}
]
}'curl https://api.openai.com/v1/embeddings \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input": "The food was delicious and the waiter...",
"model": "text-embedding-ada-002",
"encoding_format": "float"
}'curl --location --request POST 'https://xxxx.com/api/v1/rerank' \
--header 'Authorization: Bearer {{ACCESS_TOKEN}}' \
--header 'Content-Type: application/json' \
--data-raw '{
"model": "bge-rerank-m3",
"query": "Who is the director",
"documents": [
"Who are you?\nI am the assistant for the film Suzume"
]
}'curl https://api.openai.com/v1/audio/speech \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "tts-1",
"input": "The quick brown fox jumped over the lazy dog.",
"voice": "alloy"
}' \
--output speech.mp3curl https://api.openai.com/v1/audio/transcriptions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: multipart/form-data" \
-F file="@/path/to/file/audio.mp3" \
-F model="gpt-transcribe"(2) Error - Model response is empty/Model error
OneAPI can close a streaming request before returning any content.
Starting with version 4.8.10, the error log includes the request body sent to the model. Copy that body and send it to OneAPI with curl to reproduce the failure. Testing with stream=false can expose an error that the streaming response hides.
Check these possible causes:
- The model provider's content controls rejected the request.
- The model rejects optional request parameters. Test with
messagesand the required parameters first. - Parameter values exceed the model's supported range: some models reject a
temperatureof 0 or values with two decimal places;max_tokensor the context length may also exceed the limit. - The deployed model service has a problem or lacks compatible streaming support.
Use the request body from your error log in a test like this:
curl --location --request POST 'https://api.openai.com/v1/chat/completions' \
--header 'Authorization: Bearer sk-xxxx' \
--header 'Content-Type: application/json' \
--data-raw '{
"model": "xxx",
"temperature": 0.01,
"max_tokens": 1000,
"stream": true,
"messages": [
{
"role": "user",
"content": "You are hungry"
}
]
}'Testing tool calling support
Both the model provider and OneAPI must support tool calling. Test the full two-round exchange in streaming mode.
1. Request a tool call
curl --location --request POST 'https://oneapi.xxx/v1/chat/completions' \
--header 'Authorization: Bearer sk-xxxx' \
--header 'Content-Type: application/json' \
--data-raw '{
"model": "gpt-5",
"temperature": 0.01,
"max_tokens": 8000,
"stream": true,
"messages": [
{
"role": "user",
"content": "What time is it?"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "hCVbIY",
"description": "Get the current time in the user time zone.",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
}
}
],
"tool_choice": "auto"
}'2. Inspect the response
A successful response includes tool_calls. In a streaming response, inspect choices[].delta.tool_calls and collect the call ID, function name, and argument fragments. For example, the relevant part of a chunk looks like this:
{
"choices": [
{
"index": 0,
"delta": {
"tool_calls": [
{
"index": 0,
"id": "call_0n24eiFk8OUyIyrdEbLdirU7",
"type": "function",
"function": {
"name": "hCVbIY",
"arguments": "{}"
}
}
]
}
}
]
}3. Return the tool result
Send the assistant's tool call and the corresponding tool result in a second request. Use the call ID and function name from your actual first response in both messages. The example below uses kDia9S19c4RO as the call ID and a sample time as the tool result. The model should then produce an answer using that result.
curl --location --request POST 'https://oneapi.xxxx/v1/chat/completions' \
--header 'Authorization: Bearer sk-xxx' \
--header 'Content-Type: application/json' \
--data-raw '{
"model": "gpt-5",
"temperature": 0.01,
"max_tokens": 8000,
"stream": true,
"messages": [
{
"role": "user",
"content": "What time is it?"
},
{
"role": "assistant",
"tool_calls": [
{
"id": "kDia9S19c4RO",
"type": "function",
"function": {
"name": "hCVbIY",
"arguments": "{}"
}
}
]
},
{
"tool_call_id": "kDia9S19c4RO",
"role": "tool",
"name": "hCVbIY",
"content": "{\n \"time\": \"2024-09-14 22:59:21 Sunday\"\n}"
}
],
"tools": [
{
"type": "function",
"function": {
"name": "hCVbIY",
"description": "Get the current time in the user time zone.",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
}
}
],
"tool_choice": "auto"
}'Vector search scores greater than 1
Scores above 1 indicate that the embedding model produces unnormalized vectors. FastGPT currently supports normalized embedding models; use a model that normalizes its vectors.
(3) "Current group upstream load is saturated, please try again later"
If you encounter this error (e.g. request id:xxx) in the logs or response, this is typically a OneAPI channel issue. Try switching to a different model or a different relay provider.
(4) "Connection Error" in logs when using the API
Most likely the API key is pointing to OpenAI's endpoint, but the server is deployed in mainland China and can't reach overseas endpoints. Use a relay service or reverse proxy to resolve the connectivity issue.
(5) Enable image indexing reports 400
You need to correctly configure the OCR model in Admin -> System Configuration.
Last updated