fix: allow separate input/output budgets for T5 in context check

[Chessing234]

Summary

Fixes #1711.

FastChat-T5 supports up to 2K encoder tokens plus 2K decoder tokens, but the OpenAI API server treated context as a single shared budget (context_len - prompt_tokens).

Root cause

check_length always used the causal-LM formula, so a 1790-token prompt with max_tokens=512 was rejected as 2302 > 2048 even though T5 can encode 1790 tokens and still generate 512 more.

Fix

For T5 models, validate prompt length against context_len and cap completion tokens independently, matching encoder-decoder behavior in inference.py.

Test plan

• Call the API with fastchat-t5-3b-v1.0, ~1790 prompt tokens, and max_tokens=512; request should succeed
• Confirm causal models still reject prompts that leave no room for completion

Made with Cursor