Skip to content

fix(template): guard MiniCPM-V/mPLUG-Owl3 video sampling against 0-fps metadata#9750

Open
he-yufeng wants to merge 1 commit into
modelscope:mainfrom
he-yufeng:fix/minicpmv-video-zero-fps
Open

fix(template): guard MiniCPM-V/mPLUG-Owl3 video sampling against 0-fps metadata#9750
he-yufeng wants to merge 1 commit into
modelscope:mainfrom
he-yufeng:fix/minicpmv-video-zero-fps

Conversation

@he-yufeng

Copy link
Copy Markdown
Contributor

What

load_video_minicpmv_mplug_owl3 picks frames with

sample_fps = round(vr.get_avg_fps() / 1)  # FPS
frame_idx = [i for i in range(0, len(vr), sample_fps)]

decord's get_avg_fps() returns 0.0 when the container has no / broken fps metadata (and a very low sub-1 fps rounds to 0 as well), so sample_fps becomes 0 and range(0, len(vr), 0) raises ValueError: range() arg 3 must not be zero before a single frame is read. The MiniCPM-V and mPLUG-Owl3 templates call this for any user-supplied video (swift/template/templates/minicpm.py, swift/template/templates/mplug.py), and there is no upstream fps check, so an ordinary video with missing fps metadata crashes encoding. The same file already treats fps == 0 as a real case elsewhere (_video_get_metadata_local uses fps if fps > 0 else 0).

Fix

Clamp sample_fps to at least 1 so a 0-fps video falls back to sampling every frame, which the existing max_num_frames uniform_sample then downsamples. Normal videos (fps >= 1) are unaffected.

Verification

decord/torch are not installable in my environment, so I reproduced the exact sampling logic standalone:

avg fps before after
25.0 4 frames 4 frames
1.0 4 frames 4 frames
0.4 ValueError: range() arg 3 must not be zero 4 frames
0.0 ValueError: range() arg 3 must not be zero 4 frames

The added test_load_video_minicpmv_handles_zero_fps mocks a decord VideoReader whose get_avg_fps() returns 0.0 and asserts load_video_minicpmv_mplug_owl3 returns frames instead of raising (fails before this change, passes after). It needs no real video or GPU.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request ensures that sample_fps is clamped to a minimum value of 1 in vision_utils.py, preventing a zero-step range error when processing videos with 0 FPS. A corresponding unit test, test_load_video_minicpmv_handles_zero_fps, has been added to verify this fix. There are no review comments, and I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

…s metadata

load_video_minicpmv_mplug_owl3 computes sample_fps = round(vr.get_avg_fps())
and then walks the frames with range(0, len(vr), sample_fps). decord's
get_avg_fps() returns 0.0 for a video whose container has no / broken fps
metadata (and a very low sub-1 fps rounds to 0 too), so sample_fps becomes 0
and range() raises "ValueError: range() arg 3 must not be zero" before any
frame is read. The MiniCPM-V and mPLUG-Owl3 templates reach this for any
user-supplied video and there is no upstream fps check; the same file already
treats fps == 0 as a real case in _video_get_metadata_local (fps if fps > 0
else 0).

Clamp sample_fps to at least 1 so a 0-fps video falls back to sampling every
frame, which the existing max_num_frames uniform_sample then downsamples.
Normal videos (fps >= 1) are unaffected.
@he-yufeng he-yufeng force-pushed the fix/minicpmv-video-zero-fps branch from ac80301 to 7d8775f Compare July 15, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant