Hello Everyone,
A new version of my latest project Video Model Studio (VMS) has been pushed to my GitHub (and Hugging Face for easier duplication and deployment).
This new release adds automatic FPS detection and injection to the prompt!
For those who don't know, VMS is my current active project, still in early development phase: a web UI to train AI video models! Here is a video demo/tutorial:
https://x.com/flngr/status/1892349991890341985
What is the frame rate and why it matters
The frame rate, or FPS (Frame Per Second) is the number of single images (frames) per second of footage in a video stream.
It is an important factor when training or fine-tuning an AI video model, if you ever saw AI videos you must have already observed that some of them looks a bit "slow-motion" like.
This can be caused by training datasets extracted from stock video footage platforms, they might contain heavily-edited videos made for advertising, in slow-motion or 60 FPS. In itself this is fine, but this also requires each video clip to be properly tagged as such.
In some other cases, this might be due to how video frames are assembled at inference time to produce the final video: some models don't produce enough FPS, so to inflate the numbers it is common to see them being played back at a slower rate to increase the duration of footage, sometimes in combination with an AI frame interpolator as well.
How FPS is handled in Video Model Studio
I have pushed some changes to:
Automatically detect the FPS of each individual clip using FFmpeg (ffprobe)
Automatically inject this FPS into to the prompt (done after the global prompt prefix)
The generated prompt will contain additional descriptors like: "24 FPS, ", "30 FPS, ".
This feature works pretty well and doesn't cost many tokens, so it is currently always enabled.