Does PTS have to start at 0?

Question

I've seen a number of questions regarding video PTS values not starting at zero, or asking how to make them start at zero. I'm aware that using ffmpeg I can do something like ffmpeg -i <video> -vf="setpts=PTS-STARTPTS" <output> to fix this kind of thing

However it's my understanding that PTS values don't have to start at zero. For instance, if you join a live stream then odds are it has been going on for an hour and the PTS is already somewhere around 3600000+ but your video player faithfully displays everything just fine. Therefore I would expect there to be no problem if I intentionally created a video with a PTS value starting at, say, the current wall clock time.

I want to send a live stream using ffmpeg, but embed the current time into the stream. This can be used both for latency calculation while the stream is live, and later to determine when the stream was originally aired. From my understanding of PTS, something as simple as this should probably work:

ffmpeg -i video.flv -vf="setpts=RTCTIME" rtmp://<output>

When I try this, however, ffmpeg outputs the following:

frame=   93 fps= 20 q=-1.0 Lsize=    9434kB time=535020:39:58.70 bitrate=   0.0kbits/s speed=1.35e+11x

Note the extremely large value for "time", the bitrate (0.0kbits), and the speed (135000000000x!!!)

At first I thought the issue might be my timebase, so I tried the following:

ffmpeg -i video.flv -vf="settb=1/1K,setpts=RTCTIME/1K" rtmp://<output>

This puts everything in terms of milliseconds (1 PTS = 1 ms) but I had the same issue (massive time, zero bitrate, and massive speed)

Am I misunderstanding something about PTS? Is it not allowed to start at non-zero values? Or am I just doing something wrong?

Update

After reviewing @Gyan's answer, I formatted my command like so:

ffmpeg -re -i video.flv -vf="settb=1/1K, setpts=(RTCTIME-RTCSTART)/1K" -output_ts_offset $(date +%s.%N) rtmp://<output>

This way the PTS values would match up to "milliseconds since the stream started" and would be offset by the start time of the stream (theoretically making PTS = timestamp on the server)

This looked like it was encoding better:

frame=  590 fps=7.2 q=22.0 size=   25330kB time=00:01:21.71 bitrate=2539.5kbits/s dup=0 drop=1350 speed=   1x 

Bitrate was now correct, time was accurate, and speed was not outrageous. The frames per second was still a bit off, though (the source video is 24 fps but it's reporting 7.2 frames per second)

When I tried watching the stream from the other end, the video was out of sync with the audio and played at about double normal speed for a while, then the video froze and the audio continued without it

Furthermore, when I dumped the stream to a file (ffmpeg -i rtmp://<output> dump.mp4) and look at the PTS timestamps with ffprobe (ffprobe -show_entries packet=codec_type,pts dump.mp4 | grep "video" -B 1 -A 2) the timestamps didn't seem to show server time at all:

...
--
[PACKET]
codec_type=video
pts=131072
[/PACKET]
[PACKET]
codec_type=video
pts=130048
[/PACKET]
--
[PACKET]
codec_type=video
pts=129536
[/PACKET]
[PACKET]
codec_type=video
pts=130560
[/PACKET]
--
[PACKET]
codec_type=video
pts=131584
[/PACKET]

Is the problem just an incompatibility with RTMP?

Update 2

I've removed the video filter and I'm now encoding like so:

ffmpeg -re -i video.flv -output_ts_offset $(date +%s.%N) rtmp://<output>

This is encoding correctly:

frame=  910 fps= 23 q=25.0 size=   12027kB time=00:00:38.97 bitrate=2528.2kbits/s speed=0.981x 

In order to verify that the PTS values are correct, I'm dumping the output to a file like so:

ffmpeg -i rtmp://<output> -copyts -write_tmcd 0 dump.mp4

I tried saving it as dump.flv (since it's RTMP) however this threw the error:

[flv @ 0x5600f24b4620] Audio codec mp3 not compatible with flv

This is a bit weird since the video isn't mp3-encoded (it's speex) - but whatever.

While dumping this file the following error pops up repeatedly:

frame=    1 fps=0.0 q=0.0 size=       0kB time=00:00:09.21 bitrate=   0.0kbits/s dup=0 dr
43090023 frame duplication too large, skipping
43090027 frame duplication too large, skipping
    Last message repeated 3 times
43090031 frame duplication too large, skipping
    Last message repeated 3 times
43090035 frame duplication too large, skipping

Playing the resulting video in VLC plays an audio stream but displays no video. I then attempt to probe this video with ffprobe to look at the video PTS values:

ffprobe -show_entries packet=codec_type,pts dump.mp4 | grep "video" -B 1 -A 2

This returns only a single video frame whose PTS is not large like I would expect:

[PACKET]
codec_type=video
pts=1020
[/PACKET]

This has been a surprisingly difficult task

Wiredraw

It depends on the protocol/container. Every one is different. Rtmp, per specification should start at 0. But I know some encoders that don’t. – Pash

Answer 1

This puts everything in terms of milliseconds (1 PTS = 1 ms) but I had the same issue (massive time, zero bitrate, and massive speed)

This is just a side-effect of the stats calculation which doesn't measure time with reference to the observed starting PTS. The bitrate (bits/second) and speed (output duration / real time) calculations are simply casualties of this design.

---

If the container allows it, you can start at non-zero PTS. The advisable way to do this is using the option -output_ts_offset N where N is in seconds.

Changing the timestamps prior to encoding is risky since ffmpeg's framerate conversion method uses timestamps to check sync drift and make decisions about dropping or duplicating frames. Especially important for CFR output. Encoders' rate-control flow may also go haywire.

Quintuplicate

This definitely worked a lot better, but I'm still having some awkward issues. I'll update my question with the command I used and the results – Wiredraw

a) the setpts filter sets the PTS at the time of evaluation. Filters like setpts process frames very fast, are limited by input flow, typically that is fast as well. realtime filter needs to be used beforehand. b) audio flow needs to be controlled and timestamps need to be correspondingly altered. c) because of how the timestamps are set, ffmpeg will drop frames to match the output framerate d) at the output, ffmpeg will discard the starting offset, use copyts to avoid that. – Quintuplicate

Try ffmpeg -re -i video.flv -output_ts_offset $(date +%s.%N) rtmp://<output> – Quintuplicate

I already had the -re parameter (I accidentally neglected to type it when posting my update). Although removing the setpts and settb video filters did fix the frame rate and playback. But there is still the issue that when I ran ffprobe the output_ts_offset did not seem to have any effect. The PTS values were reported to start at 0. Is this just a quirk of ffprobe? If I manually parse the file would I expect to find non-0 PTS values? – Wiredraw

After streaming to the RTMP endpoint, I have a second ffmpeg process dumping the stream like so: ffmpeg -i rtmp://<output> dump.mp4. I then read this using ffprobe like so: ffprobe -show_entries packet=codec_type,pts dump.mp4 | grep "video" -B 1 -A 2 – Wiredraw

You need to add -copyts when dumping since ffmpeg will strip starting offset. – Quintuplicate

So I think output_ts_offset is doing something, but I'm running into constant issues. Adding -copyts caused ffmpeg to stop dumping video properly - I just get an audio stream with a single frame. Although while dumping I get repeated error message of the form "43069023 frame duplication too large, skipping". I think that's an offset PTS value. I'm also getting a timecode stream that seems to prevent ffprobe from working. Using -write_tmcd 0 I can get rid of this - but then ffprobe shows the ONE frame with a PTS of 1000 or so (depending on how long I recorded) – Wiredraw

Don't transcode during dump. Just copy. ffmpeg -copyts -i rtmp://<output> -c copy dump.flv – Quintuplicate

Definitely an improvement. I didn't realize that without the -c parameter ffmpeg would automatically transcode. The video now streams properly, can be viewed in a browser while live, and when dumped to dump.flv the PTS values don't start at zero. There's only two issues (one of which I can ignore). 1. There's a single frame with a very low PTS before the real frames start (in my case the first frame had a PTS of 83, followed by all of the others. Not a real issue) 2. The PTS values were in the range of 1965472779. I'd like them to be 1530973872000 (milliseconds since epoch) – Wiredraw

FLV stores 32-bit timestamps. So stored TS is input epoch ms mod (2^32-1) – Quintuplicate

Just did the math and you're exactly right. I can work with this. – Wiredraw