I have a stream with caption data in more than one field, but only captions from one field are being returned
You may want to confirm the SEI NAL units are constructed according to the CEA-608 or CEA-708 specification. Specifically:
I am getting the wrong times for only some captions, other captions are correct
If your content includes B-frames, there is an existing issue to fix this problem. We recommend that you ensure that all the segments in your source start with keyframes, to allow proper caption timing.