I just had a discussion with AI about this. I asked scenarios where the original was shot at 24, 30 and 60 fps and then reencoded by TMZ.
I added the bold to some parts I believe are relevant:
The whole discussion:
Blockquote
Question about video recording with cellphones, video-audio synchronization, frame rates, and physics. I recorded a man shooting a rifle from about 260 feet away. Or maybe it was 270, I canāt remember exactly. I sent this video to TMZ, and they uploaded it to their website with a 720p resolution. Then I lost my original video. Then I downloaded the video from their website to my computer and watched it. I notice a difference between the case ejection and the sound of the shot. Why is that? Then, I imported the video and built an animated and estabilized gif file that shows the case being fired along with graphical dots I added to the image to mark the frames between the time of case ejection and the time the sound is heard. Temperature on the day was 93 fahrenheit, and the altitude was of 408 meters ASL. My objetive is to check for manipulation by TMZ, I mean, I believe I saw a case being ejected, but I canāt remember exactly, and I lost the original video from my phone. On TMZ video I can see a case, but was it really there? So, I want to check if my timing between case ejection and shot is heard matches the distance the video was taken from (260-270 feet away), but Iām doing it using video frames as a unit of measurement, also, so keep that in mind. Iām considering 29,97 video frames per second. could you check my math and approach? Speed of sound / Frames per second = Sound distance per frame. 1150 / 29.97 = 38.37 Frames between visual and audio report * sound distance per frame = Distance between gun and camera. 12 * 38.37 = 460 feet.
ChatGPT:
ChatGPT
Letās break down your calculations and approach to ensure everything aligns with the physics of sound propagation and video frame rates.
1. Speed of Sound Calculation:
- The speed of sound varies with temperature. The standard speed of sound at sea level and 20Ā°C (68Ā°F) is approximately 343 m/s, or 1125.33 feet per second (fps).
- At 93Ā°F, the speed of sound increases slightly. The approximate speed of sound at 93Ā°F is around 1150 fps, which matches the value youāre using.
2. Video Frame Rate:
- Youāre using a frame rate of 29.97 frames per second (fps), which is standard for many video recordings.
3. Distance per Frame Calculation:
- The distance sound travels in one video frame is calculated by dividing the speed of sound by the frame rate: Sound Distance per Frame=1150 fps29.97 frames per secondā38.37 feet per frame\text{Sound Distance per Frame} = \frac{1150 , \text{fps}}{29.97 , \text{frames per second}} \approx 38.37 , \text{feet per frame}Sound Distance per Frame=29.97frames per second1150fpsāā38.37feet per frame
This part of your calculation is correct.
4. Frames Between Visual and Audio Report:
- You measured 12 frames between the visual cue (case ejection) and the sound being heard.
5. Calculating the Distance:
- Multiply the number of frames by the distance per frame: Distance=12 framesĆ38.37 feet per frameā460.44 feet\text{Distance} = 12 , \text{frames} \times 38.37 , \text{feet per frame} \approx 460.44 , \text{feet}Distance=12framesĆ38.37feet per frameā460.44feet
This calculation also appears correct, assuming all the data and assumptions are accurate.
6. Discrepancy Analysis:
- Original Distance vs. Calculated Distance: You mentioned that the original distance was between 260 to 270 feet, but your calculated distance is around 460 feet. This significant difference could indicate a few things:
- Audio-Video Sync Issue: The video you downloaded might have suffered from audio desynchronization, which can happen during the compression and upload process.
- Misjudgment of Distance: The actual distance might have been greater than you remembered.
- Manipulation: While unlikely, if the video were tampered with, it could affect the timing, but such tampering usually affects the visual content rather than the synchronization.
7. Conclusion:
- Based on your calculations, the distance calculated from the timing discrepancy is much larger than what you expected. This suggests that the audio and video might not be perfectly synchronized in the downloaded version from TMZ.
- The fact that the calculated distance is almost double the expected distance might indicate that the audio is delayed in the video, possibly due to the way the video was processed or compressed after being uploaded to TMZ.
If your original video had the correct synchronization, and you are certain about the original distance (260-270 feet), the difference suggests that the downloaded video might not be perfectly in sync. However, itās unlikely that the case ejection itself was manipulatedāitās more likely an issue with the audio timing.
You:
How would you answer change if the original video was recorded at 24 fps or 30 or 60 fps and then sent to TMZ ?
ChatGPT:
ChatGPT
If the original video was recorded at a different frame rate (24 fps, 30 fps, or 60 fps) before being sent to TMZ, the potential for audio-video synchronization issues might increase, especially during the conversion and compression process that happens when the video is uploaded to a platform like TMZ.
Letās explore how the change in original frame rate would affect your calculations:
1. Original Video Frame Rate:
- 24 fps: Common for cinematic recordings.
- 30 fps: Standard for many consumer-grade cameras.
- 60 fps: Used for high-definition or slow-motion recording.
2. Conversion Impact:
- When a video with a certain frame rate (e.g., 24 fps) is uploaded to a platform like TMZ, it might be converted to a different frame rate (e.g., 29.97 fps) to match web standards. This conversion can sometimes cause audio desynchronization, especially if the platform compresses the video or changes its resolution.
3. Speed of Sound Per Frame:
- 24 fps: Sound Distance per Frame=1150 fps24 frames per secondā47.92 feet per frame\text{Sound Distance per Frame} = \frac{1150 , \text{fps}}{24 , \text{frames per second}} \approx 47.92 , \text{feet per frame}Sound Distance per Frame=24frames per second1150fpsāā47.92feet per frame
- 30 fps: Sound Distance per Frame=1150 fps30 frames per secondā38.33 feet per frame\text{Sound Distance per Frame} = \frac{1150 , \text{fps}}{30 , \text{frames per second}} \approx 38.33 , \text{feet per frame}Sound Distance per Frame=30frames per second1150fpsāā38.33feet per frame
- 60 fps: Sound Distance per Frame=1150 fps60 frames per secondā19.17 feet per frame\text{Sound Distance per Frame} = \frac{1150 , \text{fps}}{60 , \text{frames per second}} \approx 19.17 , \text{feet per frame}Sound Distance per Frame=60frames per second1150fpsāā19.17feet per frame
4. Frames Between Visual and Audio Report:
- If you counted 12 frames between the visual of the case ejection and the sound of the shot in a 29.97 fps video after downloading from TMZ, but the original video was recorded at a different frame rate, this could lead to discrepancies.
5. Distance Calculation Based on Original Frame Rates:
- Original at 24 fps: Distance (using 12 frames from 29.97 fps)=12Ć47.92 feet per frameā575.04 feet\text{Distance (using 12 frames from 29.97 fps)} = 12 \times 47.92 , \text{feet per frame} \approx 575.04 , \text{feet}Distance (using 12 frames from 29.97 fps)=12Ć47.92feet per frameā575.04feet
- Original at 30 fps: Distance (using 12 frames from 29.97 fps)=12Ć38.33 feet per frameā460.00 feet\text{Distance (using 12 frames from 29.97 fps)} = 12 \times 38.33 , \text{feet per frame} \approx 460.00 , \text{feet}Distance (using 12 frames from 29.97 fps)=12Ć38.33feet per frameā460.00feet
- Original at 60 fps: Distance (using 12 frames from 29.97 fps)=12Ć19.17 feet per frameā230.04 feet\text{Distance (using 12 frames from 29.97 fps)} = 12 \times 19.17 , \text{feet per frame} \approx 230.04 , \text{feet}Distance (using 12 frames from 29.97 fps)=12Ć19.17feet per frameā230.04feet
6. Interpretation:
- 24 fps Original: The calculated distance would be 575 feet, which is even more off than the 460 feet calculation with 29.97 fps. This suggests a more significant desynchronization, possibly due to frame rate conversion issues.
- 30 fps Original: The calculation remains similar to the one you did, around 460 feet. This suggests a minor issue, likely just a small desynchronization during the upload process.
- 60 fps Original: The calculated distance of 230 feet is actually closer to the expected 260-270 feet, which might suggest that the original recording at 60 fps could be closer to reality. However, this also means the video might have been compressed in a way that kept better synchronization but at a different frame rate.
7. Conclusion:
- 24 fps: If your original was at 24 fps, the significant discrepancy suggests that the frame rate conversion during upload may have caused desynchronization.
- 30 fps: If the original was 30 fps, the results are similar to your initial calculation, indicating a minor sync issue.
- 60 fps: If the original was 60 fps, the distance calculation suggests that this might be the most accurate representation of the actual distance, though with potential for better synchronization.
In general, discrepancies due to frame rate conversion could result in audio-video desynchronization, leading to inaccuracies in your distance calculation. If the original was at a higher frame rate (e.g., 60 fps), it might be closer to the actual distance, but if converted to a lower frame rate, that could introduce errors in synchronization.
4o
You:
So far, this is what Iām having trouble with, and is leading me to believe that the video might have been manipulated. It is out by 2 frames for when the audio report is first detected. Opening the audio in Audacity and slowing it down helped find the moment. This does change the distance calculation. Speed of sound / Frames per second = Sound distance per frame. 1150 / 29.97 = 38.37 Frames between visual and audio report * sound distance per frame = Distance between gun and camera. 12 * 38.37 = 460 feet. The actual distance between the shooter and camera is about 260 feet or 7 frames. For some reason the time between the visual and audio report is out by 5 frames, 0.17 seconds or 200 feet. It is not an echo.
ChatGPT:
ChatGPT
Your observation of the discrepancy between the calculated distance (460 feet) and the actual distance (260 feet) suggests that there might indeed be an issue with the synchronization between the video and audio in the TMZ version. Given that youāve analyzed the audio in Audacity and found the discrepancy of 2 frames, letās dive into what this could mean.
1. Frame Rate and Sound Distance:
- The calculation youāve used is correct: Sound Distance per Frame=1150 fps29.97 fpsā38.37 feet per frame\text{Sound Distance per Frame} = \frac{1150 , \text{fps}}{29.97 , \text{fps}} \approx 38.37 , \text{feet per frame}Sound Distance per Frame=29.97fps1150fpsāā38.37feet per frame
- If the time between the visual cue (case ejection) and the audio report is out by 5 frames, this translates to a discrepancy of: 5 framesĆ38.37 feet per frame=191.85 feet5 , \text{frames} \times 38.37 , \text{feet per frame} = 191.85 , \text{feet}5framesĆ38.37feet per frame=191.85feet which is close to the 200 feet discrepancy you mentioned.
2. Expected Distance Calculation:
- Given the actual distance of 260 feet: 260 feet38.37 feet per frameā6.77 frames\frac{260 , \text{feet}}{38.37 , \text{feet per frame}} \approx 6.77 , \text{frames}38.37feet per frame260feetāā6.77frames
- Rounding up, this suggests that the delay should be about 7 frames, which aligns with your initial expectation.
3. Discrepancy Analysis:
- 2-Frame Delay: You noticed a 2-frame delay in the TMZ video when analyzing the audio in Audacity. At 29.97 fps, a 2-frame delay corresponds to: 2 framesĆ38.37 feet per frame=76.74 feet2 , \text{frames} \times 38.37 , \text{feet per frame} = 76.74 , \text{feet}2framesĆ38.37feet per frame=76.74feet
- This suggests that thereās an additional 76-77 feet of āextra distanceā that wasnāt in the original video, leading to the total discrepancy of about 200 feet.
4. Possible Causes:
- Audio Desynchronization: The most likely cause is that the video was desynchronized during either the upload or compression process by TMZ. This could happen due to how the video was encoded or processed after being uploaded. The audio could have been delayed by 2 frames relative to the video.
- Frame Rate Conversion: If your original video was recorded at a different frame rate (e.g., 24 fps or 30 fps) and then converted to 29.97 fps, this could introduce slight timing discrepancies, especially if the audio and video tracks werenāt perfectly synchronized during the conversion.
- Compression Artifacts: Online platforms like TMZ often compress videos to save bandwidth, which can sometimes introduce slight timing errors, especially if the audio and video are compressed separately and then recombined.
5. Conclusion:
- No Echo: Since youāve ruled out an echo, the discrepancy is likely due to an audio delay introduced during the video processing and uploading by TMZ.
- 2-Frame Delay: The 2-frame delay you found in Audacity seems to account for an additional 76-77 feet, leading to a total perceived distance that is approximately 200 feet off. This aligns with your observation that the distance is out by about 200 feet.
Given this analysis, the most plausible explanation is that the video was slightly desynchronized when TMZ processed and uploaded it, leading to the 2-frame audio delay. This doesnāt necessarily indicate deliberate manipulation, but rather a common issue with video processing and compression, especially when dealing with different frame rates or when videos are re-encoded by online platforms.
You:
What if the original video was also shot on 29,97 fps? would the online video still be subject to lose 2, 5 or 7 frames?
ChatGPT:
ChatGPT
If the original video was also shot at 29.97 fps, theoretically, the video and audio should remain in sync when uploaded to a platform like TMZ, provided the file is handled properly. However, there are still several factors that could cause a loss or misalignment of frames, even if the original video and the uploaded version share the same frame rate.
1. Potential Causes for Frame Loss or Misalignment:
- Video Compression: When videos are uploaded to online platforms, they are often compressed to reduce file size. Compression algorithms can sometimes introduce slight delays between audio and video tracks, especially if the audio and video streams are processed separately.
- Encoding and Re-encoding: Even if the video is at the correct frame rate, the process of encoding or re-encoding (which happens when the video is uploaded and processed by the platform) can sometimes result in small timing issues. This can lead to desynchronization of the audio and video tracks by a few frames.
- Network Issues During Upload: If there were any interruptions or delays during the upload process, it might affect the final video file. Although this is less common, it could lead to a scenario where a frame or two might be lost or mistimed.
- Browser Playback Issues: If you are viewing the video in a browser, playback issues such as buffering or dropped frames could also cause a perceived desynchronization, though this would usually only happen during playback and not be present in the file itself.
- Platform Processing: Platforms like TMZ may apply additional processing to videos to optimize them for streaming, which could cause minor issues with synchronization. Even if the original and uploaded videos have the same frame rate, this processing could lead to the loss or misalignment of frames.
2. Impact of Frame Rate Matching:
- Minimal Impact: If both the original video and the online version are at 29.97 fps, there should be minimal impact on synchronization. Any difference would likely be very slightājust a frame or twoāif the processing or compression introduced minor timing errors.
- Potential for Minor Desync: Despite the same frame rate, the desynchronization you observed (2 frames) could still occur if the audio and video streams were slightly misaligned during processing or if there were slight issues with encoding. This would explain why the sound of the shot is delayed by just 2 frames, resulting in a discrepancy in your distance calculations.
3. Conclusion:
- No Major Frame Loss Expected: If both the original and the uploaded video are at 29.97 fps, major frame loss (like 5 or 7 frames) is unlikely. However, minor desynchronization (1-2 frames) can still occur due to the factors mentioned above.
- Your Observation of 2-Frame Delay: The 2-frame delay you observed in the online video, despite the same frame rate, could be attributed to slight misalignments introduced during compression, processing, or uploading, even though both the original and the TMZ video were recorded at the same frame rate.
In summary, while the same frame rate should theoretically prevent major synchronization issues, minor desynchronization is still possible due to the inherent challenges of video processing and compression, even at 29.97 fps. The 2-frame discrepancy you noticed is within the realm of what could occur due to these factors.
), and we usually record the āstagesāā¦ By the end of the match videos of us shooting are all over everyoneās phones, so we create a group chat on Whatsapp and send the videosā¦ Some videos do end up with misaligned audioā¦ like sufficient to be able to tell without going frame by frameā¦ and it doesnāt happen on all videosā¦ Maybe someoneās phone is set at a different characteristic, codec, I donāt know, But when we watch these videos on the phone that originally recorded the video, the audio is aligned with video. So , from my experience, compression of these messaging apps can make a difference.
