Skip to content

Commit 4485109

Browse files
committed
Add initial commit
0 parents  commit 4485109

12 files changed

+1032
-0
lines changed

.gitignore

+74
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Logs
2+
logs
3+
*.log
4+
npm-debug.log*
5+
yarn-debug.log*
6+
yarn-error.log*
7+
8+
# Runtime data
9+
pids
10+
*.pid
11+
*.seed
12+
*.pid.lock
13+
14+
# Directory for instrumented libs generated by jscoverage/JSCover
15+
lib-cov
16+
17+
# Coverage directory used by tools like istanbul
18+
coverage
19+
20+
# nyc test coverage
21+
.nyc_output
22+
23+
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
24+
.grunt
25+
26+
# Bower dependency directory (https://bower.io/)
27+
bower_components
28+
29+
# node-waf configuration
30+
.lock-wscript
31+
32+
# Compiled binary addons (http://nodejs.org/api/addons.html)
33+
build/Release
34+
35+
# Dependency directories
36+
node_modules/
37+
jspm_packages/
38+
39+
# Typescript v1 declaration files
40+
typings/
41+
42+
# Optional npm cache directory
43+
.npm
44+
45+
# Optional eslint cache
46+
.eslintcache
47+
48+
# Optional REPL history
49+
.node_repl_history
50+
51+
# Output of 'npm pack'
52+
*.tgz
53+
54+
# Yarn Integrity file
55+
.yarn-integrity
56+
57+
.coverage
58+
.vagrant
59+
.idea/
60+
*.pyc
61+
*~
62+
63+
.python-version
64+
*.egg-info
65+
.pytest_cache/
66+
67+
.ipfs
68+
69+
.idea
70+
.vscode
71+
72+
venv/
73+
config/settings/dev.py
74+
db.sqlite3

LICENSE

+674
Large diffs are not rendered by default.

README.md

+115
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
![Python 3.7](https://img.shields.io/badge/Python-3.7-blue.svg)
2+
3+
# VIDEO COMPRESSOR
4+
Script to reduce the **size of video files** using FFMPEG.
5+
6+
Idea of this script: https://coderunner.io/shrink-videos-with-ffmpeg-and-preserve-metadata/
7+
8+
9+
Which codecs are supported?
10+
--------------------------------------
11+
- **H.264**:
12+
- **CRF** (Constant Rate Factor). Basically translates as *"try to keep this quality overall"*, and will use more or less bits at different parts of the video, depending on the content. (the **bitrate* is variable**).
13+
- **Rest of video properties**. They are not modified.
14+
- **Videos with different codecs**:
15+
- They are copied to another folder without being modified
16+
17+
18+
How to use
19+
------------
20+
- Install **Python** (3.7 version recommended)
21+
- Install **FFmpeg** and add it to system PATH
22+
- Install **FFprobe** (*This is installed on ffmpeg installation by default*)
23+
- Look at script options: `python main.py -h`
24+
- Execute it: `python main.py [VIDEOS_FOLDER] [--ANOTHER_OPTIONS]`
25+
26+
27+
TIPS
28+
-----------
29+
You can create an isolated Python environment to install required libraries with virtualenv:
30+
- Create a virtualenv: `python -m venv [VENV_FOLDER]`
31+
- Activate virtualenv: `source [VENV_FOLDER]/bin/activate`
32+
33+
34+
F.A.Q
35+
------------
36+
37+
### What about original video metadata?
38+
39+
Original video metadata will be copied to the new modified video:
40+
- **Container metadata**. All the original container metadata is copied using ffmpeg `-map_metadata` option
41+
- **FILE dates**. Access and modification file dates.
42+
43+
### What happens if I have h.264 videos and another videos which use different codecs in the same folder?
44+
45+
Non-h.264 videos will be copied to the destination_folder/other_codecs by default **without being modified**
46+
47+
### What happens if in the middle of the process there is a failure with one video?
48+
49+
That video will be copied to the `failures` folder so that you can analyze why later
50+
51+
52+
More source information that helped to build this script
53+
---------------------------------------------------
54+
55+
### History
56+
57+
#### Timeline video codecs
58+
[![Timeline video codecs](/readme_images/codecs_history.jpg "Timeline video codecs")](https://www.slideshare.net/mohieddin.moradi/an-introduction-to-versatile-video-coding-vvc-for-uhd-hdr-and-360-video-135899487)
59+
<br/>
60+
[Source link](https://www.slideshare.net/mohieddin.moradi/an-introduction-to-versatile-video-coding-vvc-for-uhd-hdr-and-360-video-135899487)
61+
62+
[![MPEG and VCEG codecs history](/readme_images/mpeg_vceg_history.jpg "MPEG and VCEG codec history")](https://blog.wildix.com/understanding-video-codecs/)
63+
<br/>
64+
[Source link](https://blog.wildix.com/understanding-video-codecs/)
65+
66+
67+
#### Bitrate reduction
68+
![Bitrate reduction](/readme_images/bitrate_reduction.jpg "Bitrate reduction")
69+
70+
#### Comparison of video codecs and containers
71+
http://download.das-werkstatt.com/pb/mthk/info/video/comparison_video_codecs_containers.html
72+
73+
#### ISO vs IEC vs ITU (the Big Three international standards organizations)
74+
75+
[![ISO, IEC and ITU organizations](/readme_images/big_three.JPG "ISO, IEC and ITU organizations")](https://slideplayer.com/slide/4687304/)
76+
[Source link](https://slideplayer.com/slide/4687304/)
77+
78+
79+
### Video Quality
80+
- Different methods to compare video quality after modifying videos
81+
https://superuser.com/questions/338725/compare-two-video-files-to-find-out-which-has-best-quality
82+
83+
- Compare video quality with FFMPEG
84+
https://github.com/stoyanovgeorge/ffmpeg/wiki/How-to-Compare-Video
85+
86+
87+
### H.264
88+
89+
#### Tips about H.264
90+
https://trac.ffmpeg.org/wiki/Encode/H.264
91+
92+
93+
#### Bits per frame war -> CRF (Constant Rate Factor) vs CBR (Constant Bitrate)
94+
https://slhck.info/video/2017/02/24/crf-guide.html
95+
https://slhck.info/video/2017/03/01/rate-control.html
96+
97+
### VFR (Variable Frame Rate) and CFR (Constant Frame Rate)
98+
99+
#### Know video VFR (Variable Frame Rate)
100+
https://github.com/stoyanovgeorge/ffmpeg/wiki/Variable-Frame-Rate
101+
102+
Using ffmpeg: `ffmpeg -i [VIDEO] -vf vfrdet -f null -`
103+
104+
Result example:
105+
```
106+
[Parsed_vfrdet_0 @ 0x56518fa3f380] VFR:0.400005 (15185/22777) min: 1801 max: 3604)
107+
```
108+
109+
**A non-zero value for VFR indicates a VFR stream**. The first value in brackets (15185) is the number of frames with a duration different than the expected duration implied by the detected frame rate of the stream. The 2nd value (22777) is number of frames having the expected duration. The VFR value (0.400005) is the ratio of the first number to the sum of both.
110+
111+
If there were frames with variable delta, than it will also show min and max delta encountered.
112+
113+
114+
#### FFMPEG uses CFR (Constant Frame Rate) by default for MP4 output
115+
https://trac.ffmpeg.org/wiki/ChangingFrameRate

main.py

+169
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
#!/usr/bin/env python
2+
3+
# Source https://coderunner.io/shrink-videos-with-ffmpeg-and-preserve-metadata/
4+
5+
import traceback
6+
import os
7+
import subprocess
8+
import platform
9+
# shutil is consisting of high-level Python specific functions. shutil is on top of Python `os module`.
10+
# Thus, we can use the shutil module for high-level operations on files.
11+
# For example: Use it to copy files and metadata
12+
import shutil
13+
import argparse
14+
15+
import pprint
16+
import json
17+
18+
19+
def preserve_file_dates(source_file, destination_file):
20+
"""
21+
Preserve original FILE dates.
22+
"""
23+
24+
stat = os.stat(source_file)
25+
# Preserve access and modification date FILE attributes (EXIF are other dates)
26+
os.utime(destination_file, (stat.st_atime, stat.st_mtime))
27+
28+
29+
def get_video_metadata(video_path):
30+
31+
# run the ffprobe process, decode stdout into utf-8 & convert to JSON
32+
ffprobe_output = subprocess.check_output([FFPROBE_BIN, '-v', 'quiet',
33+
'-print_format', 'json',
34+
'-show_streams',
35+
video_path]).decode('utf-8')
36+
video_metadata = json.loads(ffprobe_output)
37+
38+
# DEBUG - Prints all the metadata available:
39+
# pp = pprint.PrettyPrinter(indent=2)
40+
# pp.pprint(video_metadata)
41+
42+
video_stream = next(
43+
(stream for stream in video_metadata['streams'] if stream['codec_type'] == 'video'), None)
44+
audio_stream = next(
45+
(stream for stream in video_metadata['streams'] if stream['codec_type'] == 'video'), None)
46+
47+
return {'video': video_stream,
48+
'audio': audio_stream}
49+
50+
51+
if __name__ == '__main__':
52+
"""
53+
Main operation of this script:
54+
1. NON-H.264 videos are copied to the other_codecs folder without being modified
55+
2. H.264 videos:
56+
2.1 Reduce video quality
57+
2.1.1 Use a fixed quality that the human eye can not detect
58+
2.2 Preserve FILE metadata (dates...)
59+
3. Invalid video files are copied to failures folder
60+
"""
61+
62+
# INPUT arguments
63+
###
64+
parser = argparse.ArgumentParser(description='Compress Video files size')
65+
# Source folder is mandatory
66+
parser.add_argument('source_folder',
67+
help='videos source folder')
68+
parser.add_argument('--destination_folder',
69+
help='videos destination folder. Default is `source_folder/results`')
70+
parser.add_argument('--failures_folder',
71+
help='videos destination folder. Default is `destination_folder/failures`')
72+
parser.add_argument('--crf',
73+
type=int,
74+
choices=range(0, 51),
75+
default=23,
76+
help='video crf between 1-51`. Default is 23')
77+
78+
args = parser.parse_args()
79+
80+
source_folder = args.source_folder
81+
destination_folder = args.destination_folder or f'{source_folder}/results'
82+
failures_folder = args.failures_folder or f'{destination_folder}/failures'
83+
other_codecs_folder = f'{destination_folder}/other_codecs'
84+
# Because ffmpeg needs a str for CRF
85+
crf = str(args.crf)
86+
#
87+
###
88+
89+
# Check that it is an allowed platform
90+
assert (platform.system().upper() in ['LINUX', 'WINDOWS']), 'OS not allowed'
91+
92+
if platform.system().upper() == 'LINUX':
93+
FFMPEG_BIN = 'ffmpeg'
94+
FFPROBE_BIN = 'ffprobe'
95+
elif platform.system().upper() == 'WINDOWS':
96+
FFMPEG_BIN = 'ffmpeg.exe'
97+
FFPROBE_BIN = 'ffprobe.exe'
98+
99+
# Destination folder. Create if does not exist
100+
if not os.path.exists(destination_folder):
101+
os.makedirs(destination_folder)
102+
103+
# Process videos
104+
with os.scandir(source_folder) as entries:
105+
for entry in entries:
106+
if entry.is_file():
107+
try:
108+
109+
print(entry.name)
110+
111+
video_source_path = f'{source_folder}/{entry.name}'
112+
video_destination_path = f'{destination_folder}/{entry.name}'
113+
114+
video_metadata = get_video_metadata(video_source_path)
115+
116+
# Only process videos with this codec (at this moment)
117+
if video_metadata['video']['codec_name'] == 'h264':
118+
print(f"Video format detected: {video_metadata['video']['codec_name']}")
119+
120+
# "copy_unknown" -> "", //if there are streams ffmpeg doesn't know about, still copy them (e.g some GoPro data stuff)
121+
# "map_metadata" -> "0", //copy over the global metadata from the first (only) input
122+
# "map" -> "0", //copy *all* streams found in the file, not just the best audio and video as is the default (e.g. including data)
123+
# "codec" -> "copy", //for all streams, default to just copying as it with no transcoding
124+
# "preset" -> "medium" //fmpeg speed preset to use
125+
#
126+
# "codec:v" -> "libx264", //specifically for the video stream, reencode to x264
127+
# "pix_fmt" -> "yuv420p", //default pix_fmt
128+
# "crf" -> "23" //default constant rate factor for quality. 0-52 where 18 is near visually lossless
129+
#
130+
# "codec:a" -> "libfdk_aac", //specifically for the audio stream, reencode to aac
131+
# "vbr" -> "4" //variable bit rate quality setting
132+
133+
# Use the same pix_fmt than the source video
134+
pix_fmt = video_metadata['video']['pix_fmt']
135+
136+
subprocess.call([FFMPEG_BIN, '-i', video_source_path,
137+
'-copy_unknown',
138+
'-map_metadata', '0',
139+
'-map', '0',
140+
'-codec', 'copy',
141+
'-codec:v', 'libx264',
142+
'-pix_fmt', pix_fmt,
143+
'-preset', 'slow',
144+
'-crf', crf, video_destination_path])
145+
146+
# Preserve file dates that are not in the video metadata. Example: modification_time
147+
preserve_file_dates(source_file=video_source_path,
148+
destination_file=video_destination_path)
149+
else:
150+
print(f"Non supported video format detected: {video_metadata['video']['codec_name']}")
151+
152+
# Create folder if it does not exist
153+
if not os.path.exists(other_codecs_folder):
154+
os.makedirs(other_codecs_folder)
155+
156+
video_other_codecs_path = f'{other_codecs_folder}/{entry.name}'
157+
# Copy files with other video formats
158+
shutil.copy2(video_source_path, video_other_codecs_path)
159+
except Exception as exception:
160+
# Create failures folder if it does not exist
161+
if not os.path.exists(failures_folder):
162+
os.makedirs(failures_folder)
163+
164+
video_failure_path = f'{failures_folder}/{entry.name}'
165+
# Copy files that have raised an exception to the failure folder
166+
shutil.copy2(video_source_path, video_failure_path)
167+
168+
# Show exception stack trace
169+
traceback.print_exc()

readme_images/big_three.JPG

70 KB
Loading

readme_images/bitrate_reduction.jpg

18.9 KB
Loading

readme_images/codecs_history.jpg

79.4 KB
Loading

readme_images/mpeg_vceg_history.jpg

58.5 KB
Loading
31.3 MB
Binary file not shown.
25 MB
Binary file not shown.
13.7 MB
Binary file not shown.

sample_videos/sample-mpeg4.avi

10.6 MB
Binary file not shown.

0 commit comments

Comments
 (0)