A queue of videos and a multi-core server suggests running several at once. It usually makes everything slower, because the assumption underneath is wrong.
One job already uses the machine
FFmpeg's encoders are multi-threaded. A single encode will occupy every available core.
So a second job does not add capacity: it divides the same capacity. Two jobs each run at roughly half speed, the total time is about the same, and the machine is now unresponsive for everything else including the website.
That last part is the real cost on a server doing anything besides encoding.
When parallelism does help
Many small files. Starting a process, reading the file and writing the output cost time that is not encoding. On clips of a few seconds that overhead dominates, and running several genuinely helps.
Operations that do not encode. Remuxing, extracting audio, generating thumbnails. These are limited by storage rather than by the processor, so several can proceed at once. Remuxing and stream copy explains which operations those are.
Many more cores than one encode can use. Encoder scaling flattens out past a point, so a large machine may genuinely have capacity for two or three jobs.
Measure one job first
time ffmpeg -i input.mp4 -c:v libx264 -crf 23 -preset medium out.mp4
Watch the processor while it runs. If one job already sits near full utilisation, parallelism will not help. If it hovers well below, there is room.
Without that number the rest of this page is guesswork. Performance optimisation explains making the single job faster, which is usually the better lever.
Limit threads per job
If you do run several, stop them fighting:
ffmpeg -threads 2 -i input.mp4 -c:v libx264 -crf 23 out.mp4
Two jobs at two threads on a four-core machine is a deliberate arrangement. Two jobs both trying to use four cores is contention, and contention costs more than it appears to because of the switching involved.
Leave the machine usable
nice -n 19 ionice -c3 ffmpeg -i input.mp4 ... out.mp4
This tells the system that encoding may have whatever is spare and must yield to everything else.
The encode takes slightly longer and the website keeps responding, which on a server doing both is the correct trade, and it is a one-word change.
A queue instead of a fan-out
The workable pattern is a small fixed number of workers pulling from a list:
find ./in -name '*.mov' | xargs -P 2 -I{} sh -c \
'nice -n 19 ffmpeg -threads 2 -i "{}" -c:v libx264 -crf 23 "./out/$(basename {} .mov).mp4"'
-P 2 is the concurrency limit. It is the number to tune, starting at two and only increasing if measurement shows headroom.
For anything beyond a one-off run, a real queue is worth having: it survives reboots, retries failures and gives you a record of what was processed. Building a batch video processing pipeline goes over that, and running an application as a systemd service walks through keeping the worker alive.
Memory, not just processor
Each job holds frames in memory, and high-resolution sources use a great deal.
Several concurrent jobs on a machine with modest memory will exhaust it, and the system then starts terminating processes: frequently not the one at fault. Bound each job's memory as well as its threads, particularly when the input is not yours. There is more on the limits in processing untrusted video uploads safely.
On shared hosting
None of this applies, because the account's processing allowance is small and shared. One encode is already at the limit, and running several is the fastest way to have the account restricted.
What you can and cannot do with FFmpeg on shared hosting walks through the boundary.
Watch the machine while a batch runs
Deciding the concurrency from measurement means watching the right figures during a real run rather than after it.
vmstat 2 10 uptime
Three columns answer it. High r means processes are queued and waiting for the processor: too many jobs. Non-zero si and so mean the machine is swapping, which is worse than any slowdown and means memory is the constraint in place of the processor. And the load figure divided by the core count says whether work is clearing or accumulating.
Increase concurrency only while all three stay comfortable. The point at which any of them turns is the limit, and it arrives sooner than the core count suggests. Understanding load average deals with reading them properly.
Storage is frequently the real limit
Several concurrent encodes read and write large files at once, and on shared or network storage that saturates before the processor does.
iostat -x 2 5
A device near full utilisation with a growing queue means adding jobs makes everything slower rather than faster, and the symptom is processors that look underused, which reads as headroom and is not.
Writing the temporary file and the output to different devices helps where that is possible. Where it is not, the concurrency limit is set by the disk regardless of how many cores exist.
Separate the cheap jobs from the expensive ones
Treating every job identically wastes the distinction that matters most.
Remuxing, thumbnail extraction and metadata edits are storage-bound and finish in seconds; several can run at once without contention. Encodes are processor-bound and should not.
Two queues with different concurrency limits. A wide one for copy operations and a narrow one for encodes, is a small amount of extra structure and it stops a queue of quick jobs waiting behind one long encode.
That waiting is the usual complaint about a pipeline: a thumbnail that should take a second appears twenty minutes later. Building a batch pipeline sets out the queue structure.
Bound each job so one cannot take the machine
Concurrency limits control how many jobs run. They do not control what one job consumes.
systemd-run --scope -p MemoryMax=2G -p CPUQuota=100% \ nice -n 19 ffmpeg -nostdin -i input.mp4 ... out.mp4
That caps a single job's memory and processor share regardless of what the input turns out to be, which matters when the input came from outside, since a crafted file can consume far more than a typical one.
Without a bound, one unusual upload exhausts the machine's memory and the system begins terminating processes, frequently not the one at fault. Processing untrusted video uploads safely goes into the rest of the limits.