Robocopy features a lot of parameters. While some are well known and used, there is maybe the most underrated parameter: /MT for multithreading. I have checked how this parameter impacts the speed of your copy jobs.
Update: Here is a new article about the impact of more or less threads.
Most Windows admins should know robocopy, the built-in copy and mirror tool from microsoft. This tool comes with a lot of parameters, some are more known and some might not be that present to all users.
For this test, I prepared the following scenarios:
- A large single file (20GB or 21.474.836.480 byte)
- 3016 Images with 7-8 MB each (~20GB or 21.473.798.962 byte)
- 73.488 text files with 400-600 KB each (~20GB or 21.486.178.224 byte)
All files are copied via network. So depending on your network and storage, the result can vary, but the general idea and improvements should apply to your setup too.
I copied the scenarios ten times without the multi threading parameter and ten times with /MT. The other parameters were left to default (Single thread: /DCOPY:DA /COPY:DAT /R:1000000 /W:30. Multi thread: /DCOPY:DA /COPY:DAT /MT:8 /R:1000000 /W:30)
Large Single File
The large single file copies pretty fast (compared to the other files):
The average is below 500 seconds (499,8) with a maximum of 612 seconds and a minimum of 450 seconds.
I would expect multi threading to have little to no impact to the speed in this scenario.
But I was wrong:
The speed nearly doubled: The time required dropped to 255,7 seconds average with a maximum of 289 seconds and a minimum of 224 seconds.
Image Files
The copy job for the image files took significantly longer than the single file: The average was 841 seconds with 770 seconds the fastest and 946 seconds the slowest.
I would assume that multi threading helps here a lot as we have several files to copy. But the impact was not as good as with the single file: The average dropped to 489 seconds, which is 58% of the singlethreaded time (Compare: The single file multithread job took 51%).
Text Files
Copying so many small files takes awfully long compared to the other jobs: The average was 4860 seconds or nine times longer than the single file job.
But here is the greatest time saving possible: Multi thread took only 849 seconds or 17% of the original time.
Summary
From my test and my experience using the /MT parameter won’t hurt, but can easily speed up your jobs. So whenever possible, use it or let me know if you know a scenario, where multi threading is a bad idea to use.
Update: Here is a new article about the impact of more or less threads.
This was very useful, thanks andy! 🙂
“The other parameters were left to default (Single thread: /DCOPY:DA /COPY:DAT /R:1000000 /W:30. Multi thread: /DCOPY:DA /COPY:DAT /MT:8 /R:1000000 /W:30)”
Robocopy /?
“/MT[:n] :: Do multi-threaded copies with n threads (default 8).
n must be at least 1 and not greater than 128.
This option is incompatible with the /IPG and /EFSRAW options.
Redirect output using /LOG option for better performance.”
That’s from Robocopy 6.3. Notice the DEFAULT is 8?
What version of robocopy was used for these tests?
8 is the default value, if you specify the MT parameter without a thread count. This has not been changed since at least Server 2008. This tests have been performed with Windows 10.
Might be just me, but I’m not seeing what values you are running with that is having the increased performance. Is it 128 threads? Are there other flags that you are using?
I didn’t specify any other parameter, which leaves them to default:
Single thread: /DCOPY:DA /COPY:DAT /R:1000000 /W:30. Multi thread: /DCOPY:DA /COPY:DAT /MT:8 /R:1000000 /W:30
Would be interresting to see how changing the number of threads affect the result.
Does changing to 16 or even 32 threads cause the transfer to go faster or slower?
How many cores/threads does the CPU you tested on have?
Thank you Andy. Any chance you could rerun with /MT:32 or MT:64? I don’t see what would be making any difference since the default is 8 threads. Also interested in whether the overhead of managing 128 threads might slow the copy due to overhead. Kinda like this is your processor; ladida; and this is your processor trying to manage 128 threads, along with all the overhead of the OS, networking, etc. *(&*holy_sh@t!**&%what_the_Fukc. OK I am going just a bit covid-stir-crazy. 🙂
I too am interested in seeing results for /mt:128 as this is what I am using on a copy across an ipsec vpn tunnel that takes 20 minutes locally and 20 hours over the link.
Very helpful article, the speed and the time was my problem always with robocopy , now I use a multithreaded GUI copy program called Gs richcopy 360, it has high speed and robust performance, now I don’t have any problems with speed or time .
Why should your robocopy parameter /MT:8 affect the speed in any way if the default value is already exactly /MT:8?
I assume that 8 is default if you are using /MT without specifying the number of threads. Without /MT it’s singlethreaded.
I ran some file copy jobs in 2020, using about 7 million small files. At that volume of small files, I actually ran into performance issues with anything higher than /mt:12, because the disk queues became a major issue, choking either source or target system, and in one case, causing a major outage when it overloaded my SAN. (The SAN itself handed the extremely long disk queue fine, but it slowed the SAN response time *just* enough, that my Citrix hosts panicked and rebooted.)