?Copy with Verification false alarm?
Moderators: white, Hacker, petermad, Stefan2
?Copy with Verification false alarm?
Hi,
I was copying 50Gb of small files between two internal Sata NTFS HDD´s with verify enabled, when TC warned me that a file and its copy didn´t pass the test.
I chose ignore and later used the "Vergleich nach Inhalt" comparison, which told me they were identical.
Could this be a bug in the program or faulty hardware?
Is it possible that the two methods used for comparing lead to different results?
Or are some false alarms to be expected with the comparison used while copying?
edit: windows 7 prof, SP1 64
I was copying 50Gb of small files between two internal Sata NTFS HDD´s with verify enabled, when TC warned me that a file and its copy didn´t pass the test.
I chose ignore and later used the "Vergleich nach Inhalt" comparison, which told me they were identical.
Could this be a bug in the program or faulty hardware?
Is it possible that the two methods used for comparing lead to different results?
Or are some false alarms to be expected with the comparison used while copying?
edit: windows 7 prof, SP1 64
Last edited by Herbert on 2013-10-06, 19:12 UTC, edited 1 time in total.
Happens frequently here.
The affected files vary.
edit:
Not at home right now, so I can't check 100%, but come to think of it all those files that threw an error so far have been in numbered sequences with quite long filenames.
e.g.
country_city_street_house_room_table_leg_bolt_1_4.dat
country_city_street_house_room_table_leg_bolt_2_4.dat
And
country_city_street_house_room_table_leg_bolt_3_4.dat
would throw a false crc error.
might be coincidence, but maybe something gets mixed up.
Beta 5.
Fastcopy works fine doing the same.
I will run extensive memory-tests, just to be sure.
The affected files vary.
edit:
Not at home right now, so I can't check 100%, but come to think of it all those files that threw an error so far have been in numbered sequences with quite long filenames.
e.g.
country_city_street_house_room_table_leg_bolt_1_4.dat
country_city_street_house_room_table_leg_bolt_2_4.dat
And
country_city_street_house_room_table_leg_bolt_3_4.dat
would throw a false crc error.
might be coincidence, but maybe something gets mixed up.
Beta 5.
Fastcopy works fine doing the same.
I will run extensive memory-tests, just to be sure.
- ghisler(Author)
- Site Admin
- Posts: 48130
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Sorry, cannot reproduce - I just copied 50 GB of files, no problem at all.
Some questions:
1. What are these .dat files? Are they database files? Maybe they are open and changed while TC was copying?
2. Which Antivirus do you use? It's possible that it was interfering
3. Which copy method do you use? Configuration-Options-Copy/Delete
4. How big were the files for which you got the error? The verify function is different when the file is >10MB than when it's <=10MB.
Some questions:
1. What are these .dat files? Are they database files? Maybe they are open and changed while TC was copying?
2. Which Antivirus do you use? It's possible that it was interfering
3. Which copy method do you use? Configuration-Options-Copy/Delete
4. How big were the files for which you got the error? The verify function is different when the file is >10MB than when it's <=10MB.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Thanks for your reply.
1. Sorry, the .dat was supposed to be just an example.
The files vary. I wrote some down at the beginning and those were a .zip and a two .flv
Each time I did a copy operation, other files would throw an error, though.
2. I use avast! free. I´ll run a test without Virussoftware these days as well as checking my RAM and report back.
3.
Standard Kopiermethode
Kopieren von Datum/Zeit von Verzeichnissen
Verifizieren nach dem Kopieren
F8/Entf löscht in den Papierkorb
4. All the errors I wrote down were with files bigger than 10 Mb.
1. Sorry, the .dat was supposed to be just an example.
The files vary. I wrote some down at the beginning and those were a .zip and a two .flv
Each time I did a copy operation, other files would throw an error, though.
2. I use avast! free. I´ll run a test without Virussoftware these days as well as checking my RAM and report back.
3.
Standard Kopiermethode
Kopieren von Datum/Zeit von Verzeichnissen
Verifizieren nach dem Kopieren
F8/Entf löscht in den Papierkorb
4. All the errors I wrote down were with files bigger than 10 Mb.
There's a good chance that avast interferes here with its Network Scanner and/or other modules that are known to cause trouble (incomplete loading of websites, massive slowdowns on instant messengers etc). Try disabling all modules but FileSystem, Mail and Behaviour (Verhaltensscanner in German, don't know the correct English term).
Regards
Dalai
Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Thanks for your reply!
I completely disabled avast and the firewall but the problems stay. Did´t change Windows 7 UAC so far, though. Would that be worth a try?
My conclusions so far:
Copying the same data between two disks will throw an error faster than copying data on the same disk.
It doesn´t seem to be the files themselves or at least not the files alone.
If the very same copy operation ist repeated some of the same files MIGHT throw an error again.
If I copy just the subfolder with the files which caused the error, the pattern of offending files will change.
(Might be the reason why I saw a difference when copying within the same hdd )
If I change something else, like copying user permissions the pattern changes again.
In copies of 10 to 150 GB the error might happen 0-10 times. No relation between the overall size of the data copied and how often the error is thrown.
Seems to happen mainly with files in numbered sequences.
I did some shorter RAM tests without error, but yet have to do an extensive one.
edit: I have 32Gb of RAM installed, Windows Pagefile set to 200Mb-2Gb
I completely disabled avast and the firewall but the problems stay. Did´t change Windows 7 UAC so far, though. Would that be worth a try?
My conclusions so far:
Copying the same data between two disks will throw an error faster than copying data on the same disk.
It doesn´t seem to be the files themselves or at least not the files alone.
If the very same copy operation ist repeated some of the same files MIGHT throw an error again.
If I copy just the subfolder with the files which caused the error, the pattern of offending files will change.
(Might be the reason why I saw a difference when copying within the same hdd )
If I change something else, like copying user permissions the pattern changes again.
In copies of 10 to 150 GB the error might happen 0-10 times. No relation between the overall size of the data copied and how often the error is thrown.
Seems to happen mainly with files in numbered sequences.
I did some shorter RAM tests without error, but yet have to do an extensive one.
edit: I have 32Gb of RAM installed, Windows Pagefile set to 200Mb-2Gb
Please try the copy method for large files.
Regards
Dalai
Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
- ghisler(Author)
- Site Admin
- Posts: 48130
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
I will add some more logging details to the next beta to find out what's wrong.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
- ghisler(Author)
- Site Admin
- Posts: 48130
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Yes, this other copy method uses ReadFile/WriteFile, so it has direct access to the copied data (which it doesn't when using CopyFileEx).
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
- ghisler(Author)
- Site Admin
- Posts: 48130
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
To those who had this problem: Can you please retry with beta 7 now?
Please turn on logging in Configuration - Options - Log file. Please log file operations and log successful and failed operations.
When you get the error, you should get more details about the checked file size, and the MD5 of the source and target. Please post them here, and compare them with the real size and MD5 of the copied file.
Thanks!
Please turn on logging in Configuration - Options - Log file. Please log file operations and log successful and failed operations.
When you get the error, you should get more details about the checked file size, and the MD5 of the source and target. Please post them here, and compare them with the real size and MD5 of the copied file.
Thanks!
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
- ghisler(Author)
- Site Admin
- Posts: 48130
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Since no one is willing to test this, and probably only a few people are using verify anyway, I will unfortunately have to remove the verify function from the next beta.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
I just copied a little over 1TB of data from one disk to another using standard copy method with verify enabled, total over 700 000 files with various sizes (from zero to hundereds of megabytes). Windows 7, TC 8.50 beta 12 x64, clean ini except enabled logging and verification. The result - zero errors. So as I see it, it works fine.
- ghisler(Author)
- Site Admin
- Posts: 48130
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Thanks for your tests. The problem is that this error doesn't happen everywhere. It must be caused by some third party program interfering with TC, e.g. a virus scanner. If I can't get feedback from users with this problem, I have to remove the function because it's just too unreliable.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com