![]() ![]() I do know that while about a dozen people use the fileserver daily, I am the only person using this particular node/local disk. I am also not a system administrator - I am only a (novice) user - so I don't have access to more detailed information on the load that is on the disks. I am sure that buying new, faster hardware disks, but I don't have access to such resources. So, my question is: Is there a faster alternative to cp for copying large files in Linux? (Or are there any flags within cp that I could use which would speed up copying?) Even if I could somehow shave a minute off this copying time, that would help immensely. I need to copy ~200 large files - each ~20 GB - from the fileserver to one of the local disks. I am surprised that copying from local to same local is not faster than 9 minutes. So apparently the fileserver is somewhat slower than the local disk, as expected, but perhaps not significantly slower. Using cp to copy the same ~20 GB file from one folder on the local disk to another folder on the same local disk takes less time - about 9 minutes in real time (~51 seconds in system time, according to time). I know that this cp operation is not very efficient because (1) time tells me that the system time for such a copy is only ~45 seconds and because (2) when I examine top during the copy, %CPU is quite low (by inspection, roughly 0-10% on average). Using cp to copy a ~20 GB file from the fileserver to one of the local disks takes about 11.5 minutes in real time on average (according to time). ![]() About a dozen or so people use the fileserver. I do know from experience that the local disks are significantly faster than the fileserver in terms of I/O. I am a relative Linux novice, so I am not sure what are the specs of the fileserver in terms of speed, networking ability, etc. So the group owns a fileserver with many TBs of storage space. Each node of the cluster has its own local disk, but these local disks are relatively small and are not equipped with automatic backup. I am a graduate student, and the group in which I work maintains a Linux cluster.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |