I would not expect formatting or otherwise wiping the target drive to make any significant difference to transfer speed but of course the only way to be sure in any specific environment is to try it. Bear in mind that SSDs manage data in a different way to rotating hard disks, as there is no latency issue and nothing to be gained by defragmentation. The internal SSD logic moves data around on the drive to equalise wear on the memory cells as flash memory has a finite number of read write cycles. There are also spare cells to allow for this wearout.
You will get a greater variation when using standard hard disks due to latency and interleave timings, and allowable variations of rotational speed, usually plus or minus 5%. Fragmentation levels also play a part.
Finally, the fact that a copy dialog has completed does not necessarily mean that the copy operation has completed, just that the data transfer has completed. There is still a distinct possibility that data is still being written from memory to hard disk in the background especially if the software uses write behind caching.
Use of standard or jumbo frames at the ethernet level would also affect performance, as would error rates on the network (which a network monitor would show up)
You have not mentioned how large a variation you encounter each time but as you can see there are many factors that can affect the precise transfer time.