Why do I get an error uploading large files?
|Question||Why do I get an error uploading large files?|
|Answer||There are a couple of intentional limits set for uploading:
The last one, 'Uploads must complete within 60 minutes' may be a little more contentious but it is important that I protect the server from unexpected events like a connection being held open indefinitely due to an error or a hack attempt.
UPDATE April 2017: The time limit has been changed to 60 minutes.
How long will my upload take?
That's a great question because most people don't realise that their Internet download speed is not the same as their upload speed.
You will have one of two types of connection, Synchronous or Asynchronous.
If you have a fibre connection there is a good chance you have a Synchronous connection and your upload speed is the same as your download speed. This is wholly dependent on your provider who may choose to restrict your upload speed.
Any other type of connection is almost certainly Asynchronous for a very simple reason, your provider is trying to give you the best performance possible for downloading but there is a limit to how much traffic will pass over copper wire (total bandwidth).
Most users seldom upload very much data. Basic web browsing or downloads requires a small fraction of the total data to flow outward from your machine/device whilst it must allow a large amount inward to give you a quick/smooth experience.
It's hard to put a figure on it because every provider is different but, as a rule-of-thumb, your upload speed is usually no more than 10% of your download speed. So, if you have a 10Mb (megabit) connection it is likely that your upload speed is 1Mb.
Bits & Bytes
For those of you that are only vaguely familiar with this terminology, here are a few pointers that may be helpful.
A Byte is made up of 8 Bits therefore 1 Megabyte = 8 Megabits or put another way, a 1 Megabit upload connection will transfer a maximum of 0.125 megabytes (125KB) per second.
In addition to the actual data you wish to transfer, every byte that is uploaded or downloaded needs to be checked to ensure it's integrity is good so there is always an additional overhead.
What this means is that about 2% (4% for IPv6) of your transfer is NOT part of the file or web page and is there just to ensure everything is flowing correctly.
If you have a bad Internet connection this overhead could increase to 50% or even more as packets have to be resent because they fail the integrity check. Some very big US providers are notorious for this problem!
Back to the question! Here are a couple of practical examples:
If you have a 10Mb Asynchronous connection it's likely you have an upload speed of 1Mb. If you want to upload a 100Mb file it will take around 14 minutes on a good connection.
File size * bytes to bits + overhead / your upload speed in bits per second / 60 = minutes
(100MB * 8) = 800Mb +4% = 832Mb / 1Mb = 832 seconds = 13.8 minutes.
If you have a 20Mb Synchronous (fibre) connection to your ISP the same file would take less than 1 minute.
(100MB * 8) +4% = 832Mb / 20Mb = 41.6 second = < 1 minute.
This means the practical limit for uploading is (((upload speed/8) *0.96)*60)*30 eg.
For 1 Mb upload speed = (((1,000,000/8)*.96)*60)*30=216MB
Please keep in mind that this is a for a near perfect Internet connection that is not being shared by other users on the same connection ie. the wife or kids are downloading a recipe or playing an on-line game.