Displayed file upload progress is out of sync with actual transferred data

[oakkitten]

You may have noticed that when uploading files you can see the circular progress bar filling to 100% quickly, and that there's a dot rotating counterclockwise that can rotate for a while until upload is finished. Rotating left means that we have uploaded 100% of data, and waiting for the response from the server. The problem is that in the case of smaller files, the displayed upload progresses faster than the actual data transfer, and data continues being sent while we are “waiting for server response”.

The problem here is that we are tracking source bytes as they are pushed into Okio's sink, which is then ultimately stuffed into the system socket send buffer, which on my device is whopping 2MiB. See square/okhttp#1078. This difference is hard to account for. Even if we could easily track how much data is sitting unsent in the socket with the idea to subtract that number from the number of sent bytes, we would have to move our tracking further from source, past the compression and encryption layers of OkHttp, and to be quite smart about this all what with the various retries and subtleties of the whole process.

One straightforward way to deal with the issue would be to decrease the system socket send buffer. However, this might significantly impact upload times. For instance, with our current code, using a send buffer of 1KiB, a typical upload of a photo takes 4x the current time. A buffer of 16KiB performs significantly better, but it's hard to tell how exactly it should affect upload time. I suppose that a good send buffer value should depend on file size, network speed, network latency, and would require a complex algorithm to adjust.

Perhaps we could try using a reasonably high send buffer dependent on the file size only. For instance, file size / 20, but no less than 32KiB and no more than the default size. This would produce 15KB send buffer size for a 3MB file. However, even such a change might produce afwul results on an unlikely high speed and high latency network.

Or we can leave this as it is, it's just a progress bar.