XMLHttpRequest progress event issue with gzipped content (except FireFox)

On my application I request large JSON files using the XMLHttpRequest API. This works as expected.

I want to display a progress bar while the user waits. Some of the JSON files are between 6 to over 10Mb, so a progress bar would help improve the overall experience.

Here is an example of what I am doing:

var progress = document.getElementById('progress');

function getData(url) {
    var req = new XMLHttpRequest();

  req.onprogress = function(event) {
    if (event.lengthComputable) {
      progress.value = event.loaded / event.total * 100;
    } else {
      console.log('lengthComputable failed');
    }
  }

  req.onload = function() {
    console.log('Finished loading');
    // ... handle data
  }

  req.open('GET', url, true);
  req.overrideMimeType('application/json');
  req.send(null);
}

getData('https://raw.githubusercontent.com/datasets/geo-countries/master/data/countries.geojson');

Also on jsfiddle: https://jsfiddle.net/ctL5f73s/1/

If the content is not gzipped, the progress bar seems to work fine on all the browsers I have tested so far. But if the server sends it as gzip, only firefox works.

For context, I am requesting the data from GitHub. What I understand is that they serve all assets as gzip and I have no control over it.

I am also aware of XHR's progress events should handle gzipped content. Most of what they say just flew over my head, but what I gather is that there is no plan for changing the way lengthComputable is handled when content is gzipped.

On the Response Headers I see the Content-Encoding and Content-Length are correct:

The question is: is there any way to make the progress event work with gzip content on browsers other than FireFox?

If it is not possible, can anyone point me to an alternative? preferably native JS and not a library.

I don't mind for compatibility too far back, but if it doesn't work with the most recent versions of Chrome, then it is a problem.

Answers


I did something like this. It's not exact, but it's good enough.

req.onprogress = function(event) {
  if (event.lengthComputable) {
    progress.value = event.loaded / event.total * 100
  } else {
    var total = req.getResponseHeader('content-length')
    var encoding = req.getResponseHeader('content-encoding')
    if (total && encoding && (encoding.indexOf('gzip') > -1)) {
      // assuming average gzip compression ratio to be 25%
      total *= 4 // original size / compressed size
      progress.value = Math.min(100, event.loaded / total * 100)
    } else {
      console.log('lengthComputable failed')
    }
  }
}

Obviously sometimes progress will jump to 100% and sometimes it will be stuck on 100%, but only for a brief moment, so it's hardly noticeable.


The problem is probably your Content-Type: text/plain. Use GET from curl at a shell command line (without --compressed, to immediately see the binary output) -- does the response show immediately?

Chrome, for example, does not hit the onprogress callback immediately for 'text/plain', but it would if you used 'application/json', for example. Try 'Content-Type: application/json' in your response headers and see if that makes a difference.


Need Your Help

MSMQ concurrent processing design issue

c# concurrency windows-services msmq

I have a windows service written in C# that reads from MSMQ and based on the type of the message it assigns them to Agents that process that message in a worker thread. The application starts with no

Overload "->*" operator

c++ operator-overloading operator-keyword member

I have gone through operator overloading tutorial. I haven't find any example of overloading "->*". It has been told that the mentioned operator is rare and used mostly in expression template.