Browser Performance & Web Server & ES6

Since very early in the development of Dragons Abound I have done all my work in Chrome.  This was not through any particular devotion to Google's product, but simply because it was impossible to run Dragons Abound in Firefox.  A run that takes about 90 seconds in Chrome would take 900 seconds or more in Firefox, if it would complete at all.  Most times, Firefox's memory size would grow very large and the browser would crash.

At the time Firefox was my daily browser, so it was pretty inconvenient to have to run Chrome in parallel just for development.  But I soured on Firefox for other reasons (primarily the decision to cripple extensions) and ended up switching to Chrome for all my browsing needs.  When Firefox Quantum came out to much fanfare, I checked performance and while I found it somewhat improved it was still unusable for development.

Earlier today I loaded up Firefox for other reasons and decided to check again to see if any progress had been made.  The first thing I discovered was that Firefox objected to a number of practices in my code.  Chrome is fairly forgiving of minor syntax problems within SVG, but Firefox apparently much less so.  After correcting those errors I made a side-by-side run and was shocked to find Firefox is now considerably faster than Chrome.  On the test run, Chrome took 192 seconds and Firefox took 110 -- that's almost 75% faster.

The two browsers don't report performance in identical ways so it is hard to identify exactly where the majority of the speedup occurs, but it appears that Firefox might do a better job executing certain in-browser SVG functionality, such as getTotalLength.

When I repeated this test on a more powerful machine, I got different results.  In this case, Chrome took about 47 seconds and Firefox about 66 seconds.  It might be that with a more powerful graphics card, the browsers offload the work of rendering the page differently.

Interestingly, on this machine, Firefox registered a large number of errors about fonts and font attributes.  I spent a fair amount of time correcting as many of these problems as I could.  Many of the problems were caused by using empty values for different style attributes.  This didn't bother Chrome, but caused Firefox to throw an error.  Firefox also seemed to have a problem with font names that start with numbers; this was fixed by renaming the fonts.

More serious than these problems was that Firefox on this machine had many problems with the fonts I am using, throwing up many messages along these lines:

downloadable font: GSUB: too large substitute: 1650 

I'm not sure whether these errors were at fault but fonts had problems in this Firefox.  Even Google Webfonts were displaying poorly or incorrectly.

In general, I don't have the time/energy to make Dragons Abound run well in more than one browser, and since Chrome appears to be more tolerant and working fine for now I'll stick with it.  Might be worth looking in on Firefox from time to time to see if these problems get resolved and performance continues to improve.

---

The above portion of the post was written about six months ago; I decided to look in again on Firefox to see how it has progressed.  I see that I didn't actually note what version of Firefox I was using above.  The current version is 60.0.1 (32-bit).  On the Chrome side, I'm running 65.0.3325.181 (Official Build) (64-bit).  I tested by running each browser once, generating the same map in a fresh tab.  Not very scientific, but enough to give me a notion of the speed of each browser.

Chrome took 157 seconds to generate and render the map; Firefox took 82 seconds.  Firefox still complains about a lot of the fonts I'm using, and it seemed to take a long time to load the page initially.  But to my eye the Firefox map looks identical to the Chrome map.  This is a rather astonishing time difference, so let's see how it breaks down.

These are the top ten time users in the Firefox profile:

Time	Percent	Function
14,407.47 ms	17.54%	Graphics
8,211.89 ms	10%	GC
7,897.69 ms	9.62%	pickNextTerritoryLoc
5,739.89 ms	6.99%	Gecko
4,374.89 ms	5.33%	windModel
3,043.17 ms	3.71%	distance
3,026.53 ms	3.68%	bezierCurveTo
2,742.50 ms	3.34%	addVectors
2,623.89 ms	3.19%	JIT
2,518.81 ms	3.07%	get
2,424.13 ms	2.95%	valueLocDynamic

“Graphics" is presumably the rendering of the SVG onto the screen.  The SVG that Dragons Abound generates at this point is large and complex, so it's no surprise this is a big job.  Garbage Collection, Gecko and the Just In Time compiler (JIT) are also in the Top Ten -- these are all browser functions that execute the Javascript.  “pickNextTerritoryLoc" is a function that builds the different countries on the map.  Figuring out which location to add next to a territory is laborious and probably unnecessarily inefficient.  I've talked about the “windModel" before.  It just takes a lot of computation to propagate winds across the map.  “distance" is the usual function for calculating distance between two points.  As you might imagine, it's used extensively all over the code.  Two more interesting functions are “bezierCurveTo" and “get".  The former is part of d3.js and is used when I draw anything with a Bezier curve.  The latter is part of the parameter management I wrote about previously.  For every parameter the program uses, it has to check several structures and that apparently adds up (at least in Firefox).

Here's the Top Ten for Chrome:

Time	Percent	Function
17790.2 ms	11.4%	getTotalLength
16300.7 ms	10.4%	windModel
10744.6 ms	6.9%	pickNextTerritoryLoc
9744.7 ms	6.2%	createTerritories
5415.5 ms	3.5%	Recalculate Style
4383.1 ms	2.8%	point
3984.8 ms	2.6%	createBiomes
3808.5 ms	2.4%	Minor GC
3754.4 ms	2.4%	distFromOtherLabelsFn
3173.4 ms	2.0%	polyclip
3074.5 ms	2.0%	removeChild

Amazingly, it's largely a different list.  Only “windModel", “pickNextTerritoryLoc," “Graphics/Recalculate Style," and “GC" are in both lists.  Looking at those, it appears that Chrome is much faster at displaying the SVG and managing memory to avoid garbage collections.  “getTotalLength" is a browser function that measures the length of an SVG path; it looks like Chrome's implementation of that may be inefficient.  On the other hand, Chrome calculates distance about 12x as fast as Firefox (!).

I did these tests on a moderately powerful desktop.  My development desktop has more memory and a better graphic card.  It's running Chrome 66.0.3359.181 (Official Build) (64-bit) and Firefox 60.0.1 (64-bit).  On this machine, Chrome took 63 seconds and Firefox took about 53 seconds.  The top ten functions were roughly the same, with some swapping around of places.  So whatever makes Firefox much faster on the slower machine appears to be less relevant on the more powerful machine.

---

On another topic, I currently run Dragons Abound as a standalone Javascript application directly from the file system.  (It loads in the browser as a FILE url.)   There are some problems with working this way, which I've noted before.  Specifically, there's something in modern browsers called the same-origin policy.   The same-origin policy is intended to keep Javascript from one web page executing Javascript from another web page, since this could be used to do things like steal your PIN number out of your banking web page.  However, same-origin policy is only well-defined for protocols like HTTP and HTTPS.  And in particular, it has never been defined for file URLs.  This leads to all sorts of problems when you try to access anything on the local file system from your browser.

I've mostly worked around those problems, but I'm getting ready to switch my codebase over to ES6 modules, and there's another problem with file urls and ES6 modules.  One of the security features of ES6 modules is that they're supposed to only load Javascript -- which they determine by looking at the file's MIME type.  When you try to load an ES6 module into Chrome from the local file system, it complains that the file doesn't have the right MIME type.  (Firefox doesn't complain -- as I said, browser are free to treat FILE urls pretty much as they like!)  It isn't clear whether Windows doesn't report a MIME type or Chrome ignores it -- there are a couple of relevant Chromium bugs open -- but at any rate it creates a problem.

The best solution to all these problems is to run Dragons Abound through a standalone local web server.  Then everything is referenced through HTTP urls and all of the problems with FILE urls go away.  I've resisted this in the past primarily because of inertia, and just the added headache of setting up a local web server and running it whenever I want to work on the program.  So I spent a little time today seeing if I could find a very simple turnkey web server that would make it easy to run a local web server.

I found something called Mongoose.  Mongoose is marketed as a web server for embedded devices -- small electronic computers used in parts of a larger system, like a computer in your toaster.  Obviously I don't care about the embedded device part, but what's relevant to me is that Mongoose is very small and simple, and is available as a one-click Windows application.  I just have to drop it into the folder with the Dragons Abound source code, double-click, and it pops up in my web browser.

Switching over does require a little re-arrangement of my fonts and images (because Mongoose can only see files in the folder it starts in, or subfolders) but that's not a huge problem.

So far Mongoose is working flawlessly. Recommended.

---

Next up on “Things I've Been Meaning to Do" is to switch over to ES6 modules.  Modules are a way of splitting up code into units that don't interfere with each other.  You can do whatever you want inside a module, and then you just make visible only the important parts.  This makes it much easier to write libraries and generally keep code concerns separate.

Unfortunately, Javascript was originally designed as a kind of quick and dirty scripting language for within web pages, and no one realized that it would grow into the monster it is today.  Consequently, no thought was given to things like modules.  People eventually added in the functionality of modules through various packages like RequireJS, SystemJS and so on.  Of course, all these different approaches were only somewhat compatible.  Eventually the Guardians of Javascript decided to build a real module system into the language, and that was ES6 modules.  They've only recently become available across all browsers, so code remains a confusing mix of many different module systems.

Dragons Abound has generally used RequireJS, although for some packages it relies upon I've had to make accommodations for other systems.  But I've wanted for some time to switch over to ES6 modules, if only because the syntax is simpler and cleaner.  So I decided to bite the bullet this weekend and see if I can make the switch.

The first hurdle was just translating all of my source code files from RequireJS to ES6 modules.  This was made a little easier by amdtoes6, a “transpiler" that does the required translation.  I was able to turn it loose on the directory of source files and it did the bulk of the work.  

I say “bulk" because there were a few things I had to fix by hand.  The key part of using ES6 modules is to import into your code the modules you are using.  This is typically shown like this:

import Utils from 'utils';

However, this syntax didn't work for me; I got complaints about relative paths and other problems.  Eventually I settled on:

import Utils from './utils.js';

Which works for me.  The remaining problems revolve around software packages that I use.  A couple (primarily d3js) load as normal scripts fine.  Others (like seedrandom) I can get to work by tweaking the code slightly.  The biggest stumbling block is one of the libraries I use for computing intersections of polygons -- martinez.   This loads without any errors but is not exporting correctly.  All of these libraries have code that tries to work with a variety of module schemes (ReasonJS, CommonJS, etc.) and my best guess is that martinez thinks it is in a different environment than it really is.  When I load it up the export is a function rather than the expected structure.

The solution (or at least a solution) is to add some code that puts the proper structure onto the global where I expect to find that:

if (typeof window === 'object')
window.Martinez = boolean;

In this case, “Martinez" is my name for this module, and “boolean" is the name of the proper structure within the martinez code.  (An odd name, but that's what the author called it.)  The result of this is to make the exported code available as Martinez.intersection(), etc.

With that, everything is working again, now in ES6: