Cleanup Time

 With the map compass series done (at least for now) I'm going to take a break from dedicated development and spend some time on cleaning up the existing code.  I'd like to get to the point where I have a number of styles I can use to generate a map that will almost always produce an acceptable result.  There are a few major problems I need to address but most of the problems are fairly minor annoyances that just have to be beaten down one at a time like a whack-a-mole game.

For no particular reason I'll start with forests.  One style of forest that DA uses is called “forest masses" and draws forests as large clumps, as so:

One problem with this can be seen in the individual trees around the Toplant River at the top of the map.  These individual trees clumps shouldn't be used with the forest mass style; they're really intended to be outliers when the forest is drawn as individual trees.  That's a fairly easy fix.

The second problem is the texture of bumps inside the forest mass.  It has a couple of problems.  The bumps tend to line up in rough columns, and there are bumps close to the edge of the mass where they get clipped.  

I wrote this style of forest fairly early on, and the bump texture inside the forest masses is getting placed using a sort of perturbed grid.  Since then I've realized that a Poisson disc sampling is a better approach for filling an area in this way, and in fact I can reuse the basic approach I used for the Tolkien forest style.

Surprisingly, the borrowed code works on the first try.  I just have to tweak the distance between the bumps some to get a decent range of textures.

Pulling the bumps in from the edges of the woods is more problematic.  To do this requires “shrinking" the polygon in which I'm drawing the bumps.  Shrinking/expanding a polygon is a non-trivial problem, as I've discussed in these pages before.  In no wise should you consider writing this code yourself.  In this case, I'm going to try using the polygon-offset library from Alexander Milevski; it is partially based upon the Martinez library, which I have also recommended. 

There are many ways for polygon offsetting to fail, or at least to produce difficult results, such as numerous tiny polygon “islands."  Many of the failure modes have to do with sharp vertices close together -- like the bumps on the forest masses.  So rather than shrink those outlines and let every bump invite trouble, I'll shrink the “unbumped" original forest outlines.  This helps reduce the number of errors, but it doesn't entirely eliminate problems. So this is an area of the code where it is helpful to catch exceptions and route around the damage.

With that and some tuning of the shrinking distance, the bumps are kept from running off the edge of the forest:

There are several other options that control the look of this style forest, but they all seem to work fine.

I most often draw these sorts of forests pulled back to reveal the rivers, as above.  There is an option to draw forests with “hidden" rivers (they're not really hidden, just displayed differently).  Turning that on doesn't work.  There turn out to be a number of problems, mostly “code rot" where I've made changes but older code hasn't been updated (or not properly) to use the new code.  It takes quite a bit of time to run down all the problems, but eventually I have it working again: 

It's kind of a nice effect, looks like a fold in the forest where the river runs.

The ocean visualization is pretty solid, although there's a small bug that causes the ocean to vanish occasionally.  That's pretty easy to fix, and along the way I realized that a map with the ocean as blank space is actually a nice variant.

Here's an interesting problem:

The hills are being drawn in green.  At first it looks like hills are getting filled with the forest color instead of the land color, but on further investigation, that isn't the problem.  The color is a hard-coded value hold over from the Knurden-style map, where the land color is green.  It shouldn't be hard-coded in any case, so I'll make the color derive from the land color if it isn't otherwise specified.

That looks better.  I like the subtle shading on these mountains.

Dragons Abound also crashes fairly frequently when drawing mountains, so I decided to run down and correct some of these.  

The crash has something to do with the the “topline" of the mountain.  The topline is the entire outline of the mountain, and there's a function that tries to clean up the topline if for some reason the topline dips below the baseline of the mountain.  After some tracing, there seems to be a missing parameter in my configuration file, and that causes the topline to get created incorrectly in one case.  I replaced the default value of missing parameter and that fixed the error.  I didn't see any problems in the resulting mountains, so apparently the default value is good enough.  I'm not sure why I had commented it out.

Here's a problem that popped up while I was working on labels:

In the upper part of this image, you can see a bay outlined in gray and labeled “Turquoise Point."  DA has mis-identified the bay as a point.  The primary difference between a bay and a point is that a point has land between the two end points of the green line and a bay has water.  Since there's clearly water between the end points of the green line, something has gone wrong.

This turns out to take nearly a full day of debugging to fix.  The details are complex, but the basic problem has to do with how to tell whether an arbitrary point on the map is in the ocean or in the land.  What makes this complex is that although the general shape of the land is created with a procedural generation algorithm operating on an underlying Voronoi graph, the final coastlines can vary from that for any number of reasons.  The two ends of Turquoise “Point" above are across a small bit of ocean that actually has some land in it -- you can see a small island.  And this was confusing the algorithm.

After no small amount of thrashing around I have a new algorithm that checks for land based upon the actual coastlines as drawn, and that fixes that particular problem:

Now there's no longer the spurious Turquoise Point.  There's now a legitimate point on the far side of the bay -- Dusky Point -- but there's a problem with the label.  That's discussed and fixed in the blog post on labels.

Another problem that popped up while working on labels:

This problem is somewhat similar to the problem with Turquoise Point.  It happens because the Voronoi tiles that make up the map are considered land if any part of the tile is above sea level. In places like this little strait, most of the tiles underneath it have at least one corner on the land, so to the forest algorithm, it all looks like land.  The solution is to use a stricter definition of land which requires the midpoint of the tile to be on land.  

There's still some small overlaps where the rounded edges of the forest intrude on the ocean, but that's fine.

Then there's this:

Bloody hell.  I won't be using that random seed again!  

(More seriously, this is some sort of problem with the polygons that define the forest masses but it seems to be a rare error.  I'll give it more serious consideration if it continues to pop up.)

Next posting I'll switch over to labels as mentioned above.

Map Compasses Retrospective

I started the map compasses project with two primary goals.  First, to create a procedural map compass generator, and secondly to experiment with sharing my code.

Generating map compasses has been on my list since seeing Oleg's version a couple of years ago.  Oleg was kind enough to share his code with me, but I decided that the compass roses he generates were not usually suitable for my maps and decided to write my own.  I also wanted to experiment with creating what I think of as a conservative generator.

There's usually a trade-off in procedural generation between creativity and acceptability.  If you build a procedural generator that can be very creative and come up with unusual and unique solutions, it's usually the case that the generator will produce a substantial fraction of solutions that aren't acceptable.  In the past I've leaned to the creative side of the spectrum, and (for example) my city icon generator can invent a wide variety of city icons, but there's a good chance that any particular icon won't be acceptable for one reason or another.  Creative generators tend to be only lightly constrained.  They're able to “break the rules" and sometimes that produces a wonderful result, but often it produces a poor result.  That's why a lot of artistic endeavors have “rules of thumb" -- they outline the areas where good results heavily outweigh poor results -- and if you break those rules you can sometimes do something really new, but often at the cost of a lot of failures.

In general, you might not think it a problem to have a procedural generator that occasionally produces a bad result.  Just generate a new result until you get one you like.  And in isolation, that's fine.  But problems arise when you have a program like Dragons Abound which incorporates a number of separate procedural generation systems.  Even though each PG system may only occasionally throw up a bad design, when you have several running in parallel, the chances of at least one of them providing a poor result goes up.  And this is particularly annoying when your program can take several minutes to run.  Having to run over and over trying to hit upon a map where everything is good is annoying.  

And with creative generators, I always think that I'll look at the output and think about the good and bad results and then add those insights back into the generator to improve the good/bad balance.  For example, I might see that “city walls" in city icons only turn out well in certain specific instances and then go back and tune the rule set accordingly.  But in practice, I don't really ever do that.  By the time I'm seeing the bad city icon results, I've moved on to other problems, and about all I ever do at that point is just turn off a rule entirely.  No more city walls, that fixed the problem!

For those reasons, when I started the map compass generator I leaned in the other direction and tried to create rules that would almost always produce an acceptable result.  Overall, I'm happy with how that decision turned out.  The generator mostly creates acceptable (if “vanilla") results, and I don't usually have to re-run the generator to avoid a bad result.  I also think that on a practical level, I'm more likely to see an example of a compass I like and then go back and add that to the generator.  But we'll have to see how that works out.

One feature I thought about while creating of the compass map generator is a “temperature" control.  The idea would be to have a dial (parameter) that I could turn during generation to be either conservative (cold) or more creative (hot).  With rule-based systems, one way this could work would be to assign a temperature range to every option in a rule, e.g., 

<radialElement> => {0} <thinCircle> | {10} <thickCircle> | {50} <weirdCircle>

Here the number in the curly brackets indicates the lowest temperature for this option, i.e., thinCircle could be used even at the coldest setting, but weirdCircle wouldn't be a possibility until you turned the temperature up to 50 or above. This would give me the option to play it safe or gamble, depending upon what I was trying to do.  That's something I'll be thinking about for the future.  

The second goal for map compass generation was to experiment with sharing my code.  I often get asked to share my code, and I usually feel a little guilty saying no.  After all, I got started down the Dragons Abound road because years ago Martin O'Leary was generous enough to share his map generator, and in a way I feel I owe the same grace to the community.  And I often hear the benefits of open source lauded -- maybe I was missing out on the benefits of a thriving community around the Dragons Abound code.  But at the same time, I recognized that sharing my code would have some substantial costs -- not just eliminating any commercial value to the code, but also in terms of the time and energy it would take to share.  I didn't really have any good way to judge whether that balance would be positive or negative for Dragons Abound, so I decided to do an experiment with sharing some code to see how hard it was and how it was received.  Map compasses was the first thing that came along that fit the bill.

In terms of the mechanics of sharing, it proved to be about as difficult as I had expected.  One of the challenges was in producing code as a companion to the blog entries.  I wanted readers to be able to access and run the code as it was in each blog entry.  I ended up doing this by using Git branches -- there's a separate Git branch in the repository for each of the 18 blog entries in the compass series.  This turned out fine, but it was a bit of work.  I had to get used to working in a separate branch for each blog entry, and remembering to start a new branch with each new entry, and so on.  There were several occasions where I screwed this up and had to spend an evening fixing up the branches.

I also didn't want to release branches ahead of the blog entries, and since Git doesn't offer a way to make private branches public (at least not at the free level), it meant I couldn't actually use Github until I was ready to release the code. 

 I also wanted to keep the code self-contained to make it as easy as possible for people to grab the code and start using it.  This involved including small web servers for Windows, Linux and Mac in the repositories, as well as providing a starting web page.

Finally, I wanted to give people a way to run the code online if they just wanted to see the generator run.  I did this using Netlify and once I had established an account, set up a workflow and so on, this was fairly straightforward.  For each new branch I created a new site, gave it a recognizable name, and Netlify handled the rest.  This was probably my most pleasant tool interaction.

There were also some costs in writing the code.  First, I naturally took more care in writing code that I intended to share.  I tried to keep the code straightforward and understandable, so I often avoid things like list comprehensions that less experienced programmers might find confusing.  I also took care to comment anything that might need explanation, and to keep my formatting consistent.  In my “regular" Dragons Abound programming I often leave old versions of functions or temporary code in place as a reminder of the development process or in case I might need them again, but in the shared code I edited such things out.

Another unexpected cost was that I had to copy or rewrite a lot of utility functionality.  In Dragons Abound I have a large library of utility functions to do things like manipulate polygons, work with the map, draw lines, and so on.  Using those in the compass generation required either including lots of Dragons Abound code, spending time adapting the code for the compass generator, or rewriting the functionality.  I ended up mostly adapting/rewriting the code to keep the overall project as simple and understandable as possible.

A final cost of sharing the code was that I took the time at the end of each blog posting to suggest ways in which people might extend or modify the code for that posting.  Sometimes these ideas were extensions of what I'd done in that posting, and sometimes they were foreshadowing something coming in a later blog post.

Overall, the costs of sharing were considerable. I'd say it slowed down my development pace by 50% or perhaps more.  On the other hand, did code sharing have any positive benefits?

Well... A few people seemed to look at the code:

I'd typically get a couple of visitors to the Github repository after every new blog post. 8 users “starred" the repository.  Nobody forked the repository or tried to make a commit to any of the branches.  Although I asked in most blog posts for people to share anything they'd done with the code, I got no comments or emails indicating that anyone had used the code.  (I thought about leaving an obvious bug in the code to see if that generated any feedback but in the end decided against it.)

One of the benefits often touted for open-source code is outside contributors, but my experience has been that few projects get significant outside contributions (and few projects are welcoming of outside contributions, but that's a different problem).  That certainly seems to be the case here.  Of course, there could be many reasons for that.  Although I tried to make the code accessible, few people have the programming skills required to contribute.  And it's not like map compass generation is a problem of wide interest.  Also, my version of it might be poorly publicized or poorly executed.  But whatever the reasons, there was no interest in contributing to the code.

For another data point, we can look at Azgaar's fantasy map generation project which is obviously of broader interest.  Azgaar makes his code available and does a lot of work to create and support an active community of users.  Looking at his code statistics, it's clear that out of his thousands of users only a small handful have made any contribution to the code.

I often get emails or messages asking me to share the Dragons Abound code, and many of these messages claim to be interested in modifying or contributing to the code.  I made sure to bring the map compass experiment to the attention of several people who had made those sorts of requests; none of them responded or engaged with the code.

My conclusion is that there's virtually no software development benefit in sharing the code for this kind of project.  It has significant costs, and seems unlikely to provide any return on the investment.  If you have an interesting and well-crafted project, you'll likely get a fair number of people wanting access to the code, but it seems clear that almost all of these people want to use the code, not contribute to maintaining or improving it.  Which is not to say you shouldn't share your code or make it public, just don't do it expecting to find an engaged and contributing development community.

I've focused above on code contributions, but another possible benefit of the code sharing might be that blog readers get more out of postings that were paired with actual code.  But I don't think this was the case.  I don't have any hard statistics, but I get the sense that people engaged less with the code sharing postings.  (Certainly I didn't get any feedback praising the addition of code.)  I'd guess that most people prefer to read about the highlights and insights of the software development rather than the nitty-gritty details of the code itself.  And that's understandable.  It's a lot of work to read code and comprehend what it is doing.  Unless you're planning on modifying the code yourself, there may not be much to get out of that.

Overall I rate the map compass experiment a success.  The whole process of writing the blog and the code intertwined was new and I think I got something out of it.  I also enjoyed figuring out how to make the code public and accessible.  And it did bring some clarity to my thinking about code sharing.

And now onward to other things!