Map Compasses Retrospective

I started the map compasses project with two primary goals.  First, to create a procedural map compass generator, and secondly to experiment with sharing my code.

Generating map compasses has been on my list since seeing Oleg's version a couple of years ago.  Oleg was kind enough to share his code with me, but I decided that the compass roses he generates were not usually suitable for my maps and decided to write my own.  I also wanted to experiment with creating what I think of as a conservative generator.

There's usually a trade-off in procedural generation between creativity and acceptability.  If you build a procedural generator that can be very creative and come up with unusual and unique solutions, it's usually the case that the generator will produce a substantial fraction of solutions that aren't acceptable.  In the past I've leaned to the creative side of the spectrum, and (for example) my city icon generator can invent a wide variety of city icons, but there's a good chance that any particular icon won't be acceptable for one reason or another.  Creative generators tend to be only lightly constrained.  They're able to “break the rules" and sometimes that produces a wonderful result, but often it produces a poor result.  That's why a lot of artistic endeavors have “rules of thumb" -- they outline the areas where good results heavily outweigh poor results -- and if you break those rules you can sometimes do something really new, but often at the cost of a lot of failures.

In general, you might not think it a problem to have a procedural generator that occasionally produces a bad result.  Just generate a new result until you get one you like.  And in isolation, that's fine.  But problems arise when you have a program like Dragons Abound which incorporates a number of separate procedural generation systems.  Even though each PG system may only occasionally throw up a bad design, when you have several running in parallel, the chances of at least one of them providing a poor result goes up.  And this is particularly annoying when your program can take several minutes to run.  Having to run over and over trying to hit upon a map where everything is good is annoying.  

And with creative generators, I always think that I'll look at the output and think about the good and bad results and then add those insights back into the generator to improve the good/bad balance.  For example, I might see that “city walls" in city icons only turn out well in certain specific instances and then go back and tune the rule set accordingly.  But in practice, I don't really ever do that.  By the time I'm seeing the bad city icon results, I've moved on to other problems, and about all I ever do at that point is just turn off a rule entirely.  No more city walls, that fixed the problem!

For those reasons, when I started the map compass generator I leaned in the other direction and tried to create rules that would almost always produce an acceptable result.  Overall, I'm happy with how that decision turned out.  The generator mostly creates acceptable (if “vanilla") results, and I don't usually have to re-run the generator to avoid a bad result.  I also think that on a practical level, I'm more likely to see an example of a compass I like and then go back and add that to the generator.  But we'll have to see how that works out.

One feature I thought about while creating of the compass map generator is a “temperature" control.  The idea would be to have a dial (parameter) that I could turn during generation to be either conservative (cold) or more creative (hot).  With rule-based systems, one way this could work would be to assign a temperature range to every option in a rule, e.g., 

<radialElement> => {0} <thinCircle> | {10} <thickCircle> | {50} <weirdCircle>

Here the number in the curly brackets indicates the lowest temperature for this option, i.e., thinCircle could be used even at the coldest setting, but weirdCircle wouldn't be a possibility until you turned the temperature up to 50 or above. This would give me the option to play it safe or gamble, depending upon what I was trying to do.  That's something I'll be thinking about for the future.  

The second goal for map compass generation was to experiment with sharing my code.  I often get asked to share my code, and I usually feel a little guilty saying no.  After all, I got started down the Dragons Abound road because years ago Martin O'Leary was generous enough to share his map generator, and in a way I feel I owe the same grace to the community.  And I often hear the benefits of open source lauded -- maybe I was missing out on the benefits of a thriving community around the Dragons Abound code.  But at the same time, I recognized that sharing my code would have some substantial costs -- not just eliminating any commercial value to the code, but also in terms of the time and energy it would take to share.  I didn't really have any good way to judge whether that balance would be positive or negative for Dragons Abound, so I decided to do an experiment with sharing some code to see how hard it was and how it was received.  Map compasses was the first thing that came along that fit the bill.

In terms of the mechanics of sharing, it proved to be about as difficult as I had expected.  One of the challenges was in producing code as a companion to the blog entries.  I wanted readers to be able to access and run the code as it was in each blog entry.  I ended up doing this by using Git branches -- there's a separate Git branch in the repository for each of the 18 blog entries in the compass series.  This turned out fine, but it was a bit of work.  I had to get used to working in a separate branch for each blog entry, and remembering to start a new branch with each new entry, and so on.  There were several occasions where I screwed this up and had to spend an evening fixing up the branches.

I also didn't want to release branches ahead of the blog entries, and since Git doesn't offer a way to make private branches public (at least not at the free level), it meant I couldn't actually use Github until I was ready to release the code. 

 I also wanted to keep the code self-contained to make it as easy as possible for people to grab the code and start using it.  This involved including small web servers for Windows, Linux and Mac in the repositories, as well as providing a starting web page.

Finally, I wanted to give people a way to run the code online if they just wanted to see the generator run.  I did this using Netlify and once I had established an account, set up a workflow and so on, this was fairly straightforward.  For each new branch I created a new site, gave it a recognizable name, and Netlify handled the rest.  This was probably my most pleasant tool interaction.

There were also some costs in writing the code.  First, I naturally took more care in writing code that I intended to share.  I tried to keep the code straightforward and understandable, so I often avoid things like list comprehensions that less experienced programmers might find confusing.  I also took care to comment anything that might need explanation, and to keep my formatting consistent.  In my “regular" Dragons Abound programming I often leave old versions of functions or temporary code in place as a reminder of the development process or in case I might need them again, but in the shared code I edited such things out.

Another unexpected cost was that I had to copy or rewrite a lot of utility functionality.  In Dragons Abound I have a large library of utility functions to do things like manipulate polygons, work with the map, draw lines, and so on.  Using those in the compass generation required either including lots of Dragons Abound code, spending time adapting the code for the compass generator, or rewriting the functionality.  I ended up mostly adapting/rewriting the code to keep the overall project as simple and understandable as possible.

A final cost of sharing the code was that I took the time at the end of each blog posting to suggest ways in which people might extend or modify the code for that posting.  Sometimes these ideas were extensions of what I'd done in that posting, and sometimes they were foreshadowing something coming in a later blog post.

Overall, the costs of sharing were considerable. I'd say it slowed down my development pace by 50% or perhaps more.  On the other hand, did code sharing have any positive benefits?

Well... A few people seemed to look at the code:

I'd typically get a couple of visitors to the Github repository after every new blog post. 8 users “starred" the repository.  Nobody forked the repository or tried to make a commit to any of the branches.  Although I asked in most blog posts for people to share anything they'd done with the code, I got no comments or emails indicating that anyone had used the code.  (I thought about leaving an obvious bug in the code to see if that generated any feedback but in the end decided against it.)

One of the benefits often touted for open-source code is outside contributors, but my experience has been that few projects get significant outside contributions (and few projects are welcoming of outside contributions, but that's a different problem).  That certainly seems to be the case here.  Of course, there could be many reasons for that.  Although I tried to make the code accessible, few people have the programming skills required to contribute.  And it's not like map compass generation is a problem of wide interest.  Also, my version of it might be poorly publicized or poorly executed.  But whatever the reasons, there was no interest in contributing to the code.

For another data point, we can look at Azgaar's fantasy map generation project which is obviously of broader interest.  Azgaar makes his code available and does a lot of work to create and support an active community of users.  Looking at his code statistics, it's clear that out of his thousands of users only a small handful have made any contribution to the code.

I often get emails or messages asking me to share the Dragons Abound code, and many of these messages claim to be interested in modifying or contributing to the code.  I made sure to bring the map compass experiment to the attention of several people who had made those sorts of requests; none of them responded or engaged with the code.

My conclusion is that there's virtually no software development benefit in sharing the code for this kind of project.  It has significant costs, and seems unlikely to provide any return on the investment.  If you have an interesting and well-crafted project, you'll likely get a fair number of people wanting access to the code, but it seems clear that almost all of these people want to use the code, not contribute to maintaining or improving it.  Which is not to say you shouldn't share your code or make it public, just don't do it expecting to find an engaged and contributing development community.

I've focused above on code contributions, but another possible benefit of the code sharing might be that blog readers get more out of postings that were paired with actual code.  But I don't think this was the case.  I don't have any hard statistics, but I get the sense that people engaged less with the code sharing postings.  (Certainly I didn't get any feedback praising the addition of code.)  I'd guess that most people prefer to read about the highlights and insights of the software development rather than the nitty-gritty details of the code itself.  And that's understandable.  It's a lot of work to read code and comprehend what it is doing.  Unless you're planning on modifying the code yourself, there may not be much to get out of that.

Overall I rate the map compass experiment a success.  The whole process of writing the blog and the code intertwined was new and I think I got something out of it.  I also enjoyed figuring out how to make the code public and accessible.  And it did bring some clarity to my thinking about code sharing.

And now onward to other things!