Over the long weekend the World Science Fiction Society Business Meeting met several times. It was a marathon, much longer than is usual at Worldcon, because there was a lot to discuss. So much so, in fact, that I’m not going to attempt to cover it all in one post. This post, which is quite long enough, will be devoted solely to measures that affect how the Hugo Awards are conducted. The majority of these measures were intended, in some way or another, to combat the efforts of Pox Day and his Rabid Crypuppies.
Of course I was not actually at any of the meetings. However, video of the meetings has been posted online, and I am particularly indebted to Mx. Rachael Acks for their superb live-blogging of the meetings.
I want to start with the theory that the Puppy Problem will go away because they will get bored of pumping their money into voting when it is clear that they can’t actually game the final ballot and will those always end up getting thumped with a No Award. Pox may think that by putting people like Neil Gaiman on his slate that he can claim to have “won†because so many other people love Neil’s work too, but Neil has made it very clear that he dislikes the Puppy project just as much as anyone else.
An analysis on the Chaos Horizon website makes the claim that the Puppies are already running out of steam because Pox got 437 nominations for Editor: Long Form in the first round of voting, but only 165 first preference votes. They attribute this to large numbers of Puppies having participated in the nominating stage on the strength of the two-year eligibility rule for that part of the voting, but being unwilling to splash an additional $50 on a supporting membership of MidAmeriCon II to be able to vote on the final ballot.
This analysis is somewhat flawed because the voting methods are different. You get five equally weighted votes in the nominations, but have to rank them in the final ballot. There’s no guarantee that every Puppy voter will have put their Glorious Leader in first place when other Puppy Picks are available. Pox was eliminated fairly quickly in Editor: Long Form, so we can’t gauge the level of support he had in preferences. However, in Fanzine, where far fewer people cast ballots, the Castalia House Blog ended up with 298 votes in the final round of redistribution.
Still, 298 is a lot fewer than 437, and I’m not entirely surprised that some of the Puppy supporters have given up. Pox may be able to convince himself that he’s winning, but his fans will be a lot less invested in his infallibility. As the Puppies will have to pony up cash again to nominate next year, we may see rather fewer of them.
I want now to look at a couple of measures that were not specifically aimed at the Puppies, but which are relevant to the way in which the votes are counted and which will feed into the discussion later. Both are measures which received first passage at Sasquan last year and were ratified this year, so they will be in operation in Helsinki.
The first measure removes the requirement for finalists to have at least 5% of the vote. This is the rule that resulted in us having fewer than 5 finalists in Short Story for several years because the nominations were so diverse. One of the arguments for keeping it was that we may end up with lots of finalists because of a big tie for 5th place. That now may not happen, for reasons I shall explain later.
The other measure is the so-called Nominee Diversity motion, which is specifically aimed at the Dramatic Presentation: Short Form category. Basically it means that if there are lots of Doctor Who episodes on the final ballot, they must all be written by different people (or possibly not — see updates below). The salient point here is that WSFS members have got fed up of having a category for TV episodes in which almost every finalist comes from the same show (that’s definitely right). This too has relevance for later discussion.
The first set of specifically anti-Puppy measures to be discussed were those intended to limit participation in the voting pool. Currently you can participate in the nominating stage if you have a membership in the current Worldcon, the previous one, or the next one. There were motions to cut this down to just the current and previous years, and to the current year only. The former was passed (and requires ratification next year), the latter was not.
I’ve been a bit concerned about these motions because I don’t like the idea of us retreating into elitism as an anti-Puppy measure. We should be encouraging participation, not discouraging it. Having seen the debate, I think we did the right thing.
The right to nominate if you are a member of the following year’s Worldcon is a relatively new thing, and it is now opposed by experienced Hugo Administrators who cite problems with the system. There’s no doubt who is a member of the previous year’s Worldcon. No one is going to be joining it after it has happened. Next year’s convention, on the other hand, has a growing membership, and that leads to a requirement to coordinate closely with that event to ensure that everyone who has a right to vote can do so.
That would be OK if it meant that lots of new people were getting to vote, but most of the people who join a Worldcon more than a year in advance are WSFS regulars who join every year. We were told that including next year’s members resulted in the franchise being extended to very few additional people, and it wasn’t worth the effort.
Restricting nominating rights to just the current year is a different matter. From the debate that was very clearly an attempt to make it more expensive to vote. Also, as Lisa Hayes cogently pointed out, many people join Worldcon the current too late to participate in the nominations stage, or even the final ballot. The two year system gives those people a stake in the Hugos, which is a Good Thing.
Well done, Business Meeting, in both of those cases. We seem to have got it right.
Now we get to the serious changes in the voting mechanism. First up was a new proposal, Three Stage Voting (3SV), which I discussed before the convention. At the time most of the negative points I had seen made were about the additional load placed on the Hugo Administrators, which I addressed. At the Business Meeting three new groups of objections emerged.
The first was that the proposed calendar simply didn’t allow enough time for everyone to read and judge 15 semi-finalists in each category. That’s a fair point, and I’d like to see if any adjustments could be made to give people more time. However, it is also true that no one has enough time to read every eligible work before the nominations stage, and yet we still have it, so I’m not sure that this objection holds water.
Some people were also concerned that 3SV could be used by the Puppies to kick deserving finalists off the ballot. I’m not worried about this. The threshold to remove a potential finalist is quite high, and the Puppies have proved twice now that they just don’t have the numbers to game anything other than the nominations.
The final objection was that 3SV is purely a negative proposal designed to kick Puppy Picks off the final ballot. Effectively it is a proposal to move the process of No Awarding forward in time so that we end up with 5 finalists in each category, each of which people are happy do not need to be thumped with a No Award.
No one (expect possibly Pox and his cult followers) particularly likes how contentious the Hugos have become. But using No Award is a wholly negative action and we have been doing that a lot lately. 3SV doesn’t make the process more negative, it just moves the negativity to an earlier stage of the process.
At this point I should note that when I first wrote about 3SV I did not fully understand how it was proposed to work. I apologize profusely for this. Kevin has since put me right, but there was no time to go into the detail before the convention as a complicated amendment would have been required.
I thought that in stage 2 of 3SV we’d be allowed to up-vote works on the long lists as well as down-vote them. It seemed obvious to me that we would want to do that. However, the current proposal only allows for down-voting.
Allowing up-voting as well would have two benefits. Firstly it would allow people to change their votes based on items on the long lists that they had not heard about before. It is by no means unheard of for a work that got the fewest nominations to win the final ballot, because being on the final ballot exposes it to a lot more readers. Allowing up-voting in stage 2 would enhance that effect. In addition, if we had up-voting, the second stage would be much less of a solely Puppy-kicking exercise.
Kevin tells me that the proposers of 3SV decided not to allow up-voting because they feared that people would demand that the Voter Packet include all 15 items in each category on the long list, which would be a massively complex operation. I think that was foolish, for two reasons:
- People will demand to have these works in the Voter Packet anyway. Indeed, someone at the meeting did so.
- As I noted the other week, part of the value of 3SV is crowdsourcing eligibility checking. The items on the long list would not have been checked for eligibility so we have a solid excuse for not putting them all in the Voter Packet.
Talking of eligibility checking, Jonathan Cowie pointed out that the nominations statistics for this year show 83 nominations for “The Coode Street Podcast (Jonathan Strahan)†and separately 66 nominations for “Coode Street Podcast (Wolfe/Strahan)â€. This is an obvious error. Thankfully, even if the two had been combined, the show would not have made the final ballot, but at least one mistake like that has been made in the past that did affect who got to be a finalist. We need extra eyes on eligibility checking.
Obviously allowing up-voting would be a greater change, and therefore can’t be tacked on during the ratification process next year, but I think it is worth doing. More to the point, I think that WSFS members will demand it, once they get to see all of the good stuff on the long lists that they wish they had nominated.
3SV was passed by the meeting, which then proceeded to discuss ratification of E Pluribus Hugo (EPH), the complicated statistical procedure intended to combat slate voting of the type practiced by the Puppies.
Up until recently, we had no idea what effect EPH would have. This year we had actual test data. There are two reports, one looking at the 2014 and 2015 Hugo Awards, and one looking at this year’s data (they also look at the Retro Hugos from 2014 and 2016, but those tests are not as useful because a lot fewer people vote). The reports are quite long, but the salient points are as follows:
- EPH will not get rid of all of the Puppy Picks on the final ballot. At best it will remove one or two.
- EPH will also kick off some non-Puppy works. Indeed, in 2014 it would have kicked off one of the eventual winners.
- The effect of EPH on the Dramatic Presentation categories is currently unknown because it was not tested on them.
The effect of EPH on this year’s ballot was quite encouraging. It didn’t get rid of all of the Puppy Picks, but it added enough good material to allow a contest in almost every category. That is, there were at least two finalists worth voting for in most of the categories. But not all of them. It made no difference to Fan Artist, and in Fanzine Lady Business again finished sixth. Given the validation of EPH, Black Gate may not have withdrawn.
The effect on previous years was less good, and in particular it is notable that EPH made five changes to the final ballot for 2014, a year in which the Puppies were not active. In particular it kicked Galen Dara out of Professional Artist, a category that she went on to win.
What is going on here? You may well ask. Surely the point of EPH is to defeat slate voting. It should not make any difference to the ballots if there were no slates in operation. But it does, because it detects what we might call “natural slatesâ€. That is, if a whole bunch of people happen to nominate the same works, EPH will penalize that, even if no collusion took place. Algorithms have no political bias. They do their job in all cases.
One thing we can say about this is that it has proved the Puppies have a point. While I am satisfied that there was no SJW slate in operation in 2014, nevertheless EPH found that some categories did indeed suffer from “group thinkâ€, and it acted to produce a more diverse set of finalists. I like the idea that Joey Hi-Fi would have been a finalist in 2014.
So I have no objection to the detection of “natural slatesâ€. Politically, however, I suspect it will be a minefield. If, next year, when EPH is used on the actual voting, people who are not on the Puppy slates get eliminated by it, I think that there will be an outcry. Fandom at large is expecting EPH to get rid of all of the Puppies, and no one else. It will not do either. People are not going to be happy.
Another potential issue here is the effect that EPH will have on Helsinki in particular. Finnish fans will presumably want to vote for Finnish works. Because there are a lot fewer Finnish writers than non-Finnish ones, there will be much less diversity in their nominations. I suspect that EPH will see the Finnish votes as a slate and kick some of the nominees off. That too will make some people unhappy, including me.
By far the biggest problem with EPH, however, is its effect on the Dramatic Presentation categories (and my thanks to Martin Easterbrook for alerting me to this). None of the tests presented to the Business Meeting included these categories. Apparently it was too time consuming to do this. So my first question is, what about the poor Hugo Admins for Helsinki who will have to run EPH on those categories? It sounds like they are in for a lot of work.
In addition I note that the Dramatic Presentations are two of the highest profile, and highest participation, categories in the Hugos. If there is a problem with them, there will be serious outrage. If I were being offered a new piece of software and the tests I was shown deliberately excluded two of the most important cases I would have been very worried indeed, and would have sent the people trying to sell me the software packing until they had done all of the tests I wanted.
So what will be the effect of EPH on the Dramatic Presentation categories? I quote from Rachael Acks’ live blog here: “Dramatic presentation will be the most changed category.â€
Why? Because these are the categories that have the least diversity in nominees. To EPH, both Dramatic Presentation categories look like slates have been at work, and it will act to diversify the finalists.
To some extent it will be right. The Dramatic Presentation: Short From category has had slate voting of a sort for as long as it has existed. Fans of each series get together online to decide which episode(s) to nominate. No one has cared too much in the past, partly because it is only one category, partly because the slates are not run by arseholes determined to wreck the Awards, and partly because there are several competing slates each year.
However, as we have seen from the passage of the Nominee Diversity amendment, WSFS members have become fed up of Dramatic Presentation: Short From being dominated by Doctor Who every year, and they have acted to try to stop that. EPH will, I think assist with that process. Which is perhaps a good thing.
Except that the Doctor Who fans won’t like it. If they see some the episodes they nominated getting kicked off by EPH I suspect that complaints will be made. In fact, given the amount of dummy-spitting I have seen because of Jessica Jones winning the category this year, I suspect that very vociferous complaints will be made.
Something else that concerned me about EPH is what was said by the people doing the testing. Jameson Quinn was still in favor of EPH, but it is his baby. Dave McCarty was much more reticent. Dave was one of this year’s Hugo Administrators, he’s a past Worldcon chair, and like Kevin and I he is a member of the Hugo Awards Marketing Committee, the people charged with dealing with any bad publicity that the Hugos get. This is what Dave had to say in the testing report:
The changes to the Ballot and Long list are not easily verified and for people reviewing the detailed results at the end the only way to check that the process is working correctly would require access to secret nomination data and significant time. The difficulty in verification means that to check any result requires time which is NOT available to award administrators when it is time to close the nominating and prepare for the Ballot announcement. These are significant hurdles for a process that is generally designed to be open and democratic.
All of which makes me quite worried about the effect that EPH is going to have. Fortunately there is a clause in the motion implementing it that allows us to suspend it for a year at the Business Meeting. So if things go very badly pear-shaped in Helsinki we can keep voting to suspend it until the BM can vote to remove it again.
Having said all of that, EPH does have one unexpected benefit. If you have looked at the testing reports you will see that, whereas the traditional nomination system always gives integral numbers of votes, EPH gives fractional numbers. That’s a result of the algorithm. This will, I think, make it much more difficult for ties to happen, which will remove one of the major worries over the effect of dropping the 5% rule.
Finally, on Puppy issues, the Business Meeting ratified the 4 and 6 proposal. Or rather they didn’t, because they voted to change it to 5 and 6 and ratified that. Yes, they watered it down, making it less effective as an anti-Puppy measure.
Why would they do that? Well partly because they felt it was safe to do so. They had already passed EPH, so they felt the Puppy threat was being dealt with. 5 and 6 is a very different thing to 4 and 6. It is basically a proposal to increase the number of finalists on the ballot. It is a “share the love more widely†exercise. Business Meeting attendees are a bunch of softies at heart, and they tend to like any proposal that means giving out more prizes. They also dislike having anything taken away from them. 4 and 6 would have reduced the number of works they can nominate in each category, whereas 5 and 6 does not.
Overall I think we have made progress this weekend. I am fairly confident that next year’s final ballot will have at least one, hopefully two, deserving winners in each category, even if Pox and his cult do decide to try to piss in our beer again. But I also expect a fair amount of controversy over the fact that EPH does not do what people expected it to do. It will not get rid of all of the Puppy Picks, and it will kick off some non-Puppy finalists.
It’s the Hugo Awards. There will be drama.
Updates: A couple of small but significant changes here. I mis-spelled Jameson Quinn’s name, for which profuse apologies. Also supporting membership of Mac II was $50 (though it is down to $40 for Helsinki and San José).
The other update is a bit more complicated and pending. The Nominee Diversity motion isn’t as clear as I, with my editor hat on, would like. I read it one way. Kevin, with his knowledge of the intent of the proposers, says it means something different. I need to talk to him more about that. What I did miss is that you can have two similar works on the ballot, not just one.
Normally I get Kevin to proof this stuff, but he’s driving home from Worldcon and so has no time to look at anything in detail.
