Awards, Code
E Pluribus Hugo simulator
by
This is pretty deep in “inside baseball” territory, so if you haven’t been following the Hugo awards controversy you’re probably safe ignoring it.
The short version is, after the Sad Puppies kerfuffle this year that screw up the nominations, the WSFS (which administers the Hugos) voted to adopt a new nominating system called “E Pluribus Hugo”. It’s described over here in detail. If the vote is confirmed at next year’s WorldCon, EPH will be used for the 2017 and future Hugos.
Roughly speaking, it works like this. Everybody votes for up to five works as normal. Each ballot is given one point, which is split among the works nominated (so .2 each if you put five) and all the points from all the ballots are added up. The works with the least number of points (at least two, more in the case of ties) are compared, and the one with the fewest votes (i.e. present on the fewest ballots) is eliminated from contention. Then the points are recounted for another round — if one of your works was eliminated, its points are reallocated to the others on your ballot. (So if you nominated five works, at .2 each, and one was eliminated first round, the remaining four would count for .25 each, and so on.)
The point here is to help consolidate the field and reduce the advantages of slate voting. As I wrote earlier, the problem is that slate voting under the “naïve” rules (where you just add up all the votes) is a dominant strategy. Inasmuch as we agree that the awards SHOULD be about nominating stuff you personally thought was the best of the year, rather than coordinating with some slate, then the nomination rules needed to be amended to favor that “strategy” over coordination.
Important Caveat: I am not a voting theory expert! Smarter people than me have thought about this. However, I am a programmer of sorts, and interested in this stuff. So, I wrote up a thing that runs the EPH algorithm on test data. (I obviously don’t have access to actual Hugo data!) I thought other people might get something out of it, so I’m posting it here.
Here is the EPHConsole project as a Visual Studio ’13 project.
Here is the compiled self-installer for the EPHConsole project.
Here is the EXE file, which should work if you have .NET installed on your machine.
Here is an example data file.
Usage is pretty simple: run “EPHConsole.exe [datafile]”, hit enter when it pauses after displaying the parsed ballots, observe the results. The datafile is text, with each ballot on its own line as a set of integers separated by spaces. (“1 2 3 4 5”) There is no validation that ballots are restricted to five votes, or can’t duplicate votes.
If anybody wants to adapt this code to other purposes, modify it, etc, feel free. Shoot me a line and I’ll put a link here. It’s definitely not optimized (for one thing, I used VS and C#, which is way overkill but let me bang it out in a few hours) but it runs on datasets of realistic size in less than a second, so probably not an issue.
(If you use this and find something that you think is a bug, just send me a description and the data file you were using, and I’ll update accordingly.)
Here’s an example, using the data file above. I generated a random assortment of 800 voters choosing among 40 works, with ballots of varying sizes. Then I added 200 voters choosing among 7 works (the “slate”, numbers 41-47) similarly. It gives you a good sense of how EPH works and how it helps the slate problem.
First, it lists the ballot data, which I’ve snipped here. In Round 1, before any eliminations, you can see that the “non-slate” works have between 16.8 and 23.3 points. (800 points divided among 40 works, clustered around 20. Stats!) The “slate” works have between 26.3 and 31.9 points. (200 points divided among 7 works.) So the slate is at the top of the list. (The list is displaying points/# of votes for each work.)
However, as works at the bottom are eliminated and their points reallocated, the non-slate field converges. By round 9, a non-slate work breaks the top five. In round 15, two slate works are pitted against each other, and one is eliminated, consolidating the slate vote. And so on as the rounds continue.
The final tally shows one slate work, the most popular, along with four non-slate works. This is more or less correct, in the sense that the voters were 80% non-slate and 20% slate; the slate voters were neither completely shut out nor completely dominant. So in that sense, EPH works as advertised!
One concern, though, is that the breadth of the field is important. A run with similar numbers of voters, but 80 non-slate works instead of 40, gives two slate winners (though the naïve method would give ALL the slots to the slate), and with 160 gives four slate winners. This gets pretty deep into the voting system weeds — arguably, with 800 voters split over 160 candidates, the 200 voters focused on their seven deserve to win. (The problem really stems from “exhausted ballots” — among the 800, ballots with no remaining candidates don’t have any say at all. But since we can’t get people to do an exhaustive ranking of all possible Hugo candidates, this is inevitable.)
Anyway, enjoy! You can use the contact form to get in touch with me if you have problems, or if you want to share interesting data. I will update this post if I get stuff or need to fix things.
Round 1
16 16.88333/81
14 16.91666/80
5 17.3/83
10 17.35/84
17 17.48333/84
22 17.6/85
30 17.63333/83
34 17.65/86
24 17.8/85
3 17.98333/86
13 18.18333/86
39 18.28333/86
2 18.51666/88
12 18.58333/88
18 18.95/91
1 19.4/93
26 19.55/93
28 19.58334/94
32 19.65/93
9 19.85/96
4 19.9/96
25 20.41667/95
23 20.41667/96
36 21.06667/99
31 21.25001/102
35 21.40001/100
8 21.40001/102
6 21.53334/101
20 21.56668/103
27 21.56668/102
40 21.85001/104
21 21.86668/104
11 22.06668/103
38 22.13334/105
37 22.20001/107
33 22.33334/107
19 22.50001/107
7 22.78335/107
15 23.23335/108
29 23.36668/111
43 26.31668/98
41 26.78334/99
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 16.91666
Eliminating 14.
Round 2
16 17.23333/81
5 17.45/83
10 17.65/84
22 17.8/85
17 17.83333/84
34 17.9/86
30 18.05/83
24 18.26666/85
3 18.33333/86
13 18.6/86
39 18.75/86
12 19.05/88
2 19.08333/88
18 19.35/91
1 19.71667/93
26 19.75/93
28 20.11667/94
9 20.18334/96
32 20.2/93
4 20.28334/96
23 20.70001/96
25 21.53334/95
8 21.73334/102
36 21.81668/99
20 21.81668/103
31 21.83334/102
27 21.83334/102
6 21.91667/101
35 21.96667/100
40 22.33334/104
38 22.43334/105
21 22.45001/104
37 22.53334/107
19 22.85001/107
33 22.86668/107
11 22.93334/103
7 23.18335/107
15 23.66668/108
29 24.00001/111
43 26.31668/98
41 26.78334/99
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 17.45
Eliminating 16.
Round 3
22 18.03333/85
10 18.05/84
5 18.06666/83
17 18.16666/84
34 18.23333/86
30 18.48334/83
24 18.51667/85
39 18.9/86
3 18.98334/86
2 19.33333/88
12 19.33333/88
13 19.38333/86
18 19.71667/91
1 20.21667/93
26 20.35/93
9 20.51667/96
32 20.51667/93
28 20.55/94
4 20.63334/96
23 21.28334/96
25 21.73334/95
35 22.11667/100
8 22.25001/102
31 22.31667/102
6 22.41667/101
27 22.41668/102
20 22.46668/103
36 22.58334/99
40 22.66667/104
21 22.73335/104
38 23.25001/105
37 23.25001/107
33 23.26668/107
11 23.30001/103
19 23.45001/107
7 23.85001/107
15 24.10001/108
29 24.56668/111
43 26.31668/98
41 26.78334/99
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 18.05
Eliminating 10.
Round 4
5 18.5/83
22 18.5/85
34 18.65/86
17 18.76666/84
30 18.8/83
24 18.81667/85
39 19.23334/86
3 19.26667/86
2 19.63334/88
13 19.83333/86
18 19.96667/91
12 19.98334/88
26 20.55/93
1 20.68334/93
4 20.93334/96
32 21.01667/93
9 21.13334/96
28 21.25001/94
23 21.78334/96
25 22.06668/95
35 22.35001/100
8 22.75001/102
6 22.90001/101
20 23.08334/103
31 23.08334/102
36 23.13334/99
27 23.30001/102
21 23.33335/104
40 23.35001/104
37 23.63334/107
33 23.68335/107
38 23.80001/105
19 23.88334/107
11 23.91668/103
7 24.60001/107
15 24.76668/108
29 25.06668/111
43 26.31668/98
41 26.78334/99
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 18.5
Eliminating 5.
Round 5
22 18.83334/85
34 19.18334/86
30 19.21667/83
24 19.26667/85
17 19.4/84
39 19.5/86
3 19.98334/86
2 20.2/88
13 20.33333/86
18 20.33334/91
12 20.43333/88
1 21.15/93
26 21.23334/93
9 21.46667/96
32 21.50001/93
4 21.63334/96
28 21.66667/94
25 22.48334/95
23 22.61667/96
35 22.66668/100
8 23.00001/102
6 23.23334/101
36 23.55001/99
31 23.65001/102
21 23.70001/104
20 23.80001/103
40 23.93334/104
37 23.96667/107
27 24.15001/102
33 24.18335/107
11 24.28334/103
38 24.38334/105
19 24.55001/107
7 25.31668/107
15 25.35002/108
29 25.85001/111
43 26.31668/98
41 26.78334/99
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 19.18334
Eliminating 22.
Round 6
30 19.71667/83
34 19.73334/86
17 19.81667/84
24 19.91667/85
39 20.26667/86
3 20.51667/86
2 20.7/88
13 20.85/86
12 20.91667/88
18 21.13334/91
1 21.55/93
26 21.73334/93
4 21.86667/96
32 21.86667/93
28 21.96667/94
9 22.25001/96
25 22.71667/95
23 22.86667/96
35 23.20001/100
8 23.28334/102
6 23.95001/101
36 23.98335/99
21 24.25001/104
31 24.45001/102
37 24.53334/107
20 24.55001/103
40 24.61667/104
27 24.66668/102
38 24.73334/105
33 24.76668/107
19 24.91668/107
11 25.00001/103
15 25.81668/108
7 26.25001/107
43 26.31668/98
29 26.65001/111
41 26.78334/99
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 19.73334
Eliminating 30.
Round 7
34 20.21667/86
17 20.55/84
39 20.55/86
24 20.58334/85
3 20.91667/86
13 20.98334/86
2 21.10001/88
18 21.7/91
12 21.7/88
4 22.11667/96
26 22.28334/93
1 22.31668/93
28 22.33334/94
9 22.66667/96
32 22.68334/93
23 23.18334/96
25 23.40001/95
35 23.70001/100
8 23.70001/102
6 24.65001/101
36 24.71668/99
21 24.83334/104
37 24.90001/107
31 25.01668/102
40 25.08334/104
38 25.33334/105
19 25.36668/107
27 25.55001/102
11 25.61668/103
33 25.65001/107
20 25.78335/103
43 26.31668/98
7 26.70001/107
41 26.78334/99
15 26.96668/108
29 27.15001/111
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 20.55
Eliminating 17.
Round 8
34 20.56667/86
39 21.08334/86
3 21.13334/86
2 21.41667/88
24 21.43334/85
13 21.9/86
18 22.33334/91
1 22.58334/93
12 22.78334/88
4 22.88334/96
26 22.96668/93
9 23/96
28 23.00001/94
32 23.16667/93
23 23.61667/96
25 23.91668/95
8 24.23334/102
35 24.35001/100
6 25.06668/101
36 25.16668/99
21 25.33334/104
31 25.48335/102
19 25.63335/107
40 25.78334/104
37 25.86668/107
38 25.95001/105
11 26.26668/103
43 26.31668/98
33 26.38334/107
20 26.43335/103
27 26.51668/102
41 26.78334/99
7 27.81668/107
15 27.96668/108
29 27.96668/111
47 28.05001/104
42 28.06668/101
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 21.08334
Eliminating 34.
Round 9
3 21.66667/86
39 21.8/86
24 21.95/85
2 22.25001/88
13 22.88334/86
1 22.95001/93
18 23.13334/91
28 23.36668/94
26 23.48334/93
12 23.51667/88
4 23.63334/96
9 23.70001/96
32 23.76667/93
25 24.45001/95
23 24.48334/96
35 24.58334/100
8 24.73334/102
6 25.63334/101
19 25.93335/107
36 26.01668/99
21 26.05001/104
43 26.31668/98
37 26.38334/107
31 26.41668/102
40 26.45001/104
33 26.75001/107
41 26.78334/99
11 26.85001/103
38 26.88335/105
20 26.96668/103
27 27.23335/102
47 28.05001/104
42 28.06668/101
15 28.65001/108
29 28.70001/111
7 28.73335/107
44 28.83335/107
46 30.05002/113
45 31.90002/118
Threshhold: 21.8
Eliminating 3.
Round 10
24 22.51667/85
39 22.68334/86
2 23.13334/88
1 23.31668/93
18 23.40001/91
13 23.45001/86
28 23.50001/94
12 23.91667/88
26 24.10001/93
9 24.35001/96
4 24.38334/96
32 24.48334/93
23 25.00001/96
25 25.00001/95
35 25.08334/100
8 25.93334/102
43 26.31668/98
19 26.40001/107
36 26.63334/99
6 26.68334/101
41 26.78334/99
21 26.80001/104
40 27.13334/104
31 27.20001/102
37 27.28334/107
33 27.38334/107
11 27.51668/103
38 27.75001/105
27 27.85002/102
20 27.96668/103
47 28.05001/104
42 28.06668/101
44 28.83335/107
15 29.45001/108
7 29.58335/107
46 30.05002/113
29 30.11668/111
45 31.90002/118
Threshhold: 22.68334
Eliminating 24.
Round 11
39 23.53334/86
2 23.60001/88
18 23.91667/91
28 23.96668/94
1 24.01668/93
26 24.58334/93
13 24.71668/86
4 25.06668/96
9 25.08334/96
32 25.16667/93
25 25.26668/95
23 25.71667/96
35 25.71668/100
12 25.76668/88
43 26.31668/98
41 26.78334/99
8 26.88334/102
19 26.91668/107
6 27.03334/101
36 27.28335/99
21 27.33334/104
40 27.73334/104
37 27.88334/107
47 28.05001/104
31 28.05001/102
42 28.06668/101
38 28.45001/105
33 28.48334/107
27 28.55001/102
44 28.83335/107
20 28.91668/103
11 29.16668/103
15 29.86668/108
46 30.05002/113
7 30.31668/107
29 31.01668/111
45 31.90002/118
Threshhold: 23.60001
Eliminating 39.
Round 12
2 24.20001/88
28 24.43334/94
18 24.56667/91
1 24.68335/93
26 25.21667/93
32 25.50001/93
13 25.68334/86
4 25.83334/96
9 25.90001/96
25 26.21668/95
43 26.31668/98
23 26.36668/96
35 26.68334/100
41 26.78334/99
12 27.36668/88
8 27.50001/102
6 27.63334/101
19 27.75001/107
36 27.96668/99
21 28.01668/104
40 28.03334/104
47 28.05001/104
42 28.06668/101
37 28.70001/107
44 28.83335/107
31 28.98335/102
33 29.26668/107
27 29.40001/102
38 29.40001/105
20 29.83334/103
46 30.05002/113
15 30.30001/108
11 30.68334/103
7 31.38335/107
29 31.50001/111
45 31.90002/118
Threshhold: 24.43334
Eliminating 2.
Round 13
18 25.18334/91
1 25.41668/93
28 25.60001/94
26 25.95/93
43 26.31668/98
32 26.48334/93
13 26.56668/86
41 26.78334/99
4 26.86668/96
25 26.88334/95
23 26.90001/96
9 26.91667/96
35 27.46667/100
47 28.05001/104
42 28.06668/101
8 28.15001/102
12 28.23334/88
36 28.61668/99
44 28.83335/107
6 28.88334/101
19 28.96668/107
40 28.98334/104
21 29.13334/104
31 29.60001/102
37 29.63335/107
46 30.05002/113
33 30.16668/107
38 30.38335/105
27 30.60001/102
20 30.63334/103
15 30.93335/108
11 31.06668/103
45 31.90002/118
7 32.35001/107
29 32.43335/111
Threshhold: 25.41668
Eliminating 18.
Round 14
28 26.03334/94
43 26.31668/98
1 26.50001/93
41 26.78334/99
26 26.8/93
13 27.15001/86
9 27.26667/96
32 27.31667/93
47 28.05001/104
42 28.06668/101
23 28.33334/96
4 28.40001/96
25 28.43334/95
35 28.65001/100
8 28.80001/102
44 28.83335/107
19 29.25001/107
6 29.55001/101
36 29.60001/99
40 29.61668/104
21 29.63334/104
46 30.05002/113
31 30.26668/102
37 30.61668/107
12 30.81668/88
27 30.90001/102
33 31.35001/107
15 31.41668/108
38 31.61668/105
20 31.65001/103
11 31.88334/103
45 31.90002/118
29 33.25001/111
7 33.90001/107
Threshhold: 26.31668
Eliminating 28.
Round 15
43 26.31668/98
41 26.78334/99
1 27.30001/93
26 27.73334/93
13 27.95001/86
47 28.05001/104
42 28.06668/101
9 28.08334/96
44 28.83335/107
25 28.98335/95
23 29.13335/96
32 29.30001/93
4 29.38334/96
35 29.91668/100
46 30.05002/113
8 30.25001/102
36 30.41668/99
21 30.46668/104
6 30.48335/101
19 30.70001/107
40 30.75001/104
12 31.41668/88
31 31.63335/102
37 31.66668/107
45 31.90002/118
27 31.96668/102
33 32.21668/107
38 32.25002/105
20 32.45001/103
11 32.51668/103
15 32.80001/108
29 34.21668/111
7 35.01668/107
Threshhold: 26.78334
Eliminating 43.
Round 16
1 27.30001/93
26 27.73334/93
13 27.95001/86
9 28.08334/96
25 28.98335/95
23 29.13335/96
32 29.30001/93
4 29.38334/96
41 29.80001/99
35 29.91668/100
8 30.25001/102
36 30.41668/99
21 30.46668/104
6 30.48335/101
19 30.70001/107
40 30.75001/104
12 31.41668/88
31 31.63335/102
37 31.66668/107
47 31.81668/104
27 31.96668/102
33 32.21668/107
38 32.25002/105
20 32.45001/103
11 32.51668/103
42 32.58334/101
15 32.80001/108
44 33.06668/107
29 34.21668/111
46 34.46668/113
7 35.01668/107
45 38.26667/118
Threshhold: 27.73334
Eliminating 1.
Round 17
26 28.35001/93
9 28.78334/96
13 29.21667/86
23 29.61668/96
41 29.80001/99
25 30.26668/95
4 30.31668/96
35 30.61668/100
8 30.81668/102
32 31.16668/93
40 31.20001/104
6 31.21668/101
36 31.23334/99
47 31.81668/104
21 31.85002/104
19 31.93335/107
12 32.20001/88
31 32.43334/102
42 32.58334/101
38 32.90002/105
33 32.91668/107
44 33.06668/107
37 33.13335/107
27 33.20001/102
15 34.05001/108
11 34.25001/103
46 34.46668/113
20 34.88335/103
29 35.15001/111
7 37.30001/107
45 38.26667/118
Threshhold: 28.78334
Eliminating 26.
Round 18
13 29.56668/86
41 29.80001/99
9 30.00001/96
23 30.83335/96
4 30.95001/96
35 31.63334/100
32 31.63334/93
25 31.78334/95
47 31.81668/104
8 31.86668/102
40 32.00001/104
6 32.48334/101
21 32.56668/104
42 32.58334/101
36 32.63335/99
12 32.91667/88
31 33.06667/102
44 33.06668/107
19 33.38335/107
38 33.40002/105
33 34.21667/107
46 34.46668/113
37 34.51668/107
27 34.95001/102
11 35.58334/103
15 35.85001/108
20 36.21668/103
29 36.56668/111
45 38.26667/118
7 39.38333/107
Threshhold: 29.80001
Eliminating 13.
Round 19
41 29.80001/99
9 31.11667/96
23 31.56668/96
47 31.81668/104
4 31.98335/96
32 32.01668/93
35 32.18334/100
42 32.58334/101
25 32.75001/95
44 33.06668/107
40 33.33334/104
6 33.35001/101
21 33.35001/104
12 33.71667/88
36 34.05001/99
8 34.06667/102
31 34.08334/102
19 34.43335/107
46 34.46668/113
33 35.38334/107
11 35.78334/103
38 35.78334/105
27 36.28334/102
15 36.73334/108
20 37.01668/103
37 37.63334/107
29 38.15002/111
45 38.26667/118
7 40.23333/107
Threshhold: 31.11667
Eliminating 9.
Round 20
41 29.80001/99
47 31.81668/104
42 32.58334/101
23 32.70002/96
35 33.01668/100
44 33.06668/107
32 33.15001/93
25 33.58334/95
4 33.70001/96
46 34.46668/113
6 34.68334/101
21 34.70001/104
8 34.90001/102
12 35.15001/88
40 35.25/104
36 35.41667/99
31 35.58334/102
19 35.93335/107
33 36.35/107
27 37/102
38 37.16667/105
11 37.18334/103
15 37.7/108
20 37.90001/103
45 38.26667/118
37 38.61667/107
29 40.86668/111
7 42.44999/107
Threshhold: 31.81668
Eliminating 41.
Round 21
23 32.70002/96
35 33.01668/100
32 33.15001/93
25 33.58334/95
4 33.70001/96
6 34.68334/101
21 34.70001/104
8 34.90001/102
12 35.15001/88
40 35.25/104
36 35.41667/99
31 35.58334/102
19 35.93335/107
33 36.35/107
27 37/102
38 37.16667/105
11 37.18334/103
47 37.33334/104
42 37.66666/101
15 37.7/108
20 37.90001/103
44 38.41667/107
37 38.61667/107
46 40.75/113
29 40.86668/111
7 42.44999/107
45 45.83332/118
Threshhold: 33.01668
Eliminating 23.
Round 22
35 34.81667/100
4 35.25001/96
21 35.53334/104
32 35.58334/93
8 35.7/102
12 35.7/88
25 35.75/95
36 36.56667/99
40 36.71667/104
19 36.76668/107
6 36.93333/101
47 37.33334/104
31 37.5/102
42 37.66666/101
33 37.9/107
27 38.08334/102
11 38.2/103
44 38.41667/107
38 38.73333/105
15 39.55/108
20 39.95001/103
37 40.38333/107
46 40.75/113
29 42.35001/111
7 45.03333/107
45 45.83332/118
Threshhold: 35.25001
Eliminating 4.
Round 23
32 36.16668/93
35 36.45/100
12 36.95/88
8 37.21667/102
47 37.33334/104
36 37.48334/99
42 37.66666/101
21 37.7/104
25 37.76667/95
19 38.03335/107
44 38.41667/107
6 38.43334/101
40 39.06667/104
27 39.16668/102
31 39.25/102
11 39.3/103
33 40.53333/107
46 40.75/113
15 40.98333/108
38 41.2/105
20 41.63334/103
37 42.39999/107
29 43.26667/111
45 45.83332/118
7 47/107
Threshhold: 36.45
Eliminating 32.
Round 24
47 37.33334/104
12 37.61666/88
42 37.66666/101
8 38.05/102
44 38.41667/107
35 38.45/100
36 38.9/99
25 39.6/95
19 39.78334/107
21 39.86668/104
6 40.1/101
40 40.23333/104
27 40.5/102
31 40.58333/102
11 40.71667/103
46 40.75/113
33 41.7/107
38 42.69999/105
15 43.23333/108
20 43.88334/103
37 43.89999/107
45 45.83332/118
29 46.35/111
7 48.83333/107
Threshhold: 37.61666
Eliminating 12.
Round 25
47 37.33334/104
42 37.66666/101
8 38.38333/102
44 38.41667/107
46 40.75/113
36 40.78333/99
35 40.88333/100
25 41.01667/95
19 41.31667/107
40 41.66666/104
27 42.08333/102
21 42.08333/104
33 42.53333/107
6 42.75/101
31 42.96666/102
38 43.96666/105
15 43.98333/108
11 44.18333/103
20 45.03334/103
37 45.33332/107
45 45.83332/118
29 48.31667/111
7 49.71666/107
Threshhold: 37.66666
Eliminating 42.
Round 26
8 38.38333/102
36 40.78333/99
35 40.88333/100
25 41.01667/95
19 41.31667/107
40 41.66666/104
27 42.08333/102
21 42.08333/104
33 42.53333/107
6 42.75/101
31 42.96666/102
38 43.96666/105
15 43.98333/108
11 44.18333/103
20 45.03334/103
37 45.33332/107
47 45.49999/104
44 46.83332/107
29 48.31667/111
7 49.71666/107
46 49.83332/113
45 56.83332/118
Threshhold: 40.78333
Eliminating 36.
Round 27
8 39.54999/102
35 41.96666/100
25 42.6/95
19 42.78334/107
27 43.41666/102
40 43.71667/104
21 44/104
6 44.83334/101
33 45.03333/107
31 45.38333/102
47 45.49999/104
15 45.94999/108
38 46.46665/105
44 46.83332/107
11 47.18332/103
20 47.28333/103
37 48/107
46 49.83332/113
29 50.86666/111
7 50.96666/107
45 56.83332/118
Threshhold: 41.96666
Eliminating 35.
Round 28
8 41.18333/102
25 43.93333/95
47 45.49999/104
27 45.91666/102
19 46.04999/107
31 46.13333/102
6 46.34999/101
40 46.35/104
21 46.76666/104
44 46.83332/107
20 48.25/103
15 48.36665/108
38 48.84998/105
37 49.46666/107
33 49.58332/107
46 49.83332/113
11 50.09999/103
7 52.43333/107
29 53.26665/111
45 56.83332/118
Threshhold: 43.93333
Eliminating 25.
Round 29
8 44.48333/102
47 45.49999/104
44 46.83332/107
31 47.28333/102
19 47.89998/107
27 48.33333/102
6 48.64999/101
21 48.73333/104
40 48.98333/104
20 49.33333/103
46 49.83332/113
38 49.84998/105
15 49.86665/108
37 51.68333/107
33 53.24999/107
11 53.48332/103
7 53.89999/107
29 55.26665/111
45 56.83332/118
Threshhold: 45.49999
Eliminating 8.
Round 30
47 45.49999/104
44 46.83332/107
46 49.83332/113
6 50.06665/101
31 50.08332/102
19 50.73331/107
38 50.93332/105
27 50.93332/102
40 51.23333/104
21 51.53333/104
20 51.96667/103
15 52.69999/108
37 53.78333/107
11 55.78332/103
33 55.91666/107
45 56.83332/118
7 57.06665/107
29 57.26665/111
Threshhold: 46.83332
Eliminating 47.
Round 31
6 50.06665/101
31 50.08332/102
19 50.73331/107
38 50.93332/105
27 50.93332/102
40 51.23333/104
21 51.53333/104
20 51.96667/103
15 52.69999/108
37 53.78333/107
11 55.78332/103
33 55.91666/107
7 57.06665/107
29 57.26665/111
44 57.83332/107
46 63.33332/113
45 69.83333/118
Threshhold: 50.08332
Eliminating 6.
Round 32
31 51.58332/102
27 52.81666/102
20 52.9/103
21 52.94999/104
38 53.34998/105
15 54.19999/108
40 54.33332/104
19 54.39999/107
11 57.78332/103
44 57.83332/107
37 58.74999/107
29 59.69999/111
33 59.83332/107
7 60.39998/107
46 63.33332/113
45 69.83333/118
Threshhold: 52.81666
Eliminating 31.
Round 33
20 55.15/103
15 56.11665/108
27 56.14999/102
38 56.18332/105
21 56.95/104
19 56.98332/107
44 57.83332/107
40 58.58332/104
11 60.78332/103
37 61.33332/107
33 63.16666/107
29 63.2/111
46 63.33332/113
7 63.39998/107
45 69.83333/118
Threshhold: 56.11665
Eliminating 20.
Round 34
44 57.83332/107
27 59.24999/102
19 59.86666/107
21 60.28333/104
15 60.86665/108
38 61.53333/105
40 62.08332/104
46 63.33332/113
11 63.78332/103
37 63.91666/107
29 65.24999/111
33 65.5/107
7 65.66666/107
45 69.83333/118
Threshhold: 59.24999
Eliminating 27.
Round 35
44 57.83332/107
46 63.33332/113
21 63.53333/104
19 63.69999/107
38 64.53333/105
40 65.58334/104
15 65.69999/108
37 66.16666/107
11 67.2/103
29 68.24999/111
7 68.41666/107
33 68.91666/107
45 69.83333/118
Threshhold: 63.33332
Eliminating 44.
Round 36
21 63.53333/104
19 63.69999/107
38 64.53333/105
40 65.58334/104
15 65.69999/108
37 66.16666/107
11 67.2/103
29 68.24999/111
7 68.41666/107
33 68.91666/107
46 84/113
45 89/118
Threshhold: 63.69999
Eliminating 21.
Round 37
19 67.08333/107
40 68.41666/104
38 68.91666/105
37 69.66666/107
15 70.25/108
11 70.66666/103
29 70.83334/111
7 72.16666/107
33 72.99999/107
46 84/113
45 89/118
Threshhold: 68.41666
Eliminating 40.
Round 38
19 70.58333/107
37 71.66666/107
29 74/111
15 74.41666/108
11 74.41667/103
38 75.08333/105
33 75.16666/107
7 76.66666/107
46 84/113
45 89/118
Threshhold: 71.66666
Eliminating 19.
Round 39
37 75.49999/107
38 77.83333/105
11 78/103
33 78.66667/107
15 79.83334/108
29 80.33334/111
7 80.83334/107
46 84/113
45 89/118
Threshhold: 77.83333
Eliminating 38.
Round 40
37 79.66666/107
33 82.16667/107
11 82.16667/103
46 84/113
15 84.66666/108
29 84.66667/111
7 84.66667/107
45 89/118
Threshhold: 82.16667
Eliminating 11.
Round 41
37 83.83333/107
46 84/113
33 86/107
15 87.16666/108
7 88.16666/107
29 88.83333/111
45 89/118
Threshhold: 84
Eliminating 37.
Round 42
46 84/113
45 89/118
33 91.16666/107
15 91.66666/108
7 93.16666/107
29 95/111
Threshhold: 89
Eliminating 46.
Final Results:
33 91.16666/107
15 91.66666/108
7 93.16666/107
29 95/111
45 118/118
So you were linked to by a comment on File770.
And I’m glad you are playing around with the EPH algorithm. I wonder, though, how it would do on a more realistic, but still parametrized, nomination generator. You would expect to have some correlation between different nominating ballots, otherwise recommendation engines wouldn’t work at all.
I sometimes think something like the following might work: each work and each voter is a random vector in n-dimensional space (multivariate normal, if you like). Each voter nominates the 5 works that have the greatest inner product with his position.
That’s a really interesting idea for generating data, but I suspect implementing it is beyond my powers!
Something a little simpler would be to generate the number of nominations along a power law distribution — using a single random gives you a bell curve, which means few works with a few and many nominations, and many works with about average. A power law (yielding a few leaders and increasingly many works further down) would probably be more realistic.
Hopefully, we can eventually have some real data to test on!
I think this is a fantastic idea. What would be really great is if you could get last year’s actual vote data and run it through your program. That way we could see how the new rules would have affected the nomination process.
Thanks! I would love to, but it’s looking unlikely that the Hugo people will release that data — the problem is privacy concerns, since you’d need the actual ballots.
I wrote a node.js web app with the ’84 data here:https://hugoslatesimulator.herokuapp.com/
It works well to show how superior EPH is when dealing with slates, even with the dirty state of the data file.
I sure hope they figure out a way to anonymize the current data. It’d probably be much more relevant to test against.
Very nice! Yeah, unfortunately anonymization can be difficult. I’m hoping that the Hugo people can just run EPH (using my code or whoever’s) and post the results.
Just found this commentary, and thanks for the analysis. I was appalled at the puppies, and consider it more a scandal than feature. I hope something like this is adopted.