Amber Eye is an engine which plays Slay the Spire Ironclad A20H runs. In a sample of 100 games, Amber Eye won 90, including a streak of 19. The engine was trained using AlphaZero-style techniques: a neural network starts with almost no prior knowledge of game mechanics and learns to win through playing games and observing their outcomes. Amber Eye was trained from scratch on my desktop computer for about one month, playing about 13.8 million games over that time. Its playstyle is not quite like any other player I have watched, and I believe that the ideas exhibited by its gameplay are worth further study.
Why build an engine for Slay the Spire?
I have uploaded the full set of 100 sample games for viewing. Below are a few of the games I found interesting.
The 100-game sample was played with the engine controlling a real game instance, using a modified version of CommunicationMod. The engine was given a search budget of 262k positions per move (about 30-60 minutes per game).
The engine does not have knowledge of the seed or RNG state*1. I have ensured this to the best of my ability by only sharing visible or previously revealed state from the game with the engine. Formally, the model aims to learn to play “ideal Spire”, an abstract version of the game where random outcomes are drawn truly at random, instead of from a seeded RNG. This is the version that players generally agree is more interesting (optimally solving “real Spire” would involve reverse engineering the seed). Both the training and evaluation settings use the RNGFix mod; otherwise, the model would be able to learn certain aspects of correlated RNG.
My first reaction when seeing the engine’s gameplay was that it constantly made mistakes. My second reaction was that it wins a lot despite this, so something interesting is happening. I am a “mediocre” player myself (~30-40% Ironclad A20H winrate) so I am not particularly qualified to analyze its macro, but I will do my best here to describe its overall approach to the game.
The engine appears to aim for hard forcing looping decks in the endgame (exhaust down and play many Pommel, Shrug, Rage, Dropkick etc.), although it does sometimes incorporate alternate win conditions (Snecko, strength scaling, Barricade). It is perfectly happy to play greedily to enable looping strategies, e.g. by picking Dark Embrace on floor 1 or buying The Abacus in the Act 1 early shop. Notably, it is comfortable using Havoc or Sever Soul as core components of its exhaust engine and is flexible about how it gets its loop payoff, opening more paths to building this type of deck.
The engine’s micro ability is usually rather terrible. It bleeds HP and wastes damage or potions for no reason, often multiple times per run. For example, in run 51 of the sample, it manages to throw a very strong position entering the Heart fight, with the help of a bad draw order. (I replayed the fight and won easily.) Occasionally, however, the model is able to pull off some flashy lines.
Overall, the general ideas are quite similar to the approaches used by good players, but the details look very off in a way that I’m not able to understand. Watching the engine’s gameplay, I was very surprised that it was capable of reaching such a high winrate.
The engine gives estimated winning probabilities for each position, which we can use to generate Objectively Correct™ rankings for all items in the game. We can directly measure the engine-estimated impact of each choice on winrate from a sample of logged games.
For the charts below, we define the value of a choice as the difference in win probability in percentage points between that choice and the next best option (or skipping), or 0 if it is not the best choice. The charts are sorted by average value. The values were measured on ~43 000 self-play games played at the end of the training run. Due to being played with a lower search budget, they have a slightly lower winrate (~85%) than the runs in the sample. The lines on the chart surrounding the dots represent the 95% confidence interval.
The engine really likes option 3, as long as it is not rare colorless or max HP. The downsides are generally rated as lose max HP > take damage > lose gold > curse. If option 3 is not good, option 2 is usually pickable.
Boss swap is rated very poorly by the engine, never being picked and costing about 7-8 percentage points of winrate. Most swaps were rated as negative; only Pandora’s Box, Astrolabe, Runic Pyramid, and Fusion Hammer were clearly positive.
On the two-option blessing (Lament vs +10% max HP), the engine picks Lament almost every time, but only slightly favours it in terms of value. On average, the two-option blessing is rated about 2 percentage points worse for winrate than the full blessing.
All the premium exhaust cards are highly rated, as expected given the engine’s playstyle. Offering is in a tier of its own. A significant portion of the card pool is rated as garbage and never picked.
The floor 1 card ratings don’t appear too out of the ordinary. The biggest surprises for me are Sever Soul and Reckless Charge as high as they are, Hemokinesis relatively far down the list, and Fire Breathing being picked a significant fraction of the time.
In the sample runs, the engine appears to be significantly more willing to pick the garbage attack commons early than indicated on this chart. In floors 1-6, Cleave was picked 4 times, Wild Strike 6 times, Thunderclap 4 times, and Sword Boomerang 2 times, which is roughly double the rate expected from the self-play games. This is the only major macro discrepancy I have noticed between the sample and the self-play games. A possible explanation is that the larger search in the sample looks farther ahead into the next hallway fights and sees the value of the attacks. Given this difference only appears with a deep search, the impact on winrate is likely marginal.
Colorless cards have a much smaller sample size. Because the engine generally doesn’t like picking rare colorless from Neow, most of the rare card picks come from Sensory Stone, which probably underrates them in terms of strength (by Act 3, most runs are in a strong position and have less room to increase their win chance). Notably, Hand of Greed and The Bomb are not rated highly, which likely explains why it dislikes the Neow option.
The three books are highly rated, in agreement with how often the engine will acquire them. The HP spent to get them is not included in this chart’s rating, which inflates their placement a bit. Pocketwatch far outclasses the rest of the non-boss relics. Dead Branch is picked only 80% of the time but is still near the top of the chart, while Bottled Flame is skipped 80% of the time at the bottom. Horn Cleat, Toxic Egg, Bag of Preparation, and Centennial Puzzle outperform their rarity.
To no one’s surprise, Pyramid is literally off the charts in terms of value. On average, getting Pyramid is the highest-impact choice in the whole run. Unexpectedly to me, Pandora’s Box and Runic Cube are the next-highest rated. Generally, the engine is quite happy to pick Runic Cube even from the Act 1 boss reward. It also doesn’t care too much for energy: 31 of the games in the sample were played with no energy boss relic.
Fairy is rated far above all the other potions. This is maybe not surprising; given the engine’s horrible potion management, it is better able to use the undrinkable Fairy in contrast to its tendency to waste the other drinkable potions. The Fairy is used both as a short-term resource to be spent and as Act 4 insurance. It is also a high-priority purchase in shops.
Many shop relics are highly valued, especially The Abacus, which often turns into the core of the deck’s block engine. Despite a model limitation where the engine’s visibility into the deck order is limited to the top two cards, Frozen Eye is still often purchased.
Shop card purchase rates roughly track their general rating. The engine is quite willing to purchase removes even in Act 1.
Colorless card purchases are more niche.
Potions are frequently bought (and thrown away on the next floor). Fairy is the top-priority purchase in the shop on average.
For Scrap Ooze, the engine clicks at least once about 96% of the time. Most of the time, it will click at least 3-4 times before giving up, depending on how much HP it has. Interestingly, it often skips the Golden Idol, although this choice probably does not impact winrate significantly. It never takes the upgrades in Shining Light; generally, it does not rate upgraded basics highly.
Notably, the engine virtually never upgrades all basics in Ancient Writing, which is generally considered to be one of the strongest Act 2 event options. In the context of the engine being very remove-pilled, however, this makes a bit more sense. It also rarely takes Apparitions and Bites. JAX tends to be taken with Runic Cube. In The Joust, the engine tragically has little faith in Noodles.
For the Mysterious Sphere, the engine is usually in a strong position entering Act 3, so it often doesn’t consider the risk of fighting the double Orb Walkers worth it. Mark of the Bloom is never taken. It also takes the Madnesses a surprising amount. I don’t see the vision, as the Madnesses seem to typically not perform well in its decks, but who am I to judge.
The engine actually chooses to trade faces sometimes, which seems terrible to me, but maybe it doesn’t rate the bad faces as badly.
HP lost in this chart refers to average net HP change, including healing from Burning Blood, Reaper etc.
The engine doesn’t tend to take the Dead Adventurer Lagavulin fight in especially weak positions, as the chart might suggest. Taking the fight too often may be one of the engine’s blind spots. More generally, the engine seems to prefer spreading out the risk more throughout the course of the run, instead of concentrating it in Act 4.
“Never” covers games the engine loses.
The engine prefers to take the keys notably earlier than most players, although this probably has a small overall impact on winrate. In a large majority of games, it has already Recalled before entering Act 3.
The rates here denote the percentage of time the option is taken versus the total number of rest site visits. For Lift, Dig, and Toke, the percentage is over the rest sites where the relic is owned.
The engine doesn’t mind resting a lot and generally only values a few key upgrades (Pommel Strike, Feel No Pain, Body Slam, Havoc, Offering etc.) When it owns a rest site relic, it usually makes use of it.
Now, on to the technical details of how I built the engine (with a bit of Spire trivia thrown in). If you are not a giga nerd who is interested in the nitty gritty of how to build such an engine, you can probably stop reading here or after the next couple of sections. Check out the runs from the sample, and I will be working on extending the engine to the other three characters.
The AlphaZero paper demonstrated top-performing engines for chess, go, and shogi, but more importantly provided a general-purpose method to train computers to play a relatively broad family of games (discrete action space, two-player, deterministic, perfect information). The recipe works roughly as follows.
Apart from very simple games, it is impossible to exhaustively enumerate each game state and derive a perfect strategy from it. Thus, since very early on, game-playing programs have combined position evaluation (estimate the value of the current state) and search to improve the evaluation (simulate playing some moves to figure out the best line and get a more accurate judgement of the position). Historically, the main research areas have been to make the value estimation more accurate and to improve the search algorithm. AlphaZero’s contribution is to bundle value and search into a self-improving loop driven by the learning of a neural network.
More specifically, the AlphaZero network takes as input a game position and predicts both its estimated value and a policy, a probability distribution over the legal moves from the position, indicating what the network thinks is the best move. The policy can be incorporated into the search algorithm to concentrate on moves which the network thinks are good. The model generates its own training data in self-play, where it plays games against itself using the search; labels are obtained from observing the games’ outcomes and statistics from each search.
The two main ideas are that the value and policy predictions complement each other (more accurate policy leads to improved search leads to more accurate value estimation and vice versa), and that the network augmented by the search is a bit stronger than the network itself, so the model can continually bootstrap itself by learning from the stronger search-improved targets.
Spire differs in many important ways from the setting in the AlphaZero paper, and so needs domain-specific adaptation:
While these are significant challenges, not all the solutions need to be invented from scratch, as other projects such as KataGo and Leela Chess Zero have already improved on AlphaZero’s method. Most of the techniques that follow are adapted from followup research to AlphaZero.
Reinforcement learning is monstrously compute-hungry. The model’s improvement is driven by self-play, requiring a lot of compute to generate a tiny amount of value signal from the game outcomes. As the model learns, its strength increases and thus the training target also changes, requiring more fresh self-play data, and so on. Given this constraint, the guiding design principle motivating all of the following items was to make the learning as compute-efficient as possible.
In total, Amber Eye was trained from scratch in about one month, including time spent on experimentation. I think that by applying learnings from the initial run, a model of equal strength could be trained in about one week.
When searching, Amber Eye uses its own reimplementation of the game engine, built using sts_lightspeed with some minor enhancements. The implementation is quite fast, searching about 800 000 positions per second on small (CPU-bound) networks on my 8-core machine.
sts_lightspeed is mostly accurate to the real game, but contains some bugs, which were discovered and patched over time by having Amber Eye play in the simulator and the real game in parallel and flag state divergences. The patched simulator is faithful enough to not significantly impact the model’s winrate, but Spire itself has a lot of weird bugs which were not worth replicating:
The simulator gives a minor amount of help to the engine, mostly to reduce compute cost:
There are different options for what algorithm to use when performing the search. They all use the network’s outputs to guide the search, but differ in how they use the outputs to decide what move to explore next. AlphaZero uses PUCT, but Amber Eye uses Gumbel AlphaZero. The implementation details of the algorithms are not very important to describe here; the main benefits of using Gumbel search are that it empirically performs better than PUCT at playing Slay the Spire at equal search budgets, and that Gumbel search makes a much more favourable tradeoff between exploration and playing strength.
For PUCT to work well in AlphaZero, a significant amount of exploration noise has to be added to the network predictions, in addition to randomly sampling some moves from the policy instead of taking the top choice. With Gumbel search, I found it was sufficient to only apply the more modest Gumbel noise to the search and no sampling was needed.
Gumbel search also scales down to much lower search budgets than PUCT. To generate policy targets, AlphaZero used the distribution of visit counts from search, which is very noisy for low visit counts. Gumbel search also uses the value network’s output to construct the policy target, reducing its variance.
To accurately model ideal Spire, the search must also not reveal or exploit hidden information, e.g. draw pile order, future enemy intents, or relic queue order. However, the search must still be able to progress past actions with random outcomes, or else it would barely be able to look ahead.
A simple way to address this is to sample random outcomes. When stepping the simulator, it flags if any hidden information was revealed by this step, which can be done by checking if any of the RNGs were used. Some hidden information, like draw order or intent under Runic Dome, is randomized at an earlier step (e.g. when reshuffling the deck), so these cases are also detected and resampled. When a step reveals hidden information, its RNG is rerolled to sample a different random outcome consistent with the prior game state.
If we performed a fresh reroll every time a step was visited, the search would never be able to progress past that step. Instead, the search adopts double progressive widening for random outcomes: it puts a limit on the number of resamples per move which slowly grows as the move gets revisited. If the move is visited when the random sample count is already at the limit, the search proceeds deeper into one of the existing random outcomes rather than generating a new one.
Playing Defend \(\rightarrow\) Strike or Strike \(\rightarrow\) Defend results in the same position2, but a regular tree search would treat these lines as completely distinct, wasting computation searching the same position in two different branches and adding noise from the two branches’ differing random outcomes. Monte-Carlo Graph Search addresses this by merging the identical positions and searching a DAG instead of a tree. In the case of Spire, cycles are possible in the position graph (e.g. playing a “useless” infinite), so we must additionally detect and cut these cycles when implementing the graph search.
Because of the complexity of its implementation, MCGS was not used in self-play games. However, the benefit of MCGS grows as the search budget increases, so it was implemented to play the sample games. Empirically, applying MCGS to Spire effectively cuts the average branching factor by about 1.2\(\times\). This has a modest benefit at the search budget used in self-play, but compounds to about a 15\(\times\) effective compute advantage versus tree search at the 262k-position search budget used in the sample.
Spire positions can roughly be divided into combat and non-combat states (commonly referred to as “micro” and “macro”, respectively). The consequences of in-combat decisions are mostly local to that combat, in that playing better lines saves HP for that fight and leads to a better post-fight position. It is tempting, then, to also perform this division on the model side, and have separate networks handle combat and non-combat positions. However, micro and macro are more intertwined than they look at first glance. Resource use (potions), meta-scaling items (Feed), and relic setup (Incense Burner) obviously carry over to future combats, but other combat decisions can depend more subtly on the overall game state. For example, one might choose to play a higher-variance line which loses more HP on average if it opens a chance of highrolling and being able to path to an extra elite. Because of this, optimally playing combats requires planning beyond just the combat.
For this reason, I consider it very important that a single network evaluates all game states. A single network better applies the bitter lesson; it can learn for itself how micro interacts with macro, avoiding the need to figure out how to stitch the networks together.
Amber Eye’s network primarily consists of a transformer which processes the game state. Most people’s experience with transformers is with large language models, and so they are commonly associated with next-token prediction, but this is a bit of an oversimplification. At a high level, a transformer is a type of neural network which can process variable-size sets of features and which is very effective at learning interactions between them. To enable this, the features are represented by points in an abstract high-dimensional embedding space.
More specifically, the data follows this computation path:
To process the game state in the transformer, its information must first be preprocessed into tokens. Each token’s inputs consist of three parts:
The token then passed into the transformer is the sum of its type embedding, its categorical embeddings, and a type-specific projection of its features, all LayerNormed. A game state has an average of about 40 tokens, mostly from cards and relics.
A game like chess easily admits a fixed-size action space (source square \(\times\) target square). In Spire, many types of action are possible, and are contextual on the game state. Thus, we also prepare a list of legal actions as an input, and the policy is directly computed over this list. An action can be represented by the following:
The action list is used by the model to directly fetch the source and target tokens from the transformer’s output and compute the hidden state for the policy. More precisely, for each action \(a\) we compute
\[e_a = \text{emb}(k_a) + W_s\, h_{s_a} + W_t\, h_{t_a} + W_f\, f_a + W_c\, c\]where \(\text{emb}(k_a)\) is the action kind embedding, \(h_{s_a}, h_{t_a}\) are the source and target output tokens respectively, \(f_a\) is the vector of action features, \(c\) is a pooled summary of other tokens, and the \(W\)s are learned weight matrices. If there is no source or target for an action, a “none” embedding is used instead. The vector \(e_a\) is then projected by a MLP to a logit, and the policy is obtained by taking the softmax of these logits over the action list.
This setup has the advantages of only needing to compute the policy over legal actions, and automatically fetching the relevant tokens in the policy computation.
The heads convert the processed token stream into the value and policy predictions. Apart from the policy heads, each head consumes a summary consisting of the encoded Run token plus the average of other tokens processed by a shared summary MLP. Each head is a two-layer MLP processing this summary to produce its prediction. The prediction task and role of each head is discussed in the following section.
In a pure AlphaZero setup, only the recorded outcome of the game and search policy would be used to train the network, meaning we would only extract 1 bit of information for the value head to train on per game, a terrible ratio given the tens of thousands of network evaluations spent to play the game. However, we can make use of more information from the game to better train the network’s internal representation.
The network has the following heads:
At the start of training, playing games to purely maximize the network’s predicted win chance quickly collapses into a state of learned helplessness. The network is weak to start and cannot win any games, so it quickly learns to predict that all states are completely lost and never improves. Instead, we can make use of the other prediction heads to nudge the engine in a direction that makes progress until it is able to win. Formally, we define a utility function for the position based on the heads’ predictions and ask the engine to maximize the utility instead of the win chance.
In the beginning, the utility was defined in the following way. Define the floor progress reward for a position \(s\) as
\[F(s) = \frac1{57} \left(\sum_{f=1}^{56} f \cdot P(\text{die on floor } f \mid s)\right) + P(\text{win} \mid s).\]Effectively, \(F(s)\) gives the self-play agent a reward of \(1/57\) when it advances a floor or wins. The probabilities are estimated from the model’s outcome head. Similarly, we can define the HP reward as
\[H(s) = \frac{1}{56}\left((\text{floor}(s)-1) + \sum_{f=\text{floor}(s)}^{56} \frac{\mathbb E[\text{HP on floor } f \mid s]}{255}\right),\]which rewards the model for preserving HP. The utility function initially used was \(U(s) = F(s) + H(s)\).
The model made consistent progress under this utility and won its first game after about 460 000 games played and 1.5 hours of training. I was quite surprised that only these terms were sufficient, as long fights still don’t get any intermediate signal from the reward, and long-term objectives like keys are hard to learn about.
After a few more hours, the model reached significant winrate under this utility. I then updated the utility to \(P(\text{win} \mid s) + 0.25H(s)\) for out-of-combat states and \(F(s) + 0.5H(s)\) for in-combat states. Empirically, these improved the performance in self-play, and the changes can be interpreted as allowing focusing on building decks for the endgame in out-of-combat states. I expected to eventually be able to entirely remove the other shaping terms and purely optimize for \(P(\text{win} \mid s)\), but this utility form continued to perform better even at the end of training. One possible interpretation of this fact is that the model’s estimation of \(P(\text{win} \mid s)\) has some model error associated with it, and the other shaping terms help mitigate the error, even though they introduce a bias which does not perfectly align with optimizing for winrate.
Because training data is generated via self-play, training samples originating from the same game are correlated with each other. This presents challenges for training the value network. Samples from the same game all have the same outcome, so the value network can quickly learn to memorize properties of the game (deck and relics) and use that to predict the game’s result, overfitting instead of properly generalizing.
To address this, first we apply playout cap randomization as in KataGo. Moves are selected with probability \(p = 0.25\) to undergo a full search with a larger number of simulations, and the remainder of the moves are played using a fast search with much fewer simulations. Only the full moves are used in training. This increases the number of games that can be played per unit of compute, as we can spend less compute on the fast searches, thus increasing the diversity of value labels that can be generated in the same amount of time. While the overall production of training samples is reduced, the tradeoff is worth it to gain more value labels. The fast moves also cost self-play strength, but this is also partially mitigated by not applying exploration noise to them. Empirically, using a fast move budget \(1/8\) the size of the full move budget cost roughly 1 percentage point of winrate.
Playout cap randomization partially mitigates the memorization problem, but due to Spire’s longer game length, significant memorization is still observed at \(p = 0.25\). \(p\) could be lowered even further to reduce it, but this increases the proportion of compute spent on fast moves which don’t generate training data. This tradeoff becomes quite unfavourable for small \(p\). Thus, I altered the value label itself as further mitigation.
The value labels were redefined as a combination of the search result and an exponential weighted average of future moves’ search outcomes. This injects diversity into the value labels to reduce overfitting, although the labels from positions close to each other in the sequence of moves are still correlated. It also corresponds well to the game itself, where the long-horizon outcome is noisy and dependent on future random rolls, so it makes sense to downweight learning from outcomes far in the future.
This label definition also has the downside of having a low weight on the real outcome, so the value estimate can remain persistently miscalibrated. Notably, the engine significantly underestimates its own winrate, and the underestimation increases in positions earlier in the game (where the end outcome is more distant). The gap is 10 percentage points or more at the start of the game. However, as long as the engine is able to score better positions as higher value, the actual numerical value of the predicted winrate is irrelevant.
If the network learns that certain choices are bad early on and starts never picking them, it will never get the signal it needs to correct misjudging these choices. For example, midway through the training run, I noticed that the network rated Apotheosis as the worst card in the game by far, because it never picked it and reconsidered. Typical exploration strategies like sampling from the policy or adding uniform random noise still barely make the choices that need to be sampled, and had a large cost to play strength. To address this, I created a domain-specific exploration scheme inspired by the Quantile Balancing technique used by Kimi K3.
For choices like card picks, event choices, Neow bonuses, and relics, we first set a target percentage of games that each choice should appear in (I used 1-3%). For items below the target, we can keep track in the gamelog of how far they are below the choice actually taken (in terms of policy logits). We can then determine the bias that needs to be added to each of the items below the target such that the target pick count is reached. In the search, the below-target items are forced to be kept as a final candidate, any search that includes a below-target item becomes a full search, and the bias is added after search completes to force the picking of unexplored items at roughly the target rate.
Instead of a hard threshold like in Quantile Balancing, the boosted item is picked with probability
\[\min(1,\,c\cdot\exp(-m/T)),\]where \(c\) is a parameter computed per item by binary search to hit the target pick rate, \(m\) is the gap between the boosted item and the top pick after the bias is added, and \(T = 4\) is a temperature parameter. Effectively, situations where the item is closer to being the top pick are more likely to be picked under this scheme, but situations with larger gaps are also chosen occasionally to have more variety in the picks.
The forced picks successfully corrected the bias on items like Apotheosis, and only affected ~0.2% of moves with a modest play strength cost of ~3 percentage points.
There are many ways in which more compute can be traded for more playing strength. Generally, you have to spend more compute than you get playing strength back, so it saves a lot of time to avoid spending more compute when the model is weak and there are low-hanging fruit for improvement, and only spend once it becomes necessary. I applied this idea to several axes:
To train the model, we need to store a large replay buffer of training examples across a large number of games to ensure each batch has a good mix of training data. At the end of training, I used a replay buffer of 10M states, but a naive storage of the raw training data would take about 7-8 KB per state, which is way too large for my machine. I used the following compression techniques:
Search in an AlphaZero setting is difficult to efficiently implement as described. In most abstract descriptions of search algorithms, the search tree/graph is expanded one node at a time, and each node needs a GPU evaluation, leading to many roundtrips between CPU and GPU. This naive approach also does not utilize the GPU properly, which is much more efficient at processing large batches in parallel.
In self-play, these issues can partially be addressed by playing many games simultaneously. On my 8-core, 16-thread machine, I ran 14 worker threads each running hundreds of self-play games. In each search, network evaluation requests were submitted into a queue, and separate threads packed the requests from the queue into larger batches and passed them to the GPU. The latency of GPU network evaluations is hidden by having many games in flight.
The large game count is still not enough to fully utilize the GPU, and does not apply to the single-game setting, so we additionally allow multiple searches in the same game tree using virtual visits. In self-play, up to 4 nodes per candidate move’s subtree were allowed to be visited in parallel. For the single-game evaluation, much more parallelism needs to be extracted, and the strength penalty is not as large for a large simulation count, so 2048 parallel nodes per candidate were allowed. Both the above techniques are standard in AlphaZero-esque setups.
On the GPU side, the model is small enough that its weights in FP16 fit comfortably in my GPU’s L2 cache, and it has high arithmetic intensity, so it was relatively easy to get high compute utilization. The kernel achieved about 75% MFU in self-play after discounting padded attention operations.
I am not sure why anyone would read this very long article, but hopefully it was informative.
Much more is still possible. I have many ideas for further improving the engine’s performance, and of course the other three characters are still unexplored. I am excited to continue developing the engine and sharing the results.
There are three situations I am aware of where hidden information is leaked to the engine.
Each of these situations occur very rarely, and their knowledge gives a very small advantage to the player, so the overall impact on winrate is negligible. ↩
In the absence of other effects like Juggernaut. ↩
Towards the end of training, the observed scaling was very roughly 1 percentage point of winrate per quadrupling of search budget. ↩