Dear
@likeawizard, I am hereby writing to maybe help us (some of us) and maybe you as side effect.
From your whole series of blogs journalling your chess engine development from scratch (well beside general C.S. notion you might have become familiar with, but not specifically about chess and its algorithmic models (or the high level math. models their might be implementing, or were implementing in the past, before it became code that started inventing something often now in need to high level extraction in terms equally rigorous, but absrtacting machine language into human language. so a bit of math.. might be needed to bridge things). That is my outlook. I am writing to you, because i know you have had that concern of explaining to some idea of general audience what you were experiencing, learning and doing, including your trials and errors.
It has been my understanding that even your code has been constructed with clarity of purpose as part of your global optmization criteria (for the whole project, not just the arena behavior).
So I would like to complement my current understanding of chess engines of the familiy you have been following in your design, which includes our mighty, beloved and revered without much question, I name Stockfish tree of versions (and its fishtest pool of other instances, of which probably we get to see the code post optimization, at release time).
Avoiding chessprogramming wiki terminolgy at the point where post AB pruning heuristics start being considered to further reduce the explored tree diversity of candidate branches being explored toward scoring a single position, usually the user current position.
I will make a list of what I think I know.. and would ask you to try to complements trying to regroups things abstracted from the chessporgramming wiki various terminologies that might have been fortuitous choices that seemed good at innovation time, but do not help, figuring out what the code might be doing at the level your blogs were aiming at or I am trying to understand.
I don,t fear math. i fear words more than math.. and words to math fuzzyness, across math. sub-fields.. so i suggest we try to be parsimonious. if you agree. and use best natural language if we can..
list of things, if you please could complete even partially, about what your engine has so far tried, rejectied, and included.
exhaustive legal tree exploration
root position scoring
AB pruning
Iterative deepening
Quiescence search
Leaf node evaluation
Now that could be the basic notions (besides the move encoding and the position information encoding), as chess user of engine analysis might want to become familiar with, and they are all explained by your efforts in your blogs. I suggest avoiding implementation terms like transposition table (it is not only about transposition anymore, has not been for a long time in SF).
Let me set the tabula rasa level. You have mentioned history, good choice of term.. to which i would had memory of history.
Which ties in to the notion of making statistics internally during the construction of the explored tree, about various things that all the incrementally composed new heuristics you are trying on top of above core model for chess audience.
The killer move list.. which is an example of how such engine design keeps having to develop a bush of recursive heuristics possibly synergizing and repairing each others weaknesses, but also possibly messing things up, perhaps the source of your spleen.. (i may not have read or understood the blog carefully, let me know if I should revisit).
I have questions for you that might help you answer the more general, possibly impossible to answer questions (depends on how fast you went for the ELO whale, i assume you have been consistent though).
So I have a basic model i alrady shared with you, that you responded was basically the skeleton of interactions between the search module and the evaluation module. (at the time). They are increasingly tangled as your engine evolved. and I would let you speak about how move versus position gets dissected and filtered in that interaction, but the point is always to reduce computing cost and memory usage (which are probably interacting costs in same direction). but that is not necessary for chess model we want, is it. can we still talk in SAN notation ply-moves and positions as FEN, even if those are not internally preserved?
I will. SAN move being the individual mobile unit notation that determines if applied (that is important that applied part i think for chess audience to understand, do you agree?), what the next position will be (or candidate next position, if move not yet applied). The point is that a SAN move can be processed in the search module for many recursively coded (the whole coder and all heuristics are usually coded like that, very compact writing), which regard to which position it would be applied to or the position resulting.. A lot happens in search before, a candidat move is dispatched to the evaluation module (expllicit function of position information or what is left of it in the particular engine design) with modelize input as FEN.
My question to you, as coder, and open source code repository developper. If we were to have a generic model within that frame i might have suggested. could I assume that all your heuristic experiements beyond the core model above, have a code reading connectted set of lines vertically, and possibly even a name for the conditional test (and its bottom case), to the point
where I could ask a subsequent question:
How much trouble would it be to have the full set of all nodes and all their heuristic dispatching as node label (not the chessprogramming topology or node type, or other confusing terms to me at least, unless you went consistent with only one there and did not need more node types.) So node and heuristics code block labelling . modifying code, in single root search, to keep dumping in some persitent memory, all the node processing, and all the dispatching calls.
if the leaf evaluation further makes types of nodes (the moves) or the position (I let you sort that out if it no longer makes sense to talk about piece-move or position. In case the processing has spilled over to the evaluation modules, for more cost efficiency i can't imagine. not being a developpper myself..
Can you help me understand further and which of your blogs might cover that already.. the parts that can.
I hope i have just show to you and possibly other chess engine developpper, and most importantly chess lovers and users of engine as analytical tools, what I meant in previous rambling about internal debug mode.
We can start with the as-hoc tree topology nomenclature using terminolgooy from your code of your choosing. You are good at such thing) or just function# or block number if some node processing is not yet sub-modularized.
I suppose such full singe root search explored tree tracing with all the heuristic treatment labels from code (not wiki), is stil of the finite set category, each heuristic or testing and dispatching steps are still human mind size. or reasonable amount of words.
then. the data analysis can start becoming conceivable., and we would have the real model that corresponds to your code, that some non-dev chess users could still include in their own thinking, analysing chess board, and discussion in the community.
I already explained why I am eager that one day, such memory effort dumping becomes a norm for engine being used in their analytical role in chess.
I hope that i made. sense to op. and at least someone else. or more. i welcome any mode of discussion seeking some truth. not a competition here.