Concept Measurements

[nkondapa]

Hi,

I am trying to reproduce the trends observed in Figure 4 from the paper.

I want to measure: middle_game_evaluation, middle_game_piece_value, has_bishop_pair_active, can_capture_queen_alive on the data from the example_test_dataset. I have modified the code/dataset to (1) sample 5000 board positions, (2) repeat the dataset while substituting different ELO values (total 7), and (3) use Stockfish or some simple code to evaluate each position.

I extract the features from the penultimate layer (just before the three heads) and conduct 5-fold linear regression on the concept values. Unfortunately, the fits I'm getting are extremely poor. Do you have any demo code using the example_test_data that can reproduce the rough behavior in Figure 4?

Thanks