Env sweeps over battle and drone

[vyeoms]

Drone

The default clip_coef is lower than the default bound, so added a smaller min to the sweep config to drone. Config file in discord, summary:

• perf = 0.916312
• score = 380.626
• hover/perf = 0.876612
• hover/score = 749.812
• race/perf = 0.956011
• race/score = 11.4407
• 128x4 policy net

There's a config that got perf ~0.94 but had hidden dim 512

Hover rollout

drone_rollout_hover.mp4

Race rollout

drone_rollout_race.mp4

Constellation:

Battle

Fixed a few numerics on logging, no other modifications functionally. Perf is fraction of episodes with positive return. Maybe could be good to make the different types of units more distinguishable for the render?

Config file in discord, summary:

• perf = 0.808419
• score = 3.2605
• 512x3 policy net

Rollout of the policy:

battle_rollout_trained.mp4

For comparison, this is a random init policy:

battle_rollout_randomInit.mp4

Constellation: