Open-ended evolution – the presence of rich evolutionary dynamics that continuously produce novel, complex communities and species – is a key feature of the natural world. Understanding the conditions that enable open-endedness is a major challenge in artificial life and evolutionary computation. The MODES toolbox, consisting of metrics for detecting change, novelty, ecology, and complexity, is a promising approach for quantifying open-endedness. However, MODES has only been applied to a few systems so far, with limited opportunity for controlled experiments or cross-system comparisons. To address this gap, we implement a custom digital evolution platform (Evo-Sandbox) designed specifically for this purpose. Evo-Sandbox includes configurable modules for that can be combinatorially combined to create diverse environments. We investigate two diversity promoting mechanisms, fit-when-rare, and parasites, to test MODES across a range of conditions. Our experiments reveal that the regions of parameter space in which different hallmarks of open-endedness are maximized are non-intuitive, and that MODES is, in fact, a valuable tool for understanding the resulting behaviors.