Automatically identifying the stepping-stones that will lead to a predetermined final solution presents a significant challenge for optimization algorithms, yet is essential for solving complex problems. This study, inspired by Picbreeder, investigates a variation of NeuroEvolution of Augmenting Topologies (NEAT) which aims to perform an image replication task using Compositional Pattern Producing Networks (CPPNs) without a human in the loop. This challenge is central to many similar problems in evolutionary computation and artificial life, where identifying key intermediate goals known as stepping-stones is crucial but difficult, often requiring precise fine-tuning of solutions. We leverage techniques from deep learning computer vision research: a fitness function based on perceptual-similarity to help avoid deceptive optima, Fourier features to diversify the CPPNs’ inputs, and gradient-based backpropagation to balance the exploration of evolutionary search with goal-directed exploitation. Back-propagation has the additional benefit of smoothing the fitness landscape of topological network mutations. Our results indicate that combining these approaches with CPPN-NEAT yields more diverse and higher fitness solutions compared to traditional NEAT. This hybrid method not only preserves diversity, but also leverages the strengths of both evolutionary algorithms and gradient descent to achieve more detailed and accurate image generation. We speculate that this is a promising avenue for algorithm design, where exploiting gradient information can be balanced with maintaining robust diversity in the search process.