Learning the procedure of counting represents a major step in children's development of the concept of the natural numbers. How children acquire generalized concepts of number and counting skills is still under debate. Here we investigate how a neural network agent develops representations for key concepts of counting while learning to perform several different counting tasks in a multimodal, interactive environment. We identify neural activity and connection patterns that realize a) a representation of the entity to count that was invariant to the task, b) a mapping from entity to number-word, and c) a representation of the number of entities that have been counted that was shared between tasks. The results support the notion that abstract representations of number can arise from integrating experiences across a range of number-related tasks.