We consider various setups where large language models (LLMs) communicate solely with themselves or other LLMs. In accordance with similar results known for program representations (like λ-expressions or automata), we observe a natural tendency for the evolution of self-replicating text pieces, i.e., LLM prompts that cause any receiving LLM to produce a response similar to the original prompt. We argue that the study of these self-replicating patterns, which exist in natural language and across different types of LLMs, may have important implications on artificial intelligence, cultural studies, and related fields.