Imagine pouring a thousand different ideas—love, entropy, a bicycle, the number 7—into a blender and then asking an AI to arrange them in a neat, continuous landscape. That’s essentially what modern representation learning does: it maps concepts to points in high-dimensional spaces, hoping those points lie on a low-dimensional manifold. But reality, as a recent deep-dive on Habr shows, is far less tidy. The article titled “How Models Package Concepts into Manifolds: Looking at Theory, Destroying the Ideal World, and Looking for Circles” (published in July 2026) takes a hard look at where the theory of manifold learning meets the gritty truth of real data—and why sometimes you just have to go hunting for circles.
The Promise of a Perfect Manifold
At the heart of many AI breakthroughs lies the manifold hypothesis: real-world data lives on or near a low-dimensional manifold embedded in a high-dimensional ambient space. Think of a crumpled sheet of paper in 3D: the paper itself is 2D, but it twists and folds in three dimensions. Models like autoencoders, t-SNE, and UMAP try to “unfold” that paper, learning a representation where similar concepts cluster together and transitions between concepts are smooth. The theory is elegant, mathematically grounded, and has powered everything from facial recognition to natural language embeddings.
The Habr article begins by revisiting this ideal—a world where data points flow like a Riemannian manifold, where local neighborhoods are linear, and where geodesic distances reflect semantic similarity. The authors (the article is presented as a collective work by the Habr community) walk through classic examples: how word2vec arranges analogies as linear translations, or how variational autoencoders generate smooth interpolations between digits. The implicit promise is that once you find the right manifold, you have captured the “shape of meaning.”
Destroying the Ideal World: When Manifolds Break
Then comes the twist. The article systematically dismantles the fairy tale by introducing real-world data pathologies:
- Disconnected Components: Not all concepts are connected. The space of “animals” and the space of “tools” may have no smooth path between them. Models forced to embed everything into one continuous manifold create artifacts.
- Curse of Dimensionality Revisited: Even if the intrinsic dimension is low, the required sample size to estimate the manifold grows exponentially. In practice, models often see only sparse, noisy samples.
- Non-Euclidean Geometry: Data often lives on non-Euclidean structures—spheres, tori, or discrete graphs. The Euclidean distance used by most algorithms distorts relationships.
- Labeling Artifacts: Supervised models may warp the manifold to fit labels, creating unnatural separations that don’t reflect true concept boundaries.
The article calls this “destroying the ideal world” because it forces practitioners to abandon the assumption that a single, smooth manifold exists. Instead, the authors advocate for a more nuanced view: concepts are often clumped into clusters or wrapped around topological features like loops (circles) or holes.
The Hunt for Circles: Topological Data Analysis to the Rescue
The most intriguing part of the Habr piece is the search for “кружочки” (circles). This refers to one-dimensional topological features—cycles—detected via persistent homology, a tool from topological data analysis (TDA). The authors show that many real datasets contain circular patterns that traditional manifold learning misses.
For example, consider the space of 2D shapes generated by rotating an object. The angle of rotation creates a loop: 0° and 360° are the same point. A standard PCA might embed this as a line (breaking the circle), whereas a topological approaching can preserve the circular structure. Similarly, in time-series data, recurrent states may form cycles. In natural language, the concept of “parts of speech” might loop through verb tenses.
The article demonstrates how to compute persistent homology (using tools like the Ripser library or GUDHI) on point clouds and identify which cycles are “persistent” (i.e., real structure) versus noise. It then contrasts this with the “ideal world” assumption that manifolds are simply connected—something that is rarely true in practice.
| Technique | Assumptions | What It Preserves | Where It Fails |
|---|---|---|---|
| t-SNE | Local neighborhoods are roughly linear | Local clusters | Global topology (e.g., cycles) |
| UMAP | Data lies on a Riemannian manifold | Local and some global structure | Non-Euclidean intrinsic geometry |
| Isomap | Geodesic distances approximate true distances | Global distances on convex manifolds | Non-convex shapes, holes |
| Persistent Homology | No assumption about manifold; only distances | Topological features (connected components, cycles, voids) | Does not produce a direct embedding |
The table above, adapted from the Habr discussion, illustrates that no single method is a silver bullet. TDA offers a complementary lens: instead of trying to flatten everything into a Euclidean space, it asks “what topological shape does the data actually form?”.
Practical Implications and Case Study
The article includes a concrete case study: a dataset of handwritten digits (MNIST) is often used as a poster child for manifold learning. But the Habr authors show that the manifold of digit “8” has a hole in the middle (the loop of the two circles), which t-SNE and UMAP typically break apart because they prioritize locality over global topology. By applying persistent homology, they identify that the true shape is a torus-like structure with a circular hole. They then use a technique called “circular coordinates” (based on Hodge theory) to parameterize that hole explicitly.
This has real-world consequences: when models ignore these topological features, they fail to capture essential geometric invariances (e.g., rotation, periodic structure). The article suggests that for domains like molecular conformations, robotics joint angles, or sound pitch, explicitly modeling cycles can improve generalization.
Where Do We Go From Here?
The Habr article concludes by arguing for a hybrid approach: use manifold learning for local structure, but supplement it with topological descriptors for global shape. It also calls for more research into “neural topology” – networks that learn to preserve both metric and topological properties.
For developers and data scientists working on representation learning, the message is clear: don’t assume your data lives on a perfect sheet of paper. It might be a crumpled, punctured, loop-filled mess. And that’s okay—as long as you know how to look for the circles.
Interested in applying these ideas to real-world AI systems? Platforms like ASI Biont provide tools to connect your models to custom data pipelines and explore embeddings with advanced analytics. Discover how at asibiont.com.
Comments