A research paper examines how the scale of a network's initial weights influences which features individual ReLU neurons end up learning during training. The authors argue that this initialization-dependent effect, framed as a hidden gauge, helps explain why trained networks organize representations the way they do.
A new arXiv paper reframes the AddUNet model through the lens of perfect reconstruction and describes how it can operate at full rate. The authors introduce a residual full-rate perfect-reconstruction design aimed at task-directed representation learning, building on the architecture's survivor-skip structure. The work is theoretical and architectural rather than a released product or benchmark result.
Researchers propose a representation learning framework built on SL(n) that operates in an intrinsic mixed-curvature space rather than relying on manually composed product manifolds. The approach aims to provide higher curvature capacity and deeper order-aware composition for capturing complex geometric structure. It is presented as an alternative to existing product-manifold methods that require hand-specified curvature combinations.
A new preprint sets out a framework for describing when a learned or acquired representation broadens what a system can do, and where that broadening stops. Its central claim is that a system can gain new cognitive capabilities from a representation without inheriting the abilities that were used to create it. The work aims to make the scope and limits of such capability gains specifiable.
A new arXiv paper analyzes how representations inside transformer models change across layers, treating each learned update as a combination of a component that keeps the existing direction and one that shifts it elsewhere. The authors frame this as a functional geometry, aiming to explain what the model preserves versus reorients as information flows through the network.
Researchers introduce a multimodal, multiresolution foundation model trained from scratch for lunar remote sensing. It was pretrained on SomBench, a geographically partitioned dataset of roughly two million co-registered tile bundles covering 11 sensor modalities at two spatial resolutions of 1 m/pixel. The work targets general-purpose representation learning for planetary surface analysis.
A theoretical paper examines the foundations of contrastive learning, a method that uses data augmentation to learn feature representations without large labeled datasets. The authors analyze how the structure of the augmentation graph relates to whether neural networks can approximate the resulting objective. The work aims to fill gaps in the theoretical understanding of why contrastive learning works in practice.
A preprint on arXiv cs.LG describes a method that learns representations of human cortical folding patterns, which form before birth and stay largely stable afterwards. The authors suggest these folding signatures could serve as early markers of neurodevelopment. The abstract is truncated, so reported results could not be fully assessed.
A newly revised arXiv paper proposes a perturbation-based technique for locating and evaluating linguistic representations inside deep neural language models, framing it as an adversarial tracer. The authors note that representation discovery remains unresolved, and that loosely constrained alignment procedures can make the very notion of a representation vacuous. Their approach aims to provide a simpler and more efficient way to probe how such models encode language.
A revised arXiv paper argues that feature learning in neural networks is not fully captured by the prevailing view that networks converge on a single global low-dimensional representation. The authors point to neuron specialization inside multilayer perceptrons as an additional, separate mechanism through which features are acquired and organized. The work aims to broaden the theoretical picture of how networks structure what they learn.
A new arXiv paper presents a method for learning feature transformations on tabular data by searching a continuous embedding space with a learned policy. The approach is hierarchical and invariant to the ordering of input features, allowing it to generalize across feature arrangements. Its stated aim is to build informative abstractions from raw features that improve downstream predictive performance.
A new arXiv study investigates self-supervised speech foundation models, which are widely deployed for speech recognition and to supply tokens for speech-capable language models. The authors analyze what these models' internal representations encode, testing whether they capture genuine word-level linguistic structure rather than only acoustic patterns. The results bear on how such models should be interpreted and used in downstream speech applications.