AI/ML solutions
Theoretical
in the physics of complex systems power-law scaling behavior is predicted for a wide variety of circumstances, where global behavior is observed (phase transitions or magnetization for example). In the process of examining such complex systems physicists have asked and (partly) answered questions about how much system data is needed to have the necessary amount of data-capture in order to then properly predict the global behavior for the system. This understanding is our best theoretical understanding of the key breakthrough leading to the Neural-Net based LLMs that began to demonstrate scaling behavior in 2017.
From physics we see that the long-range correlations in the data must not be entirely destroyed by the sampling process (it can be significantly suppressed, and will be, to get the ‘best of both worlds’). In modern LLM/GAI this is achieved by use of the Attention feature extraction method. From physics we also see there are dimensionality issues in the sampling process, where sampling in the maximal dimensionality is necessary to capture global behavior. And that’s it, as far as capturing sufficient information to properly recover phase transition (magnetization) effect in the model. In the context of statistical learning (using neural nets) a phase transition might correspond to transition to a state with generalized learning behavior. Brute-force computational scaling has been performed, given the predicted scaling, to achieve AGI, or close to it. Now the objective is to repeat this breakthrough with 1/1000 the computational muscle by use of software improvements.
Engineering
At the engineering level is where most of the excitement is (especially the Agentic AI applications) because this is where we actually see the scaling laws happen and can make decisions for massive computational projects and actually see them scale and work as projected (the origination of ChatGPT3.0 for example). In the scramble to create more and more massive projects, most projects are nowhere near fully trained before the next (larger) project is undertaken. A limit has yet to be reached but a limiting (bending in log-scale curve) is beginning to be seen. With the shift to focus on the highly impactful empirically observed scaling law behavior a paradigm shift is occurring in the theoretical perspective to the theory and physics of critical phenomena (from thermodynamics, statistical mechanics, and condensed matter physics – see recent physics book publications for further details). As noted above, in scaling law behavior from theoretical physics there is an association with large-length scale correlations and a sufficiency of local model correlation description has been shown to capture that global model correlation insofar as critical structure is concerned (like the critical exponent behavior and the existence of phase transitions in the Ising model). In statistical learning this would be a transition that marks ‘learning’. The physics of scaling law behavior in developing large language models would indicate a long-range correlation in the text data itself – how can this be? It is not only true, but has been known for quite some time (an oddity at first, but now understood from a number of excellent studies).
Applications
At the application level the LLM’s are being coupled to signal processing and data analysis and to graphics, music, and other generators. Suffice it to say that the AGI-level LLM backbone provides a process whereby audio and video gets pulled into an overall multimodal AGI interface, with amazing generative capabilities in text, image, video, and audio domains.