Unlike the brain, existing LLM transformer models use staggering amounts of energy.
Simply put, not only are the transformer models in LLMs expensive to run, they do not align with sustainability objectives for the planet.
SSMs are models with three views. A continuous view, and when discretized, a recurrent as well as a convolutive view. SSMs have an ability to handle very long sequences (number of tokens), generally with a lower number of parameters than other models (ConvNet or transformers), while still being very fast. SSMs can be applied to text, vision, audio and time-series tasks (or even graphs).
Jensen Huang, NVIDIA: "And I think the work around state-space models, or SSMs, that allow you to learn extremely long patterns and sequences without growing quadratically in computation, probably is the next transformer."
