SambaNova Systems, maker of dedicated AI hardware and software systems, has launched a new AI chip, the SN40, that will be utilized in the company’s full-stack large language model (LLM) platform, the SambaNova Suite.First introduced
in March, the SambaNova Suite utilizes custom processors and running systems for AI inference training. It’s created to be an option to power-hungry and pricey GPUs.To update the hardware so soon after launch implies that there ought to be a huge dive in performance, and there is. The SN40L serves up to a 5 trillion criterion LLM with 256K+ sequence length possible on a single system node, according to the vendor.Each SN40L processing system is made up of 51 billion transistors (total 102 billion per bundle), which is a substantial boost over the 43 billion transistors in the previous SN30 product. The SN40L also uses 64 GB of HBM memory, which is new to the SambaNova line, and uses more than 3x greater memory bandwidth to speed data in and out of the processing cores. It has 768 GB of DDR5 per processing system(1.5 TB total)vs. 512 GB (1.0 TB) in the SN30.SambaNova’s processor is various from Nvidia’s GPU because it uses a RDU-based (reconfigurable dataflow unit )environment, which is reconfigurable on-demand, practically like an FPGA. This is helpful when business start handling multimodal AI, where they are moving in between various inputs and outputs.On the software application side, SambaNova is using what it calls a turnkey option for generative AI. SambaNova’s full AI
stack consists of pre-trained, open-source designs such as the Meta Llama2 LLM design, which companies can customize with their own content to develop their own internal LLM. It also consists of the business’s SambaFlow software application, which instantly evaluates and enhances processing based on the requirements of the particular jobs. Dan Olds, primary research study officer at Intersect360 Research study, said this is a major upgrade both in regards to hardware and, as importantly, the surrounding
software application stack. He keeps in mind that the 5 trillion specification limitation of the SN40 is nearly three times larger than the 1.7 trillion parameter estimated size of GPT-4.”The larger memory, plus the addition of HBM, are crucial factors in driving the performance of this new processor. With bigger memory spaces, customers can get more of
their models into main memory, which suggests much faster processing. Including HBM to the architecture enables the system to move data in between main memory and the cache-like HBM in much larger portions, which also speeds processing,”stated Olds. The ability to run much larger models in relatively little systems and to run multiple designs simultaneously with high efficiency, plus the integration of open-source LLMs to help customers get off the ground quickly with their own generative AI tasks, mark a huge advance for SambaNova, Olds said.”It gives them hardware that can really take on GPU-based systems on big models and a suite of software application that ought to take a lot of the secret(and time )out of building a custom LLM for end users,” he said. Copyright © 2023 IDG Communications, Inc. Source
