NVIDIA's TensorRT updates for RTX GPUs also enable some big performance uplifts to GenAI workloads such as Stable Diffusion.
Stable Diffusion & GenAI Gets Boost Through TensorRT Support on NVIDIA's Gaming & Pro RTX GPUs
We have already detailed how TensorRT-LLM is bringing faster AI capabilities to Windows on RTX hardware and GenAI is another area where consumers owning an RTX GPU will be able to see a direct benefit.
It's no secret that NVIDIA's GPUs are amongst the most popular solutions for Stable Diffusion and Generative AI workloads. We have seen NVIDIA being ahead of almost everyone in this field but the most recent and upcoming CPU (from AMD & Intel) launches have started to include a dedicated NPU unit that can offload the AI tasks from the CPU/GPU & complete the work in a very low-power & efficient mode for the vast majority of users.
NVIDIA states that it is great to see the push to accelerate AI being made by infusing CPUs with AI and they will mostly be used for lightweight AI tasks running at low power and the GPU is going to be for more demanding use cases. Both NPU and GPUs are offline and locally available resources, providing low latency and data locality/privacy features while cloud data centers are targetting the Heavy AI workloads for very large models and use-on-demand. NVIDIA's RTX GPUs are said to offer anywhere from 20x to 100x more performance than these NPUs.
TensorRT acceleration is now available for Stable Diffusion in the popular Web UI by Automatic1111 distribution. It speeds up the generative AI diffusion model by up to 2x over the previous fastest implementation.
via NVIDIA
In a Stable Diffusion performance demonstration, NVIDIA shows the GeForce RTX 4090 running WebUI from Automatic 1111 and outputting 27 images per minute using the PyTorc xFormers implementation but running it with TensorRT doubles the performance to 52 images per minute.
NVIDIA also compares the performance against Apple's M2 Ultra (72 Core Variant) which costs a base price of $5000 US. This system only outputs 7 Images per minute using the CoreML model. Meanwhile, you can build a very high-end system with two GeForce RTX 4090 GPUs on the same budget.
The company has announced that TensorRT is now available in WebUI (Automatic 1111) and is available to download from GitHub.com/NVIDIA.
WccftechContinue reading/original-link]
