Jon Allman | Puget Systems https://www.pugetsystems.com/bios/jonallman/ Workstations for creators. Tue, 06 May 2025 17:27:17 +0000 en-US hourly 1 https://wordpress.org/?v=6.8.3 NVIDIA Blackwell GPU GenAI Software Support https://www.pugetsystems.com/labs/articles/nvidia-blackwell-gpu-genai-software-support/ https://www.pugetsystems.com/labs/articles/nvidia-blackwell-gpu-genai-software-support/#respond Tue, 06 May 2025 17:27:14 +0000 https://www.pugetsystems.com/?post_type=article&p=36704 After the recent release of Pytorch 2.7, what is the current state of support for Blackwell GPUs in popular generative AI applications?

The post NVIDIA Blackwell GPU GenAI Software Support appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/nvidia-blackwell-gpu-genai-software-support/feed/ 0 Exploring Hybrid CPU/GPU LLM Inference https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/ https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/#respond Thu, 20 Mar 2025 20:41:02 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=35198 A brief look into using a hybrid GPU/VRAM + CPU/RAM approach to LLM inference with the KTransformers inference library.

The post Exploring Hybrid CPU/GPU LLM Inference appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/feed/ 0 NVIDIA GeForce RTX 5090 & 5080 AI Review https://www.pugetsystems.com/labs/articles/nvidia-geforce-rtx-5090-amp-5080-ai-review/ https://www.pugetsystems.com/labs/articles/nvidia-geforce-rtx-5090-amp-5080-ai-review/#respond Sat, 22 Feb 2025 00:16:26 +0000 https://www.pugetsystems.com/?post_type=article&p=34053 How do NVIDIA's new GeForce RTX 5090 and 5080, released with fanfare regarding their new features and capabilities, perform in real world AI applications?

The post NVIDIA GeForce RTX 5090 & 5080 AI Review appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/nvidia-geforce-rtx-5090-amp-5080-ai-review/feed/ 0 Exploring GPU Performance Across LLM Sizes https://www.pugetsystems.com/labs/articles/exploring-gpu-performance-across-llm-sizes/ https://www.pugetsystems.com/labs/articles/exploring-gpu-performance-across-llm-sizes/#respond Thu, 16 Jan 2025 20:49:25 +0000 https://www.pugetsystems.com/?post_type=article&p=33242 Does the size of a Large Language Model affect relative performance when testing a variety of GPUs?

The post Exploring GPU Performance Across LLM Sizes appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/exploring-gpu-performance-across-llm-sizes/feed/ 0 What’s the deal with NPUs? https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/ https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/#respond Fri, 25 Oct 2024 19:55:03 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=31072 An introduction to NPU hardware and its growing presence outside of mobile computing devices.

The post What’s the deal with NPUs? appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/feed/ 0 LLM Inference – NVIDIA RTX GPU Performance https://www.pugetsystems.com/labs/articles/llm-inference-professional-gpu-performance/ https://www.pugetsystems.com/labs/articles/llm-inference-professional-gpu-performance/#respond Thu, 22 Aug 2024 16:41:27 +0000 https://www.pugetsystems.com/?post_type=article&p=29660 How do a selection of GPUs from NVIDIA's professional lineup compare to each other in the llama.cpp benchmark?

The post LLM Inference – NVIDIA RTX GPU Performance appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/llm-inference-professional-gpu-performance/feed/ 0 LLM Inference – Consumer GPU performance https://www.pugetsystems.com/labs/articles/llm-inference-consumer-gpu-performance/ https://www.pugetsystems.com/labs/articles/llm-inference-consumer-gpu-performance/#respond Thu, 22 Aug 2024 16:41:26 +0000 https://www.pugetsystems.com/?post_type=article&p=30017 How do a selection of GPUs from NVIDIA's GeForce series compare to each other in the llama.cpp benchmark?

The post LLM Inference – Consumer GPU performance appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/llm-inference-consumer-gpu-performance/feed/ 0 Tech Primer: What hardware do you need to run a local LLM? https://www.pugetsystems.com/labs/articles/tech-primer-what-hardware-do-you-need-to-run-a-local-llm/ https://www.pugetsystems.com/labs/articles/tech-primer-what-hardware-do-you-need-to-run-a-local-llm/#respond Mon, 12 Aug 2024 21:34:43 +0000 https://www.pugetsystems.com/?post_type=article&p=29116 What considerations need to be made when starting off running LLMs locally?

The post Tech Primer: What hardware do you need to run a local LLM? appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/tech-primer-what-hardware-do-you-need-to-run-a-local-llm/feed/ 0 Effects of CPU speed on GPU inference in llama.cpp https://www.pugetsystems.com/labs/articles/effects-of-cpu-speed-on-gpu-inference-in-llama-cpp/ https://www.pugetsystems.com/labs/articles/effects-of-cpu-speed-on-gpu-inference-in-llama-cpp/#respond Mon, 01 Jul 2024 17:20:22 +0000 https://www.pugetsystems.com/?post_type=article&p=28690 What effect, if any, does a system's CPU speed have on GPU inference with CUDA in llama.cpp?

The post Effects of CPU speed on GPU inference in llama.cpp appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/effects-of-cpu-speed-on-gpu-inference-in-llama-cpp/feed/ 0 Puget Mobile 17″ vs M3 Max MacBook Pro 16″ for AI Workflows https://www.pugetsystems.com/labs/articles/puget-mobile-17-vs-m3-max-macbook-pro-16-for-ai-workflows/ https://www.pugetsystems.com/labs/articles/puget-mobile-17-vs-m3-max-macbook-pro-16-for-ai-workflows/#respond Tue, 28 May 2024 19:17:34 +0000 https://www.pugetsystems.com/?post_type=article&p=27770 How does the new Puget Mobile 17" compare to the MacBook Pro M3 Max 16" in performance across a variety of AI-powered workloads?

The post Puget Mobile 17″ vs M3 Max MacBook Pro 16″ for AI Workflows appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/puget-mobile-17-vs-m3-max-macbook-pro-16-for-ai-workflows/feed/ 0