Personal AI: laptops, workstations and their capabilities
Updated: 1 day ago
The arrival of high-performance processors from NVIDIA, AMD, and Apple Silicon, together with large amounts of shared memory, has made local AI accessible in different formats: from laptops to compact home workstations. This new class of computers, designed primarily to run AI models locally, is also sometimes called "agentic computers".
They share an architecture in which the processor and integrated graphics accelerator use the same large pool of high-speed memory. This allows a large, capable LLM model to stay close to the compute units, without constantly transferring parts of it over a slow connection to a separate graphics card.
The main practical reason to run AI locally is the ability to process documents, source code, and internal knowledge bases without sending them to a cloud service. For a company, this is useful and often essential when working with confidential information or a private corporate code repository.
Another advantage of hosting an LLM locally is that there is no charge for each API request when using a local model. The direct operating cost of such a model is the electricity needed to power the computer. You should also account for the cost of buying the machine, although it will serve you for several years. Running locally also lets you work without an internet connection and choose the model best suited to particular tasks: coding, agentic work, translation, generating images, video or audio, and so on.
Why is a large amount of shared memory so important?
A conventional PC with a discrete GPU can also be an excellent machine for local AI. As long as the selected model fits entirely in VRAM and leaves room for the KV cache and working buffers, the accelerator can process requests very quickly. It would therefore be inaccurate to consider all computers with a separate graphics card unsuitable for AI.
The limitation appears when a large model outgrows video memory. It is precisely these large models that are most valuable and necessary for professional use. The software either reports insufficient memory, moves part of the work to the CPU and system RAM, or loads weights from RAM or an SSD as needed. These transfers and slower storage can sharply increase latency, potentially making it unacceptable for an interactive assistant. There is no universal slowdown factor: the runtime software, the proportion of offloaded weights, the model, and the hardware all matter. In practice, however, performance in these situations can fall by a factor of 5-10.
A large amount of system RAM alongside a small discrete GPU does not automatically become the same amount of fast VRAM.
Solutions based on NVIDIA RTX/DGX Spark, AMD Ryzen AI Max, and Apple Silicon Max/Ultra have a different emphasis: a large amount of shared memory is available to the graphics accelerator within the supported architecture. This broadens the choice of large models.
128GB or more is the benchmark chosen in this article for large models capable of handling tasks for which users normally turn to cloud models through an API or subscription. Smaller specialized models, or experiments with heavily quantized and distilled models with a limited context window, may need less memory.
Large memory capacities also provide headroom for less compression of the weights, preserving quality better, and for a larger context window.
Where local AI helps in everyday work
Assistant for internal documents. If you have a large documentation database, it may make sense to build a RAG-based system that first finds relevant passages in the local knowledge base, then passes them to the LLM model for an answer with references to the documents. An LLM can search documentation by keyword, but if working with document archives is the main task, RAG can make the search more accurate and efficient because it is a vector database.
Working with source code. The model helps explain parts of a project, suggest changes, write code and test it. Top models from Anthropic or OpenAI will produce better results, but these cloud solutions are not available to everyone, cannot always be used because of confidentiality, and can be very expensive. If cloud solutions are permitted in your work environment, you can distribute the workload: local models handle routine operations, while top cloud models handle the final review and architecture.
Running tests and routine operations in the background or overnight. Your team may have recurring tasks that need to run according to a particular workflow, event or schedule and involve tools: parsing and analyzing logs, testing releases, or finding new documents in local sources and on the internet.
Shared assistant for a small team. A compact workstation can sit in the office, with several computers accessing the local model. NVIDIA PAIR can distribute independent Ollama or LM Studio requests across connected compatible systems.
Image generation. For a computer with 128GB of memory, consider FLUX.1-dev for images: the selected Diffusers weight set, together with text encoders and the VAE, takes up around 33.74GB. NVIDIA publishes instructions for running it through ComfyUI on DGX Spark.
Video generation. An example of a local model for this use case is Wan2.2-T2V-A14B: the original set with two experts in FP32, a text encoder, and a VAE takes up around 126.18GB. Its official 720p example requires at least 80GB.
Speech synthesis. For speech generation in different languages, you can use Qwen3-TTS-12Hz-1.7B-CustomVoice together with the speech tokenizer; the weight files take up around 4.52GB. A larger alternative is MOSS-TTS-v1.5 with 8B parameters: 31 languages, voice cloning from a sample, long recordings, and control over duration, pronunciation through IPA and pauses. The main weights take up around 16.98GB, while the required MOSS-Audio-Tokenizer takes another 7.10GB, for around 24.08GB in total. The Apache 2.0 license permits commercial use. This is a broader feature set, not proof of better sound for every text.
Speech recognition. For local transcription of meetings, interviews, and voice notes, consider Qwen3-ASR-1.7B: the model supports 30 languages, including Russian, automatic language detection, and streaming recognition. The BF16 weight files take up around 4.70GB. Word timestamps additionally use Qwen3-ForcedAligner-0.6B with around 1.84GB of weights. Both models are available under Apache 2.0. Runtime memory usage exceeds the file size and depends on buffers and parallel streams.
A large amount of shared memory alone does not guarantee that a model will run: for each chip, especially AMD and Apple, check runtime compatibility, the amount of memory available to the accelerator, and the supported precision.
Which models suit computers with 32-768GB?
Although we have emphasized that serious professional work with models capable of replacing cloud solutions in many situations requires a machine with 128GB of memory, the following section also provides a reference in the form of the dense model Qwen3.8-27B. It can also be used for coding and can be loaded on a machine with less memory.
The main reference points, however, will be the modern and highly capable open MoE models Qwen3.8-Flash-Next and GLM-5.3-Flash. Qwen3.8 Flash is a cloud release, while Flash-Next is an open release for local installation.
MoE models activate only some of their parameters for each token, but the full set of weights must fit in memory.
Model quantization reduces the size of weights and the cache through a more compact representation of numbers; mixed UD variants use different precision for different parts of the model. Quantization inevitably affects quality, but the effect depends on how aggressive it is. Quantization down to 4 bits (Q4) is considered potentially acceptable in practice, while Q8 is often very close to the original. The following table lists specific published GGUF versions and the sum of all parts of their weights. File size is not the memory usage of the entire application: additional memory is needed for the OS, working buffers, the KV cache and the chosen context length.
If a machine or cluster has plenty of free memory, it can host several copies of a model or different models for simultaneous use.
Memory | Selection conditions | |||
|---|---|---|---|---|
32GB | Q4 + headroom does not fit | Q4 + headroom does not fit | Qwen27B: 32K context fits; 64K is borderline; 128K+ does not fit. | |
64GB | Q4 + headroom does not fit | Q4 + headroom does not fit | Qwen27B: context up to 256K fits. | |
128GB | Q4 + headroom does not fit | Qwen27B: up to 256K. Flash-Next: 32K/64K fits; 128K is borderline. | ||
192GB | Q4 + headroom does not fit | Qwen27B and Flash-Next: up to 256K. | ||
256GB | Context up to 256K fits. | |||
512GB | Context up to 256K. A cluster of 4 nodes needs another 36GB for Qwen27B, 60GB for Flash-Next, and 72GB for GLM Flash: the OS and buffers are needed on every node. | |||
768GB | Context up to 256K. A cluster of 4 nodes needs another 36GB for Qwen27B, 60GB for Flash-Next, and 72GB for GLM Flash: the OS and buffers are needed on every node. |
Comparing models on the same test versions
The values below are percentages from publications by the developers of the original releases, rather than results for the selected quantized GGUF versions. Identical test versions do not guarantee the same agent harness or settings. For example, Qwen3.8-27B was tested in SWE-bench Pro and DeepSWE 1.1 with the Claude Code agent harness and 256K context.
Alongside open models, the table provides benchmark values for well-known cloud models as a reference.
SWE-bench Pro and DeepSWE show coding performance;
Terminal-Bench 2.1 covers autonomous agent tasks;
GPQA Diamond and HLE with tools cover difficult questions and reasoning, with tools in the latter case.
Model and source | SWE-bench Pro | Terminal-Bench 2.1 | DeepSWE v1.1 | GPQA Diamond | HLE with tools |
|---|---|---|---|---|---|
61.7 | 73.0 | 42.2 | 89.2 | No value | |
62.5 | No value | 58.7 | 91.7 | No value | |
No value | 84.3 | 63.4 | No value | 55.3 | |
No value | 90.6 | 74.2 | 90.9 | 63.9 | |
No value | No value | No value | No value | 64.5 | |
No value | No value | No value | 89.9 | 49.0 | |
No value | 88.8 | 72.7 | No value | 64.5 | |
53.4 | No value | No value | 91.3 | No value | |
No value | 85.0 | 58.0 | No value | 57.9 |
Comparing processor performance and memory speed
The peak figures below come from manufacturer documentation. NVIDIA's FP4 performance and AMD's total TOPS describe different operating modes and do not provide a direct speed ranking. Apple does not publish a comparable peak AI metric in the cited specifications, so compute cores and memory bandwidth are listed instead.
Chip in the highest configuration | Published performance | Memory and its speed | Source |
|---|---|---|---|
NVIDIA RTX Spark N1X | Up to 1PFLOP FP4; CPU with 20 cores, GPU with 6144 CUDA cores | Up to 128GB shared memory; 300GB/s, 256-bit bus according to NVIDIA documentation | |
NVIDIA GB10 Grace Blackwell | Up to 1PFLOPS FP4 with sparsity; CPU with 20 cores | 128GB LPDDR5X; 273GB/s | |
Ryzen AI Max+ / PRO 395 | 126TOPS total, NPU up to 50TOPS; CPU with 16 cores, Radeon 8060S with 40 CU | 128GB; Framework uses LPDDR5X 8000MT/s, 256 bits (calculated: 256GB/s) | |
Ryzen AI Max+ PRO 495 | 131TOPS total, NPU up to 55TOPS; CPU with 16 cores, Radeon 8065S with 40 CU | 192GB from Framework; LPDDR5X 8533MT/s, 256 bits (calculated: 273.1GB/s) | |
Apple M4 Max | CPU with 16 cores, GPU with 40 cores, NPU with 16 cores | 128GB; 546GB/s | |
Apple M3 Ultra | CPU with 32 cores, GPU with 80 cores, NPU with 32 cores | Up to 512GB at launch; 819GB/s | |
Apple M5 Max | CPU with 18 cores, GPU with 40 cores, NPU with 16 cores | 128GB; 614GB/s | |
Apple M5 Ultra | CPU with 36 cores, GPU with 80 cores, NPU with 32 cores | Up to 512GB; 1.2TB/s |
RTX Spark N1X and GB10 have similar architectures: their highest versions both have 20 Arm CPU cores and a Blackwell GPU with 6144 CUDA cores. However, the official RTX Spark specifications distinguish the laptop N1X with a TDP of 45-80W from the desktop version with a processor TDP of 140W; GB10 also has a TDP of 140W. There is also a lower-tier laptop N1X with 18 CPU cores and 5120 CUDA cores. Pay attention to the processor version.
For memory, NVIDIA specifies 300GB/s for N1X, and 273GB/s for GB10.
N1X is designed for Windows on Arm, while DGX Spark uses Linux.
The checked sources did not contain a paired N1X and GB10 test using the same model, quantization, and context, so we can only assume that the desktop N1X will deliver results comparable to GB10.
For AMD, direct measurements are available: Tom's Hardware compared Ryzen AI Halo on Max+ 395 with Dell Pro Max on GB10 in llama.cpp through llama-benchy. The table shows results with a context depth of 8192 tokens, an input of 2048, and an output of 256 tokens; the author used GGUF Unsloth Q4_K_M. The AMD machine had 128GB LPDDR5X-8000 and a nominal TDP of 120W. TTFT is the delay until the first token; during generation, more tokens per second means a faster response.
Test model | AMD 395: TTFT | GB10: TTFT | AMD 395: generation | GB10: generation |
|---|---|---|---|---|
Qwen3.6-35B-A3B | 11.4s | 5.0s | 43.8 tokens/s | 58.9 tokens/s |
gpt-oss-120B | 26.0s | 5.9s | 46.4 tokens/s | 50.7 tokens/s |
In these two tests, GB10 wins on both TTFT and generation speed. The author notes that input caching can reduce the TTFT difference.
The speed of a complete agent task can produce a different result. In AMD's own HEPA test both machines ran Qwen3.6-35B-A3B in a 4-bit format and prepared a presentation from the same local documents. AMD Halo finished in 311.6s and DGX Spark in 367.1s, although the individual GPU stage was faster on Spark: 137.2s versus 159.5s. Seven of the eight stages ran on the CPU. This is an example of a specific manufacturer test: for an agent, it is worth measuring time to a finished result as well, including search, document processing, and answer verification.
To compare Apple and NVIDIA solutions, the developer EXO quotes around 26TFLOPS FP16 for M3 Ultra and around 100TFLOPS FP16 for DGX Spark. These estimates suggest a difference of roughly a factor of 3.8 in peak FP16 performance, but that is not a guaranteed difference in local LLM speed.
What matters more for speed: memory or compute units?
When generating a response, a dense model such as Qwen3.8-27B uses all its layers and weights. With one or a few requests, the accelerator often waits for data to be read, making memory bandwidth important: it affects how quickly subsequent tokens appear.
During prefill, the computer processes the input text and creates the KV cache; a long input provides more opportunities for parallel matrix operations. At this prefill stage, strong compute units can shorten the wait for the first token. The effect of large request batches and prediction is discussed in a separate NVIDIA article.
For an agent that repeatedly receives context and returns a short response or a tool call, prefill can account for a significant share of the wait. The MoE models mentioned, Qwen3.8-Flash-Next and GLM-5.3-Flash, also use only selected experts, reducing the demand on memory speed compared with dense models, although it remains an important factor in overall performance.
The developer EXO measured the same Llama-3.1 8B FP16: an input of 8192 tokens and an output of 32 tokens. On DGX Spark, prefill took 1.47s, generation 2.87s, and the total 4.34s; on M3 Ultra Mac Studio, the figures were 5.57s, 0.85s, and 6.42s. In this test, Spark won on input processing and the Mac on generation.
Clusters: 512GB from NVIDIA and 768GB from AMD
With GB10-based DGX Spark, each top-configuration machine has 128GB. Four machines provide 512GB in total. NVIDIA Sync supports setting up such a cluster; four nodes require a suitable QSFP network switch, while a direct connection suits smaller configurations.
AMD also offers a path to distributed execution: the company publishes an example of four nodes with llama.cpp RPC. In addition, Framework states that four Desktop machines with Ryzen AI Max+ PRO 495 and 192GB per node can be combined for up to 768GB in total.
In both cases, the combined memory does not automatically become a single address space. The orchestration software must distribute the model across the nodes; speed depends on the network and the parallelization method.
A cluster can be configured to host a large model, speed up a single request, or increase a service's total throughput. To do this, you can use several distribution strategies:
Tensor parallel: nodes simultaneously compute parts of the matrices in the same layers. They process the request together but exchange results frequently.
Expert parallel: MoE experts are placed on different nodes, and tokens are routed to the required experts. Parallelization can help, but data exchange and uneven utilization can consume the gains.
Pipeline parallel: groups of layers are split into successive stages. One token passes through them in order; multiple requests or microbatches can occupy different stages simultaneously. Pipeline idle time is possible, but this is an effective scenario for a large stream of sequential tasks.
Data parallel: separate model copies serve independent requests. This increases the service's overall throughput when there is enough memory for the copies, rather than automatically speeding up one request.
Modes can be combined, for example tensor + pipeline or tensor + expert, if the selected software supports it. Simply having four nodes does not guarantee a speedup: the network, synchronization, batch size, and load balance matter. NVIDIA provides a separate explanation, showing how to choose a model split to suit the hardware.
DGX Spark has an official two-node vLLM recipe: Qwen3.8-27B NVFP4, tensor parallel across two nodes, and continuous batching of multiple requests. This example shows that computing a model jointly can also serve parallel requests.
Local AI model solutions from leading computer manufacturers
The tables list computer families with top configurations of 128GB or more.
A separate column specifies the charging protocol an adapter must support and the power it must deliver to supply the computer.
NVIDIA RTX Spark N1X laptops and workstations
Model, chip and memory range | Charging with a USB-C adapter (PD) | Documentation and store |
|---|---|---|
Acer SFF RTX Spark | NVIDIA RTX Spark | up to 128GB unified memory | - | |
ASUS ProArt P16 H7607 | N1X | 32-128GB | PD3.1, 140W | |
ASUS ProArt P14 H7407 | N1X | 32-128GB | PD3.1, 140W | |
ASUS ProArt GR1X | NVIDIA RTX Spark N1X | 64/128GB shared memory | USB-C PD IN, EPR PD 3.1; 240W adapter included; required input power and profile are not specified | |
Dell Creator Edition Desktop | NVIDIA RTX Spark | up to 128GB unified memory | - | |
Dell XPS 16 Creator Edition | N1X | 128GB | Power Delivery is specified for three USB-C 4.0 ports | |
HP OmniBook Ultra 16, 16-bk0000 | N1X | 32-128GB | PD3.2 USB-C 140W | |
HP OmniBook X 14 | NVIDIA RTX Spark N1X | up to 128GB unified memory | USB PD 3.2; USB-C GaN adapter 140W | |
Lenovo Yoga Pro 9n / Pro 9n 15 Gen 11 | N1X | 32-128GB | USB-C PD 3.0; a 140W USB-C adapter is supplied with selected configurations; maximum input power is not specified. | |
Microsoft Surface Laptop Ultra | N1X | 24-128GB | USB PD 3.1, at least 115W | |
Microsoft Surface RTX Spark Dev Box | NVIDIA RTX Spark | 128GB unified memory | - | |
MSI EdgeMesa N AI+ | NVIDIA RTX Spark N1X | up to 128GB LPDDR5X unified memory | USB-C PD IN; adapter 240W | |
MSI Prestige N16 Flip AI+ | N1X | 32-128GB | PD3.1, 65-140W |
NVIDIA GB10 / DGX Spark workstations
Model, chip and memory range | Charging with a USB-C adapter (PD) | Documentation and store |
|---|---|---|
Acer Veriton GN100 | GB10 | 128GB | Included USB-C 240W power supply | |
ASUS Ascent GX10 | GB10 | 64/128GB | PD3.1 EPR. The datasheet lists an input of up to 180W; the included power supply is 240W | |
Dell Pro Precision with GB10 (FCM1253; formerly Dell Pro Max with GB10) | GB10 | 128GB | Supplied USB-C power supply rated at 280W, 48V5.83A; PD version not specified. This differs from the standard 240W-48V5A mode | |
GIGABYTE AI TOP ATOM | GB10 | 128GB | USB-C PD IN; 240W adapter included; required input power, voltage profile and PD revision are not specified. | |
HP ZGX Nano G1n AI | GB10 | 128GB | Dedicated USB-C input, 240W power supply. | |
Lenovo ThinkStation PGX | GB10 | 128GB | Dedicated USB-C input with USB PD 3.1; supplied 240W power supply | |
MSI EdgeXpert MS-C931 | GB10 | 128GB | Supplied USB PD 3.1 power supply, 240W, 48V5A | |
NVIDIA DGX Spark | GB10 | 128GB | Supplied USB-C PD3.1 240W power supply | |
PNY NVIDIA DGX Spark | GB10 | 128GB | Power supply included; power and PD mode not specified |
Ryzen AI Max+ 395 and new models with 192GB
Model, chip and memory range | Charging with a USB-C adapter (PD) | Other DC input or mains power supply | Documentation and store |
|---|---|---|---|
Acer Veriton RA110 | Max+ 395 | 128GB | - | Separate rear DC connector; the included adapter's power rating is not specified in the checked source. | |
AMD Ryzen AI Halo | Max+ 395 | 128GB | Dedicated USB-C input for charging with the supplied power supply | - | |
AMD Ryzen AI Halo | Max+ PRO 495 | 192GB unified memory | - | - | |
ASUS ProArt PX13 HN7306 | Max+ 395 | 128GB | USB PD 45-100W supported | Rectangular DC input: 200W power supply, 20V10A | |
ASUS ROG Flow Z13 GZ302 | Max+ 395 in the highest version | 32-128GB | USB PD 3.0, 45-100W | Rectangular DC input: 200W power supply, 20V10A | |
Beelink GTR9 Pro | Max+ 395 | 128GB | - | Internal 230W mains power supply | |
CORSAIR AI Workstation 300 | Max+ 395 | 64/128GB | - | Internal Flex ATX 300W power supply | |
Framework Desktop | 32/64/128/192GB; Max+ 395: 64/128GB, Max+ PRO 495: 192GB | - | Internal Flex ATX 400W power supply | |
GEEKOM A9 Mega | AMD Ryzen AI Max+ 395 | 128GB shared memory | USB-C PD input is not specified in the checked materials | DC-in; 330W adapter included | |
GMKtec EVO-X2 | Max+ 395 | 64-128GB | - | DC5525: 19.5V11.8A power supply, around 230W | |
GMKtec EVO-X5 Pro | Ryzen AI Max+ PRO 495 | 192GB unified memory | USB-C power input is not supported | DC IN 19.5V16.9A, 330W; connector 5.5 x 2.5 mm | |
HP Z2 Mini G1a | Max+ PRO 395 in the highest version | 16-128GB | - | Internal 300W mains power supply | |
HP ZBook Ultra 14 G1a | Max+ PRO 395 in the highest version | 16-128GB | USB-C 140W power supply | - | |
HP ZBook Ultra G3a 16 | Ryzen AI Max+ PRO 495 | up to 192GB unified memory | - | - | |
Lenovo ThinkCentre X Ultra | Ryzen AI Max+ PRO 495 | up to 128GB unified memory | - | - | |
MINISFORUM MS-S1 MAX | Max+ 395 | 64/128GB | - | Internal 320W mains power supply | |
MINISFORUM MS-S1 MAX-P495 | Max+ PRO 495 | 192GB unified memory | - | Built-in power supply 320W | |
MSI PRO MAX EDGE AI+ 11M | Max+ 395 | 128GB | USB PD input not specified; both USB-C ports: PD output 15W | Internal 300W Flex mains power supply |
Apple Silicon: the two latest generations
For MacBook Pro, this covers M5 Max and M4 Max with a 40-core GPU and 128GB. For Mac Studio, it covers the M5 Max / M5 Ultra generation and the previous M4 Max / M3 Ultra. Apple announced the 512GB M3 Ultra version at launch, but the current specifications page shows a maximum of 256GB.
Model, chip and memory range | Charging with a USB-C adapter (PD) | Other DC input or mains power supply | Documentation and store |
|---|---|---|---|
MacBook Pro 14" (2024) | M4 Max | 36-128GB; 128GB requires a 40-core GPU | PD3.0 (<100W) | MagSafe 3: from PD 3.0 | Specifications | Fast charging | Amazon.com(36Gb) |
MacBook Pro 16" (2024) | M4 Max | 36-128GB; 128GB requires a 40-core GPU | PD3.1 (140W) | MagSafe 3: from PD 3.1 | Specifications | Fast charging | Amazon.com(48Gb) |
Mac Studio (2025) | M4 Max | 36-128GB; 128GB requires a 40-core GPU | - | Built-in mains power supply; 480W | Specifications | Amazon.com(36Gb) |
Mac Studio (2025) | M3 Ultra | 96-256GB in the current specification; 512GB at launch | - | Built-in mains power supply; 480W | Specifications | Amazon.com(256Gb) |
Mac Studio | M5 Ultra | 96/256/512GB; 512GB: shipping later in October 2026 | - | Built-in mains power supply; 480W | |
MacBook Pro 14" | M5 Max | 36-128GB; 128GB requires a 40-core GPU | PD3.0 (<100W) | MagSafe 3: from PD 3.0 | Specifications | Fast charging | Amazon.com(128Gb) |
MacBook Pro 16" | M5 Max | 36-128GB; 128GB requires a 40-core GPU | PD3.1 (140W) | MagSafe 3: from PD 3.1 | |
Mac Studio | M5 Max | 36-128GB; 128GB requires a 40-core GPU | - | Built-in mains power supply; 480W | Specifications | Amazon.com(36Gb) |
MagSafe 3 on a MacBook is a proprietary connector on the laptop side that receives power from a USB-C charging adapter through the appropriate charging cable.
How to ensure a reliable power supply for your computer
Check what is included: some computer configurations are sold without an adapter, especially in Europe, while others include one.
A computer that will run a local AI model for many hours every day needs a reliable source capable of continuously delivering the required power without interruptions.
Check voltage, current, protocol, and the permitted cable against the documentation for the specific machine. If the manufacturer requires a proprietary power supply or a non-standard mode, a USB-C connector does not justify replacing it with any high-power charger.
If you choose a multiport power supply, pay attention to the power delivered by one port and how it changes when a second device is connected, if that scenario is relevant to you.