Browser LLMs in PWAs – Field Test, Storage Silos and the 400 MB Hurdle

You are engineering a Progressive Web App and want to deliver on the promise of offline on-device AI: zero cloud latency, zero server bills, and absolute privacy on client devices. But the initial field test delivers a harsh reality check: a 480-megabyte download for even the smallest coherent model, thermal throttling, and answers that crumble on basic reasoning tasks. Why is there such a massive gulf between WebGPU demo hype and real-world production viability?

Local language models in PWAs fail in practice due to three architectural and physical barriers: the web’s Same-Origin isolation that duplicates model weights per origin, the 400-megabyte minimum threshold for multilingual utility, and the total lack of standardized native browser APIs on mobile devices. While vector embeddings at 25 megabytes work reliably offline, generative in-browser chat models remain an expensive storage and user-experience antipattern for mobile PWAs.

Why Does the Vision Collide with the Web's Storage Architecture?

The foundational security model of the modern web is the Same-Origin Policy (A browser security mechanism that prevents scripts on one origin from accessing storage, cookies, or databases of another origin.). Every web application runs inside an isolated sandbox. While essential for credential security, this sandbox turns into an architectural disaster when handling multi-hundred-megabyte weight files:

When a user installs three separate Progressive Web Apps (PWAs) (Web applications built with modern browser APIs that can be installed to the home screen and function offline via Service Workers.), each offering an on-device chatbot or offline assistant, no app can read or share the model cache of its neighbor.

Origin Isolation: The Multi-App Storage Trap

On native operating systems, applications share common shared libraries or query a single operating-system neural engine (such as Apple Intelligence or Windows Copilot Runtime). The web platform lacks this shared tier entirely:

1. Origin A (notes-app.dev) downloads a quantized Qwen model (483 MB) into its private Cache Storage API (A browser API for storing network request/response pairs persistently, typically managed by a Service Worker.).
2. Origin B (lukiluki.me) downloads the exact same 483 MB binary weights into its own isolated origin cache.
3. Origin C (task-tracker.app) downloads the weights a third time.

Across three installed PWAs, identical tensor matrices consume nearly 1.5 gigabytes of flash storage on the client device. On smartphones where users constantly juggle storage limits for photos, operating-system updates, and videos, this uncoordinated footprint triggers immediate frustration and uninstalls.

Cache Eviction on Mobile Engines

The storage dilemma is aggravated by aggressive eviction heuristics enforced by mobile browser engines:

• WebKit (iOS Safari): Safari subjects client storage to strict lifetime rules. If a web app is not launched frequently or added to the home screen, Intelligent Tracking Prevention (ITP) and storage pressure algorithms evict IndexedDB and CacheStorage. When the user opens the PWA mid-flight, the model directory is gone.
• Chromium (Android): When free disk space on a device runs low, Chromium purges non-persisted origins using a least-recently-used (LRU) algorithm. A 500 MB download offers zero durability guarantee.

While navigator.storage.persist() exists to request durable storage, browsers grant this privilege conservatively based on internal heuristics and site engagement scores.

The Model Bottleneck: Why 400 Megabytes Is the Multilingual Floor

Why not simply deploy an ultra-compact 50 MB language model? The answer lies in the fundamental relationship between parameter scale, token density, and reasoning depth. For generative chat in the browser, developers are effectively restricted to three model families:

The Three In-Browser Model Families

The Quality Cliff Below One Billion Parameters

Models under 500 million parameters (such as SmolLM2-135M or 360M) simply lack the dimensionality to represent nuanced world models, multilingual grammar, and strict instruction adherence:

1. Language Distortion: The vast majority of sub-billion pre-training corpora is English. On non-English prompts, token density drops, resulting in hallucinated tokens, mid-sentence language switching, or broken agreement.
2. System Prompt Collapse: When passing a RAG context (Retrieval-Augmented Generation: Supplying extracted document chunks directly inside the prompt so the model bases its answer on verified facts.), sub-500M models regularly ignore system instructions to quote strictly from source material and invent plausible falsehoods instead.
3. WebGPU Buffer Allocations: WebGPU enforces strict limits via maxBufferSize and maxStorageBufferBindingSize. On integrated mobile GPUs, allocating intermediate attention tensors frequently fails with requested maxBufferSize exceeds limit.

To give users coherent multilingual output, developers are forced to use the 1-billion-parameter tier – requiring a 700 MB initial payload.

Where Does the Chrome Prompt API Stand and Why Is a Standard Missing?

Wouldn’t the logical architectural solution be a browser-supplied foundation model accessible via standard JavaScript? This is the vision behind Google's Prompt API (An experimental W3C community group specification (LanguageModel) exposing an in-browser foundation model directly to JavaScript.) and Gemini Nano.

However, the reality in 2026 demonstrates why web architects cannot build mobile PWA stacks on this API:

Google's 22-Gigabyte Hardware Gate

The Prompt API (window.LanguageModel) shipped in Chrome 148 for desktop operating systems. But its hardware gates exclude the mobile PWA market completely:

• Zero Support on Android and iOS: Neither Chrome on Android nor Chrome on iOS supports the built-in foundation-model APIs.
• The 22 GB Storage Gate: Chrome downloads Gemini Nano only if the volume containing the user profile has at least 22 GB of free space. If free space drops below 10 GB later, the model is automatically evicted.
• Hardware Prerequisites: Strictly more than 4 GB of VRAM for the GPU pipeline, or at least 16 GB of RAM and 4 CPU cores for CPU inference.
• No Worker Execution: The APIs are not exposed inside Dedicated Web Workers or Service Workers.

Why Apple and Mozilla Oppose the Specification

The Prompt API is not a W3C Recommendation track standard; it is a Draft Community Group Report produced by the Web Machine Learning Community Group. Both major competing engine vendors have officially rejected the proposal:

• Mozilla (Firefox): Holds an official stance of "Negative Standards Position". The reasoning: shipping browser-specific models destroys web interoperability. Prompts tuned for Gemini Nano produce erratic, divergent behavior when executed against a different model like Phi-4-mini in another browser.
• Apple (WebKit/Safari): Holds an official stance of "Oppose Position", citing critical privacy concerns, lack of portability, and vendor lock-in that would force websites into browser-sniffing branches.

Without consensus between Google, Apple, and Mozilla, there is no shared platform layer for web AI. Any site attempting on-device chat today must bundle and ship hundreds of megabytes of raw weights itself.

Field Test in Looki Looki Me: Benchmarks and Code

During the Progressive Web App migration of lukiluki.me, we ran an empirical test: What happens when an app built for speed and lean resource consumption integrates a real in-browser chat model?

Empirical Metrics: Application Code vs. Model Weights

The production build figures highlight the architectural imbalance:

• Complete lukiluki.me offline manifest: 74.6 MB (49 pre-rendered routes, 232 static assets, typography, and the full compiled ONNX WebAssembly SIMD runtime).
• Adding the smallest multilingual chat model (Qwen3-0.6B-q4f16_1): +483 MB additional download.
• Ratio: The AI model accounts for 86.6% of the entire PWA’s installed footprint.
• Cold start to first token: On a flagship smartphone, decompressing tensor weights and compiling WebGPU pipelines requires 9 to 16 seconds. During pipeline compilation, UI frame rates in adjacent tabs drop noticeably.

Resilient Download and Quota Pre-Flight in TypeScript

If your project requires on-device models for specialized enterprise offline workflows, you must never precache weights during the Service Worker install phase. Instead, execute an explicit pre-flight quota check and stream weights in cancelable chunks:

1/**
2 * Production-ready quota verification and chunked weight streaming.
3 * Prevents out-of-quota aborts and supports clean cancellation via AbortSignal.
4 */
5interface ModelDownloadOptions {
6  modelUrl: string;
7  expectedBytes: number;
8  signal: AbortSignal;
9  onProgress: (ratio: number) => void;
10}
11
12export async function prepareLocalModelStorage({
13  modelUrl,
14  expectedBytes,
15  signal,
16  onProgress,
17}: ModelDownloadOptions): Promise<ArrayBuffer> {
18  // 1. Pre-flight storage quota before saturating the network
19  if (navigator.storage && navigator.storage.estimate) {
20    const { quota, usage } = await navigator.storage.estimate();
21    const availableBytes = (quota ?? 0) - (usage ?? 0);
22
23    // Maintain a 200 MB safety buffer for background browser operation
24    if (availableBytes < expectedBytes + 200 * 1024 * 1024) {
25      throw new DOMException(
26        `Insufficient storage: ${Math.round(availableBytes / (1024 * 1024))} MB free, ` +
27        `required: ${Math.round(expectedBytes / (1024 * 1024))} MB.`,
28        "QuotaExceededError"
29      );
30    }
31  }
32
33  // 2. Fetch with explicit stream reading and cancellation support
34  const response = await fetch(modelUrl, {
35    signal,
36    headers: { Accept: "application/octet-stream" },
37  });
38
39  if (!response.ok || !response.body) {
40    throw new Error(`Model weight transfer failed with status ${response.status}`);
41  }
42
43  const contentLength = Number(response.headers.get("Content-Length")) || expectedBytes;
44  const reader = response.body.getReader();
45  const chunks: Uint8Array[] = [];
46  let receivedBytes = 0;
47
48  while (true) {
49    const { done, value } = await reader.read();
50    if (done) break;
51
52    chunks.push(value);
53    receivedBytes += value.length;
54    onProgress(Math.min(1, receivedBytes / contentLength));
55  }
56
57  // 3. Assemble binary tensor buffer in memory
58  const result = new Uint8Array(receivedBytes);
59  let offset = 0;
60  for (const chunk of chunks) {
61    result.set(chunk, offset);
62    offset += chunk.length;
63  }
64
65  return result.buffer;
66}

How Does Your Current Browser Behave?

Use the following auditor to inspect your device's actual WebGPU adapter, query available storage quota, and calculate the compounding storage duplication across multiple installed PWAs.

The auditor illustrates the core dilemma: even if your current browser supports WebGPU and allocates generous quota, storage consumption scales linearly with every additional web app. There is zero cross-origin deduplication.

The Pragmatic Architecture: What Actually Works Offline

Does this invalidate the concept of offline AI for the web? Not at all. It simply proves that generative chat models are the wrong architectural fit for mobile PWAs today.

The production architecture powering lukiluki.me separates semantic filtering from generative text synthesis:

To understand how lightweight vector math operates without multi-hundred-megabyte baggage, explore our deep dive into Embeddings – Finding Similar Content.

Architecture Checklist for PWA Engineers

Before deciding to integrate an in-browser language model into your web application or PWA, verify these five criteria:

• Storage budget audited: Does the model payload combined with application assets remain strictly within a defensible cache budget of under 50 MB for mobile users?
• Target languages verified: Has the model been evaluated against real-world non-English prompts to ensure it does not switch languages or suffer syntactic breakdown?
• Eviction resiliency confirmed: Does the core application continue to function gracefully when Safari or Android evicts the model cache after several days of inactivity?
• Vector embeddings evaluated: Can the functional requirement (e.g. search, document triage, classification) be solved using a 25 MB embedding pipeline instead of a 500 MB generative model?
• Data transparency provided: Is the exact download size, memory requirement, and battery impact disclosed to the user before initiating any transfer?

Sources and Next Steps

The following primary sources document technical storage boundaries and the active browser standardization debates:

• W3C Prompt API Explainer – Community Group specification draft for the LanguageModel interface.
• Chrome for Developers: Built-in AI Requirements – Official Google documentation covering the 22 GB storage gate and desktop constraints for Gemini Nano.
• Mozilla Standards Positions: Issue #1213 – Mozilla’s formal rejection of the Prompt API due to web interoperability concerns.
• WebKit Standards Positions: Issue #495 – Apple’s opposing position regarding Prompt API inclusion in Safari.
• WebLLM Paper (ArXiv:2412.15803) – Technical evaluation of WebGPU storage buffer allocations and in-browser inference limits.
• MDN Web Docs: Storage Quotas and Eviction Criteria – Comprehensive rules governing browser cache lifetimes and quota policies.

Review the architectural runtime foundation in Offline AI in Web Apps, explore Embeddings – Finding Similar Content for resource-friendly vector search, and read WebNN – Machine Learning in the Browser for upcoming hardware-accelerated NPU standards.