You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8a72618
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: docs/apfel-vs-fm.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ macOS 27 ships Apple's own command-line front end for the on-device Foundation M
4
4
5
5
This page is the single source of truth for the comparison. Every number below was measured, and the exact commands are listed at the end so you can re-run them.
6
6
7
-
**Measured on:** apfel 1.12.0 (main, 2026-10-05, built against the macOS 27.0 SDK, deployment floor macOS 26.0), Apple `fm` 1.0 (`/usr/bin/fm` as shipped with macOS 27.0.1, build 26A434), MacBook Air with Apple M2 and 24 GB (this chip gets the 4096-token AFM 3 Core model; M3+ Macs with 12 GB+ get the 8192-token AFM 3 Core Advanced model - the 8192 reading is [#192](https://github.com/Arthur-Ficial/apfel/issues/192), the first GA run on such a Mac is [#509](https://github.com/Arthur-Ficial/apfel/issues/509)), 2026-10-05. Numbers are re-measured for every apfel release and after every macOS point update.
7
+
**Measured on:** apfel 1.13.0 (the released binary, built against the macOS 27.0 SDK, deployment floor macOS 26.0), Apple `fm` 1.0 (`/usr/bin/fm` as shipped with macOS 27.0.1, build 26A434), MacBook Air with Apple M2 and 24 GB (this chip gets the 4096-token AFM 3 Core model; M3+ Macs with 12 GB+ get the 8192-token AFM 3 Core Advanced model - the 8192 reading is [#192](https://github.com/Arthur-Ficial/apfel/issues/192), the first GA run on such a Mac is [#509](https://github.com/Arthur-Ficial/apfel/issues/509)), 2026-10-05. Numbers are re-measured for every apfel release and after every macOS point update.
8
8
9
9
## TL;DR
10
10
@@ -95,14 +95,14 @@ All measurements on the machine in the "Measured on" line above, warm model, no
95
95
96
96
| Measurement | apfel |`fm`|
97
97
|---|---|---|
98
-
| CLI one-shot, "Reply with exactly: hello", warm model, median of 5 |1.38 s (best 1.19 s) |1.29 s (best 1.18 s) |
99
-
| CLI one-shot, "Write a haiku about autumn." |2.50 s (best 2.02 s) |2.17 s (best 1.81 s) |
100
-
| CLI one-shot, "List five European capitals, one per line." |1.89 s (best 1.79 s) |1.69 s (best 1.59 s) |
101
-
| Process start without the model (`apfel --version` / `fm --help`), median of 10 | 0.08 s | 0.13 s |
102
-
| HTTP `/v1/chat/completions`, "Reply with exactly: hello", non-streaming, median of 5 |1.30 s | 0.73 s (`stream: false`) |
98
+
| CLI one-shot, "Reply with exactly: hello", warm model, median of 5 |0.42 s (best 0.41 s) |0.32 s (best 0.32 s) |
99
+
| CLI one-shot, "Write a haiku about autumn." |0.98 s (best 0.96 s) |0.83 s (best 0.81 s) |
100
+
| CLI one-shot, "List five European capitals, one per line." |0.87 s (best 0.86 s) |0.78 s (best 0.73 s) |
101
+
| Process start without the model (`apfel --version` / `fm --help`), median of 10 | 0.008 s | 0.008 s |
102
+
| HTTP `/v1/chat/completions`, "Reply with exactly: hello", non-streaming, median of 5 |0.48 s | 0.30 s (`stream: false`) |
103
103
| Binary size on disk (SI megabytes) | 22.1 MB | 3.4 MB |
104
104
105
-
The CLI gap of 0.1 to 0.3 s per call is apfel's pre-flight work: it counts prompt tokens with the real tokenizer to budget the context window and reports `usage`, `fm respond` does not. The larger server gap (about 0.6 s on this request) is on apfel's side and tracked as a performance issue; the model time itself is identical.
105
+
The CLI gap of about 0.1 s per call is apfel's pre-flight work: it counts prompt tokens with the real tokenizer to budget the context window and reports `usage`, `fm respond` does not. The server gap (about 0.18 s on this request) is on apfel's side and tracked as [#504](https://github.com/Arthur-Ficial/apfel/issues/504); the model time itself is identical. All numbers were taken on an idle machine; an earlier run of the same script under background load read three times higher across the board, so compare only against measurements taken the same way.
106
106
107
107
Token counts agree exactly between the two tools because both call `SystemLanguageModel.tokenCount(for:)`:
108
108
@@ -146,7 +146,7 @@ apfel --count-tokens -o json 'Summarize this short sentence.'
146
146
fm count-tokens -q 'Summarize this short sentence.'
147
147
```
148
148
149
-
Latency, one-shot prompt (run each line five times, take the median):
149
+
Latency, one-shot prompt (idle machine, run each line five times, take the median):
0 commit comments