Skip to content

file-extraction.md

Latest commit

 

History

History
71 lines (51 loc) · 3.66 KB

file-extraction.md

File metadata and controls

71 lines (51 loc) · 3.66 KB
Raw

File extraction (-f and piped files)

apfel turns files into prompt text on-device before sending them to the model. Text files pass through unchanged; PDFs and images are extracted with Apple's Vision and PDFKit via the shared lesbar package (also used by auge). No cloud, no API keys, no network.

What each file type produces

Input How it is extracted What the model receives
Text (.txt, .md, source, JSON, ...) UTF-8 decode the raw text, unchanged
PDF PDFKit text layer; per-page Vision OCR fallback for scanned pages === name (pdf) === header + the text
Image (JPEG, PNG, HEIC, TIFF, GIF, BMP, WebP) Vision OCR and Vision classification === name (image) === + "what the image shows" (classification) + the OCR text
Unknown / unsupported binary rejected a clear error, no silent garbage

For images, OCR alone is not enough: a photo with no text still gets a sense of its content from classification, and a photo with text gets both. A photo of a receipt yields its line items; a photo of a beach yields "beach, ocean, sky".

Attach a file with -f

apfel -f report.pdf "Summarize the key findings"
apfel -f receipt.jpg "What is the total?"

Attach several files at once:

apfel -f old.swift -f new.swift "What changed between these two files?"

Pipe a file straight in

Piping a PDF or image works the same way as -f:

cat report.pdf | apfel "Summarize this"
cat photo.jpg | apfel "What is in this picture?"

Native image input on macOS 27

On macOS 27 the model itself accepts images, so apfel -f photo.jpg (and piped image bytes) sends BOTH: the extracted OCR/classification text stays in the prompt - small print is only reliable through OCR - and the image rides along as a native attachment for the model to look at. --debug says which happened (image attached natively vs image sent text-only). On macOS 26 nothing changes: images are text-only. --count-tokens does not include the attachment (the runtime prices it at generation time) and says so on stderr. One macOS 27 extra: an image Vision can neither read nor label no longer errors out - it still reaches the model natively, as long as the bytes decode as an image.

Check the token budget first

Extraction is model-free, so you can preflight how much a file adds to the prompt:

apfel --count-tokens -f report.pdf "Summarize this"

Honest limits

  • On macOS 27 the default guardrails sometimes block prompts whose attached text came from OCR (measured: the Apollo 11 plaque photo was refused 3 times out of 5 on 27.0.1, never on macOS 26.1). If apfel -f photo.jpg ... exits 3 with [guardrail], add --permissive: it selects Apple's guardrail set for transforming user-provided content, which is what file extraction is (docs/PERMISSIVE.md).
  • The on-device model has a small context window (4096 tokens on macOS 26 and on M1/M2 Macs, 8192 on macOS 27 with an M3 or newer chip and 12 GB+ (Apple's AFM 3 Core Advanced model) - read at runtime). A large PDF can exceed it; use --count-tokens to check.
  • OCR quality depends on the image. Engraved, handwritten, or low-contrast text may come out partial. apfel reports what Vision actually read and never invents text.
  • Image classification labels are Vision's best guess. When nothing is confident, apfel says so ("could not confidently identify the image") rather than making something up.
  • HTML and web archives are not extracted (they can fetch remote resources); save as PDF or text first.