ai-raccoon on the Neural Engine: Twice as Fast at a Quarter of the Energy
How ai-raccoon 1.53 moved its embedding model from the GPU to the Apple Neural Engine on an M4: 77 mJ per chunk instead of 283, half the time, faster searches. What it took to get there (a re-exported fp16 model in Apple's ANE layout, published on Hugging Face, and a background CoreML compile) and a plain-language primer on NPUs.
[ Read more → ]