if you're already starting to fine-tune open source models for smaller tasks, using something like the @runanywhereai sdk can let you offload some of the inference to user devices, lowering cloud inference costs and latency but there's probably more fun/unique applications i'm not thinking of
ollama for mobile, picking up attention

Nov 7, 2025 · 11:01 PM UTC

7
6
30
federated learning. This will unlock that if the experience is smooth.
1
if you're already starting to fine-tune open source models for smaller tasks, using something like the runanywhereai sdk can let you offload some of the inference to user devices, lowering cloud inference cost but there's probably more fun/unique applications i'm not thinking of
1
Happy birthday pippin🦄❤️
1
When pippin
2
Rapid SDK adoption is fueling bold new mobile AI experiments. On-device inference isn't just about lower costs-it's a launchpad for creative apps. Curious to see what devs build next.
1
Edge-offload unlocks new plays: on-device RAG for private docs, client-side eval loops for agents, and split compute (server for recall, device for generation) to cut cost/latency. Add PII-safe prompts and offline fallbacks. 🤖📱⚡