Add internal Apple's on-device Foundation Model integration in Tahoe
complete
Louis Faure
Tahoe now enables access to Apple's on-device LLM from within apps (https://developer.apple.com/documentation/foundationmodels).
Integrating this model within Kerlig would be wonderful! Here is how someone has already experimented with that approach: https://github.com/gety-ai/apple-on-device-openai
Jarek Ceborski
complete
Foundation Model is now added in Kerlig 2.5.0
https://www.kerlig.com/releases#2.5.0
Louis Faure
I have just tested the AppleOnDevice app from https://github.com/gety-ai/apple-on-device-openai which effectively expose Apple's on-device LLM as an OpenAI compatible API.
When adding it to Kerlig using a custom integration, it correctly detects the model name but no message is generated (error message showing). After some investigation, I discovered that the issue occurs because Kerlig sends a POST request with a structured content array in the message body, which the app doesn’t process. Once support for that format is enabled, it works correctly!
Simon
It would also be good to use Apple's speech to text model to quickly dictate prompts. Apparently, the built-in, offline model in the new MacOS beta is as accurate and faster than OpenAI's whisper (https://9to5mac.com/2025/06/18/apple-devices-offer-amazing-speech-to-text-transcription-in-developer-betas-shows-test/)
anschmieg
Simon and TTS would be nice as well :)