Support multimodal inputs

[yeroc]

Would be awesome if support for multimodal input were added? Is this planned at all?

[kherud]

Hey @yeroc I would love to have support for multimodal! It's on my agenda, but a rather complicated addition (can't give an ETA yet).

[bernardladenthin]

I noticed the repository hasn't seen recent activity, so I've taken the liberty of forking it and continuing development at: https://github.com/bernardladenthin/java-llama.cpp

See also #125

[bernardladenthin]

Implemented in the downstream fork at https://github.com/bernardladenthin/java-llama.cpp
in PR #189. The upstream mtmd library is linked into jllama
(CMakeLists.txt:125-145,253-255); ModelParameters exposes the mmproj
flags (ModelParameters.java:1250-1281); and the typed image API ships as
ContentPart + ChatMessage(role, List<ContentPart>) +
InferenceParameters.setMessages(List<ChatMessage>), which emit the OAI
multipart content the upstream chat path already routes through mtmd.
See #103 for the parallel write-up.