Zero-Shot, But at What Cost? Unveiling the Hidden Overhead of MILS’s LLM-CLIP Framework for Image Captioning

This paper written by the NStarX Engineers is the first to expose and quantify the hidden computational costs of MILS (Multimodal Iterative LLM Solver), a recently published framework for zero-shot image captioning. While MILS claims LLMs can "see and hear without training" and demonstrates good performance, the authors reveal it achieves this through an expensive multi-step iterative refinement process. They show that alternative models like BLIP-2 and GPT-4V achieve competitive results more efficiently with single-pass approaches. The key contribution is systematically measuring the trade-offs between output quality and computational overhead, challenging the assumption that zero-shot performance comes without heavy resource demands and providing insights for designing more efficient multimodal models.