This survey explores the computational and memory challenges inherent in Video Large Language Models (VideoLLMs), which integrate video representations with pretrained LLMs to perform tasks like temporal grounding and captioning. As these models scale, their resource requirements grow significantly alongside frame counts and context lengths, hindering deployment in real-time or mobile environments. The researchers categorize inference-efficiency mechanisms based on their stage in the processing pipeline, specifically targeting bottlenecks in frame sampling, modality encoding, connector-level token reduction, and the LLM prefilling and decoding phases. The authors analyze various methodologies designed to reduce parameter counts, FLOPs per input, latency, and the total number of visual or audio tokens. By reviewing literature published since late 2022, the study provides comparative analyses of accuracy versus computational cost, where shared host models and input protocols allow for direct measurement. The survey identifies critical research gaps, particularly regarding audiovisual efficiency and the lack of standardized evaluation metrics for these complex systems. Ultimately, the work serves as a comprehensive guide to the current landscape of efficiency-driven video and audiovisual model architectures.