A Systematic Taxonomy of Large Language Model Inference Optimization: Bridging Model-Level and API-Layer Latency Reduction Strategies
Yao-Liang Chung
Progress in Computing Applications, Vol. 15, No. 2, 2026.
DOI: https://doi.org/10.6025/pca/2026/15/2/82-103
Link: https://www.dline.info/pca/fulltext/v15n2/pcav15n2_3.pdf
Abstract:
The deployment of Large Language Models (LLMs) as autonomous agents in next generation infrastructures,
such as Agentic AI-Native 6G networks, introduces severe latency, memory, and operational cost bottlenecks.
Existing optimization approaches frequently treat model compression and system orchestration in isolation,
failing to adequately address the multidimensional constraints of modern production environments. This
paper presents a systematic, multi criteria taxonomy of LLM inference optimization, bridging granular
model level compression techniques with macro level API-layer orchestration strategies. We conduct a
rigorous comparative analysis of post training quantization algorithms specifically highlighting activationaware
methods like AWQ alongside GPTQ and GGUF evaluating their trade offs across memory efficiency,
inference throughput, and predictive quality. Our findings reveal that deployment efficiency is not solely
dictated by numerical precision but is fundamentally a co-design problem requiring optimized hardware
kernels and adaptive, importance aware parameter protection. Furthermore, we demonstrate that systemlevel
interventions, such as intelligent model routing, prompt caching, and deterministic decoupling, yield
immediate latency reductions without altering underlying model weights. Ultimately, this study provides a
comprehensive, scenario based decision framework for matching specific quantization algorithms and APIlayer
strategies to distinct deployment environments, ranging from high throughput GPU serving to resourceconstrained
edge devices and safety critical autonomous systems. By unifying these perspectives, we establish
a foundational guide for practitioners aiming to build reliable, high performance inference pipelines that
ensure scalable and economically sustainable AI integration across diverse industrial domains.