OpenClaw requests are primarily dominated by long, tool-augmented prefixes, including system prompts, conversation history, and tool outputs fed back into the context window. For this workload, with about 28k-30k input tokens and 500 output tokens per request, the service quality is governed by throughput, TTFT (Total Time to First Byte), and tail latency rather than short-prompt throughput alone.
This report studies the GLM-5 serving-parameter tuning within a MaaS multi-model inference optimization architecture. The scope focuses on the Single-Node Optimization block of the inference-optimization layer, where chunked prefill, tensor parallelism (TP), pipeline parallelism (PP), and request concurrency are tuned for one GLM-5 serving deployment. In this report, 'Single-Node Optimization' denotes the architecture block, while experiments run on a two-node, sixteen-GPU cluster.
Within the tested parameter space, the best configuration is chunked-prefill-size=3072, tp=4, pp-size=4, and max-running-requests=24. Compared to the conservative 2048/4/4/16 baseline, this configuration increases request throughput from 0.43 to 0.48 req/s and total token throughput from 9029.64 to 9993.23 tok/s, while reducing average TTFT from 8.98 to 6.69 seconds and P90 latency from 40.23 to 32.64 seconds. Under the same hardware footprint, this corresponds to an estimated 10.4% lower serving cost per request and 9.6% lower cost per token. The results show that the optimum is workload-specific: larger chunk sizes and deeper queueing do not monotonically improve performance. Therefore, we recommend 3072 / tp4 / pp4 / max24 as the default OpenClaw deployment profile.
Blogger's Review: This article meticulously analyzes the optimization potential of GLM-5 for long-context processing, demonstrating how tuning service parameters can significantly enhance performance and reduce costs under specific load conditions. It provides valuable insights for future research and practical applications.