话显存占用还是持久化的保存着。对于本地计算资源比较紧张的用户来说这也是一种资源的浪费。本文提供一个简单的指令可以将闲置的缓存释放。运行指令核心的缓存释放操作比较简单直接运行如下指令即可$ curl -X POST http://localhost:30000/flush_cache这个操作不一定只能在本机运行一般如果是远端会话结束也可以用这条指令清空缓存开启一个新的会话。单用户场景先看下只有一个用户在使用时释放缓存的效果。在释放缓存之前这个sglang任务所占据的显存空间为27196MiB而运行上述指令清空缓存之后剩下27064MiB的显存空间。由于显卡资源比较有限没必要让所有的会话都一直用缓存占据着空间闲时或者有紧急任务穿插的时候可以释放一部分显存出来用。$ nvidia-smiThu Jul 2 16:00:53 2026±----------------------------------------------------------------------------------------| NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |±----------------------------------------±-----------------------±---------------------| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. || | | MIG M. |||| 0 NVIDIA GeForce RTX 5090 D Off | 00000000:01:00.0 Off | N/A || 0% 35C P8 12W / 600W | 27220MiB / 32607MiB | 0% Default || | | N/A |±----------------------------------------±-----------------------±---------------------±----------------------------------------------------------------------------------------| Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage |||| 0 N/A N/A 1948 G /usr/lib/xorg/Xorg 4MiB || 0 N/A N/A 3414955 C sglang::scheduler 27196MiB |±----------------------------------------------------------------------------------------$ curl -X POST http://localhost:30000/flush_cacheCache flushed.Please check backend logs for more details. (When there are running or waiting requests, the operation will not be performed.)$ nvidia-smiThu Jul 2 16:00:56 2026±----------------------------------------------------------------------------------------| NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |±----------------------------------------±-----------------------±---------------------| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. || | | MIG M. |||| 0 NVIDIA GeForce RTX 5090 D Off | 00000000:01:00.0 Off | N/A || 0% 36C P1 59W / 600W | 27088MiB / 32607MiB | 0% Default || | | N/A |±----------------------------------------±-----------------------±---------------------±----------------------------------------------------------------------------------------| Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage |||| 0 N/A N/A 1948 G /usr/lib/xorg/Xorg 4MiB || 0 N/A N/A 3414955 C sglang::scheduler 27064MiB |±----------------------------------------------------------------------------------------多用户场景当下版本的KTransformer相比于旧版本多出来一个多用户的特性一个本地模型加载起来之后支持多路的会话。那么同时也会导致显存空间有更多的占用用户越多每个会话的缓存占用也会越大。例如下方案例就是多用户的一个使用场景在释放缓存之前其显存占用来到了29564MiB。还是用一样的指令释放缓存之后显存占用变回了27064MiB。$ nvidia-smiThu Jul 2 16:06:02 2026±----------------------------------------------------------------------------------------| NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |±----------------------------------------±-----------------------±---------------------| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. || | | MIG M. |||| 0 NVIDIA GeForce RTX 5090 D Off | 00000000:01:00.0 Off | N/A || 0% 38C P1 155W / 600W | 29588MiB / 32607MiB | 54% Default || | | N/A |±----------------------------------------±-----------------------±---------------------±----------------------------------------------------------------------------------------| Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage |||| 0 N/A N/A 1948 G /usr/lib/xorg/Xorg 4MiB || 0 N/A N/A 3414955 C sglang::scheduler 29564MiB |±----------------------------------------------------------------------------------------$ curl -X POST http://localhost:30000/flush_cacheCache flushed.Please check backend logs for more details. (When there are running or waiting requests, the operation will not be performed.)$ nvidia-smiThu Jul 2 16:06:53 2026±----------------------------------------------------------------------------------------| NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |±----------------------------------------±-----------------------±---------------------| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. || | | MIG M. |||| 0 NVIDIA GeForce RTX 5090 D Off | 00000000:01:00.0 Off | N/A || 0% 37C P1 59W / 600W | 27088MiB / 32607MiB | 0% Default || | | N/A |±----------------------------------------±-----------------------±---------------------±----------------------------------------------------------------------------------------| Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage |||| 0 N/A N/A 1948 G /usr/lib/xorg/Xorg 4MiB || 0 N/A N/A 3414955 C sglang::scheduler 27064MiB |±----------------------------------------------------------------------------------------总结概要本文主要介绍一个在sglang启用本地大模型进行对话之后远程清除对话缓存的一个指令。可以在闲时或者有紧急任务时释放出一部分的显存出来作其他用途。