Skip to content

Commit 60ab884

Browse files
committed
Update 2025-12-21 17:57:26
1 parent c3cca9b commit 60ab884

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+5529
-6289
lines changed

_sources/advanced_features/lora.ipynb

Lines changed: 197 additions & 210 deletions
Large diffs are not rendered by default.

_sources/advanced_features/router.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ SGLang Model Gateway is a high-performance model-routing gateway for large-scale
4646

4747
### Control Plane
4848
- **Worker Manager** discovers capabilities (`/get_server_info`, `/get_model_info`), tracks load, and registers/removes workers in the shared registry.
49-
- **Job Queue** serializes add/remove requests and exposes status (`/workers/{url}`) so clients can track onboarding progress.
49+
- **Job Queue** serializes add/remove requests and exposes status (`/workers/{worker_id}`) so clients can track onboarding progress.
5050
- **Load Monitor** feeds cache-aware and power-of-two policies with live worker load statistics.
5151
- **Health Checker** continuously probes workers and updates readiness, circuit breaker state, and router metrics.
5252

@@ -171,11 +171,13 @@ curl -X POST http://localhost:30000/workers \
171171
# Inspect registry
172172
curl http://localhost:30000/workers
173173

174-
# Remove a worker
175-
curl -X DELETE http://localhost:30000/workers/grpc%3A%2F%2F0.0.0.0%3A31000
174+
# Remove a worker (RESTful: delete by UUID)
175+
# Tip: POST /workers returns a JSON body containing worker_id and a Location header.
176+
WORKER_ID="$(curl -s http://localhost:30000/workers | jq -r '.workers[0].id')"
177+
curl -X DELETE "http://localhost:30000/workers/${WORKER_ID}"
176178
```
177179

178-
Legacy endpoints (`/add_worker`, `/remove_worker`, `/list_workers`) remain available but will be deprecated. `/workers/{url}` returns both registry data and queued job status. The worker url in the removal request should be escaped.
180+
Legacy endpoints (`/add_worker`, `/remove_worker`, `/list_workers`) remain available but will be deprecated. `/workers/{worker_id}` returns both registry data and queued job status.
179181

180182
---
181183

@@ -337,7 +339,7 @@ Use CLI flags to select parsers:
337339
| `GET`/`DELETE` | `/v1/conversations/{id}/items/{item_id}` | Inspect/delete conversation item. |
338340
| `GET` | `/workers` | List registered workers with health/load. |
339341
| `POST` | `/workers` | Queue worker registration. |
340-
| `DELETE` | `/workers/{url}` | Queue worker removal. |
342+
| `GET`/`PUT`/`DELETE` | `/workers/{worker_id}` | Get/update/remove a worker by UUID. |
341343
| `POST` | `/flush_cache` | Flush worker caches (HTTP workers). |
342344
| `GET` | `/get_loads` | Retrieve worker load snapshot. |
343345
| `GET` | `/liveness` / `/readiness` / `/health` | Health probes. |

_sources/advanced_features/separate_reasoning.ipynb

Lines changed: 106 additions & 106 deletions
Large diffs are not rendered by default.

_sources/advanced_features/speculative_decoding.ipynb

Lines changed: 276 additions & 540 deletions
Large diffs are not rendered by default.

_sources/advanced_features/structured_outputs.ipynb

Lines changed: 156 additions & 178 deletions
Large diffs are not rendered by default.

_sources/advanced_features/structured_outputs_for_reasoning_models.ipynb

Lines changed: 162 additions & 151 deletions
Large diffs are not rendered by default.

_sources/advanced_features/tool_parser.ipynb

Lines changed: 185 additions & 210 deletions
Large diffs are not rendered by default.

_sources/advanced_features/vlm_query.ipynb

Lines changed: 215 additions & 222 deletions
Large diffs are not rendered by default.

_sources/basic_usage/native_api.ipynb

Lines changed: 266 additions & 248 deletions
Large diffs are not rendered by default.

_sources/basic_usage/offline_engine_api.ipynb

Lines changed: 474 additions & 477 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)