Running a 120-billion-parameter reasoning model across two GPUs is no longer an exotic exercise — in 2026, with NVIDIA’s FP8 quantization and the Hopper architecture’s native FP8 tensor cores, it is an afternoon project that delivers genuinely production-grade speed. This article walks through the complete, end-to-end deployment of Nemotron-3-Super-120B-A12B on a server with 2× NVIDIA H200 (143 GB each), served by vLLM behind a standard OpenAI-compatible API, with a 1-million-token context window, thinking/reasoning mode, tool calling, multi-turn conversations, and API-key authentication.

There is already an excellent guide for running this model on a single Blackwell RTX PRO 6000 using the NVFP4 (4-bit floating point) checkpoint. This article is the companion piece for the Hopper / H200 case — and as you’ll see, the model variant and several flags must change, because Hopper and Blackwell have different native low-precision capabilities. They are a different hardware, so the installation process turned out to be quite different.

The Target Hardware

The Critical Decision: FP8, Not NVFP4

This is the single most important thing to get right, and it is where a blind copy of the Blackwell guide will bite you.

Nemotron-3-Super-120B-A12B ships in three quantization variants on Hugging Face:

• NVFP4 — 4-bit floating point weights (~80 GB). Requires Blackwell architecture (B200/B300, RTX PRO 6000 Blackwell) with native FP4 tensor cores. On Hopper, FP4 has no native tensor-core support — it would be emulated and slow.
• FP8 — 8-bit floating point weights (~128 GB). This is the native sweet spot for Hopper (H100/H200). The H200’s FP8 tensor cores run this at full speed.
• BF16 — 16-bit, ~240 GB. Overkill and a waste of VRAM unless you need the absolute maximum accuracy.

For 2× H200, the correct choice is nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8. At ~128 GB it fits comfortably across both GPUs with tensor parallelism, leaving roughly 160 GB of headroom for a large FP8 KV cache — which is exactly what lets you run the full 1M-token context.

Rule of thumb: Blackwell (sm_120+) ? NVFP4. Hopper (sm_90) ? FP8. Do not mix these up — the performance difference is dramatic.

The Model: Nemotron-3-Super-120B-A12B

A quick refresher on why this model is special and how it achieves its speed:

• 120 billion total parameters, only 12 billion active per token — a hybrid Latent Mixture-of-Experts design (22 experts routed out of 512 per layer). This is why it is fast despite its size.
• Mamba-2 hybrid layers interleaved with MoE and attention layers. The Mamba layers have O(1) recurrent state (no growing KV cache), which is precisely how the model supports a 1,048,576-token (1M) context.
• Configurable reasoning/thinking mode via the chat template’s enable_thinking flag — the model produces a reasoning trace, then a final answer.
• Native tool calling (function calling) and multi-turn conversation support.
• Multi-Token Prediction (MTP) layers for faster speculative generation.

On Hugging Face it lives at nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8. It is public and ungated — no token or license acceptance required to download.

Step 1 — Connect to the Server

ssh root@YOUR-SERVER-IP

Verify the GPUs are visible and idle:

nvidia-smi

You should see two H200s with ~143 GB each and 0 MiB used.

Step 2 — Install Docker and GPU Passthrough

Unlike the Blackwell guide (where Docker was pre-installed), our H200 server had no Docker at all — only the nvidia-container-toolkit package. Install Docker first:

apt-get update
apt-get install -y docker.io
systemctl enable docker
systemctl start docker

Then configure the NVIDIA container runtime so Docker can pass GPUs through to containers:

nvidia-ctk runtime configure --runtime=docker
systemctl restart docker

Verify both GPUs are visible inside a container:

docker run --rm --runtime=nvidia --gpus all ubuntu:22.04 \
  nvidia-smi --query-gpu=name,memory.total --format=csv,noheader

You should see:

NVIDIA H200, 143771 MiB
NVIDIA H200, 143771 MiB

Step 3 — Create a Python Environment

We need a small venv on the host for downloading the model. On Ubuntu 24.04, the python3-venv package must be installed explicitly:

apt-get install -y python3.12-venv

python3 -m venv /opt/nemotron-venv
/opt/nemotron-venv/bin/pip install --upgrade pip wheel setuptools
/opt/nemotron-venv/bin/pip install "huggingface_hub[cli]" hf_transfer

Step 4 — Download the Model (128 GB)

mkdir -p /opt/models

HF_HUB_CACHE=/opt/models /opt/nemotron-venv/bin/python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download(
    "nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8",
    cache_dir="/opt/models",
    max_workers=16,
)
PY

This pulls 26 safetensors shards (~128 GB) plus the chat template, custom modeling code (modeling_nemotron_h.py), and a reasoning parser (super_v3_reasoning_parser.py). With 16 parallel workers on a fast connection, expect roughly 10–15 minutes.

Note the snapshot path — you’ll need it later:

find /opt/models -type d -path "*Nemotron-3-Super*FP8*/snapshots/*"

Step 5 — Generate an API Key

Create a random API key that clients must present as a Bearer token:

API_KEY="nm-$(openssl rand -hex 24)"
echo "$API_KEY"

mkdir -p /opt/nemotron-serve
echo "API_KEY=$API_KEY" > /opt/nemotron-serve/.api_key
chmod 600 /opt/nemotron-serve/.api_key

Keep this key secret — every request to the server must carry it.

Step 6 — Pull the vLLM Image

The model’s README pins vLLM 0.20.0. Use the official Docker image — it bundles a matched CUDA 13.x toolchain so all the FP8 kernels compile and load cleanly. Do not be tempted by pip install vllm; the pip path hits JIT compiler/header mismatches on Hopper that the Docker image avoids entirely.

docker pull vllm/vllm-openai:v0.20.0

Step 7 — The Launch Script

This is the heart of the deployment. Create the launch script:

cat > /opt/nemotron-serve/launch_docker.sh <<'EOF'
#!/bin/bash
set -euo pipefail

API_KEY="$(grep -oP 'nm-[0-9a-f]+' /opt/nemotron-serve/.api_key)"
# Path as it appears INSIDE the container (the HF_HOME mount point)
MODEL_PATH="/root/.cache/huggingface/models--nvidia--NVIDIA-Nemotron-3-Super-120B-A12B-FP8/snapshots/SNAPSHOT_HASH"

echo "Using model path: $MODEL_PATH"
echo "Using API key:    $API_KEY"

docker rm -f nemotron-serve 2>/dev/null || true

exec docker run --name nemotron-serve \
  --runtime nvidia --gpus all \
  --network host \
  --shm-size 16g \
  --restart unless-stopped \
  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
  -e HF_HUB_OFFLINE=1 \
  -e HF_HOME=/root/.cache/huggingface \
  -e HF_MODULES_CACHE=/root/.cache/hf_modules \
  -v /opt/models:/root/.cache/huggingface:ro \
  -v /opt/hf-cache/modules:/root/.cache/hf_modules:rw \
  vllm/vllm-openai:v0.20.0 \
    --model "$MODEL_PATH" \
    --served-model-name nvidia/nemotron-3-super \
    --host 0.0.0.0 \
    --port 20000 \
    --api-key "$API_KEY" \
    --async-scheduling \
    --dtype auto \
    --kv-cache-dtype fp8 \
    --tensor-parallel-size 2 \
    --enable-expert-parallel \
    --trust-remote-code \
    --gpu-memory-utilization 0.90 \
    --enable-chunked-prefill \
    --max-model-len 1048576 \
    --max-num-seqs 32 \
    --mamba-ssm-cache-dtype float32 \
    --reasoning-parser nemotron_v3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder
EOF

chmod +x /opt/nemotron-serve/launch_docker.sh
mkdir -p /opt/hf-cache/modules

Replace SNAPSHOT_HASH with the actual hash from Step 4.

What each key flag does

• --tensor-parallel-size 2 — splits the model across both H200s. Essential for a 128 GB checkpoint.
• --enable-expert-parallel — distributes the MoE experts across GPUs (NVIDIA’s recommended setting for multi-GPU Hopper).
• --max-model-len 1048576 + VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 — unlocks the full 1M-token context. The model’s max_position_embeddings is 262144, but the Mamba/attention hybrid legitimately extends to 1M; the env var lets vLLM exceed the conservative default.
• --kv-cache-dtype fp8 — FP8 KV cache, halving attention memory and fitting longer contexts in VRAM.
• --mamba-ssm-cache-dtype float32 — the Mamba-2 recurrent state in fp32 (required for the hybrid layers).
• --reasoning-parser nemotron_v3 — loads the parser that splits reasoning traces into the reasoning field of the response.
• --enable-auto-tool-choice --tool-call-parser qwen3_coder — enables function-calling in the OpenAI-compatible API.
• --max-num-seqs 32 — allows up to 32 concurrent sequences for batching.
• --enable-chunked-prefill — processes long prompts in chunks for smoother scheduling.

Launch it

nohup /opt/nemotron-serve/launch_docker.sh > /opt/nemotron-serve/docker.log 2>&1 &

Startup takes about 2 minutes: ~30 s to load 128 GB of weights across two GPUs, then ~90 s for torch.compile and CUDA graph capture. The server is ready when the log shows:

INFO ... Application startup complete.

Follow along with:

tail -f /opt/nemotron-serve/docker.log

The Two Pitfalls (and Their Fixes)

I hit both of these during bring-up. The launch script above already avoids them, but they are worth understanding.

Pitfall #1 — --swap-space not recognized

Symptom: vllm: error: unrecognized arguments: --swap-space 0.

Cause: The model’s README (written for a newer vLLM) includes --swap-space 0, but vLLM 0.20.0’s argument parser does not accept this flag.

Fix: Simply remove --swap-space 0 from the launch script.

Pitfall #2 — Model path resolution inside the container

Symptom: HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/models/...' or LocalEntryNotFoundError: Cannot find an appropriate cached snapshot folder.

Cause: There are two traps here. First, if you pass the host path (/opt/models/...), it does not exist inside the container (the weights are mounted at /root/.cache/huggingface/...). Second, if you pass the repo ID (nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8) with HF_HUB_OFFLINE=1, vLLM’s get_model_path() tries to look up a specific revision and fails because offline mode prevents the lookup.

Fix: Pass the path as it appears inside the container — i.e., under the HF_HOME mount point (/root/.cache/huggingface/models--nvidia--.../snapshots/HASH), not the host path. The launch script above does exactly this.

Step 8 — Verify Everything Works

Load the API key into a variable:

API_KEY=$(grep -oP 'nm-[0-9a-f]+' /opt/nemotron-serve/.api_key)

List models — confirms the 1M context

curl -s http://localhost:20000/v1/models \
  -H "Authorization: Bearer $API_KEY" | python3 -m json.tool

The response includes "max_model_len": 1048576 — your 1M context is active.

Thinking / reasoning mode

Pass "reasoning": {"effort": "auto"} in the request body to activate the reasoning parser. The model’s thinking trace appears in the reasoning field, and the final answer in content:

curl -s http://localhost:20000/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/nemotron-3-super",
    "messages": [{"role":"user","content":"A bat and ball cost $1.10 total. The bat costs $1.00 more than the ball. How much is the ball?"}],
    "max_tokens": 500,
    "temperature": 0.6,
    "reasoning": {"effort": "auto"}
  }'

You can also toggle thinking per-request via the chat template:

"chat_template_kwargs": {"enable_thinking": true}

Tool calling

curl -s http://localhost:20000/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/nemotron-3-super",
    "messages": [{"role":"user","content":"What is the weather in Tokyo?"}],
    "max_tokens": 400,
    "tools": [{"type":"function","function":{
      "name":"get_weather",
      "description":"Get current weather for a city",
      "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}
    }}]
  }'

The model returns "finish_reason": "tool_calls" with a structured call to get_weather({"city": "Tokyo"}).

Multi-turn conversations

Simply pass the full message history — the model retains context across turns:

curl -s http://localhost:20000/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/nemotron-3-super",
    "messages": [
      {"role":"user","content":"My name is Alice. Remember it."},
      {"role":"assistant","content":"Nice to meet you, Alice!"},
      {"role":"user","content":"What is my name?"}
    ],
    "max_tokens": 200
  }'

API key enforcement

# Wrong key ? HTTP 401
curl -o /dev/null -w "%{http_code}" http://localhost:20000/v1/models \
  -H "Authorization: Bearer wrong-key"

# No key ? HTTP 401
curl -o /dev/null -w "%{http_code}" http://localhost:20000/v1/models

# Correct key ? HTTP 200
curl -o /dev/null -w "%{http_code}" http://localhost:20000/v1/models \
  -H "Authorization: Bearer $API_KEY"

Benchmarks

Measured against the live server with single requests (unless noted), using the OpenAI-compatible API:

~160 tokens/second steady-state decode from a 120B-parameter model — and remarkably flat across output lengths (144 ? 163), confirming the FP8 KV cache and Mamba SSM state are healthy. This is roughly 1.7× faster than the ~94 tok/s achieved on a single Blackwell RTX PRO 6000, thanks to the H200’s higher memory bandwidth plus tensor parallelism across two GPUs.

Under 8-way concurrency, aggregate output throughput reaches 281 tok/s, demonstrating that the batching engine (--max-num-seqs 32) scales well. For context: a full 800-token coding response generates in under 5 seconds.

Using the Server

The server speaks the standard OpenAI API. Point any OpenAI-compatible client at it:

from openai import OpenAI

client = OpenAI(
    base_url="http://YOUR-SERVER-IP:20000/v1",
    api_key="nm-YOUR-API-KEY",
)

resp = client.chat.completions.create(
    model="nvidia/nemotron-3-super",
    messages=[{"role": "user", "content": "Design a REST API for a todo app."}],
    max_tokens=2000,
    temperature=0.6,
    extra_body={"reasoning": {"effort": "auto"}},
)
print(resp.choices[0].message.reasoning)  # thinking trace
print(resp.choices[0].message.content)     # final answer

Tool calling works the same way — pass tools=[...] and the model emits structured function calls. Multi-turn conversations work as expected (pass the full message history).

Operations Cheatsheet

# check container status
docker ps --filter name=nemotron-serve

# tail logs
docker logs -f nemotron-serve

# restart (picks up the launch script)
docker restart nemotron-serve

# GPU usage
nvidia-smi

# rotate the API key
API_KEY="nm-$(openssl rand -hex 24)"
echo "API_KEY=$API_KEY" > /opt/nemotron-serve/.api_key
docker restart nemotron-serve

GPU Memory Breakdown

With the 128 GB FP8 checkpoint split across two H200s at 90% memory utilization:

GPU 0: 129,757 MiB / 143,771 MiB used
GPU 1: 129,757 MiB / 143,771 MiB used

That is ~65 GB of weights per GPU plus a large FP8 KV cache and Mamba SSM state — leaving healthy headroom for the 1M-token context window.

Conclusion

The era of running a 120-billion-parameter reasoning model on affordable datacenter GPUs is firmly here. With two H200s and vLLM, you get a 1M-context reasoning model at ~160 tok/s, with native tool calling and thinking mode, behind a standard OpenAI-compatible API secured with an API key.

The key insight — and the reason this article exists alongside the Blackwell guide — is matching the quantization variant to your architecture: NVFP4 for Blackwell, FP8 for Hopper. Get that one decision right, and the rest of the deployment is straightforward infrastructure work: Docker GPU passthrough, the correct model path inside the container, and the right cache/memory flags. Once those are solved, the server is stable, fast, and ready for production traffic.

Model: NVIDIA-Nemotron-3-Super-120B-A12B-FP8, governed by the NVIDIA Nemotron Open Model License. Serving: vLLM 0.20.0 on 2× NVIDIA H200. All benchmarks are real, measured live on the described hardware.

Introduction

Running a 120-billion-parameter reasoning model on a single workstation GPU used to be impossible. In 2026, thanks to NVIDIA’s NVFP4 quantization and the Blackwell architecture’s native FP4 tensor cores, it is now a practical afternoon project. This article walks through the complete, end-to-end deployment of nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 on a single RTX PRO 6000 Blackwell (96 GB) — every command, every pitfall, and the real benchmarks at the end.

We will cover: connecting to the server, choosing the right model artifact, installing the serving stack, fixing the four problems you will hit on Blackwell, and measuring throughput.

The Target Hardware

The Model: Nemotron-3-Super-120B-A12B

This is the star of the show. Nemotron-3-Super-120B-A12B is a hybrid Latent Mixture-of-Experts model:

• 120 billion total parameters, but only 12 billion active per token (22 experts routed out of 512 per layer). This is why it runs fast despite its size.
• Architecture: interleaved Mamba-2 state-space layers, MoE layers, and select attention layers. The Mamba layers have O(1) recurrent state — they do not grow a KV cache — which is exactly how the model supports a 1,048,576-token (1M) context on a single GPU.
• NVFP4 quantization: experts are stored in 4-bit floating point (group size 16); attention and shared layers remain FP8. This shrinks the weights to ~80 GB, fitting comfortably in 96 GB VRAM with room for a large KV cache.
• Capabilities: configurable reasoning/thinking mode (via the chat template’s enable_thinking flag), native tool calling, and a 1M-token context window.

On Hugging Face it lives at nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4. It is public and ungated — no token or license-acceptance gate is required to download it.

Important naming note: There is no model literally called “Nemotron 3 Super NVFP4.” The official repository id is nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4. An older, unrelated family member (nvidia/Llama-3.3-Nemotron-Super-49B-v1) is a DeciLM/NAS model — make sure you grab the right one.

Step 1 — Create a Python Environment

We do not need a full CUDA toolkit on the host; vLLM ships with everything inside its Docker image. We do need a small venv on the host for downloading the model and running benchmark scripts.

python3 -m venv /opt/nemotron-venv
/opt/nemotron-venv/bin/pip install --upgrade pip wheel setuptools
/opt/nemotron-venv/bin/pip install "huggingface_hub[cli]" hf_transfer

Step 2 — Download the Model (80 GB)

export HF_HUB_CACHE=/opt/models
mkdir -p /opt/models

/opt/nemotron-venv/bin/python - <<'PY'
from huggingface_hub import snapshot_download
snapshot_download(
    "nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4",
    cache_dir="/opt/models",
    max_workers=16,
)
PY

This pulls 17 safetensors shards (~80 GB) plus the chat template and a custom reasoning parser (super_v3_reasoning_parser.py). Copy the parser somewhere stable — we will mount it into the container:

MODEL_PATH=$(find /opt/models -type d -path "*Nemotron-3-Super*/snapshots/*" | head -1)
cp "$MODEL_PATH/super_v3_reasoning_parser.py" /opt/super_v3_reasoning_parser.py

Step 3 — Generate an API Key

API_KEY="nm-$(openssl rand -hex 24)"
echo "$API_KEY"
mkdir -p /opt/nemotron-serve
echo "API_KEY=$API_KEY" > /opt/nemotron-serve/.api_key
chmod 600 /opt/nemotron-serve/.api_key

Keep this key secret — every request to the server must carry it as a Bearer token.

Step 4 — Fix Docker GPU Passthrough (Pitfall #1)

The host had Docker installed but no nvidia-container-toolkit, so --gpus all failed with could not select device driver "" with capabilities: [[gpu]]. The NVIDIA apt repository was already configured, so the fix was a one-liner:

apt-get install -y nvidia-container-toolkit
nvidia-ctk runtime configure --runtime=docker
systemctl restart docker

Verify the GPU is visible inside a container:

docker run --rm --runtime=nvidia --gpus all ubuntu:22.04 \
  nvidia-smi --query-gpu=name,memory.total --format=csv,noheader

# NVIDIA RTX PRO 6000 Blackwell Workstation Edition, 97887 MiB

Step 5 — Pull the vLLM Image

The model README pins vLLM 0.20.0 with a purpose-built Blackwell image. Do not be tempted by a newer pip install vllm — the pip path hit two more problems (see the pitfalls below). Use the official image, which bundles a matched CUDA 13.x toolkit:

docker pull vllm/vllm-openai:v0.20.0

Step 6 — Launch the Server

Create the launch script. The flags come straight from the model’s official README, with two critical additions discovered during bring-up (explained in the pitfalls section).

cat > /opt/nemotron-serve/launch_docker.sh <<'EOF'
#!/bin/bash
set -euo pipefail

API_KEY="$(grep -oP 'nm-[0-9a-f]+' /opt/nemotron-serve/.api_key)"
MODEL_PATH="/root/.cache/huggingface/models--nvidia--NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4/snapshots/4f0cf9daaeb7a4d5e23f80a00e7ed15f0e03caf6"

docker rm -f nemotron-serve 2>/dev/null || true

exec docker run --name nemotron-serve \
  --runtime nvidia --gpus all \
  --network host \
  --shm-size 16g \
  --restart unless-stopped \
  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
  -e HF_HUB_OFFLINE=1 \
  -e HF_HOME=/root/.cache/huggingface \
  -e HF_MODULES_CACHE=/root/.cache/hf_modules \
  -v /opt/models:/root/.cache/huggingface:ro \
  -v /opt/hf-cache/modules:/root/.cache/hf_modules:rw \
  -v /opt/super_v3_reasoning_parser.py:/app/super_v3_reasoning_parser.py:ro \
  vllm/vllm-openai:v0.20.0 \
    --model "$MODEL_PATH" \
    --served-model-name nvidia/nemotron-3-super \
    --host 0.0.0.0 \
    --port 20000 \
    --api-key "$API_KEY" \
    --async-scheduling \
    --dtype auto \
    --kv-cache-dtype fp8 \
    --tensor-parallel-size 1 \
    --trust-remote-code \
    --gpu-memory-utilization 0.90 \
    --enable-chunked-prefill \
    --max-model-len 1048576 \
    --max-num-seqs 32 \
    --mamba-ssm-cache-dtype float16 \
    --reasoning-parser-plugin /app/super_v3_reasoning_parser.py \
    --reasoning-parser super_v3 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder
EOF
chmod +x /opt/nemotron-serve/launch_docker.sh

# writable HF modules dir for trust-remote-code
mkdir -p /opt/hf-cache/modules

# launch
nohup /opt/nemotron-serve/launch_docker.sh > /opt/nemotron-serve/docker.log 2>&1 &

What each key flag does

• --max-model-len 1048576 + VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 — unlocks the full 1M-token context (the model’s max_position_embeddings is 262144, but the Mamba/attention hybrid legitimately extends to 1M; the env var lets vLLM exceed the conservative default).
• --kv-cache-dtype fp8 — FP8 KV cache, halving attention memory and fitting longer contexts in VRAM.
• --mamba-ssm-cache-dtype float16 — the Mamba-2 recurrent state in fp16 (required for the hybrid layers).
• --reasoning-parser super_v3 + --reasoning-parser-plugin — loads the custom parser that splits reasoning traces into the reasoning_content field.
• --enable-auto-tool-choice --tool-call-parser qwen3_coder — enables function-calling in the OpenAI-compatible API.
• --max-num-seqs 32 — see Pitfall #4.

Startup takes about 90 seconds: ~25 s to load 80 GB of weights, then ~60 s for torch.compile and CUDA graph capture. The server is ready when the log shows:

INFO ... Application startup complete.

Step 7 — Verify It Works

API_KEY=$(grep -oP 'nm-[0-9a-f]+' /opt/nemotron-serve/.api_key)

# list models — confirms the 1M context
curl -s http://localhost:20000/v1/models \
  -H "Authorization: Bearer $API_KEY" | python3 -m json.tool

# chat
curl -s http://localhost:20000/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/nemotron-3-super",
    "messages": [{"role":"user","content":"What is 17 * 23?"}],
    "max_tokens": 500,
    "temperature": 1.0,
    "top_p": 0.95
  }'

The Four Pitfalls (and Their Fixes)

I hit all four of these. If you follow the script above you will avoid them — but they are worth understanding.

Pitfall #1 — Docker GPU passthrough missing

Symptom: could not select device driver "" with capabilities: [[gpu]].

Cause: nvidia-container-toolkit not installed.

Fix: apt-get install -y nvidia-container-toolkit && nvidia-ctk runtime configure --runtime=docker && systemctl restart docker.

Pitfall #2 — Read-only filesystem for trust-remote-code

Symptom: OSError: [Errno 30] Read-only file system: '/root/.cache/huggingface/modules'.

Cause: The model uses trust-remote-code, which writes its custom nemotron_h Python module into a cache directory. I had mounted the HF cache as :ro.

Fix: Keep the weights mount read-only, but add a separate writable volume for HF_MODULES_CACHE: -e HF_MODULES_CACHE=/root/.cache/hf_modules -v /opt/hf-cache/modules:/root/.cache/hf_modules:rw.

Pitfall #3 — max_num_seqs exceeds available Mamba blocks

Symptom: ValueError: max_num_seqs (1024) exceeds available Mamba cache blocks (685). Each decode sequence requires one Mamba cache block, so CUDA graph capture cannot proceed.

Cause: The hybrid Mamba-2 layers allocate one fixed-size state block per concurrent sequence. vLLM’s default max_num_seqs of 1024 was too high for the VRAM left after the 69 GB of weights.

Fix: Add --max-num-seqs 32 (the model README suggests 4 for DGX Spark, but 32 fits comfortably in the 685 available blocks and gives better batch throughput).

Pitfall #4 — Why not just pip install vllm?

I tried a native pip install vllm (which pulled v0.26.0 with torch 2.11 / CUDA 13). It hit two Blackwell-specific JIT problems that the Docker image avoids entirely:

1. Missing nvcc at runtime: vLLM’s CUTLASS NVFP4 kernels need to JIT-compile. The host had no CUDA toolkit, only the driver. Fixable by pointing CUDA_HOME at the pip-installed nvidia/cu13 package — but that led to problem #2.
2. CUDA compiler / header mismatch: FlashInfer’s sm_120 FP4/NVFP4 GEMM JIT (gen_gemm_sm120) failed with #error "CUDA compiler and CUDA toolkit headers are incompatible". The pip nvidia-cuda-nvcc (13.3) disagreed with the bundled CCCL/libcudacxx headers.

The purpose-built vllm/vllm-openai:v0.20.0 image contains a coherently-matched CUDA 13.x toolchain, so all the NVFP4 kernels compile and load cleanly on the first try. Use the Docker image.

Benchmarks

Measured against the live server with a single concurrent request:

~94 tokens/second steady-state decode from a 120B-parameter model on a single workstation GPU. That is remarkable, and it comes from three things working together: NVFP4 weights (4× compression with minimal accuracy loss), Blackwell’s native FP4 tensor cores, and the 12B-active Mamba/MoE hybrid architecture that keeps the per-token FLOPs low. Throughput is flat across output lengths, confirming the KV and Mamba caches are healthy.

Using the Server

The server speaks the standard OpenAI API. Point any OpenAI-compatible client at it:

from openai import OpenAI

client = OpenAI(
    base_url="http://175.28.230.22:20000/v1",
    api_key="nm-<your-key>",
)

resp = client.chat.completions.create(
    model="nvidia/nemotron-3-super",
    messages=[{"role": "user", "content": "Design a REST API for a todo app."}],
    max_tokens=2000,
    temperature=1.0,
    top_p=0.95,
)
print(resp.choices[0].message.content)

Tool calling works the same way — pass tools=[...] and the model emits structured function calls. Multi-turn conversations work as expected (pass the full message history).

Thinking mode

The model’s chat template defaults to enable_thinking=True. When thinking is on, the model first produces a reasoning trace inside <think>...</think> tags, then the final answer. The super_v3 reasoning parser splits these into the response’s reasoning_content and content fields. To toggle it per-request:

resp = client.chat.completions.create(
    model="nvidia/nemotron-3-super",
    messages=[{"role": "user", "content": "Prove the Pythagorean theorem."}],
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
    ...
)

Operations Cheatsheet

# check container status
docker ps --filter name=nemotron-serve

# tail logs
docker logs -f nemotron-serve

# restart (picks up the launch script)
docker restart nemotron-serve

# GPU usage
nvidia-smi

# rotate the API key
API_KEY="nm-$(openssl rand -hex 24)"
echo "API_KEY=$API_KEY" > /opt/nemotron-serve/.api_key
docker restart nemotron-serve

Conclusion

The era of “a 120-billion-parameter model on one GPU” has arrived. The Nemotron-3-Super-120B-A12B design — LatentMoE with Mamba-2 hybrid layers, trained natively in NVFP4 — is purpose-built for exactly this hardware moment. With an RTX PRO 6000 Blackwell you get a 1M-context reasoning model at 94 tok/s, behind a standard OpenAI-compatible API, for the cost of the electricity to run one card.

The tricky parts are all infrastructure: getting Docker GPU passthrough working, matching the CUDA toolchain for the NVFP4 JIT kernels, and sizing the Mamba/sequence caches correctly. Once those are solved (and the launch script above solves them), the deployment is stable and fast.

Model: NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4, governed by the NVIDIA Nemotron Open Model License. Serving: vLLM 0.20.0. All benchmarks are real, measured live on the described hardware.

Introduction

Running a model too large for your GPU is a problem most people solve by giving up. AirLLM takes the opposite approach: it streams the model’s layers (or expert sublayers, for MoE) from disk into the GPU one at a time, so even a 2.8-trillion-parameter model like Kimi K3 can run on a single 16 GB card. The catch is that AirLLM is a Python library, not a server — there is no HTTP endpoint, no API key, no streaming, no tool-call parsing. You have to write that yourself.

That’s what AirWrapperLLM is: an OpenAI-compatible FastAPI server that wraps AirLLM and gives it the surface every modern agent expects. This guide walks through the complete install + first-run, including the hardware you’ll actually need and the speed you’ll actually get — both of which AirLLM’s marketing sometimes glosses over.

What AirWrapperLLM gives you

After one pip install and one server start, you have:

• POST /v1/chat/completions — full OpenAI schema, both streaming and non-streaming responses.
• GET /v1/models — model listing.
• GET /health — liveness check, reports server version + uptime.
• Bearer-token API-key auth — the key is auto-generated on first start and persisted with chmod 600.
• Multi-turn conversations — pass message history, the tokenizer’s chat template renders it.
• Thinking / reasoning mode — the model’s <think> trace is split out into message.reasoning.
• Tool / function calling — pass tools=[...] in the standard OpenAI format; AirWrapperLLM parses the model’s XTML <tools> channel and returns structured tool_calls.
• Thinking-effort control — reasoning={"effort": "low" | "medium" | "high" | "max"} in the request body, or lower-level chat_template_kwargs.

The XTML parser is a standalone, dependency-free module (airwrapper_xtml.py) with 7 unit tests. If you want to build your own service around AirLLM, you can lift the parser straight out.

Use it as a backend for your agent

AirWrapperLLM speaks the standard OpenAI HTTP API, so any agent that supports an OpenAI-compatible base_url works against it without code changes. Just point the agent at http://host:port/v1 with the Bearer token:

What works through the agent: multi-turn, tool calling, reasoning/thinking. What to watch out for: latency is high (first-token takes tens of seconds to minutes; tokens then trickle out as layers stream from disk), there is no live token-by-token streaming (the full sequence arrives in one batch), and the server is single-tenant — one generation at a time, period.

Hardware requirements (be honest with yourself)

AirLLM’s tagline is “run any model on any GPU.” That’s true in the sense that the model will load and produce output. It is not true in the sense that it will be fast. Before you commit, measure against this table:

Three hardware rules that aren’t optional:

1. Disk speed matters as much as GPU speed. The model is read off NVMe for every token. A SATA SSD will make 0.5 tok/s into 0.05 tok/s. Get NVMe with at least 1 GB/s sequential read.
2. RAM matters too. AirLLM can pin host memory for layer prefetching; 16 GB is a minimum, 64 GB is comfortable for the 100B+ class.
3. You need 2–3× the model size in free disk during the initial layer-split transform (the original Hugging Face snapshot plus the per-shard directory).

If you need vLLM-class speed (dozens to hundreds of tok/s), AirLLM is the wrong tool. AirLLM is the right tool when the model is too big to fit any other way and you’re willing to wait.

Install

Clone and create a virtualenv:

git clone https://github.com/sufehmi/AirWrapperLLM.git
cd AirWrapperLLM
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip wheel setuptools

Install PyTorch with the CUDA wheel matching your driver. The README’s recommendation is CUDA 12.8 for Blackwell GPUs (RTX 50-series, cc 10.0/12.0); check vast-capabilities | jq .hardware.gpu.cuda or your equivalent to confirm:

pip install torch --index-url https://download.pytorch.org/whl/cu128

Then everything else:

pip install -r requirements.txt

The pinned requirements include AirLLM 3.x, transformers 4.56.x (some remote-code models — Kimi K3 — refuse to load on 5.x), tiktoken, fla-core, bitsandbytes, compressed-tensors, fastapi, uvicorn, pydantic.

If your model uses flash attention, install it against the CUDA toolkit that matches your torch wheel:

CUDA_HOME=/usr/local/cuda-12.8 pip install flash-attn --no-build-isolation

There are no prebuilt flash-attn wheels for torch 2.11, so this takes a few minutes to compile. On Blackwell (cc 12.0) the compile is straightforward; on older arches you may need a CUDA toolkit match the driver, and on Hopper (cc 9.0) flash-attn 2.8.3 builds cleanly against CUDA 12.x.

Download a model

AirWrapperLLM is engine-agnostic over any AirLLM-compatible checkpoint. The current AirLLM readme highlights Kimi K3; smaller alternatives (Qwen3 32B, Llama 3.1 70B, DeepSeek-V3) work the same way.

python3 -c "
from huggingface_hub import snapshot_download
snapshot_download(
    'nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8',  # or any AirLLM-compatible model
    local_dir='./models/model-name',
    max_workers=16,
)
"

With max_workers=16 and a fast connection, expect roughly 10–15 minutes per 100 GB of weights. The Kimi K3 download alone is around 1.5 TB.

Start the server

python airwrapper.py \
  --model ./models/model-name \
  --host 0.0.0.0 \
  --port 20002 \
  --max-seq-len 32768 \
  --compression 4bit \
  --delete-original

Or use the bundled launch wrapper, which sources the right environment variables first:

./launch_docker.sh

On a successful start you’ll see:

================================================================
AirWrapperLLM — OpenAI-compatible server for AirLLM
================================================================
API key : air-XXXXXXXXXXXXXXXXXXXXXXXXXXXX
Model   : ./models/model-name
Listen  : 0.0.0.0:20002
Compress: 4bit
Delete original after split: True
Max seq : 32768
================================================================
...
[AirWrapperLLM] Model loaded in 1500s.
[AirWrapperLLM] Starting uvicorn on 0.0.0.0:20002
INFO:     Uvicorn running on http://0.0.0.0:20002

The first start is slow — AirLLM splits the checkpoint into per-layer shards on disk (and optionally 4bit/8bit-compresses them). Subsequent starts reuse the cached shards and are fast.

CLI flags and environment variables

Test it

Copy the API key from the server’s stdout, then:

API_KEY=air-XXXXXXXXXXXXXXXXXXXXXXXXXXXX

# Health check (no auth required)
curl http://localhost:20002/health
# {"status":"ready","version":"1.0.0","model":"...","uptime":...}

# Models
curl http://localhost:20002/v1/models \
  -H "Authorization: Bearer $API_KEY"

# Chat with thinking mode
curl http://localhost:20002/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model",
    "messages": [{"role":"user","content":"What is 17 * 23?"}],
    "max_tokens": 200,
    "temperature": 0.6,
    "reasoning": {"effort": "auto"}
  }'

Python with the official OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:20002/v1",
    api_key="air-XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
)

resp = client.chat.completions.create(
    model="your-model",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=100,
    extra_body={"reasoning": {"effort": "auto"}},
)
print(resp.choices[0].message.reasoning)   # thinking trace, if any
print(resp.choices[0].message.content)     # final answer

Tool calling

curl http://localhost:20002/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model",
    "messages": [{"role":"user","content":"What is the weather in Tokyo?"}],
    "tools": [{"type":"function","function":{
      "name":"get_weather",
      "description":"Get the current weather for a city",
      "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}
    }}]
  }'

The response includes structured tool_calls:

{
  "choices": [{
    "finish_reason": "tool_calls",
    "message": {
      "role": "assistant",
      "tool_calls": [{
        "id": "call_abc123...",
        "type": "function",
        "function": {"name": "get_weather", "arguments": "{\"city\": \"Tokyo\"}"}
      }]
    }
  }]
}

Multi-turn

Just pass the full conversation history — the tokenizer’s chat template handles rendering:

resp = client.chat.completions.create(
    model="your-model",
    messages=[
        {"role": "user", "content": "My name is Alice. Remember it."},
        {"role": "assistant", "content": "Nice to meet you, Alice!"},
        {"role": "user", "content": "What is my name?"},
    ],
    max_tokens=50,
)

Run as a managed service

For production deployments you’ll want auto-restart on crash. The launch wrapper is supervisor-friendly:

[program:airwrapper]
environment=PROC_NAME="%(program_name)s"
command=/path/to/AirWrapperLLM/launch_docker.sh
autostart=true
autorestart=unexpected
startsecs=2
stopasgroup=true
killasgroup=true
stopsignal=TERM
stdout_logfile=/dev/stdout
redirect_stderr=true
stdout_logfile_maxbytes=0

Or under systemd. The server is single-process, single-GPU, so a simple supervisor/service is sufficient — no clustering, no shared state.

Operational notes

Rotating the API key: kill the server, regenerate the key file, restart.

echo "API_KEY=air-$(openssl rand -hex 24)" > /path/to/.airwrapper_api_key
chmod 600 /path/to/.airwrapper_api_key
supervisorctl restart airwrapper

Inspecting logs: with the supervisor config above, logs go to /dev/stdout and surface in the supervisor journal. The model’s quantization + loading phase prints tens of thousands of progress-bar lines — pipe through grep -v "Compressing\|Applying quant" if you want clean output.

When generation hangs: AirLLM is single-tenant. If a request times out on the client, the server still holds the generation lock. Wait for the in-flight run to finish (or use supervisorctl restart airwrapper to hard-stop — the server doesn’t expose a “cancel” signal yet).

Why no live token streaming? AirLLM returns the full sequence only after the last token. AirWrapperLLM emits the parsed result as SSE deltas when the run completes, not token-by-token as they generate. The OpenAI stream=true contract is honored, but in practice clients will see all tokens arrive at once. Improving this requires changes to AirLLM itself.

Limitations

• Single request at a time. AirLLM has no batched-inference path. Two requests in flight queue; they don’t run in parallel.
• No live token-by-token streaming. See above.
• No LoRA / adapter support. AirLLM doesn’t expose this.
• XTML parser is Kimi-K3-aware. Other models that use a different structural-token scheme may need a tweaked parser.
• No multi-GPU. AirLLM runs on one device by design; AirWrapperLLM cannot override that without a different engine underneath.

Conclusion

AirWrapperLLM turns AirLLM’s “run any model from disk” engine into a normal OpenAI-compatible HTTP service: API key auth, multi-turn, thinking mode, tool calling, versioned releases. If you have a model that’s too big for your VRAM and you’re willing to wait for tokens, it’s the cleanest end-to-end stack available in 2026.

If you want real speed, use vLLM with a model that fits your VRAM. AirLLM and AirWrapperLLM exist for the case where that’s not an option.

AirWrapperLLM v1.0.0 — MIT licensed. Project: github.com/sufehmi/AirWrapperLLM. AirLLM is Apache 2.0 by Gavin Li.

I’ve been interviewed by many researchers, and everyone pretty much always asked this particular question:

“Why fact-checking is not working?” or “Is it enough to solve the mis/disinformation problem by doing fact-checking?”

By which my answer is always the same: doing fact-checking, alone, will NEVER solve this problem.

There is no silver bullet for this monster.

We need to do so much in order to be able to combat hoax, our umbrella term for the whole spectrum of mis/disinformation, effectively.

We need to educate the public. We need to advocate the governments, so they can develop the right regulations & policies. We need to work together with academics and researchers, so they can pinpoint the right courses of actions. We need to work with journalists, so people will always have trusted sources of information. And of course we need to do hoax busting / fact-checking as well.

It’s a massive scope of work which require massive amount of efforts & resources.

Therefore I’m always so grateful to Mafindo’s volunteers. Due to their sheer numbers and depth & breadth of their skills capacity & capabilities, Mafindo has been able to do all of the above.

Indonesia is so lucky to have them. I hope we will always be able to support their efforts effectively.

“Isometric Exercise” adalah olah raga yang membebani otot, namun tidak bergerak / minim gerakan. Contoh: planking, yoga, pilates, dll. Dan penelitian terbaru menemukan bahwa ini adalah jenis olah raga yang paling sehat.

Kelebihan isometric exercise daripada olahraga lainnya adalah (1) tidak membutuhkan ruang yang luas (2) bisa dilakukan tanpa alat, sehingga (3) jadi bisa dilakukan kapan saja

Beberapa contoh isometric exercises yang mudah dilakukan adalah sbb:

Overhead Hold

Otot yang terdampak : core, triceps, shoulder girdle, upper trapezius

Alat yang diperlukan : beban ringan, seperti barbel 2 kg, atau bahkan kaleng makanan.

1. Angkat tangan ke atas, dan tahan. Pastikan otot core / tubuh terasa turut bereaksi.
2. Pastikan tangan dianggkat lurus ke atas. Karena jika bengkok, maka otot yang terdampak akan beda (hanya biceps dan triceps)
3. Tahan selama 20-30 detik (tapi jangan segan turunkan sebelum itu jika dirasa beban bisa terlepas dari tangan)
4. Istirahat sejenak, lalu ulangi lagi dari poin 1
5. Ulangi sebanyak 2 atau 3 kali

Variasi : lakukan dengan berdiri di satu kaki.

High plank

Alat yang diperlukan : tidak ada

Otot yang terdampak : abdominals, quadriceps, glutes, semua otot tangan, dada, pundak

1. Start dengan posisi seperti push-up, dengan bertumpu di lutut
2. Tegakkan kedua tangan, dan lalu luruskan kaki. Sehingga tubuh Anda jadi seperti di posisi saat naik di push-up. Pastikan tangan sama rata dengan pundak, kaki lurus, dan seluruh otot core terasa aktif.
3. Tahan selama bisa, istirahat sejenak, lalu ulang 2 kali.

Variasi : bertumpu di lengan

Side Plank

Alat yang diperlukan : tidak ada

Otot yang terdampak : obliques (otot samping perut), spinal stabilizers, quadriceps, glutes, serratus anterior, shoulder stabilizers, hip abductors

1. Mulai dengan berbaring menyamping.
2. Bertumpu pada lengan, dan naikkan badan.
3. Tangan yang satu lagi bisa di samping, atau diluruskan ke atas
4. Pastikan postur tubuh lurus dari ujung kaki sampai kepala
5. Tahan selama mungkin, istirahat sejenak, lalu ganti sisi.

Variasi : tangan tumpuan diluruskan.

Dunia medsos jadi memungkinkan siapa saja untuk tampil di panggung. Ini seperti pisau bermata dua – ketika yang muncul adalah yang baik, maka jadi banyak yang bisa mendapatkan manfaatnya.

Namun jika yang muncul adalah yang buruk, atau jahat – maka juga jadi banyak yang bisa dirugikan, atau terzalimi.

Salah satu fenomena yang sudah cukup lama marak adalah para pakar palsu. Mengandalkan penampilan yang memukau, dan kata-kata yang manis – mereka menipu & mengecoh banyak orang.

Salah satunya adalah Edy Nurhan, dengan akun @edynurhan di berbagai medsos- videonya sedang viral di WhatsApp, karena dianggap menyampaikan info yang bagus tentang diabetes.

Padahal sebenarnya banyak yang salah, dan bahkan menyesatkan.

Beberapa yang langsung jelas misalnya adalah:

1/ Fungsi insulin ketika ada gula yang berlebih adalah mengubahnya menjadi lemak.

Karena itu kelebihan gula / karbohidrat menyebabkan kegemukan.

2/ Menyarankan kentang daripada french fries = SALAH, keduanya tetap saja sama-sama karbohidrat.

Bagi yang sedang musti diet karbohidrat, klaim-klaim seperti ini bisa berdampak fatal.

3/ Menyarankan jagung daripada popcorn = SALAH, keduanya tetap saja sama-sama karbohidrat.

Bagi yang sedang musti diet karbohidrat, klaim-klaim seperti ini bisa berdampak fatal.

4/ Klaim bahwa “olahraga adalah yang paling penting” – SALAH, yang paling penting adalah kualitas asupan makanan.

Ini sudah menjadi pemahaman umum di kalangan pakar kebugaran/ fitness.Olahraga sekeras apapun, tapi makanan / asupannya tidak diubah – maka dampaknya akan minim.

5/ Anggap diabetes lebih berbahaya daripada covid19 = lebih bodoh daripada orang awam, bisa menyesatkan, dan bisa menyebabkan korban nyawa.

Kedua-duanya sama = berbahaya.

Tidak boleh malah pakai salah satunya untuk remehkan yang lainnya.Itu adalah kelakuan yang amat serampangan.

Semoga para soothsayer seperti ini, orang bodoh tapi manis mulutnya – segera musnah dari medsos Indonesia ; sehingga tidak lagi bisa menyesatkan masyarakat awam.

ref: video ybs yang sedang menyebar di WhatsApp:

I’ve been traveling quite a lot lately, and in turn have been asked the same questions many times by the event organizer. And being rather forgetful – sometimes I forgot some of it, and had to trouble the EO at the last minutes.

So to avoid that from happening again, here they are:

1. Allergy : I’m allergic to (human skin) dust.
   
   Please ask the hotel staff to clean the room’s AC filter before my arrival.
   Otherwise my throat will inflame, and I’ll have trouble breathing.
   
   I’d prefer hotel with “split AC” / dedicated AC unit (because the filter can be cleaned).
   I’ve had many problems with hotel with Central AC facility – in many cases I fell sick, probably because its AC filter can only be cleaned by its vendor / technician.
   
   Also if there’s carpet / sofa in the hotel room – please ask the hotel staff to vacuum it, extra clean.
   My house does not have carpet because it keeps triggering this allergy. So if the hotel room have carpet, it’s alright, as long as it’s clean.
   .
2. Accommodation preference : Non-smoking, high floor, breakfast included.
   .
3. Food / dietary requirement : Halal, or vegetarian, or vegan.
   
   I have very, very low alcohol tolerance. Just a little bit of it is enough to make me nauseous.
   Please ask the caterer to ensure that there’s no alcohol in my food & drink.
   
   I’m not a picky eater, so don’t worry, I’ll appreciate anything that’s provided.
   .
4. Airplane : short flight = window seat. long flight (> 3 hours) = aisle seat.
   .
5. Train : preferably single seat / not having someone else next to me.
   .
6. Bus : sleeper seat if possible, otherwise anything is fine.
   .
7. Venue / Event : strictly non-smoking please – I suffer from asthma, cigarette’s smoke will cause me breathing problems.

Thank you.

A few days ago a client contacted me and said that their website is down. I checked the server, and indeed the server is very overloaded. In total it got 36 cores – and they’re all 100% utilized.

As usual I checked the whole stack, but today it’s something different – the MySQL / database server was the culprit. It was performing very slowly, and in turn caused the webserver to slow down as well.

In the slow query log, some queries kept showing up with crazy query times, in tens of seconds.
For comparison – all of the other queries finishes in less than a second.

MySQL’s slow query log is your friend – it enable you to find problematic queries very quickly.

And all those slow queries are in these tables:

wp_statistics_useronline
wp_statistics_visitor

I checked the currently running queries with “mysqladmin processlist“, and almost all (hundreds of them) queued queries are those involving those tables, looking for specific content in the field “ip”. They’re all looked like these:

SELECT location FROM wp_statistics_visitor WHERE ip = '88.88.88.88'
SELECT * FROM wp_statistics_useronline WHERE ip = '99.99.99.99'

On a hunch, I checked the structure of those tables. And right enough, there’s no index for “ip”

An index can increase a query’s performance by a, very, significant amount.

When the size of those tables are big enough, and you have a higher traffic than usual (they just published a very important information that’s of interest to a lot of people) – then suddenly these seemingly innocent queries were able to bring down a 36-core server to its knees.

Anyway, now we know the culprit, the solution is easy enough:

alter table wp_statistics_useronline add index (ip);
alter table wp_statistics_visitor add index (ip);

And voilà – in an instant, the website was up again, and the CPU utilization dropped to nearly zero.

Everyone’s happy, and I have also notified the developers about the issue as well.

For English (and other) speakers: this is a post about general financial management. Click on the “Translate” button on the right to have this article translated into your language.

Tidak sengaja saya membaca artikel tentang FI/RE (Financial Independence/Retiring Early). Di dunia yang ideal, semua orang punya akses ke UBI (Universal Basic Income) – yaitu dimana Pemerintah menjamin bahwa setiap bulan Anda akan selalu mendapatkan sejumlah uang.

Namun ketika ini belum ada, maka kita perlu melakukan berbagai usaha lainnya agar kondisi finansial keluarga kita selalu aman. Artikel ini adalah catatan pribadi saya tentang berbagai informasi terkait hal ini.

Disclaimer: artikel ini hanya catatan pribadi saya. Keuntungan/kerugian finansial yang terjadi karena membaca isi artikel ini bukan tanggung jawab saya. Dengan mengakses artikel ini, maka Anda telah setuju dengan disclaimer ini.

Mayoritas orang hidup dari gajian ke gajian. Jumlah tabungan minim. Atau kalaupun ada tabungan/investasi, kurang dikelola – sehingga kalah cepat dari besaran inflasi
= jumlah uang Anda malah berkurang setiap tahunnya ……

INFLASI – Target untuk dikalahkan

Selama 2010 s/d 2020, rata-rata inflasi per tahun adalah 4.48%

Artinya : nilai uang Anda berkurang sebesar 4.48% setiap tahun.

Maka, strategi investasi / tabungan Anda harus memberikan hasil yang lebih besar dari ini.
(setelah dipotong pajak, zakat, komisi, dst).

Mari kita lihat beberapa strategi investasi / tabungan yang ada:

1. EMAS

Dengan rata-rata peningkatan nilai investasi sebesar 21,67% setiap tahun, emas jelas bisa mengalahkan gerogot inflasi pada harta Anda.

RESIKO KERUGIAN INVESTASI : rendah

Tantangan

1. Penyimpanan : Safe deposit bank : bukan jaminan aman, di luar negeri sudah sering terjadi kasus safe deposit box yang lenyap.
2. Penyimpanan : Rumah : ukuran emas memang kecil sehingga mudah disembunyikan, namun jangan sampai lupa tempatnya. Keberadaan emas di rumah juga bisa membahayakan penghuni jika sampai diketahui oleh penjahat.

PROS

1. Ukuran kecil, mudah disimpan.
2. Cukup likuid / mudah dijual kembali, beda dengan misalnya perak.

CONS

1. Resiko menyimpan benda berharga yang berbentuk fisik.
2. Harganya cenderung agak mahal.
3. Musti paham emas seperti apa yang harganya stabil – beberapa bentuk emas, seperti dinar dll, harganya bisa jatuh cukup banyak ketika dijual kembali.

x

TANAH / RUMAH / PROPERTI

RESIKO KERUGIAN INVESTASI : Agak tinggi

PROS

1. Jika tepat memilih lokasi, ROI (return on investment) bisa cukup tinggi.

CONS

1. Sangat tergantung pada lokasi : jika salah pilih lokasi, malah bisa turun harga dan/atau sulit dijual kembali.
2. Cenderung tidak likuid : susah dijual dalam waktu cepat, kecuali jika harganya diturunkan jauh di bawah harga pasar.
3. Potensi resiko : mafia tanah : bukan sekali dua kali kejadian mendadak tanah sudah dikuasai oleh pihak lainnya, lengkap dengan sertifikat tanah asli.
4. Modal tinggi : membutuhkan dana dalam jumlah besar untuk melakukan investasi jenis ini.

.

LAIN-LAIN

Tentu saja masih ada banyak sekali skema-skema tabungan & investasi lainnya. Misalnya di luar negeri ada berbagai institusi Fund Management yang bisa membantu mengelola dana Anda. Anda bisa turut bergabung dengan dana yang tidak besar dan cenderung aman.

Silakan jika ada saran / masukan, bisa disampaikan via Telegram atau Facebook di bawah ini.

.

Penutup

Artikel ini masih belum selesai, karena saya masih terus mendalami soal ini.
Artikel ini akan terus diperbaharui setiap kali saya mendapatkan informasi baru.

Berbagai data & chart di artikel ini bisa dilihat di Google Sheet ini :

“Performa Berbagai Investasi versus Inflasi Indonesia” = https://docs.google.com/spreadsheets/d/1pRF3W8BuAYGyG5RVHtvN-DeZXuz2CEKCkzl5lm4XGKo/

Jika ada data yang ingin Anda sumbangkan, silakan kontak saya via Telegram di “sufehmi” (respons cepat), atau via Facebook https://www.facebook.com/sufehmi (jarang saya cek).
Sumbangan data silakan dikirim dengan format CSV, dan lalu akan saya gabungkan ke dokumen di atas.

Jika ada kekeliruan / koreksi / masukan, jangan segan kontak saya via Telegram / Facebook di atas.

Demikian artikel ini, semoga bermanfaat bagi Anda.

Referensi / Bacaan tambahan

1. https://brianlovin.com/writing/investing-for-designers-and-developers
2. https://i.imgur.com/Vlt0DOR.png

In my years of experience as IT architect, it’s quite shocking to see how many institutions are slacking about their backup system once they moved to the cloud. Especially with their DRC (disaster recovery center). They thought that once they go “up” to the cloud, then it’s all right. No need to worry anymore with troublesome stuff such as backup.

As harsh as it may sound, my friend said that “cloud is other people’s computer”, and it’s a fact. And computer will fail. It’s just a matter of when, not if. And cloud did indeed fail from time to time.

When your organization does not have a solid backup system, then when the cloud fail – you are in for a very unpleasant experience.

“There’s no such thing as too much backup” – this is another principle that’s true. I have been in various data loss incidents, one of them were saved by the fifth (5th) backup mechanism. All other four failed.

But of course the implementation of the backup system will need to balance between levels of data safety and actual available resources.

A DRC can be of various shapes and sizes, customized to fit one’s system recovery needs versus available resources/budget. There are 8 levels of Disaster Preparedness, and we can choose the one that fits our needs & available resources.
But it simply has to exist. Any institution with data & systems considered important, need to have a working DRC facility.

And a DRC does not always have to be complex or expensive. There are ways to make a fully working DRC with minimum resources. And along time, it can be tweaked even further.

Moving to the cloud is not an excuse to avoid having a good backup strategy. We don’t need to be caught with our pants down.

This mouse feels good to touch. It just feels nice. The scroll wheel, called Magspeed Wheel, feels really good to use with its tactile feeling. However try pressing the black button in the middle – then it changed from Ratchet mode to Freespin mode , basically it flies. You can spin it really, really fast. Awesome.

However, Logitech does not provide its configuration software on Linux. But no worries, we can use LogiOps for that.

Copy-paste these lines to set it up ; these are for Ubuntu 20.04, if you’re using different Linux distro, you might need to change some of it.

sudo apt-get install -y install cmake libevdev-dev libudev-dev libconfig++-dev

cd /tmp ; wget https://github.com/PixlOne/logiops/archive/refs/heads/master.zip ; mkdir tmp ; cd tmp ; unzip ../master.zip ; cd logiops-master ; mkdir build ; cd build

cmake .. ; make ; sudo make install 

sudo nano /etc/logid.cfg

sudo systemctl enable --now logid

You may notice that we’re creating a configuration file name logid.cfg , there’s a [ guide to create it ], however some may find it confusing.

Therefore please find a sample logid.cfg for MX Anywhere 3 below. It will enable a reasonably nice usage of the device, and also enable you to change the mouse’s DPI by pressing the side buttons.

Enjoy.

// Logiops (Linux driver) configuration for Logitech MX Master 3.
// Includes gestures, smartshift, DPI.
// Tested on logid v0.2.2-35-g1c209ed.

// File location: /etc/logid.cfg

devices: ({
  name: "MX Anywhere 3";

  smartshift: {
    on: true;
    threshold: 15;
  };

  hiresscroll: {
    hires: true;
    invert: false;
    target: true;
       up: {
            mode: "Axis";
            axis: "REL_WHEEL_HI_RES";
            axis_multiplier: 1;
        },
        down: {
            mode: "Axis";
            axis: "REL_WHEEL_HI_RES";
            axis_multiplier: -1;
        },
  };

  dpi: 1600; // max=4000


    buttons: (
        {
            cid: 0x52;
            action =
            {
                type: "Gestures";
                gestures: (
                    {
                        direction: "Left";
                        mode: "OnInterval";
			interval: 10;
                        action =
                        {
                            type: "Keypress";
                            keys: ["KEY_VOLUMEDOWN"];
                        };
                    },
                    {
                        direction: "Right";
                        mode: "OnInterval";
			interval: 10;
                        action =
                        {
                            type: "Keypress";
                            keys: ["KEY_VOLUMEUP"];
                        };
                    },
                    {
                        direction: "None"
                        mode: "OnRelease";
                        action =
                        {
			    type: "Keypress";
			    keys: ["BTN_MIDDLE"];
                        }
                    }
                );
            };
        },
        {
            cid: 0x53;
            action =
            {
//                type: "Keypress";
//                keys: ["KEY_BACK"];
		type: "ChangeDPI";
              	inc: -1000;            
            };
	},
        {
            cid: 0x56;
            action =
            {
//                type: "Keypress";
//                keys: ["KEY_FORWARD"];
		type: "ChangeDPI";
              	inc: 1000;            
            };
	}
    );
}
);

I used to play Roblox with my kids from my Linux-based (Ubuntu) laptop – by having a virtual machine set up (on VirtualBox) with Windows, and play there. Of course it’s slow, but I still CAN play.

However one day Roblox decided to disable playing from Virtual Machine. And there goes my little bit of happiness with my children.

One day I found this article about using Steam Link to enable remote access to a Windows desktop, and it got me an idea – can I use this trick to play Roblox again from my laptop?

So I setup Google Chrome to be streamable from my gaming computer on my house’s second floor – and voila, it showed up in my Steam account indeed !

By invoking Google Chrome on my Steam app on Linux – then I can browse to Roblox.com, and then play all the games there.

Mission accomplished !

Use Steam to Stream Your Desktop Instead of Your Games : https://lifehacker.com/use-steam-to-stream-your-desktop-instead-of-your-games-1818722875

I used to use my Android smartphone as webcam for Zoom / Skype / Google Meet / etc because my laptop’s webcam is so bad. This is possible thanks to the Droidcam app.

But sometimes there are problems, like the wifi got interference so the video would slow down or freeze for a while. And for long conference / meeting, it got my phone pretty hot because sometimes I have to charge it. And of course I can’t use my phone while it’s being used as webcam.

So I bought Logitech C920 Pro webcam, and started using it instead. In Linux it’s recognized and can bs used straight away.
But you may need to tweak its image quality a bit using guvcview before being used for work.

The picture quality is not as good as my smartphone’s , because my smartphone is heavily processing the images, so it came out even with HDR quality, in real-time. But as a daily work webcam, this Logitech webcam is good enough.

Then I need to start using Green screen as well with this webcam. There’s one problem – my Green screen is not wide enough to cover the webcam’s wide angle.

With Droidcam, this is not a problem, there’s a “Zoom” feature. So I just Zoom-in, until the green screen fills the view.
But since Logitech does not provide any kind of software for this webcam on Linux, I use OBS instead.

Using OBS, I can set up green screen in it, so we don’t need to use Zoom’s green screen / Virtual Background feature. And also that means all other software (Google Meet, Skype, etc) will automatically got the already green screened video from OBS.

To zoom-in in OBS, I just enlarge the webcam’s image box, until the green screen fill the view.
To activate green screen, I use the Chroma key filter.

To make OBS become a video source, we’ll need to install obs-v4l2sink : https://github.com/CatxFish/obs-v4l2sink

Turned out there are a few problems installing it in Ubuntu 20.04 , we’ll discuss here the workaround for those:

# Another possible solution is using Snap's version of OBS, 
# which already include v4l2loopback kernel module 
# & obs-v4l2sink plugin
# sudo snap install obs-studio.
# sudo modprobe v4l2loopback video_nr=10 card_label=”OBS Video Source” exclusive_caps=1

# If by any reason you can't use this Snap-based solution, 
# then continue : 


# download needed software for compilation
sudo apt-get update ; sudo apt-get install -y install obs-studio git cmake build-essential libobs-dev ffmpeg qtbase5-dev

cd /tmp ; mkdir myobscode ; cd myobscode

# get OBS' source code
git clone --recursive https://github.com/obsproject/obs-studio.git

# get plugin's source code
git clone https://github.com/CatxFish/obs-v4l2sink

# compile the OBS plugin
cd ~/obs-v4l2sink
mkdir build && cd build
cmake -DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" -DCMAKE_INSTALL_PREFIX=/usr ..

make -j4
sudo make install
sudo cp v4l2sink.so /usr/lib/obs-plugins/
sudo cp /usr/lib/obs-plugins/v4l2sink.so /usr/lib/x86_64-linux-gnu/obs-plugins/

# Turned out we need to compile and build v4l2loopback by ourselves - this is because the Ubuntu's version is too old
# Thanks to user jplandrain : 
# https://github.com/CatxFish/obs-v4l2sink/issues/54#issuecomment-722966599
cd ..
sudo apt-get remove v4l2loopback-dkms
git clone --branch v0.12.5 https://github.com/umlaeute/v4l2loopback.git
cd v4l2loopback
make && sudo make install

# make v4l2loopback automatically loaded by kernel after reboot
echo "v4l2loopback" >> /etc/modules-load.d/modules.conf
echo 'options v4l2loopback video_nr=2' >> /etc/modprobe.d/v4l2loopback.conf

echo 'options v4l2loopback card_label="VirtualCam"' >> /etc/modprobe.d/v4l2loopback.conf

echo 'options v4l2loopback exclusive_caps=1' >> /etc/modprobe.d/v4l2loopback.conf

# load the loopback module into kernel now
sudo modprobe v4l2loopback video_nr=10 card_label="OBS Video Source" exclusive_caps=1

# start OBS - now there should be a new menu : 
#     Tools - V4L2 Video Output
# also you'll need to tick "Autostart" option after choosing that menu
obs &

IPv4 address is becoming rarer by each day. In some cases, it can be pretty hard to get multiple IPv4 address for your Proxmox server.

Thankfully, Proxmox is basically a Debian Linux OS with Proxmox layer on top of that. So that gives us quite a lot of flexibility.

This tutorial will help you to create a fully functional Proxmox server running multiple containers & virtual machines, using only a single IPv4 address.

These are the main steps :

1. Create port forwarding rules
2. Make sure it’s executed automatically everytime the server is restarted
3. Setup a reverse-proxy server : to forward HTTP/S requests to the correct container / virtual machine
4. Setup HTTPS

For CT (container) / VM (virtual machine) that contains webserver, point 3 is important – because there’s only one public IP address, so there’s only one port 80 and 443 that’s facing the Internet.

By forwarding port 80 and 443 to a reverse-proxy in a CT, then we’ll be able to forward incoming visitors, by hostname / domain name, to the correct CT/VM.

1. CREATE PORT FORWARDING RULES

Modify the following to match your host’s interface name & CT/VM’s internal IP addresses, then copy-paste to terminal :

###### All HTTP/S traffic are forwarded to reverse proxy
iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 80 -j DNAT --to 10.10.50.1:80

iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 443 -j DNAT --to 10.10.50.1:443

###### SSH ports to each existing CT/VM
iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22101 -j DNAT --to 10.10.50.1:22

iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22102 -j DNAT --to 10.10.50.2:22

iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22103 -j DNAT --to 10.10.50.3:22

iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 22104 -j DNAT --to 10.10.50.4:22

Then we save it :

iptables-save > /etc/iptables.rules

2. EXECUTE IPTABLES AT SERVER RESTART

Edit /etc/network/interfaces file, find your network interface name that’s facing the Internet (in my case, vmbr0) – then add the pre-up line as follows :

auto vmbr0
pre-up iptables-restore < /etc/iptables.rules

3. SETUP REVERSE-PROXY

In a CT, install Nginx. Then for each domain, create a configuration file like this, for example: /etc/nginx/sites-available/www.my_website.com :

server {
listen 80;
server_name www.my_website.com;

location / {
    proxy_pass http://10.10.50.2:80;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

}

To activate it (assuming you’re using Ubuntu) link it to /etc/nginx/sites-enabled/ , then restart Nginx :

ln -s /etc/nginx/sites-available/www.my_website.com /etc/nginx/sites-enabled/www.my_website.com

/etc/init.d/nginx restart

note: as noted before, all HTTP/s traffic will have to go through this reverse-proxy. You may wish to tune this Nginx installation accordingly.

4. SETUP HTTPS

It’s very easy with Let’s Encrypt once you’ve done point 3 above. Do the following on the reverse-proxy CT :

sudo apt-get update ; sudo apt-get install -y certbot python3-certbot-nginx

sudo certbot --nginx

sudo /etc/init.d/nginx restart

Reference:

https://gist.githubusercontent.com/basoro/b522864678a70b723de970c4272547c8/raw/a985657453f72683040fbe38b1db6b1989618116/proxmox-proxy

Today I needed to have the latest version of HTTrack installed to make a (static) mirror of a website that I managed

After a few attempts, this is how you compile & install HTTrack from source on Ubuntu :

wget "http://download.httrack.com/cserv.php3?File=httrack.tar.gz"

mv cserv.php3\?File\=httrack.tar.gz  httrack.tar.gz

tar xzvf httrack.tar.gz

cd httrack-3.49.2/

### the following is the key to a successful install
apt-get install zlib1g-dev libssl-dev build-essential

./configure && make && make install