DiffusionBlocks-MoE distributed training  (dbmoe-v4-20261105)
====================================================================
A 33,217,025-parameter byte-level language model (6 DiffusionBlocks, each 4 transformer layers with a
4-expert MoE FFN, d=256, ctx=256) trained on HuggingFaceFW/fineweb-edu:sample/10BT/*.parquet.
Every block owns one slice of the noise range, so each computer trains a different block with no gradient traffic
between them; only the trained block (about 22 MB) goes back to the coordinator.

  updates 18,784   training bytes 3,181,252,096   active workers 8   AR bits/byte 2.0898

Join: one command, no account, no token, nobody to ask (Linux or macOS, Python 3.9+, a GPU is optional):
  curl -fsSL https://codex.opentransformers.online/dbmoe/join.sh | sh
By hand, also on Windows (use py instead of python3 there, and curl.exe in PowerShell):
  python3 -m pip install torch --index-url https://download.pytorch.org/whl/cpu
  curl -fsSL https://codex.opentransformers.online/dbmoe/code -o dbmoe.py
  python3 dbmoe.py worker [--device cuda|rocm|xpu|mps] [--threads 4] [--hours 1]
Want your own token first?  python3 dbmoe.py enrol   (optional; the worker does this by itself)
Many machines, one identity: python3 dbmoe.py enrol, copy worker.token to each one, run python3 dbmoe.py worker --token @worker.token --name MACHINE
AMD GPUs need a ROCm build of PyTorch: https://github.com/Marxist-Leninist/dbmoe#amd-gpus-rocm
Macs: https://github.com/Marxist-Leninist/dbmoe#apple-mac-macos
Intel GPUs need an XPU build of PyTorch: https://github.com/Marxist-Leninist/dbmoe#intel-gpus-xpu
The worker gets its own credential by itself, trains one block at a time and serves that block for network inference
(--no-serve turns serving off).  Every update is tested on held-out text before it is merged.  Ctrl+C stops it.
Leaving for good: curl -fsSL https://codex.opentransformers.online/dbmoe/leave.sh | sh     (or python3 dbmoe.py leave; takes you off the public list and ends your credential)
Live status: https://codex.opentransformers.online/dbmoe/status     Sample text: https://codex.opentransformers.online/dbmoe/sample?prompt=The%20&n=120     Network sample: https://codex.opentransformers.online/dbmoe/sample?net=1&route=auto&prompt=The%20&n=32