48.8566° N, 2.3522° E

OPEN-SOURCE AI LAB

// COMPLEXITY
MACHINE LEARNING

Building efficient transformer architectures
with Mu-Guided Dynamics and Token-Routed MLP

GitHubPaper
SCROLL

// PROJECTS

What We're Building

Complexity-Deep
Active

Token-Routed MLP with Mu-Guided Dynamics. Deterministic expert routing + PID-inspired control for efficient transformers.

PyTorchTritonMoELLM
GitHubPyPI
Pacific-Prime
Training

1.5B parameter language model trained with Complexity-Deep architecture. Mu-guided attention and token-routed experts.

LLM1.5BBF16HuggingFace
🤗HuggingFace
Complexity Framework
Stable

Base transformer architecture library. Foundation for all Complexity models with modern attention patterns.

PyTorchGQARoPEFlashAttention
GitHubPyPI
CGGR Kernels
Active

Contiguous Group GEMM Routing - Triton kernels achieving 5-6x speedup for expert routing operations.

TritonCUDAOptimization
GitHub

// PUBLICATIONS

Research

Complexity-Deep: Token-Routed MLP with Mu-Guided Dynamics for Efficient Transformer Architectures

Boris Peyriguere

Zenodo • 2026

We present Complexity-Deep, a novel transformer architecture that combines deterministic token-routed MLP with mu-guided dynamics for efficient and stable training.

Read Paper
DOI: 10.5281/zenodo.18293026

Cite Our Work

@software{peyriguere2026complexity,
  author       = {Peyriguere, Boris},
  title        = {Complexity-Deep: Token-Routed MLP with
                  Mu-Guided Dynamics for Efficient
                  Transformer Architectures},
  year         = 2026,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.18293026},
  url          = {https://doi.org/10.5281/zenodo.18293026}
}

// ABOUT

Our Mission

Complexity-ML is dedicated to developing efficient and innovative transformer architectures. Our research focuses on making large language models more accessible through novel routing mechanisms and dynamics-inspired control systems.

μ

Mu-Guided Dynamics

PID-inspired control mechanism that maintains context across layers through velocity and mu accumulation.

Token-Routed MLP

Deterministic expert routing based on token identity. Perfect load balance without routing collapse.

CGGR Kernels

Custom Triton kernels for contiguous group GEMM routing. 5-6x speedup over naive implementations.