Warn when LoRA skips the dense and shared expert layers of MoE models on transformers v5 with peft 0.19 to 0.21.2

[albertvillanova]

This PR makes the trainers warn when a LoRA config would silently not train the dense and shared expert layers of an MoE model, which happens with transformers v5 and peft 0.19 to 0.21.2.

Closes #7767.

Motivation

With transformers v5 and peft 0.19 to 0.21.2, the latest release, target_modules like ["gate_proj", "up_proj", "down_proj"] or "all-linear" only train the routed experts of MoE models that have a dense first layer or shared experts (GLM-4-MoE, DeepSeek-V3, ...): the dense and shared expert layers get no LoRA, with no warning. It is fixed on peft main, but these versions remain in TRL's supported range. See #7767 for the root cause.

Solution

Before creating the PEFT model, the trainers log a warning when all of these hold:

• the config is a LoraConfig, with transformers>=5.0.0 and peft>=0.19.0,<0.21.3.dev0 (peft main, which has the fix, is 0.21.3.dev0)
• the model has fused routed experts (3D parameters under .experts.)
• the model also has nn.Linear layers named like expert projections (gate_proj, up_proj, down_proj, w1, w2, w3), i.e. the dense or shared expert layers
• target_modules would reach them (a string such as "all-linear", or a leaf name in that set)

It is a warning rather than an error: the routed experts are still trained, and no target_modules spelling avoids the issue with these versions, so an error would block training these models on the latest PEFT release.

Tests will be added in a follow-up PR, together with those for the #7769 guard. Verified locally with SFT on transformers 5.19.0:

peft	GLM-4-MoE MLP targets	all-linear	attention only	Qwen3-MoE MLP targets	dense model
0.19.1	warning	(fails in peft)	none	none	none
0.20.0, 0.21.2	warning	warning	none	none	none
main (0.21.3.dev0)	none	none	none	none	none

On transformers 4.56.2 with peft 0.21.2, nothing is logged.

Changes

• Log a warning before get_peft_model in the SFT, DPO, KTO, GRPO, RLOO, Reward and Distillation trainers, with the same block in each

---

Note

Low Risk
Logging-only change before PEFT wrapping; no training logic or defaults are modified.

Overview
Adds a pre-get_peft_model warning in SFT, DPO, KTO, GRPO, RLOO, Reward, and Distillation trainers when LoRA on transformers v5 with peft 0.19.0–0.21.2 would only adapt routed MoE experts and leave dense/shared MLP projections (gate_proj, up_proj, down_proj, etc.) without adapters—matching the silent gap fixed on peft main (≥0.21.3.dev0).

The check mirrors the existing #7765 guard (same MoE + target_modules heuristics) but uses logger.warning instead of raising, so training can continue on the latest supported peft release while users are pointed to upgrade peft or install from main (#7767).

^{Reviewed by Cursor Bugbot for commit 7273cb2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.