Skip to content

Conversation

@shantanugupta2004
Copy link
Contributor

Fixes #2949
This PR replaces all occurrences of the deprecated warmup_ratio argument with warmup_steps across the examples/directory and documentation.
Specific changes include:

  • Zero Warmup: Replaced --warmup_ratio 0.0 with --warmup_steps 0 in SFT example scripts and Accelerate documentation.
    Using an integer 0 is compatible with previous transformers versions and functionally equivalent.
  • Ratio Warmup: Replaced --warmup_ratio 0.03 with --warmup_steps 0.03 in various finetuning example READMEs (MiSS,
    PiSSA, Bone, CoRDA).

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR attempts to replace the deprecated warmup_ratio argument with warmup_steps across examples and documentation to address issue #2949. The changes fall into two categories: zero warmup cases (0.0 → 0) in shell scripts and documentation, and ratio-based warmup cases (0.03 → 0.03) in README examples.

Key changes:

  • Shell scripts and documentation updated to replace --warmup_ratio 0.0 with --warmup_steps 0 (functionally correct)
  • README examples updated to replace --warmup_ratio 0.03 with --warmup_steps 0.03 (incorrect - critical bug)

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
examples/sft/run_unsloth_peft.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/sft/run_peft_qlora_fsdp.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/sft/run_peft_qlora_deepspeed_stage3.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/sft/run_peft_multigpu.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/sft/run_peft_fsdp_gptq.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/sft/run_peft_fsdp.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/sft/run_peft_deepspeed.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/sft/run_peft.sh Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup
examples/pissa_finetuning/README.md Incorrectly replaces --warmup_ratio 0.03 with --warmup_steps 0.03 (should be an integer)
examples/miss_finetuning/README.md Incorrectly replaces --warmup_ratio 0.03 with --warmup_steps 0.03 (should be an integer)
examples/corda_finetuning/README.md Incorrectly replaces --warmup_ratio 0.03 with --warmup_steps 0.03 (should be an integer)
examples/bone_finetuning/README.md Incorrectly replaces --warmup_ratio 0.03 with --warmup_steps 0.03 (should be an integer)
docs/source/accelerate/fsdp.md Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup in two locations
docs/source/accelerate/deepspeed.md Replaces --warmup_ratio 0.0 with --warmup_steps 0 for zero warmup in two locations

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@BenjaminBossan
Copy link
Member

Sorry for the copilot review, it was triggered on accident. Feel free to ignore.

Copy link
Member

@BenjaminBossan BenjaminBossan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this so quickly. PR LGTM. We should wait for the transformers v5 release before merging this though, so leaving it open for now.

@BenjaminBossan BenjaminBossan added the wait-transformers-v5 Don't merge before transformers v5 release. label Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wait-transformers-v5 Don't merge before transformers v5 release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deprecation of warmup_ratio argument in transformers v5

2 participants