What does this PR do?

Earlier PR #3560 broke FSDP2 for non-compiled models due to omitted keys from the named_params dict. This change fixes the mistake in the if-replace logic, by conditionally stripping the "_orig_mod." prefix only when it's present as intended.

Fixes #3554

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@S1ro1, @SunMarc

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.