Skip to content

Feature proposal: finetuning from checkpoint #107

@yqzhishen

Description

@yqzhishen

This proposed feature allows finetuning specific parameters from a given checkpoint. If finetuning is enabled, the trainer loads all matched parameters from the source ckpt when a new training task is started.

Need 4 new configuration keys:

  • finetune_enabled: whether finetuning is enabled
  • finetune_ckpt: path to the ckpt to be finetuned
  • finetune_ignored_params: params (name prefixes) to be dropped from the ckpt before finetuning starts
  • finetune_strict_shapes: if set to True, raise errors when tensor shapes mismatch; otherwise skip the mismatching parameters

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions