[Flux Kontext] Support Fal Kontext LoRA

[linoytsaban]

example:

import torch
from diffusers import FluxKontextPipeline
from diffusers.utils import load_image

pipe = FluxKontextPipeline.from_pretrained("black-forest-labs/FLUX.1-Kontext-dev", torch_dtype=torch.bfloat16)
pipe.load_lora_weights("fal/Plushie-Kontext-Dev-LoRA", weight_name="plushie-kontext-dev-lora.safetensors")

pipe.to("cuda")

input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png")

image = pipe(
  image=input_image,
  prompt="Convert to plushie style",
  guidance_scale=2.5
).images[0]

[linoytsaban]

@bot /style

[github-actions]

Style bot fixed some files and pushed the changes.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Thanks for doing it! Since it shares a significantly amount of overlap with the existing conversion methods, could we try to leverage them instead?

[linoytsaban]

IIRC it overlaps the most with bfl_control lora but diverges in the norms, context_embedder, X_embedder, guidance and time_text_embed (and other minor things like the block prefix). Could try to refactor the bfl control conversion to account for both use cases

[sayakpaul]

Cool, let's merge this PR and then do a refactor in the future one?

[linoytsaban]

Cool, let's merge this PR and then do a refactor in the future one?

SG to me :)

[linoytsaban]

@sayakpaul these failing tests seem unrelated, no?

[sayakpaul]

Yeah failures don't seem related.