include input parameters in PipelineOutput

[keturn]

It's useful to be able to associate a pipeline result with the parameters that generated it. It provides a record for the experimenter themselves, other people looking to learn from them, those interested in reproducible results, and community-wide surveys.

Current pipelines have a number of values they set by default when they are not provided, so the only way for an application to know all the inputs is either
a) use a pipeline that provides no defaults for optional arguments, requiring the application to explicitly specify every value, or
b) have the output of a pipeline run include the values of all the inputs it used, including any that were set from a default value or heuristic.

I propose throwing a bunch more fields on StableDiffusionPipelineOutput (and presumably other PipelineOutput classes too), that include most of the args in their __call__ signature.

Potential complications:

• batching. With current pipeline implementations, most parameters are consistent across the batch, but prompt may vary.
• seed. People want this, but the current StableDiffusionPipeline doesn't bother to explicitly seed the PRNG at all, so there's nothing for it to report. There is a way for the calling application to handle this: provide its own seeded Generator, or its own latents, The pipeline run can return its initial latents as well (as proposed in Add better compatibility with diffusers-interpret (and possibly other use cases!) #506).

This is a fairly straightforward change. The reason I ask about it up front instead of diving right in to a PR is because I have the feeling I don't understand all the existing PipelineOutput use cases, i.e. whether there are places that depend on PipelineOutput.to_tuple to stay a particular size.

[patrickvonplaten]

Hey @keturn,

Nice issue and sorry to reply so late here!

Are you talking only about default values for the __call__ function about the pipeline or all config attributes?
I don't really see why not setting defaults or returning the defaults would help here to be honest. Also note that both inputs and outputs should ideally only be tensors or fields that influence the output and outputs should only be tensors or information that results from the call method (such as the "is_nsfw" flag).

Regarding providing a seed this is another topic IMO, but note that we can pass generators instead

[keturn]

This is a stepping stone toward being able to record a metadata file with all the information needed to reproduce, as discussed at https://github.com/invoke-ai/InvokeAI/discussions/689#discussion-4402574

Some of that the caller knows (because they specified it),
some of it they can extract from the pipeline configuration.
But if, for example, the caller doesn't specify num_inference_steps, the metadata writer would be at a loss as to what to record for that.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.