[Refactor] Minor Improvement for import utils

[ishan-modi]

What does this PR do?

At times pkg_name (used for import) and dist_name (used for install) are different. The _is_package_available function doesnt work for that see here. This PR helps resolve this through _is_package_available function.

Who can review?

@DN6 @hlky

[hlky]

@bot /style

[github-actions]

Style fixes have been applied. View the workflow run here.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[hlky]

Thanks for the iteration. Looks like packages_distributions was only added in 3.10
https://docs.python.org/3/library/importlib.metadata.html#importlib.metadata.packages_distributions
Can you look into alternatives?

[ishan-modi]

Couldn't find alternatives, but pulled out some code from packages_distribution from python-3.10 and added to _is_package_available function

import importlib
import importlib.metadata as importlib_metadata
import inspect
from collections import defaultdict

def _is_package_available(pkg_name: str, get_dist_name: bool = False):
    pkg_exists = importlib.util.find_spec(pkg_name) is not None
    pkg_version = "N/A"

    if pkg_exists:
        try:
            package_map = importlib_metadata.packages_distributions()
        except AttributeError:
            try:
                # Fallback for Python < 3.10
                package_map = defaultdict(list)
                for dist in importlib_metadata.distributions():
                    _top_level_declared = (dist.read_text('top_level.txt') or '').split()
                    _infered_opt_names = {f.parts[0] if len(f.parts) > 1 else inspect.getmodulename(f) for f in (dist.files or [])} - {None}
                    _top_level_inferred = filter(lambda name: '.' not in name, _infered_opt_names)
                    for pkg in _top_level_declared or _top_level_inferred:
                        package_map[pkg].append(dist.metadata['Name'])
            except:
                package_map = {}
        try:
            if get_dist_name and pkg_name in package_map and package_map[pkg_name]:
                if len(package_map[pkg_name]) > 1:
                    print(f"Multiple distributions found for package {pkg_name}. Picked distribution: {package_map[pkg_name][0]}")
                pkg_name = package_map[pkg_name][0]
            pkg_version = importlib_metadata.version(pkg_name)
        except (ImportError, importlib_metadata.PackageNotFoundError):
            pkg_exists = False

    return pkg_exists, pkg_version

_is_package_available("optimum", get_dist_name=True)

Let me know if this looks good to you ? I have tested works fine for python [3.8, 3.9, 3.10]

[hlky]

LGTM, thanks!

[Copilot]

Pull Request Overview

This PR refactors the _is_package_available function to handle the case where the package name used for import differs from the distribution name used for installation. It also consolidates the logic for determining package version and updates the optimum_quanto check accordingly.

• Updated _is_package_available signature to return a tuple (availability, version) and added an optional get_dist_name parameter.
• Integrated a fallback mechanism for package distribution lookup and refactored the optimum_quanto availability check.
• Added necessary imports and extended the collections import for defaultdict.

Comments suppressed due to low confidence (1)

src/diffusers/utils/import_utils.py:87

• [nitpick] Consider using a separate variable (e.g., 'dist_name') instead of overwriting 'pkg_name' when get_dist_name is True. This improves code clarity by distinguishing between the module name and the distribution name.

pkg_name = package_map[pkg_name][0]

[yiyixuxu]

thanks!