fix conditional useHref call

[abeljohn]

Closes #7028

✅ Pull Request Checklist:

• Included link to corresponding React Spectrum GitHub Issue.
• Added/updated unit tests and storybook for this change (for new code or code which already has tests).
• Filled out test instructions.
• Updated documentation (if it already exists for this component).
• Looked at the Accessibility Practices for this feature - Aria Practices

📝 Test Instructions:

🧢 Your Project:

[snowystinger]

thanks for opening a PR and starting to look at what makes sense for the types

You'll need to sign the CLA https://react-spectrum.adobe.com/contribute.html#contributor-license-agreement then you may need to close and reopen the PR to get it to take hold.

[abeljohn]

Thanks @snowystinger , should be good to go now!