Unable to pass "downloader_config" key

[philnewm]

Summary

I'm trying to create content for a pulp file repository directly via file_url.
As the artifacts I want to sync are pretty large (up to 9GB) the default timeout is too low and kills the download process (or rather the worker) in some cases from the server side while its still running.
Details in this issue: #7641

So I got pointed to the "downloader_config" key for file content creation which is described like this on my servers integrated docs page:

object
Configuration for the download process (e.g., proxies, auth, timeouts). Only applicable when providing a 'file_url.

The public api docs don't seem to include that part yet.

Steps to reproduce

Try creating pulp file content like this via pulp-glue:

pulp_ctx = PulpContext(
    api_root="/pulp/",
    api_kwargs={
        "base_url": pulp_url,
        "username": pulp_user_name,
        "password": pulp_password,
    },
    verify_ssl=False,
    timeout=timedelta(1800),
)


pulp_file_ctx = PulpFileContentContext(pulp_ctx=self.pulp_context, repository_ctx=self.pulp_repo_ctx)
pulp_file_ctx.create(
  body={
    "file_url": artifact_url,
    "relative_path": relative_path,
    "repository": self.pulp_repo_ctx.entity["name"],
    "downloader_config": {
        "total_timeout": 1800.0
    },
  }
)

Expected behavior

A download from the provided artifact url should start on the pulp server and respect the set "total_timeout" so it can finish even for large artifacts.

Stacktrace/Error log

pulp_glue.common.exceptions.PulpException: Task /pulp/api/v3/tasks/019dd4cf-158e-7f11-aadb-b0921030fa0c/ failed: '{'downloader_config': {'non_field_errors': [ErrorDetail(string='Invalid data. Expected a dictionary, but got str.', code='invalid')]}}'

And when I send it like this "downloader_config": json.dumps({"total_timeout": 1800.0}
I get a different error:

pulp_glue.common.exceptions.ValidationError: Validation failed for 'content_file_files_create':
  application/x-www-form-urlencoded: 'body[downloader_config]' is expected to be a object.
  multipart/form-data: 'body[downloader_config]' is expected to be a object.

At this point I assume an issue on the client side.

Pulp and pulp-cli version info

pulp --version
Pulp3 Command Line Interface, Version 0.39.0
Plugin Versions:
  common: 0.39.0

Left out the other plugins and worker results as they are all up-to-date

pulp status
Notice: Cached api is outdated. Refreshing...
{
  "versions": [
    {
      "component": "core",
      "version": "3.110.0",
      "package": "pulpcore",
      "module": "pulpcore.app",
      "domain_compatible": true
    },
    {
      "component": "file",
      "version": "3.110.0",
      "package": "pulpcore",
      "module": "pulp_file.app",
      "domain_compatible": true
    }
  "database_connection": {
    "connected": true
  },
  "redis_connection": {
    "connected": false
  },
  "storage": {
    "total": 261292556288,
    "used": 79406088192,
    "free": 181886468096
  },
  "content_settings": {
    "content_origin": "http://containervm:8080",
    "content_path_prefix": "/registry/"
  },
  "domain_enabled": false
}

Additional context

I can't use a file remote as the provider doesn't offer a repository server.

[mdellweg]

I cannot reproduce this with the current main branch:
pulp --dry-run debug openapi call --id content_file_files_create --body '{"file_url": "asdf", "relative_path": "test", "downloader_config": {"total_timeout": 1800.0}}' aborts after that validation step.

More context to compare:

pulp debug openapi schema --name file.FileContent
{
  "type": "object",
  "description": "Serializer for File Content.",
  "properties": {
    "repository": {
      "type": "string",
      "format": "uri",
      "writeOnly": true,
      "description": "A URI of a repository the new content unit should be associated with."
    },
    "pulp_labels": {
      "type": "object",
      "additionalProperties": {
        "type": [
          "string",
          "null"
        ]
      },
      "description": "A dictionary of arbitrary key/value pairs used to describe a specific Content instance."
    },
    "artifact": {
      "type": "string",
      "format": "uri",
      "description": "Artifact file representing the physical content"
    },
    "relative_path": {
      "type": "string",
      "minLength": 1,
      "description": "Path where the artifact is located relative to distributions base_path"
    },
    "file": {
      "type": "string",
      "format": "binary",
      "writeOnly": true,
      "description": "An uploaded file that may be turned into the content unit."
    },
    "upload": {
      "type": "string",
      "format": "uri",
      "writeOnly": true,
      "description": "An uncommitted upload that may be turned into the content unit."
    },
    "file_url": {
      "type": "string",
      "writeOnly": true,
      "minLength": 1,
      "description": "A url that Pulp can download and turn into the content unit."
    },
    "downloader_config": {
      "allOf": [
        {
          "$ref": "#/components/schemas/RemoteNetworkConfig"
        }
      ],
      "writeOnly": true,
      "description": "Configuration for the download process (e.g., proxies, auth, timeouts). Only applicable when providing a 'file_url."
    }
  },
  "required": [
    "relative_path"
  ]
}

pulp debug openapi schema --name RemoteNetworkConfig
{
  "type": "object",
  "description": "Shared network configuration fields and validation logic used by both\nRemoteSerializer and UploadSerializerFieldsMixin.",
  "properties": {
    "ca_cert": {
      "type": [
        "string",
        "null"
      ],
      "minLength": 1,
      "description": "A PEM encoded CA certificate used to validate the server certificate presented by the remote server."
    },
    "client_cert": {
      "type": [
        "string",
        "null"
      ],
      "minLength": 1,
      "description": "A PEM encoded client certificate used for authentication."
    },
    "client_key": {
      "type": [
        "string",
        "null"
      ],
      "writeOnly": true,
      "minLength": 1,
      "description": "A PEM encoded private key used for authentication."
    },
    "tls_validation": {
      "type": "boolean",
      "description": "If True, TLS peer validation must be performed."
    },
    "proxy_url": {
      "type": [
        "string",
        "null"
      ],
      "minLength": 1,
      "description": "The proxy URL. Format: scheme://host:port"
    },
    "proxy_username": {
      "type": [
        "string",
        "null"
      ],
      "writeOnly": true,
      "minLength": 1,
      "description": "The username to authenticte to the proxy."
    },
    "proxy_password": {
      "type": [
        "string",
        "null"
      ],
      "writeOnly": true,
      "minLength": 1,
      "description": "The password to authenticate to the proxy. Extra leading and trailing whitespace characters are not trimmed."
    },
    "username": {
      "type": [
        "string",
        "null"
      ],
      "writeOnly": true,
      "minLength": 1,
      "description": "The username to be used for authentication when syncing."
    },
    "password": {
      "type": [
        "string",
        "null"
      ],
      "writeOnly": true,
      "minLength": 1,
      "description": "The password to be used for authentication when syncing. Extra leading and trailing whitespace characters are not trimmed."
    },
    "max_retries": {
      "type": [
        "integer",
        "null"
      ],
      "format": "int64",
      "description": "Maximum number of retry attempts after a download failure. If not set then the default value (3) will be used."
    },
    "total_timeout": {
      "type": [
        "number",
        "null"
      ],
      "format": "double",
      "minimum": 0.0,
      "description": "aiohttp.ClientTimeout.total (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
    },
    "connect_timeout": {
      "type": [
        "number",
        "null"
      ],
      "format": "double",
      "minimum": 0.0,
      "description": "aiohttp.ClientTimeout.connect (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
    },
    "sock_connect_timeout": {
      "type": [
        "number",
        "null"
      ],
      "format": "double",
      "minimum": 0.0,
      "description": "aiohttp.ClientTimeout.sock_connect (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
    },
    "sock_read_timeout": {
      "type": [
        "number",
        "null"
      ],
      "format": "double",
      "minimum": 0.0,
      "description": "aiohttp.ClientTimeout.sock_read (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
    },
    "headers": {
      "type": "array",
      "items": {
        "type": "object"
      },
      "description": "Headers for aiohttp.Clientsession"
    },
    "download_concurrency": {
      "type": [
        "integer",
        "null"
      ],
      "format": "int64",
      "description": "Total number of simultaneous connections. If not set then the default value will be used.",
      "minimum": 1
    },
    "rate_limit": {
      "type": [
        "integer",
        "null"
      ],
      "format": "int64",
      "description": "Limits requests per second for each concurrent downloader"
    }
  }
}

[philnewm]

Thanks for checking,

I can confirm this one works fine for me as well

pulp --dry-run debug openapi call --id content_file_files_create --body '{"file_url": "asdf", "relative_path": "test", "downloader_config": {"total_timeout": 1800.0}}'
Error: Call aborted due to safe mode

I'm running version 0.39.0

pip show pulp-cli
Name: pulp-cli
Version: 0.39.0
Summary: Command line interface to talk to pulpcore's REST API.

But for the same version this keeps failing right on creation

pulp_file_ctx.create(
  body={
    "file_url": artifact_url,
    "relative_path": relative_path,
    "repository": self.pulp_repo_ctx.entity["name"],
    "downloader_config": {
      "total_timeout": 1800.0
    }
  }
)

And I get this traceback

  File "/home/dev_test/artifact_compare/src/sync.py", line 118, in sync
    pulp_file_ctx.create(
  File "/home/dev_test/artifact_compare/.venv/lib64/python3.12/site-packages/pulp_glue/file/context.py", line 60, in create
    result = super().create(body=body, parameters=parameters, non_blocking=non_blocking)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev_test/artifact_compare/.venv/lib64/python3.12/site-packages/pulp_glue/common/context.py", line 1788, in create
    return super().create(body=body, parameters=parameters, non_blocking=non_blocking)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev_test/artifact_compare/.venv/lib64/python3.12/site-packages/pulp_glue/common/context.py", line 1133, in create
    result = self.call(
             ^^^^^^^^^^
  File "/home/dev_test/artifact_compare/.venv/lib64/python3.12/site-packages/pulp_glue/common/context.py", line 791, in call
    return self.pulp_ctx.call(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/dev_test/artifact_compare/.venv/lib64/python3.12/site-packages/pulp_glue/common/context.py", line 561, in call
    result = self.wait_for_task(result)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev_test/artifact_compare/.venv/lib64/python3.12/site-packages/pulp_glue/common/context.py", line 624, in wait_for_task
    while not self._task_finished(task, expect_cancel=expect_cancel):
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dev_test/artifact_compare/.venv/lib64/python3.12/site-packages/pulp_glue/common/context.py", line 591, in _task_finished
    raise PulpException(
pulp_glue.common.exceptions.PulpException: Task /pulp/api/v3/tasks/019df36c-fdb7-7f98-b690-cf262f64121d/ failed: '{'downloader_config': {'non_field_errors': [ErrorDetail(string='Invalid data. Expected a dictionary, but got str.', code='invalid')]}}'

Will step through the code with a debugger later today and try to figure out where the dictionary changes to a string.

[mdellweg]

while not self._task_finished(task, expect_cancel=expect_cancel):
So a task (and therefore the http request) was dispatched. So I conclude it failed on the server side.

[philnewm]

It might originate on the server side.
I went digging a bit deeper into this issue and found the following.
In case some terms don't make sense in the context I use them: I used some AI help to dig into this as I'm not too proficient debugging such client - server issues.

For the dictionary assigned to "downloader_config" to make it to the server formatted correctly a content type of "application/json" would be required.
But my server seems to not advertise that content type at all so pulp-glue doesn't even try to send it this way.

So I got this

pulp debug openapi spec | python3 -c "
import json,sys
s=json.load(sys.stdin)
rb=s['paths']['/pulp/api/v3/content/file/files/']['post']['requestBody']
print(json.dumps(list(rb['content'].keys()), indent=2))
"
[
  "multipart/form-data",
  "application/x-www-form-urlencoded"
]

And checking for the definition of "downloader_config" per content type I got this

pulp debug openapi spec | python3 -c "
import json,sys
s=json.load(sys.stdin)
rb=s['paths']['/pulp/api/v3/content/file/files/']['post']['requestBody']
for ct, v in rb['content'].items():
    schema = v.get('schema', {})
    ref = schema.get('\$ref')
    props = {'(ref)': ref} if ref else schema.get('properties', {})
    dc = props.get('downloader_config', 'NOT PRESENT')
    print(ct, '->', json.dumps(dc))
"
multipart/form-data -> "NOT PRESENT"
application/x-www-form-urlencoded -> "NOT PRESENT"

I also managed to set up this (ugly) workaround which did solve the issue in my python code during testing.

op: OpenAPISpec = self.pulp_context.api._api_spec
path_spec: PathItem = op.paths["/pulp/api/v3/content/file/files/"]
post_spec = path_spec.post
rb = post_spec.request_body

multipart_media = rb.content["multipart/form-data"]
rb.content["application/json"] = multipart_media

result = pulp_file_ctx.create(
  body={
    "file_url": artifact_url,
    "relative_path": relative_path,
    "repository": self.pulp_repo_ctx.entity["pulp_href"],
    "downloader_config": {
      "total_timeout": 1800.0
    },
  }
)

This way the timeout was passed without issues and an installer sync that failed before due to timeouts on the server site made it through on first try.

And for reference the full status output from the server site:

pulp status
{
  "versions": [
    {
      "component": "core",
      "version": "3.110.0",
      "package": "pulpcore",
      "module": "pulpcore.app",
      "domain_compatible": true
    },
    {
      "component": "ansible",
      "version": "0.29.8",
      "package": "pulp-ansible",
      "module": "pulp_ansible.app",
      "domain_compatible": true
    },
    {
      "component": "container",
      "version": "2.27.8",
      "package": "pulp-container",
      "module": "pulp_container.app",
      "domain_compatible": true
    },
    {
      "component": "deb",
      "version": "3.8.1",
      "package": "pulp_deb",
      "module": "pulp_deb.app",
      "domain_compatible": true
    },
    {
      "component": "gem",
      "version": "0.7.5",
      "package": "pulp-gem",
      "module": "pulp_gem.app",
      "domain_compatible": true
    },
    {
      "component": "hugging_face",
      "version": "0.3.0",
      "package": "pulp_hugging_face",
      "module": "pulp_hugging_face.app",
      "domain_compatible": true
    },
    {
      "component": "maven",
      "version": "0.12.0",
      "package": "pulp-maven",
      "module": "pulp_maven.app",
      "domain_compatible": true
    },
    {
      "component": "npm",
      "version": "0.7.1",
      "package": "pulp-npm",
      "module": "pulp_npm.app",
      "domain_compatible": true
    },
    {
      "component": "ostree",
      "version": "2.6.0",
      "package": "pulp-ostree",
      "module": "pulp_ostree.app",
      "domain_compatible": true
    },
    {
      "component": "python",
      "version": "3.29.0",
      "package": "pulp-python",
      "module": "pulp_python.app",
      "domain_compatible": true
    },
    {
      "component": "rpm",
      "version": "3.36.0",
      "package": "pulp-rpm",
      "module": "pulp_rpm.app",
      "domain_compatible": true
    },
    {
      "component": "certguard",
      "version": "3.110.0",
      "package": "pulpcore",
      "module": "pulp_certguard.app",
      "domain_compatible": true
    },
    {
      "component": "file",
      "version": "3.110.0",
      "package": "pulpcore",
      "module": "pulp_file.app",
      "domain_compatible": true
    }
  ],
  "online_workers": [
    {
      "name": "10105@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:37.551083Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10129@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:37.559773Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10060@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:37.566107Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",Root cause is likely DRF stripping nested object fields from auto-generated form schemas when a file (format: binary) field is present on the same serializer
The server does accept application/json with downloader_config correctly when forced — so it's purely a schema advertisement problem, not a server-side handling issue
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10123@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:37.565226Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    }
  ],
  "online_api_apps": [
    {
      "name": "10137@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:32.587207Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10138@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:32.590510Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    }
  ],
  "online_content_apps": [
    {
      "name": "10155@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.647243Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10178@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.647741Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10159@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.641383Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10169@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.646638Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10162@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.648276Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10170@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.648310Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10166@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.646301Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    },
    {
      "name": "10175@fdfdc1984c3a",
      "last_heartbeat": "2026-05-06T18:43:36.647394Z",
      "versions": {
        "deb": "3.8.1",
        "gem": "0.7.5",
        "npm": "0.7.1",
        "rpm": "3.36.0",
        "core": "3.110.0",
        "file": "3.110.0",
        "maven": "0.12.0",
        "ostree": "2.6.0",
        "python": "3.29.0",
        "ansible": "0.29.8",
        "certguard": "3.110.0",
        "container": "2.27.8",
        "hugging_face": "0.3.0"
      }
    }
  ],
  "database_connection": {
    "connected": true
  },
  "redis_connection": {
    "connected": false
  },
  "storage": {
    "total": 261292556288,
    "used": 168202866688,
    "free": 93089689600
  },
  "content_settings": {
    "content_origin": "http://containervm:8080",
    "content_path_prefix": "/registry/"
  },
  "domain_enabled": false
}

Not sure if it's of any use but this was claude's conclusion

• Root cause is likely DRF stripping nested object fields from auto-generated form schemas when a file (format: binary) field is present on the same serializer
• The server does accept application/json with downloader_config correctly when forced — so it's purely a schema advertisement problem, not a server-side handling issue

[mdellweg]

Can you use pulp -vv [...] with the original api spec to see what actually gets on the wire?

[philnewm]

I'm not entirely sure which kind of command you are talking about but I checked for these:

curl -u <user>:<password> -o api.json http://<server-address>/pulp/api/v3/docs/api.json
From the output the file content related specs look like this

"file.FileContent": {
    "type": "object",
    "description": "Serializer for File Content.",
    "properties": {
        "repository": {
            "type": "string",
            "format": "uri",
            "writeOnly": true,
            "description": "A URI of a repository the new content unit should be associated with."
        },
        "pulp_labels": {
            "type": "object",
            "additionalProperties": {
                "type": "string",
                "nullable": true
            },
            "description": "A dictionary of arbitrary key/value pairs used to describe a specific Content instance."
        },
        "artifact": {
            "type": "string",
            "format": "uri",
            "description": "Artifact file representing the physical content"
        },
        "relative_path": {
            "type": "string",
            "minLength": 1,
            "description": "Path where the artifact is located relative to distributions base_path"
        },
        "file": {
            "type": "string",
            "format": "binary",
            "writeOnly": true,
            "description": "An uploaded file that may be turned into the content unit."
        },
        "upload": {
            "type": "string",
            "format": "uri",
            "writeOnly": true,
            "description": "An uncommitted upload that may be turned into the content unit."
        },
        "file_url": {
            "type": "string",
            "writeOnly": true,
            "minLength": 1,
            "description": "A url that Pulp can download and turn into the content unit."
        },
        "downloader_config": {
            "allOf": [
                {
                    "$ref": "#/components/schemas/RemoteNetworkConfig"
                }
            ],
            "writeOnly": true,
            "description": "Configuration for the download process (e.g., proxies, auth, timeouts). Only applicable when providing a 'file_url."
        }
    },
    "required": [
        "relative_path"
    ]
},

And the mentioned RemoteNetworkConfig shows this

"RemoteNetworkConfig": {
    "type": "object",
    "description": "Shared network configuration fields and validation logic used by both\nRemoteSerializer and UploadSerializerFieldsMixin.",
    "properties": {
        "ca_cert": {
            "type": "string",
            "nullable": true,
            "minLength": 1,
            "description": "A PEM encoded CA certificate used to validate the server certificate presented by the remote server."
        },
        "client_cert": {
            "type": "string",
            "nullable": true,
            "minLength": 1,
            "description": "A PEM encoded client certificate used for authentication."
        },
        "client_key": {
            "type": "string",
            "writeOnly": true,
            "nullable": true,
            "minLength": 1,
            "description": "A PEM encoded private key used for authentication."
        },
        "tls_validation": {
            "type": "boolean",
            "description": "If True, TLS peer validation must be performed."
        },
        "proxy_url": {
            "type": "string",
            "nullable": true,
            "minLength": 1,
            "description": "The proxy URL. Format: scheme://host:port"
        },
        "proxy_username": {
            "type": "string",
            "writeOnly": true,
            "nullable": true,
            "minLength": 1,
            "description": "The username to authenticte to the proxy."
        },
        "proxy_password": {
            "type": "string",
            "writeOnly": true,
            "nullable": true,
            "minLength": 1,
            "description": "The password to authenticate to the proxy. Extra leading and trailing whitespace characters are not trimmed."
        },
        "username": {
            "type": "string",
            "writeOnly": true,
            "nullable": true,
            "minLength": 1,
            "description": "The username to be used for authentication when syncing."
        },
        "password": {
            "type": "string",
            "writeOnly": true,
            "nullable": true,
            "minLength": 1,
            "description": "The password to be used for authentication when syncing. Extra leading and trailing whitespace characters are not trimmed."
        },
        "max_retries": {
            "type": "integer",
            "format": "int64",
            "nullable": true,
            "description": "Maximum number of retry attempts after a download failure. If not set then the default value (3) will be used."
        },
        "total_timeout": {
            "type": "number",
            "format": "double",
            "minimum": 0.0,
            "nullable": true,
            "description": "aiohttp.ClientTimeout.total (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
        },
        "connect_timeout": {
            "type": "number",
            "format": "double",
            "minimum": 0.0,
            "nullable": true,
            "description": "aiohttp.ClientTimeout.connect (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
        },
        "sock_connect_timeout": {
            "type": "number",
            "format": "double",
            "minimum": 0.0,
            "nullable": true,
            "description": "aiohttp.ClientTimeout.sock_connect (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
        },
        "sock_read_timeout": {
            "type": "number",
            "format": "double",
            "minimum": 0.0,
            "nullable": true,
            "description": "aiohttp.ClientTimeout.sock_read (q.v.) for download-connections. The default is null, which will cause the default from the aiohttp library to be used."
        },
        "headers": {
            "type": "array",
            "items": {
                "type": "object"
            },
            "description": "Headers for aiohttp.Clientsession"
        },
        "download_concurrency": {
            "type": "integer",
            "format": "int64",
            "nullable": true,
            "description": "Total number of simultaneous connections. If not set then the default value will be used.",
            "minimum": 1
        },
        "rate_limit": {
            "type": "integer",
            "format": "int64",
            "nullable": true,
            "description": "Limits requests per second for each concurrent downloader"
        }
    }
  },

I also noticed that the pulp-cli doesn't seem to support downloader-config so far.

pulp -vv --refresh-api file content create \
  --file-url "artifact_url" \
  --relative-path "relative_path" \
  --repository "repo_name" \
  --downloader-config '{"total_timeout": 1800.0}'
Usage: pulp file content create [OPTIONS]
Try 'pulp file content create --help' for help.

Error: No such option: --downloader-config
((.venv) ) [dev_test@dev-client ansible-pulp]$ pulp file content create --help
Usage: pulp file content create [OPTIONS]

  Create a file content.

Options:
  --relative-path TEXT  [required]
  --sha256 TEXT         Digest of the artifact to use [deprecated]
  --file-url TEXT       Remote url to download and create file content from
  --repository TEXT     Repository to add the content to in the form
                        '[[<plugin>:]<resource_type>:]<name>' or by href.
  --help                Show this message and exit.

Let me know if there's anything else I can provide.

[mdellweg]

I'm not entirely sure which kind of command you are talking about but I checked for these:

Right, you were calling pulp_glue. So to get to the same results, can you set the python loglevel to debug and capture the debuglogs that should contain what is sent in the request. (I want to be able to see what the server receives and then decides to reject.)

[philnewm]

Got it, here the full debug output.

pulp_glue_debug.txt

[mdellweg]

Seems like requests actually looses that data when querystring encoding the body.

<QueryDict: {'file_url': ['file:///tmp/labor/file'], 'relative_path': ['file'], 'downloader_config': ['total_timeout']}>

So the client does not send the data in the right format.
But why does the serializer fail to complain in the viewset?

[mdellweg]

So it appears that after receiving the wrong post data in www-url-encoded way, drf fails to fail the validate until we turn the QueryDict into json for the task where the the wrong data then fails validation in exactly the way I expect it to fail much earlier.

(Still investigating this.)