Skip to content

CFFI 1.16.0 fails to recognize nested structs properly for purposes of partial notation #36

Open

Description

@CrazyCasta

Using cffi==1.16.0 the following can not be handled:


ffi = FFI()

ffi.set_source("buggy",
"""
struct a {
    int b;
    struct {
        int c;
    } d;
};
""")

ffi.cdef("""
struct a {
    struct {
        ...;
    } d;
    ...;
};
""")```

The following is the error message:

```Traceback (most recent call last):
  File "~/code/cffi/bug_report_1/bug.py", line 15, in <module>
    ffi.cdef("""
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/api.py", line 112, in cdef
    self._cdef(csource, override=override, packed=packed, pack=pack)
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/api.py", line 126, in _cdef
    self._parser.parse(csource, override=override, **options)
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/cparser.py", line 389, in parse
    self._internal_parse(csource)
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/cparser.py", line 412, in _internal_parse
    self._parse_decl(decl)
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/cparser.py", line 508, in _parse_decl
    self._get_struct_union_enum_type('struct', node)
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/cparser.py", line 839, in _get_struct_union_enum_type
    type, fqual = self._get_type_and_quals(decl.type,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/cparser.py", line 674, in _get_type_and_quals
    tp = self._get_struct_union_enum_type('struct', type, name)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/cparser.py", line 832, in _get_struct_union_enum_type
    self._make_partial(tp, nested)
  File "~/code/cffi/bug_report_1/ve/lib/python3.11/site-packages/cffi/cparser.py", line 867, in _make_partial
    raise NotImplementedError("%s is partial but has no C name" %(tp,))
NotImplementedError: <struct $1> is partial but has no C name```

Looking at the code in and around cparser.py:867 it looks as if there's supposed to be some sort of support for marking the struct as nested, but it never gets called with a parameter for that argument, so it defaults to false.

I don't see a workaround, so in my application I'm just removing the inner struct and hoping I don't need it later, but this does seem as if it would be quite a nasty problem. It gets past the first checks if I put a name on the inner struct, but then fails when it tries to match with the "real" C code, so it definitely seems to be the inner struct and not have a simple workaround.

Activity

  1. arigo commented on Nov 25, 2023

    @arigo
    Contributor

    This is hard to fix. Maybe it's possible, but I don't know for sure. That's the reasoning behind why it's raising a NotImplementedError. In two words: cffi generates C code when you call ffi.compile(). This C code contains stuff like defining wrapper functions that can be called from Python, but also it defines code that put this kind of expression in some static array:

    sizeof(struct a), offsetof(struct a, b), offsetof(struct a, d)
    

    The problem is that it's harder to write these expressions if the type name is unknown, or if it just doesn't have any name at all. In particular, I'm unsure how to write offsetof() in this case to get at the field offsets inside. It's probably possible on specific compilers like gcc, using the gcc extension typeof(), but I'm looking for a solution that works for all C compilers.

  2. arigo commented on Nov 25, 2023

    @arigo
    Contributor

    Oh, according to https://stackoverflow.com/questions/10874211/finding-relative-offset-in-nested-structure then we should simply be able to write offsetof(struct a, d.subfield).

  3. arigo commented on Nov 25, 2023

    @arigo
    Contributor

    In summary, I think that it is possible to implement this. It's a matter of tweaking the build-time cffi.model.StructOrUnion instance to store extra information: "this struct is anonymously defined as a field of that other struct". Then we'd use that when generating C code. It requires some careful handling of a lot of different cases and probably not all cases can be fixed. But for common cases it should work---unless I'm missing something else. Please tell me if you want to give it a go, or if you prefer to leave it to me (for some point in the future: I'm not doing much Python any more).

    Note: you are correct that there is no way to use ...; in an anonymous substructure. As far as I can tell, the only solution is to improve CFFI.

  4. CrazyCasta commented on Nov 26, 2023

    @CrazyCasta
    Author

    Let me take a look and see if I can do anything. Need to see if I can figure out the interface between the cffi.model.StructOrUnion and C code generation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions