Array of enums with associated values

[jsbean]

Thanks for taking up the maintenance on this project!

I am working with an XML schema which requires support for heterogeneous arrays of values within a set of specified types.

Consider the following enum which defines each of the types allowable:

enum AB {
    struct A: Decodable { let value: Int }
    struct B: Decodable { let value: String }
    case a(A)
    case b(B)   
}

Currently, the Decodable implementation is as follows (based on objc.io/codable-enums):

extension AB: Decodable {
    enum CodingKeys: String, CodingKey { case a, b }
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        do {
            self = .a(try container.decode(A.self, forKey: .a))
        } catch {
            self = .b(try container.decode(B.self, forKey: .b))
        }
    }
}

A minimal example of the source XML could look something like this (though there could be arbitrary amounts of either type, in any order):

let xml = """
<?xml version="1.0" encoding="UTF-8"?>
<container>
    <a value="42"></a>
    <b value="forty-two"></b>
</container>
"""

Upon decoding it with the XMLDecoder:

let decoder = XMLDecoder()
let abs: [AB] = try! decoder.decode([AB].self, from: xml.data(using: .utf8)!)

We get the following:

// => [.a(AB.A(value: 42))]

The desired result should be:

// => [.a(AB.A(value: 42)), .b(AB.B(value: "forty-two"))]

I have tried many combinations of keyed, unkeyed, singleValue, and nested containers, yet haven't found a solution that treats the data as an Array rather than a Dictionary.

Let me know if I am holding this wrong (at either the Decodable or the XMLDecoder stages), or if this could be a current limitation of the XMLCoder implementation. I'd be happy to contribute, but if anyone notices something glaring here, this would be very helpful.

[MaxDesiatov]

Hi @jsbean, first of all thanks a lot for the report and sorry for the delay caused by our current focus on test coverage in XMLCoder. We've finally reached ~75% coverage and are ready to look into more bugfixes and disruptive changes in the codebase.

This indeed looks like a bug caused by XMLCoder wrapping the inner unkeyed box twice: first in a keyed box and then an unkeyed box on top of it. I'm currently looking into it more closely and will keep you updated.

[jsbean]

@MaxDesiatov, no worries! Apologies for disappearing for a bit. Went unplugged.

I'm glad the effort was put into enhancing the test coverage. I'll keep an eye on this. Let me know if there is anything I can do to contribute.

[MaxDesiatov]

somewhat related: https://forums.swift.org/t/automatic-codable-conformance-for-enums-with-associated-values-that-themselves-conform-to-codable/11499

[flowbe]

Hello!

I am also affected by this issue, any news?

[jsbean]

I am currently only looking at this from the decoding side of things, but here are some thoughts.

Here is a pipeline that seems important for this feature:

XMLCoderElement.flatten()-> KeyedBox
XMLStackParser.parse(...) -> KeyedBox
XMLDecoder.decode(...)

Inside XMLDecoder.decode, XMLStackParser.parse is called, returning a KeyedBox, though it is up-cast immediately to an abstract Box. Is it possible to instead return Box from XMLStackParser.parse? The keyed-ness is an issue for the following scenario (slightly simplified from the one in the original post):

Here's an enum:

enum IntOrString {
    case int(Int)
    case string(String)
}

Here's its Decodable bits:

extension IntOrString: Decodable {
    enum CodingKeys: String, CodingKey {
        case int
        case string
    }
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        do {
            self = .int(try container.decode(Int.self, forKey: .int))
        } catch {
            self = .string(try container.decode(String.self, forKey: .string))
        }
    }
}

Here's an example of it in the wild:

let xml = """
<container>
    <int>"42"</int>
    <string>"forty-two"</string>
    <int>"43"</int>
</container>
"""

If we try to decode it like this:

let values = try XMLDecoder().decode([IntOrString].self, from: xml.data(using: .utf8)!)

Within XMLDecoder.decode, the topLevel: Box we get from XMLStackParser.parse (who got it from XMLCoderElement.flatten) is a KeyedBox, with elements of:

["int": ["42", "43"], "string": "forty-two"]

In this case, it does not behoove us to immediately treat this as a KeyedBox, as we lose information: the node: XMLCoderElement returned by XMLStackParser(...) -> XMLCoderElement has everything we need.

@MaxDesiatov, am I looking in the right place for this?

[MaxDesiatov]

I've been working on this issue for the last couple of days in attempt to resolve #91 and you're right, most of the problems are in how flatten works. The name is unhelpful though, it doesn't flatten anything, only transforms a tree of XMLCoderElement to a tree of boxes. I'm going to rename it to transformToBoxTree or similar.

A temporary hack I've applied during my experimentation was to transform a KeyedBox to an UnkeyedBox, but your comment is super helpful @jsbean! I now realize that transformation would convert

{"int": ["42", "43"], "string": "forty-two"}

into

[{"int": "42"}, {"int": "43"}, {"string": "forty-two"}]

which still loses the original order of keys.

To clean this up I'm thinking of removing UnkeyedBox entirely and making KeyedBox store everything internally as an array with an additional mapping from keys to array indices. This way both KeyedContainer and UnkeyedContainer would decode from KeyedBox. KeyedContainer would work as it did previously, while UnkeyedContainer is going to get the access to the original array of elements. Some of these cases were previously worked around in master with double wrapping KeyedBox in an UnkeyedBox and vice versa, which is unnecessarily complex. I'll create PRs as I go so that we don't conflict if anyone is working on this too.