PSET-v2 Output Serialisation

[sekulicd]

If i understood correctly keypairs should be serialised in order as defined by BIP 174 and in [here].(https://github.com/ElementsProject/elements/blob/master/doc/pset.mediawiki)

Currently this is not the case for output serialisation as script is serialised before amount in here.
As per BIP 174 it should be opposite.

I can see @achow101, in PSBT2 branch, follows order defined in BIP 174/370, check here.

[achow101]

BIP 174 recommends a serialization order, but records are not required to be in that order. Your deserializer must be able to handle records in a different order and should not rely on records being provided in a specific order.

[sekulicd]

Yes i guess deserialisation is not impacted, but since PSET serialisation is not standardised then round trip test defined in here could fail(like in my case) for different serialisation implementation since input byte array(before deserialised to PSET object) doesnt need to be same as output byte array.

[apoelstra]

Yes, that unit test is specifically for rust-elements (which makes up its own canonical serialization, specifically so that we could write round-trip tests for our fuzzer).

[altafan]

BIP 174 recommends a serialization order, but records are not required to be in that order. Your deserializer must be able to handle records in a different order and should not rely on records being provided in a specific order.

Is the serialization order defined in BIP 174 actually a recommendation? Consider this, taken from the test vectors, for instance:

A creator creating a PSBT for a transaction which creates the following outputs:

• scriptPubKey: 0014d85c2b71d0060b09c9886aeb815e50991dda124d, Amount: 1.49990000
• scriptPubKey: 001400aea9a2e5f0f876a588df5546e8742d1d87008f, Amount: 1.00000000

and spends the following inputs:

• TXID: 75ddabb27b8845f5247975c8a5ba7c6f336c4570708ebe230caf6db5217ae858, Index: 0
• TXID: 1dea7cd05979072a3578cab271c02244ea8a090bbb46aa680a65ecd027048d83, Index: 1

must create this PSBT:

• Bytes in Hex:

70736274ff01009a020000000258e87a21b56daf0c23be8e7070456c336f7cbaa5c8757924f545887bb2abdd750000000000ffffffff838d0427d0ec650a68aa46bb0b098aea4422c071b2ca78352a077959d07cea1d0100000000ffffffff0270aaf00800000000160014d85c2b71d0060b09c9886aeb815e50991dda124d00e1f5050000000016001400aea9a2e5f0f876a588df5546e8742d1d87008f000000000000000000

• Base64 String:

cHNidP8BAJoCAAAAAljoeiG1ba8MI76OcHBFbDNvfLqlyHV5JPVFiHuyq911AAAAAAD/////g40EJ9DsZQpoqka7CwmK6kQiwHGyyng1Kgd5WdB86h0BAAAAAP////8CcKrwCAAAAAAWABTYXCtx0AYLCcmIauuBXlCZHdoSTQDh9QUAAAAAFgAUAK6pouXw+HaliN9VRuh0LR2HAI8AAAAAAAAAAAA=

And similar ones are defined for other roles (updater, signer, combiner). The serialization order defined MUST be preserved so, even if it would be totally fine to follow a different one, considering it would then produce hex/base64 string different from those in the BIP, should it be actaully valid?

[altafan]

cc @achow101

[achow101]

@altafan

The serialization order is a recommendation. The test vectors are for software that follow the recommendations. It is difficult to write test vectors that use different ordering while also enforce that the correct fields are added.

[sanket1729]

@achow101, what would be recommended workflow for testing whether the pset/psbt created by elementsd and the one parsed by another implementation are the same?

[achow101]

@sanket1729 Generally it would be to have both implementations output their internal representations and compare that the internal representations match each other. Obviously this cannot be generalized and will need to be specific for each implementation. For example, you could use decodepsbt and check that the data output by the decoding match what is being stored internally by the other implementation (e.g. compare that the hex of a pubkey matches the bytes of supposedly the same pubkey).