ERSArchiveTimeStampGenerator becomes unusable well before the data-object counts an archiving service reaches: 10,000 objects take about six minutes, and the cost rises by roughly a factor of eight for every doubling.
Measured on bcpkix-jdk18on 1.80 (JDK 21), one ERSByteData per object, timing only generateTimeStampRequest:
| data objects |
generateTimeStampRequest |
| 1,000 |
0.36 s |
| 2,000 |
2.5 s |
| 4,000 |
25 s |
| 10,000 |
347 s |
| 100,000 |
~93 h (extrapolated) |
The cause is in SortedHashList.add, which keeps its hashes in a LinkedList and finds the insertion point by walking it with get(index):
private final LinkedList<byte[]> baseList = new LinkedList<byte[]>();
...
int index = 1;
while (index < baseList.size() && hashComp.compare(baseList.get(index), hash) <= 0)
{
index++;
}
LinkedList.get(i) is O(i), so one add that scans to position i costs O(i²) and n adds cost O(n³). SortedIndexedHashList has the same code and the same behaviour.
Both are on the generation path, one after the other:
ERSArchiveTimeStampGenerator.getPartialHashtrees() → ERSUtil.buildIndexedHashList → SortedIndexedHashList
BinaryTreeRootCalculator.computeRootHash → SortedHashList
Neither can be replaced from outside the library — getPartialHashtrees() is private, so supplying a custom ERSRootNodeCalculator only reaches (2) and leaves the build cubic.
The fix is small: both classes only expose size(), getFirst() and toList(), so the list can be collected unsorted and sorted once in the accessors. Collections.sort is stable, so hashes that compare equal keep the order they were added in — which is where inserting after the last equal element put them — and the output order is therefore unchanged. I have a patch with tests and will open a pull request.
Still present on main (ab16374).
ERSArchiveTimeStampGeneratorbecomes unusable well before the data-object counts an archiving service reaches: 10,000 objects take about six minutes, and the cost rises by roughly a factor of eight for every doubling.Measured on bcpkix-jdk18on 1.80 (JDK 21), one
ERSByteDataper object, timing onlygenerateTimeStampRequest:generateTimeStampRequestThe cause is in
SortedHashList.add, which keeps its hashes in aLinkedListand finds the insertion point by walking it withget(index):LinkedList.get(i)is O(i), so oneaddthat scans to position i costs O(i²) and n adds cost O(n³).SortedIndexedHashListhas the same code and the same behaviour.Both are on the generation path, one after the other:
ERSArchiveTimeStampGenerator.getPartialHashtrees()→ERSUtil.buildIndexedHashList→SortedIndexedHashListBinaryTreeRootCalculator.computeRootHash→SortedHashListNeither can be replaced from outside the library —
getPartialHashtrees()is private, so supplying a customERSRootNodeCalculatoronly reaches (2) and leaves the build cubic.The fix is small: both classes only expose
size(),getFirst()andtoList(), so the list can be collected unsorted and sorted once in the accessors.Collections.sortis stable, so hashes that compare equal keep the order they were added in — which is where inserting after the last equal element put them — and the output order is therefore unchanged. I have a patch with tests and will open a pull request.Still present on
main(ab16374).