Skip to content

tsp.ers: building an evidence record is O(n³) in the number of data objects #2456

Description

@dejmekradim

ERSArchiveTimeStampGenerator becomes unusable well before the data-object counts an archiving service reaches: 10,000 objects take about six minutes, and the cost rises by roughly a factor of eight for every doubling.

Measured on bcpkix-jdk18on 1.80 (JDK 21), one ERSByteData per object, timing only generateTimeStampRequest:

data objects generateTimeStampRequest
1,000 0.36 s
2,000 2.5 s
4,000 25 s
10,000 347 s
100,000 ~93 h (extrapolated)

The cause is in SortedHashList.add, which keeps its hashes in a LinkedList and finds the insertion point by walking it with get(index):

private final LinkedList<byte[]> baseList = new LinkedList<byte[]>();
...
int index = 1;
while (index < baseList.size() && hashComp.compare(baseList.get(index), hash) <= 0)
{
    index++;
}

LinkedList.get(i) is O(i), so one add that scans to position i costs O(i²) and n adds cost O(n³). SortedIndexedHashList has the same code and the same behaviour.

Both are on the generation path, one after the other:

  1. ERSArchiveTimeStampGenerator.getPartialHashtrees()ERSUtil.buildIndexedHashListSortedIndexedHashList
  2. BinaryTreeRootCalculator.computeRootHashSortedHashList

Neither can be replaced from outside the library — getPartialHashtrees() is private, so supplying a custom ERSRootNodeCalculator only reaches (2) and leaves the build cubic.

The fix is small: both classes only expose size(), getFirst() and toList(), so the list can be collected unsorted and sorted once in the accessors. Collections.sort is stable, so hashes that compare equal keep the order they were added in — which is where inserting after the last equal element put them — and the output order is therefore unchanged. I have a patch with tests and will open a pull request.

Still present on main (ab16374).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions