Sanitizer exception in regression mode causes bogus exception for subsequent fuzzing

[Marcono1234]

Version

jazzer-junit 0.24.0

Description

It seems when a sanitizer throws an exception during regression mode, that exception is not properly cleared and causes the subsequent fuzzing run to immediately fail with the same exception and also creating a bogus 0-byte input file.

The issue might be in FuzzTestExtensions and the way it handles JazzerInternal.lastFinding: It only clears the lastFinding at the start, but after regression execution it just retrieves the value but does not clear it:

jazzer/src/main/java/com/code_intelligence/jazzer/junit/FuzzTestExtensions.java

Line 115 in c138e8f

Throwable stored = (Throwable) getLastFindingField().get(null);

This most likely then causes the fuzzing run (which happens still in the same JVM) to see that lastFinding and fail immediately, creating a bogus 0-byte input file (bogus because on the next run this does not cause an exception).
At least that is my current assumption; I have not debugged it in detail though.

How to reproduce

Create a test class which triggers a sanitizer exception, for example:

public class RegexTest {
    @FuzzTest
    void test(FuzzedDataProvider d) {
        String input = d.consumeRemainingAsString();
        try {
            Pattern.matches("\\Q" + input + "\\E", "foobar");
        } catch (PatternSyntaxException ignored) {
        }
    }
}

For all the subsequents steps always run with JAZZER_FUZZ=1.

1. Run the test method
   🔍 As expected: It should find (non-empty) input which triggers a sanitzer exception; a new crash-... input file is generated
2. Run the test method again
   🔍 As expected: The previous crash reproducer is still failing
   ❌ Bug: Fuzzing immediately fails and creates a bogus 0-byte crash-... input file

(I performed this within IntelliJ IDE, with JDK 17)

[Marcono1234]

@oetr, if possible could you please have a look at this? Maybe I can try submitting a pull request with the above mentioned change for the FuzzTestExtensions class, but unfortunately I still don't have a proper local development setup for Jazzer, so testing this would be rather cumbersome.
Also, I am not sure if I might be overlooking a case where the JazzerInternal.lastFinding field is not cleared either.

[oetr]

Nice finding, thanks for your report!

• If the last input from regression mode results in a finding, it is not cleared before the fuzzing starts.
• Clearing it solves the issue.

I opened a preliminary PR #944 and might add a few tests before wrapping it up.