Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #3930 - Splits tests to Tests/UnitTests, Tests/IntegrationTests, Tests/StressTests #3954

Merged
merged 110 commits into from
Mar 6, 2025

Conversation

tig
Copy link
Collaborator

@tig tig commented Mar 2, 2025

Fixes

Proposed Changes/Todos

  • Remove dead tests
  • Split tests to Tests/UnitTests, Tests/IntegrationTests, Tests/StressTests
  • Add Directory.Build.props to unify settings
  • Add Direcotry.Packages.props to unify package versions
  • Add a script for repeatedly running tests on Ubuntu locally to see if the failure can be reproduced
  • Create new ./Tests/UnitTestsParallelizable project and move obviously parallelizable tests.
  • Keep trying shit until tests work reliably

Pull Request checklist:

  • I've named my PR in the form of "Fixes #issue. Terse description."
  • My code follows the style guidelines of Terminal.Gui - if you use Visual Studio, hit CTRL-K-D to automatically reformat your files before committing.
  • My code follows the Terminal.Gui library design guidelines
  • I ran dotnet test before commit
  • I have made corresponding changes to the API documentation (using /// style comments)
  • My changes generate no new warnings
  • I have checked my code and corrected any poor grammar or misspellings
  • I conducted basic QA to assure all features are working

tig added 25 commits February 19, 2025 13:15
@tig tig changed the title Fixes #3952 - Test failures Fixes #3930 - Test failures Mar 2, 2025
@tig
Copy link
Collaborator Author

tig commented Mar 2, 2025

Another result (after tyring to fix stuff):

  Passed Terminal.Gui.ViewTests.HasFocusChangeEventTests.HasFocusChanged_With_SubView_Raises [< 1 ms]
  Passed Terminal.Gui.ViewTests.HasFocusChangeEventTests.HasFocusChanging_RemoveFocus_Raises [< 1 ms]
Error opening terminal: unknown.
The active test run was aborted. Reason: Test host process crashed : Error opening terminal: unknown.

Data collector 'Blame' message: System.IO.FileNotFoundException: Collect dump was enabled but no dump file was generated.
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.ProcessDumpUtility.GetDumpFiles(Boolean warnOnNoDumpFiles, Boolean processCrashed) in /_/src/Microsoft.TestPlatform.Extensions.BlameDataCollector/ProcessDumpUtility.cs:line 84
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.BlameCollector.SessionEndedHandler(Object sender, SessionEndEventArgs args) in /_/src/Microsoft.TestPlatform.Extensions.BlameDataCollector/BlameCollector.cs:line 534.

Test Run Aborted.
Total tests: Unknown
     Passed: 791
    Skipped: 10
 Total time: 41.9252 Seconds

The active Test Run was aborted because the host process exited unexpectedly. Please inspect the call stack above, if available, to get more information about where the exception originated from.
The test running when the crash occurred: 
Terminal.Gui.ApplicationTests.ApplicationScreenTests.ClearScreenNextIteration_Resets_To_False_After_LayoutAndDraw

@tig
Copy link
Collaborator Author

tig commented Mar 5, 2025

Sadly, the weird crash is still happening even after all this work:

 Passed Terminal.Gui.ConfigurationTests.ColorJsonConverterTests.DeserializesFromRgb(rgb: "\"rgb(0,0,0)\"", r: 0, g: 0, b: 0) [< 1 ms]
  Passed Terminal.Gui.ConfigurationTests.ColorJsonConverterTests.TestSerializeColor_BrightRed [< 1 ms]
Error opening terminal: unknown.
The active test run was aborted. Reason: Test host process crashed : Error opening terminal: unknown.

Data collector 'Blame' message: System.IO.FileNotFoundException: Collect dump was enabled but no dump file was generated.
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.ProcessDumpUtility.GetDumpFiles(Boolean warnOnNoDumpFiles, Boolean processCrashed) in /_/src/Microsoft.TestPlatform.Extensions.BlameDataCollector/ProcessDumpUtility.cs:line 84
   at Microsoft.TestPlatform.Extensions.BlameDataCollector.BlameCollector.SessionEndedHandler(Object sender, SessionEndEventArgs args) in /_/src/Microsoft.TestPlatform.Extensions.BlameDataCollector/BlameCollector.cs:line 534.

Test Run Aborted.
Total tests: Unknown
     Passed: 1451
 Total time: 34.4460 Seconds

The active Test Run was aborted because the host process exited unexpectedly. Please inspect the call stack above, if available, to get more information about where the exception originated from.
The test running when the crash occurred: 
Terminal.Gui.ApplicationTests.ApplicationScreenTests.ClearScreenNextIteration_Resets_To_False_After_LayoutAndDraw

Highly suspicious that ClearScreenNextIteration_Resets_To_False_After_LayoutAndDraw seems to be the culprit. I've studied the test and code and cannot see what the issue is. Some sort of race condition.

I sure wish I could get the dump to be created

@BDisp
Copy link
Collaborator

BDisp commented Mar 5, 2025

Only UnitTests project do a build release. Should the others test projects also needed to do a build release?

@tig
Copy link
Collaborator Author

tig commented Mar 5, 2025

Only UnitTests project do a build release. Should the others test projects also needed to do a build release?

I think I'll break that out to a separate action.

@BDisp
Copy link
Collaborator

BDisp commented Mar 5, 2025

Error opening terminal: unknown.

This error normally is associated to a code that want running on a terminal but in the Github Actions environments don't really have a real terminal. In these cases it's needed to identify what unit test fail first and doing a stressing test with the fail test and with others together. Normally it's needed to set ConsoleDriver.RunningUnitTests to true in the unit test and add a condition on the driver, if doesn't exist yet, to not run it if it's true.

@BDisp
Copy link
Collaborator

BDisp commented Mar 5, 2025

I think I'll break that out to a separate action.

I agree. It only need to build once for all the projects.

@BDisp
Copy link
Collaborator

BDisp commented Mar 5, 2025

The ClearScreenNextIteration_Resets_To_False_After_LayoutAndDraw unit test should set ConsoleDriver.RunningUnitTests = true; because it's using the driver of the current OS which could run code that needed to have a real terminal. In these cases an error will be throw.

@tig tig merged commit b0f3281 into gui-cs:v2_develop Mar 6, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tests are crashing again... Enable unit tests to run in parallel
2 participants