Skip to content

[nasa/nos3#577] CSS-System-Tests#598

Merged
jlucas9 merged 19 commits intodevfrom
nos3#577-css-system-tests
Apr 10, 2025
Merged

[nasa/nos3#577] CSS-System-Tests#598
jlucas9 merged 19 commits intodevfrom
nos3#577-css-system-tests

Conversation

@dacarter22
Copy link
Contributor

@dacarter22 dacarter22 commented Apr 1, 2025

Added System Tests for CSS. Steps to test below:

  1. Pull nos3#577-css-system-tests
  2. build with cfs and cosmos
  3. Launch command telemetry server and test runner.
  4. Wait for 20 seconds and change to generic_css test suite
  5. click start
  6. verify tests pass

This PR closes issue #577

Submodules to be merged into dev:

@codecov-commenter
Copy link

codecov-commenter commented Apr 1, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.88%. Comparing base (1b52bab) to head (13d11d0).
Report is 114 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #598      +/-   ##
==========================================
+ Coverage   72.77%   72.88%   +0.11%     
==========================================
  Files         744      747       +3     
  Lines       91191    91899     +708     
  Branches     6923     6975      +52     
==========================================
+ Hits        66362    66981     +619     
- Misses      24829    24918      +89     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dacarter22 dacarter22 changed the title [nasa/nos3#577] Creating top level branch [nasa/nos3#577] CSS-System-Tests Apr 1, 2025
@Isgaroth-the-Green
Copy link
Contributor

The tests all appear to run largely as they should; I did have one fail the first time through, but that was merely due to one of the numbers not getting low enough quite in time, and running it twice more, every test passed.

@zlynch2 zlynch2 changed the title [nasa/nos3#577] CSS-System-Tests [WIP] [nasa/nos3#577] CSS-System-Tests Apr 1, 2025
@dacarter22 dacarter22 changed the title [WIP] [nasa/nos3#577] CSS-System-Tests [nasa/nos3#577] CSS-System-Tests Apr 8, 2025
@dacarter22 dacarter22 changed the title [nasa/nos3#577] CSS-System-Tests WIP [nasa/nos3#577] CSS-System-Tests Apr 8, 2025
@dacarter22 dacarter22 changed the title WIP [nasa/nos3#577] CSS-System-Tests [WIP] [nasa/nos3#577] CSS-System-Tests Apr 8, 2025
@dacarter22 dacarter22 changed the title [WIP] [nasa/nos3#577] CSS-System-Tests [nasa/nos3#577] CSS-System-Tests Apr 8, 2025
@dacarter22
Copy link
Contributor Author

Increased wait time from 30 to 60 seconds

Copy link
Contributor

@kevincbruce kevincbruce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks ok. When running the test, had an error on the data loop for the AST prep I believe. One of the Voltages was pretty far off (I believe around 400). Though in all other cases, it all passed, so I'm not sure if this is an odd discrepancy, or some oversight. May require some more review.

Update: On subsequent runs, seems to work fine, so leaning towards it being an edge case from running too close to startup or some other factor. Though still, if variation is greater than 5, may be worth looking at increasing the margin and decreasing the wait time.

@Isgaroth-the-Green
Copy link
Contributor

Twice when I have run it, I have gotten a similar (large) error with both the GENERIC_CSS_DATA_TLM_RAW_CSS_3 and GENERIC_CSS_DATA_TLM_RAW_0 (the latter wrong by a value of 800, and the former wrong by 500 or so). They are always wrong together, and when I run a second time the error usually goes away.

@dacarter22
Copy link
Contributor Author

Let me double check with John, I was getting a similar issue but it seems to happen randomly, I'll see if there's a way to restrict the tests to only fire under sunsafe mode

@dacarter22
Copy link
Contributor Author

Okay, I think its a problem of running the tests too soon after launch, I was able to get them all to execute correctly on the first try after waiting 20-30 seconds after I launched telemetry server

@dacarter22
Copy link
Contributor Author

image

@kevincbruce
Copy link
Contributor

That's more or less what I was seeing too, was the first run would have issues, but once I waited a bit it would run fine. May be a consequence of not having an isvalid value for CSS, and thus not knowing if it is in sun enough to get an accurate reading (or whatever validity is checking for, I'm not 100%)

@jlucas9
Copy link
Contributor

jlucas9 commented Apr 9, 2025

I have some updates to the entire PR. Can walk through them tomorrow if desired.

Copy link
Contributor

@jlucas9 jlucas9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve after my commit modifications.

@zlynch2
Copy link
Collaborator

zlynch2 commented Apr 9, 2025

Instead of waiting 20 seconds you can wait for mode 2 sun point which I think is a tlm point from adcs.

@kevincbruce
Copy link
Contributor

Ran new version a few times in a row. Seems to work well, so approval maintained.

@zlynch2
Copy link
Collaborator

zlynch2 commented Apr 10, 2025

Ran new push, seems to work. Approving

@zlynch2 zlynch2 self-requested a review April 10, 2025 12:14
Copy link
Contributor

@Isgaroth-the-Green Isgaroth-the-Green left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good; I am no longer having the same issues as before.

@jlucas9 jlucas9 merged commit 5b7b392 into dev Apr 10, 2025
3 checks passed
@jlucas9 jlucas9 deleted the nos3#577-css-system-tests branch April 10, 2025 19:48
vedantt21 pushed a commit to vedantt21/nos3-mantle that referenced this pull request Feb 18, 2026
* [nasa#577] Changed >= to <= to fix data comparison tests
* [nasa#577] Removed cpt and lpt test files
* [nasa#577] Updates to CSS application to disable I2C and reset on enables, also updated data checks to use SIM_42_TRUTH;
* [nasa#577] Updated generic_css after submodule merge;
---------
Co-authored-by: dacarter22 <davidacarter22>
Co-authored-by: Lucas, John P. <John.P.Lucas@nasa.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

Comments