Fix: Execute setup and teardown in clails test

[tamurashingo]

Summary

Fixed an issue where suite-level setup and teardown were not executed when running tests with the clails test command.

Problem

When executing the clails test command, the following issues occurred:

• (setup ...) defined in test packages was not executed
• (teardown ...) defined in test packages was not executed
• As a result, initialization before tests and cleanup after tests were not performed, causing tests to fail

Cause

The run-suite-tests function in src/test/runner.lisp was passing a list of test symbols directly to rove:run-tests.

In rove, suite-level setup/teardown are executed within the run-suite-tests method, but clails was calling rove:run-tests directly, so setup/teardown were not executed.

Changes

1. src/test/runner.lisp

Added Helper Function

Added the run-tests-with-setup-teardown function to execute the following process:

(defun run-tests-with-setup-teardown (pkg pkg-tests style)
  "Run tests with suite setup and teardown.
   ..."
  (let ((suite (find-suite pkg))
        (result nil)
        (setup-failed nil))
    (unwind-protect
        (progn
          ;; Execute setup
          (when suite
            (handler-case
                (let ((setup-fn (rove/core/suite/package:suite-setup suite)))
                  (when setup-fn
                    (funcall setup-fn)))
              (error (e)
                (format t "~%ERROR in setup for package '~A': ~A~%" pkg e)
                (format t "Skipping tests for this package.~%")
                (setf setup-failed t))))

          ;; Run tests only if setup succeeded
          (unless setup-failed
            (setf result (run-tests pkg-tests :style style))))

      ;; Execute teardown (ensured by unwind-protect)
      (when suite
        (handler-case
            (let ((teardown-fn (rove/core/suite/package:suite-teardown suite)))
              (when teardown-fn
                (funcall teardown-fn)))
          (error (e)
            (format t "~%ERROR in teardown for package '~A': ~A~%" pkg e)))))

    ;; Return nil if setup failed, otherwise return test result
    (if setup-failed
        nil
        result)))

Error Handling

• When an error occurs in setup:

  • Display error message
  • Skip tests for that package
  • Execute teardown (guaranteed by unwind-protect)
  • Return nil as result (test failure)

• When an error occurs in teardown:

  • Display error message
  • Return test execution result as-is (success if tests passed, failure if tests failed)

Application Points

Replaced rove:run-tests with run-tests-with-setup-teardown at 3 locations in the run-suite-tests function:

1. Line 154: when packages are specified
2. Line 179: when tags are specified
3. Line 206: when running all tests

Added Import

Added find-suite to the imports from the rove package.

2. test/e2e/templates/todo-model-test.lisp

Added setup/teardown test cases to e2e tests for verification:

(defparameter *setup-called* nil)

(setup
  (setf *setup-called* t)
  (format t "~%>>> SETUP CALLED~%"))

(teardown
  (format t "~%>>TEARDOWN CALLED~%"))

(deftest-suite :model setup-test
  (ok *setup-called* "setup should be called"))

Testing

• Confirmed that setup/teardown are executed correctly in e2e tests
• Verified behavior when an error occurs in setup
• Verified behavior when an error occurs in teardown

Impact Scope

Target for Modification

• run-suite-tests function in src/test/runner.lisp

Affected Users

• Application developers using clails
• If setup/teardown are used in application tests, they will now be executed correctly with this fix

Related Issue

fix #141