Test suite issue on i386 and PowerPC 32-bit for version 1.0.2

[tillea]

Hi,
when upgrading the Debian package of estimatr from version 1.0.0 to 1.0.2 a test suite error occurs on i386 architecture as you can see in the according test log. It contains

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component “estimate”: 'is.NA' value mismatch: 0 in current 1 in target
Component “std.error”: 'is.NA' value mismatch: 0 in current 1 in target
Component “statistic”: 'is.NA' value mismatch: 0 in current 1 in target
Component “p.value”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.low”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.high”: 'is.NA' value mismatch: 0 in current 1 in target
Component “df”: 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component “estimate”: 'is.NA' value mismatch: 0 in current 1 in target
Component “std.error”: 'is.NA' value mismatch: 0 in current 1 in target
Component “statistic”: 'is.NA' value mismatch: 0 in current 1 in target
Component “p.value”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.low”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.high”: 'is.NA' value mismatch: 0 in current 1 in target
Component “df”: 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component “estimate”: 'is.NA' value mismatch: 0 in current 1 in target
Component “std.error”: 'is.NA' value mismatch: 0 in current 1 in target
Component “statistic”: 'is.NA' value mismatch: 0 in current 1 in target
Component “p.value”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.low”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.high”: 'is.NA' value mismatch: 0 in current 1 in target
Component “df”: 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:622:3'): Handle perfect fits appropriately ───
tidy(rfo)[rfo$term == "X", ] not equivalent to tidy(ro)[ro$term == "X", ].
Component “std.error”: 'is.NA' value mismatch: 1 in current 0 in target
Component “statistic”: 'is.NA' value mismatch: 1 in current 0 in target
Component “p.value”: 'is.NA' value mismatch: 1 in current 0 in target
Component “conf.low”: 'is.NA' value mismatch: 1 in current 0 in target
Component “conf.high”: 'is.NA' value mismatch: 1 in current 0 in target

[ FAIL 4 | WARN 188 | SKIP 11 | PASS 930 ]
Error: Test failures

It would be great if you could have a look.
Kind regards, Andreas.

[barracuda156]

Just got the same issue on PowerPC 32-bit:

R version 4.3.3 (2024-02-29) -- "Angel Food Cake"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(testthat)
> library(estimatr)
> 
> test_check("estimatr")
[ FAIL 4 | WARN 0 | SKIP 5 | PASS 1270 ]

══ Skipped tests (5) ═══════════════════════════════════════════════════════════
• On CRAN (3): 'test-iv-robust.R:235:3', 'test-lm-cluster.R:182:3',
  'test-lm-cluster.R:213:3'
• ivpack cannot be loaded (2): 'test-iv-robust.R:30:3',
  'test-iv-robust.R:328:3'

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component "estimate": 'is.NA' value mismatch: 0 in current 1 in target
Component "std.error": 'is.NA' value mismatch: 0 in current 1 in target
Component "statistic": 'is.NA' value mismatch: 0 in current 1 in target
Component "p.value": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.low": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.high": 'is.NA' value mismatch: 0 in current 1 in target
Component "df": 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component "estimate": 'is.NA' value mismatch: 0 in current 1 in target
Component "std.error": 'is.NA' value mismatch: 0 in current 1 in target
Component "statistic": 'is.NA' value mismatch: 0 in current 1 in target
Component "p.value": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.low": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.high": 'is.NA' value mismatch: 0 in current 1 in target
Component "df": 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component "estimate": 'is.NA' value mismatch: 0 in current 1 in target
Component "std.error": 'is.NA' value mismatch: 0 in current 1 in target
Component "statistic": 'is.NA' value mismatch: 0 in current 1 in target
Component "p.value": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.low": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.high": 'is.NA' value mismatch: 0 in current 1 in target
Component "df": 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:622:3'): Handle perfect fits appropriately ───
tidy(rfo)[rfo$term == "X", ] not equivalent to tidy(ro)[ro$term == "X", ].
Component "std.error": 'is.NA' value mismatch: 1 in current 0 in target
Component "statistic": 'is.NA' value mismatch: 1 in current 0 in target
Component "p.value": 'is.NA' value mismatch: 1 in current 0 in target
Component "conf.low": 'is.NA' value mismatch: 1 in current 0 in target
Component "conf.high": 'is.NA' value mismatch: 1 in current 0 in target

[ FAIL 4 | WARN 0 | SKIP 5 | PASS 1270 ]
Error: Test failures
Execution halted

[tillea]

Any idea how to fix the test suite?

[barracuda156]

Any idea how to fix the test suite?

Probably expect_equal should have tolerance being set. Something like in kevhuy/WALS@5c8c57d

P. S. Perhaps the issue is rather be renamed to 32-bit instead of i386.

[tillea]

Probably expect_equal should have tolerance being set. Something like in kevhuy/WALS@5c8c57d

Probably, but I did not wanted to do this without confirmation by upstream.

P. S. Perhaps the issue is rather be renamed to 32-bit instead of i386.

Not really since the testsuite is passing on armel and armhf. I've changed the issue title to what we really know.

[mr-c]

These tests still fail for i386 in version 1.0.4; also for riscv64 (which is not surprising, NA is not supported on riscv64)

[barracuda156]

I cannot verify on PowerPC at the moment, though given results on i386, I would assume ppc will fail likewise. I can check in a while once back to my hardware.