buildsystem: Remove buildtest Goal & Fix info-buildsizes-diff

[crasbe]

Contribution description

The buildtests goal was an early version used for compiling all boards.

There is still a reference in the documentation that has to be changed, but I'm not familiar with make generate-makefiles.ci, so input in this regard is very welcome.

RIOT/doc/doxygen/src/advanced-build-system-tricks.md

Lines 178 to 196 in fb7acf5

	Comparing Build Sizes {#comparing-build-sizes}
	=====================
	There is a make target for build size comparison. You can use it like that:
	~~~~~~~~~~~~~~~~~~~
	$ cd RIOT/test/test_something
	$ git checkout master
	$ BINDIRBASE=master-bin make buildtest
	$ git checkout my-branch
	$ BINDIRBASE=my-branch-bin make buildtest
	$ OLDBIN=master-bin NEWBIN=my-branch-bin make info-buildsizes-diff
	text data bss dec BOARD/BINDIRBASE
	0 0 0 0 avsextrem **← this line contains the diff**
	57356 1532 96769 155657 master-bin
	57356 1532 96769 155657 my-branch-bin

Likewise in dist/tools/compile_test/compile_test.py:

RIOT/dist/tools/compile_test/compile_test.py

Lines 127 to 130 in fb7acf5

	subprocess = Popen((MAKE, 'buildtest'),
	bufsize=1, stdin=null, stdout=PIPE, stderr=null,
	cwd=app_dir,
	env=subprocess_env)

Testing procedure

Make sure your favorite board and favorite application still build.

Make sure that no references to buildtest or buildtests remain, for example with grep -Rnw . -e buildtest.

Issues/PRs references

Fixes #9742.
Alternative approach for #21355.

[crasbe]

The example for info-buildsizes-diff should be changed IMO. Building everything with the amount of boards we have (and the amount of different platforms) will take FOREVER.

Furthermore, info-buildsizes-diff itself is incredibly slow. My proposal would be to check the boards that are actually present in the two folders and just comparing those.

Also, it will show ERR when the build size is equal, which is not really a good experience. There is no comparison to equal to 0.

RIOT/makefiles/info-global.inc.mk

Lines 129 to 130 in fb7acf5

	if [[ "$${DIFF}" -gt 0 ]]; then $(COLOR_ECHO) -n "$(COLOR_RED)"; fi; \
	if [[ "$${DIFF}" -lt 0 ]]; then $(COLOR_ECHO) -n "$(COLOR_GREEN)"; fi; \

---

The only place where dist/tools/compile_test/compile_test.py is mentioned is here:

RIOT/doc/doxygen/src/getting-started.md

Lines 318 to 331 in fb7acf5

	### Downloading and testing RIOT docker container
	Finally, download the pre-built RIOT Docker container:
	```console
	# docker pull riot/riotbuild
	```
	This will take a while. If it finishes correctly, you can then use the toolchains contained in the Docker container:
	(**from the riot root**):
	```console
	$ docker run --rm -i -t -u $UID -v $(pwd):/data/riotbuild riot/riotbuild ./dist/tools/compile_test/compile_test.py
	```

But compile_test.py will build EVERYTHING for ALL boards, which takes forever on a normal machine.
Even generate-Makefile.ci crashed my WSL because it exceeded 15GB of RAM...

Is that used for something else? Or is it even useful to begin with?
The compile_like_murdock.py script in the same folder is used by the GitHub workflows:

RIOT/.github/workflows/tools-test.yml

Lines 41 to 42 in fb7acf5

	- name: Test compile_like_murdock script
	run: cd dist/tools/compile_test/tests && ./test.sh

[maribu]

Thx for cleaning this up!

[maribu]

Ah, the good old MSB-A2 still is the "default" board in our doc :D

[maribu]

Even generate-Makefile.ci crashed my WSL because it exceeded 15GB of RAM...

And 14.2 GiB of that were the riot/riotbuild docker image :/

[crasbe]

It turns out that info-buildsizes-diff does not even work in current master because of a similar issue as described in the other PR. The make system gives the MAKEFLAGS to the submake, which defines BINDIR, which is already defined to native64, since no BOARD is given for the original call of info-buildsizes-diff.

I changed the behavior of info-buildsizes-diff to only show the boards that are actually present in the NEWBIN and OLDBIN and it also compiles a list of boards that don't have a complementary partner in the other directory.
Furthermore if there is no size difference, the number will be printed yellow. Because why not.

Now info-buildsizes-diff is actually useful again. Contrary to info-buildsizes, which I don't think has any real uses tbh.
But I could also add it back in with the same change that I did to info-buildsizes-diff to only evaluate the boards acutally present in the folder.

The documentation still has to be adapted. done.

[crasbe]

Okay so the dist/tools/ci/build_and_test.sh script became obsolete in #15667, when it was replaced with the static tests as we know them today. This was essentially the last script that called RIOT/dist/tools/compile_test/compile_test.py.

The documentation in

RIOT/doc/doxygen/src/getting-started.md

Lines 318 to 331 in fb7acf5

	### Downloading and testing RIOT docker container
	Finally, download the pre-built RIOT Docker container:
	```console
	# docker pull riot/riotbuild
	```
	This will take a while. If it finishes correctly, you can then use the toolchains contained in the Docker container:
	(**from the riot root**):
	```console
	$ docker run --rm -i -t -u $UID -v $(pwd):/data/riotbuild riot/riotbuild ./dist/tools/compile_test/compile_test.py
	```

has to be updated as well, because docker pull riot/riotdocker does not work anymore.

The riotbuild/Dockerfile also mentions this script, but does not execute it:
https://github.com/RIOT-OS/riotdocker/blob/004fdc18820f576267a332def5075b58fbdefc88/riotbuild/Dockerfile#L19-L21

[crasbe]

I removed this from the merge queue for now, so someone can take a second look at it again, now that two more scripts are removed and the documentation is updated.

[crasbe]

The makefiles/info-nproc.inc.mk script was only used by the buildtest target and the info-concurrency target was not defined as a global target, therefore not accessible anyway.

Without buildtest, this script is not needed anymore either.

[riot-ci]

Murdock results

✔️ PASSED

7543be5 buildsystem: Remove unused scripts and update docker docs

Success	Failures	Total	Runtime
10282	0	10282	09m:47s

Artifacts

• Documentation preview

[mguetschow]

@crasbe with the riotdocker CI still using buildtest would compile_like_murdock.py be the right tool to switch to? RIOT-OS/riotdocker#256 (comment)

[crasbe]

I thought I researched this more thoroughly, but apparently that got missed. Sorry for the mess that it caused.

To answer the question: yes, that should be a valid substitution. However one thing that still has to be checked it how the Github workflow evaluates whether the run was successful or not. As far as I can tell, the Python script always returns success?