tests: fix UTF-8 detection, per-test LC_* settings, CI coverage

- runtests: fix `codeset-utf8` feature detection. Before this patch it
  detected if the calling environment had UTF-8 enabled. If not, UTF-8
  tests were all skipped. After this patch, it detects if UTF-8 is
  supported by the calling environment regardless of what's currently
  enabled.
  Follow-up to 0b70b23ef4 #15039

- GHA/linux: sync `codeset-test` to also reset `LC_CTYPE` and
  `LC_NUMBER`. To give it more spin.
  Follow-up to c221c0ee59 #17938

- GHA/macos: fix to actually enable `codeset-test`. Also set `LC_ALL`,
  which seems necessary to trigger issues.
  Follow-up to c221c0ee59 #17938

- tests/data: replace `LC_CTYPE` env with `LC_ALL` in all tests
  requiring a locale. Also to avoid potential issues with a blank or
  unset `LC_ALL`, as seen earlier. And to ensure that the override works
  on all platforms (as tested in CI.)
  Slight downside is that this now resets the language/culture to `C`.
  Ref: b4c9982382 #4743
  Ref: 23208e330a #4738

- replace `en_US.UTF-8` with `C.UTF-8` to be language/culture-agnostic.

- TEST-SUITE.md: drop `UTF-8` as a requirement for tests.
  Tests shall work (or least be skipped) without UTF-8 support.

Tests requiring UTF-8 locale:
165, 962, 963, 964, 965, 966, 967, 1448, 1560, 2046, 2047
Tests requiring UTF-8 locale, but passing without one anyway:
955, 956, 957, 958, 959, 960, 961, 968, 1034, 1035

Spec 1997: https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html
Spec 2008: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Ref: c221c0ee59 #17938
Ref: 7cf8414fab
Ref: 4c140a5628
Ref: 28faaacee2 #2436
Ref: ecd1d020ab

Closes #17988
This commit is contained in:
Viktor Szakats 2025-07-16 04:25:08 +02:00
parent c37e06c642
commit 1cc8a5235f
No known key found for this signature in database
GPG key ID: B5ABD165E2AEF201
25 changed files with 51 additions and 45 deletions

View file

@ -685,7 +685,12 @@ jobs:
fi
fi
[ -x ~/venv/bin/activate ] && source ~/venv/bin/activate
[[ "${MATRIX_INSTALL_STEPS}" = *'codeset-test'* ]] && export LC_ALL=C
if [[ "${MATRIX_INSTALL_STEPS}" = *'codeset-test'* ]]; then
locale || true
export LC_ALL=C
export LC_CTYPE=C
export LC_NUMERIC=fr_FR.UTF-8
fi
if [ "${MATRIX_BUILD}" = 'cmake' ]; then
cmake --build bld --verbose --target "${TEST_TARGET}"
else