tests: fix UTF-8 detection, per-test LC_* settings, CI coverage

- runtests: fix `codeset-utf8` feature detection. Before this patch it
  detected if the calling environment had UTF-8 enabled. If not, UTF-8
  tests were all skipped. After this patch, it detects if UTF-8 is
  supported by the calling environment regardless of what's currently
  enabled.
  Follow-up to 0b70b23ef4 #15039

- GHA/linux: sync `codeset-test` to also reset `LC_CTYPE` and
  `LC_NUMBER`. To give it more spin.
  Follow-up to c221c0ee59 #17938

- GHA/macos: fix to actually enable `codeset-test`. Also set `LC_ALL`,
  which seems necessary to trigger issues.
  Follow-up to c221c0ee59 #17938

- tests/data: replace `LC_CTYPE` env with `LC_ALL` in all tests
  requiring a locale. Also to avoid potential issues with a blank or
  unset `LC_ALL`, as seen earlier. And to ensure that the override works
  on all platforms (as tested in CI.)
  Slight downside is that this now resets the language/culture to `C`.
  Ref: b4c9982382 #4743
  Ref: 23208e330a #4738

- replace `en_US.UTF-8` with `C.UTF-8` to be language/culture-agnostic.

- TEST-SUITE.md: drop `UTF-8` as a requirement for tests.
  Tests shall work (or least be skipped) without UTF-8 support.

Tests requiring UTF-8 locale:
165, 962, 963, 964, 965, 966, 967, 1448, 1560, 2046, 2047
Tests requiring UTF-8 locale, but passing without one anyway:
955, 956, 957, 958, 959, 960, 961, 968, 1034, 1035

Spec 1997: https://pubs.opengroup.org/onlinepubs/7908799/xbd/envvar.html
Spec 2008: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Ref: c221c0ee59 #17938
Ref: 7cf8414fab
Ref: 4c140a5628
Ref: 28faaacee2 #2436
Ref: ecd1d020ab

Closes #17988

.github/workflows/linux.yml

@@ -685,7 +685,12 @@ jobs:
             fi
           fi
           [ -x ~/venv/bin/activate ] && source ~/venv/bin/activate
-          [[ "${MATRIX_INSTALL_STEPS}" = *'codeset-test'* ]] && export LC_ALL=C
+          if [[ "${MATRIX_INSTALL_STEPS}" = *'codeset-test'* ]]; then
+            locale || true
+            export LC_ALL=C
+            export LC_CTYPE=C
+            export LC_NUMERIC=fr_FR.UTF-8
+          fi
           if [ "${MATRIX_BUILD}" = 'cmake' ]; then
             cmake --build bld --verbose --target "${TEST_TARGET}"
           else

.github/workflows/macos.yml

@@ -214,6 +214,7 @@ jobs:
       MATRIX_BUILD: ${{ matrix.build.generate && 'cmake' || 'autotools' }}
       MATRIX_COMPILER: '${{ matrix.compiler }}'
       MATRIX_INSTALL: '${{ matrix.build.install }}'
+      MATRIX_INSTALL_STEPS: '${{ matrix.build.install_steps }}'
       MATRIX_MACOS_VERSION_MIN: '${{ matrix.build.macos-version-min }}'
     strategy:
       fail-fast: false
@@ -381,7 +382,6 @@ jobs:
           MATRIX_CHKPREFILL: '${{ matrix.build.chkprefill }}'
           MATRIX_CONFIGURE: '${{ matrix.build.configure }}'
           MATRIX_GENERATE: '${{ matrix.build.generate }}'
-          MATRIX_INSTALL_STEPS: '${{ matrix.build.install_steps }}'
         run: |
           if [[ "${MATRIX_COMPILER}" = 'gcc'* ]]; then
             sysroot="$("${CC}" --print-sysroot)"  # Must match the SDK gcc was built for
@@ -481,6 +481,8 @@ jobs:
           TFLAGS="-j20 ${TFLAGS}"
           source ~/venv/bin/activate
           if [[ "${MATRIX_INSTALL_STEPS}" = *'codeset-test'* ]]; then
+            locale || true
+            export LC_ALL=C
             export LC_CTYPE=C
             export LC_NUMERIC=fr_FR.UTF-8
           fi