build: tidy up and simplify setmode() detection and use

- move macro to `curl_setup.h` (from curlx), and rename.
  It's required by src, test servers, libtests. Also used by unit/tunit,
  (which is fixable but this patch doesn't touch it.)
- special-case it for Windows/Cygwin/MS-DOS.
- build: drop `setmode()`/`_setmode()` detection.
  This also avoids detecting the different `setmode()` on BSDs,
  and a lot of complexity and overhead.
- use `CURL_O_BINARY`.

Follow-up to 250d613763 #15787
Follow-up to 5e70566094 #15169

Closes #20539

lib/Makefile.inc

@@ -45,7 +45,6 @@ LIB_CURLX_CFILES =      \
 
 LIB_CURLX_HFILES =      \
   curlx/base64.h        \
-  curlx/binmode.h       \
   curlx/basename.h      \
   curlx/curlx.h         \
   curlx/dynbuf.h        \