LinuxCNC and musl C library (possible?)

[NTULINUX]

Here are the steps I follow to reproduce the issue:

1. Use a Clang 19 system with musl
2. Download and attempt to build LinuxCNC
3. Wait for errors

This is what I expected to happen:

A proper compilation.

This is what happened instead:

A few errors, maybe an easy fix?

It worked properly before this:

Clang 16 and glibc built fine when I last tested this.

Information about my hardware and software:

• I am using Gentoo

• I am using this kernel version (shown by uname -a): Linux ntu-gentoo 6.12.0-rc7-BUILD SMP Mon Nov 11 10:20:31 CST 2024 x86_64 AMD Ryzen 7 2700X Eight-Core Processor AuthenticAMD GNU/Linux

• I am running ...

  • A binary version from linuxcnc.org (including buildbot.linuxcnc.org)
  • A binary I built myself
  • A binary version from some other source besides linuxcnc.org

• I am using this LinuxCNC version (shown in package manager or, for git versions, scripts/get-version-from-git): latest 2.9 git checkout

[NTULINUX]

Compiling libnml/buffer/tcpmem.cc
libnml/buffer/tcpmem.cc:108:46: error: use of undeclared identifier 'u_short'; did you mean 'short'?
  108 |     server_socket_address.sin_port = htons(((u_short) tcp_port_number));
      |                                              ^~~~~~~
      |                                              short


Compiling rtapi/uspace_rtapi_app.cc
rtapi/uspace_rtapi_app.cc:351:14: warning: variable length arrays in C++ are a Clang extension [-Wvla-cxx-extension]
  351 |     char buf[len];
      |              ^~~
rtapi/uspace_rtapi_app.cc:351:14: note: read of non-const variable 'len' is not allowed in a constant expression
rtapi/uspace_rtapi_app.cc:350:9: note: declared here
  350 |     int len = read_number(fd);
      |         ^
rtapi/uspace_rtapi_app.cc:699:18: error: use of undeclared identifier 'M_TRIM_THRESHOLD'
  699 |     if (!mallopt(M_TRIM_THRESHOLD, -1)) {
      |                  ^
rtapi/uspace_rtapi_app.cc:704:18: error: use of undeclared identifier 'M_MMAP_MAX'
  704 |     if (!mallopt(M_MMAP_MAX, 0)) {
      |                  ^
rtapi/uspace_rtapi_app.cc:1053:21: error: use of undeclared identifier 'pthread_attr_setaffinity_np'; did you mean 'pthread_setaffinity_np'?
 1053 |           if((ret = pthread_attr_setaffinity_np(&attr, sizeof(cpuset), &cpuset)) != 0)
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                     pthread_setaffinity_np
/usr/include/pthread.h:221:5: note: 'pthread_setaffinity_np' declared here
  221 | int pthread_setaffinity_np(pthread_t, size_t, const struct cpu_set_t *);
      |     ^
rtapi/uspace_rtapi_app.cc:1053:49: error: cannot initialize a parameter of type 'pthread_t' (aka 'unsigned long') with an rvalue of type 'pthread_attr_t *'
 1053 |           if((ret = pthread_attr_setaffinity_np(&attr, sizeof(cpuset), &cpuset)) != 0)
      |                                                 ^~~~~
/usr/include/pthread.h:221:37: note: passing argument to parameter here
  221 | int pthread_setaffinity_np(pthread_t, size_t, const struct cpu_set_t *);
      |                                     ^


Compiling hal/user_comps/xhc-whb04b-6/usb.cc
In file included from hal/user_comps/xhc-whb04b-6/usb.cc:20:
hal/user_comps/xhc-whb04b-6/./usb.h:269:20: error: field has incomplete type 'struct timeval'
  269 |     struct timeval mLastWakeupTimestamp;
      |                    ^
hal/user_comps/xhc-whb04b-6/./usb.h:269:12: note: forward declaration of 'XhcWhb04b6::timeval'
  269 |     struct timeval mLastWakeupTimestamp;
      |            ^
hal/user_comps/xhc-whb04b-6/usb.cc:503:40: error: variable has incomplete type 'struct timeval'
  503 |                         struct timeval now;
      |                                        ^
hal/user_comps/xhc-whb04b-6/./usb.h:269:12: note: forward declaration of 'XhcWhb04b6::timeval'
  269 |     struct timeval mLastWakeupTimestamp;
      |            ^
hal/user_comps/xhc-whb04b-6/usb.cc:517:44: error: variable has incomplete type 'struct timeval'
  517 |                             struct timeval now;
      |                                            ^
hal/user_comps/xhc-whb04b-6/./usb.h:269:12: note: forward declaration of 'XhcWhb04b6::timeval'
  269 |     struct timeval mLastWakeupTimestamp;
      |            ^


Compiling emc/task/backtrace.cc
emc/task/backtrace.cc:43:9: error: cannot assign to variable 'stderr' with const-qualified type 'FILE *const' (aka '_IO_FILE *const')
   43 |         stderr = freopen(filename, "a", stderr);
      |         ~~~~~~ ^
/usr/include/stdio.h:64:20: note: variable 'stderr' declared const here
   64 | extern FILE *const stderr;
      | ~~~~~~~~~~~~~~~~~~~^~~~~~


Compiling emc/rs274ngc/interp_o_word.cc
emc/rs274ngc/interp_o_word.cc:548:36: error: use of undeclared identifier 'basename'
  548 |     logOword("Entered:%s %s", name,basename(block->o_name));
      |                                    ^
emc/rs274ngc/interp_o_word.cc:549:36: error: use of undeclared identifier 'basename'
  549 |     it = settings->offset_map.find(basename(block->o_name));
      |                                    ^
emc/rs274ngc/interp_o_word.cc:612:28: error: use of undeclared identifier 'basename'
  612 |     settings->skipping_o = basename(block->o_name); // start skipping
      |                            ^
emc/rs274ngc/interp_o_word.cc:613:33: error: use of undeclared identifier 'basename'
  613 |     settings->skipping_to_sub = basename(block->o_name); // start skipping
      |                                 ^

[NTULINUX]

For the pthread_attr_setaffinity_np error, this commit from AWS might offer some insight:

https://github.com/awslabs/aws-c-common/pull/755/files

[BsAtHome]

struct timeval is declared in sys/time.h. It looks like it is not included. Maybe it gets included indirectly with glibc and not with musl. Try to add #include <sys/time.h> at the top of hal/user_comps/xhc-whb04b-6/usb.h where the other includes are.

[NTULINUX]

Getting closer! That solved that! Thanks!

[NTULINUX]

Shining light on basename error:

https://bugzilla.mozilla.org/attachment.cgi?id=8516179&action=diff

bluez/bluez@e2f1211

[NTULINUX]

Just squashed libnml/buffer/tcpmem.cc by adding:

#include <sys/types.h>

To the file.

WOO!!

[BsAtHome]

The declaration of "extern FILE *const stderr;" is not standard, but it is not wrong.
From the man stderr(3):
Since the symbols stdin, stdout, and stderr are specified to be macros, assigning to them is nonportable. The standard streams can be made to refer to different files with help of the library function freopen(3), specially introduced to make it possible to reassign stdin, stdout, and stderr.
(emphasis mine)

I suggest to change/fix the code to use freopen(3), without reassigning to stderr.

[edit]The return value may be used to check for failure, but the reassignment is done implicitly by freopen.[edit]

[NTULINUX]

I suggest to change/fix the code to use freopen(3), without reassigning to stderr.

Now that I don't know how to properly do.

[BsAtHome]

change the lines (src/emc/task/backtrace.cc:43):
fclose(stderr);
stderr = freopen(filename, "a", stderr);
into
freopen(filename, "a", stderr);

Description of the change: There is no need to close "old" stderr before reassigning it. The close operation is implicit in the call to freopen (man page says: The original stream (if it exists) is closed.). The assignment to stderr is the problem that freopen is supposed to solve^(*). So, just use freopen and drop the assignment.

^(*) There is some irony in the original code. The assignment to stderr and friends is the exactly the problem that freopen is supposed to solve and then you see a piece of code using freopen and assigning to stderr... Gotta love it. ;-)

[NTULINUX]

Thanks, I'm not much of a coder! I'll try it in a bit!

[NTULINUX]

Things are looking REALLY good! Only two files left; emc/rs274ngc/interp_o_word.cc and rtapi/uspace_rtapi_app.cc

I made a musl branch with the respective changes:

NTULINUX@50946b9

As for M_TRIM_THRESHOLD and M_MMAP_MAX, I spoke to a few people in the musl IRC channel and someone didn't see why 32MB of pre-faulted memory would be any different with or without mmap. It could not be doing anything on glibc builds either, i.e. changing the trim threshold modifies when free() returns memory to the OS, but as this is a real-time application, it pre-allocates and faults in all it's memory. As a result, from the perspective of the allocator, the program doesn't allocate anything as it's only done once on init so M_TRIM_THRESHOLD and M_MMAP_MAX could be doing nothing.

[BsAtHome]

The mmap problem comes into existence when you allocate large blocks. The allocator may choose to use mmap instead of sbrk. This has been explicitly disabled by setting M_MMAP_MAX to zero. I agree that the problem might be academic, but there may be many corner cases.
The same goes for the trim threshold. Once the mlockall is called requesting future pages locked, then that should prevent reducing the heap because reducing sbrk would fail. However, that failure may cause malloc and friends to fail, depending the stupidity/cleverness of the allocator. Again, corner cases may exist.

For musl, it may not apply for their allocator. That can only be answered by investigating the allocator code. For this case, it may be necessary to add a configure test to check mallopt() and the existense of the mmap and threshold flags. Then it can be coded in a preprocessor conditional.

[smoe]

Nice!
OpenWRT is another distribution with musl as their default C library IIRC.

[NTULINUX]

Yes it is!

[heatd]

The mmap problem comes into existence when you allocate large blocks. The allocator may choose to use mmap instead of sbrk. This has been explicitly disabled by setting M_MMAP_MAX to zero. I agree that the problem might be academic, but there may be many corner cases.

What's the problem here, exactly? mmap on Linux behaves the same as brk.

The same goes for the trim threshold. Once the mlockall is called requesting future pages locked, then that should prevent reducing the heap because reducing sbrk would fail.

How? munmap and brk reduction don't care about regions being locked. "The memory lock on an address range is automatically removed if
the address range is unmapped via munmap(2)."

However, that failure may cause malloc and friends to fail, depending the stupidity/cleverness of the allocator. Again, corner cases may exist.

For musl, it may not apply for their allocator. That can only be answered by investigating the allocator code. For this case, it may be necessary to add a configure test to check mallopt() and the existense of the mmap and threshold flags. Then it can be coded in a preprocessor conditional.

FWIW musl's malloc always uses mmap.