build failures using clang to build powerpc64le

[mayeut]

git bisect shows the issue was introduced by #4944

reproducer using an ubuntu:24.04 container (doing cross build but the issue also happens in native builds):

apt install lsb-release wget software-properties-common gnupg curl wget
wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
./llvm.sh 21
apt install make gcc gcc-powerpc64le-linux-gnu gfortran-powerpc64le-linux-gnu
ln -s clang-21 /usr/bin/powerpc64le-linux-gnu-clang
git bisect start
git bisect bad v0.3.29
git bisect good v0.3.28
git bisect run bash -c 'make clean && make -j$(nproc) HOSTCC="clang-21" CC="powerpc64le-linux-gnu-clang" FC="powerpc64le-linux-gnu-gfortran" ARCH=power TARGET=POWER8 DYNAMIC_ARCH=1 USE_OPENMP=0'
....
76227e2948b1f847abef003de5f6d49ea0dd3171 is the first bad commit
commit 76227e2948b1f847abef003de5f6d49ea0dd3171
Author: Chip Kerchner <chip.kerchner@ibm.com>
Date:   Fri Sep 6 14:03:31 2024 -0500

    Initial commit for vectorized BF16 GEMV. Added GEMM_GEMV_FORWARD_BF16 to enable using BF16 GEMV for one dimension matrices.  Updated unit test to support inc_x != 1 or inc_y for GEMV.
...

The reported error looks like:

In file included from ../kernel/power/sbgemv_n_vsx.c:31:
../kernel/power/sbgemv_common.c:In file included from 61:18: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
../kernel/power/sbgemv_t_vsx.c   61 | typedef :_31_:
../kernel/power/sbgemv_common.c:61:18: error: type specifier missing, defaults to 'int'; ISO C99 and later vdoe cnott osupportr implicit  IintF L[-Wimplicit-int]OA
T        vec_bf16;
      | ~~~~~~~~~~~~~~~~ ^
      | int
   61 | typedef __vector IFLOAT        ../common.hv:ec_bf13106:;16:
       | note: ~~~~~~~~~~~~~~~~ ^
expanded       from|  macroint 
'IFLOAT'
../common.h:310:16: note: expanded from macro 'IFLOAT'
    310 | #define IFLOAT  bfloat16
      |                 ^
310 | #deIn file included from f../kernel/power/sbgemv_t_vsx.ci:n31e:
 I../kernel/power/sbgemv_common.c:61:18: error: typedef redefinition with different types ('__vector int' (vector of 4 'int' values) vs 'uint16_t' (aka 'unsigned short'))
../common.h:310:16: note: expanded from macro 'IFLOAT'
  310 | #define IFLOAT  bfloat16
      |                 ^
../common.h:265:18: note: previous definition is hereF
LOAT  bfloat16
      |                 ^
  265 | typedef uint16_t bfloat16;
      |                  ^
In file included from ../kernel/power/sbgemv_n_vsx.c:31:
../kernel/power/sbgemv_common.c:61:18: error: typedef redefinition with different types ('__vector int' (vector of 4 'int' values) vs 'uint16_t' (aka 'unsigned short'))
In file included from ../kernel/power/sbgemv_t_vsx.c:../common.h31::
310:../kernel/power/sbgemv_common.c16::61 :24:note:  expanded error: from macroexpected  'IFLOAT'';' 
after top level declarator
   61 | typedef __v  e310c | t#odref iInFeL OIAFTL O A T    b f l ovaetc1_6bf
1      6| ;                ^

      |                        ^
../common.h:      265| :18                       ;:
 note: previous definition is here
  265 | typedef uint16_t bfloat16;
      |                  ^
In file included from ../kernel/power/sbgemv_n_vsx.c:31:
../kernel/power/sbgemv_common.c:61:24: error: expected ';' after top level declarator
   61 | typedef __vector IFLOAT        vec_bf16;
      |                        ^
      |                        ;
powerpc64le-linux-gnu-clang: warning: argument '-Ofast' is deprecated; use '-O3 -ffast-math' for the same behavior, or '-O3' to enable only conforming optimizations [-Wdeprecated-ofast]
powerpc64le-linux-gnu-clang: warning: overriding '-Ofast' option with '-fno-fast-math' [-Woverriding-option]
../kernel/power/sbgemv_common.c:102:13: error: unknown type name 'vec_bf16'
  102 | FORCEINLINE vec_bf16 vec_loadN(void *src, BLASLONG n)
      |             ^
../kernel/power/sbgemv_common.c:102:13: error: unknown type name 'vec_bf16'
  102 | FORCEINLINE vec_bf16 vec_loadN(void *src, BLASLONG n)
      |             ^
../kernel/power/sbgemv_common.c:108:51: error: ../kernel/power/sbgemv_common.c:108:51: error: use of undeclared identifier 'vec_bf16'
use of   108 |   __attribute__((aligned(16))) IFLOAT data[sizeof(vec_bf16) / sizeof(IFLOAT)];
      |                                                   ^~~~~~~~
undeclared identifier 'vec_bf16'
  108 |   __attribute__((aligned(16))) IFLOAT data[sizeof(vec_bf16) / sizeof(IFLOAT)];
      |                                                   ^~~~~~~~
../kernel/power/sbgemv_common.c:109:26: error: use of undeclared identifier 'vec_bf16'
  109 |   memset(data, 0, sizeof(vec_bf16));
      |                          ^~~~~~~~
../kernel/power/sbgemv_common.c:109:26: error: use of undeclared identifier 'vec_bf16'
  109 |   memset(data, 0, sizeof(vec_bf16));
      |                          ^~~~~~~~
../kernel/power/sbgemv_common.c:121:11: error: use of undeclared identifier 'vec_bf16'
  121 |   re../kernel/power/sbgemv_common.c:121:11: error: use of undeclared identifier 'vec_bf16'
turn   121 |   return (vec_bf16)vec_load_vec(data);
      |           ^~~~~~~~
(vec_bf16)vec_load_vec(data);
      |           ^~~~~~~~
../kernel/power/sbgemv_common.c:125:56: error: unknown type name 'vec_bf16'
  125 | FORCEINLINE vec_f32 vec_loadNHi(void *src, BLASLONG n, vec_bf16 zero)
      |                                                        ^
../kernel/power/sbgemv_common.c:125:56: error: unknown type name 'vec_bf16'
  125 | FORCEINLINE vec_f32 vec_loadNHi(void *src, BLASLONG n, vec_bf16 zero)
      |                                                        ^
../kernel/power/sbgemv_common.c:127:3: error: use of undeclared identifier 'vec_bf16'
  127 |   vec_bf16 data = vec_loadN(src, n);
      |   ^~~~~~~~
../kernel/power/sbgemv_common.c:128:18: error: use of undeclared identifier 'data'
  128../kernel/power/sbgemv_common.c | :  return BF16_HI(data, zero);
      |                  ^~~~
127:3: error: use of undeclared identifier 'vec_bf16'
  127 |   vec_bf16 data = vec_loadN(src, n);
      |   ^~~~~~~~
../kernel/power/sbgemv_common.c:128:18: error: use of undeclared identifier 'data'
  128 |   return BF16_HI(data, zero);
      |                  ^~~~
../kernel/power/sbgemv_common.c:163:44: error: unknown type name 'vec_bf16'
  163 | FORCEINLINE vec_f32 vec_mult(vec_f32 *inp, vec_bf16 in0, vec_bf16 zero)
      |                                            ^
../kernel/power/sbgemv_common.c:163:44: error: unknown type name 'vec_bf16'
  163 | FORCEINLINE vec_f32 vec_mult(vec_f32 *inp, vec_bf16 in0, vec_bf16 zero)
      |                                            ^
../kernel/power/sbgemv_common.c:163:58: error: unknown type name 'vec_bf16'
  163 | FORCEINLINE vec_f32 vec_mult(vec_f32 *inp, vec_bf16 in0, vec_bf16 zero)
      |                                                          ^
../kernel/power/sbgemv_common.c:163:58: error: unknown type name 'vec_bf16'
  163 | FORCEINLINE vec_f32 vec_mult(vec_f32 *inp, vec_bf16 in0, vec_bf16 zero)
      |                                                          ^
../kernel/power/sbgemv_common.c:171:35: error: unknown type name 'vec_bf16'
  171 | FORCEINLINE vec_f32 vec_load_mult(vec_bf16 *in, vec_f32 *inp, vec_bf16 zero)
      |                                   ^
../kernel/power/sbgemv_common.c:171:35: error: unknown type name 'vec_bf16'
  171 | FORCEINLINE vec_f32 vec_load_mult(vec_bf16 *in, vec_f32 *inp, vec_bf16 zero)
      |                                   ^
../kernel/power/sbgemv_common.c:171:63: error: unknown type name 'vec_bf16'
  171 | FORCEINLINE vec_f32 vec_load_mult(vec_bf16 *in, vec_f32 *inp, vec_bf16 zero)
      |                                                               ^
../kernel/power/sbgemv_common.c:171:63: error: unknown type name 'vec_bf16'
  171 | FORCEINLINE vec_f32 vec_load_mult(vec_bf16 *in, vec_f32 *inp, vec_bf16 zero)
      |                                                               ^
../kernel/power/sbgemv_common.c:173:3: error: use of undeclared identifier 'vec_bf16'
  173 |   vec_bf16 in0 = (vec_bf16)vec_load_vec(in);
      |   ^~~~~~~~
../kernel/power/sbgemv_common.c:175:24: error: use of undeclared identifier 'in0'
  175 |   return vec_mult(inp, in0, zero);
      |                        ^~~
../kernel/power/sbgemv_common.c:173:3: error: use of undeclared identifier 'vec_bf16'
  173 |   vec_bf16 in0 = (vec_bf16)vec_load_vec(in);
      |   ^~~~~~~~
../kernel/power/sbgemv_common.c:175:24: error: use of undeclared identifier 'in0'
  175 |   return vec_mult(inp, in0, zero);
../kernel/power/sbgemv_common.c      :| 178:                       ^~~32
: error: unknown type name 'vec_bf16'
  178 | FORCEINLINE void vec_load_vec2(vec_bf16 *in, BLASLONG i, vec_f32 *v_x0, vec_bf16 zero)
      |                                ^
../kernel/power/sbgemv_common.c:178:32:../kernel/power/sbgemv_common.c :178:error: 73: unknown typeerror:  name unknown'vec_bf16' type
 name 'vec_bf16'
  178   | 178F | OFROCRECIENILNILNIEN E voviodid v evce_cl_olaoda_d_vevce2c(2v(evce_cb_fb1f61 6* i*ni,n ,B LBALSALSOLNOGN Gi ,i ,v evce_cf_3f23 2* v*_vx_0x,0 ,v evce_cb_fb1f61 6z ezreor)o)
      
|       |                                                                         ^
                               ^
../kernel/power/sbgemv_common.c../kernel/power/sbgemv_common.c::180178::373::  error: error: useunknown  oftype  undeclaredname  identifier'vec_bf16' 'vec_bf16'

  180   | 178  |  FvOeRcC_EbIfN1L6I NiEn p v=o i(dve cv_ebcf_1l6o)avde_cv_elco2a(dv_evce_cb(f&1i6n [*ii]n),; B
L      A| SL  ^~~~~~~~O
NG i, vec_f32 *v_x0, vec_bf16 zero)
      |                                                                         ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
../kernel/power/sbgemv_common.c:180:3: error: use of undeclared identifier 'vec_bf16'
  180 |   vec_bf16 inp = (vec_bf16)vec_load_vec(&in[i]);
      |   ^~~~~~~~
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
20 errors generated.
make[1]: *** [Makefile.L2:513: sbgemv_n_POWER8.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile.L2:515: sbgemv_t_POWER8.o] Error 1
make[1]: Leaving directory '/openblas/kernel'
make: *** [Makefile:203: libs] Error 1

[martin-frbg]

Probably needs something like

diff --git a/kernel/power/gemm_common.c b/kernel/power/gemm_common.c
index adfbb85c2..0aaeddf0b 100644
--- a/kernel/power/gemm_common.c
+++ b/kernel/power/gemm_common.c
@@ -28,7 +28,7 @@
 #define USE_VECTOR_PAIRS
 #endif
 
-#ifdef _AIX
+#if defined(_AIX) || defined(__clang__)
 #include<stdbool.h>
 typedef __vector unsigned short vec_bf16;
 #else

but I haven't found a ppc machine with LLVM newer than 19 in the GCC Compile Farm and building my own will take some time.

[mayeut]

I came to the same conclusion for a fix but can't easily test (using QEMU for ppc64le).
No need for clang-21, I just got a failed build with the same error using the Ubuntu packaged clang in osuosl/ubuntu-ppc64le:18.04 used by Jenkinsfile.pwr so clang 19 should reproduce as well.
I'm not sure CI still runs with Jenkins for ppc64le ? recent reports only mention 1 Jenkins feedback for OpenBLAS-Z.

[martin-frbg]

CI still runs (with occasional hickups) but both ppc and ibmz end up in the same slot in the github overview and one needs to be logged in to the OSUOSL Jenkins service (with my account :-) ) to access the logs - a bit messy but better than not having these in CI at all.

[martin-frbg]

BTW just realized there is https://github.com/IBM/llvm-project providing AIX and RH binaries of LLVM releases

[mayeut]

clang 6 on ubuntu 18.04 still fails to build after the fix for other errors but not worth fixing IMHO.
clang 10 on osuosl/ubuntu-ppc64le:20.04 failed before the fix and builds successfully after.
clang 18 on osuosl/ubuntu-ppc64le:24.04 failed before the fix and builds successfully after.

build command used in both cases: make CC=clang TARGET=POWER8

seeing there's already a PR for this fix, should I open one adding a clang build in Jenkins using osuosl/ubuntu-ppc64le:20.04 ?

[mayeut]

BTW just realized there is https://github.com/IBM/llvm-project providing AIX and RH binaries of LLVM releases

FYI I first ran into this as part of MacPython/openblas-libs#238, using https://github.com/mayeut/static-clang-images which can be installed on any linux