Improve installation process

[abetlen]

Open to suggestions / assistance on how to make installation easier and less error prone.

One thought is to add better platform detection to the cmakelists and provide better docs / links if required environment variables aren't set / libraries can't be found.

[ElliottDyson]

GPU detection to decide on what llama.cpp build to use? For example, if Arc is detected, but no Nvidia nor AMD card, then use the build that actively supports it.

As for the current process it has to be built anew each time, which is particularly a pain for those that struggle with this process with the various specific compilers that have to be used for OneAPI.

[vriesdemichael]

Wouldn't llama.cpp be the logical place to implement these improvements?

One thing improvement that would logically fall in the responsibility of llama-cpp-python is using the prebuilt llama.cpp when using windows, but other than that it would just be another layer of cmake

[kristaller486]

Prebuilt wheels with GPU support for all platforms (on Github or PyPI). According to my observations, installing llama-cpp-python with GPU support is the most popular problem when installing llama-cpp-python, prebuilt packages should fix it.

[abetlen]

Prebuilt wheels with GPU support for all platforms (on Github or PyPI). According to my observations, installing llama-cpp-python with GPU support is the most popular problem when installing llama-cpp-python, prebuilt packages should fix it.

I'm partial to this, PyPI is a little annoying because we would need different package names for each but if we did it using seperate indexes (similar to pytorch) this should work. Ideally this would be done via seperate index URLs for metal, CUDA, etc. Maybe can be done with a GitHub pages run on each release.

Update : Started working on this and it's very much do-able, the process is straightforward and much of the hard work to figure out how to build these wheels has thankfully been done by @jllllll and @oobabooga

1. On new releases we can build wheels for each target (trying to keep number to a minimum).
2. Generate an index html file for each target
3. Deploy to github pages

Basic example of this in https://github.com/abetlen/github-pages-pypi-index

As an example, to install the latest llama-cpp-python version:

pip install llama-cpp-python --extra-index-url https://abetlen.github.io/github-pages-pypi-index/whl/cpu

In the future this will likely be found at https://abetlen.github.io/llama-cpp-python/whl/cpu

[DvitryG]

will the installation without wheels be supported? I just tried to update the package after a long break and got an error: Getting requirements to build wheel did not run successfully.

This is the first time I've come across wheels, so I want to ask if performance will deteriorate from using pre-build libraries? I don't have the most powerful computer, so it's important for me to make the most of it.

[abetlen]

@DvitryG yes source installation will always be the default (pip install llama-cpp-python etc) and it should offer the most control / performance.

[waheedi]

Thank you for making the effort to create these bindings, it would have been a nightmare ;D

One important improvement, specifically if building from a cloned source repo, is to keep the newly built wheel in place, maybe somewhere like dist/llama_cpp_python-0.2.76-cp310-cp310-linux_x86_64.whl

Currently there is no wheel you can grab in your hand but rather its directly installed into packages

It seems the setup.py way is going obsolete.. But it still can be used independently.

I just made it to work with llama for now :)

import os
import subprocess
from setuptools import setup, find_packages
from setuptools.command.build_ext import build_ext

class CMakeBuild(build_ext):
    def run(self):
        # Ensure CMake is installed
        try:
            subprocess.check_call(['cmake', '--version'])
        except OSError:
            raise RuntimeError("CMake must be installed to build the following extensions: " +
                               ", ".join(e.name for e in self.extensions))

        # Directory where the current setup.py is located
        source_dir = os.path.abspath(os.path.dirname(__file__))
        build_temp = self.build_temp

        # Create the build directory if it doesn't exist
        if not os.path.exists(build_temp):
            os.makedirs(build_temp)

        # Define the CMake arguments
        cmake_args = [
            '-D', 'LLAMA_HIPBLAS=ON',
            '-D', 'CMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang',
            '-D', 'CMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++',
            '-D', 'CMAKE_PREFIX_PATH=/opt/rocm',
            source_dir  # Add the source directory
        ]

        # Change to the build directory and run CMake
        subprocess.check_call(['cmake'] + cmake_args, cwd=build_temp)
        subprocess.check_call(['make', '-j8'], cwd=build_temp)

        # Ensure the shared library is moved to the expected location

#build/temp.linux-x86_64-cpython-310/libllama.so

        build_lib = os.path.abspath(self.build_lib)
        lib_path = os.path.join(build_temp, 'vendor/llama.cpp/libllama.so')
        target_path = os.path.join(source_dir, 'llama_cpp', 'libllama.so')
        self.copy_file(lib_path, target_path)

setup(
    name='llama_cpp_python',
    version='0.2.76',  # Adjust the version as necessary
    author='abetlen',
    description='Python bindings for llama.cpp',
    long_description=open('README.md').read(),
    long_description_content_type='text/markdown',
    url='https://github.com/abetlen/llama-cpp-python',
    packages=find_packages(),
    include_package_data=True,
    install_requires=[
        'numpy',
    ],
    package_data={
        'llama_cpp': ['libllama.so'],
    },
    classifiers=[
        'Programming Language :: Python :: 3',
        'License :: OSI Approved :: MIT License',
        'Operating System :: OS Independent',
    ],
    python_requires='>=3.6',
    cmdclass={
        'build_ext': CMakeBuild,
    },
)

[neviaumi]

I recently try install llama-cpp-python on my project in X86 macbook pro.

The simple way would be

brew install llama.cpp
pdm add llama-cpp-python -Ccmake.args="-DGGML_BLAS=ON;-DGGML_BLAS_VENDOR=OpenBLAS" 

Without brew install first, it would keep complain Ninja build error

(experimental-lllama.cpp-3.12) ➜  experimental-lllama.cpp git:(main) ✗ tail -n 256 /Users/davidng/Library/Logs/pdm/pdm-install-7d8v9uww.log                           
unearth.evaluator: Skipping package Package(name='packaging', version='16.2'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.5'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.3'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='16.1'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.4'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.3'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='16.1'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.4'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='16.0'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.3'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='16.0'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.3'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.3'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.3'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.2'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.3.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.2'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.2.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.1'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.2.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.1'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.2.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.0'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.2.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='15.0'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.2.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.5'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.2.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.5'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.6'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.4'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.6'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.4'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.5'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.3'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.5'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.3'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.4'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.2'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.4'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.2'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.3'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.1'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.3'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.1'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.0'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='14.0'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='20.6'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.4.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='packaging', version='20.6'): version doesn't match ==24.1
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.4.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0'): version doesn't match ==0.10.7
pdm.termui: Fetching hashes for packaging@24.1
pdm.termui: Fetching hashes for scikit-build-core@0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0rc1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b2'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0b0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a1'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.1.0a0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.4.0'): version doesn't match ==0.10.7
unearth.evaluator: Skipping package Package(name='scikit-build-core', version='0.4.0'): version doesn't match ==0.10.7
pdm.termui: Fetching hashes for scikit-build-core@0.10.7
pdm.termui: Synchronization complete.
pdm.termui: *** scikit-build-core 0.10.7 using CMake 3.30.5 (wheel)
pdm.termui: *** Configuring CMake...
pdm.termui: loading initial cache file /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/CMakeInit.txt
pdm.termui: -- The C compiler identification is AppleClang 12.0.0.12000026
pdm.termui: -- The CXX compiler identification is AppleClang 12.0.0.12000026
pdm.termui: -- Detecting C compiler ABI info
pdm.termui: -- Detecting C compiler ABI info - done
pdm.termui: -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/clang - skipped
pdm.termui: -- Detecting C compile features
pdm.termui: -- Detecting C compile features - done
pdm.termui: -- Detecting CXX compiler ABI info
pdm.termui: -- Detecting CXX compiler ABI info - done
pdm.termui: -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/clang++ - skipped
pdm.termui: -- Detecting CXX compile features
pdm.termui: -- Detecting CXX compile features - done
pdm.termui: -- Found Git: /usr/local/bin/git (found version "2.40.0")
pdm.termui: -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
pdm.termui: -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
pdm.termui: -- Found Threads: TRUE
pdm.termui: -- Accelerate framework found
pdm.termui: -- Metal framework found
pdm.termui: -- The ASM compiler identification is AppleClang
pdm.termui: -- Found assembler: /Library/Developer/CommandLineTools/usr/bin/clang
pdm.termui: -- Found OpenMP_C: -Xclang -fopenmp (found version "4.5")
pdm.termui: -- Found OpenMP_CXX: -Xclang -fopenmp (found version "4.5")
pdm.termui: -- Found OpenMP: TRUE (found version "4.5")
pdm.termui: -- OpenMP found
pdm.termui: -- Could NOT find BLAS (missing: BLAS_LIBRARIES) 
pdm.termui: CMake Warning at vendor/llama.cpp/ggml/src/CMakeLists.txt:253 (message):
pdm.termui:   BLAS not found, please refer to
pdm.termui:   https://cmake.org/cmake/help/latest/module/FindBLAS.html#blas-lapack-vendors
pdm.termui:   to set correct GGML_BLAS_VENDOR
pdm.termui: 
pdm.termui: 
pdm.termui: -- Using llamafile
pdm.termui: -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
pdm.termui: -- CMAKE_SYSTEM_PROCESSOR: x86_64
pdm.termui: -- x86 detected
pdm.termui: CMake Warning (dev) at CMakeLists.txt:9 (install):
pdm.termui:   Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
pdm.termui: Call Stack (most recent call first):
pdm.termui:   CMakeLists.txt:73 (llama_cpp_python_install_target)
pdm.termui: This warning is for project developers.  Use -Wno-dev to suppress it.
pdm.termui: 
pdm.termui: CMake Warning (dev) at CMakeLists.txt:17 (install):
pdm.termui:   Target llama has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
pdm.termui: Call Stack (most recent call first):
pdm.termui:   CMakeLists.txt:73 (llama_cpp_python_install_target)
pdm.termui: This warning is for project developers.  Use -Wno-dev to suppress it.
pdm.termui: 
pdm.termui: CMake Warning (dev) at CMakeLists.txt:9 (install):
pdm.termui:   Target ggml has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
pdm.termui: Call Stack (most recent call first):
pdm.termui:   CMakeLists.txt:74 (llama_cpp_python_install_target)
pdm.termui: This warning is for project developers.  Use -Wno-dev to suppress it.
pdm.termui: 
pdm.termui: CMake Warning (dev) at CMakeLists.txt:17 (install):
pdm.termui:   Target ggml has PUBLIC_HEADER files but no PUBLIC_HEADER DESTINATION.
pdm.termui: Call Stack (most recent call first):
pdm.termui:   CMakeLists.txt:74 (llama_cpp_python_install_target)
pdm.termui: This warning is for project developers.  Use -Wno-dev to suppress it.
pdm.termui: 
pdm.termui: -- Configuring done (2.6s)
pdm.termui: -- Generating done (0.1s)
pdm.termui: -- Build files have been written to: /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build
pdm.termui: *** Building project with Ninja...
pdm.termui: Change Dir: '/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build'
pdm.termui: 
pdm.termui: Run Build Command(s): /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-env-rt5uund2-overlay/lib/python3.12/site-packages/ninja/data/bin/ninja -v
pdm.termui: [1/36] cd /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/vendor/llama.cpp/ggml/src && echo Embedding\ Metal\ library && sed -e '/#include "ggml-common.h"/r /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-common.h' -e '/#include "ggml-common.h"/d' < /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-metal.metal > /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.metal && echo .section\ __DATA,__ggml_metallib > /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.s && echo .globl\ _ggml_metallib_start >> /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.s && echo _ggml_metallib_start: >> /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.s && echo .incbin\ \"/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.metal\" >> /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.s && echo .globl\ _ggml_metallib_end >> /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.s && echo _ggml_metallib_end: >> /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.s
pdm.termui: Embedding Metal library
pdm.termui: [2/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Accelerate.framework -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Foundation.framework -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Metal.framework -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/MetalKit.framework -isystem /usr/local/include -O3 -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/__/__/__/__/autogenerated/ggml-metal-embed.s.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/__/__/__/__/autogenerated/ggml-metal-embed.s.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/__/__/__/__/autogenerated/ggml-metal-embed.s.o -c /var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/tmpyodc81uo/build/autogenerated/ggml-metal-embed.s
pdm.termui: [3/36] cd /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp && /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-env-rt5uund2-overlay/lib/python3.12/site-packages/cmake/data/bin/cmake -DMSVC= -DCMAKE_C_COMPILER_VERSION=12.0.0.12000026 -DCMAKE_C_COMPILER_ID=AppleClang -DCMAKE_VS_PLATFORM_NAME= -DCMAKE_C_COMPILER=/Library/Developer/CommandLineTools/usr/bin/clang -P /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/common/cmake/build-info-gen-cpp.cmake
pdm.termui: -- Found Git: /usr/local/bin/git (found version "2.40.0")
pdm.termui: [4/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-alloc.c.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-alloc.c.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-alloc.c.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-alloc.c
pdm.termui: [5/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-aarch64.c.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-aarch64.c.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-aarch64.c.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-aarch64.c
pdm.termui: [6/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-metal.m.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-metal.m.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-metal.m.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-metal.m
pdm.termui: FAILED: vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-metal.m.o 
pdm.termui: /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-metal.m.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-metal.m.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-metal.m.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-metal.m
pdm.termui: /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-metal.m:449:69: error: use of undeclared identifier 'MTLGPUFamilyApple7'
pdm.termui:     ctx->support_simdgroup_reduction  = [ctx->device supportsFamily:MTLGPUFamilyApple7];
pdm.termui:                                                                     ^
pdm.termui: /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-metal.m:452:61: error: use of undeclared identifier 'MTLGPUFamilyApple7'
pdm.termui:     ctx->support_simdgroup_mm = [ctx->device supportsFamily:MTLGPUFamilyApple7];
pdm.termui:                                                             ^
pdm.termui: /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-metal.m:1765:57: error: use of undeclared identifier 'MTLGPUFamilyApple7'
pdm.termui:                         if ([ctx->device supportsFamily:MTLGPUFamilyApple7] &&
pdm.termui:                                                         ^
pdm.termui: /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-metal.m:2071:57: error: use of undeclared identifier 'MTLGPUFamilyApple7'
pdm.termui:                         if ([ctx->device supportsFamily:MTLGPUFamilyApple7] &&
pdm.termui:                                                         ^
pdm.termui: 4 errors generated.
pdm.termui: [7/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-backend.c.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-backend.c.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-backend.c.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-backend.c
pdm.termui: [8/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wmissing-declarations -Wmissing-noreturn -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/llamafile/sgemm.cpp.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/llamafile/sgemm.cpp.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/llamafile/sgemm.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/llamafile/sgemm.cpp
pdm.termui: [9/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-grammar.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-grammar.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-grammar.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/llama-grammar.cpp
pdm.termui: [10/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-sampling.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-sampling.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-sampling.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/llama-sampling.cpp
pdm.termui: [11/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-vocab.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-vocab.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-vocab.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/llama-vocab.cpp
pdm.termui: [12/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-quants.c
pdm.termui: [13/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/unicode.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/unicode.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/unicode.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/unicode.cpp
pdm.termui: [14/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml.c.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml.c.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml.c.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml.c
pdm.termui: [15/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/llama.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/llama.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/llama.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/llama.cpp
pdm.termui: ninja: build stopped: subcommand failed.
pdm.termui: 
pdm.termui: 
pdm.termui: *** CMake build failed
pdm.termui: Error occurs adding llama-cpp-python: 
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 29, in install_candidate
    self.manager.install(can)
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/installers/manager.py", line 33, in install
    prepared.build(),
    ^^^^^^^^^^^^^^^^
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/models/candidates.py", line 411, in build
    self._cached = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/base.py", line 84, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/wheel.py", line 26, in build
    filename = self._hook.build_wheel(out_dir, self.config_settings, metadata_directory)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 209, in build_wheel
    return self._call_hook('build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 311, in _call_hook
    self._subprocess_runner(
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/base.py", line 281, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/builders/base.py", line 128, in log_subprocessor
    raise build_error(e) from None
pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines of the build output:
[10/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-sampling.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-sampling.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-sampling.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/llama-sampling.cpp
[11/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-vocab.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-vocab.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/llama-vocab.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/llama-vocab.cpp
[12/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml-quants.c
[13/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/unicode.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/unicode.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/unicode.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/unicode.cpp
[14/36] /Library/Developer/CommandLineTools/usr/bin/clang -DACCELERATE_LAPACK_ILP64 -DACCELERATE_NEW_LAPACK -DGGML_BUILD -DGGML_METAL_EMBED_LIBRARY -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -DGGML_USE_ACCELERATE -DGGML_USE_LLAMAFILE -DGGML_USE_METAL -DGGML_USE_OPENMP -D_DARWIN_C_SOURCE -D_XOPEN_SOURCE=600 -Dggml_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/. -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk/System/Library/Frameworks -isystem /usr/local/include -O3 -DNDEBUG -std=gnu11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -march=native -Xclang -fopenmp -MD -MT vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml.c.o -MF vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml.c.o.d -o vendor/llama.cpp/ggml/src/CMakeFiles/ggml.dir/ggml.c.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/ggml.c
[15/36] /Library/Developer/CommandLineTools/usr/bin/clang++ -DGGML_USE_METAL -DLLAMA_BUILD -DLLAMA_SHARED -Dllama_EXPORTS -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/. -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/../include -I/private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/ggml/src/../include -O3 -DNDEBUG -std=gnu++11 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -fPIC -MD -MT vendor/llama.cpp/src/CMakeFiles/llama.dir/llama.cpp.o -MF vendor/llama.cpp/src/CMakeFiles/llama.dir/llama.cpp.o.d -o vendor/llama.cpp/src/CMakeFiles/llama.dir/llama.cpp.o -c /private/var/folders/4s/18hpf60j7h388m2d1j2pw2wc0000gn/T/pdm-build-enqn7f1i/vendor/llama.cpp/src/llama.cpp
ninja: build stopped: subcommand failed.


*** CMake build failed
pdm.termui: Error occurs
Traceback (most recent call last):
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/termui.py", line 260, in logging
    yield logger
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/cli/actions.py", line 301, in do_sync
    synchronizer.synchronize()
  File "/Users/davidng/Library/Application Support/pdm/venv/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 226, in synchronize
    raise InstallationError("Some package operations failed.")
pdm.exceptions.InstallationError: Some package operations failed.

I believe it was missing some package on my Mac as one of message show pdm.termui: -- Could NOT find BLAS (missing: BLAS_LIBRARIES)

[andreamoro]

It's not clear whether this is still an open issue. I only recently decided to give llama.cpp a go as I always though performance were nearly the same for ollama. I couldn't be more wrong.

Regardless, if this might open a conversational point, I believe there should be an independent package doing some machine introspection, completing a config file to be passed to the builder.
An independent process would allow user's fine tuning if any is needed.