-
Notifications
You must be signed in to change notification settings - Fork 0
Update llama cpp python 0.3.16 #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Update version from 0.2.24 to 0.3.16 - Update SHA256 hash - Update llama.cpp dependency from 0.0.1660 to 0.0.6653 - Add new dependency: jinja2 >=2.11.3 - Add pyyaml >=5.1 (server dependency) - Reset build number to 0
2f5e0a3 to
9980451
Compare
cbouss
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you build the corresponding llama.cpp version in the llama.cpp feedstock instead?
Otherwise we'll end up having a non optimized llama-cpp-python.
The llama.cpp feedstock package is not used due to an API incompatibility issue (llama.cpp feedstock is too new while llama-cpp-python is still old): llama.cpp b6872 removed the llama_get_kv_self() function that llama-cpp-python 0.3.16 depends on. This causes compilation/runtime failures when trying to link against the newer llama.cpp. To use our latest llama.cpp, need to wait llama-cpp-python release a new version compatible. |
You can build an older version of llama.cpp on the llama.cpp feedstock. The vendored one matches b6173. |
98b27f0 to
0581df6
Compare
cbouss
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can simplify the recipe quite a bit by removing any compiler related portion.
No build occurs, the python code loads llama.cpp at runtime. (I made suggestion but haven't tried a local build, apologies if I made a mistake there.)
You also don't need to do manual builds for cuda, since you don't have a cuda compiler.
However looking at the cuda logs, I see that the solver picked the non cuda variant of llama.cpp. llama.cpp 0.0.6188 needs a rebuild because the cuda variant has the same build number than the cpu variant. Instead, a higher build number should be used to tell the solver to prefer the cuda variant. (Same goes for the metal variant.)
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Failed to submit graph for 3a68363Message: Failed to run command: 'pbp generate from-feedstock --output /tmp/tmp270sxn26/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmp270sxn26/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-3a683637ffee6d75aa33471e47a84da2a2f262f7 llama-cpp-python-feedstock/recipe', with error: 'INFO:root:Found feedstock llama-cpp-python-feedstock at /tmp/tmp270sxn26/llama-cpp-python-feedstock/recipe INFO:root:Build/Test graph from llama-cpp-python-feedstock Created by sbx_user1051 on 2025-12-17 18:22:46 Command line: /opt/conda/envs/webhook-listener/bin/pbp generate from-feedstock --output /tmp/tmp270sxn26/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmp270sxn26/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-3a683637ffee6d75aa33471e47a84da2a2f262f7 llama-cpp-python-feedstock/recipeINFO:root:Creating graph from feedstock at: llama-cpp-python-feedstock During handling of the above exception, another exception occurred: Traceback (most recent call last): |
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Co-authored-by: Charles Bousseau <[email protected]>
Failed to submit graph for 4366976Message: Failed to run command: 'pbp generate from-feedstock --output /tmp/tmp7riuw1jh/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmp7riuw1jh/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-4366976b18d4fdf6d81e71e2214a2d0e7026cd26 llama-cpp-python-feedstock/recipe', with error: 'INFO:root:Found feedstock llama-cpp-python-feedstock at /tmp/tmp7riuw1jh/llama-cpp-python-feedstock/recipe INFO:root:Build/Test graph from llama-cpp-python-feedstock Created by sbx_user1051 on 2025-12-17 18:25:24 Command line: /opt/conda/envs/webhook-listener/bin/pbp generate from-feedstock --output /tmp/tmp7riuw1jh/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmp7riuw1jh/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-4366976b18d4fdf6d81e71e2214a2d0e7026cd26 llama-cpp-python-feedstock/recipeINFO:root:Creating graph from feedstock at: llama-cpp-python-feedstock During handling of the above exception, another exception occurred: Traceback (most recent call last): |
Failed to submit graph for 66d81c0Message: Failed to run command: 'pbp generate from-feedstock --output /tmp/tmpal4711ai/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmpal4711ai/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-66d81c031ae850b2524eddbb46d8fa663f9b7e35 llama-cpp-python-feedstock/recipe', with error: 'INFO:root:Found feedstock llama-cpp-python-feedstock at /tmp/tmpal4711ai/llama-cpp-python-feedstock/recipe INFO:root:Build/Test graph from llama-cpp-python-feedstock Created by sbx_user1051 on 2025-12-17 18:25:29 Command line: /opt/conda/envs/webhook-listener/bin/pbp generate from-feedstock --output /tmp/tmpal4711ai/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmpal4711ai/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-66d81c031ae850b2524eddbb46d8fa663f9b7e35 llama-cpp-python-feedstock/recipeINFO:root:Creating graph from feedstock at: llama-cpp-python-feedstock During handling of the above exception, another exception occurred: Traceback (most recent call last): |
Failed to submit graph for be75f42Message: Failed to run command: 'pbp generate from-feedstock --output /tmp/tmpkix3yi1d/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmpkix3yi1d/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-be75f42b2e5213e659a9ec49196f6c932ca4ffb3 llama-cpp-python-feedstock/recipe', with error: 'INFO:root:Found feedstock llama-cpp-python-feedstock at /tmp/tmpkix3yi1d/llama-cpp-python-feedstock/recipe INFO:root:Build/Test graph from llama-cpp-python-feedstock Created by sbx_user1051 on 2025-12-17 18:25:40 Command line: /opt/conda/envs/webhook-listener/bin/pbp generate from-feedstock --output /tmp/tmpkix3yi1d/llama-cpp-python-feedstock.yaml --custom-metadata-file /tmp/tmpkix3yi1d/custom_metadata.yaml --release-name llama-cpp-python-feedstock-1-be75f42b2e5213e659a9ec49196f6c932ca4ffb3 llama-cpp-python-feedstock/recipeINFO:root:Creating graph from feedstock at: llama-cpp-python-feedstock During handling of the above exception, another exception occurred: Traceback (most recent call last): |
llama-cpp-python 0.3.16
Destination channel: defaults
Links
Explanation of changes:
Version Update:
External llama.cpp Dependency:
Patches:
Build Script:
Dependency Note:
CUDA Variants