Integrate llama.cpp in Aprapipes and a module ImageToTextXForm which can describe an image#345
Open
kushaljain-apra wants to merge 25 commits intomainfrom
Open
Integrate llama.cpp in Aprapipes and a module ImageToTextXForm which can describe an image#345kushaljain-apra wants to merge 25 commits intomainfrom
kushaljain-apra wants to merge 25 commits intomainfrom
Conversation
yashrajsapra
requested changes
Apr 18, 2024
kushaljain-apra
changed the title
joiskash
requested changes
Oct 16, 2024
Collaborator
joiskash
left a comment
joiskash
left a comment
There was a problem hiding this comment.
I have left a few comments please address them, thanks!
* Introduced comprehensive documentation for the LLM framework design in ApraPipes, detailing architectural decisions, core design principles, and future extensions. * Added a guide for integrating llama.cpp and whisper.cpp using the vcpkg package manager, covering portfile structure, build configuration, and platform-specific handling. * Enhanced clarity on model management, pipeline integration, and resource management strategies within the framework.
* Introduced a new document detailing the integration issues between whisper.cpp and llama.cpp due to conflicting GGML library versions. * Described the root causes, reproduction steps, error symptoms, and current workarounds. * Outlined long-term solutions and the impact on projects requiring both libraries. * Provided links to related issues and repositories for further reference.
* Introduced a new section detailing available models for the LLM framework, starting with LLaVA-1.6-Mistral-7B. * Included model descriptions, supported use cases, and various model variants with their specifications. * Provided guidance on selecting model variants based on hardware capabilities and precision requirements.
* Introduced new SVG and PNG files for the LLM Framework, providing visual representations of the architecture and components. * Added an Excalidraw file for detailed design elements and layout. * Enhanced documentation with visual aids to improve understanding of the framework's structure and functionality.
…tants * Updated ClipEncoderProps to include a numThreads parameter for improved threading control. * Modified serialization methods to account for the new numThreads property. * Introduced default constants for model configuration in ModelStrategy, enhancing clarity and maintainability. * Adjusted ClipEncoder implementation to utilize the new threading parameter during inference. * Added validation in ImageToTextXForm to prevent changes to GPU layers post-initialization.
…praPipes into ks/llama_integration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
IMPORTANT: All PRs must be linked to an issue (except for extremely trivial and straightforward changes).
Fixes #[Issue]
Description
Alternative(s) considered
Have you considered any alternatives? And if so, why have you chosen the approach in this PR?
Type
Type Choose one: (Feature)
Screenshots (if applicable)
Checklist