Prepare
- Download llama2 gguf format model
- https://huggingface.co/TheBloke/Llama-2-7B-GGUF
- and put it into “./models/llama-2–7b/”
ls -lht models/llama-2-7b
total 7997112
-rw-r--r--@ 1 user staff 3.8G Jan 4 13:12 llama-2-7b.Q4_K_M.gguf
- 2. Build the llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make -j
- 3. Prepare the