Https Huggingface Co Thebloke Llama 2 70b Gptq
Description This repo contains GPTQ model files for Meta Llama 2s Llama 2 70B Multiple GPTQ parameter permutations are provided See Provided Files below for details of the options. Llama 2 70B Instruct v2 Description This repo contains GPTQ model files for Upstages Llama 2 70B Instruct v2. The size of Llama 2 70B fp16 is around 130GB so no you cant run Llama 2 70B fp16 with 2 x 24GB You need 2 x 80GB GPU or 4 x 48GB GPU or 6 x 24GB GPU to run fp16. The GPTQ links for LLaMA-2 are in the wiki. In the section labeled Download custom model or LoRA input TheBlokeLlama-2-70B-chat-GPTQ For downloading from specific branches for instance TheBlokeLlama-2-70B-chat..
Description This repo contains GPTQ model files for Meta Llama 2s Llama 2 70B Multiple GPTQ parameter permutations are provided See Provided Files below for details of the options. Llama 2 70B Instruct v2 Description This repo contains GPTQ model files for Upstages Llama 2 70B Instruct v2. The size of Llama 2 70B fp16 is around 130GB so no you cant run Llama 2 70B fp16 with 2 x 24GB You need 2 x 80GB GPU or 4 x 48GB GPU or 6 x 24GB GPU to run fp16. The GPTQ links for LLaMA-2 are in the wiki. In the section labeled Download custom model or LoRA input TheBlokeLlama-2-70B-chat-GPTQ For downloading from specific branches for instance TheBlokeLlama-2-70B-chat..
Hugging Face
Comments