Install GPTJ on WSL

  1. open cmd as administrator
  2. wsl –install
  3. reboot
  4. wsl
  5. install CUDA toolkit
  6. wget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh
  7. ./Anaconda3-2022.10-Linux-x86_64.sh
  8. Reboot entire machine AGAIN
  9. mkdir gptj
  10. cd gptj
  11. conda create -n gptj python=3.8
  12. conda activate gptj
  13. conda install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia
  14. pip uninstall -y transformers && pip install --no-cache-dir https://github.com/deniskamazur/transformers/archive/gpt-j-8bit.zip
  15. pip install bitsandbytes-cuda111
  16. pip install datasets==1.16.1
  17. pip install torch==1.11.0+cu115 torchvision==0.12.0+cu115 -f https://download.pytorch.org/whl/torch_stable.html
import torch
import transformers
from transformers.models.gptj import GPTJForCausalLM

device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = transformers.AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6b")
gpt = GPTJForCausalLM.from_pretrained(
    "hivemind/gpt-j-6B-8bit", low_cpu_mem_usage=True
).to(device)
raw_text = open("prompts/delandzombie.txt", "r").read()
text = raw_text
prompt = tokenizer((raw_text), return_tensors="pt")
prompt = {key: value.to(device) for key, value in prompt.items()}
out = gpt.generate(
    **prompt,
    do_sample=True,
    temperature=1.03,
    top_k=500,
    top_p=0.98,
    max_new_tokens=200,
)
out = tokenizer.decode(out[0])
text = out
print(
    "\n",
    "\n",
    str(text),
    "\n",
    "\n",
    end="",
)
raw_text += text
output = open("out.txt", "a")
output.write(
    str(text)
    + "\n"
    + "\n"
    + "------"
    + "\n"
    + "\n"
)
output.close()

0 Replies to “Install GPTJ on WSL”

  1. Great walkthrough on installing GPTJ on WSL, thoroughly enjoyed your step by step explanation. However, I’m intrigued about the specific choices you’ve made when configuring the model for text generation. You set the temperature at 1.03, top_k at 500, and top_p at 0.98. I’d love to understand your reasoning behind these particular settings, as I’ve seen different values used in other GPTJ examples. Would you mind clarifying?

    1. Might be he’s aiming for a balance between novelty and structure. Play with settings until it clicks, right?

    2. I think the settings were chosen to strike a balance between creative output and coherence. Temperature slightly above 1 increases diversity.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.