- open cmd as administrator
- wsl –install
- reboot
- wsl
- install CUDA toolkit
- wget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh
- ./Anaconda3-2022.10-Linux-x86_64.sh
- Reboot entire machine AGAIN
- mkdir gptj
- cd gptj
conda create -n gptj python=3.8conda activate gptjconda install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidiapip uninstall -y transformers && pip install --no-cache-dir https://github.com/deniskamazur/transformers/archive/gpt-j-8bit.zippip install bitsandbytes-cuda111pip install datasets==1.16.1- pip install torch==1.11.0+cu115 torchvision==0.12.0+cu115 -f https://download.pytorch.org/whl/torch_stable.html
import torch
import transformers
from transformers.models.gptj import GPTJForCausalLM
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = transformers.AutoTokenizer.from_pretrained("EleutherAI/gpt-j-6b")
gpt = GPTJForCausalLM.from_pretrained(
"hivemind/gpt-j-6B-8bit", low_cpu_mem_usage=True
).to(device)
raw_text = open("prompts/delandzombie.txt", "r").read()
text = raw_text
prompt = tokenizer((raw_text), return_tensors="pt")
prompt = {key: value.to(device) for key, value in prompt.items()}
out = gpt.generate(
**prompt,
do_sample=True,
temperature=1.03,
top_k=500,
top_p=0.98,
max_new_tokens=200,
)
out = tokenizer.decode(out[0])
text = out
print(
"\n",
"\n",
str(text),
"\n",
"\n",
end="",
)
raw_text += text
output = open("out.txt", "a")
output.write(
str(text)
+ "\n"
+ "\n"
+ "------"
+ "\n"
+ "\n"
)
output.close()
Great walkthrough on installing GPTJ on WSL, thoroughly enjoyed your step by step explanation. However, I’m intrigued about the specific choices you’ve made when configuring the model for text generation. You set the temperature at 1.03, top_k at 500, and top_p at 0.98. I’d love to understand your reasoning behind these particular settings, as I’ve seen different values used in other GPTJ examples. Would you mind clarifying?
I too am curious about these specific parameters for the model’s text generation. Would love some insights!
“Cool details, much confusion?”
Parameters tailored to encourage creativity, balance randomness with coherence.
Creative chaos meets methodical madness, eh?
Sometimes it’s kinda like seasoning to taste, you know? Adjusting params until it feels just right.
Might be he’s aiming for a balance between novelty and structure. Play with settings until it clicks, right?
Exactly, it’s all about hitting that sweet, sweet algorithmic spot.
Specificity brings control over chaos; a writer’s digital spice rack.
He’s tweaking those dials to fine-tune the output. Kinda like an artist with a palette, it’s subjective.
Spices for the text, yet subjectivity rules.
I think the settings were chosen to strike a balance between creative output and coherence. Temperature slightly above 1 increases diversity.