I want to train / fine-tune a model (sh.itjust.works)

submitted 1 year ago by [email protected] to c/[email protected]

1 comments fedilink hide all child comments

I want to train, or more likely fine-tune, a model on about 20 years worth of email and text data that I've collected.

The goal would be to train it how to respond like me in simple cases.

It's there a particular base model I should start with?

I'm also interested in anyone's experience in doing this kind of thing themselves.

top 1 comments

sorted by: hot top controversial new old

[-] [email protected] 2 points 1 year ago

I'm inclined to fine tune/make a Lora for Llama (2? I'm not sure how well it's supported yet), you definitely won't want to train from scratch. The big thing seems to be formating the dataset properly, although I haven't actually done any of this myself yet. this blog seems to have a few good articles about it that you might be interested in reading.

this post was submitted on 05 Aug 2023

3 points (100.0% liked)

LocalLLaMA

2210 readers

2 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

[email protected]