@se_6c624c
2 months ago
Yes you are on the right track. What you are mentioning is called fine tuning the model. I personally have done this and used the same approach. The LLM I used was GPT-J 6B to generate MCQ's. Some tips when fine tuning large LLM's are: Do not feed all the data to the model. First create a small dataset and fine tune the model in order to check whether the fine tuning is working properly for a couple epochs. This may save you some hours down the road. Make sure you understand all the Hyperparameters and their effects first before fine tuning. Since the model is large, fine tuning might take a long time and resources and you do not want to fine tune multiple times with different parameters. So save yourselves some time and money by understanding the effects of the parameters first. Yes it is possible to load the model into lower bits (also known as quantisation) in order to reduce the model size and in turn reduce resource utilisation. Make sure to quantise before training and then before inferencing, make sure to de-quantise the model to original bits. This is important as I was getting nonsense results just because I forgot to de-quantise the model after training and before inferencing. Your first point of only using the last layer. I am not sure about this as I did not try this method. Regarding the 3rd point of using a very small learning rate, it is a hyperparameter. So you might want to tune it. But usually if the model is huge (as in my case 6 billion parameters, 100 GB) even if there is some leeway in tuning the parameters, it won't affect the results much as the model is robust enough to counter it. But again it depends on your model size! Cheers!