![]() ![]() One of the main reasons for this slow progress could be the lack of large labeled text datasets. However, this performance of deep learning models in NLP pales in comparison to the performance of deep learning in Computer Vision. If you’d like more content like this, I post on YouTube too.With the advancement in deep learning, neural network architectures like recurrent neural networks (RNN and LSTM) and convolutional neural networks (CNN) have shown a decent improvement in performance in solving several Natural Language Processing (NLP) tasks like text classification, language modeling, machine translation, etc. I hope you enjoyed this article! If you have any questions, let me know via Twitter or in the comments below. There are already a tremendous number of datasets available - and more are made available frequently - so there’s plenty to learn from! CommonSenseQA - Good luck with this one □.FiQA - Financial opinion Q&A from Reddit.You most likely won’t be using the SQuAD dataset for any fine-tuning - instead, you will be using your own data, or an open-source Q&A dataset: ![]() Learning and understanding this process is very important and, when applied correctly, can push Q&A model performance on specific use-cases much higher. However, more importantly, we have introduced the process of taking a Q&A dataset - and used it to ‘fine-tune’ a pre-trained transformer model. That’s all for this walk-through! We’ve taken a pre-trained DistilBert model, fitted it with a Q&A head - and fine-tuned it using the SQuAD dataset. This time though, we call model.eval() to switch the behavior of relevant layers from training to inference mode - and use torch.no_grad() to stop PyTorch from calculating model gradients (only needed during training): Once again, we set up a DataLoader - and then loop through each batch. Now we know how to calculate our Q&A accuracy - let’s implement it for our val_dataset! item() extracts the tensor value as a plain and simple Python int. We do this with PyTorch like so: acc = ( (start_pred = start_true).sum() / len(start_pred) ).item() To calculate the EM of each batch, we take the sum of the number of matches per batch - and divide by the total. The simplest metric we can measure here is accuracy - more specifically known as exact match (EM) - did the model get the right start or end token, or not? Of course, this doesn’t factor in models that are getting close and maybe missing by one or two tokens - but it’s a start. The start_pred and end_pred tensors will look like this - these are the model's prediction of start-end answer positions. ![]() To extract the start-end token range from our model, we can access the start_logits and end_logits tensors and perform an argmax function like so: start_pred = torch.argmax(outputs, dim=1) end_pred = torch.argmax(outputs, dim=1) Once we have trained our model, we can begin making predictions and asking our model a few questions.īear in mind here that we are already using a highly optimized base model - which gets us halfway there - but we could still benefit from experimenting with different models.īut for now, we just need to learn this approach - and how to apply it to our own more specific use-cases. Or using the generic AutoModel loader: model = om_pretrained(model_path) tokenizer = om_pretrained(model_path) Measuring Performance When loading our model again, we can use the same from_pretrained method that we use when loading models from HuggingFace - all we do is replace the online model path/name with our local path - in this case: model_path = 'models/distilbert-custom' model = om_pretrained(model_path) tokenizer = om_pretrained(model_path)
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |