# Fine Tuning GPT2 for machine translation

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-machine-translation%2F5894%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 30, 2021, 11:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-machine-translation%2F5894 "2021-04-30T23:23:04Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![yansoares](https://avatars.discourse-cdn.com/v4/letter/y/898d66/32.png) [@yansoares](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fyansoares)%5C
**Post date:** [April 30, 2021, 11:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-machine-translation%2F5894%2F1 "2021-04-30T23:23:04Z")

</div>

good evening everyone,

is it possible to fine-tune gpt2 for text translation?

if it is possible, how can I do it using my own data?

I want to translate from ASL to English, and the idea that came to me was to use gpt2 as the decoder (since it is trained in English) and use a BERT as an encoder (I would fine tune it and retrain with the ASL base)

Does anyone have a tutorial on how to do something like this?

---

<div class="post-metadata">

**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [May 2, 2021, 10:22am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-machine-translation%2F5894%2F2 "2021-05-02T10:22:49Z")

</div>

hey @yansoares, you could try using the `EncoderDecoderModel` ([docs](https://huggingface.co/transformers/model_doc/encoderdecoder.html?highlight=encoderdecodermodel#encoder-decoder-models)) to create a BERT2GPT model. I’m not aware of any tutorials using this class, but the original paper ([link](https://arxiv.org/abs/1907.12461)) is quite well written and should give you a good idea on what needs to be done.
