Version and versioning

What is the ChatGPT plug-in used for?

The plug-in enables you to create, train and use language resources based on Large Language Models (LLM) within translate5. It is possible to select from various GPT models, some of which can be trained, which serve as the basis for a language resource. These GPT language resources can then be trained with prompts and translation examples as needed. In addition, they can be fine-tuned via “Temperature”, “Top‑p”, “Presence” and “Frequency” parameters.


Sections and overviews

In translate5, the following sections are relevant for the management of GPT language resources:

  • The language resource overview:
    Language resources based on GPT models are created here.
  • The “Adjust OpenAI model” window:
    Is called up in the language resource overview and offers the option of fine-tuning models using various parameters.
  • The “Fine-tune OpenAI model” window:
    Is called up in the language resource overview and offers the possibility to test and train models with prompts and translation memories.
  • “Prompt management” in Preferences:
    You can register prompts here in order to use them as instructions when pre-translating with GPT language resources.

Creation, fine-tuning and training of GPT language resources as well as prompt management are all available to project managers.

Available engines/models

OpenAI

The models available are continuously queried at OpenAI’s platform and therefore correspond to what is currently available there.

Azure

The models that you have available in your AzureCloud are available here.

Creating GPT language resources

A language resource based on a GPT model is created like any other language resource in the language resource overview:

  1. Click on the button to open the “Add language resource” window.
  2. From the “Resource” drop-down, select the option “ChatGPT (OpenAI / MS Azure)”.
  3. Select the model you want to use from the “Engine/Model” drop-down.
    The models appearing at the top of the list are trainable, which is also indicated accordingly in their designation.
  4. Enter a meaningful name in the “Name” field.
  5. Use the “Source language” and “Target language” drop-downs to specify the language combination for which the language resource should be created.
  6. Select the clients for whom the language resource should be used from the “Use for selected clients” drop-down.
  7. Under “Read access by default”, select the clients to whose projects the language resource should be added with read rights by default.
  8. If required, select the clients for whose projects the language resource should be used in pivot language projects by default from the “Use as pivot by default” drop-down.
  9. Select the colour with which matches from this language resource should be coloured in the matches and concordance panel from the “Color” drop-down.
  10. Confirm by clicking the “Save” button.

The language resource is then created and will be visible immediately afterwards in the language resource overview.

Start typing in drop-down fields to find options more quickly. For languages, for example, you can type the ISO code: “de-de” will find “German (Germany) (de-DE)”.

Managing GPT language resources

The following options are available for GPT language resources in the language resource overview:

ButtonExplanation

Opens the “Edit language resource” window, but the basic settings can no longer be edited. However, clients can be added/removed for whom the language resource should be:

  • used;
  • used with read rights by default;
  • used as a pivot language resource by default.

Deletes the language resource. The deletion process must be completed by confirming it in a window that appears after clicking the button.

Opens the “Adjust OpenAI model” window, in which you can adjust various parameters to fine-tune the GPT language resource.

Opens the “Fine-tune OpenAI model”, via which you can train and test the GPT resource with prompts.

Fine-tuning GPT language resources

In the “Adjust OpenAI model” window, both trainable and untrainable GPT resources can be fine-tuned using the following parameters:

  • Use fine-tuning default system-message when translating with a trained model:
    • If this box is ticked, the standard prompt (the top prompt in the training window) will be used as instruction when pre-translating with the model.
    • If the box remains deselected, the following prompt will be used as instruction when pre-translating with the model: “Translate the following segments encapsulated in JSON Objects with the index and the segment as properties from [source language] as source language to [target language] as target language using all segments as context for each other”.
  • Use all trained system messages when translating with a trained model:
    • If this box is ticked, the user-specific prompts will be used as instructions when pre-translating with the model. These can be selected in the “Fine-tune OpenAI Model” window.
  • Generation Sensitivity / Temperature: Values from 0 to 2, with up to two decimal places.
  • Probability Threshold / Top P: Values from 0 to 1, with up to two decimal places.
  • Presence Penalty: Values from 0 to 2, with up to two decimal places.
  • Frequency Penalty: Values from 0 to 2, with up to two decimal places.
  • Max. target tokens (% of source tokens): A GPT model can only ever process a limited number of tokens. This maximum number of tokens includes both the sent and the returned tokens. In the case of a (pre-)translation, this includes the prompt(s), the text or batch to be translated and the returned translations. An appropriate ratio must be maintained so that there is enough “space” for the returned tokens. This setting is only relevant for batch translations, such as those used for pre-translation.

A token is the smallest linguistic unit that a language model like GPT processes. It can be a single character, a word, or a part of a word – depending on the language and structure of the text. For example, the word “translation” is usually counted as one token, whereas a long or compound term can comprise several tokens.

Parameters for fine-tuning

How creative should the translation be?

Generation sensitivity / Temperature

This parameter determines how “random” or “creative” the language model should be when generating text. A low Temperature means that the model translates more objectively and predictably, while a higher temperature means that it can translate very creatively and therefore unpredictably.

Probability threshold / Top P

The “Top P” parameter (also known as “nucleus sampling”) is a nuanced alternative to Temperature-based sampling. It is like a “spotlight” that emphasizes the most probable words. With a default value of 1.0 all words are taken into account. This parameter can help to control the distribution of word choice and thus ensure the relevance and coherence of the generated text.

Attention: If Temperature is set to a very high value, it is possible that the model will generate contradictory or meaningless content.

It is advisable to adjust either the Temperature or top P, but not both.

Please have a look at this page for further information on the two parameters Temperature and Top P.

How varied should the translation be?

Presence Penalty

This parameter is used to encourage the model to include a wide range of tokens in the generated text. This is a value that is deducted from the log probability of a token each time it is generated. A high Presence penalty value means that the model tends to generate tokens that are not yet contained in the generated text.

Frequency Penalty

This parameter is used to prevent the model from using the same words or phrases too often within the generated text. It is a value that is added to the log probability of a token each time it appears in the generated text. A high Frequency Penalty value means that the model is more careful when using recurring tokens.

Please have a look at this page for further information on the two parameters Presence Penalty and Frequency Penalty.

Max. target tokens (% of source tokens)

A GPT model can only ever process a limited number of tokens. This maximum number of tokens includes both the sent and the returned tokens. In the case of a (pre-)translation, this includes the prompt(s), the text or batch to be translated and the returned translations. An appropriate ratio must be maintained so that there is enough “space” for the returned tokens. This setting is only relevant for batch translations, such as those used for pre-translation.

Prompt management

Training GPT language resources

Once a GPT language resource has been created, it can be trained with prompts, examples, and specific translation memories. In the language resource overview, click on the button in the line of the language resource that you would like to train. The “Fine-tune OpenAI Model” window opens. It includes the following areas:

  1. The prompts and example sentences that can be loaded from a prompt set previously created in Prompt management are displayed here.
  2. Test panel in which a test text can be entered, and its translation tested using the current configuration.
  3. The TM panel, in which one or more translation memories can be added for training the model, as well as a drop-down in which the number of training iterations can be selected.

All entries in the TMs are used as examples for your training. Normally, this only makes sense for TMs that contain hand-picked examples for your training. The use of large amounts of TM data usually reduces the translation quality.

Loading and customizing prompt sets, starting a training

  1. Click on “Add some pre-configured prompt” to open the “Add some pre-configured prompt” window.
  2. It lists all the prompt sets created in Prompt management. They can be selected by ticking the box in the “Add” column.
  3. The selection is confirmed by clicking the “OK” button, which also closes the window.
  4. The prompts and examples contained in the selected prompt sets are now displayed in the “Fine-tune OpenAI Model” window and can be added to, adjusted or removed as required. The functions of the buttons correspond to their counterparts in Prompt management.
  5. You can now enter any test text in the top window of the test panel. By clicking on the “Translate” button, the text is translated using the current configuration of the selected language resource in combination with the added prompts and example sentences. The translation of the test text is displayed in the lower window.
  6. One or more task or main translation memories can also be added to the TM panel for (pre-)translation.
  7. If you are satisfied with the way the test text is translated, you can select further down in the TM panel the number of epochs with which the language resource should be trained using the existing configuration. Then start the training using the “Submit training” button.

You can add one or more of the preconfigured prompt sets.

As the use of terminology during training does not lead to good results, no TermCollections can be added in translate5 for this purpose. Instead, we recommend that you add the desired TermCollections as usual when creating tasks so that GPT can then take the resource into account for pre-translation during import.