translate5 has a core component for this. It is able to manage multiple processes at the same time.

This component e. g. is used for

• importing and exporting files
• marking terminology
• counting terminology errors, words and characters
• etc.

Worker capabilities

Asynchronous processing

The GUI is not blocked by the server side processes. The user gets informed, when the background processes finish

Chaining of

...

workers

This refers to

...

different worker types

...

, that should run over the data of the same task.

The developer can define, in which order background processes should run over the the segments. E. g. first pre-translation, then termTagging, then segment analysis. In this case first termTagging worker for a task will only start, after the last pre-translation worker is finished.

Queuing of workers

...

This refers to the same

...

worker type running in parallel on multiple tasks.

It can be defined, how many parallel processes of which worker type are allowed. E. g. max. 5 parallel import processes (because the machine needs free CPU capacity to answer user GUI requests) but additional 50 parallel MT-pretranslation requests (because MT-pretranslation runs on a different server). If there are more parallel requests than allowed, the additional ones get queued and will be started, as soon as there is free capacity.

 

Load balancing

This refers the same worker type running in parallel multiple times on the same task.

        --- chaining of jobs:
                ---- Different types of background jobs can be chained for a task. This means, translate5 ensures, that all MT-pretranslation jobs will be finished, before the termTagging jobs start.
        --- load balancing: 
            translate5 can e. g. talk to 5 different instances of an external system (number configurable) at the same time and pass chunks of the job to each of them (the size of the chunks is configurable, e. g. 50 segments per chunk).

When one chunk is returned by one of the external instances, the next chunk will be passed to it. This continues until all segments have been passed to the external system.

Worker architecture - developer notes

Worker These processes can be external processes called via REST, or internal translate5 processes.

...