...
The transferred data in the requests is JSON , and is directly done in the request body.
URL Format:
In this document, the OpenTM2 is always assumed under http://opentm2/.
To rely on full networking features (proxying etc.) the URL is configurable in Translate5 , so that the OpenTM2 instance can also reside under : http://xyz/foo/bar/.
Errors
For each request, the possible errors are listed below for each resource. In case of an error, the body should contain at least the following JSON, if it is senseful the attributes of the original representation can be added.
...
errors: [{errorMsg: 'Given tmxData is no TMX.'}]
}
http://opentm2/translationmemory/
POST - creating a new or importing an existing filebased binary OpenTM2 TM
The Parameter „name“ contains the TM Name as a string. The string has a maxlength of 256 chars. It can contain any characters except the characters backslash (\), slash(/), colon (:), question mark (?), asterisk (*), vertical line (|), less than sign (<), and greater than sign (>).
Uploading a file is optional, omitting a file means creating a empty TM only.
If an empty TM is created, the POST request contains only the JSON structure with the TM Name.
If an existing binary OpenTM2 file should be additionally imported to the new TM, the POST must be encoded as multipart/form-data.
The JSON structure with the meta data will then be in the first chunk of the multiparted request, the chunk must be named “meta”.
The second chunk contains the plain binary file content and must be named “data”. This binary data contains the TM content
The resulting body contains the name of the TM, as given in the POST request.
LOGGING:
You can optionally set here or in other requests with JSON body logging level that could impact performance.
You can set the next levels:
...
| Endpoints | ||
|---|---|---|
| 1 | Get the list of TMs | Returns JSON list of TMs |
| 2 | Create TM | Creates TM with the provided name |
| 3 | Create/Import TM in internal format | Import and unpack base64 encoded archive of .TMD, .TMI, .MEM files. Rename it to provided name |
| 4 | Delete TM | Deletes .TMD, .TMI, .MEM files |
| 5 | Import TMX into TM | Import provided base64 encoded TMX file into TM |
| 6 | Export TMX from TM | Creates TMX from tm. Encoded in base64 |
| 7 | Export in Internal format | Creates and exports archive with .TMD, .TMI, .MEM files of TM |
| 8 | Status of TM | Returns status\import status of TM |
| 9 | Fuzzy search | Returns enrties\translations with small differences from requested |
| 10 | Concordance search | Returns entries\translations that contain requested segment |
| 11 | Entry update | Updates entry\translation |
| 12 | Entry delete | Deletes entry\translation |
| 13 | Save all TMs | Flushes all filebuffers(TMD, TMI files) into the filesystem |
| 14 | Shutdown service | Flushes all filebuffers into the filesystem and shutting down the service |
Values | |
|---|---|
| %service% | Name of service(default - t5memory, could be changed in t5m3mory.conf file |
| %tm_name% | Name of Translation Memory |
| ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Returns JSON list of TMs | |||||||||
| Request | GET /%service%/ | |||||||||
| Params | - | |||||||||
| ||||||||||
2. Create TM | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Creates TM with the provided name | |||||||||
| Request | Post /%service%/%tm_name%/ | |||||||||
| Params | Required: name, sourceLang | |||||||||
| ||||||||||
3. Create/Import TM in internal format | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Import and unpack base64 encoded archive of .TMD, .TMI, .MEM files. Rename it to provided name | |||||||||
| Request | POST /%service%/%tm_name%/ | |||||||||
| Params | { "name": "examle_tm", "sourceLang": "bg-BG" , "data":"base64EncodedArchive" } | |||||||||
| ||||||||||
4. Delete TM | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Deletes .TMD, .TMI, .MEM files | ||||||||||||||||||
| Request | Delete /%service%/%tm_name%/ | ||||||||||||||||||
| Params | - | ||||||||||||||||||
| |||||||||||||||||||
5. Import provided base64 encoded TMX file into TM | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Import provided base64 encoded TMX file into TM. Starts another thead for import. For checking import status use status call | |||||||||
| Request | POST /%service%/%tm_name%/import | |||||||||
| Params | {"tmxData": "base64EncodedTmxFile" } | |||||||||
| ||||||||||
6. Export TMX from TM | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Creates TMX from tm. | |||||||||
| Request | GET /%service%/%tm_name%/ | |||||||||
| Headers | Accept - applicaton/xml | |||||||||
| ||||||||||
7. Export in internal format | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Creates and exports archive with .TMD, .TMI, .MEM files of TM | |||||||||
| Request | GET /%service%/%tm_name%/ | |||||||||
| Headers | application/zip | |||||||||
| ||||||||||
8. Get the status of TM | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Request | GET /%service%/%tm_name%/status | |||||||||
| Params | - | |||||||||
| ||||||||||
9. Fuzzy search | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Returns enrties\translations with small differences from requested | |||||||||
| Request | POST /%service%/%tm_name%/fuzzysearch | |||||||||
| Params | Required: source, sourceLang, targetLang iNumOfProposal - limit of found proposals - max is 20, if 0 → use default value '5' | |||||||||
| ||||||||||
10. Concordance search | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Returns entries\translations that contain requested segment | |||||||||
| Request | POST /%service%/%tm_name%/concordancesearch | |||||||||
| Params | Required: searchString - what we are looking for , searchType ["Source"|"Target"|"SourceAndTarget"] - where to look iNumOfProposal - limit of found proposals - max is 20, if 0 → use default value '5' | |||||||||
| ||||||||||
11. Update entry | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Updates entry\translation | |||||||||
| Request | POST /%service%/%tm_name%/entry | |||||||||
| Params | Only sourceLang, targetLang, source and target are required | |||||||||
| ||||||||||
12. Delete entry | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Deletes entry\translation | |||||||||
| Request | POST /%service%/%tm_name%/entrydelete | |||||||||
| Params | Only sourceLang, targetLang, source, and target are required Deleting based on strict match(including tags and whitespaces) of target and source | |||||||||
| ||||||||||
13. Save all TMs | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Flushes all filebuffers(TMD, TMI files) into the filesystem. Reset 'Modified' flags for file buffers. Filebuffer is a file instance of .TMD or .TMI loaded into RAM. It provides better speed and safety when working with files. | |||||||||
| Request | GET /%service%/%tm_name%_service/savetms | |||||||||
| Params | - | |||||||||
| ||||||||||
14. Shutdown service | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Purpose | Safely shutting down the service with\without saving all loaded tm files to the disk | |||||||||
| Request | GET /%service%/%tm_name%_service/shutdown?dontsave=1 | |||||||||
| Params | dontsave=1(optional in address) - skips saving tms, for now value doesn't matter, only presence | |||||||||
| ||||||||||
OTHER SERVICE DETAIL
You can configure the service in ~/.t5service/t5memory.conf
Logging | ||
|---|---|---|
| Level | Mnemonic | Description |
| 0 | DEVELOP | could make code work really slow, should be used only when debugging some specific places in code, like binary search in files, etc. |
| 1 | DEBUG | logging values of variables. Wouldn't delete temporary files(In MEM and TMP subdirectories), like base64 encoded\decoded tmx files and archives for import\export |
| 2 | INFO | logging top-level functions entrances, return codes, etc. Default value. |
| 3 | WARNING | logging if we reached some commented or hardcoded code. Usually commented code here is replaced with new code, and if not, it's marked as ERROR level |
| 4 | ERROR | errors, why and where something fails during parsing, search, etc |
| 5 | FATAL | you shouldn't reach this code, something is really wrongOther values would be ignored. The set level would stay the same till you change it in a new request or close the app. Logs suppose to be written into a file with date\time name under ~/.OtmMemoryService/Logs and errors/fatal are supposed to be duplicated in another log file with FATAL suffices |
| 6 | TRANSACTION | - Logs only things like begin\end of request etc. No purpose to setup this hight |
Logging could impact application speed very much, especially during import or export. POST http://localhost:4040/t5memory/example_tm/{ Or in t5memory.conf file in line | ||
| Working directory | |
|---|---|
| Path | Description |
| ~/.t5memory | The main directory of service. Should always be under the home directory. Consists of nested folders and t5memory.conf file(see Config file). All directories\files below are nested |
| LOG | lIncludes log files. It should be cleanup manualy. One session(launch of service) creates two files Log_Thu May 12 10:15:48 2022 .log and Log_Thu May 12 10:15:48 2022 .log_IMPORTANT |
| MEM | Main data directory. All tm files is stored here. One TM should include .TMD(data file), .TMI(index file), .MEM(properties file) with the same name as TM name |
| TABLE | Services reserved readonly folder with tagtables, languages etc. |
| TEMP | For temporary files that were created for mainly import\export. On low debug leved(DEVELOP, DEBUG) should be cleaned manualy |
| t5memory.conf | Main config file(see config file) |
Config directory should be located in a specific place | |
Config file | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| field | default | Description | |||||||||
| name | t5memory | name of service that we use under %service% in address | |||||||||
| port | 8080 | service port | |||||||||
| timeout | 3600 | service timeout | |||||||||
| threads | 1 | ||||||||||
| logLevel | 2 | logLevel - > see logging | |||||||||
| AllowedRAM_MB | 1500 | Ram limit to operate openning\closing TM(see Openning and closing TM) Doesn't include services RAM | |||||||||
| TriplesThreshold | 33 | Level of pre-fuzzy search filtering based on combinations of triples of tokens(excluding tags). Could impact fuzzy search perfomance. For higher values service is faster, but could skip some segments in result. Not always corelated with resulted fuzzyRate | |||||||||
Config file should be located under ~/.t5memory/t5memory.conf Anyway, all field has default values so the service could start without the conf file Reading\applying configs happen only once at service start Once service started you should be able to see setup values in logs.
| |||||||||||
Openning and closing TM | |
|---|---|
Opening and closing a TMIn first concept it was planned to implement routines to open and close a TM. While concepting we found some problemes with this approach:
This leads to the following conclusion in implementation of opening and closing of TMs: OpenTM2 has to automatically load the requested TMs if requested. Also OpenTM2 has to close the TMs after a TM was not used for some time. That means that OpenTM2 has to track the timestamps when a TM was last requested.
http://opentm2/translationmemory/[TM_Name]/openHandleGET – Opens a memory for queries by OpenTM2Note: This method is not required as memories are automatically opened when they are accessed for the first time. http://opentm2/translationmemory/[TM_Name]/openHandleDELETE – Closes a memory for queries by OpenTM2Note: This method is not required as memories are automatically opened when they are accessed for the first time.
|
http://opentm2/translationmemory/
POST - creating a new or importing an existing filebased binary OpenTM2 TM
The Parameter „name“ contains the TM Name as a string. The string has a maxlength of 256 chars. It can contain any characters except the characters backslash (\), slash(/), colon (:), question mark (?), asterisk (*), vertical line (|), less than sign (<), and greater than sign (>).
Uploading a file is optional, omitting a file means creating a empty TM only.
If an empty TM is created, the POST request contains only the JSON structure with the TM Name.
If an existing binary OpenTM2 file should be additionally imported to the new TM, the POST must be encoded as multipart/form-data.
The JSON structure with the meta data will then be in the first chunk of the multiparted request, the chunk must be named “meta”.
The second chunk contains the plain binary file content and must be named “data”. This binary data contains the TM content
The resulting body contains the name of the TM, as given in the POST request.
...
To OpenTM2 – without data / creating an empty TM:
...
POST http://opentm2/translationmemory HTTP/1.1
Content-Type: multipart/form-data; boundary="autogenerated"
-- autogenerated
Content-Type: application/json; charset=utf-8/1.1
Content-DispositionType: multipart/form-data; nameboundary=meta
{"name":"TM Name", sourceLang:"en"}"autogenerated"
-- autogenerated
Content-Type: image/jpeg
Content-Disposition: form-dataapplication/json; name=data; filename=Original Filename.jpg
...TM content ...
--autogenerated--In both cases from OpenTM2 - HTTP 200 OK:
{
name: „TM Name“
}
Errors:
- 400 Bad Request – if parameters are missing or are not well formed.
- 409 Conflict – if a memory with the given name already exists.
- 500 Server Error – for other technical problems.
http://opentm2/translationmemory/[TM_Name]/import
POST import a TMX file into an existing OpenTM2 TM
To OpenTM2:
multipart/form-data like on POST above, expect that no separate JSON section is needed here.
Call answers directly after the upload is done, but before the import starts with HTTP 201 – this means: Import is created and will be started now.
From OpenTM2 - HTTP 201 OK:
charset=utf-8
Content-Disposition: form-data; name=meta
{"name":"TM Name", sourceLang:"en"}
--autogenerated
Content-Type: image/jpeg
Content-Disposition: form-data; name=data; filename=Original Filename.jpg
...TM content ...
--autogenerated--In both cases from OpenTM2 - HTTP 200 OK:
{
name: „TM Name“{ // empty JSON object, since no data expected as result here!
}
Errors:
- 400 Bad Request – if parameters are missing or are not well formed.
- 404 Not Found 409 Conflict – if the a memory of with the given name does not existalready exists.
- 500 Server Error – for other technical problems.
http://opentm2/translationmemory/[TM_Name]/
...
import
POST import a TMX file into an existing OpenTM2 TM
To OpenTM2:
multipart/form-data like on POST above, expect that no separate JSON section is needed here.
Call answers directly after the upload is done, but before the import starts with HTTP 201 – this means: Import is created and will be started now.
From OpenTM2 - HTTP 200 201 OK:
{ ‘status’:’import’ // allowed status values: import, available, errorempty JSON object, since no data expected as result here!
}
Errors:
- 400 Bad Request – if parameters are missing or are not well formed.
- 404 Not Found – if the memory of the given name does not exist
- 500 Server Error – for other technical problems.
http://opentm2/translationmemory/
GET – retrieving a list of available TM Files
To OpenTM2: -
From OpenTM2 - HTTP 200 OK:
[{
name: 'my nice TM'
}]
Errors:
- formed.
- 404 Not Found – if the memory of the given name does not exist
- 500 Server Error – for other technical problems.
http://opentm2/translationmemory/[TM_Name]/
...
status
GET
...
status of a
...
TM
...
To OpenTM2:
multipart/form-data like on POST above, expect that no separate JSON section is needed here.
From OpenTM2 - HTTP 200 OK: HTTP 200 OK:
{
‘status’:’import’ //allowed status values: import, available, error
}Same as POST from OpenTM2 result.
Errors:
- 404 Not Found – if TM file to given [TMID] in URL was not found
- 500 Server Error – for other technical problems.
DELETE – deletes an existing TM File
Adressed by the given URL, no body needed.
Errors:
- 400 Bad Request – if parameters are missing or are not well formed.
- 404 Not Found – if TM file to given [TMID] in URL was not foundthe memory of the given name does not exist
- 500 Server Error – for other technical problems.
PUT – updating an existing TM File in one request
Currently not needed, would be only to change the TM name
GET – list of all segments from TM
Currently not needed.
Opening and closing a TM
In first concept it was planned to implement routines to open and close a TM. While concepting we found some problemes with this approach:
- First one is the realization: opening and closing a TM by REST would mean to update the TM Resource and set a state to open or close. This is very awkward.
- Since in translate5 multiple tasks can be used to the same time, multiple tasks try to access one TM. Closing TMs is getting complicated to prevent race conditions in TM usage.
- Since OpenTM2 loads the whole TM in memory, OpenTM2 must control itself which TMs are loaded or not.
This leads to the following conclusion in implementation of opening and closing of TMs:
OpenTM2 has to automatically load the requested TMs if requested. Also OpenTM2 has to close the TMs after a TM was not used for some time. That means that OpenTM2 has to track the timestamps when a TM was last requested.
http://opentm2/translationmemory/[TM_Name]/openHandle
GET – Opens a memory for queries by OpenTM2
Note: This method is not required as memories are automatically opened when they are accessed for the first time.
http://opentm2/translationmemory/[TM_Name]/openHandle
DELETE – Closes a memory for queries by OpenTM2
http://opentm2/translationmemory/
GET – retrieving a list of available TM Files
To OpenTM2: -
From OpenTM2 - HTTP 200 OK:
[{
name: 'my nice TM'
}]
Errors:
- 500 Server Error – for other technical problems.
http://opentm2/translationmemory/[TM_Name]/
TM_Name is URL-encoded
GET – retrieving a single TM File
To OpenTM2: -
From OpenTM2 - HTTP 200 OK:
Same as POST from OpenTM2 result.
Errors:
- 404 Not Found – if TM file to given [TMID] in URL was not found
- 500 Server Error – for other technical problems.
DELETE – deletes an existing TM File
Adressed by the given URL, no body needed.
Errors:
- 404 Not Found – if TM file to given [TMID] in URL was not found
- 500 Server Error – for other technical problems.
PUT – updating an existing TM File in one request
Currently not needed, would be only to change the TM name
GET – list of all segments from TM
Currently not neededNote: This method is not required as memories are automatically opened when they are accessed for the first time.
http://opentm2/translationmemory/[TM_Name]/entry/
...
targetLang: 'en', ← rfc5646
matchRate: '100',
documentName: 'my file.sdlxliff',
DocumentShortName: 'shortnam.txt',
id: 'identifier',
type: 'Manual',
matchType: 'Exact',
segmentNumber: 123,
markupTable: 'XYZ: '100',
timestamp documentName: '2015-05-12 13:46:12my file.sdlxliff',
author DocumentShortName: „Thomas Lauria“.'shortnam.txt',
id context: 'identifier',
addInfo: ''
}]}
Errors:
- 400 Bad Request – if search, query or language parameters are missing or are not well formed.
- 404 Not Found – if TM file to given [TM_Name] in URL was not found
- 500 Server Error – for other technical problems.
SearchPosition / NewSearchPosition
...
type: 'Manual',
matchType: 'Exact',
segmentNumber: 123,
markupTable: 'XYZ',
timestamp: '2015-05-12 13:46:12',
author: „Thomas Lauria“.
context: '',
addInfo: ''
}]}
Errors:
- 400 Bad Request – if search, query or language parameters are missing or are not well formed.
- 404 Not Found – if TM file to given [TM_Name] in URL was not found
- 500 Server Error – for other technical problems.