...
- import new openTM2-TMs
- delete openTM2-TMs
- create new empty openTM2-TM
- import TMX
- open TM and close TM: not possible see extra section in this document. Maybe we need trigger to flush tm to the disk, but also it could be done in some specific cases...
- query TM for Matches: one query per TM, not quering multiple TMs at once.
- query TM for concordance search
- save new entry to TM
- delete entry from TM
- localy clone TM
- reorganize TM
- get some statistics about service
- also you can use tagreplacement endpoint to test tag replacement mechanism
This can be achieved by the following specification of a RESTful HTTP Serive, the specification is given in the following form:
- URL of the HTTP Resource, where servername and an optional path prefix is configurable.
- HTTP Method with affected functionality
- Brief Description
- Sent and returned Body.
Request Data Format:
The transferred data in the requests is JSON and is directly done in the request body. It's should be pretty json and ends with '\n}" symbol, because of bug in proxygen that caused garbage after valid data.
URL Format:
In this document, the OpenTM2 is always assumed under http://opentm2/.
...
Values | |
---|---|
%service% | Name of service(default - t5memory, could be changed in t5m3mory.conf file |
%tm_name% | Name of Translation Memory |
Example | http://localhost:4040/t5memory/examle_tm/fuzzysearch/? |
Endpoints overview | default endpoint/example | Is async? | ||||
---|---|---|---|---|---|---|
1 | Get the list of TMs | Returns JSON list of TMs | GET | / |
%service%/ | /t5memory/ | ||||
2 | Create TM | Creates TM with the provided name | POST | /%service%/ | / |
t5memory/ | |||||
3 | Create/Import TM in internal format | Import and unpack base64 encoded archive of .TMD, .TMI, .MEM files. Rename it to provided name | POST | /%service%/ | / |
t5memory/ | ||||
4 | Clone TM Localy | Makes clone of existing tm | POST | / |
%service%/%tm_name%/clone | /t5memory/my+TM/clone (+is placeholder for whitespace in tm name, so there should be 'my TM.TMD' and 'my TM.TMI'(and in pre 0.5.x 'my TM.MEM' also) files on the disk ) tm name IS case sensetive in url | |||
5 | Reorganize TM | Reorganizing tm(replacing tm with new one and reimporting segments from tmd) - async | GET | / |
%service%/%tm_name%/reorganize | /t5memory/my+other_tm/reorganize | + in 0.5.x and up | ||
5 | Delete TM | Deletes .TMD, .TMI files | DELETE | / |
%service%/%tm_name%/ | /t5memory/%tm_name%/ | ||||
6 | Import TMX into TM | Import provided base64 encoded TMX file into TM - async | POST | /%service%/%tm_name%/import | / |
t5memory/ |
%tm_ |
name%/import | + | |||
7 | Export TMX from TM | Creates TMX from tm. Encoded in base64 | GET | / |
%service%/%tm_name%/ | /t5memory/%tm_name%/ | |||
8 | Export in Internal format | Creates and exports archive with .TMD, .TMI files of TM | GET | / |
%service%/%tm_name%/ | /t5memory/%tm_name%/status | ||||
9 | Status of TM | Returns status\import status of TM | GET | /%service%/%tm_name%/status | / |
t5memory/ |
%tm_ |
name%/status | |||||
10 | Fuzzy search | Returns entries\translations with small differences from requested | POST | /%service%/%tm_name%/fuzzysearch | / |
t5memory/ |
%tm_ |
name%/fuzzysearch | ||||
11 | Concordance search | Returns entries\translations that contain requested segment | POST | / |
%service%/%tm_name%/concordancesearch | /t5memory/%tm_name%/concordancesearch | |||
12 | Entry update | Updates entry\translation | POST | / |
%service%/%tm_name%/entry | /t5memory/%tm_name%/entry | |||
13 | Entry delete | Deletes entry\translation | POST | / |
%service%/%tm_name%/entrydelete | /t5memory/%tm_name%/entrydelete | |||
14 | Save all TMs | Flushes all filebuffers(TMD, TMI files) into the filesystem | GET | / |
%service%_service/savetms | /t5memory_service/saveatms | |||
15 | Shutdown service | Flushes all filebuffers into the filesystem and shutting down the service | GET | / |
%service%_service/shutdown | /t5memory_service/shutdown | |||
16 | Test tag replacement call | For testing tag replacement | POST | / |
%service%_service/tagreplacement | /t5memory_service/tagreplacement | ||||
17 | Resources | Returns resources and service data | GET | /%service%_service/resources | / |
t5memory_service/resources |
Available end points
List of TMs | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Returns JSON list of TMs | |||||||||
Request | GET /%service%/ | |||||||||
Params | - | |||||||||
Returns list of open TMs and then list of available(excluding open) in the app.
|
Create TM | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Creates TM with the provided name(tmd and tmi files in/MEM/ folder) | |||||||||
Request | Post /%service%/%tm_name%/ | |||||||||
Params | Required: name, sourceLang | |||||||||
|
Create/Import TM in internal format | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Purpose | Import and unpack base64 encoded archive of .TMD, .TMI, .MEM(in pre 0.5.x versions) files. Rename it to provided name | |||||||||||||||||
Request | POST /%service%/%tm_name%/ | |||||||||||||||||
Params | { "name": "examle_tm", "sourceLang": "bg-BG" , "data":"base64EncodedArchive" } | |||||||||||||||||
Do not import tms created in other version of t5memory. Starting from 0.5.x tmd and tmi files has t5memory version where they were created in the header of the file, and different middle version(0.5.x) or global version(0.5.x) would be represented as This would create example_tm.TMD(data file) and example.TMI(index file) in MEM folder
|
...
Clone TM localy | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Creates TM with the provided name | |||||||||
Request | Post /%service%/%tm_name%/clone | |||||||||
Params | Required: name, sourceLang | |||||||||
Endpoint is sync(blocking)
|
...
Delete TM | |||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Purpose | Deletes .TMD, .TMI, .MEM files | ||||||||||||||||||
Request | Delete /%service%/%tm_name%/ | ||||||||||||||||||
Params | - | ||||||||||||||||||
|
...
Import provided base64 encoded TMX file into TM | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Import provided base64 encoded TMX file into TM. Starts another thead for import. For checking import status use status call | |||||||||
Request | POST /%service%/%tm_name%/import | |||||||||
Params | {"tmxData": "base64EncodedTmxFile" } | |||||||||
TM must exist
|
Reorganize TM | |
---|---|
Purpose | Reorganizes tm and fixing issues. |
Request | GET /%service%/%tm_name%/reorganize |
Headers | Accept - applicaton/xml |
up to v0.4.x reorganize is sync, so t5memory reorganize would check this condition
, and in case if this condition is true and then it passes segment to putProposal function, which is also used by UpdateRequest and ImportTmx request, so other
{ |
Reorganize TM
Accept - applicaton/xml
Export TMX from TM | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Creates TMX from tm. | |||||||||
Request | GET /%service%/%tm_name%/ | |||||||||
Headers | Accept - applicaton/xml | |||||||||
|
...
Export in internal format | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Creates and exports archive with .TMD, .TMI, .MEM files of TM | |||||||||
Request | GET /%service%/%tm_name%/ | |||||||||
Headers | application/zip | |||||||||
returns archive(.tm file) consists with .tmd and .tmi files
|
...
Get the status of TM | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Request | GET /%service%/%tm_name%/status | |||||||||
Params | - | |||||||||
Would return status of TM. It could be 'not found', 'available' if it's on the disk but not loaded into the RAM yet, and 'open' with additional info. In case if there was at least one try to import tmx or reorganize tm since it was loaded into the RAM, additional fields would appear and stay in the statistics till memory would be unloaded.
|
...
Update entry | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Updates entry\translation | |||||||||
Request | POST /%service%/%tm_name%/entry | |||||||||
Params | Only sourceLang, targetLang, source and target are required | |||||||||
This request would made changes only in the filebuffer(so files on disk would not be changed)
|
...
Delete entry | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Purpose | Deletes entry\translation | |||||||||
Request | POST /%service%/%tm_name%/entrydelete | |||||||||
Params | Only sourceLang, targetLang, source, and target are required Deleting based on strict match(including tags and whitespaces) of target and source | |||||||||
This request would made changes only in the filebuffer(so files on disk would not be changed)
|
...
Working directory | |
---|---|
Path | Description |
~/.t5memory | The main directory of service. Should always be under the home directory. Consists of nested folders and t5memory.conf file(see Config file). All directories\files below are nested |
LOG | lIncludes log files. It should be cleanup manualy. One session(launch of service) creates two files Log_Thu May 12 10:15:48 2022 .log and Log_Thu May 12 10:15:48 2022 .log_IMPORTANT |
MEM | Main data directory. All tm files is stored here. One TM should include .TMD(data file), .TMI(index file), .MEM(properties file) with the same name as TM name |
TABLE | Services reserved readonly folder with tagtables, languages etc. |
TEMP | For temporary files that were created for mainly import\export. On low debug leved(DEVELOP, DEBUG) should be cleaned manualy |
t5memory.conf | Main config file(see config file) |
Config directory should be located in a specific place |
Config file - obsolete - use commandline flags instead | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
field | default | Description | |||||||||
name | t5memory | name of service that we use under %service% in address | |||||||||
port | 8080 | service port | |||||||||
timeout | 3600 | service timeout | |||||||||
threads | 1 | ||||||||||
logLevel | 2 | logLevel - > see logging | |||||||||
AllowedRAM_MB | 1500 | Ram limit to operate openning\closing TM(see Openning and closing TM) Doesn't include services RAM | |||||||||
TriplesThreshold | 33 | Level of pre-fuzzy search filtering based on combinations of triples of tokens(excluding tags). Could impact fuzzy search perfomance. For higher values service is faster, but could skip some segments in result. Not always corelated with resulted fuzzyRate | |||||||||
Config file should be located under ~/.t5memory/t5memory.conf Anyway, all field has default values so the service could start without the conf file Reading\applying configs happen only once at service start Once service started you should be able to see setup values in logs.
|
...
Openning and closing TM | |
---|---|
In first concept it was planned to implement routines to open and close a TM. While concepting we found some problemes with this approach:
This leads to the following conclusion in implementation of opening and closing of TMs: OpenTM2 has to automatically load the requested TMs if requested. Also OpenTM2 has to close the TMs after a TM was not used for some time. That means that OpenTM2 has to track the timestamps when a TM was last requested.
http://opentm2/translationmemory/[TM_Name]/openHandle GET – Opens a memory for queries by OpenTM2 Note: This method is not required as memories are automatically opened when they are accessed for the first time. http://opentm2/translationmemory/[TM_Name]/openHandle DELETE – Closes a memory for queries by OpenTM2 Note: This method is not required as memories are automatically opened when they are accessed for the first time.
|
TM files structure and other related info | ||
---|---|---|
Info below is actual for version 0_5_x TM file is just archive with tmi and tmd files. |
TAG REPLACEMENT | ||
---|---|---|
Tag replacement
...
Previous documentation:
|
...