...

import new openTM2-TMs
delete openTM2-TMs
create new empty openTM2-TM
import TMX
open TM and close TM: not possible see extra section in this document.
query TM for Matches: one query per TM, not quering multiple TMs at once.
query TM for concordance search
save new entry to TM
get some statistics about service

This can be achieved by the following specification of a RESTful HTTP Serive, the specification is given in the following form:

...

The transferred data in the requests is JSON and is directly done in the request body. It's should be pretty json and ends with '\n}" symbol, because of bug in proxygen that caused garbage after valid data.

URL Format:

In this document, the OpenTM2 is always assumed under http://opentm2/.

...

Endpoints overview
1	Get the list of TMs	Returns JSON list of TMs	GET	/%service%/
2	Create TM	Creates TM with the provided name	POST	/%service%/
3	Create/Import TM in internal format	Import and unpack base64 encoded archive of .TMD, .TMI, .MEM files. Rename it to provided name	POST	/%service%/
4	Clone TM Localy	Makes clone of existing tm	POST	/%service%/%tm_name%/
5	Reorganize TM	Reorganizing tm(replacing tm with new one and reimporting segments from tmd) - async	GET	/%service%/%tm_name%/reorganize
5	Delete TM	Deletes .TMD, .TMI , .MEM files	DELETE	/%service%/%tm_name%/
6	Import TMX into TM	Import provided base64 encoded TMX file into TM - async	POST	/%service%/%tm_name%/import
7	Export TMX from TM	Creates TMX from tm. Encoded in base64	GET	/%service%/%tm_name%/
8	Export in Internal format	Creates and exports archive with .TMD, .TMI , .MEM files of TM	GET	/%service%/%tm_name%/
9	Status of TM	Returns status\import status of TM	GET	/%service%/%tm_name%/status
10	Fuzzy search	Returns enrtiesentries\translations with small differences from requested	POST	/%service%/%tm_name%/fuzzysearch
11	Concordance search	Returns entries\translations that contain requested segment	POST	/%service%/%tm_name%/concordancesearch
12	Entry update	Updates entry\translation	POST	/%service%/%tm_name%/entry
13	Entry delete	Deletes entry\translation	POST	/%service%/%tm_name%/entrydelete
14	Save all TMs	Flushes all filebuffers(TMD, TMI files) into the filesystem	GET	/%service%_service/savetms
15	Shutdown service	Flushes all filebuffers into the filesystem and shutting down the service	GET	/%service%_service/shutdown
16	Test tag replacement call	For testing tag replacement	POST	/%service%_service/tagreplacement
17	Resources	Returns resources and service data	GET	/%service%_service/resources

...

List of TMs

Purpose

Returns JSON list of TMs

Request

GET /%service%/

Params

-

Returns list of open TMs and then list of available(excluding open) in the app.

Code Block

language	js
title	Response
collapse	true

Response example:
{
    [
        {name:examle_tm
        },
        {name:mem_gt_issue
        }
    ]
}

Create TM

PurposeCreates TM with the provided nameRequestPost /%service%/%tm_name%/Params

Required: name, sourceLang

Code Block

language	js
title	Response
collapse	true

Request example {

"Open": [
        {
            "name": "

examle_tm",

mem2"
        }

"sourceLang":

"bg-BG"

],

["data

    "Available on disk": [
        {
            "name": "

base64_encoded_archive_see_import_in_

mem_internal_format"

] ["loggingThreshold": 0] } Response example:Success:{


        },
        {
            "name": "

examle_tm

mem1"
        },
        {
            "name": "newBtree3"
        },
        {
            "name": "newBtree3_cloned"
        }
    ]
}

Create TM

Purpose

Creates TM with the provided name

Request

Post /%service%/%tm_name%/

Params

Required: name, sourceLang

Code Block

language	js
title	Response
collapse	true

Request example 
{    "name": "examle_tm",    
     "sourceLang": "bg-BG" 
   ["data": "base64_encoded_archive_see_import_in_internal_format"]
   ["loggingThreshold": 0]
}

Response example:Success:{
"name": "examle_tm",
}
TM already exists:
{
  "ReturnValue": 1,
  "ErrorMsg": ""
}

Create/Import TM in internal format
Purpose	Import and unpack base64 encoded archive

, } TM already exists: { "ReturnValue": 1, "ErrorMsg": "" }

Create/Import TM in internal format

Purpose

Import and unpack base64 encoded archive of .TMD, .TMI, .MEM files. Rename it to provided name

Request

POST /%service%/%tm_name%/

Params

{ "name": "examle_tm", "sourceLang": "bg-BG" , "data":"base64EncodedArchive" }

Code Block

language	js
title	Response
collapse	true

Request example:{ "name": "mem_internal_format", "sourceLang": "bg-BG", "data":"UEsDBBQACAgIAPmrhVQAAAAAAAAAAAAAAAAWAAQAT1RNXy1JRDE3NS0wXzJfNV9iLk1FTQEAAADtzqEKgDAQgOFTEHwNWZ5swrAO0SBys6wfWxFBDILv6uOI2WZQw33lr38GbvRIsm91baSiigzFEjuEb6XHEK\/myX0PXtXsyxS2OazwhLDWeVTaWgEFMMYYY\/9wAlBLBwhEWTaSXAAAAAAAAAAACAAAAAAAAFBLAwQUAAgICAD5q4VUAAAAAAAAAAAAAAAAFgAEAE9UTV8tSUQxNzUtMF8yXzVfYi5UTUQBAAAA7d3Pa5JxHMDxz+Ns09phDAYdPfaDyQqWRcYjS9nGpoYZhBeZMCISW2v2g5o6VkqQONk\/0KVzh4IoKAovnboUo1PHbuuwU8dSn8c9Pk2yTbc53y+R5\/P9fL7P1wf5Ps9zep5vIOy3iMiSiPLn0yPrQ7In+rStTQARi\/bV9chEyHcxGPIKAGDnPonl21SsHNmUYNgfHZ70nnKNDo9ET0dHozFn2L+Ll9uxZPzazPz1mYQAAAAAAAAAAAAAAAAAAAAAAAAAANDtBkXRoj5Zk7OqSFZ9q35Vn6khNa6W2wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAdBKbKHK4Em1omT5DxV6J7FrmkKFypBKt9FczvYaKtr+2DLpiqPTWVayGiq2uYjFUpC7VI6aElN8F8JPn\/QEAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2ANW7U0Ag9Iv60MnT4j8uLBZ\/X5+7dxn1ztX6Uy5AgAAAAAAAAAAAAAAAAAAgA6nL1qFjmc1rAO2IwNN9bL9u4ulVUeEfcQqQAfxSNtltshZaytB7jalZZ2a5KhFGT3Qr\/ztv1pkzAnP1v06+F7UxL22tRzSNf6aFq08MdoiY078\/znmkTZo5Qm2YdoOSLSyDdbaVUop\/Cj3cDm14I6\/uqf++nDUN1u4lS+k9MbKXL4QK72+775U+phOpp8sucdK728X5nK5hVT+weJqbTiHjMiNzWG1yNxWvI8rvxZ9cTfycj71NH1nsZgbf54uJlKryWy6GFlueBT6xHrzJRupDqkPXc9eyyduJmbLkf6\/mlYRDgQDPtO++3\/uYvsazANfYHx68vLEsSvOKedxqa\/hAGowD4Jh\/1X\/dH1X5sEBZpoH6E6\/AVBLBwj3gRyzjAIAAAAAAAAAAAEAAAAAAFBLAwQUAAgICAD5q4VUAAAAAAAAAAAAAAAAFgAEAE9UTV8tSUQxNzUtMF8yXzVfYi5UTUkBAAAA7d3PS9NhHMDxz\/Y1nbp0zfw2Vw6CEjooJkkFPs9DZZaFCiIRHRxKoJUIFXk06iB0kS5Fvw6dhDp28FDgOSqiIKQ\/ICQMhIIuYVnJt2f7eK2M2Ps1xp49b8Y+fP6ArXegJy4iV0RiPx6BNAXyT6ysrKhXlLZ49PwlkKP9hw\/19XcKAOD3PZX42+PDP0+JWN9AT765u3P33vbm1nxbvj0\/3DLQ0y3r5uClsZGhC2eGxgUAAAAAAAAAAAAAAAAAAAAAAAAAgFKXllh0ahQbLHeInDb3Xc6NWrF77Jibcr22zC2YY6bVLNoX5qp97Pa5SbPc8ci8sqHpd1k7a2+ZN+6eFQAAAAAAAAAAAAAAAAAAAAAAAAAAAAD4YxISk8bVUyq6eVa905dtqtxO3fBlqyqnkrW+ZFVZCGp8aVDl9ZeELxlVjhRNsEWVa+UffAlVuf78rC\/1eoK20JfNqnzt3OhLnSp1DZW+bFJl\/467vqRUuVxV5UutKts\/JX2pUWUyXvie9OopE5U7QWEHSfWZXdmPvlSr8i75xJcqVT7fPOdLpSqj5+t9Sahy8UBhOxWqLEph6nJVHhZNvUFPXbS3MlXyYWFvgSon3xf2FldlpGiCmCoPiiYQVbLR3or\/ZT0tS04AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMC6K4t+ZSAtOWkKQpOSeTfnZty0m3CDrsu1uNB9swv2pZ21IlN23J6w1uZsuV0y82bOzJhpM2EGTZdpMaERAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPjrUmteK0RypXifid5n1tyX6j7+9\/vvUEsHCGo104BhAgAAAAAAAAAAAQAAAAAAUEsBAgAAFAAICAgA912FVERZNpJcAAAAAAgAABYABAAAAAAAAAAAALSBAAAAAE9UTV8tSUQxNzUtMF8yXzVfYi5NRU0BAAAAUEsBAgAAFAAICAgA\/F2FVPeBHLOMAgAAAAABABYABAAAAAAAAAAAALSBrAAAAE9UTV8tSUQxNzUtMF8yXzVfYi5UTUQBAAAAUEsBAgAAFAAICAgA\/F2FVGo104BhAgAAAAABABYABAAAAAAAAAAAALSBiAMAAE9UTV8tSUQxNzUtMF8yXzVfYi5UTUkBAAAAUEsGBiwAAAAAAAAAHgMtAAAAAAAAAAAAAwAAAAAAAAADAAAAAAAAANgAAAAAAAAAOQYAAAAAAABQSwYHAAAAABEHAAAAAAAAAQAAAFBLBQYAAAAAAwADANgAAAA5BgAAAAA=" }
Response example:{
"name": "examle_tm"
}

TM already exists:
{
  "ReturnValue": 65535,
  "ErrorMsg": ""
}

...

Clone TM localy

Purpose

Creates TM with the provided name

Request

Post /%service%/%tm_name%/clone

Params

Required: name, sourceLang

Code Block

language	js
title	Response
collapse	true

Request example 
{    "newName": "examle_tm"
}

Response example:
Success:
ID111-0_4_9_cloned_2 
{
    "msg": "newBtree3_cloned2 was cloned successfully",
    "time": "5 ms"
}

 Failure: 
{
'dstMemPath    "ReturnValue": -1,
    "ErrorMsg": "'dstTmdPath' = /home/or/.t5memory/MEM/ID111-0_4_9newBtree3_cloned_2.MEMTMD already exists; for request for mem newBtree3; with body = {\n    \"newName\": \"newBtree3_cloned\"\n}"
}

Delete TM

Purpose

Deletes .TMD, .TMI, .MEM files

Request

Delete /%service%/%tm_name%/

Params

-

Code Block

language	js
title	Response
collapse	true

Response example:%empty_anyway%:
success:
{
    "newBtree3_cloned2": "deleted"
}

Code Block

language	js
title	Response
collapse	true

Response example:%empty_anyway%:
failed:
{
    "newBtree3_cloned2": "not found"
}

Import provided base64 encoded TMX file into TM

Purpose

Import provided base64 encoded TMX file into TM. Starts another thead for import. For checking import status use status call

Request

POST /%service%/%tm_name%/import

Params

{"tmxData": "base64EncodedTmxFile" }

Code Block

language	js
title	Response
collapse	true

Request example:{
  "tmxData":   "PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPHRteCB2ZXJzaW9uPSIxLjQiPgogIDxoZWFkZXIgY3JlYXRpb250b29sPSJTREwgTGFuZ3VhZ2UgUGxhdGZvcm0iIGNyZWF0aW9udG9vbHZlcnNpb249IjguMCIgby10bWY9IlNETCBUTTggRm9ybWF0IiBkYXRhdHlwZT0ieG1sIiBzZWd0eXBlPSJzZW50ZW5jZSIgYWRtaW5sYW5nPSJlbi1HQiIgc3JjbGFuZz0iYmctQkciIGNyZWF0aW9uZGF0ZT0iMjAxNTA4MjFUMDkyNjE0WiIgY3JlYXRpb25pZD0idGVzdCIvPgogIDxib2R5PgoJPHR1IGNyZWF0aW9uZGF0ZT0iMjAxODAyMTZUMTU1MTA1WiIgY3JlYXRpb25pZD0iREVTS1RPUC1SNTlCT0tCXFBDMiIgY2hhbmdlZGF0ZT0iMjAxODAyMTZUMTU1MTA4WiIgY2hhbmdlaWQ9IkRFU0tUT1AtUjU5Qk9LQlxQQzIiIGxhc3R1c2FnZWRhdGU9IjIwMTgwMjE2VDE2MTMwNVoiIHVzYWdlY291bnQ9IjEiPgogICAgICA8dHV2IHhtbDpsYW5nPSJiZy1CRyI+CiAgICAgICAgPHNlZz5UaGUgPHBoIC8+IGVuZDwvc2VnPgogICAgICA8L3R1dj4KICAgICAgPHR1diB4bWw6bGFuZz0iZW4tR0IiPgogICAgICAgIDxzZWc+RXRoIDxwaCAvPiBkbmU8L3NlZz4KICAgICAgPC90dXY+CiAgICA8L3R1PgogIDwvYm9keT4KPC90bXg+Cg=="
}Response example:Error in case of errorFrom v0_2_15
{ "%tm_name%":"deleted"} in case of success
Check status of import using status call

...

Export in internal format

Purpose

Creates and exports archive with .TMD, .TMI, .MEM files of TM

Request

GET /%service%/%tm_name%/

Headers

application/zip

returns archive(.tm file) consists with .tmd and .tmi files

Code Block

language	js
title	Response
collapse	true

Response example:%binary_data%

...

Get the status of TM

Request

GET /%service%/%tm_name%/status

Params

-

Would return status of TM. It could be 'not found', 'available' if it's on the disk but not loaded into the RAM yet, and 'open' with additional info. In case if there was at least one try to import tmx or reorganize tm since it was loaded into the RAM, additional fields would appear and stay in the statistics till memory would be unloaded.

Code Block

title
language	js

Code Block

language	js
title	Response
collapse	true

Response example:
{//just opened tm
	, without import\reorganize called
    "status": "open",
      "tmxImportStatuslastAccessTime": "available",
      "lastAccessTimecreationTime":  "%lastAccessTime20230703T122212Z" }
{//not opened but available tm
	"status,
    "tmCreatedInT5M_version": "available0:5:1"
}

{//not openedafter butreorgainize availablewas tm
	called
    "status": "not foundopen",
}

The  tmxImportStatus could be"reorganizeStatus": "available",
    "importreorganizeProgress" or: 100,
    "failedreorganizeTime": if"Overall thereorganize importtime hadis errors. If there were at least one import to that tm, new fields would appear
{//tm in process of import
    "status: 0:00:02\n",
    "segmentsReorganized": 1739,
    "invalidSegments": 2,
    "reorganizeErrorMsg": "open",
      "tmxImportStatuslastAccessTime": "available", 
     "importProgresscreationTime" : 56,    "importTime"20230703T122212Z", 
    "tmCreatedInT5M_version": "000:005:13",    "segmentsImported": 1356,    "invalidSegments": 23,    "invalidSymbolErrors": 2,    "importErrorMsg": "", 
    "lastAccessTime":  "%lastAccessTime"}

Fuzzy search

Purpose

Returns enrties\translations with small differences from requested

Request

POST /%service%/%tm_name%/fuzzysearch

1"
}


{//not opened but available on the disk 
	"status": "available"
}
{//not found tm {
    "status": "not found",
    "res": 48 // 48- both tmi and tmd files are no found, 16- only TMD file not found, 32 - only TMI file not found
}
  The tmxImportStatus could be "available", "import" or "failed" if the import had errors. If there were at least one import to that tm, new fields would appear
{//tm in process of import
    "status": "open",
    "tmxImportStatus": "import",    
	"importProgress" : 56,    
	"importTime": "00:00:13",    
	"segmentsImported": 1356,    
	"invalidSegments": 23,    
	"invalidSymbolErrors": 2,    
	"importErrorMsg": "", 
    "lastAccessTime":  "%lastAccessTime",
    "creationTime": "20230703T122212Z",
    "tmCreatedInT5M_version": "0:5:1" 
}

Required: source, sourceLang, targetLang

iNumOfProposal - limit of found proposals - max is 20, if 0 → use default value '5'

Fuzzy search

Purpose

Returns enrties\translations with small differences from requested

Request

POST /%service%/%tm_name%/fuzzysearch

Params

Required: source, sourceLang, targetLang

iNumOfProposal - limit of found proposals - max is 20, if 0 → use default value '5'

Code Block

language	js
title	Response
collapse	true

Request example:


Request example:
{  "sourceLang":"en-GB",   "targetLang":"de",   "source":"For > 100 setups.",   ["documentName":"OBJ_DCL-0000000845-004_pt-br.xml"],   ["segmentNumber":""],   ["markupTable":"OTMXUXLF"],   ["context":"395_408"],   ["numOfProposals":20],   ["loggingThreshold": 0]}





Response example:

Success:
{
  "ReturnValue": 0,
  "ErrorMsg": "",
  "NumOfFoundProposals": 1, 
  "results": [
  {
     "source": "For > 100 setups.",
     "target": "Für > 100 Aufstellungen.",
     "segmentNumber": 10906825,
     "id": "",
     "documentName": "none",
    "documentShortName": "NONE",
    "

Params

Code Block

language	js
title	Response
collapse	true

Request example:


Request example:
{  "sourceLang":"en-GB",   "targetLang":"de",   "source":"For > 100 setups.",   ["documentName":"OBJ_DCL-0000000845-004_pt-br.xml"],   ["segmentNumber":""],   ["markupTable":"OTMXUXLF"],   ["context":"395_408"],   ["numOfProposals":20],   ["loggingThreshold": 0]}





Response example:

Success:
{
  "ReturnValue": 0,
  "ErrorMsg": "",
  "NumOfFoundProposals": 1, 
  "results": [
  {
     "source": "For > 100 setups.",
     "target": "Für > 100 Aufstellungen.",
     "segmentNumber": 10906825,
     "id": "",
     "documentName": "none",
    "documentShortName": "NONE",
    "sourceLang": "en-GB",
    "targetLang": "de-DE",
    "type": "Manual",
    "matchType": "Exact",
    "author": "",
    "timestamp": "20190401T084052Z",
    "matchRate": 100,
    "markupTable": "OTMXML",
    "context": "",
   "additionalInfo": ""
   }
 ]
}

Not found:

{
"ReturnValue": 133,
"ErrorMsg": "OtmMemoryServiceWorker::concordanceSearch::"
}

Concordance search

PurposeReturns entries\translations that contain requested segmentRequestPOST /%service%/%tm_name%/concordancesearchParams

Required: searchString - what we are looking for , searchType ["Source"|"Target"|"SourceAndTarget"] - where to look

iNumOfProposal - limit of found proposals - max is 20, if 0 → use default value '5'

Code Block

language	js
title	Response
collapse	true

Request example:
{
    "searchString": "The",
    "searchType": "source",
    ["searchPosition": "",]
    ["numResults": 20,]
    ["msSearchAfterNumResults": 250,]
	["loggingThreshold": 0]
}
Response example:Success:
{
  "ReturnValue": 0,
  "NewSearchPosition": null,
  "results": [
  {
     "source": "For > 100 setups.",
     "target": "Für > 100 Aufstellungen.",
     "segmentNumber": 10906825,
     "id": "",
     "documentName": "none",
     "documentShortName": "NONE",
     "sourceLang": "en-GB",← rfc5646 
    
     "targetLang": "de-DE",← rfc5646
     "type": "Manual",
     "matchType": "undefinedExact",
     "author": "",
     "timestamp": "20190401T084052Z",
     "matchRate": 0100,
     "markupTable": "OTMXML",
     "context": "",
     "additionalInfo": ""
   }
  ],
 "ErrorMsg": ""
}

Success, but with NewSearchPosition - not all TM was checked, use this position to repeat search:Not found:

{
  "ReturnValue": 0133,
  "NewSearchPositionErrorMsg": "OtmMemoryServiceWorker::concordanceSearch::"
}

Concordance search

Purpose

Returns entries\translations that contain requested segment

Request

POST /%service%/%tm_name%/concordancesearch

Params

Required: searchString - what we are looking for , searchType ["Source"|"Target"|"SourceAndTarget"] - where to look

iNumOfProposal - limit of found proposals - max is 20, if 0 → use default value '5'

Code Block

language	js
title	Response
collapse	true

Request example:
{
    "searchString": "The",
    "searchType": "source",
    ["searchPosition": "",]
    ["numResults": 20,]
    ["msSearchAfterNumResults": 250,]
	["loggingThreshold": 0]
}
Response example:Success:
{
  "ReturnValue": 0,
  "NewSearchPosition": null,
  "results": [
  {
     "source": "For > 100 setups.",
     "target": "Für > 100 Aufstellungen.",
     "segmentNumber": 10906825,
     ""8:1",
  "results": [
  {
     "source": "For > 100 setups.",
     "target": "Für > 100 Aufstellungen.",
     "segmentNumber": 10906825,
     "id": "",
     "documentName": "none",
     "documentShortName": "NONE",
     "sourceLang": "en-GB",
← rfc5646     
     "targetLang": "de-DE",← rfc5646
     "type": "Manual",
     "matchType": "undefined",
     "author": "",
     "timestamp": "20190401T084052Z",
     "matchRate": 0,
     "markupTable": "OTMXML",
     "context": "",
     "additionalInfo": ""
   }
  ],
 "ErrorMsg": ""
}

SearchPositionSuccess, /but NewSearchPositionFormat: "7:1"
First is segmeng\record number, second is target number
The NextSearchposition is an internal key of the memory for the next position on sequential access. Since it is an internal key, maintained and understood by the underlying memory plug-in (for EqfMemoryPlugin is it the record number and the position in one record),
no assumptions should be made regarding the content. It is just a string that, should be sent back to OpenTM2 on the next request, so that the search starts from there.
So is the implementation in Translate5: The first request to OpenTM2 contains SearchPosition with an empty string, OpenTM2 returns than a string in NewSearchPosition, which is just resent to OpenTM2 in with NewSearchPosition - not all TM was checked, use this position to repeat search:
{
  "ReturnValue": 0,
  "NewSearchPosition": "8:1",
  "results": [
  {
     "source": "For > 100 setups.",
     "target": "Für > 100 Aufstellungen.",
     "segmentNumber": 10906825,
     "id": "",
    "documentName": "none",
    "documentShortName": "NONE",
    "sourceLang": "en-GB",
    "targetLang": "de-DE",
    "type": "Manual",
     "matchType": "undefined",
     "author": "",
     "timestamp": "20190401T084052Z",
     "matchRate": 0,
     "markupTable": "OTMXML",
     "context": "",
     "additionalInfo": ""
   }
  ],
 "ErrorMsg": ""
}
SearchPosition / NewSearchPositionFormat: "7:1"
First is segmeng\record number, second is target number
The NextSearchposition is an internal key of the memory for the next position on sequential access. Since it is an internal key, maintained and understood by the underlying memory plug-in (for EqfMemoryPlugin is it the record number and the position in one record),
no assumptions should be made regarding the content. It is just a string that, should be sent back to OpenTM2 on the next request, so that the search starts from there.
So is the implementation in Translate5: The first request to OpenTM2 contains SearchPosition with an empty string, OpenTM2 returns than a string in NewSearchPosition, which is just resent to OpenTM2 in the next request.

Not found:{
"ReturnValue": 0,
"NewSearchPosition": null,
"ErrorMsg": ""
}TM not found:{
"ReturnValue": 133,
"ErrorMsg": "OtmMemoryServiceWorker::concordanceSearch::"
}

...

Save all TMs

Purpose

Flushes all filebuffers(TMD, TMI files) into the filesystem. Reset 'Modified' flags for file buffers.

Filebuffer is a file instance of .TMD or .TMI loaded into RAM. It provides better speed and safety when working with files.

Request

GET /%service%_service/savetms

Params

-

Code Block

language	js
title	Response
collapse	true

Response example:{
   'saved 4 files': '/home/or/.t5memory/MEM/bg_internal_formatmem2.TMD;, /home/or/.t5memory/MEM/bg_internal_formatmem2.TMI;, /home/or/.t5memory/MEM/mem_gt_issue.TMD;newBtree3.TMD, /home/or/.t5memory/MEM/mem_gt_issuenewBtree3.TMI; EQFSYSW.PRP; '
}
 List of saved files

Shutdown service

Purpose

Safely shutting down the service with\without saving all loaded tm files to the disk

Request

GET /%service%_service/shutdown?dontsave=1

Params

dontsave=1(optional in address) - skips saving tms, for now value doesn't matter, only presence

If try to save tms before closing, would check if there is still import process going on
If there is some, would wait 1 second and check again.
Repeats last step up to 10 min, then closes service anyway.

Code Block

language	js
title	Response
collapse	true

Response example:%Empty%

...

Openning and closing TM

In first concept it was planned to implement routines to open and close a TM. While concepting we found some problemes with this approach:

First one is the realization: opening and closing a TM by REST would mean to update the TM Resource and set a state to open or close. This is very awkward.
Since in translate5 multiple tasks can be used to the same time, multiple tasks try to access one TM. Closing TMs is getting complicated to prevent race conditions in TM usage.
Since OpenTM2 loads the whole TM in memory, OpenTM2 must control itself which TMs are loaded or not.

This leads to the following conclusion in implementation of opening and closing of TMs:

OpenTM2 has to automatically load the requested TMs if requested. Also OpenTM2 has to close the TMs after a TM was not used for some time. That means that OpenTM2 has to track the timestamps when a TM was last requested.

Concept endpoints, not implemented

http://opentm2/translationmemory/[TM_Name ]/openHandle

GET – Opens a memory for queries by OpenTM2

Note: This method is not required as memories are automatically opened when they are accessed for the first time.

http://opentm2/translationmemory/[TM_Name ]/openHandle

DELETE – Closes a memory for queries by OpenTM2

Note: This method is not required as memories are automatically opened when they are accessed for the first time.

For now we open TM in case of call to work with it. TM stays opened till the shutdown we wouldn't try to open more TM's, exceeding the RAM limit setupped in config file.
In that case we would close TM in order of longest not used, till we would fit in limit including TM that we try to open.
TM size is calcucated basicaly as sum .TMD and .TMI files
Ram limit doesn't include service RAM and temporary files

TM files structure and other related info

Info below is actual for version 0_5_x

Starting from version 0_5_0 .mem file is excluded from TM files - tm now consists only with .tmd and .tmi files. That files have 2kb headers which have some useful information, like creation date and version in which that file was created. In general, changing mid_version number means binary incompatible files. During reorganize there would be created new empty tm and then segments would be reimported from previous, and then old files would be deleted and new ones would be renamed to replace old files. That means that reorganize would also update creation t5memory version of files to the newest.

TM file is just archive with tmi and tmd files.

tmd and tmi files should be flushed in a safe way - saved on disk with temporary filename and then replacing old files.(Should be implemented)

There is tmmanager(as singletone) which have list of tm, and one tm instance have two binary trees(for both (tmd)data and (tmi)index files), with each have own filebuffer instance(before there used to be a pool of filebuffers and it's files operation functions, like write, read, close and open was handling requests).

Request handler - it's an instance of class in request handler hierarhy classes. For each type of requests there is class to handle it. In general it have private functions "parseJSON"(would parse json if provided and would return error if json is invalid), "checkData"(whould check if all required fields was provided), "requestTM"(would request readOnly, write or service tm handlers. It would load tm if it is not loaded in RAM yet) and "execute" - original requests code. And also it has public function "run" which is stategy template to operate listed private function.

The TMs is saved in TMManager using smart pointers(it's pointer which track references to itself and call destructor automaticaly). That means that on request it's possible to clear list from some TM, while it would still be active in other thread(like in fuzzy search). Then ram would be freed at the end of last request handling that TM.
In case if in the middle of some request(like fuzzy search) there was a call to delete tm, first we clear TMlist(but we keep smart pointer in fuzzy requests thread, so this is not calling destructor yet, but would after fuzzy request would be done). Destructor would try to flush filebuffer into filesystem but because there is no files in the disk, filebuffers would not create them again and it would just clean the RAM(in that case log would be writen about filebuffer flush not founding file in the folder).

From TMManager, request could ask for one of 3 types of tm handers - readonly, write or service. ReadOnly\write requests here have it's name from inside-tm perspective(so operations with tm files in filesystem is service requests).
ReadOnly(concordance search, fuzzy search, exportTmx) would be provided if there is no write handlers, for write handlers(deleteEntry, updateEntry, importTmx) there should be no other write handlers and no readOnly handlers. Service handlers could mean different for different requests. For example status request should be able to access something like readonly handler, but it shouldn't be blocked if there is any write requests, since it's used for checking import\reorganize status and progress. For some filesystem requests(deleteTM, createTM, cloneTM, importTM, exportTM(internal format)) there should be other blocking mechanism, since most of them even doesn't require to load tm into the ram.

In case if tm is not in RAM, requesting handler from TMManager would try to load TM into the RAM, considering RAM limit explained in this document.

TAG REPLACEMENT

Tag replacement

...

Page tree

Versions Compared

Old Version 31

New Version 32

Key

Endpoints overview

List of TMs

Create TM

Create TM

Create/Import TM in internal format

Create/Import TM in internal format

Clone TM localy

Delete TM

Import provided base64 encoded TMX file into TM

Export in internal format

Get the status of TM

Fuzzy search

Fuzzy search

Concordance search

Concordance search

Save all TMs

Shutdown service

Openning and closing TM

TM files structure and other related info

TAG REPLACEMENT

Tag replacement

Page tree

Page History

Versions Compared

Old Version 31

New Version 32

Key

Endpoints overview

List of TMs

Create TM

Create TM

Create/Import TM in internal format

Create/Import TM in internal format

Clone TM localy

Delete TM

Import provided base64 encoded TMX file into TM

Export in internal format

Get the status of TM

Fuzzy search

Fuzzy search

Concordance search

Concordance search

Save all TMs

Shutdown service

Openning and closing TM

TM files structure and other related info

TAG REPLACEMENT

Tag replacement