Search is made segment-by segment, and it's checking segment if it fits selected filters. You can search for EXACT or CONCORDANCE matches in this fields:
source, target, document, author, addInfo, context
To set filter, use it's SearchMode field, otherwise filter would be disabled. So you have sourceSearchMode, targetSearchMode, documentSearchMode,
authorSearchMode, addInfoSearchMode, contextSearchMode
Search mode should be set explicitly to CONTAINS/CONCORDANCE or EXACT, otherwise filter would be ignored. But also each searchMode could have additional search parameters "CONTAINS, caseinsensetive, WHITESPACETOLERANT, INVERTED", all that values is not important, as well as delimiter. By default search is case sensetive. If you add Inverted option, check for that filter would be reverted.
To check how filters would be parsed, check json in responce. Field with that info could look like this: "Filters":"
Search filter, field: SOURCE FilterType::CONTAINS SearchStr: 'THE'; Options: SEARCH_FILTERS_NOT|SEARCH_CASEINSENSITIVE_OPT|SEARCH_WHITESPACETOLERANT_OPT|;\n
Search filter, field: TARGET FilterType::EXACT SearchStr: ''; Options: SEARCH_CASEINSENSITIVE_OPT|;\n
Search filter, field: ADDINFO FilterType::CONTAINS SearchStr: 'some add info'; Options: SEARCH_WHITESPACETOLERANT_OPT|;\n
Search filter, field: CONTEXT FilterType::EXACT SearchStr: 'context context'; Options: ;\nSearch filter, field: AUTHOR FilterType::CONTAINS SearchStr: ''; Options: ;\n
Search filter, field: DOCUMENT FilterType::CONTAINS SearchStr: 'evo3_p1137_reports_translation_properties_de_fr_20220720_094902'; Options: SEARCH_FILTERS_NOT|;\n
Search filter, field: TIMESTAMP FilterType::RANGE Range: 20000121T115234Z - 20240121T115234Z Options: ;\n", It's possible to apply filter just with SearchMode, like if you would type "authorSearchMode": "exact",but there would be no "author" field, it would look for segments, where author field is empty.
Also there are timespan parameter, to set it, use this fields and format: "timestampSpanStart":"20000121T115234Z",
"timestampSpanEnd":"20240121T115234Z", You should set both parameters to apply filter, otherwise you would get error as return. Check output to see how it was parsed and applied.
By default all mentioned filters is applied in logical and combination, but you can change that globaly with adding "logicalOr": 1,
Then all mentioned filters would be applied in logical or combination(please, use 1 to set this to true, boolean type is not supported by json parser in t5memory). Supported since 0.6.5
"onlyCountSegments":1 Instead of returning segments, just count them and return counter in "NumOfFoundSegments":22741
Also there are lang filters, they would always be applied to selection of segments that passed previous filters, so value of "logicalOr": 1, wouldn't be applied to that.
To set language filters, use this fields: "sourceLang":"en-GB",
"targetLang":"de", Lang filters could be applied with major lang feature, so source lang in this case would be applied as exact filter for source lang, but target lang would check if langs is in the same lang group. That check is done in languages.xml file with isPreferred flag.
Lang filters and if filters is combined in logical or or logical and you can check in GlobalSearchOptions field of responce. It could look like this: "GlobalSearchOptions":"SEARCH_FILTERS_LOGICAL_OR|SEARCH_EXACT_MATCH_OF_SRC_LANG_OPT, lang = en-GB|SEARCH_GROUP_MATCH_OF_TRG_LANG_OPT, lang = de", Other that you can send is: "searchPosition":"8:1",
"numResults":2,
"msSearchAfterNumResults":250
"loggingThreshold": 4 - check other requests,
So search position is position where to start search internaly in btree. This search is limited by num of found segment(set by numResults) or timeout(set by msSearchAfterNumResults), but timeout would be ignored in case if there are no segments in the tm to fit params. Max numResults is 200.
You can send empty json and search would work fine, but it would just return first 5 segments in tm
You can go through all segment with using this 2 fields
"searchPosition":"8:1",
"numResults":200 and just updating searchPosition with NewSearchPositionfrom responce.| Code Block |
|---|
| language | js |
|---|
| title | Response |
|---|
| collapse | true |
|---|
| {
"logicalOr": 1,
"source":"the",
"sourceSearchMode":"CONTAINS, CASEINSENSETIVE, WHITESPACETOLERANT, INVERTED",
"target":"",
"targetSearchMode":"EXACT, CASEINSENSETIVE",
"document":"evo3_p1137_reports_translation_properties_de_fr_20220720_094902",
"documentSearchMode":"CONTAINS, INVERTED",
"author":"some author",
"authorSearchMode":"CONTAINS",
"timestampSpanStart": "20000121T115234Z",
"timestampSpanEnd": "20240121T115234Z",
"addInfo":"some add info",
"addInfoSearchMode":"CONCORDANCE, WHITESPACETOLERANT",
"context":"context context",
"contextSearchMode":"EXACT",
"sourceLang":"en-GB",
"targetLang":"SV",
"searchPosition": "8:1",
"numResults": 2,
"msSearchAfterNumResults": 25,
"loggingThreshold": 3
}
So here search would be done in logical or way, so if any of source, target, document, context, author, timestamp filters returns true, result would be added to set, which then would be filtered out by sourceLang on exact match check and targetLang on groupLang check.
Search would start from position "8:1"(tm data start at "7:1" but if you wan't to start from the beggining, just avoid that param.
numResuts:2 - so if there would be 2 segments found, search would end
"msSearchAfterNumResults": 25 - 25ms after first found segment, search would end, even if more segments was found, responce would contain "NewSearchPosition": "10:1", which can be used in searchPosition to continue search
Response example:Success:
example{
"Filters": "Search filter, field: SOURCE FilterType::CONTAINS SearchStr: 'THE'; Options: SEARCH_FILTERS_NOT|SEARCH_CASEINSENSITIVE_OPT|SEARCH_WHITESPACETOLERANT_OPT|;\n
Search filter, field: TARGET FilterType::EXACT SearchStr: ''; Options: SEARCH_CASEINSENSITIVE_OPT|;\n
Search filter, field: ADDINFO FilterType::CONTAINS SearchStr: 'some add info'; Options: SEARCH_WHITESPACETOLERANT_OPT|;\n
Search filter, field: CONTEXT FilterType::EXACT SearchStr: 'context context'; Options: ;\n
Search filter, field: AUTHOR FilterType::CONTAINS SearchStr: ''; Options: ;\n
Search filter, field: DOCUMENT FilterType::CONTAINS SearchStr: 'evo3_p1137_reports_translation_properties_de_fr_20220720_094902'; Options: SEARCH_FILTERS_NOT|;\n
Search filter, field: TIMESTAMP FilterType::RANGE Range: 20000121T115234Z - 20240121T115234Z Options: ;\n",
"GlobalSearchOptions": "SEARCH_FILTERS_LOGICAL_OR|SEARCH_EXACT_MATCH_OF_SRC_LANG_OPT, lang = en-GB|SEARCH_GROUP_MATCH_OF_TRG_LANG_OPT, lang = sv",
"ReturnValue": 0,
"ReturnMessage": "FOUND",
"NewSearchPosition": "10:1",
"results": [
{
"source": "Congratulations on the purchase of a <ph x=\"101\"/> machine control system.",
"target": "Gratulerar till köpet av maskinstyrningsystemet <ph x=\"101\"/>.",
"segmentNumber": 5740419,
"id": "",
"documentName": "none",
"sourceLang": "en-GB",
"targetLang": "SV-SE",
"type": "Manual",
"author": "",
"timestamp": "20170327T091814Z",
"markupTable": "OTMXUXLF",
"context": "",
"additionalInfo": "",
"internalKey": "8:1"
},
{
"source": "The <ph x=\"101\"/> System is an ideal tool for increasing productivity in all aspects of the construction earthmoving industry.",
"target": "Systemet <ph x=\"101\"/> är ett verktyg som lämpar sig perfekt för att öka produktiviteten inom alla delar av bygg- och anläggningsområdet.",
"segmentNumber": 5740420,
"id": "",
"documentName": "none",
"sourceLang": "en-GB",
"targetLang": "SV-SE",
"type": "Manual",
"author": "",
"timestamp": "20170327T091814Z",
"markupTable": "OTMXUXLF",
"context": "",
"additionalInfo": "",
"internalKey": "9:1"
}
]
}
SearchPosition / NewSearchPositionFormat: "7:1"
First is segment\record number, second is target number
The NextSearchposition is an internal key of the memory for the next position on sequential access. Since it is an internal key, maintained and understood by the underlying memory plug-in (for EqfMemoryPlugin is it the record number and the position in one record),
no assumptions should be made regarding the content. It is just a string that, should be sent back to OpenTM2 on the next request, so that the search starts from there.
So is the implementation in Translate5: The first request to OpenTM2 contains SearchPosition with an empty string, OpenTM2 returns than a string in NewSearchPosition, which is just resent to OpenTM2 in the next request.
Not found:{
"ReturnValue": 0,
"NewSearchPosition": null,
"ErrorMsg": ""
}TM not found:{
"ReturnValue": 133,
"ErrorMsg": "OtmMemoryServiceWorker::concordanceSearch::"
} |
Here is search request with all possible parameters:
{
"logicalOr": 1, "source":"the", "sourceSearchMode":"CONTAINS, CASEINSENSETIVE, WHITESPACETOLERANT, INVERTED", "target":"", "targetSearchMode":"EXACT, CASEINSENSETIVE", "document":"evo3_p1137_reports_translation_properties_de_fr_20220720_094902", "documentSearchMode":"CONTAINS, INVERTED", "author":"some author",
"authorSearchMode":"CONTAINS", "timestampSpanStart": "20000121T115234Z", "timestampSpanEnd": "20240121T115234Z", "addInfo":"some add info", "addInfoSearchMode":"CONCORDANCE, WHITESPACETOLERANT", "context":"context context", "contextSearchMode":"EXACT", "sourceLang":"en-GB", "targetLang":"SV", "searchPosition": "8:1", "numResults": 2, "msSearchAfterNumResults": 25,
"loggingThreshold": 3
}
All fields is optional, but some depends on other, so error should be returned in case of not providing required field So request with this body would also work:
{
} | Parameter | valueType | default value | possible values | requireField | description |
|---|
| sourceLang | string | "" | langs that can
be matched to
langs in languages.xml | - | Filter segments on src/trg lang attribute, If specified lang is preffered, matching is done based on lang family,
otherwise on exact match | | targetLang | | searchPosition | string | "" (search would
start from "7:1"
then | "8:1" etc | point where to start search in tmd file | | numResults | int | 5 | (0....200] | points how many matches return in current request | | msSearchAfterNumResults | 0 | no check | sets how many ms should pass between first found segment and search stop, if it didn't reach the end yet. | | loggingThreshold | -1 | [0...6] | additional field to set log level on the run | | logicalOr | int | 0 | 0 for false, any other number as true,
example:
"logicalOr": 1,
"onlyCountSegments": 1 | by default source, target, document, author, context, addinfo, timestamp is combined in logical AND, but by sending here "OR" you can switch that to logical OR, any other value would left it in default AND state.
Doesn't apply to sourceLang and targetLang filters, they are always in AND state | | onlyCountSegments | instead of returning segment, would go in search till the end of tm and return total number of segments, that returns true with selected filters | | source | string | "" | any string, example
"source": "data in the segment" | sourceSearchMode | Sets what to look for in source of the segments, based on type of search, specified in sourceSearchMode(exact, concordance). If sourceSearchMode is not specified, returns an error. | | target | targetSearchMode | --//–(the same as above but for corresponding fields) | | document | documentSearchMode | | author | authorSearchMode | | context | contextSearchMode | | addInfo | addInfoSearchMode | | timestampSpanStart | string | string with date in format
"20240121T115234Z" | timestampSpanEnd | Sets filter for time. You need to provid both timestamps, or none, otherwise request would return an error. Could be used in "OR" combination in
"logicalOr": 1,, but, maybe, it's better to change that behaviour to similar like with langs(Always AND) | | timestampSpanEnd | timestampSpanStart | | sourceSearchMode | string | "" | String with required
EXACT or CONCORDANCE
(or CONTAINS, what's equal to CONCORDANCE)
words and some optional, like
CASEINSENSETIVE for non case sensetive comparison,
WHITESPACETOLERANT for
modifying whitespaces(result of this actions you can see in filters in responce)
INVERTED for applying filter in inverted state, so to return false on match and true
if no match. Logical NOT
Attributes is not case sensetive,
Separator doen't matters | - | Sets type of search for corresponding field.
If you set, for example, "authorSearchMode" = "EXACT", but don't provide any
author in request, author field would be "", so request would look for segments,
where author equals to "". The same is true for other fields
Examples:
1)
"source": "the text inside"
"sourceSearchMode":"CONTAINS, CASEINSENSETIVE, WHITESPACETOLERANT, INVERTED", - search would be for all segments, which doesn't contains "the text inside" in non case sensetive mode and with normalizing whitespaces.
2)
"author": "Ed Sheeran",
"authorSearchMode" = "Exact", -search would be done on exact case sensetive matches with "Ed Sheeran" in author field
3)
"author": "Ed Sheeran",
"authorSearchMode" = "CASEINSENSETIVE", - ERROR, search mode(Exact\Contains) is not selected 4) "author": "Ed Sheeran", - ERROR, search mode(Exact\Contains) is not selected
5) "authorSearchMode" = "CONTAINS",- OK, filter would check if segment contains "", so every segment would return true then
| | targetSearchMode | | documentSearchMode | | authorSearchMode | | contextSearchMode | | addInfoSearchMode |
|