Pretranslate-only corpora have no quote convention

[Enkidu93]

Because Serval analyzes the quote convention at a parallel corpus level using only the chapters/books selected for training on the target side, if a parallel corpus is only used for inferencing, no quote convention is detected. Also, if the target filter specified is not the same as source filter since we will analyze the corpus based on the target filter, the user may have an unexpected experience. In one sense, these are both particularities of SF's use of Serval. We need to decide what reasonable behavior in Serval is, add it to the documentation, and then request changes to SF as needed.

[Enkidu93]

This is preventing quote denormalization from working on SF. We could temporarily remove the book filtering if we think this is urgent enough.

[ddaspit]

So, what should we be analyzing? Both the training and inferencing corpora? Just the inferencing corpus?

[Enkidu93]

For context, this issue emerged from this thread in Slack: https://sil-language-software.slack.com/archives/C07PH69AC1F/p1766179281939389?thread_ts=1766008927.310609&cid=C07PH69AC1F.

So, what should we be analyzing? Both the training and inferencing corpora? Just the inferencing corpus?

I think we either need to ask SF to use a single parallel corpus OR we need to analyze the convention at a different level (e.g. build, engine, monolingual corpus, or file). The problem is this: SF creates two parallel corpora for a given engine - one for training and one for inferencing - even though they often reference the same projects/monolingual corpora. During the build job, the target quote convention is analyzed correctly for the training parallel corpus but not for the inferencing parallel corpus because nothing is filtered for training. When SF goes to fetch the USFM, it specifies the id of the inferencing parallel corpus which is then used to look up the quote convention but none exists since the successful analysis is only associated with the training parallel corpus.

My instinct is that associating the analysis with the build is probably the best direction if we want to analyze the convention at a different level. If we associate it with the engine, file, or monolingual corpus, we may have more of a problem dealing with the analysis becoming out-dated.

An easier (maybe short-term) solution would be to grab the first non-empty target quote convention among the parallel corpora quote convention analyses when fetching USFM. Since the target project is likely to be the same across parallel corpora, this is probably safe to do. Of course, we could always document it in swagger.

What do you guys think, @ddaspit @pmachapman?

[pmachapman]

I think we either need to ask SF to use a single parallel corpus

I think that will only work when the training and inferencing source corpora are the same? Not all projects are set up that way, and some projects will have two training source corpora.

My instinct is that associating the analysis with the build is probably the best direction if we want to analyze the convention at a different level.

Yes, I think so too. The training and inferencing parallel corpora currently have the same target corpus, although that could possibly change in future. Maybe we need multiple values, i.e. per corpus referenced in each parallel corpus in each build? That might get confusing though...

[Enkidu93]

I think that will only work when the training and inferencing source corpora are the same? Not all projects are set up that way, and some projects will have two training source corpora.

Since the ParallelCorpusFilter filters at a monolingual corpus level, I do think you could manage having just one parallel corpus, but you would have to be explicit and careful in your filtering. And if the target corpus is almost always the same and the drafting and training source corpora are often the same, then I could see using one parallel corpus making sense. I don't think Serval's current behavior is unreasonable, but it's certainly easier to adjust Serval's behavior than migrate all of the SF corpora haha. Regardless, we definitely need to add some testing for this set-up with two parallel corpora.

[ddaspit]

Should we have a single convention that is computed across all of the target monolingual corpora in a build, or should we compute a convention for each target monolingual corpus separately?

The convention is used to denormalize the quotation marks at a model-level, which would indicate that we should have a single convention for the build, which is computed using all of the target-side training data.

[ddaspit]

Also, the quotation denormalization should work irrespective of how the calling application has setup the parallel corpora.

[Enkidu93]

Well, the complication is that you specify a parallel corpus id when fetching USFM. It is possible that you could have multiple parallel corpora on an engine with different quote conventions but only want the convention to take effect which is relevant to the build that produced the pretranslations for the parallel corpus whose id you supply. But there may just not be clean a way to juggle that case and SF's usage.

We could examine the build itself in the USFM endpoint and grab the target quote convention of the parallel corpus that was used for training rather than the one specified by the client (?). It would add some complexity, but I think it would work correctly.

[ddaspit]

Is it possible to compute the convention across multiple corpora? I think what we want is to compute a single convention using all of the training data. We would only want to include training data in the target language of the engine.

[Enkidu93]

Is it possible to compute the convention across multiple corpora? I think what we want is to compute a single convention using all of the training data. We would only want to include training data in the target language of the engine.

Absolutely! With the recent updates to Machine, we can combine analyses. I think maybe we just need to talk about this face-to-face. There are a lot of different considerations depending on what kinds of multi-parallel-corpus set-ups we want to support - e.g., if you have different target corpora across the parallel corpora with different quote conventions, you may want to only register the quote convention of whatever corpus you're inferencing on; or what if the source training data is filtered by book but the target is unfiltered - ultimately, we will only train on the parallel data, but should we analyze only where there is parallel data or just on all target data?

[Enkidu93]

I would say that this issue is very high priority since it's keeping QD from working at all. Do we want to push a hotfix in the short term while we ruminate on the best long-term solution? Options for a quick fix could be:

• Remove the book-level filtering when analyzing the quote convention (this may hamper the analysis)
• Fallback to any non-empty target quote convention among parallel corpora associated with the build/engine when denormalizing (for the way that SF is using Serval, this should result in the correct behavior)

[ddaspit]

I like the idea of a hotfix. I would go with the second option of falling back to the other parallel corpus.