Translate doesn't start in some instances

[DavidMarquezF]

I've seen that the service, in some instances doesn't even start translating:

Submitted: 223 documents, 417188 bytes.
09:47:48.0863709	Status: NotStarted	In progress: 0	Success: 0	Fail: 0
09:47:50.7703304	Status: NotStarted	In progress: 0	Success: 0	Fail: 0

From this, it doesn't print anything else

The heartbeat still works though:

Translation Request submitted. Status: NotStarted
4.5524552 START - Translation service request with SAS token. StatusID: *************
5.7596823 Http status: 200 OK
5.7618146 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
6.979743 Http status: 200 OK
6.9816031 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
8.1819809 Http status: 200 OK
8.183962 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
9.385535 Http status: 200 OK
9.3875677 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
10.5896654 Http status: 200 OK
10.5915891 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
11.8077684 Http status: 200 OK
11.809774 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
13.0188996 Http status: 200 OK
13.0210184 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
14.2335563 Http status: 200 OK
14.2354616 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
15.4479572 Http status: 200 OK
15.4498513 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
16.6604599 Http status: 200 OK
16.6623357 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
The thread 0x67d4 has exited with code 0 (0x0).
The thread 0xa1d0 has exited with code 0 (0x0).
The thread 0x77f4 has exited with code 0 (0x0).
The thread 0xa694 has exited with code 0 (0x0).
17.8749834 Http status: 200 OK
The thread 0xaac8 has exited with code 0 (0x0).
The thread 0x38d0 has exited with code 0 (0x0).
17.879706 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
The thread 0x5ea4 has exited with code 0 (0x0).
The thread 0x9af8 has exited with code 0 (0x0).
The thread 0x98d8 has exited with code 0 (0x0).
The thread 0x4458 has exited with code 0 (0x0).
The thread 0x5f20 has exited with code 0 (0x0).
The thread 0x376c has exited with code 0 (0x0).
The thread 0x3374 has exited with code 0 (0x0).
The thread 0x45f4 has exited with code 0 (0x0).
The thread 0x68dc has exited with code 0 (0x0).
The thread 0x2370 has exited with code 0 (0x0).
The thread 0x5d0c has exited with code 0 (0x0).
The thread 0x1998 has exited with code 0 (0x0).
The thread 0x38bc has exited with code 0 (0x0).
The thread 0x5488 has exited with code 0 (0x0).
The thread 0x1e1c has exited with code 0 (0x0).
The thread 0x56d4 has exited with code 0 (0x0).
19.0858959 Http status: 200 OK
19.0873632 Service status: 23/06/2023 10:06:49 +00:00 NotStarted
The thread 0x813c has exited with code 0 (0x0).
20.3029608 Http status: 200 OK
....

This happens consistently when I'm trying to translate a high number of documents (200 for instance), but for lower amounts with the exact same settings (storage strings, token, glossaries, etc.) it works well.

If I check in Azure, the three blob containers have been created (glossary, target and source) and have all the files there. So it seams that for some reason the translator just doesn't start. Any ideas why this might happen??

[chriswendt1]

Possible that one of the limits that are listed here were exceeded, and the client-side code isn't catching it:
https://learn.microsoft.com/en-us/azure/cognitive-services/translator/service-limits#document-translation

Maybe the total content size of 250MB was exceeded?

I thought the log file should record the failure reason, if reported by the service. But maybe it isn't?

[chriswendt1]

Whether the service reports a failure or not: The client isn't currently able to work around these listed limits. Let me know if you can find which limit was exceeded. Then we can design a workaround, for instance split the job up into smaller chunks, inside the DocumentTranslationService run.

[DavidMarquezF]

Hi, thanks for the quick response!

It doesn't seem like I'm over the limits. I'm sending Submitted: 223 documents, 417188 bytes. which is nowhere close. There are no failure logs at all, it seems like the service doesn't even try to start. It's stuck at "Not started" status.

[chriswendt1]

I agree. It looks like you are way below the limits.
If you retry with the exact same set, does it continue to "Not start"?
How long are you waiting before declaring the job failed? In the above log you waited 20 seconds. Maybe that's too short?

[DavidMarquezF]

Yes it happens consistently. I've tried waiting up to 1 hour. I've noticed that if it doesn't start the operation withtin a few seconds it just means it will be stuck until the end of times. Has it ever happened to you?

[chriswendt1]

It has happened to me in the past. Microsoft said at the time that there was an intermittent service issue.

[DavidMarquezF]

Well it has been happening consistently for a few weeks now. Any other ideas of what i could be doing wrong?? The thing that I find weird is that there is no error thrown on Azure's side so I'm finding it very hard to understand what else I can d to fix it

[DavidMarquezF]

I was messing with it a bit more and it happens when I am uploading more than 19 documents (I don't know if the number of documents is the reason but it might be).

Consistenly:
Submitted: 19 documents, 19774 bytes. works properly

Submitted: 20 documents, 20616 bytes.doesn't work properly

Any documents less than 19 and it still works. Any documents more than 20 and it doesn't work. Maybe it has to do with the byte count, but I don't think so because my files are quite small. I've managed to translate the whole thing (the 223 documents) in batches of 15. It is not ideal but it works (so it has nothing to do with specific files)

I'm using Custom Translator, does it have different limits than the raw Document Translator?

[chriswendt1]

Hi @DavidMarquezF, can you please submit an Azure service request with the request ID of your failing transaction?

FYI I cannot repro, one try with a custom translation system and two tries with standard translation, using 23 small files.

[DavidMarquezF]

I will do that. Thanks!