AI Langauge Translation for Stake Conference etc.

[wgotzmann]

I thought I would start a New topic since my ideas on closed captions is really a different subject
I have been looking pondering what might be done for some time. I have felt some inspiration to look in a new direction.

The world is changing and new tools are coming , I tend to push things a little. LOL

kenaycock suggesting looking at Microsoft Translator - I had dismissed this before, but things change. I looked at it capabilities again. This time I understood a lot more and saw a way to useit.

I found a way to use a phone as a source and some laptops as a Language translator, to then feed this into the standard Zoom interpretation function. (Outside interpretors)

Well it has worked very well - in my test runs. "Two Languages" We can add as many languages as we have Laptops and Internet bandwidth. Since we only need audio I turn off the video feed and transmission. This reduced the data use.
There are several voices for the spoken languages which are remarkable

I can share how, if people are interested? when I have documented the process

It is a little complicated but can be done with some middle software to connect Microsoft Translate to Zoom - but doable.

I realise this is not human translation but that is really "interpretation" - based on individuals. This is not consistent and also difficult to arrange. It is surprisingly good with both content and context
Having something is much better than nothing in my opinion.

.
The Microsoft engine used is based on GPT 4.0 and will only get better. I believe this is a very important tool to help people become more connected and part of wards and Stakes
The translation device which I had originally intended, "is costly" but does work a little more easily Time Kettle X1

Thanks

[wgotzmann]

Note thi sis Spoken language translation - I am referring to!

[dave.swartz]

Please document the process you have used. I have experimented with using Power Point to capture the microphone audio (based on what I saw at the Central Point Oregon stake) and to in real time put a text translation at the bottom of the display which is then passed to OBS studio as an overlay or chroma key over the video feed and then passing this to Zoom. Translation (Samoan -> English and then English-> Samoan) on the same screen starts getting a bit complex since you need two computers to generate the text and a third computer to put all the overlays together.

If you have a pure audio approach so much the better. Yes please share with us.

[BrianEdwards]

Also very interested in the details.

[kenaycock]

Wow, this sounds neat. We are most interested in Haitian Creole for which there is not currently a spoken language option. I am still looking into options to cleanly capture the translated captions for video overlay. There is a Microsoft Translator API with Azure... Not sure I want to get that involved just yet though.

Thanks, all, for the ideas!

P.S., for the adventurous, I also experimented with combining Whisper+NLLB to perform realtime transcription and translation locally. There are too many bugs to work out (e.g., random profanity, need for a higher-end workstation), but I think a local solution will be practical in the near future.
Demo (see description for details):
https://youtu.be/aElcY9hNKX4