EV3 Text-to-Speech with MaryTTS

Post your EV3 projects, project ideas, etc here!

Moderators: roger, gloomyandy, skoehler

Posts: 30
Joined: Sun Sep 08, 2013 4:33 pm

EV3 Text-to-Speech with MaryTTS

Postby fuzzycow » Thu Sep 26, 2013 2:57 pm


My current project is to create a robot which will amuse my 3 year old son.

As part of this I wanted to give the robot a large vocabulary of spoken sentences. There's a a dozen ways of going about this - Simple pre-recorded vocabulary, Google Android TTS, with output to cellphone (first thing I tried), FreeTTS (bit too heavy for EV3), porting flite (outputs 16bit audio unless you recompile the voices), etc.

Finally I settled on MaryTTS ( home - http://mary.dfki.de/, live demo - http://mary.dfki.de:59125/ )

Future-set of MaryTTS is quite amazing. I recommend checking out online demo with dfki-spike voice + effects + vocalizations.

MaryTTS which has a client/server architecture - with a server running on another "nearby" Linux/Windows machine, and its client-side is comparatively light. I've used MaryTTS provided client class, but according to MaryTTS docs any HTTP client would do the trick.

* This is a POC-level code ! Use at your own risk
* EV3 has 8bit sound and Java AudioSystem sound conversion isn't amazing. Don't expect miracles regarding sound quality.
* Some sound skipping/word swallowing issues, which I haven't sorted out yet
* I'm not a java pro, so pardon the poor style
* Uses modified version of Lejos EV3 Sound.java sample to output to sound device. If/when NXT code for outputting samples is merged into EV3 code this will not be needed.
* Calls code from my mini-logger class, but replacing Logger calls with System.out should be trivial
* If it turns out that this implementation is too heavy for running concurrently with other threads - dictionary per-compilation, with in-memory or on-disk cache of pre-tts'd sound clips will be quite easy to implement

Edit 1:
I discovered a couple of problems after I integrated the original class implementation into my main bot package:
* May not be much point in making TTS run in its own thread, since it is rather hard to hear TTS voice over the engine noise :roll:
* Code below works, but threading approach used below is rather poor. If you want to reuse it - remove all the multithreading. I'll be posting a better sample in a few days.
* Although MaryTTS client is fast, its not fast enough. So for now I'm switching to pre-building and caching TTS sentence vocabulary.

Code: Select all

package org.fuzzycow.lego.util;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
import java.util.Set;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;

import java.util.concurrent.TimeUnit;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;

import lejos.internal.io.NativeDevice;
import lejos.nxt.Battery;
import lejos.nxt.LocalBattery;
import lejos.nxt.Sound;
import lejos.util.Delay;
import marytts.MaryInterface;
import marytts.client.RemoteMaryInterface;

public class TTS  implements Runnable  {
   public final static TTS INSTANCE = new TTS();
   private final static NativeDevice dev = new NativeDevice("/dev/lms_sound");
   private static final byte OP_PLAY = 2;
   private static final byte OP_SERVICE = 4;
   private static final int PCM_BUFFER_SIZE = 250;
   public static final int VOL_MAX = 100;
   private int vol = VOL_MAX;
   private MaryInterface marytts = null;
   private BlockingQueue<String> queue = new LinkedBlockingQueue<String>();
   public static void main(String[] args) throws Exception {

      Battery battery = new LocalBattery();
      float voltLevel = battery.getVoltage();
      Logger.info("Volatage " + voltLevel);
       TTS tts = TTS.getInstance();
       Thread thread = new Thread(tts);
      // just for this test;
       tts.connect("", 59125);
                // should be true in real scenario
      tts.say("I am a robot");
      tts.say("yes, yes, a robot");
      tts.say("1 2 3 4 5 6 7 8 9 11 12 13 14 15");



   public static TTS getInstance() {
      return INSTANCE;
   public void run() {
      while (true) {
         String mesg = null;
         try {
            //while ((mesg = queue.poll(100,TimeUnit.MILLISECONDS)) != null) {
            while (  ( mesg = queue.take() ) != null ) {
         } catch (InterruptedException e) {

   public  synchronized void setVolume(int v) {
      vol = v;
   public int getQueueSize() {
      return queue.size();
   public int say(String text) {
      Logger.debug("TTS: adding to queue " + text);
      return queue.size();
   public synchronized boolean  isConnected() {
      if ( marytts != null )
         return true;
         return false;
   public synchronized void connect(String hostname, int port) {
      Logger.info("TTS Connecting to service");
      try {
         marytts = new RemoteMaryInterface(hostname, port);
         Set<String> voices = marytts.getAvailableVoices();
         Logger.info("TTS available voices: " + voices);
         // marytts.setVoice(voices.iterator().next());getClass();

                        // NOTE !!! change the voice name to one you have installed !!!
         // TractScaler    amount:1.4;
         // Robot amount:40.0;
         // marytts.setAudioEffects("Volume(Amount=3.0), Robot(Amount=40), TractScaler(amount=1.4)");
         //marytts.setAudioEffects("Volume(Amount=2.0), TractScaler(amount=1.4)");
         Logger.info("TTS Available audio effects: " + marytts.getAudioEffects() );
         Logger.info("TTS Connected to service");
      } catch (IOException ioe) {
         marytts = null;

   public void clear() {
   private void sayString(String text) {

      Logger.info("TTS will now say: [" + text + "]");

      AudioInputStream ais = null;
      try {
         ais = marytts.generateAudio(text);
      } catch (Exception e) {

      AudioFormat outDataFormat = new AudioFormat(
            AudioFormat.Encoding.PCM_UNSIGNED, 8000.0F, 8, 1, 1, 8000.0F,

      AudioInputStream lowResAIS = null;

      if (!AudioSystem.isConversionSupported(outDataFormat, ais.getFormat())) {
         Logger.error("Can't convert audiostream into " + outDataFormat);

      try {
         lowResAIS = AudioSystem.getAudioInputStream(outDataFormat, ais);
      } catch (IllegalArgumentException iae) {

      playAudioInputStream(lowResAIS, vol);

   private synchronized void playAudioInputStream(AudioInputStream ais, int vol) {
      byte[] buf = new byte[PCM_BUFFER_SIZE * 4 + 1];
      byte[] header = new byte[3];
      // get ready to play, set the volume
      header[0] = OP_PLAY;
      header[1] = (byte) ((vol * 8) / 100);
      header[2] = (byte)'T';

      int dataLen = 0;
      int offset = 0;

      Logger.debug("TTS writing to sound device");
      try {
         dev.write(header,0, 3);

         // buf[1] = 0;

         while ((dataLen = ais.read(buf, 1, buf.length - 1)) > 0) {

            offset = 0;

            while (offset < dataLen) {
               buf[offset] = OP_SERVICE;
               int len = dataLen - offset;
               if (len > PCM_BUFFER_SIZE)
                  len = PCM_BUFFER_SIZE;
               int written = dev.write(buf, offset, len + 1);
               if ( written < dataLen) {
               offset += written;
         Logger.debug("TTS wrote to sound device " + offset);
      } catch (Exception e) {


Last edited by fuzzycow on Fri Sep 27, 2013 3:02 pm, edited 4 times in total.

User avatar
leJOS Team Member
Posts: 6002
Joined: Fri Sep 28, 2007 2:06 pm
Location: UK

Re: EV3 Text-to-Speech with MaryTTS

Postby gloomyandy » Thu Sep 26, 2013 4:18 pm

great to see people doing some interesting projects with the EV3. Do you have a video clip of it in action? Would be great to hear how it sounds...


Posts: 30
Joined: Sun Sep 08, 2013 4:33 pm

Re: EV3 Text-to-Speech with MaryTTS

Postby fuzzycow » Sun Sep 29, 2013 6:48 am

No video yet I'm afraid.
However I believe you could see the above in action by pointing it at marytts online demo page.

I've been struggling with low voice volume for a couple of days. Currently you need to be next to the bot to make out what its "saying".
Updated version of the code is synchronous (since voice can't be heard over the motors anyway), and uses tricks like building text->wav cache for known phrases on startup.
I'll post the code in a day or two.

Ian bell
New User
Posts: 1
Joined: Tue Jun 28, 2016 7:14 am

Re: EV3 Text-to-Speech with MaryTTS

Postby Ian bell » Tue Jun 28, 2016 7:37 am

Another good text to speech software is Text Speaker. everything is self contained so you don't need the extra PC, and you can generate the wav or mp3's for cached phrases.

Return to “EV3 Projects”

Who is online

Users browsing this forum: No registered users and 1 guest