Navigation through Twine with Voice - Week 9/10

This week was spent mostly on loading the pronunciation phonemes for each word in every option into Pocketsphinx.js. The first step was getting all the phonemes which was already accomplished earlier and made into jsgf files. Instead of using the jsgf files, the pronunciations will be loaded directly into the html page in two steps.

The first step will be to make an entire dictionary for all the word present on that particular page. For example for the sentence “Change the word recently”, the word dictionary will be [“change”, “CH EY N JH”], [“the”, “DH AH”], [“word”, “W ER D”], [“recently”, “R IY S AH N T L IY”]. This is done for all the options and then appended to an entire list.

The second step will be to tell Pocketsphinx.js when to expect each word in the sentence, i.e. which order the words will be appearing in. Using the same sentence as above the format in which it will have to be loaded is {from: 0, to:1, word:“change”},{from: 1, to:2, word:“the”},{from: 1, to:2, word:“the”},{from: 2, to:3, word:“word”},{from: 3, to:4, word:“recently”}.

The code for these need to be written into each html page along with the Pocketsphinx.js recognizer. For this I wrote a python parser which automatically writes the code into each html file with only a only a few lines of difference between each page. These lines include the options specific to each page and the pronunciations for each of these options. As of right now the recognizer can sometimes be inaccurate resulting in none of the options being selected which needs to be remedied.