POSTS
Final Report
The final work period is done and the application is up and running at https://snd96.pythonanywhere.com/. The github repository for my work can be found here: https://github.com/SND96/twine-stories.
These are the tasks that the application can so far perform.
Parse through an entire Twine story and then extract the relevant data like questions and choice statements. The entire story can then be navigated using Flask.
Generate the phonemes for each choice statement and then loading it into Pocketsphinx.js.
Uses the voice recording UI found in https://github.com/SND96/pocketsphinx-scores, to record the user’s voice. The recording can then be replayed.
The recording can also be evaluated to see if it matches any of the choices in the Twine node. If it does, the application automatically navigates to the next node.
The application can also perform intelligibility remediation on the phrase “we drank tea in the afternoon and watched tv”. This node can be reached by clicking on the option “Click for more pronunciation evaluation” in the starting node of the story. The user can record their saying of the phrase and on pressing evaluate, the intelligibility scores for each word will be displayed.
Right now the voice recognition for choosing an option is still buggy and can result in wrong phrases being chosen.
Also the server for the intelligibility scores can be slow and doesn’t respond for many minutes.
Due to a lack of recordings only one phrase is available for intelligibility scoring.
The application as of now is ready to take in a new Twine story and convert it into a voice navigated story. I tried to do as much as I could to get it ready in time. I’m happy with what I’ve accomplished but still wish I could’ve done more. For now, I’m done with GSoC 2017 :D