NVDA Workshop in Japan

NVDA Workshop in Japan is held as the satellite event of PyCon JP 2012 on September 16-17, 2012 in Tokyo, Japan.

NVDA Japanese Team is very happy that Mr. Michael Curran (NV Access Limited, Australia) and Dr. Jerry Wang (Taiwan Digital Talking Books Association) accepted our invitation to visit Japan for this purpose.

Mr. Hiroshi Kawamura (Past President, DAISY Consortium) will also give valuable advices to the NVDA community.

We would like to express a hearty welcome to all who are interested in NVDA, accessibility, screen reader, speech synthesizer, braille display, open standards, and inclusive development.

Notice: PyCon JP registration is required for the Sunday program.

New community and activities of NVDA in Japan

At the end of April, I became one of the members of NVDA translations.
At the same time, I started the new group, NVDA Japanese Team, in Japan.
Official site is http://en.sourceforge.jp/projects/nvdajp/devel/.
Localized beta version of NVDA is available.
We also release Java Access Bridge 2.0.2 installer and older versions of LibreOffice and OpenOffice.org with the accessibility option enabled.
Source code of nvdajp is also available at launchpad.net/nvdajp.

Our goal is to contribute our efforts to the main stream of NVDA.
Toward Windows 8, it is important to consider security model or digital signatures.
Japanese Team is planning to make executables with digital signatures by ourselves, however, it is not desired solution in the long run.

Besides the translation of documents, messages and character tables, Japanese Team is working for the issues as follows:

  • Japanese Text to Speech engine. eSpeak is difficult to support Japanese. We have added JTalk, which can be used as the installer’s default voice. It depends on morphological analysys engine MeCab with the dictionary of one hundred megabytes.
  • Japanese input method support. We use Python and C++ code. To announce the change of IME status, low-level keyboard hook is used. To announce the content of composition window, Text service framework (TSF) must be supported. Note that some Windows XP users disable TSF because of the compatibility issues. For x64 applications, remote procedure call (RPC) is also required, because NVDA itself is 32bit process. Candidate window is accessible using MSAA. Our implementation should be re-written to complete the application support, to improve stability, and to support other East Asian languages.
  • Japanese Braille support. It consists of (1) extending translation engine and (2) supporting domestic devices. Braille translation of NVDA is relying on liblouis library, however, it is not enough for supporting Japanese language. To reduce the development effort, it is desireble for us to reuse morphological analysys engine for TTS as the basis of Japanese braille translation.
    As the domestic device support, we added a new KGS Braille Memo driver for 2012.2jp-beta release.
    Because of the limitation of vendor’s library and testers’ demand, serial port selection can’t be ignored. So we both implemented automatic port scanning and COM port selector of the braille setting dialog.
    Although Japanese braille translation is far from perfect, Japanese NVDA users are highly interested in our work.
  • Supporting domestic applications and use cases. AppModules and global plugins can be used.

It is a new challenge to make our contributions into NVDA-addons, which will be available for NVDA 2012.2 and later versions, as much as possible.

As a first step, we are making the JTalk Japanese TTS as the add-on, which runs on NVDA 2012.2 beta versions.
If you are interested in Japanese speech engine, please evaluate the current version as follows:
Release Note
Download
Note that JTalk2 is experimental multi-lingual engine which uses both eSpeak and JTalk internally.
It is tested on Windows XP Japanese, however, I am not sure how this package is depending on locale.
(note: JTalk addon is updated on May 28 19:10 JST. Moved from dropbox to sourceforge.jp on June 13.)

NVDA Japanese Team started discussions with Taiwanese NVDA developers.
We welcome intenational collaborations, to make the NVDA project more active and attractive.

NVDA 2012.1j (nvdajp)

We would like to provide information on the activities of Japanese community of NVDA. Currently 2012.1j-beta2 is released.

Release by NVDA Japanese Project (nvdajp), based on NVDA 2012.1 from nvda-project.org.

Release manager: Takuya Nishimoto (nishimotz atmark gmail.com, Twitter @nishimotz)

This release is Japanese localization of the NVDA open-source screen reader for Windows.

Changes from original 2012.1:


  • Application icon is changed to the NVDAJP Telescope Goldfish.

  • NonConvert key of Japanese keyboard can be used as NVDA modifier key (optional).

  • Japanese Text-to-Speech engine JTalk is added (based on Open JTalk, MMDAgent, Mecab project)

  • Japanese input method editor (IME) support (for 32bit/64bit environment). Some applicaitions are not fully supported.
    You can disable this feature using “Keyboard settings” – “Support IME (nvdajp)”.

  • AppModules for Search Aid and VoicePopper (popular self-voicing applications in Japan) are added.

  • Japanese translations added for Input Help Mode: “IME Non convert”, “IME Convert”, “Pause”.

  • Character descriptions can be customized by user.
    If characterDescriptions-(locale).dic file is found in userConfig file,
    it supersedes built-in dictionary.
    This feature is requested from the Japanese users, because commercial screen readers use their own character description dictionaries, and the users preferences are so different.
    We are also developing alternative dictionary to replace the current Japanese dictionary.

  • Experimental JTalk2 driver: language switching among Japanese and other languages supported by eSpeak.

  • Experimental Japanese braille display driver DirectBM (for KGS devices).
    Japanese braille support requires the process of inserting spaces between words.
    Current implementation of nvdajp uses the text analysis module (mecab and naist-jdic) for both JTalk and DirectBM.
    Although we would like to accelerate the development, the development members have very limited number of devices.

  • “Braille Viewer” is added in Tools menu. Available for Japanese language.

  • Report Time (NVDA+f12) is modified for Japanese environment.

  • Help documents are opened by HTML application (HTA) window, not the system’s default web browser.
    This is because Japanese popular voice web browser for screen reader users is difficult to use with NVDA.

What’s New in this version:


  • NVDA 2012.1 (rev 5086) is merged.
    http://bzr.nvaccess.org/nvda/releases/2012.1

  • Some libraries are updated: liblouis 2.4.1 and espeak 1.46.12

  • Some Japanese translation of messages are modified.

  • Japanese Input Method Editor (IME) support is improved.
    This work is unfinished. Some limitations and bugs are remained.

  • JTalk (Japanese TTS) is updated.

  • Dictionary for Japanese text analysis is updated.

We use launchpad.net as bazaar source-code repository.

Versions of Japanese resources (shown in NVDA log):


  • nvdajptext.dll 20111017 mecab:0.98

  • nvdajp-jtalk-dic 20120403-053143

  • libopenjtalk 20120323-070011

Repositories are as follows:


  • lp:~nishimotz/nvdajp/releases_2012.1

  • lp:~nishimotz/nvdajp/MiscellaneousDependencies rev 77

NVDA development in Japan

NVDA (Non-Visual Desktop Access) is developed by an NPO based
in Australia.

I am participating as the project leader of Japanese localization of NVDA since 2011.

Assisitive technlogy is making evolutions, as well as the other technologies.

In Japan, most of the core technologies, such as text input method, speech synthesis, and automatic braille translation, have not been freely available, so the Japanese community is behind the times.
This is the reason I committed to the project so deeply since the earthquake.

nvdajp 2011.3j was released on Dec 23.
Its enhancement includes Japanese TTS, IME support and braille support.
Default speech engine is JTalk, an open-source Japanese TTS, based on Open JTalk.

Functions of reporting Japanese input method has limitations as follows:

(1) IME support does not work properly with Wordpad, Microsoft Word, and Internet Explorer.

(2) With Internet Explorer and some applications, after writing some Japanese characters within a textbox,
pressing Escape key without finishing composition may cause collision of escaping edit mode with canceling Japanese composition.

(3) With 32bit applications, key-echo of Japanese input is delayed, because the report is performed at the key-up events.

IME support function can be disabled using Keyboard setting.

DirectBM is experimental braille driver which supports KGS Japanese braille displays.

Although the activity is separated from the mainstream development now, international cooperation, such as helping development of Chinese text imput support, is very important.

We started discussions with an NPO in Japan to obtain the support. I am also seeking the consultation business related to the software, or a position of full-time researcher where I can keep the engagement of NVDA development. I hope I can inform the good news this year.

Research on Effective Designs and Evaluation for Speech Interface Systems

This post is excerpt from the draft version of the abstract of doctoral dissertation.
My public hearing for the dissertation will be held this month at Waseda University.
Although the thesis itself is written in Japanese, I am willing to write the related topics here in English.

This paper describes a systematic way of enabling of developers and designers to build information-communication systems successfully with speech technologies, such as speech synthesis and speech recognition. As the results of this work, application systems of speech technologies can be used easily for everyone.

This work also describes four research projects including the development of speech applications and the evaluations of speech interfaces, which are performed based on the proposed methodology.

Continue reading “Research on Effective Designs and Evaluation for Speech Interface Systems”

ICCHP 2010 talk

My talk on audio CAPTCHA at ICCHP finished yesterday in Vienna.

I enjoyed using Twitter during the conference. Thank you.

Dear new friends,
Usually I tweet in Japanese with @nishimotz account. I prefer using facebook for English conversation. So, please feel free to unfollow me.
Both facebook and @nishimotz account on Twitter can be used for English conversation.
I will tweet in Japanese with @nishimtz account.

My slide and my tweets at ICCHP is as follows: Takuya Nishimoto, Takayuki Watanabe: The Evaluations of Deletion-Based Method and Mixing-Mased Method for Audio CAPTCHAs.

Notice (2010-10-16): Updated slide of this topic (for Interspeech in Sep. 2010) is available.

Continue reading “ICCHP 2010 talk”

DMCPP: development of another dialog manager

An experimental dialog manager of Galatea for Linux, using lib-julius and OpenCV, written in C++, is under development.
The project is focusing on the low-level multimodal event handling, while Galatea Dialog Studio is focusing on higher-level dialog management using VoiceXML.
Although current version is just the skeleton of applications, I would like to ask for feedback from the developers.

DMCPP in English
DMCPP in Japanese

Please join mailing lists for discussions.

pyAA

To test the MSAA-related features of Microsoft Japanese Input Method Editor MS-IME 2002 for Japanese version of Windows, I am working with pyAA.
This is a preliminary work for localization of NVDA for Japanese users.

I tried to adopt original pyAA to Python 2.6.x.
At first I obtained the source code (of simpler branch) from CVS repository, then I built it with Visual Studio 2008 and SWIG. I also modified the code so that the Value-property can be accessed correctly under multibyte charactor coding environments.

We are successfully proceeding the work with it at the moment.
At the previous meeting of NVDAjp project, we added some code to NVDA, and verified that the WinEvent of MS-IME 2002 can be captured and the Value property can be accessed using IAccessible interface.

Related pages in Japanese (not yet translated) : pyaa and nvdajp

Notes on Feb 27 : I created a github repository of pyaa.