This article was accepted into the corpus but its outbound wikilinks were never NER-processed — typical at the deepest BFS hop or when the run's entity cap was reached. No expansion funnel to show.
| UniDic | |
|---|---|
| Name | UniDic |
| Language | Japanese |
| Type | Morphological dictionary |
| License | Various (open-source variants) |
UniDic UniDic is a Japanese morphological dictionary designed for use with morphological analyzers and natural language processing tools. It provides detailed lexical entries with pronunciation, part-of-speech tags, inflectional paradigms, and readings to support tokenization, compound analysis, and linguistic research for applications used by organizations such as National Institute for Japanese Language and Linguistics, NHK, Rakuten, Yahoo! Japan, and LINE Corporation. The project intersects with projects and tools developed by communities linked to MeCab, JUMAN, Kuromoji, Sudachi, and academic initiatives at The University of Tokyo and Kyoto University.
UniDic supplies standardized lexical information for Japanese words and morphemes to drive analyzers like MeCab and Kuromoji and to integrate with corpora such as the Balanced Corpus of Contemporary Written Japanese and the National Corpus of Japan. Entries include surface forms used in texts such as editions by Kodansha, Shogakukan, and Iwanami Shoten; pronunciations aligned with standards from NHK and dictionaries like Daijirin and Kōjien; and morphological annotations compatible with tagsets used in research at NINJAL and projects funded by Japan Science and Technology Agency. UniDic aims to harmonize lexical resources across platforms developed by companies like Google Japan, Microsoft Japan, and research labs at Tohoku University.
Development traces to collaborations among researchers and engineers at institutions including NINJAL, Kyoto University, Osaka University, and corporate teams from NTT and Yahoo! Japan. Early morphological efforts relied on resources such as IPAdic and lexica underlying tools from ChaSen and JUMAN. UniDic emerged as part of efforts in the late 2000s to provide richer morphological segmentation inspired by corpora produced by The National Institute for Japanese Language and Linguistics (NINJAL) Modern Japanese Lexicon Project and projects led by scholars at Waseda University and Keio University. Funding and contributions came from grants by MEXT and collaborations with industry partners like Rakuten and LINE Corporation. Subsequent versions incorporated feedback from international conferences such as COLING, ACL, EMNLP, and workshops at NAACL and LREC.
Entries in UniDic encode surface forms, lemmas, readings, accent information aligned with accent dictionaries like those from NHK and Tokyo University of Foreign Studies; part-of-speech tags related to schemes used in corpora from BCCWJ and research by NINJAL; and morphological decomposition informed by resources from NII and projects at The University of Tokyo. The structure supports inflection models for verb classes known from treatments by linguists at Kyoto University and Harvard University Japan studies, and includes derivational morphology relevant to publishers like Shueisha and Kadokawa. Data formats follow conventions compatible with analyzers such as MeCab, JUMAN++, SudachiPy, and libraries in languages maintained by teams at GitHub involving contributors linked to Google Research and Microsoft Research Asia.
UniDic is used in search engines developed by Google, Yahoo! Japan, and Bing for Japanese indexing; in machine translation systems from teams at Google Translate and Microsoft Translator; in speech recognition stacks by Nuance Communications affiliates and research at Nagoya University; and in linguistic annotation pipelines used by projects at NINJAL and The National Institute of Informatics. It supports sentiment analysis in products by Rakuten and LINE Corporation, named-entity recognition in services by Recruit Holdings and Mercari, and academic corpora annotation for studies at Kyoto University and Osaka University. UniDic integrates into tokenization modules for frameworks like Mecab-ipadic-NEologd and feeds training data for neural models developed by groups at Preferred Networks and RIKEN AIP.
Compared with IPAdic, UniDic emphasizes richer morphological segmentation and standardized readings compatible with corpora from BCCWJ and conventions adopted by NINJAL. Compared to lexica used by JUMAN and ChaSen, UniDic targets interoperability with modern statistical and neural analyzers developed in research at The University of Tokyo and industry teams at Google Japan. Alternative resources such as NEologd focus on neologisms and social media forms curated by communities linked to GitHub and Twitter researchers, while UniDic emphasizes formal coverage, accent information, and inflection models used in academic evaluations at venues like ACL and LREC.
UniDic distributions have been released under licenses permitting academic and commercial use in many versions; contributors include researchers and engineers from institutions including NINJAL, National Institute of Informatics, and corporate partners like Yahoo! Japan and Rakuten. Binary packages and source distributions are made available in repositories used by deployments at GitHub and package managers serving projects at PyPI and Maven Central. Commercial adopters include corporations such as LINE Corporation, Google Japan, and Microsoft Japan, while academic users span The University of Tokyo, Kyoto University, Waseda University, and Keio University.
UniDic is packaged for use with morphological analyzers and libraries including MeCab, Kuromoji, Sudachi, SudachiPy, JUMAN++, and wrappers in languages supported by teams at GitHub and ecosystems at PyPI and Maven Central. Implementations power NLP pipelines in research projects at NINJAL, industrial systems at Rakuten and Yahoo! Japan, and open-source toolchains maintained by contributors associated with Preferred Networks, RIKEN, and academic groups at Osaka University and Nagoya University. Integration examples include tokenizers for spaCy, bindings used by TensorFlow models from Google Research, and preprocessing modules in toolkits developed by Microsoft Research Asia and participants at conferences like EMNLP and ACL.
Category:Japanese dictionaries