Recent advances in natural language processing have increased interestin automatic question generation, particularly in education (e. g.math, biology, law, medicine, and languages) due to its efficiency inassessing comprehension. Specifically, multiple-choice questions havebecome popular, especially in standardized language proficiency tests.However, manually creating high-quality tests is time-consuming andchallenging. Distractor generation, a critical aspect of multiple-choicequestion creation, is often overlooked, yet it plays a crucial role in testquality. Generating appropriate distractors requires ensuring they areincorrect but related to the correct answer (semantically or contextually),are grammatically correct, and of similar length to the targetword. While various languages have seen research in automatic distractorgeneration, Japanese has received limited attention. This Master’sThesis addresses this gap by automatically generating cloze tests,including distractors, for Japanese language proficiency tests, evaluatingthe generated questions’ quality, difficulty, and preferred distractortypes, and comparing them to human-made questions throughautomatic and manual evaluations.