| Abstract [eng] |
The thesis investigates the automatic and semi-automatic evaluation of programming assignments in teaching of programming, exams and competitions. Manual evaluation of programming assignments is still dominant in programming teaching in Lithuania. In some contexts (i.e. programming competitions, basics of programming and algorithms course) there is widely applied automatic and semi-automatic evaluation of programming tasks based on black-box testing approach. However, automatic evaluation based on black-box testing is criticized for its inability to demonstrate the strengths and the weaknesses of the program being evaluated. Automatic evaluation is not capable of identifying the level of achievement in incomplete programs. The thesis addresses the problem of evaluation of practical programming assignments in Information Technology maturity exam (IT VBE). This problem involves requirements to evaluate reliably thousands of programming assignment solutions in a reasonable amount of time. The thesis presents analysis of systems for automatic evaluation of programming tasks and appropriate types of assignments. The analysis led to the improvement of the evaluation method. The most important result of the research is the proposed way to improve the semi-automatic evaluation and testing method by increasing its interactivity. A semi-automatic assessment system is expanded by adding the components responsible for the modification of submitted programs, monitoring changes in the functionality of the modified program. The proposed method has been tested in practice - it was used in the IT VBE practical task assessment system developed by the author. This tool was used to test the effectiveness and the quality of evaluation by applying the proposed method. The analysis of possibilities of applying the proposed method to other semi-automatic evaluation systems was conducted. |
