The Ministry of Health of Ukraine
Decree № 419, dated February 19, 2019,
registered with the Ministry of Justice of Ukraine
under № 279/33250 on March 20, 2019

On adopting the procedure, terms, and timeframe for development and administration of the Unified State Qualification Exam and criteria for the result assessment

I. General Provisions

1. This Procedure determines the mechanism, terms, and timeframe for development and administration of the Unified State Qualification Exam (hereinafter “the USQE”) for the students enrolled in the Master’s academic programs and majoring in “22 Healthcare” branch of knowledge (hereinafter “the students”), as well as the criteria for the USQE result assessment.

This Procedure is binding for all the higher education establishments that train future professionals majoring in “22 Healthcare” branch of knowledge (hereinafter “the schools”), the State Non-Profit Enterprise “Testing Board for Professional Competence Assessment of Higher Education Trainees in Medicine and Pharmacy at the Ministry of Health of Ukraine” (hereinafter “the Testing Board at the MOH”), and the students.

In this Procedure the terms are used in the same sense as in the Law of Ukraine “On Higher Education”, in Resolution № 334 “On adoption of the Unified State Qualification Exam Procedure for the students enrolled in the Master’s academic programs and majoring in “22 Healthcare” branch of knowledge”, adopted by the Cabinet of Ministers of Ukraine on March 28, 2018, and in Decree № 1883 “On adoption of the cost estimate methodology for the development and administration of the Unified State Qualification Exam”, adopted by the Ministry of Health of Ukraine on October 19, 2018.

2. The USQE consists of the test-based components and the Objective Structured Practical (Clinical) Exam (hereinafter “the OSCE/OSPE”).

The USQE test-based components include:

• the “Krok” Integrated Test-Based Exam (hereinafter “the “Krok” Exam”);
• the English Language Proficiency Test (until September 01, 2019 it was the Foreign Language Proficiency Test that assessed the students’ occupational proficiency in any foreign language studied by them in their schools);
• the International Foundations of Medicine Exam.

3. Organization and administration of the USQE test-based components is provided by:

• the Rectors (Directors) of the schools, where it concerns preparation of exam classrooms, ensuring suitable and safe conditions during the exam, and provision of emergency medical aid for the students, if needed;
• the Director of the Testing Board at the MOH, where it concerns the organizational and technological measures, necessary for the exam administration.

Organization and administration of the OSCE/OSPE is provided by the schools.

II. Procedure, Terms, and Timeframe for the USQE Development

1. The “Krok” Exam and English Language Proficiency Test are developed by the Testing Board at the MOH.

The main characteristics of the “Krok” Exam and English Language Proficiency Test, which are the basics for their development, are entered into the Exam Specifications.

The Exam Specifications must contain the information about the exam’s goal, content structure, language of administration, format of administration, format and number of test items, duration of the exam, etc. The “Krok” Exam and English Language Proficiency Test Specifications are developed by the Testing Board at the MOH and approved by the Ministry of Health of Ukraine no later than 6 months before the date of the exam.

The OSCE/OSPE are developed by the schools and must be approved by the Ministry of Health of Ukraine.

The International Foundations of Medicine Exam is administered using the International Foundations of Medicine Basic and Clinical Science Exams, developed by a foreign agency, specializing in student knowledge assessment (hereinafter “a foreign agency”).

2. The USQE development (except for the International Foundations of Medicine Exam) includes primary development of the USQE components, their updates and relevance renewal according to the standards of higher education, contemporary scientific knowledge, and healthcare practices.

The USQE development can include:

• determining and revising the goal, structure, and content of the USQE components;
• creation, expert assessment, approbation, and revision of the exam materials (test items, assessment algorithms, case vignettes, OSCE/OSPE workstations, emulated scenarios, protocols, checklists, etc.);
• development of study guides and information materials for the USQE;
• development of software for the USQE;
• development of the USQE norms and standards, formation and standardization of the list of material and technical resources, equipment, and medical simulation mannequins for the USQE;
• administration of pilot tests;
• instruction, provision of hands-on experience, and training of the test item authors, experts, standardized patients, examiners, and other professionals involved in the USQE;
• surveys and analytical research to maintain the USQE relevance and quality, as well as setting the criteria for successfully passing the USQE components;
• organization of work meetings, panels, workshops, conferences and participation in them;
• collaboration with international organizations that assess competence and quality of healthcare education;
• delegation of Ukrainian professionals to work with the developers of international exams, particularly the International Foundations of Medicine Exam;
• communication and informational events.

3. The timeframe for the USQE components development includes the primary development, with updates and relevance renewal occurring at least every three years.

The timeframe for the “Krok” Exam and English Language Proficiency Test development depends on the current practices for administration of each specific component, taking into account student specialty, type of education, language of education, whether it is the first or a repeated attempt to pass the test, exam duration, and conditions of exam administration and registration for the exam.

The timeframe for the “Krok” Exam and English Language Proficiency Test development is determined by the Testing Board at the MOH.

The timeframe for the International Foundations of Medicine Exam development is determined by the foreign agency.

The timeframe for the OSCE/OSPE development is determined by the schools, taking into account student specialty, type of education, language of education, and whether it is the first or a repeated attempt to pass the test.

4. The number of test items in each test-based component of the USQE must not exceed 200.

III. Procedure, Terms, and Timeframe for Administration of the USQE Test-Based Components

1. The USQE test-based components are administered based on the technologies of standardized testing as paper-based tests (answer sheets) or web-based tests (in full compliance with the information security legislation).

All the USQE test-based components are administered using unified closed tests and unified methodology guidelines. They are developed by the Testing Board at the MOH for the “Krok” Exam and English Language Proficiency Test or by the foreign agency for the International Foundations of Medicine Exam.

2. The duration of the English Language Proficiency Test for Ukrainian and foreign students is calculated, allocating 2 minutes per 1 test item.

The duration of the “Krok” Exam for Ukrainian students is calculated, allocating 1 minute per 1 test item, while foreign students receive 1.2 minutes per 1 test item.

The duration of the International Foundations of Medicine Exam is 4 hours.

3. The USQE test-based components are administered according to the schedule approved by the Ministry of Health. The Schedule is planned for a calendar year and approved annually before September 01 of the previous calendar year.

The Schedule specifies the administration date of each USQE test-based component for each specialty included in the “22 Healthcare” branch of knowledge and the period (month) within which retakes of the USQE test-based components are held, taking into account that retakes must take place before the start of a new academic year or a new semester. Retake dates for the USQE test-based components are determined by the Testing Board at the MOH and published on the Testing Board official website.

The students, who were absent during any of the USQE test-based components with a reasonable excuse (properly confirmed with the relevant documents), are granted one retake of the exam. The retake date is determined by the Testing Board at the MOH.

4. The USQE test-based components are administered at the school premises.

Each classroom, where the USQE test-based components are to be held as paper-based tests (answer sheets), must satisfy the following requirements:

• no partitions, so that nothing hinders the observation of the testing process and all the examinees can be easily accessed;
• sufficient lighting;
• furnished with tables (desks, chairs with folding tables);
• equipped with a clock and a writing board or any other surface, where the time of beginning and end of the test can be written down;
• equipped with cameras or video surveillance systems that automatically record all the stages of the testing process (no less than 95% of the classroom must be visible in the recording);
• spacious enough (all the examinees must be seated facing one direction and separated by two empty seats (approximately 1.2 meters), so that they cannot communicate or copy each other’s answers).

5. The USQE test-based components can be undertaken by the properly registered students, if their current health condition allows them to undergo testing.

Registration for the USQE test-based components begins no sooner than 4 months before the date of the USQE test-based component administration and ends no later than 2 months before the date of the USQE test-based component administration.

Students are registered for the USQE test-based components:

• by the Testing Board at the MOH, where it concerns the “Krok” Exam and English Language Proficiency Test;
• by the Testing Board at the MOH along with the foreign agency, where it concerns the International Foundations of Medicine Exam.

The students’ personal data necessary for their registration is transferred abroad to the foreign agency that is the developer of the International Foundations of Medicine Exam. Such transfer occurs in full compliance with the Ukrainian legislation on personal data protection and only after a student gives his/her permission for the personal data processing.

Students are registered for participation in the USQE test-based components on the condition that the official request and the document package have been provided by their schools for the purpose of registration. The registration format is determined by the Testing Board at the MOH and/or the foreign agency that is the developer of the International Foundations of Medicine Exam.

In cases when special needs must be met during administration of the USQE test-based components for the students with medical conditions, the school must submit to the Testing Board at the MOH a medical conclusion issued by a healthcare establishment, stating that certain conditions must be created for such student.

The information about the registration procedure for the students, who are to take part in the USQE test-based components, along with the relevant forms, must be published on the official website of the Testing Board at the MOH no later than one month before the registration starts.

The Testing Board at the MOH has the right to reject the school’s request for the registration of a student for the “Krok” Exam and English Language Proficiency Test, if the document package for the registration was not executed properly, the documents do not comply with the Ukrainian legislation, and/or the request for registration was submitted after the final date.

If a student’s request for registration is rejected, because the document package for the registration was not executed properly or the documents do not comply with the Ukrainian legislation, the school can submit the registration documents again, after putting them in order.

If a student is majoring in a specialty that legally requires passing two stages of the USQE and he/she has no Certificate for successfully passing the USQE, Stage 1, the Testing Board at the MOH cannot register such student for the USQE, Stage 2.

6. The students registered for participation in the USQE test-based components have the right to:

• access the information about the USQE test-based components schedule and procedure, their time and place, the result assessment procedure and time and manner in which the USQE results will be officially published;
• be treated politely and without bias by the persons authorized to organize and administer the USQE;
• safe conditions during the USQE administration;
• receive emergency medical aid, if needed, during the USQE administration;
• appeal against the USQE administration procedure, if their rights, listed in this section, are violated, and request (appeal) to recheck the USQE results.

The schools and the Testing Board at the MOH ensure that the students’ rights, listed in this section, are observed.

7. The students registered for participation in the USQE test-based components must:

• familiarize themselves with the USQE Procedure;
• timely arrive at the exam site with their identity documents, in compliance with section 13, chapter III of the USQE Procedure;
• be polite to the other students and to the persons tasked with the USQE administration;
• comply with the instructions and requests of the persons tasked with the USQE administration, where it concerns the USQE administration procedure.

8. The students registered for participation in the USQE test-based components must not:

• bring to the USQE site any dangerous objects or substances that can endanger the life and health of people;
• use or have on their persons or at their workstations any communication devices or devices that can be used for scanning, processing, storage, and retrieval of information (including mobile phones, tablets, calculators, watches with an alarm, CPU, or memory, pagers, audio or video recorders, radio, players, etc.), as well as their separate components, any printed or handwritten materials, other kinds of gadgets, tools, and equipment not allowed during the test, backpacks, bags, outerwear, any food or beverages;
• interfere with the work of other students during the test;
• interfere with the duties of the persons tasked with the USQE administration;
• communicate in any way with other students during the test or pass them any objects and materials, including exam materials;
• copy the answers of other students;
• in any way disclose the information about the content of the test items before the exam is concluded;
• take exam booklets and answer sheets, their separate pages or copies, out of the classroom;
• continue answering the test items or erase the marks in the answer sheet after the end of the test;
• damage the property at the USQE site.

A student, who violates these rules, loses the right to continue taking the test and, upon the request of the Testing Board representative, who administers the USQE test-based component (hereinafter “the proctor”), has to immediately hand over his/her answer sheet and exam booklet and leave the exam classroom, which must be recorded in the relevant report. The submitted answer sheet in this case is not checked and marked instead as “0” (“fail”).

9. To ensure the security of closed exam booklets, they are delivered to the schools using the State Special Communications Service of Ukraine. For the delivery, the Testing Board at the MOH hermetically seals the exam booklets into a film and packs them into containers made of protective materials.

Containers with the exam booklets are delivered to the school office no later than two days before the date of the USQE test-based component administration. In the school office they are received by an authorized person, who must be appointed to this duty by the Principal’s (Director’s) decree and hold an executive administration post within the school, which is recorded in the relevant report. The exam materials are to be stored in a sealed room of Principal’s (Director’s) choice.

On the day of the exam, the authorized representative of the school must in the presence of all the proctors, who arrived to the school for exam administration, check the intactness of the sealed containers with the exam booklets and record it in the relevant report signed by all proctors and the authorized representative of the school. (The procedure can be recorded on video.)

If the package with the exam materials is confirmed to be intact, the proctors can proceed with the USQE test-based component administration.

If the package with the exam materials was tampered with, the proctors must inform of it the Testing Board at the MOH. The Testing Board, in turn, informs the Ministry of Health of Ukraine. After that, the Ministry of Health of Ukraine reaches the decision about administration of this exam in the schools.

10. To ensure the transparency and public nature of the USQE administration, video surveillance and video recording are provided in the classrooms, where the USQE test-based components are being administered. The video recording must cover the time interval from the moment, when the first student enters the classroom, to the moment, when all the students and proctors leave the classroom after the exam ends. The proctors are the last to leave the classroom.

The copies of the surveillance recordings must be sent to the Testing Board at the MOH for analysis. The original recordings are kept by the schools for four years.

If during the video recording analysis employees of the Testing Board at the MOH detect the fact of the exam procedure violation by a student, such student receives “0” score.

11. To ensure public safety and order, the Testing Board at the MOH collaborates with the Department of Police Security at the National Police of Ukraine. The measures aimed at maintaining the public safety and order are determined based on the agreements concluded between the Testing Board at the MOH and the Department of Police Security at the National Police of Ukraine.

12. The proctors/proctor and the school representative must be constantly present in the classroom for the duration of the test.

The school representative is appointed to this duty by the Principal’s (Director’s) decree. During the test the authorized school representative cannot walk the classroom or communicate with the students.

Outsiders are prohibited from entering the classroom, where the testing takes place.

13. Each USQE test-based component is to be taken by a student individually after the identification procedure at the exam site.

For identification a student must present his/her current identity document, specified by the school during the registration of this student for the exam. (For Ukrainian citizens it is their Ukrainian passport, for foreign citizens and persons without citizenship it is their foreign identity document.) The identification procedure is organized by the school representative, assisted by the proctor.

The students, who did not undergo the identification procedure, cannot take the exam and are considered to be absent without a reasonable excuse at the exam. All such cases are recorded in the relevant report.

14. After being checked for objects and devices prohibited during the test, the students are admitted into the classroom in the presence of the proctors and school representatives according to the list held by the proctors.

The students must leave their personal effects in the place allocated for this purpose by their school. The school holds the responsibility for the safety of the stored personal effects.

If a student arrives late to the exam, but the delay does not exceed 15 minutes, the student can proceed with the testing without additional time granted.

15. The proctor directs the students to their places in the classroom.

Before the start of the test, the proctor informs the examinees about the test procedure. After that, each student receives his/her exam booklet.

Exam booklets and answer sheets cannot be taken out of the classroom during the exam.

The students cannot leave the classroom during the exam without permission from the proctor. If a student needs to leave the classroom to feed the baby, to have a bathroom break, or because of a medical condition (in this case the presence of the medical condition must be properly confirmed with the relevant documents and the school needs to notify the Testing Board at the MOH about it in writing in advance), he/she must notify of it the proctor and receive the permission.

If the student’s health condition suddenly becomes worse during the exam, the proctor and the school representative present in the classroom must be immediately notified of it. If the continuation of the exam is impossible for the student, the proctor discontinues the exam for this student.

If exam had to be discontinued for a student because of a medical condition, the fact and duration of the student’s absence from the classroom must be recorded in the relevant report.

If a student needs to temporarily leave the classroom, the Testing Board representative collects his/her booklet and answer sheet and gives them back to the student upon his/her return to the classroom.

No additional time is granted to the student to compensate for the time spent outside the exam classroom.

When the student returns to the classroom, he/she needs to be checked again for any objects and devices prohibited in the exam classroom.

Additional seats in the classroom must be reserved for the students, who were allowed to leave the classroom during the exam.

16. Having finished working with the exam booklet, a student must put his/her answer sheet into his/her exam booklet, hand them over to the proctor, and leave the classroom.

The proctor inspects whether both the answer sheet and the exam booklet were handed over and whether all the answer sheets match their corresponding exam booklets. It must be done in the presence of the student.

The students, who completed their test before the exam is over, can leave the classroom one by one, no later than 10 minutes before the time when the exam ends. After it is announced that the exam ends in 10 minutes, no student can leave the classroom.

After the test, a relevant report is drawn up for each classroom, where the exam has been administered.

To ensure the transparency of the exam administration and its results, a procedure of copying the answer sheets of the examinees can be conducted at the schools. The procedure of the answer sheet copying, storing, and opening is determined by the Testing Board at the MOH.

In case of the International Foundations of Medicine Exam, the procedure of answer sheet copying can be conducted only if the exam developer allows it.

17. The students, who failed to be present at any of the USQE test-based components must submit the documents explaining the cause of their absence to the Testing Board at the MOH within 3 days after the exam.

Reasonable excuses of the absence at the exam include:

• an illness at the time of the exam, confirmed by the health certificate of temporary disability of a student at a school of I–IV accreditation levels, or illness, quarantine or other reasons for the absence of a student at a general secondary school or a preschool establishment (form №095/о, approved by Decree №110 “On adoption of the forms of primary accounting documentation and instructions on the procedure of their filling in, used at the healthcare establishments regardless of their forms of ownership and subordination”, adopted by the Ministry of Health of Ukraine on February 14, 2012 and registered with the Ministry of Justice of Ukraine on April 28, 2012 under №661/20974), or a copy of the health certificate of disability;
• a death in the family, including the in-laws;
• a natural disaster, fire, or technogenic disaster on the day of the exam or on the day before the exam.

If such students fail to submit the documents providing a reasonable excuse of their absence at the exam or if they submit them after the expiration of the three-day-period, it is considered that they were absent at the exam without a reasonable excuse and they obtain “0” score.

18. The relevant reports are drawn up in the cases specified in this chapter in sections 8, 9, 13, 15, and 16. The format of such reports is determined by the Testing Board at the MOH. The reports are signed by the persons present in the classroom during the test, as provided in sections 11 and 12.

19. If a student disagrees with his/her score obtained as the result of the “Krok” Exam and English Language Proficiency Test, he/she has the right to request a recheck of his/her test score. In this case an Appeal Request to recheck the obtained test score must be submitted to the Testing Board at the MOH no later than 5 calendar days after the date, when the exam results were sent to the schools.

Appeals concerning the results of the International Foundations of Medicine Exam are processed by the foreign agency according to its standard procedure.

20. When all the USQE test-based components are successfully passed, the student receives a Certificate:

• in the format determined by the Testing Board at the MOH for the “Krok” Exam and English Language Proficiency Test;
• in the format determined by the foreign agency for the International Foundations of Medicine Exam.

21. The information about the certificates obtained for successfully passing the “Krok” Exam and English Language Proficiency Test is entered into the Certificate Register of higher education trainees majoring in “22 Healthcare” branch of knowledge.

V. Result Assessment Criteria for the USQE Components

1. The USQE is considered successfully passed only when each USQE component is successfully passed.

The results of the “Krok” Exam and English Language Proficiency Test are presented qualitatively and quantitatively.

Qualitatively, the results are presented in the “pass” / ”fail” format and are based on the professional competence requirements for healthcare workers. The requirements are determined by the national standards for higher education in Ukraine.

Quantitatively, the primary results are presented as a portion of correctly answered test items that is converted into a multiple-point normalized scale.

The results of the International Foundations of Medicine Exam are presented quantitatively, in the format determined by the exam developers.

2. The minimum passing score for each USQE test-based component is approved by the Ministry of Health of Ukraine.

The criteria for result assessment of the USQE test-based components are based on the number of correctly answered test items.

The test item is considered correctly answered, when the examinee has chosen and marked the correct answer in the answer sheet. The test item is considered incorrectly answered, when:

• a wrong answer is marked;
• two or more answers are marked, even if one of them is the correct one;
• no answer is marked.

3. The criteria for the OSCE/OSPE result assessment are based on the evaluation of the student’s performance at a workstation. The evaluation is based on the control sheet that presents an algorithm for practical skill performance. The maximum score per 1 station is 5 points. The points are given for each part of the algorithm. All algorithm parts have different value, depending on their total number, importance, and complexity. After a student completes tasks at all the workstations, the obtained points are summarized. The maximum score can vary from 40 to 60 points, depending on the number of the workstations. The OSCE/OSPE is considered successfully passed, if upon completion of all the workstations a student obtains no less than 60% of the maximum score.