questions

Phase internal questions Methods Answers
discovery(understanding the context) What are the demographics of users who engage with the chatbot game? 1. User research to understand the needs and preferences of the target audience. rage of ages: 21-52education / profession: from medicine students touniversity lecturers and physicians.
What are the primary objectives of users when engaging with a chatbot game in medicine, and how can the game best meet these objectives?Was sind die primären Ziele der Nutzer, wenn sie sich mit einem Chatbot-Spiel in der Medizin beschäftigen, und wie kann das Spiel diese Ziele am besten erreichen? 1. Collecting data and feedback from users to inform product development. To ensure credibility, gender bias should be carefully and purposefully incorporated into the user journey. To achieve a higher effect, gender-sensitive design can be utilized through the use of illustrations or pictures of the patients in a clinical environment.
what are the expectations wishes for the project 1. Interview (research phase) **Interview (research phase):**more of a role play game, conversational dialogoue, with gamification elements chatbot type
What role will the chatbot play? (Assistant/mentor chatbot or as a simulation patient) Chatbot (game?): as a serious game or conversation partner? 1. Interview (research phase)
  1. Interview (Usability Test; A/B Test) | **Interview (research phase):**They didn’t have thought about the role of the chatbot. They assume the Bot will have both the role as the patient as well as giving feedback like a mentor. (but thats not really realistic) | | | What kind of Gender Bias can we include in the scenario? | 1. Expert Interviews
  2. Storytelling
  3. Evaluate Clinical Cases | Gender Occupational StereotypingContraceptive HesitationSexual History BiasRacial BiasesPsychosocial BiasThe majority of the students were aware of biases and stereotypes in health care but could not assign or name them.In addition, they did not expect embedded stereotypes/implicit biases in the Clinical Case so that their consultation would not be biased. | | discovery(understanding the problem) | what bothers lecturers about today's solution | 1. Interview (research phase) | Interview (research phase):
  4. There are many examples of gender bias in medical practice. Instructing and sensitizing medical students about sex, gender, and diversity-specific medical knowledge and possible intersections with implicit biases is key to improving medical teaching and practice.
  5. It makes sense to define through the pedagogical goal which requirements are needed for the course. Furthermore, if the chatbot is used independently, it should not be forgotten to draw attention to the biases it contains. | | | 1. how the lessons are prepared and conducted
  6. How do the students document their work?
  7. How do the teachers review their process? | 1. Interview (research phase) | Interview (research phase):
  8. They prepare with notes before going to class.
  9. all students have an e-Portfolio in which they must document and upload discussion points after they finish class.
  10. The instructors review the document, give feedback on it and send it back. | | concept | how sensible is the use of a chatbot? | 1. Creating and testing minimum viable products (MVPs)
  11. Expertreview
  12. Usability Testiings | Training clinical reasoning through a chatbot in the role of a patient is possible, and was evaluated as positive and effective by the students. However, students expected feedback after taking a history. | | | What is the impact of the chatbot game on user knowledge and understanding of the clinical cases, and how can this impact be maximized? | 1. Focus Content > A/B Testing
  13. Chatbot-Role Assistent and Patient (mix) | A chatbot in the role of a patient is perfectly capable of training clinical reasoning. By collecting questions generated from the testings, the dialog can be refined and made more credible. It can also be used to identify still unknown biases. | | | What is the user's experience with the chatbot game, and how can the game be improved to enhance the user's experience? | 1. observation
  14. unmoderated testing online
  15. Survey | The answers need to be better adapted to what apatient would say, so that they appear more natural and credible. For example, a patient wouldn't tell how intense his/her pain is on a scale, if not askedTo enhance the learning or aha effect, feedback should be provided to the user after the interaction with the chatbot. This could be in written form or as a list. Which can afterwards be saved/exported and be taken to the course. | | | What are the main challenges or pain points that users experience when using the chatbot game, and how can these challenges be addressed? | 1. Creating user flow and user journey maps to visualize the user experience.
  16. User research to understand the needs and preferences. | The majority of participants expecting somekind of feedback, like a comparison of what aregood questions | | | Are gamification elements needed? | 1. Interview (research phase) | It was important to us here to uncouple the classic testing & learning environment. Therefore, we tried a more playful, more technically savvy look & feel.gamification elements would support the concept and MediBot is designed for that. | | create(Usability Test) | how will/should the chatbot be designed? | 1. Designing wireframes, prototypes, and mockups to visualize product features and user interface.
  17. User-Centered Design (UCD) approach
  18. Pitching and presenting concept to stakeholders
  19. Feedback Users | Majority of participants would use the chatbot to practice medical reasoning and think itwould benefit them | | | Are multimedia assets needed to support the clinical case? | 1. observe and analyse
  20. Interview | Illustrations or providing results of tests are a plus | | | how does the visual appearance influence the result of their clinical reasoning skills? | 1. observe and analyse
  21. Survey online | Majority of participants are aware of biases and expect a more natural setting withoutstereotypes/implicit biases embed in the scenarioThe participants expect the patients to look ill or in pain | | | how do users react to the chatbot? | 1. Usability Testing (wizard of oz)
  22. observe and analyse | The majority of participants were unsure of thepurpose of the chatbot and didn’t know what todo with itReaction were mostly positive and approvingly. Butthey weren’t sure of its purpose.The participants expect the patients to look ill or in pain | | | Does the Chatbot need an introduction? | 1. Usability Testing (Wizard of Oz)
  23. Interview
  24. online survey | It is absolutely important to subtly draw the user's attention to the issue of gender bias. so that the user does not feel fooled afterwards.From the first testing we we have found out participants are aware of some kind of biasesSome of the participants were uncertain about the usage and the purpose of the exercise. It is yet to investigate further how to guide the user in order to create a better experience. | | | how are the questions being posed by the students? | 1. Usability Testing (Wizard of Oz)
  25. online testing unmoderated
  26. Interaction with chatbot (dialogflow) | Students tend to ask several questions at once. Some use a language translator. | | | What kind of motivators should we use?"Furthermore, it is encouraged that different motivators should be used to urge students to engage the chatbot" (Essel et al., 2022). | 1. Usability Testing
  27. Feedback Users
  28. Ideation | Training clinical reasoning through a chatbot in the role of a patient is possible, and was evaluated as positive and effective by the students. However, students expected feedback after taking a history. | | | how is an intro with a storytelling approach appreciated by the users? | 1. online survey
  29. prototype | The storytelling approach wasconsidered moderately engaging by theparticipants.Majority of participants rated theirexperience with MediBot as veryinformative.In terms of meaningfulness the majorityof the participant rated Sundiata’s storyvery meaningful |