| What action should be taken if there is an allegation of discrimination or bias in an AI technology? | any allegation of discrimination or indication of bias in an AI technology, the operation of such a system must be temporarily discontinued | The existence of bias in the data set can potentially affect the functioning of AI technology. If there is any allegation of discrimination or indication of bias in an AI technology, the operation of such a system must be temporarily discontinued. The manufacturer has the responsibility to eliminate the bias. | 107 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | If there is any allegation of discrimination or indication of bias in an AI technology, the operation of such a system must be temporarily discontinued. | 37 | page=37,block=0 | 0.850 | valid |
| What is a potential risk when AI technologies are implemented in healthcare? | used in healthcare, there is a possibility that the system can function independently and undermine human autonomy | When AI technologies are used in healthcare, there is a possibility that the system can function independently and undermine human autonomy. The application of AI technology into healthcare may transfer the responsibility of decision-making into the hands of machines. | 25 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | When AI technologies are used in healthcare, there is a possibility that the system can function independently and undermine human autonomy. | 24 | page=24,block=3 | 0.850 | valid |
| What do the Medical Device Rules, 2017 and its amendments in 2020 include in the definition of medical device? | any software or an accessory intended to be used for a medical purpose | Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare 48 State eHealth Authorities and Health Information Exchanges to standardize and regulate the processes related to collection, storing, transmission and use of digital health data; and to ensure reliability, data privacy, confidentiality and security of digital health data”[28]. Regulatory frameworks lay guidelines to assess the product’s probable benefits and risk as well as provide an enabling environment. The Medical Device Rules, 2017 and its amendments in 2020 have expanded the definition of medical device and includes any software or an accessory intended to be used for a medical purpose[29]. there is a need to further develop and establish norms for testing and validating SaMD in India. | 630 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | The Medical Device Rules, 2017 and its amendments in 2020 have expanded the definition of medical device and includes any software or an accessory intended to be used for a medical purpose[29]. |
| What does epidemiology focus on in relation to diseases? | identification of the factors and determinants of the diseases and the trends, patterns and prediction of diseases | AI technology can assist in self-monitoring of personal health-related parameters such as intake of nutrition, physical activity, blood pressure, glucose, lipids for identifying high risk group. AI based health coaching systems and smartphone apps using neural network and ML methods could provide solutions for medication adherence, motivation, reminders and building a care network[13]. Chatbots and robotic assistants can empower patients in self-management of Non-Communicable Diseases (NCDs) and improve decision making. Epidemiology and Prevention of Disease Epidemiology is the cornerstone of public health that guides policy decisions and evidence-based practice. The science involves identification of the factors and determinants of the diseases and the trends, patterns and prediction of diseases. | 693 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | The science involves identification of the factors and determinants of the diseases and the trends, patterns and prediction of diseases. | 18 |
| What does moderation or human intervention in AI technology involve? | Does the AI technology require human intervention/ moderation? If yes give proper details for instance, who will control the access to AI technology | Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare 51 14. Data collection, storage, sharing and access to data Describe the source of data, data collection method and storage, data access procedures, and data security 15. Informed consent Describe the type of consent (Signed consent, Verbal/Oral consent, Witnessed consent and Audio-Video (AV) consent). Ethical approval from an Ethics Committee! Who will obtain the informed consent (PI/Co-PI, Nurse/Counselor, Research Staff or Other.)? And mechanism for consent withdrawal. 16. Right to be forgotten Can a person retrieve and erase all their records or not? And if yes then how? Describe the process. 17. Moderation (human in the loop) Does the AI technology require human intervention/ moderation? If yes give proper details for instance, who will control the access to AI technology | 739 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | Moderation (human in the loop) Does the AI technology require human intervention/ moderation? If yes give proper details for instance, who will control the access to AI technology |
| What does the concept 'Human In The Loop' (HITL) emphasize in healthcare? | places human beings in a supervisory role and is more relevant for healthcare purposes | Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare 22 for health should always be appropriately supervised. Open source software must also follow best ethical practices so as to ensure that ethical consideration are not compromised at any point of time in the name of innovation. iii. The concept ‘Human In The Loop’ (HITL) places human beings in a supervisory role and is more relevant for healthcare purposes. This will ensure an individualized decision making by the health professionals keeping the interest of the patient in the center. Adoption of the HITL principle throughout the development and deployment of AI for Health also helps in optimal sharing of accountability by the team involved in development and deployment of AI-based algorithms. iv. | 373 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | The concept ‘Human In The Loop’ (HITL) places human beings in a supervisory role and is more relevant for healthcare purposes. | 35 |
| What does the term 'AI technology' include as defined in the guidelines? | AI technologies, AI applications, AI models, AI products, AI-driven solutions, or AI-based solutions | Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare 3 There is no standard and commonly accepted term and therefore the term AI technology has been used to refer to AI technologies, AI applications, AI models, AI products, AI-driven solutions, or AI-based solutions in the entire document. Scope These guidelines apply to AI based tools created for all biomedical and health research and applications involving human participants and/or their biological data. Considering the far-reaching implications of AI-based technologies in healthcare, these guidelines are applicable to health professionals, technology developers, researchers, entrepreneurs, hospitals, research institutions, organization(s), and laypersons who want to utilize health data for biomedical research and healthcare delivery using AI technology and techniques. | 213 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare 3 There is no standard and commonly accepted term and therefore the term AI technology has been used to refer to AI technologies, AI applications, AI models, AI products, AI-driven solutions, or AI-based solutions in the entire document. |
| What does the term 'algorithm' refer to in the context of artificial intelligence? | a set of rules/instructions that step- by-step define how a work is to be executed upon in order to get the expected results. | Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare 58 Glossary: 1. Algorithms: Algorithm refers to a set of rules/instructions that step- by-step define how a work is to be executed upon in order to get the expected results. 2. Artificial Intelligence (AI): The simulation of human intelligence processes by machines, especially computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using rules to reach approximate or definite conclusions) and self-correction. 3. Black-box: In science, computing, and engineering, a black box is a device, system or object which can be viewed in terms of its inputs and outputs (or transfer characteristics), without any knowledge of its internal workings. Its implementation is ‘opaque’ and is therefore referred to as ‘black box’. 4. | 148 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | Algorithms: Algorithm refers to a set of rules/instructions that step- by-step define how a work is to be executed upon in order to get the expected results. 2. |
| What does the term 'digital divide' refer to? | uneven distribution of access to, use of or effect of information and communication technologies | It would reduce the bias associated with the AI technologies. vi. AI technologies may include local languages in their user interface to overcome the language barrier associated with the accessibility of technology. It increases AI technology acceptance and more comprehensive user compliance. vii. Some AI technologies may need internet connectivity, technical expertise, electricity, and other infrastructure. Relevant stakeholders of AI technology should ensure that adequate infrastructure is available for the smooth and optimum functioning of an AI technology that is intended to be used in a low-resource setting. viii. Digital divide refers to uneven distribution of access to, use of or effect of information and communication technologies among any | 652 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | Digital divide refers to uneven distribution of access to, use of or effect of information and communication technologies among any | 39 | page=39,block=0 |
| What does the term 'explainable' refer to in the field of AI-based algorithms? | the results and interpretations provided by AI-based algorithms should be explainable based on scientific plausibility | Ethical Guidelines for Application of Artificial Intelligence in Biomedical Research and Healthcare 17 iv. Explainable, i.e., the results and interpretations provided by AI-based algorithms should be explainable based on scientific plausibility. It should be possible to understand the logic behind the results obtained so that AI technology is valid, reliable and responsible. The lack of information about the decision-making by AI algorithms has prompted some to label it as a “black box” which can prove a deterrent to its wider adoption. A well explainable AI-based solution is expected to improve the confidence of both the patient and the health professionals. v. A diagnostic AI technology may produce results that are not in line with the physicians’ views/ decision on disease. Such situations may question the credibility of the system as well as the doctor. | 126 | definition | 1.000 | https://www.icmr.gov.in/icmrobject/custom_data/pdf/Ethical-guidelines/Ethical_Guidelines_AI_Healthcare_2023.pdf | Explainable, i.e., the results and interpretations provided by AI-based algorithms should be explainable based on scientific plausibility. |