A Multimodal Knowledge Graph for Medical Decision Making Centred Around Personal Values

It is important to incorporate patient values in healthcare decision-making systems so that the system gives patient-centered decision solutions in the decision-making process. The extant studies show less integration of both. Patient values can be summarized in several medical community review data. The existing knowledge of values is distilled and then combined with existing knowledge of clinical guidelines and rehabilitation. The patient's personalized information is obtained during the consultation process and matched with the general medical data, resulting in personalized recommendations that incorporate the patient's values. An ontology model is constructed based on five domains: population features, medical treatments, personal values, side effects and rehabilitation. Clinical guideline data, rehabilitation data and patient review data are mapped and linked through the ontology to obtain a general medical knowledge graph. The patient's voice, emotion and gesture information during the consultation process are analyzed and recognized as weighted entities, which are passed into the general medical knowledge graph to form a personalized multi-modal knowledge graph of the patient. A prototype system in a man with breast cancer was designed and implemented to demonstrate the method's feasibility.