Effective Medical Visual Question Answering Using Dynamic Prompting and Decoding Knowledge Editing

DOI number:

10.1007/s41019-025-00291-0

Affiliation of Author(s):

School of Informatics, Xiamen University,

Journal:

Data Science and Engineering

Place of Publication:

China

Key Words:

"Medical visual question answering";"Pre-training";"Prompt tuning"; "Knowledge editing"

Abstract:

This paper addresses the issue of time-consuming manual prompt design and the limitations of current knowledge editing methods, which fail to utilize new knowledge fully. It integrates knowledge during the pre-training and answer prediction stages. Additionally, it proposes a medical visual question-and-answer model called DPDE, which is based on dynamic prompts and decoded knowledge editing. This model innovatively applies dynamic prompts to the field of medical visual question answering. It proposes a new paradigm for decoding knowledge editing, which enhances the language model’s knowledge editing capabilities in the decoding stage. In order to verify the effectiveness of the DPDE model, we conducted multiple sets of experiments to explore the model performance on three data sets. Experiments have proven that the dynamic prompt module and decoding knowledge editing module used in this model can effectively improve the performance of medical visual question-answering tasks.

First Author:

ZhiJie Zhou

Correspondence Author:

XiaoLi Wang

Co-author:

YeFan Huang,Xiaojie Hong

Indexed by:

Original Research

Discipline:

医疗人工智能 / 计算机科学 / 多模态 / 视觉语言模型

First-Level Discipline:

信息与计算科学 / 计算机科学 / 人工智能类

Document Type:

Journal Article

Translation or Not:

no

Date of Publication:

2025-06-24