COMe-SEE: Cross-modality Semantic Embedding Ensemble for Generalized Zero-Shot Diagnosis of Chest Radiographs
文献类型: 会议论文
第一作者: Angshuman Paul
作者: Angshuman Paul 1 ; Thomas C. Shen 1 ; Niranjan Balachandar 1 ; Yuxing Tang 1 ; Yifan Peng 2 ; Zhiyong Lu 2 ; Ronald M. Summers 1 ;
作者机构: 1.Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences, National Institutes of Health Clinical Center
2.Biomedical Text Mining Group, National Center for Biotechnology Information, National Institutes of Health
关键词: Zero-shot learning;Chest x-ray;Semantic saliency;Autoencoder;Ensemble
会议名称: Workshop on Interpretability of Machine Intelligence in Medical Image Computing
主办单位:
页码: 103-111
摘要: Zero-shot learning, in spite of its recent popularity, remains an unexplored area for medical image analysis. We introduce a first-of-its-kind generalized zero-shot learning (GZSL) framework that utilizes information from two different imaging modalities (CT and x-ray) for the diagnosis of chest radiographs. Our model makes use of CT radiology reports to create a semantic space consisting of signatures corresponding to different chest diseases and conditions. We introduce a CrOss-Modality Semantic Embedding Ensemble (COMe-SEE) for zero-shot diagnosis of chest x-rays by relating an input x-ray to a signature in the semantic space. The ensemble, designed using a novel semantic saliency preserving autoencoder, utilizes the visual and the semantic saliency to facilitate GZSL. The use of an ensemble not only helps in dealing with noise but also makes our model useful across different datasets. Experiments on two publicly available datasets show that the proposed model can be trained using one dataset and still be applied to data from another source for zero-shot diagnosis of chest x-rays.
分类号: TP181-53
- 相关文献
[1]Multi-Label Classification for Fashion Data: Zero-Shot Classifiers via Few-Shot Learning on Large Language Models. Dongming Jiang,Abhishek Shah,Stanley Yeung,Jessica Zhu,Karan Singh,George Goldenberg. 2024
[2]Confidence-based Ensemble for GBM brain tumor segmentation. Jing Huo,Eva M. van Rikxoort,Kazunori Okada,Hyun J. Kim,Whitney Pope,Jonathan Goldin,Matthew Brown. 2011
[3]Sampling-Based Ensemble Segmentation against Inter-operator Variability. Jing Huo,Kazunori Okada,Whitney Pope,Matthew Brown. 2011
作者其他论文 更多>>
-
Automated Measurement of Pericoronary Adipose Tissue Attenuation and Volume in Ct Angiography
作者:Andrew M. Nguyen;Tejas Sudharshan Mathai;Liangchen Liu;Jianfei Liu;Ronald M. Summers
关键词:Three-dimensional displays;Volume measurement;Angiography;Biomedical measurement;Attenuation measurement;Attenuation;Fats
-
Automated Classification of Multi-Parametric Body MRI Series
作者:Boah Kim;Tejas Sudharshan Mathai;Kimberly Helm;Ronald M. Summers
关键词:Protocols;Sensitivity;Automation;Magnetic resonance imaging;Radiology;Pelvis;DICOM
-
Semantic Image Synthesis for Abdominal CT
作者:Yan Zhuang;Benjamin Hou;Tejas Sudharshan Mathai;Pritam Mukherjee;Boah Kim;Ronald M. Summers
关键词:CT;Abdomen;Diffusion model;Semantic image synthesis
-
Anatomical Location-Guided Deep Learning-Based Genetic Cluster Identification of Pheochromocytomas and Paragangliomas from CT Images
作者:Bikash Santra;Abhishek Jha;Pritam Mukherjee;Mayank Patel;Karel Pacak;Ronald M. Summers
关键词:Pheochromocytoma;Paraganglioma;Genetic mutations;Transformer;CT images;Deep learning;Radiogenomics;PPGL
-
Improving Automatic Segmentation of lymphoma with Additional Medical Knowledge Priors
作者:Liangchen Liu;Benjamin Hou;Manas Kumar Nag;Jianfei Liu;Navid Hasani;Thomas C. Shen;Yingying Zhu;Babak Saboury;Jing Xiao;Ronald M. Summers
关键词:Training;Image segmentation;Shape;Computed tomography;Pipelines;Positron emission tomography;Radiotracer
-
A Dual-Branch Network with Mixed and Self-Supervision for Medical Image Segmentation: An Application to Segment Edematous Adipose Tissue
作者:Jianfei Liu;Omid Shafaat;Ronald M. Summers
关键词:Self-supervision;Mixed supervision;Dual-branch;Residual u-Net;Edema
-
A Dual-Branch Network with Mixed and Self-Supervision for Medical Image Segmentation: An Application to Segment Edematous Adipose Tissue
作者:Jianfei Liu;Omid Shafaat;Ronald M. Summers
关键词:Self-supervision;Mixed supervision;Dual-branch;Residual U-Net;Edema


