Haochen Zhang

Haochen Zhang

Applied Scientist at Amazon. Ph.D. and M.S. in ECE from UCSD; M.Eng. and B.Eng. in EE from USTC.

OCTA-based AMD Stage Grading Enhancement via Class-Conditioned Style Transfer

Published in International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2024

In this paper, we enable AMD stage classifier training on cross-instrument OCTA data (Heidelberg and Optovue). Because the Heidelberg and Optovue datasets contain retinas from different patients, paired data are unavailable. To address this limitation, we employ the CycleGAN framework, which compensates for the lack of paired training samples by introducing a cycle-consistency constraint.

Illustration of our framework utilizing CycleGAN to transfer samples between Heidelberg and Optovue domains. For simplicity, only one channel of OCTA projection is displayed, and the identity loss is omitted. In the Green cycle, a real Optovue sample $x$ undergoes translation to the Heidelberg domain using the generator $G_{O2H}(\cdot)$, resulting in $\hat{y}$. Subsequently, $\hat{y}$ is translated back to the Optovue domain through another generator $G_{H2O}(\cdot)$, producing the recovered Optovue image $\tilde{x}$. By imposing an L1 loss between $x$ and $\tilde{x}$, we ensures faithful reconstruction, preserving both content and style in Optovue domain. Adversarial loss between $y$ and $ \hat{y}$ enforces the transferred image share same style as samples in Heidelberg domain.
Illustration of our framework utilizing CycleGAN to transfer samples between Heidelberg and Optovue domains. For simplicity, only one channel of OCTA projection is displayed, and the identity loss is omitted. In the Green cycle, a real Optovue sample $x$ undergoes translation to the Heidelberg domain using the generator $G_{O2H}(\cdot)$, resulting in $\hat{y}$. Subsequently, $\hat{y}$ is translated back to the Optovue domain through another generator $G_{H2O}(\cdot)$, producing the recovered Optovue image $\tilde{x}$. By imposing an L1 loss between $x$ and $\tilde{x}$, we ensures faithful reconstruction, preserving both content and style in Optovue domain. Adversarial loss between $y$ and $ \hat{y}$ enforces the transferred image share same style as samples in Heidelberg domain.

In summary, the vanilla training loss of CycleGAN without class constraint is:

$ l_{cyclegan}(x,y) = l_{cyc}(x,\tilde{x}) + l_{cyc}(y,\tilde{y}) + \alpha( l_{GAN}(x, \hat{x}) + l_{GAN}(y, \hat{y})) + \beta ( l_{idt}(x, G_{H2O}(x)) + l_{idt}(y, G_{O2H}(y))) $

where $\alpha, \beta$ are loss weights.

To better support classifier training, we further propose explicit class constraints—applied in both supervised and unsupervised settings—to enhance the machine-recognition quality of the translated images.

The disassembly diagram showing the blue (a) and green (b) cycles, respectively, with the proposed supervised (a) and unsupervised (b) class constraints. In short, we apply a pretrained classifier $cls(\cdot)$ to the transferred image and enforcing it to remain within the same class.
The disassembly diagram showing the blue (a) and green (b) cycles, respectively, with the proposed supervised (a) and unsupervised (b) class constraints. In short, we apply a pretrained classifier $cls(\cdot)$ to the transferred image and enforcing it to remain within the same class.

Combined with the vanilla loss, the final CycleGAN training loss with supervised class constraints is:

$l_{total} = l_{cyclegan}(x,y) + \gamma( l_{cos}(label(x), cls(\hat{y})) + l_{cos}(label(y), cls(\hat{x})))$

And the final CycleGAN training loss incorporating unsupervised class constraints is:

$l_{total} = l_{cyclegan}(x,y) + \gamma( l_{cos}(cls(x), cls(\hat{y})) + l_{cos}(cls(y), cls(\hat{x})))$

where $\gamma$ is loss weight, $cls(\cdot)$ is a pretrained feature extractor.

Experimental results show that the CycleGAN-based translator becomes particularly effective when trained with these class-related constraints.

Experimental results.
Experimental results.

Paper | Slides

Please also refer to our clinical paper

Citation:

@inproceedings{zhang2024octa,
  title={OCTA-based AMD Stage Grading Enhancement via Class-Conditioned Style Transfer},
  author={Zhang, Haochen and Heinke, Anna and Broniarek, Krzysztof and Galang, Carlo Miguel B and Deussen, Daniel N and Nagel, Ines D and Michalska-Ma{\l}ecka, Katarzyna and Bartsch, Dirk-Uwe G and Freeman, William R and Nguyen, Truong Q and others},
  booktitle={International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)},
  pages={1--5},
  year={2024}
}

@article{heinke2024cross,
  title={Cross-instrument optical coherence tomography-angiography (OCTA)-based prediction of age-related macular degeneration (AMD) disease activity using artificial intelligence},
  author={Heinke, Anna and Zhang, Haochen and Broniarek, Krzysztof and Michalska-Ma{\l}ecka, Katarzyna and Elsner, Wyatt and Galang, Carlo Miguel B and Deussen, Daniel N and Warter, Alexandra and Kalaw, Fritz and Nagel, Ines and others},
  journal={Scientific Reports},
  volume={14},
  number={1},
  pages={27085},
  year={2024}
}