Aryan GoyalPublications
Medical Imaging with Deep Learning (MIDL) 2026

A Diffusion-Driven Fine-Grained Nodule Synthesis Framework for Enhanced Lung Nodule Detection from Chest Radiographs

Aryan Goyal*, Shreshtha Singh*, Ashish Mittal*, Manoj Tadepalli, Piyush Kumar, Preetham Putha
Qure.ai · Indian Institute of Technology Bombay · *equal contribution
Generated nodule on a chest X-ray patch
Same patch, five sources

Ours (diffusion)

In short

Realistic nodules, with control over what they look like

Detecting lung nodules on chest X-rays is hard, partly because detectors don't see enough examples of every kind of nodule. We train a diffusion model that paints a nodule into any chosen spot of a real X-ray, and lets you choose its shape and size (through a mask), its border, its texture, whether it is calcified, and how subtle it is.

Each characteristic is learned by its own small add-on module (a LoRA adapter). To combine two characteristics in one nodule, we train the adapters with an extra loss that keeps them from interfering with each other. Adding the generated nodules to a detector's training data improves detection on in-house and public test sets.

80%
of generated nodules judged real by a radiologist panel (real nodules: 90%)
0.902 AUC
on JSRT, up from 0.856 with real data alone
0.932 AUC
on ChestX-ray14, up from 0.901
40→52%
of the most subtle JSRT nodules correctly classified
Explore by characteristic

What each characteristic looks like, and how well it works

Radiologists describe a nodule by a few traits that matter for diagnosis. Pick one to see its definition, examples generated by our model, and the numbers for that trait. Click any image to enlarge it.

Method

How it works

The model is trained in three stages. The first two teach it to draw realistic nodules where a mask says; the third adds control over individual characteristics. Click a stage.

Training pipeline with three stages
The full pipeline (Figure 3 of the paper). Flames mark trained parts; snowflakes mark frozen parts.

From a mask to a nodule

At test time you give the model an X-ray patch and a binary mask. It fills the masked region with a nodule of that shape and size and leaves the surrounding lung untouched. Drag the slider to compare the input and the output.

Generated patch
Original patch
OriginalGenerated
Binary nodule mask
The mask used for this example. The model follows its position, size and shape.

Why separate adapters instead of one conditioning signal

A natural idea is to feed the characteristic label into the model with classifier-free guidance (CFG), the same way the mask is fed in. In practice, guiding on both the mask and the labels made them compete and blurred the nodule's boundary. So the mask keeps CFG, and each characteristic gets its own LoRA adapter. The adapters follow their characteristic much more closely (lower FID is better):

A LoRA adapter freezes the model's weights \(W_0\) and learns a small low-rank update \(\Delta W = AB\) (rank 32 here). Each adapter adds about 6.2M parameters, under 1% of the frozen model, and sits in the attention layers of all 28 transformer blocks.

Subtlety

A slider for how hard a nodule is to see

Radiologists graded each training nodule from 1 (most subtle) to 5 (most obvious). Rather than training one adapter per grade, we tie the grade to the adapter's strength \(\alpha\): subtle nodules are trained with a small \(\alpha\) and obvious ones with a large \(\alpha\),

\[ W = W_0 + \alpha(s)\,\Delta W, \qquad \alpha(s) = 2^{2+s}, \quad s \in \{1,\dots,5\}. \]

At generation time, turning \(\alpha\) down makes the same nodule fainter. Try it on one mask:

Nodule at subtlety alpha 24
LoRA scale
α = 24
more visiblemore subtle

Radiologists were shown sets of three such versions and asked to order them from most obvious to most subtle. The panel's majority order matched ours in 80% of sets. Subtle nodules also helped the detector most where it was weakest; see the subtle-nodule results.

Two characteristics at once

Combining adapters without interference

Real nodules rarely have just one trait. A benign nodule is often calcified and smooth; a worrying one is often irregular and uneven in texture. So we want to switch on two adapters at the same time.

Problem 1: they fight over the same pixels

In natural images, a "style" adapter and an "object" adapter usually act on different parts of the image. Here every adapter acts on the same small nodule region, so their updates compete. One trait tends to dominate, and artifacts appear near the mask edge.

Problem 2: their updates overlap

Adapters trained separately end up pointing in similar directions in weight space. Most of an adapter's weights are near zero (60–70% below 0.01), so the few that matter collide when you average two adapters. This hurts every merging method we tried: linear merging, switching and ZipLoRA.

Our fix is to train the two adapters together and add a penalty whenever their updates overlap:

\[ \mathcal{L}_{\text{orth}} = \big\lVert W_1 W_2^{\top} \big\rVert_F^2 \]

The penalty is zero when the two updates are orthogonal, meaning they change the model along independent directions. With it, simply averaging the adapters works, and each trait stays adjustable through its own \(\alpha\).

Linear merge vs. ours
Pick a combination, then drag the slider across the patch.
Ours
Linear merge
Linear mergeOurs

Layer-wise overlap between adapters: separate training vs cross-training
Overlap between two adapters in each of the 28 transformer layers (lower is more independent). Blue: trained separately. Red: trained together with the orthogonality loss. Joint training cuts the overlap in almost every layer.

A fair worry is that the loss just shrinks the adapters toward zero, which would also make the overlap small. It doesn't: the jointly trained adapters actually have larger weights than separately trained ones, so the loss is separating their directions rather than switching them off.

Results

Does it help?

We check three things: whether radiologists find the nodules realistic and correctly characterised, whether the nodules improve a detector, and how the model compares with other nodule generators.

Three experienced radiologists took three blinded tests. Results can be counted two ways: majority vote (the panel's decision per image) or pooled (every individual answer counted). Whiskers show 95% confidence intervals.

Count
Takeaway: most generated nodules pass as real, and radiologists recognise the intended trait in 80–100% of cases by majority vote. Calcification is the hardest trait to read reliably.
Data and cost

Datasets

The in-house data has 1.2M frontal chest X-rays from partner hospitals, about 40k of them with nodules. Every nodule has a shape outline and characteristic labels from three experienced radiologists. JSRT and ChestX-ray14 are public test sets; their nodule boxes were refined into outlines with our segmentation model and checked by radiologists.

Diffusion trainDetector trainIn-house testJSRT testChestX-ray14 test
All X-rays1,100,000100,00012,000247500
With nodules28,00010,0002,00015466

Compute

  • Backbone: DiT-XL/2 (about 680M parameters, 28 transformer blocks) on 256×256 patches, encoded to 32×32 latents. Stage 1 took about a week and Stage 2 about two days, each on four NVIDIA L40 GPUs.
  • Adapters: about 10 hours per characteristic on four L40s; 20–30 hours for a jointly trained pair.
  • Generation: 250 DDIM steps, about 10 seconds per nodule.

Limitations

Some combinations of adapters produce nodules outside what the model saw in training. We evaluate pairs of characteristics only. The loss extends to three or more adapters by penalising every pair, but the data has too few nodules labelled with several traits at once for a fair test, and many three-way combinations are rare in practice.

Citation

BibTeX

@inproceedings{goyal2026nodule,
  title     = {A Diffusion-Driven Fine-Grained Nodule Synthesis Framework
               for Enhanced Lung Nodule Detection from Chest Radiographs},
  author    = {Goyal, Aryan and Singh, Shreshtha and Mittal, Ashish and
               Tadepalli, Manoj and Kumar, Piyush and Putha, Preetham},
  booktitle = {Medical Imaging with Deep Learning (MIDL)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2603.01659}
}