mespinosami commited on
Commit
2d087d4
·
verified ·
1 Parent(s): aefe3ba

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +113 -0
README.md ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ tags:
4
+ - earth-observation
5
+ - remote-sensing
6
+ - diffusion
7
+ - generative
8
+ - copernicus
9
+ - sentinel
10
+ - major-tom
11
+ - multimodal
12
+ - latent-diffusion
13
+ library_name: cop-gen
14
+ datasets:
15
+ - Major-TOM/COP-GEN-Benchmark
16
+ ---
17
+
18
+ ![copgen-banner-github](https://cdn-uploads.huggingface.co/production/uploads/63ea69a55c837d9968ebecc0/M3pNdfNV83ukK2czA2M-1.png)
19
+
20
+ # COP-GEN: Latent Diffusion Transformer for Copernicus Earth Observation Data
21
+
22
+ [![arXiv](https://img.shields.io/badge/arXiv-2603.03239-D12424)](https://arxiv.org/abs/2603.03239)
23
+ [![GitHub](https://img.shields.io/badge/GitHub-COP--GEN-black?logo=github)](https://github.com/miquel-espinosa/COP-GEN)
24
+ [![Website](https://img.shields.io/badge/🌐-Website-grey)](https://miquel-espinosa.github.io/cop-gen/)
25
+ [![HF Collection](https://img.shields.io/badge/🤗-Collection-yellow)](https://huggingface.co/collections/mespinosami/copgen)
26
+
27
+ COP-GEN is a generative foundation model for Copernicus Earth observation data. It learns a joint distribution over all major Copernicus modalities — Sentinel-1 SAR, Sentinel-2 multispectral (L1C and L2A), DEM, and LULC — enabling both unconditional generation and cross-modal conditional synthesis (e.g. generate S2 RGB from S1 SAR, or generate all modalities jointly).
28
+
29
+ ## Model Details
30
+
31
+ - **Developed by:** Miguel Espinosa, Eva Gmelich Meijling, Valerio Marsocci, Elliot J. Crowley, Mikolaj Czerkawski
32
+ - **Model type:** Latent Diffusion Transformer (multimodal, multi-resolution)
33
+ - **Modalities:** S1RTC (VV, VH), S2L1C (all bands + cloud mask), S2L2A (all bands), DEM, LULC, timestamps, lat-lon
34
+ - **License:** CC-BY-4.0
35
+ - **Paper:** [arXiv:2603.03239](https://arxiv.org/abs/2603.03239)
36
+ - **Repository:** [github.com/miquel-espinosa/COP-GEN](https://github.com/miquel-espinosa/COP-GEN)
37
+
38
+ ### Architecture
39
+
40
+ COP-GEN operates in a shared latent space produced by a set of modality-specific KL-regularised VAEs. The diffusion backbone is a transformer trained jointly over all modalities, supporting arbitrary conditioning at inference time — any subset of modalities can be held as conditions while the rest are generated.
41
+
42
+ ## Uses
43
+
44
+ ### Direct Use
45
+
46
+ Generate synthetic Copernicus EO scenes, either unconditionally or conditioned on one or more input modalities. Useful for data augmentation, gap-filling missing modalities, and studying cross-sensor relationships.
47
+
48
+ ### Downstream Use
49
+
50
+ The latent representations and generated samples can serve as inputs to downstream EO tasks: land cover classification, change detection, cloud removal, SAR-to-optical translation, and more.
51
+
52
+ ## How to Get Started
53
+
54
+ ```python
55
+ from libs.copgen import CopgenModel
56
+
57
+ model = CopgenModel(
58
+ model_path="path/to/model_checkpoint.pth",
59
+ config_path="path/to/model_config.py"
60
+ )
61
+
62
+ # Conditional generation: provide one or more modalities as conditions
63
+ samples = model.generate(
64
+ modalities=["S2L2A_B02_B03_B04_B08", "S1RTC_vh_vv"],
65
+ conditions={"S1RTC_vh_vv": s1_tensor},
66
+ n_samples=4,
67
+ )
68
+
69
+ # Unconditional generation
70
+ samples = model.generate(
71
+ modalities=["S2L2A_B02_B03_B04_B08", "S1RTC_vh_vv"],
72
+ n_samples=4,
73
+ )
74
+ ```
75
+
76
+ See [examples/conditional_generation.py](https://github.com/miquel-espinosa/COP-GEN/blob/main/examples/conditional_generation.py) and [examples/unconditional_generation.py](https://github.com/miquel-espinosa/COP-GEN/blob/main/examples/unconditional_generation.py) for full worked examples.
77
+
78
+ ## Training Details
79
+
80
+ ### Training Data
81
+
82
+ Trained on [Major-TOM](https://huggingface.co/Major-TOM) global Copernicus data, covering Sentinel-1 RTC, Sentinel-2 L1C and L2A, DEM, and LULC. A pre-compiled Edinburgh subset is available at [mespinosami/copgen-edinburgh-subset](https://huggingface.co/datasets/mespinosami/copgen-edinburgh-subset) for local development and reproduction.
83
+
84
+ ### Training Procedure
85
+
86
+ 1. Modality-specific KL-VAEs are trained separately per modality and resolution.
87
+ 2. All modalities are encoded into a shared latent space.
88
+ 3. A diffusion transformer backbone is trained jointly over the merged latents, with random masking of modalities to enable conditional generation at inference.
89
+
90
+ See the [GitHub README](https://github.com/miquel-espinosa/COP-GEN) for full training instructions.
91
+
92
+ ## Evaluation
93
+
94
+ Evaluated on the [COP-GEN-Benchmark](https://huggingface.co/datasets/Major-TOM/COP-GEN-Benchmark) test set (495 held-out global scenes). Distribution-level metrics (FID and related) are reported in Table 1 of the paper. To reproduce:
95
+
96
+ ```bash
97
+ pip install -r benchmark/stochastic/requirements.txt
98
+ python -m benchmark.stochastic.run --output metrics.csv
99
+ ```
100
+
101
+ ## Citation
102
+
103
+ ```bibtex
104
+ @article{copgen2026,
105
+ title = {COP-GEN: Latent Diffusion Transformer for Copernicus Earth
106
+ Observation Data},
107
+ author = {Espinosa, Miguel and Gmelich Meijling, Eva and Marsocci,
108
+ Valerio and Crowley, Elliot J. and Czerkawski, Mikolaj},
109
+ year = {2026},
110
+ journal = {arXiv preprint arXiv:2603.03239},
111
+ url = {https://arxiv.org/abs/2603.03239},
112
+ }
113
+ ```