【图像去噪】第七期论文复现赛——SwinIR

作者引入Swin-T结构应用于低级视觉任务，包括图像超分辨率重建、图像去噪、图像压缩伪影去除。SwinIR网络由一个浅层特征提取模块、深层特征提取模块、重建模块构成。重建模块对不同的任务使用不同的结构。浅层特征提取就是一个3×3的卷积层。深层特征提取是k个RSTB块和一个卷积层加残差连接构成。

☞☞☞AI 智能聊天, 问答助手, AI 智能搜索, 免费无限量使用 DeepSeek R1 模型☜☜☜

论文复现——Low-level算法 SwinIR (去噪)

SwinIR: Image Restoration Using Swin Transformer——基于Swin Transformer的用于图像恢复的强基线模型

官方源码：https://github.com/JingyunLiang/SwinIR

复现地址：https://github.com/sldyns/SwinIR_paddle

脚本任务：https://aistudio.baidu.com/aistudio/clusterprojectdetail/3792518

PaddleGAN版本：https://github.com/PaddlePaddle/PaddleGAN/blob/develop/docs/zh_CN/tutorials/swinir.md

1. 简介

 SwinIR的结构比较简单，如果看过Swin-Transformer的话就没什么难点了。作者引入Swin-T结构应用于低级视觉任务，包括图像超分辨率重建、图像去噪、图像压缩伪影去除。SwinIR网络由一个浅层特征提取模块、深层特征提取模块、重建模块构成。重建模块对不同的任务使用不同的结构。浅层特征提取就是一个3×3的卷积层。深层特征提取是k个RSTB块和一个卷积层加残差连接构成。每个RSTB（Res-Swin-Transformer-Block）由L个STL和一层卷积加残差连接构成。

2. 复现精度

在 CBSD68 测试集上测试，达到验收最低标准34.32：

Pytorch34.42Paddle34.32

注：源代码八卡训练的iteration为 1,600,000，我们四卡只训练到了 426,000 就超时停止了.

3. 数据集、预训练模型、文件结构

3.1训练数据

DIV2K (800 training images) + Flickr2K (2650 images) + BSD500 (400 training&testing images) + WED(4744 images)

已经整理好的数据：放在了 Ai Studio 里.

训练数据放在：data/trainsets/trainH 下

3.2测试数据

测试数据为 CBSD68：放在了 Ai Studio 里.

解压到：data/testsets/CBSD68

In [2]

# 解压!cd data && unzip -oq -d testsets/ data147756/CBSD68.zip!cd data && unzip -oq -d trainsets/ data149405/trainH.zip

   In [3]

# 添加软链接!cd work && ln -s ../data/trainsets trainsets && ln -s ../data/testsets testsets

3.2 预训练模型

已放在文件夹 work/pretrained_models 下：

官方预训练模型，已转为 paddle 的，名为 005_colorDN_DFWB_s128w8_SwinIR-M_noise15.pdparams.复现的模型，名为 SwinIR_paddle.pdparams.

3.3 文件结构

SwinIR_Paddle    |-- data                                        # 数据相关文件    |-- models                                   # 模型相关文件    |-- options                                   # 训练配置文件    |-- trainsets         |-- trainH                                # 训练数据    |-- testsets         |-- CBSD68                             # 测试数据    |-- test_tipc                                  # TIPC: Linux GPU/CPU 基础训练推理测试    |-- pretrained_models                  # 预训练模型    |-- utils                                          # 一些工具代码    |-- config.py                                  # 配置文件    |-- generate_patches_SIDD.py      # 生成数据patch    |-- infer.py                                     # 模型推理代码    |-- LICENSE                                   # LICENSE文件    |-- main_test_swinir.py                  # 模型测试代码    |-- main_train_psnr.py                   # 模型训练代码    |-- main_train_tipc.py                    # TICP训练代码    |-- README.md                             # README.md文件    |-- train.log                                    # 训练日志

4. 环境依赖

PaddlePaddle >= 2.3.2

scikit-image == 0.19.3

In [1]

!pip install scikit-image

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simpleCollecting scikit-image  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2d/ba/63ce953b7d593bd493e80be158f2d9f82936582380aee0998315510633aa/scikit_image-0.19.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.5 MB)     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.5/13.5 MB 717.8 kB/s eta 0:00:0000:0100:01Requirement already satisfied: scipy>=1.4.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image) (1.6.3)Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image) (21.3)Requirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image) (2.4)Requirement already satisfied: numpy>=1.17.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image) (1.19.5)Collecting tifffile>=2019.7.26  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d8/38/85ae5ed77598ca90558c17a2f79ddaba33173b31cf8d8f545d34d9134f0d/tifffile-2021.11.2-py3-none-any.whl (178 kB)     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.9/178.9 kB 25.2 kB/s eta 0:00:00a 0:00:01Requirement already satisfied: pillow!=7.1.0,!=7.1.1,!=8.3.0,>=6.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image) (8.2.0)Collecting PyWavelets>=1.1.1  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ae/56/4441877073d8a5266dbf7b04c7f3dc66f1149c8efb9323e0ef987a9bb1ce/PyWavelets-1.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.4 MB)     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.4/6.4 MB 1.3 MB/s eta 0:00:0000:0100:01Requirement already satisfied: imageio>=2.4.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image) (2.6.1)Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from networkx>=2.2->scikit-image) (4.4.2)Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging>=20.0->scikit-image) (3.0.9)Installing collected packages: tifffile, PyWavelets, scikit-imageSuccessfully installed PyWavelets-1.3.0 scikit-image-0.19.3 tifffile-2021.11.2[notice] A new release of pip available: 22.1.2 -> 22.3[notice] To update, run: pip install --upgrade pip

5. 快速开始

配置文件在work/options下，可修改学习率、batch_size等参数

5.1 模型训练

为更好的体验，建议使用单机多卡训练，例如fork并运行脚本任务： https://aistudio.baidu.com/aistudio/clusterprojectdetail/3792518

In [ ]

# 单机单卡!cd work && python main_train_psnr.py --opt options/train_swinir_multi_card_32.json

/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/__init__.py:107: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import MutableMapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Iterable, Mapping/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working  from collections import Sizedexport CUDA_VISIBLE_DEVICES=0,1,2,3number of GPUs is: 4/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/distributed/parallel.py:159: UserWarning: Currently not a parallel execution environment, `paddle.distributed.init_parallel_env` will not do anything.  "Currently not a parallel execution environment, `paddle.distributed.init_parallel_env` will not do anything."1LogHandlers setup!22-10-17 16:47:13.877 :   task: swinir_denoising_color_15  model: plain  gpu_ids: [0, 1, 2, 3]  dist: True  n_channels: 3  path:[    root: denoising    pretrained_netG: None    pretrained_netE: None    task: denoising/swinir_denoising_color_15    log: denoising/swinir_denoising_color_15    options: denoising/swinir_denoising_color_15/options    models: denoising/swinir_denoising_color_15/models    images: denoising/swinir_denoising_color_15/images    pretrained_optimizerG: None  ]  datasets:[    train:[      name: train_dataset      dataset_type: dncnn      dataroot_H: trainsets/trainH      dataroot_L: None      H_size: 128      sigma: 15      sigma_test: 15      dataloader_shuffle: True      dataloader_num_workers: 8      dataloader_batch_size: 2      phase: train      scale: 1      n_channels: 3    ]    test:[      name: test_dataset      dataset_type: dncnn      dataroot_H: testsets/CBSD68      dataroot_L: None      sigma: 15      sigma_test: 15      phase: test      scale: 1      n_channels: 3    ]  ]  netG:[    net_type: swinir    upscale: 1    in_chans: 3    img_size: 128    window_size: 8    img_range: 1.0    depths: [6, 6, 6, 6, 6, 6]    embed_dim: 180    num_heads: [6, 6, 6, 6, 6, 6]    mlp_ratio: 2    upsampler: None    resi_connection: 1conv    init_type: default    scale: 1  ]  train:[    G_lossfn_type: charbonnier    G_lossfn_weight: 1.0    G_charbonnier_eps: 1e-09    E_decay: 0.999    G_optimizer_type: adam    G_optimizer_lr: 0.0002    G_optimizer_wd: 0    G_optimizer_clipgrad: None    G_optimizer_reuse: True    G_scheduler_type: MultiStepLR    G_scheduler_milestones: [800000, 1200000, 1400000, 1500000, 1600000]    G_scheduler_gamma: 0.5    G_regularizer_orthstep: None    G_regularizer_clipstep: None    G_param_strict: True    E_param_strict: True    manual_seed: 42    checkpoint_test: 2000    checkpoint_save: 2000    checkpoint_print: 400    F_feature_layer: 34    F_weights: 1.0    F_lossfn_type: l1    F_use_input_norm: True    F_use_range_norm: False    G_optimizer_betas: [0.9, 0.999]    G_scheduler_restart_weights: 1  ]  opt_path: options/train_swinir_multi_card_32.json  is_train: True  merge_bn: False  merge_bn_startpoint: -1  scale: 1  find_unused_parameters: True  use_static_graph: False  num_gpu: 4  nranks: 1Random seed: 42Dataset: Denosing on AWGN with fixed sigma. Only dataroot_H is needed.Dataset [DatasetDnCNN - train_dataset] is created.22-10-17 16:47:13.913 : Number of train images: 8,694, iters: 4,347Dataset: Denosing on AWGN with fixed sigma. Only dataroot_H is needed.Dataset [DatasetDnCNN - test_dataset] is created.W1017 16:47:14.995301  3546 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2W1017 16:47:14.999406  3546 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.Pass this initialization! Initialization was done during network definition!Pass this initialization! Initialization was done during network definition!Training model [ModelPlain] is created.Copying model for E ...22-10-17 16:47:15.620 : Networks name: SwinIRParams number: Tensor(shape=[1], dtype=int64, place=Place(gpu:0), stop_gradient=False,       [11504163])Net structure:SwinIR(  (conv_first): Conv2D(3, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)  (patch_embed): PatchEmbed(    (norm): LayerNorm(normalized_shape=[180], epsilon=1e-05)  )  (patch_unembed): PatchUnEmbed()  (pos_drop): Dropout(p=0.0, axis=None, mode=upscale_in_train)  (layers): LayerList(    (0): RSTB(      (residual_group): BasicLayer(dim=180, input_resolution=(128, 128), depth=6        (blocks): LayerList(          (0): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): Identity()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (1): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (2): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (3): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (4): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (5): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )        )      )      (conv): Conv2D(180, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)      (patch_embed): PatchEmbed()      (patch_unembed): PatchUnEmbed()    )    (1): RSTB(      (residual_group): BasicLayer(dim=180, input_resolution=(128, 128), depth=6        (blocks): LayerList(          (0): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (1): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (2): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (3): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (4): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (5): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )        )      )      (conv): Conv2D(180, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)      (patch_embed): PatchEmbed()      (patch_unembed): PatchUnEmbed()    )    (2): RSTB(      (residual_group): BasicLayer(dim=180, input_resolution=(128, 128), depth=6        (blocks): LayerList(          (0): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (1): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (2): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (3): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (4): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (5): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )        )      )      (conv): Conv2D(180, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)      (patch_embed): PatchEmbed()      (patch_unembed): PatchUnEmbed()    )    (3): RSTB(      (residual_group): BasicLayer(dim=180, input_resolution=(128, 128), depth=6        (blocks): LayerList(          (0): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (1): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (2): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (3): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (4): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (5): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )        )      )      (conv): Conv2D(180, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)      (patch_embed): PatchEmbed()      (patch_unembed): PatchUnEmbed()    )    (4): RSTB(      (residual_group): BasicLayer(dim=180, input_resolution=(128, 128), depth=6        (blocks): LayerList(          (0): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (1): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (2): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (3): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (4): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (5): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )        )      )      (conv): Conv2D(180, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)      (patch_embed): PatchEmbed()      (patch_unembed): PatchUnEmbed()    )    (5): RSTB(      (residual_group): BasicLayer(dim=180, input_resolution=(128, 128), depth=6        (blocks): LayerList(          (0): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (1): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (2): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (3): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (4): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=0, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )          (5): SwinTransformerBlock(dim=180, input_resolution=(128, 128), num_heads=6, window_size=8, shift_size=4, mlp_ratio=2            (norm1): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (attn): WindowAttention(dim=180, window_size=(8, 8), num_heads=6              (qkv): Linear(in_features=180, out_features=540, dtype=float32)              (attn_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (proj): Linear(in_features=180, out_features=180, dtype=float32)              (proj_dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)              (softmax): Softmax(axis=-1)            )            (drop_path): DropPath()            (norm2): LayerNorm(normalized_shape=[180], epsilon=1e-05)            (mlp): Mlp(              (fc1): Linear(in_features=180, out_features=360, dtype=float32)              (fc2): Linear(in_features=360, out_features=180, dtype=float32)              (act): GELU(approximate=False)              (dropout): Dropout(p=0.0, axis=None, mode=upscale_in_train)            )          )        )      )      (conv): Conv2D(180, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)      (patch_embed): PatchEmbed()      (patch_unembed): PatchUnEmbed()    )  )  (norm): LayerNorm(normalized_shape=[180], epsilon=1e-05)  (conv_after_body): Conv2D(180, 180, kernel_size=[3, 3], padding=1, data_format=NCHW)  (conv_last): Conv2D(180, 3, kernel_size=[3, 3], padding=1, data_format=NCHW))22-10-17 16:47:15.819 :  |  mean  |  min   |  max   |  std   || shape                | -0.002 | -0.941 |  1.199 |  0.274 | (180, 3, 3, 3) || conv_first.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || conv_first.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || patch_embed.norm.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || patch_embed.norm.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.0.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.0.norm1.bias | -0.000 | -0.040 |  0.040 |  0.017 | (225, 6) || layers.0.residual_group.blocks.0.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.0.residual_group.blocks.0.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.0.residual_group.blocks.0.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.0.residual_group.blocks.0.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.0.residual_group.blocks.0.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.0.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.0.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.0.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.0.residual_group.blocks.0.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.0.residual_group.blocks.0.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.0.residual_group.blocks.0.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.0.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.0.residual_group.blocks.1.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.1.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.1.norm1.bias | -0.001 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.0.residual_group.blocks.1.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.0.residual_group.blocks.1.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.0.residual_group.blocks.1.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.0.residual_group.blocks.1.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.0.residual_group.blocks.1.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.1.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.1.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.1.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.0.residual_group.blocks.1.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.0.residual_group.blocks.1.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.0.residual_group.blocks.1.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.1.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.2.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.2.norm1.bias |  0.001 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.0.residual_group.blocks.2.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.0.residual_group.blocks.2.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.0.residual_group.blocks.2.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.0.residual_group.blocks.2.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.0.residual_group.blocks.2.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.2.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.2.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.2.norm2.bias |  0.001 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.0.residual_group.blocks.2.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.0.residual_group.blocks.2.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.0.residual_group.blocks.2.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.2.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.0.residual_group.blocks.3.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.3.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.3.norm1.bias | -0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.0.residual_group.blocks.3.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.0.residual_group.blocks.3.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.0.residual_group.blocks.3.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.0.residual_group.blocks.3.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.0.residual_group.blocks.3.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.3.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.3.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.3.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.0.residual_group.blocks.3.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.0.residual_group.blocks.3.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.0.residual_group.blocks.3.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.3.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.4.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.4.norm1.bias |  0.000 | -0.039 |  0.040 |  0.018 | (225, 6) || layers.0.residual_group.blocks.4.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.0.residual_group.blocks.4.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.0.residual_group.blocks.4.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.0.residual_group.blocks.4.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.0.residual_group.blocks.4.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.4.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.4.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.4.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.0.residual_group.blocks.4.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.0.residual_group.blocks.4.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.0.residual_group.blocks.4.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.4.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.0.residual_group.blocks.5.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.5.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.5.norm1.bias | -0.000 | -0.040 |  0.039 |  0.018 | (225, 6) || layers.0.residual_group.blocks.5.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.0.residual_group.blocks.5.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.0.residual_group.blocks.5.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.0.residual_group.blocks.5.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.0.residual_group.blocks.5.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.5.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.0.residual_group.blocks.5.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.5.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.0.residual_group.blocks.5.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.0.residual_group.blocks.5.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.0.residual_group.blocks.5.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.0.residual_group.blocks.5.mlp.fc2.bias | -0.000 | -0.155 |  0.158 |  0.035 | (180, 180, 3, 3) || layers.0.conv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.0.conv.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.0.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.0.norm1.bias | -0.000 | -0.040 |  0.039 |  0.018 | (225, 6) || layers.1.residual_group.blocks.0.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.1.residual_group.blocks.0.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.1.residual_group.blocks.0.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.1.residual_group.blocks.0.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.1.residual_group.blocks.0.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.0.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.0.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.0.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.1.residual_group.blocks.0.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.1.residual_group.blocks.0.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.1.residual_group.blocks.0.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.0.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.1.residual_group.blocks.1.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.1.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.1.norm1.bias |  0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.1.residual_group.blocks.1.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.1.residual_group.blocks.1.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.1.residual_group.blocks.1.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.1.residual_group.blocks.1.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.1.residual_group.blocks.1.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.1.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.1.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.1.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.1.residual_group.blocks.1.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.1.residual_group.blocks.1.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.1.residual_group.blocks.1.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.1.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.2.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.2.norm1.bias | -0.001 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.1.residual_group.blocks.2.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.1.residual_group.blocks.2.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.1.residual_group.blocks.2.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.1.residual_group.blocks.2.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.1.residual_group.blocks.2.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.2.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.2.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.2.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.1.residual_group.blocks.2.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.1.residual_group.blocks.2.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.1.residual_group.blocks.2.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.2.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.1.residual_group.blocks.3.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.3.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.3.norm1.bias |  0.001 | -0.040 |  0.040 |  0.017 | (225, 6) || layers.1.residual_group.blocks.3.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.1.residual_group.blocks.3.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.1.residual_group.blocks.3.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.1.residual_group.blocks.3.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.1.residual_group.blocks.3.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.3.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.3.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.3.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.1.residual_group.blocks.3.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.1.residual_group.blocks.3.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.1.residual_group.blocks.3.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.3.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.4.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.4.norm1.bias |  0.000 | -0.040 |  0.040 |  0.017 | (225, 6) || layers.1.residual_group.blocks.4.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.1.residual_group.blocks.4.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.1.residual_group.blocks.4.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.1.residual_group.blocks.4.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.1.residual_group.blocks.4.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.4.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.4.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.4.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.1.residual_group.blocks.4.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.1.residual_group.blocks.4.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.1.residual_group.blocks.4.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.4.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.1.residual_group.blocks.5.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.5.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.5.norm1.bias |  0.000 | -0.039 |  0.040 |  0.018 | (225, 6) || layers.1.residual_group.blocks.5.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.1.residual_group.blocks.5.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.1.residual_group.blocks.5.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.1.residual_group.blocks.5.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.1.residual_group.blocks.5.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.5.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.1.residual_group.blocks.5.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.5.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.1.residual_group.blocks.5.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.1.residual_group.blocks.5.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.1.residual_group.blocks.5.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.1.residual_group.blocks.5.mlp.fc2.bias |  0.000 | -0.155 |  0.147 |  0.035 | (180, 180, 3, 3) || layers.1.conv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.1.conv.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.0.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.0.norm1.bias | -0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.2.residual_group.blocks.0.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.2.residual_group.blocks.0.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.2.residual_group.blocks.0.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.2.residual_group.blocks.0.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.2.residual_group.blocks.0.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.0.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.0.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.0.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.2.residual_group.blocks.0.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.2.residual_group.blocks.0.mlp.fc1.bias |  0.001 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.2.residual_group.blocks.0.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.0.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.2.residual_group.blocks.1.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.1.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.1.norm1.bias |  0.000 | -0.039 |  0.039 |  0.017 | (225, 6) || layers.2.residual_group.blocks.1.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.2.residual_group.blocks.1.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.2.residual_group.blocks.1.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.2.residual_group.blocks.1.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.2.residual_group.blocks.1.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.1.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.1.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.1.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.2.residual_group.blocks.1.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.2.residual_group.blocks.1.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.2.residual_group.blocks.1.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.1.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.2.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.2.norm1.bias | -0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.2.residual_group.blocks.2.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.2.residual_group.blocks.2.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.2.residual_group.blocks.2.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.2.residual_group.blocks.2.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.2.residual_group.blocks.2.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.2.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.2.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.2.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.2.residual_group.blocks.2.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.2.residual_group.blocks.2.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.2.residual_group.blocks.2.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.2.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.2.residual_group.blocks.3.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.3.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.3.norm1.bias | -0.000 | -0.039 |  0.039 |  0.018 | (225, 6) || layers.2.residual_group.blocks.3.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.2.residual_group.blocks.3.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.2.residual_group.blocks.3.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.2.residual_group.blocks.3.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.2.residual_group.blocks.3.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.3.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.3.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.3.norm2.bias | -0.001 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.2.residual_group.blocks.3.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.2.residual_group.blocks.3.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.2.residual_group.blocks.3.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.3.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.4.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.4.norm1.bias | -0.000 | -0.040 |  0.040 |  0.017 | (225, 6) || layers.2.residual_group.blocks.4.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.2.residual_group.blocks.4.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.2.residual_group.blocks.4.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.2.residual_group.blocks.4.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.2.residual_group.blocks.4.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.4.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.4.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.4.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.2.residual_group.blocks.4.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.2.residual_group.blocks.4.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.2.residual_group.blocks.4.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.4.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.2.residual_group.blocks.5.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.5.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.5.norm1.bias |  0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.2.residual_group.blocks.5.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.2.residual_group.blocks.5.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.2.residual_group.blocks.5.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.2.residual_group.blocks.5.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.2.residual_group.blocks.5.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.5.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.2.residual_group.blocks.5.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.5.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.2.residual_group.blocks.5.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.2.residual_group.blocks.5.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.2.residual_group.blocks.5.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.2.residual_group.blocks.5.mlp.fc2.bias |  0.000 | -0.159 |  0.157 |  0.035 | (180, 180, 3, 3) || layers.2.conv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.2.conv.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.0.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.0.norm1.bias | -0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.3.residual_group.blocks.0.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.3.residual_group.blocks.0.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.3.residual_group.blocks.0.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.3.residual_group.blocks.0.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.3.residual_group.blocks.0.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.0.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.0.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.0.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.3.residual_group.blocks.0.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.3.residual_group.blocks.0.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.3.residual_group.blocks.0.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.0.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.3.residual_group.blocks.1.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.1.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.1.norm1.bias | -0.001 | -0.039 |  0.040 |  0.018 | (225, 6) || layers.3.residual_group.blocks.1.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.3.residual_group.blocks.1.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.3.residual_group.blocks.1.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.3.residual_group.blocks.1.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.3.residual_group.blocks.1.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.1.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.1.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.1.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.3.residual_group.blocks.1.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.3.residual_group.blocks.1.mlp.fc1.bias |  0.001 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.3.residual_group.blocks.1.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.1.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.2.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.2.norm1.bias | -0.001 | -0.040 |  0.040 |  0.017 | (225, 6) || layers.3.residual_group.blocks.2.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.3.residual_group.blocks.2.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.3.residual_group.blocks.2.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.3.residual_group.blocks.2.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.3.residual_group.blocks.2.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.2.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.2.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.2.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.3.residual_group.blocks.2.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.3.residual_group.blocks.2.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.3.residual_group.blocks.2.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.2.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.3.residual_group.blocks.3.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.3.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.3.norm1.bias |  0.000 | -0.040 |  0.039 |  0.018 | (225, 6) || layers.3.residual_group.blocks.3.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.3.residual_group.blocks.3.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.3.residual_group.blocks.3.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.3.residual_group.blocks.3.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.3.residual_group.blocks.3.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.3.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.3.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.3.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.3.residual_group.blocks.3.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.3.residual_group.blocks.3.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.3.residual_group.blocks.3.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.3.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.4.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.4.norm1.bias |  0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.3.residual_group.blocks.4.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.3.residual_group.blocks.4.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.3.residual_group.blocks.4.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.3.residual_group.blocks.4.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.3.residual_group.blocks.4.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.4.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.4.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.4.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.3.residual_group.blocks.4.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.3.residual_group.blocks.4.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.3.residual_group.blocks.4.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.4.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.3.residual_group.blocks.5.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.5.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.5.norm1.bias |  0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.3.residual_group.blocks.5.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.3.residual_group.blocks.5.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.3.residual_group.blocks.5.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.3.residual_group.blocks.5.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.3.residual_group.blocks.5.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.5.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.3.residual_group.blocks.5.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.5.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.3.residual_group.blocks.5.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.3.residual_group.blocks.5.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.3.residual_group.blocks.5.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.3.residual_group.blocks.5.mlp.fc2.bias | -0.000 | -0.166 |  0.169 |  0.035 | (180, 180, 3, 3) || layers.3.conv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.3.conv.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.0.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.0.norm1.bias |  0.001 | -0.040 |  0.040 |  0.017 | (225, 6) || layers.4.residual_group.blocks.0.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.4.residual_group.blocks.0.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.4.residual_group.blocks.0.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.4.residual_group.blocks.0.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.4.residual_group.blocks.0.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.0.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.0.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.0.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.4.residual_group.blocks.0.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.4.residual_group.blocks.0.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.4.residual_group.blocks.0.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.0.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.4.residual_group.blocks.1.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.1.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.1.norm1.bias | -0.001 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.4.residual_group.blocks.1.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.4.residual_group.blocks.1.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.4.residual_group.blocks.1.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.4.residual_group.blocks.1.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.4.residual_group.blocks.1.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.1.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.1.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.1.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.4.residual_group.blocks.1.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.4.residual_group.blocks.1.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.4.residual_group.blocks.1.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.1.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.2.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.2.norm1.bias | -0.000 | -0.040 |  0.039 |  0.018 | (225, 6) || layers.4.residual_group.blocks.2.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.4.residual_group.blocks.2.attn.relative_position_index |  0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.4.residual_group.blocks.2.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.4.residual_group.blocks.2.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.4.residual_group.blocks.2.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.2.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.2.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.2.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.4.residual_group.blocks.2.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.4.residual_group.blocks.2.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.4.residual_group.blocks.2.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.2.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.4.residual_group.blocks.3.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.3.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.3.norm1.bias | -0.001 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.4.residual_group.blocks.3.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.4.residual_group.blocks.3.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.4.residual_group.blocks.3.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.4.residual_group.blocks.3.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.4.residual_group.blocks.3.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.3.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.3.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.3.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.4.residual_group.blocks.3.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.4.residual_group.blocks.3.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.4.residual_group.blocks.3.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.3.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.4.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.4.norm1.bias |  0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.4.residual_group.blocks.4.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.4.residual_group.blocks.4.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.4.residual_group.blocks.4.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.4.residual_group.blocks.4.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.4.residual_group.blocks.4.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.4.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.4.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.4.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.4.residual_group.blocks.4.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.4.residual_group.blocks.4.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.4.residual_group.blocks.4.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.4.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.4.residual_group.blocks.5.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.5.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.5.norm1.bias |  0.000 | -0.040 |  0.039 |  0.017 | (225, 6) || layers.4.residual_group.blocks.5.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.4.residual_group.blocks.5.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.4.residual_group.blocks.5.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.4.residual_group.blocks.5.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.4.residual_group.blocks.5.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.5.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.4.residual_group.blocks.5.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.5.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.4.residual_group.blocks.5.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.4.residual_group.blocks.5.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.4.residual_group.blocks.5.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.4.residual_group.blocks.5.mlp.fc2.bias | -0.000 | -0.148 |  0.169 |  0.035 | (180, 180, 3, 3) || layers.4.conv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.4.conv.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.0.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.0.norm1.bias |  0.001 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.5.residual_group.blocks.0.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.5.residual_group.blocks.0.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.5.residual_group.blocks.0.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.5.residual_group.blocks.0.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.5.residual_group.blocks.0.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.0.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.0.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.0.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.5.residual_group.blocks.0.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.5.residual_group.blocks.0.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.5.residual_group.blocks.0.mlp.fc2.weight | -0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.0.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.5.residual_group.blocks.1.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.1.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.1.norm1.bias |  0.001 | -0.039 |  0.040 |  0.018 | (225, 6) || layers.5.residual_group.blocks.1.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.5.residual_group.blocks.1.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.5.residual_group.blocks.1.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.5.residual_group.blocks.1.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.5.residual_group.blocks.1.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.1.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.1.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.1.norm2.bias | -0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.5.residual_group.blocks.1.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.5.residual_group.blocks.1.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.5.residual_group.blocks.1.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.1.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.2.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.2.norm1.bias |  0.000 | -0.039 |  0.040 |  0.018 | (225, 6) || layers.5.residual_group.blocks.2.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.5.residual_group.blocks.2.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.5.residual_group.blocks.2.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.5.residual_group.blocks.2.attn.qkv.bias |  0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.5.residual_group.blocks.2.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.2.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.2.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.2.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.5.residual_group.blocks.2.mlp.fc1.weight | -0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.5.residual_group.blocks.2.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.5.residual_group.blocks.2.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.2.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.5.residual_group.blocks.3.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.3.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.3.norm1.bias | -0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.5.residual_group.blocks.3.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.5.residual_group.blocks.3.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.5.residual_group.blocks.3.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.5.residual_group.blocks.3.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.5.residual_group.blocks.3.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.3.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.3.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.3.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.5.residual_group.blocks.3.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.5.residual_group.blocks.3.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.5.residual_group.blocks.3.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.3.mlp.fc2.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.4.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.4.norm1.bias |  0.000 | -0.040 |  0.040 |  0.017 | (225, 6) || layers.5.residual_group.blocks.4.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.5.residual_group.blocks.4.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.5.residual_group.blocks.4.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.5.residual_group.blocks.4.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.5.residual_group.blocks.4.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.4.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.4.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.4.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.5.residual_group.blocks.4.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.5.residual_group.blocks.4.mlp.fc1.bias | -0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.5.residual_group.blocks.4.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.4.mlp.fc2.bias | -6.152 | -100.000 | -0.000 | 24.029 | (256, 64, 64) || layers.5.residual_group.blocks.5.attn_mask |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.5.norm1.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.5.norm1.bias |  0.000 | -0.040 |  0.040 |  0.018 | (225, 6) || layers.5.residual_group.blocks.5.attn.relative_position_bias_table | 112.000 |  0.000 | 224.000 | 48.713 | (64, 64) || layers.5.residual_group.blocks.5.attn.relative_position_index | -0.000 | -0.040 |  0.040 |  0.018 | (180, 540) || layers.5.residual_group.blocks.5.attn.qkv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (540,) || layers.5.residual_group.blocks.5.attn.qkv.bias | -0.000 | -0.040 |  0.040 |  0.018 | (180, 180) || layers.5.residual_group.blocks.5.attn.proj.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.5.attn.proj.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || layers.5.residual_group.blocks.5.norm2.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.5.norm2.bias |  0.000 | -0.105 |  0.105 |  0.061 | (180, 360) || layers.5.residual_group.blocks.5.mlp.fc1.weight |  0.000 | -0.000 |  0.000 |  0.000 | (360,) || layers.5.residual_group.blocks.5.mlp.fc1.bias |  0.000 | -0.105 |  0.105 |  0.061 | (360, 180) || layers.5.residual_group.blocks.5.mlp.fc2.weight |  0.000 | -0.000 |  0.000 |  0.000 | (180,) || layers.5.residual_group.blocks.5.mlp.fc2.bias | -0.000 | -0.152 |  0.160 |  0.035 | (180, 180, 3, 3) || layers.5.conv.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || layers.5.conv.bias |  1.000 |  1.000 |  1.000 |  0.000 | (180,) || norm.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || norm.bias | -0.000 | -0.166 |  0.159 |  0.035 | (180, 180, 3, 3) || conv_after_body.weight |  0.000 |  0.000 |  0.000 |  0.000 | (180,) || conv_after_body.bias |  0.000 | -0.120 |  0.133 |  0.035 | (3, 180, 3, 3) || conv_last.weight |  0.000 |  0.000 |  0.000 |  0.000 | (3,) || conv_last.bias

       In [ ]

# 单机四卡# !cd work && python -m paddle.distributed.launch main_train_psnr.py --opt options/train_swinir_multi_card_32.json

训练过程会将模型参数保存在 work/denoising/swinir_denoising_color_15/models/ 文件夹下.

训练日志将会保存在 work/denoising/swinir_denoising_color_15/models/train.log

本人单机四卡的训练日志为 work/train.log

5.2 模型评估与预测

在 CBSD68 测试数据上作测试，加强度为15的噪声，结果将存放在 work/results/swinir_color_dn_noise15/ 文件夹下

In [14]

!cd work && python main_test_swinir.py --task color_dn --noise 15 --model_path pretrained_models/SwinIR_paddle.pdparams --folder_gt testsets/CBSD68/

loading model from pretrained_models/SwinIR_paddle.pdparamsW1016 16:39:45.440834 13414 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2W1016 16:39:45.444900 13414 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.Testing 0 101085               - PSNR: 31.32 dB; SSIM: 0.9118; PSNR_Y: 32.88 dB; SSIM_Y: 0.9210; PSNR_B: 0.00 dB.Testing 1 101087               - PSNR: 35.39 dB; SSIM: 0.9517; PSNR_Y: 37.14 dB; SSIM_Y: 0.9603; PSNR_B: 0.00 dB.Testing 2 102061               - PSNR: 35.21 dB; SSIM: 0.9348; PSNR_Y: 36.97 dB; SSIM_Y: 0.9477; PSNR_B: 0.00 dB.Testing 3 103070               - PSNR: 35.86 dB; SSIM: 0.9448; PSNR_Y: 37.79 dB; SSIM_Y: 0.9562; PSNR_B: 0.00 dB.Testing 4 105025               - PSNR: 33.11 dB; SSIM: 0.9426; PSNR_Y: 34.87 dB; SSIM_Y: 0.9509; PSNR_B: 0.00 dB.Testing 5 106024               - PSNR: 37.57 dB; SSIM: 0.9560; PSNR_Y: 39.35 dB; SSIM_Y: 0.9652; PSNR_B: 0.00 dB.Testing 6 108005               - PSNR: 33.86 dB; SSIM: 0.9337; PSNR_Y: 35.73 dB; SSIM_Y: 0.9470; PSNR_B: 0.00 dB.Testing 7 108070               - PSNR: 31.96 dB; SSIM: 0.9222; PSNR_Y: 33.71 dB; SSIM_Y: 0.9353; PSNR_B: 0.00 dB.Testing 8 108082               - PSNR: 34.37 dB; SSIM: 0.9420; PSNR_Y: 36.06 dB; SSIM_Y: 0.9520; PSNR_B: 0.00 dB.Testing 9 109053               - PSNR: 34.67 dB; SSIM: 0.9309; PSNR_Y: 36.52 dB; SSIM_Y: 0.9428; PSNR_B: 0.00 dB.Testing 10 119082               - PSNR: 34.75 dB; SSIM: 0.9570; PSNR_Y: 36.56 dB; SSIM_Y: 0.9655; PSNR_B: 0.00 dB.Testing 11 12084                - PSNR: 33.08 dB; SSIM: 0.9168; PSNR_Y: 35.40 dB; SSIM_Y: 0.9340; PSNR_B: 0.00 dB.Testing 12 123074               - PSNR: 34.70 dB; SSIM: 0.9363; PSNR_Y: 36.45 dB; SSIM_Y: 0.9465; PSNR_B: 0.00 dB.Testing 13 126007               - PSNR: 35.77 dB; SSIM: 0.9334; PSNR_Y: 37.60 dB; SSIM_Y: 0.9484; PSNR_B: 0.00 dB.Testing 14 130026               - PSNR: 32.89 dB; SSIM: 0.9189; PSNR_Y: 34.51 dB; SSIM_Y: 0.9319; PSNR_B: 0.00 dB.Testing 15 134035               - PSNR: 33.90 dB; SSIM: 0.9480; PSNR_Y: 35.66 dB; SSIM_Y: 0.9559; PSNR_B: 0.00 dB.Testing 16 14037                - PSNR: 37.21 dB; SSIM: 0.9387; PSNR_Y: 38.95 dB; SSIM_Y: 0.9533; PSNR_B: 0.00 dB.Testing 17 143090               - PSNR: 37.57 dB; SSIM: 0.9513; PSNR_Y: 39.62 dB; SSIM_Y: 0.9639; PSNR_B: 0.00 dB.Testing 18 145086               - PSNR: 33.79 dB; SSIM: 0.9310; PSNR_Y: 35.46 dB; SSIM_Y: 0.9461; PSNR_B: 0.00 dB.Testing 19 147091               - PSNR: 34.33 dB; SSIM: 0.9289; PSNR_Y: 36.20 dB; SSIM_Y: 0.9468; PSNR_B: 0.00 dB.Testing 20 148026               - PSNR: 32.43 dB; SSIM: 0.9511; PSNR_Y: 34.22 dB; SSIM_Y: 0.9619; PSNR_B: 0.00 dB.Testing 21 148089               - PSNR: 32.71 dB; SSIM: 0.9363; PSNR_Y: 34.36 dB; SSIM_Y: 0.9462; PSNR_B: 0.00 dB.Testing 22 157055               - PSNR: 34.56 dB; SSIM: 0.9495; PSNR_Y: 36.59 dB; SSIM_Y: 0.9616; PSNR_B: 0.00 dB.Testing 23 159008               - PSNR: 34.26 dB; SSIM: 0.9460; PSNR_Y: 35.95 dB; SSIM_Y: 0.9578; PSNR_B: 0.00 dB.Testing 24 160068               - PSNR: 35.17 dB; SSIM: 0.9682; PSNR_Y: 36.88 dB; SSIM_Y: 0.9758; PSNR_B: 0.00 dB.Testing 25 16077                - PSNR: 33.49 dB; SSIM: 0.9201; PSNR_Y: 35.28 dB; SSIM_Y: 0.9347; PSNR_B: 0.00 dB.Testing 26 163085               - PSNR: 34.90 dB; SSIM: 0.9267; PSNR_Y: 36.75 dB; SSIM_Y: 0.9421; PSNR_B: 0.00 dB.Testing 27 167062               - PSNR: 37.95 dB; SSIM: 0.9449; PSNR_Y: 39.81 dB; SSIM_Y: 0.9601; PSNR_B: 0.00 dB.Testing 28 167083               - PSNR: 31.24 dB; SSIM: 0.9596; PSNR_Y: 32.78 dB; SSIM_Y: 0.9645; PSNR_B: 0.00 dB.Testing 29 170057               - PSNR: 34.72 dB; SSIM: 0.9203; PSNR_Y: 36.58 dB; SSIM_Y: 0.9371; PSNR_B: 0.00 dB.Testing 30 175032               - PSNR: 30.79 dB; SSIM: 0.9515; PSNR_Y: 32.49 dB; SSIM_Y: 0.9566; PSNR_B: 0.00 dB.Testing 31 175043               - PSNR: 32.28 dB; SSIM: 0.9442; PSNR_Y: 34.06 dB; SSIM_Y: 0.9519; PSNR_B: 0.00 dB.Testing 32 182053               - PSNR: 33.90 dB; SSIM: 0.9614; PSNR_Y: 35.50 dB; SSIM_Y: 0.9692; PSNR_B: 0.00 dB.Testing 33 189080               - PSNR: 37.19 dB; SSIM: 0.9205; PSNR_Y: 38.89 dB; SSIM_Y: 0.9385; PSNR_B: 0.00 dB.Testing 34 19021                - PSNR: 33.28 dB; SSIM: 0.9276; PSNR_Y: 34.99 dB; SSIM_Y: 0.9394; PSNR_B: 0.00 dB.Testing 35 196073               - PSNR: 31.92 dB; SSIM: 0.8469; PSNR_Y: 33.31 dB; SSIM_Y: 0.8625; PSNR_B: 0.00 dB.Testing 36 197017               - PSNR: 33.46 dB; SSIM: 0.9228; PSNR_Y: 35.10 dB; SSIM_Y: 0.9352; PSNR_B: 0.00 dB.Testing 37 208001               - PSNR: 34.05 dB; SSIM: 0.9204; PSNR_Y: 36.01 dB; SSIM_Y: 0.9396; PSNR_B: 0.00 dB.Testing 38 210088               - PSNR: 38.00 dB; SSIM: 0.9620; PSNR_Y: 40.78 dB; SSIM_Y: 0.9756; PSNR_B: 0.00 dB.Testing 39 21077                - PSNR: 34.18 dB; SSIM: 0.9032; PSNR_Y: 35.83 dB; SSIM_Y: 0.9194; PSNR_B: 0.00 dB.Testing 40 216081               - PSNR: 33.35 dB; SSIM: 0.9367; PSNR_Y: 35.04 dB; SSIM_Y: 0.9494; PSNR_B: 0.00 dB.Testing 41 219090               - PSNR: 34.88 dB; SSIM: 0.9243; PSNR_Y: 36.50 dB; SSIM_Y: 0.9377; PSNR_B: 0.00 dB.Testing 42 220075               - PSNR: 35.54 dB; SSIM: 0.9521; PSNR_Y: 37.61 dB; SSIM_Y: 0.9650; PSNR_B: 0.00 dB.Testing 43 223061               - PSNR: 33.88 dB; SSIM: 0.9557; PSNR_Y: 35.49 dB; SSIM_Y: 0.9627; PSNR_B: 0.00 dB.Testing 44 227092               - PSNR: 37.97 dB; SSIM: 0.9423; PSNR_Y: 39.80 dB; SSIM_Y: 0.9547; PSNR_B: 0.00 dB.Testing 45 229036               - PSNR: 32.15 dB; SSIM: 0.9262; PSNR_Y: 33.76 dB; SSIM_Y: 0.9353; PSNR_B: 0.00 dB.Testing 46 236037               - PSNR: 32.16 dB; SSIM: 0.9350; PSNR_Y: 33.94 dB; SSIM_Y: 0.9431; PSNR_B: 0.00 dB.Testing 47 24077                - PSNR: 35.38 dB; SSIM: 0.9678; PSNR_Y: 37.68 dB; SSIM_Y: 0.9761; PSNR_B: 0.00 dB.Testing 48 241004               - PSNR: 35.61 dB; SSIM: 0.9070; PSNR_Y: 37.23 dB; SSIM_Y: 0.9253; PSNR_B: 0.00 dB.Testing 49 241048               - PSNR: 32.33 dB; SSIM: 0.9273; PSNR_Y: 33.99 dB; SSIM_Y: 0.9364; PSNR_B: 0.00 dB.Testing 50 253027               - PSNR: 33.92 dB; SSIM: 0.9310; PSNR_Y: 35.58 dB; SSIM_Y: 0.9421; PSNR_B: 0.00 dB.Testing 51 253055               - PSNR: 36.10 dB; SSIM: 0.9168; PSNR_Y: 37.78 dB; SSIM_Y: 0.9341; PSNR_B: 0.00 dB.Testing 52 260058               - PSNR: 35.37 dB; SSIM: 0.8999; PSNR_Y: 36.95 dB; SSIM_Y: 0.9184; PSNR_B: 0.00 dB.Testing 53 271035               - PSNR: 34.40 dB; SSIM: 0.9299; PSNR_Y: 36.10 dB; SSIM_Y: 0.9428; PSNR_B: 0.00 dB.Testing 54 285079               - PSNR: 33.00 dB; SSIM: 0.9271; PSNR_Y: 34.77 dB; SSIM_Y: 0.9383; PSNR_B: 0.00 dB.Testing 55 291000               - PSNR: 30.12 dB; SSIM: 0.9457; PSNR_Y: 31.99 dB; SSIM_Y: 0.9523; PSNR_B: 0.00 dB.Testing 56 295087               - PSNR: 33.49 dB; SSIM: 0.9405; PSNR_Y: 35.32 dB; SSIM_Y: 0.9515; PSNR_B: 0.00 dB.Testing 57 296007               - PSNR: 34.75 dB; SSIM: 0.8944; PSNR_Y: 36.36 dB; SSIM_Y: 0.9144; PSNR_B: 0.00 dB.Testing 58 296059               - PSNR: 34.66 dB; SSIM: 0.9062; PSNR_Y: 36.29 dB; SSIM_Y: 0.9223; PSNR_B: 0.00 dB.Testing 59 299086               - PSNR: 35.32 dB; SSIM: 0.9082; PSNR_Y: 37.03 dB; SSIM_Y: 0.9262; PSNR_B: 0.00 dB.Testing 60 300091               - PSNR: 35.18 dB; SSIM: 0.9137; PSNR_Y: 36.80 dB; SSIM_Y: 0.9295; PSNR_B: 0.00 dB.Testing 61 302008               - PSNR: 38.71 dB; SSIM: 0.9649; PSNR_Y: 40.80 dB; SSIM_Y: 0.9749; PSNR_B: 0.00 dB.Testing 62 304034               - PSNR: 32.27 dB; SSIM: 0.9455; PSNR_Y: 34.11 dB; SSIM_Y: 0.9549; PSNR_B: 0.00 dB.Testing 63 304074               - PSNR: 31.85 dB; SSIM: 0.9043; PSNR_Y: 33.38 dB; SSIM_Y: 0.9154; PSNR_B: 0.00 dB.Testing 64 306005               - PSNR: 33.75 dB; SSIM: 0.9265; PSNR_Y: 35.72 dB; SSIM_Y: 0.9444; PSNR_B: 0.00 dB.Testing 65 3096                 - PSNR: 42.69 dB; SSIM: 0.9768; PSNR_Y: 44.83 dB; SSIM_Y: 0.9854; PSNR_B: 0.00 dB.Testing 66 33039                - PSNR: 30.98 dB; SSIM: 0.9585; PSNR_Y: 32.50 dB; SSIM_Y: 0.9619; PSNR_B: 0.00 dB.Testing 67 351093               - PSNR: 32.45 dB; SSIM: 0.9607; PSNR_Y: 34.08 dB; SSIM_Y: 0.9672; PSNR_B: 0.00 dB.results/swinir_color_dn_noise15 -- Average PSNR/SSIM(RGB): 34.32 dB; 0.9344-- Average PSNR_Y/SSIM_Y: 36.10 dB; 0.9465

输出如下：

— Average PSNR/SSIM(RGB): 34.32 dB; 0.9344

接近了验收精度.

5.3 单张图像去噪测试

导入单张图像，测试去噪效果，首先需要在work/test_images里上传一张图片.

In [13]

# 先上传一张图片import os.path as ospfrom IPython.display import displayfrom PIL import Imageimg_path = 'bird.png' # 改成自己上传的图片名称full_img_path = osp.join(osp.abspath('work/test_images/'), img_path)img = Image.open(full_img_path).convert('RGB')print('以下为上传的图片:')display(img)

以下为上传的图片:

需要指定干净图像和噪声图像，可以只给一张噪声图片，也可以只给一张干净图片，也可以都给.

给定一张噪声图片：指定参数noisy_img，直接输出去噪图片.

给定一张干净图片：指定参数clean_img和noisyL，后者为噪声水平，默认为15，输出加噪图片和去噪图片.

给定噪声图片和干净图片：直接输出去噪图片.

In [24]

# 仅给定干净图片，噪声水平为15!cd work && python predict_single.py --clean_img $full_img_path --save_images --noisyL 15 --model_path pretrained_models/SwinIR_paddle.pdparams

loading model from pretrained_models/SwinIR_paddle.pdparamsW1016 17:20:03.374689 17743 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2W1016 17:20:03.378355 17743 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2.only clean image provided, noise level is 15PSNR on test data 34.9994

       In [25]

# 去噪效果查看import globfrom IPython.display import displayfrom PIL import Imageimgs = glob.glob('work/test_images/*')for path in imgs:    print(path)    img = Image.open(path)    display(img)

work/test_images/bird_denoised.png

work/test_images/bird_noised.png

work/test_images/bird.png

以上就是【图像去噪】第七期论文复现赛——SwinIR的详细内容，更多请关注创想鸟其它相关文章！