FITS file doubles in size after preprocessing

[Rudi]

I just got a ZWO ASI 1600 MM Cool this weekend, and I tested it last night.
As expected, the 4656x3520 pixel images takes up approximately 32 MB disk space, the camera is 12 bpp => 2 bytes.
However, after stacking, the resulting fits file is 64 MB !

The fits header for a single light, says "BITPIX = 16", where as the stacked fits file says: "BITPIX = 32".

I have upgraded from a Canon DSLR, so I have made sure, that in [Preferences]->[RAW images], "DCRAW demosaic on load" is disabled and also that Custom RAW is Disabled.
Could I be missing something else out?

A single luminance:
https://www.dropbox.com/s/scdclciu0uncb ... 1.fit?dl=0

Stacked luminance:
https://www.dropbox.com/s/f7ocjyp1yz4rd ... L.fit?dl=0

[Dunmunro]

I just got a ZWO ASI 1600 MM Cool this weekend, and I tested it last night.
As expected, the 4656x3520 pixel images takes up approximately 32 MB disk space, the camera is 12 bpp => 2 bytes.
However, after stacking, the resulting fits file is 64 MB !

The fits header for a single light, says "BITPIX = 16", where as the stacked fits file says: "BITPIX = 32".

I have upgraded from a Canon DSLR, so I have made sure, that in [Preferences]->[RAW images], "DCRAW demosaic on load" is disabled and also that Custom RAW is Disabled.
Could I be missing something else out?

A single luminance:
https://www.dropbox.com/s/scdclciu0uncb ... 1.fit?dl=0

Stacked luminance:
https://www.dropbox.com/s/f7ocjyp1yz4rd ... L.fit?dl=0

Which stacking method were you using?

[Rudi]

Hi Dunmunro,

I am using "Sigma+Sum".

[Dunmunro]

Hi Dunmunro,

I am using "Sigma+Sum".

It seems that AA7 converts 16 bit fits files to 32 bit whenever it makes changes to the image, such as demosaicing and/or stacking. I haven't really considered this but it is probably necessary to preserve the data in the resulting files. It happens with my 16bit images as well.

Hopefully Fabio will comment further.

[fabdev]

Hello, the stacked image has a range of one million ADU (because it's a Sum) so it's correctly saved as 32 bit.
16 bit FITS images are limited to approx. 0..65000 ADU.
The ADU range of an image is always shown on the status bar of the image (R = ... )

You may keep the range to 16 bit using "Average" or "Sigma with average" (normal Sigma).
"Sum" is useful for 8-12 bit images where quantization is an issue.

[Rudi]

Thanks for the answer Fabio.
What stacking method would You recommend for a ZWO ASI 1600MM Pro at unity gain or below?
It’s a 12 bit camera.

[fabdev]

I think it depends on the number of images:
up to 20-30 images there will not be any quantization and I'd use "Average" to obtain a 16 bit result.
For 50+ images I would use "Sum".

[Rudi]

Hi Fabio,

I tried using stacking method "Average", but still, the result file is 64 MB and it has BITPIX = 32 !!!

If I follow the steps below, the change seems to happen when the picture is saved.

1. Preprocessing 16 bit fit files using "Average" method
   
2. AstroArt 7 grinding the data
   
3. Data has been processed, file still not saved, BITPIX is 16
   
4. After the file has been saved, and without re-opening it, BITPIX is now 32!!!
   And the saved file is twice the size as each of the input fits files.
   

If I am doing it right, it seems that something is going wrong in AA7 when saving the file?

[fabdev]

Hello, what is the Range of the image?

Only if the Range is less than 0..65535 an image can be saved as 16 bit.
https://en.wikipedia.org/wiki/16-bit

( BITPIX is updated only when the image is saved. )

[Rudi]

Statistics for one of the lights:

Code: Select all

 Name  ...........  #2 m100_L_60s_001.fit
 Path  ...........  C:\Users\Bruger\Rudi Dropbox\Main\astro\2019\0512\
 Size  ...........  4656 x 3520
 Telescope  ......  12 22.9  +15 48
 Pixels  .........  16389120
 Min  ............  3952 @ 80,3357
 Max  ............  65504 @ 4210,2681
 Background  .....  4976
 Average  ........  5071.38
 Sum  ............  83115456176
 Std.deviation  ..  432.87219

Statistics after stacking:

Code: Select all

 Name  ...........  #1 Preprocessing00.fit
 Path  ...........  C:\Users\Bruger\Rudi Dropbox\Main\astro\2019\0512\
 Size  ...........  4656 x 3520
 Telescope  ......  12 22.9  +15 48
 Pixels  .........  16389120
 Min  ............  437 @ 0,0
 Max  ............  68691 @ 36,488
 Background  .....  2988
 Average  ........  2991.8873
 Sum  ............  49034400157
 Std.deviation  ..  359.3689

So, somehow it exceeded 2^16 ?

Max from all lights:

Code: Select all

 #1:	Max  ............  65504 @ 4210,2681
 #2:	Max  ............  65504 @ 4210,2679
 #3:	Max  ............  65504 @ 4210,2678
 #4:	Max  ............  65504 @ 4209,2679
 #5:	Max  ............  65504 @ 4211,2679
 #6:	Max  ............  65504 @ 4210,2679
 #7:	Max  ............  65504 @ 4208,2679
 #8:	Max  ............  65504 @ 4208,2679
 #9:	Max  ............  65504 @ 4208,2677
#10:	Max  ............  65504 @ 4208,2678
#2-1:	Max  ............  65504 @ 1461,2481
#2-2:	Max  ............  65504 @ 2205,164
#2-3:	Max  ............  65504 @ 2202,164
#2-4:	Max  ............  65504 @ 2204,162
#2-5:	Max  ............  65504 @ 2204,162
#2-6:	Max  ............  65504 @ 2203,164

[fabdev]

OK, the range is 437..68691 (it can be read on the status bar of the image too), so it exceeds the 16 bit range.

The cause is the flat field correction. This can happen if the bias of the camera is low and the vignetting is strong.

To force a 16 bit range before saving there are several ways, for example the command: Arithmetic / Clamp / Maximum.

For a scientifically correct result the flat field must also be corrected from its dark frame.

[Rudi]

Ok Fabio. I will use dark flats next time.
I was also recommended doing that for my new camera on a danish Astro site.