Page 1 of 1
FITS file doubles in size after preprocessing
Posted: 13 May 2019, 07:55
by Rudi
I just got a ZWO ASI 1600 MM Cool this weekend, and I tested it last night.
As expected, the 4656x3520 pixel images takes up approximately 32 MB disk space, the camera is 12 bpp => 2 bytes.
However, after stacking, the resulting fits file is 64 MB !
The fits header for a single light, says "BITPIX = 16", where as the stacked fits file says: "BITPIX = 32".
I have upgraded from a Canon DSLR, so I have made sure, that in [Preferences]->[RAW images], "DCRAW demosaic on load" is disabled and also that Custom RAW is Disabled.
Could I be missing something else out?
A single luminance:
https://www.dropbox.com/s/scdclciu0uncb ... 1.fit?dl=0
Stacked luminance:
https://www.dropbox.com/s/f7ocjyp1yz4rd ... L.fit?dl=0
Re: FITS file doubles in size after preprocessing
Posted: 14 May 2019, 06:49
by Dunmunro
Rudi wrote: ↑13 May 2019, 07:55
I just got a ZWO ASI 1600 MM Cool this weekend, and I tested it last night.
As expected, the 4656x3520 pixel images takes up approximately 32 MB disk space, the camera is 12 bpp => 2 bytes.
However, after stacking, the resulting fits file is 64 MB !
The fits header for a single light, says "BITPIX = 16", where as the stacked fits file says: "BITPIX = 32".
I have upgraded from a Canon DSLR, so I have made sure, that in [Preferences]->[RAW images], "DCRAW demosaic on load" is disabled and also that Custom RAW is Disabled.
Could I be missing something else out?
A single luminance:
https://www.dropbox.com/s/scdclciu0uncb ... 1.fit?dl=0
Stacked luminance:
https://www.dropbox.com/s/f7ocjyp1yz4rd ... L.fit?dl=0
Which stacking method were you using?
Re: FITS file doubles in size after preprocessing
Posted: 14 May 2019, 10:43
by Rudi
Hi Dunmunro,
I am using "Sigma+Sum".
Re: FITS file doubles in size after preprocessing
Posted: 15 May 2019, 00:58
by Dunmunro
Rudi wrote: ↑14 May 2019, 10:43
Hi Dunmunro,
I am using "Sigma+Sum".
It seems that AA7 converts 16 bit fits files to 32 bit whenever it makes changes to the image, such as demosaicing and/or stacking. I haven't really considered this but it is probably necessary to preserve the data in the resulting files. It happens with my 16bit images as well.
Hopefully Fabio will comment further.
Re: FITS file doubles in size after preprocessing
Posted: 15 May 2019, 10:17
by fabdev
Hello, the stacked image has a range of one million ADU (because it's a Sum) so it's correctly saved as 32 bit.
16 bit FITS images are limited to approx. 0..65000 ADU.
The ADU range of an image is always shown on the status bar of the image (R = ... )
You may keep the range to 16 bit using "Average" or "Sigma with average" (normal Sigma).
"Sum" is useful for 8-12 bit images where quantization is an issue.
Re: FITS file doubles in size after preprocessing
Posted: 15 May 2019, 15:37
by Rudi
Thanks for the answer Fabio.
What stacking method would You recommend for a ZWO ASI 1600MM Pro at unity gain or below?
It’s a 12 bit camera.
Re: FITS file doubles in size after preprocessing
Posted: 16 May 2019, 14:35
by fabdev
I think it depends on the number of images:
up to 20-30 images there will not be any quantization and I'd use "Average" to obtain a 16 bit result.
For 50+ images I would use "Sum".
Re: FITS file doubles in size after preprocessing
Posted: 17 May 2019, 06:59
by Rudi
Hi Fabio,
I tried using stacking method "Average", but still, the result file is 64 MB and it has BITPIX = 32 !!!
If I follow the steps below, the change seems to happen when the picture is saved.
- Preprocessing 16 bit fit files using "Average" method
- AstroArt 7 grinding the data
- Data has been processed, file still not saved, BITPIX is 16
- After the file has been saved, and without re-opening it, BITPIX is now 32!!!
And the saved file is twice the size as each of the input fits files.
If I am doing it right, it seems that something is going wrong in AA7 when saving the file?
Re: FITS file doubles in size after preprocessing
Posted: 17 May 2019, 09:25
by fabdev
Hello, what is the Range of the image?
Only if the Range is less than 0..65535 an image can be saved as 16 bit.
https://en.wikipedia.org/wiki/16-bit
( BITPIX is updated only when the image is saved. )
Re: FITS file doubles in size after preprocessing
Posted: 17 May 2019, 09:52
by Rudi
Statistics for one of the lights:
Code: Select all
Name ........... #2 m100_L_60s_001.fit
Path ........... C:\Users\Bruger\Rudi Dropbox\Main\astro\2019\0512\
Size ........... 4656 x 3520
Telescope ...... 12 22.9 +15 48
Pixels ......... 16389120
Min ............ 3952 @ 80,3357
Max ............ 65504 @ 4210,2681
Background ..... 4976
Average ........ 5071.38
Sum ............ 83115456176
Std.deviation .. 432.87219
Statistics after stacking:
Code: Select all
Name ........... #1 Preprocessing00.fit
Path ........... C:\Users\Bruger\Rudi Dropbox\Main\astro\2019\0512\
Size ........... 4656 x 3520
Telescope ...... 12 22.9 +15 48
Pixels ......... 16389120
Min ............ 437 @ 0,0
Max ............ 68691 @ 36,488
Background ..... 2988
Average ........ 2991.8873
Sum ............ 49034400157
Std.deviation .. 359.3689
So, somehow it exceeded 2^16 ?
Max from all lights:
Code: Select all
#1: Max ............ 65504 @ 4210,2681
#2: Max ............ 65504 @ 4210,2679
#3: Max ............ 65504 @ 4210,2678
#4: Max ............ 65504 @ 4209,2679
#5: Max ............ 65504 @ 4211,2679
#6: Max ............ 65504 @ 4210,2679
#7: Max ............ 65504 @ 4208,2679
#8: Max ............ 65504 @ 4208,2679
#9: Max ............ 65504 @ 4208,2677
#10: Max ............ 65504 @ 4208,2678
#2-1: Max ............ 65504 @ 1461,2481
#2-2: Max ............ 65504 @ 2205,164
#2-3: Max ............ 65504 @ 2202,164
#2-4: Max ............ 65504 @ 2204,162
#2-5: Max ............ 65504 @ 2204,162
#2-6: Max ............ 65504 @ 2203,164
Re: FITS file doubles in size after preprocessing
Posted: 17 May 2019, 10:57
by fabdev
OK, the range is 437..68691 (it can be read on the status bar of the image too), so it exceeds the 16 bit range.
The cause is the flat field correction. This can happen if the bias of the camera is low and the vignetting is strong.
To force a 16 bit range before saving there are several ways, for example the command: Arithmetic / Clamp / Maximum.
For a scientifically correct result the flat field must also be corrected from its dark frame.
Re: FITS file doubles in size after preprocessing
Posted: 17 May 2019, 13:44
by Rudi
Ok Fabio. I will use dark flats next time.
I was also recommended doing that for my new camera on a danish Astro site.