Download speed on ASCOM vs. native drivers

[AstroBernd]

I noticed that the download speed of images is much slower in ASCOM vs. native drivers in my new QHYCCD CMOS camera (QHY268C). When i use Sharpcap, i see 5.6 images/ second (fps) downloaded and stored as FITS files without any dropped frames. So, the whole setup (camera, driver, USB3, Software, hardware) is capable of doing that. When i use the ASCOM driver provided by the Camera manufacturer, the download time is getting much longer, i.e. 4 seconds per frame (a whopping factor of 20!). This means this is related to the ASCOM driver architecture and how it passes on the data to the receiving software (here: Sharpcap). I observed similar download times with AstroArt 7, so, no extra delays .
* Is this an observation others can share as well?
* Any plans to get the native drivers for ZWO or QHYCCD cameras directly interfaced into AstroArt? (Sorry, i am not a Software expert, so this question may be not exactly correct, but i hope you can see what i mean): Faster download times also in AstroArt!
Just a thought
Bernd

[Rudi]

That’s true, data download via ASCOM is significantly slower than through native drivers, at least for Canon and ZWO, which is the only cameras I’ve been using.
For deep sky photography, it doesn’t really matter for me, if every two minute exposure takes one second or 1/20 second to download.
But when focusing with AstroArt, both manual and automatic Full Frame Auto Focus, it would speed things up considerably,if AstroArt used native camer drivers.
I don’t know how easy ZWO’s SDK is to use, that’s up to Fabio to decide.

[AstroBernd]

I fully agree, Rudi, no big deal for deep sky images, but a lot of possible “unproductive “ time saving.

[fabdev]

Interesting: first, is the quality really the same? Downloading a multimegapixel image in 0.2 seconds could mean that readout mode was different (more quantization, faster ADC setting, lossy compression?). The following test could tell: a 5-10 seconds flatfield-like image at half range (30000 ADU), downloaded in both modes, then compare the standard variation and the quantization of both images.
Second question: if the two images are really the same, why the ASCOM driver is slower? COM is able to transmit megabytes of data really fast, all the operations are done in memory.

[Rudi]

Perhaps it’s because the live view in ZWO’s software is lower quality, I don’t know.
The focus liveview probably also uses ROI.

[Rudi]

I found an old post on the Danish forum astronet here:
https://www.astronet.dk/forum/viewtopic.php?t=2460

Here I wrote a little about the download speed.
To put it short, my ZWO 1600MM with 4,656 x 3,520 pixels at 16 BPP = 262,225,920 Bit should be able to transfer ca 20 FPS over USB 3. However, the real number is ca 14 FPS.
I have measured 10-12 FPS with FireCaputre, when using a dedicated USB 3 cable to the camera.

[AstroBernd]

This is about the different behaviors of the drivers supplied by camera manufacturers:

First, i have taken Rudi's calculation (see below) for the QHY268C (APS-C, 16bit, 26.8MPixel) and get a max theoretical frame rate per sec(fps) on USB3 (max 5GBit/s) of: 6280 x 4210 pixels @16BPP = 423,020,800 (432MBit) of 12.7 fps.
The specified download rate from QHYCCD is 6FPS @16BIT, binning 1x1.

As mentioned earlier, i measured the download rates on my PC system with SharpCap (v3.2.6442.0, 32 bit). Those results are also being shared with QHYCCD. I am waiting for their response.
When using the native QHY-driver, the frame rate shown in SharpCap was 5.9fps . this is confirmed by the timestamps of the 10 files downloaded and saved.
This is well below the theoretical limit, but very close to the manufacturer specified value and it includes my whole computer system including file saving. The files are true 16bpp.

When using the ASCOM driver provided by QHYCCD in the same setup and software, i measured a total download rate for 10 images of 46sec, i.e. 4.6sec per image. This is 27times SLOWER than with the native driver. By the way, the download time in AstroArt with the QHYCCD-ASCOM driver is 15% faster (4.0sec).

I have taken Fabio's comment below on the image quality and done some measurements with my system in the twilight to generate something like flat fields (not real flat fields). The telescope was pointing downward and collected reflected light from the floor. As this setup is very light sensitive at f/2, my exposure times were only 0.07sec (ASCOM) and 0.036sec (native driver) to get a decent mid-range exposure between 0 and 65536. I took 10 dark frames, 10 "flat fields" and averaged, dark frame subtracted and analyzed those in AstroArt7.
Here is a summary of the analysis:

Min Max Background Avg Sum Std. Dev rel. Std. Dev.
native driver: 8613 30318 20298 21323 5,57E+11 6209 29,1%
QHY-ASCOM driver 10316 35968 23646 25186 6,58E+11 7373 29,3%

The absolute levels are different, but it also took me some time to get everything switched from one to the other driver while night was falling. The images also look very similar.
My conclusion is there is no discernible difference in the image quality.
Rudi has seen similar effects on Canon and ZWO CMOS cameras, so these CMOS cameras can not show there full speed and application potential when you have to use the ASCOM driver.
I don't know the underlying reason, but as a user, i am throwing away a lot of precious time under the stars if i have to use the slower driver.

That is why i asked about "* Any plans to get the native drivers for ZWO or QHYCCD cameras directly interfaced into AstroArt?" in my first post. For classical DSO imagers, it would help saving on "unproductive" time (image selection, estimation of seeing, even focus, especially full frame focus like Rudi mentioned). in addition, the community of Deep Sky Lucky imagers, EEA (electronic assisted astronomy) and planetary imaging with video capture is ever increasing and can not use software that can not connect to the native drivers.

Any thoughts on this?
How many of the AstroArt users use or plan to use CMOS cameras like ASI ZWO or QHY, but also DSLRs? You will have the same issue.

[fabdev]

So, that ASCOM interface added a latency of 0.15 seconds per Megapixel.
A native driver could be very useful for OSC cameras with more than 10 Megapixel, where you are forced to download images in binning 1x1. It will be developed. Meanwhile if QHY solves the ASCOM problem please report here.
I didn't understand the test by the way: averaging 10 images will mask any quantization and other defects, may you compare a single flat field in native mode and a single one in ASCOM mode ?

P.S.
5 years ago downloading 26 Megapixels in 4 seconds would have been an excellent result (!)

[AstroBernd]

Fabio, thanks for your quick reaction.
I agree, the added latency added by the ASCOM driver doesn't sound like a lot and is not bad, BUT if the native driver downloads up to 6fps full frame in 1x1, it is still more than an order of magnitude slower.
Yes, the QHY268C and the corresponding ASI ZWO 2600MCPro are APS-C OSC cameras with 26.8MPixel, the corresponding full-size cameras have 60 MPixels (!).
I appreciate very much that you are going to add it to your developments. Thanks a lot for considering it, Fabio!
I will also post any answers from QHY regarding the topic (nothing yet as of today).

Sorry for the wrong testing, my fault. I have looked at single images as well. Here is a screen dump of 2 single frames (ASCOM left, native right) acquired with Sharpcap and analyzed in AA7.

Also, the statistics have not changed significantly ( I am sending you a private link to the original data as well for your analysis, if i have missed something). Here are the statistics of the 5th file each:

. Min Max Background Avg Sum Std. Dev rel. Std. Dev.
native driver: 8613 31448 20926 21610 5,64E+11 6227 28,8%
QHY-ASCOM driver 10554 38064 24906 25976 6,78E+11 7468 28,7%

PS: I fully agree with you, those speeds have been completely unheard of just 5 years ago, but they are also setting new standards and expectations and opening completely new possibilities.