Register    Login    Forum    Search    FAQ

Board index » Software » Linux




Post new topic Reply to topic  [ 14 posts ]  Go to page 1, 2  Next
Author Message
 Post Posted: Mon Mar 27, 2017 10:58 pm 
Offline

Joined: Sun Mar 19, 2017 10:01 pm
Posts: 9
Dear All

I am having problems obtaining long exposures from the ASI120MC when using the RAW16 format. When gathering long exposures of greater than 1000ms I get a large number of failed exposures. This problem does not occur in RAW8, RGB24 or Y8.

I am using the following test code. When running this code with an exposure of 5 seconds I get, at least, 60% to 70% failed exposures. My camera has been flashed to use the ZWO compatible firmware because the standard firmware does not seem to work with Linux.


Code:
 

initialise_camera();

int exposure_count = 0;
int failed = 0;
int success = 0;

ASISetControlValue(0, ASI_EXPOSURE, 5000 * 1000, ASI_FALSE);
ASISetControlValue(0,  ASI_GAIN, 0, ASI_FALSE);

uint8_t *raw_Buffer = new uint8_t[m_Width * m_Height * 2];

while (true)
{
        exposure_count++;
        std::cout << "Exposure Count: " << exposure_count << "\n";
       
        ASIStartExposure(0, ASI_FALSE);
        status = ASI_EXP_WORKING;
        usleep(10000);

        while (status == ASI_EXP_WORKING)
        {
            ASIGetExpStatus(0, &status);
        }

        if (status == ASI_EXP_FAILED)
        {
            failed++;
            std::cout << "Failed Exposure: " << failed << "\n";
            continue;
        }

        // m_Width = 1280, m_Height = 960
        ASIGetDataAfterExp(0, raw_Buffer, m_Width * m_Height * 2);
       
        success++;
        std::cout << "Successful Exposure: " << success << "\n";
}


I would be grateful for any help offered.
NikkiM


Top 
 Profile  
Reply with quote  
 Post Posted: Tue Mar 28, 2017 7:42 am 
Offline

Joined: Tue Feb 21, 2017 8:45 am
Posts: 53
Dear Nikki,

Hi - I don't have this camera, but I do have the USB-3 version (120MM-S). I've been working with Yang to try and get this to work with The Sky X on a Raspberry Pi, and it does now more or less work.

In doing that, I had some code which looked a lot like yours (well, not as elegant ... :D), and it did work.

However, Richard Wright from Software Bisque has a 120MM, and he was struggling to get data from the camera in anything other than 8 bit - see the quote below from the ASI Imaging Website:

*Note:Our USB2.0 camera is not compatible very well with Mac OSX & Linux. USB3.0 cameras is recommended for Mac & Linux users and fully compatible with USB2.0 host.this thread may help you if you want to try our USB2.0 cameras on Mac OSX.

A couple of suggestions - try and download Kstars/INDI onto your system and see if that works with 16 bit.

Secondly, use the set properties routine to reduce the bandwidth to 40 before starting the image - that will give you the best chance of making this work.

Good luck!

Colin


Top 
 Profile  
Reply with quote  
 Post Posted: Tue Mar 28, 2017 8:21 am 
Offline

Joined: Tue Sep 17, 2013 4:50 pm
Posts: 77
Sorry for the negativity, but I've never made it to work decently. I think simply there is not enough bandwidth with the small size of transfers in compatible mode (one can argue why bulk transfers are used instead of isochronous ones but this could be a long story). I use the old ASI120MM in 8bit mode just for guiding now. I posted a patch in another thread here to disable the enforcing of compliant transfer size (this was introduced somewhere around 4.4 maybe, but perhaps it has been backported because it can be seen as a security issue). With it applied you can use the firmware with non standard endpoint size.

If you somehow manage to make it work decently at 16bit with the compatible firmware please share your solution, it would be pretty interesting.


Top 
 Profile  
Reply with quote  
 Post Posted: Tue Mar 28, 2017 11:51 pm 
Offline

Joined: Sun Mar 19, 2017 10:01 pm
Posts: 9
Thanks for your replies, it's appreciated.

Whilst I understand the issue related to ZWO using a maxPacket size of 1024 bit instead of 512 in their standard firmware. I don't believe (however, I may be wrong) that my long exposure RAW16 failure problem is related to USB bandwidth. My Linux application is actually capable of achieving frame rates as high, if not, higher than SharpCap (Windows). For example, I can achieve the following frame rates without any problems and with few dropped frames:

    RAW8: 320px x 240px: (Realtime debayering with a High Quality Linear Interpolation algorithm). Frame rates is upwards of 200 fps.
    RAW8: 1280px x 960px: (Realtime debayering) Frame rate is around 18 fps.
    RAW16: 1280px x 960px (Realtime debayering) Frame rate is around 9 fps.
    RGB24: 320px x 240px: Frame rate is upwards of 240 fps.
    RGB24: 1280px x 960px: Frame rate is around 18 fps.

My problem seems to stem from the following simple block of code. I have, of course, tested the return code of ASIGetExpStatus() and it always returns ASI_SUCCESS. It's just that the loop terminates, most of the time with ASI_EXP_FAILED. This can happen thousands of times in a row and other times I get perfect exposures each and every time. This fail cycle seems to be random but with the failures occurring most of the time.

I do set the ASI_BANDWIDTHOVERLOAD value to be 40 and set ASI_HIGH_SPEED_MODE to false.

Code:

// I doubt this function is using much USB bandwidth
while (status == ASI_EXP_WORKING)
{
      ASIGetExpStatus(0, &status);
}

if(status == ASI_EXP_WORKING)
{
continue;
}

// This function will use more USB bandwidth
ASIGetDataAfterExp(0, m_RAW_Buffer, m_Width * m_Height * 2);



I tried another experiment tonight. I removed the code block which checks the exposure status and replaced it with a usleep(exposure_length * 1 second) and got similar results. In the above code example, the data extraction function is the USB bandwidth heavy function but most of the time the execution does not reach that function.

If I can find the time I may try patching my kernel and use ZWO's vanilla firmware. The camera works perfectly on SharpCap (Windows) with the compatible firmware so I don't believe that there is a problem with the hardware. However, it's not impossible that there is a bug in the Linux SDK.

Anyway, thanks for you help so far. I would be very interested to see what a ZWO representative has to say on this issue.

Many thanks
Nikki


Last edited by NikkiM on Thu Mar 30, 2017 6:04 pm, edited 1 time in total.

Top 
 Profile  
Reply with quote  
 Post Posted: Wed Mar 29, 2017 7:45 am 
Offline

Joined: Tue Sep 17, 2013 4:50 pm
Posts: 77
The fact that it works for Windows is a data point I missed (I have only Linux boxes at home), thanks. I will give another look this weekend if I have time. BTW here: https://github.com/chripell/yaaca/blob/ ... ng/asill.c is a driver that directly handles the ASI120MM/MC without using the proprietary ASI one. I was able to find the data sheet for the old camera sensors (unfortunately I've never got to those for newer ones ... if anyone has them I can promise a lot of Guinness for a fair exchange ;-p ) and wrote it. It might be worth trying a different PCLK setting. I must admit that bandwidth problem is most probably not the right term. I remember seeing a lot of "broken frames", like if some piece of data were missing. Most probably the USB host is not polling fast enough to avoid some FIFO buffer in the sensor to USB glue logic to overflow. Maybe using a lower PCLK will help? (I remember changing it is well documented in the MT9M034 data sheet).


Top 
 Profile  
Reply with quote  
 Post Posted: Wed Mar 29, 2017 12:32 pm 
Offline

Joined: Tue Feb 21, 2017 8:45 am
Posts: 53
chripell wrote:
Sorry for the negativity, but I've never made it to work decently.


Remember, I have the ASI120MM-S, which is the USB 3 version - that may be the difference.

Colin


Top 
 Profile  
Reply with quote  
 Post Posted: Wed Mar 29, 2017 12:41 pm 
Offline

Joined: Tue Feb 21, 2017 8:45 am
Posts: 53
NikkiM wrote:
I tried another experiment tonight. I removed the code block which checks the exposure status and replaced it with a usleep(exposure_length * 1 second) and got similar results. In the above code example, the data extraction function is the USB bandwidth heavy function but most of the time the execution does not reach that function.


Interesting - where does your function fail? I was experimenting with an X2 driver (Software Bisque's standard), and there I got a lot of ASI_EXP_FAILED about mid-way through the exposure. This seemed to be because The Sky X was asking for or a lot of camera status requests. When this was reduced (went from 90,000 per exposure to ~ 100!), the driver worked a lot better - though not perfectly.

When I used standalone code, I got no errors at all, and that would have very rapid call rates to get the exposure status.

Colin


Top 
 Profile  
Reply with quote  
 Post Posted: Fri Mar 31, 2017 1:17 pm 
Offline

Joined: Sun Mar 19, 2017 10:01 pm
Posts: 9
Dear All

Over the last few days, I have undertaken various additional tests. My previous statement saying that it worked on Windows SharpCap was, in fact, incorrect. Testing again on Windows I discovered that RAW16 long exposures do not work.

mcgillca wrote:
Interesting - where does your function fail?


The failure only occurs when using RAW16 and with exposures greater than 1000ms. The failure happens in the following function. As you can see, it never reaches the function which actually downloads the buffer.

while (status == ASI_EXP_WORKING)
{
ASIGetExpStatus(0, &status);
}

When the loop ends, the variable "status" is, for the best part, equal to ASI_EXP_FAILED.

Here is an updated summary and my understanding of the problem so far.

If I use RAW16 (maximum resolution) with short exposures and realtime debayering (using a high-quality linear interpolation algorithm) I am able to obtain 7-9 fps. This clearly shows me that my USB bus is able to handle RAW16 traffic and at a reasonable frame rate. Surely, if I can obtain RAW16 7-9 fps with short exposures I should be able to handle RAW16 exposures greater than 1000ms.

Furthermore, my software is able to obtain frame rates of around 18 fps when using RGB24 (maximum resolution). The ZWO SDK requires that an 8 bit unsigned integer array (aka unsigned char) is passed into ASIGetDataAfterExp() for all types, including RAW16. Therefore the buffer length of RGB24 is width * height * 3. That buffer length is a larger than the buffer required for RAW16 (width * height * 2). So, if there were a USB bottleneck, surely it would present itself when using RGB24?

Now, we don't know exactly what ZWO are doing in their firmware or SDK but it does look like there is a problem with the compatible firmware and 16 bit raw output. This problem could well be related to the maxPacket size of 512 but, if so I'm now convinced that it's not a limitation of the USB2 bus per se.

It's a shame that we have not had any comments from ZWO on this thread as I am sure that this problem could be resolved by their engineers.

I am very disappointed by this and I hope there's a solution. :(

Kind Regards
Nikki


Last edited by NikkiM on Sat Apr 01, 2017 12:12 am, edited 1 time in total.

Top 
 Profile  
Reply with quote  
 Post Posted: Fri Mar 31, 2017 1:53 pm 
Offline

Joined: Tue Sep 17, 2013 4:50 pm
Posts: 77
With the ASI1600 there is a threshold in the exposure, above which only one bulk transfer is used instead of many rolling ones. I would suggest to set the environment variable LIBUSB_DEBUG to a high value before starting you program to check what is going on.


Top 
 Profile  
Reply with quote  
 Post Posted: Mon Apr 03, 2017 6:38 am 
Offline

Joined: Tue Feb 21, 2017 8:45 am
Posts: 53
NikkiM wrote:
The failure only occurs when using RAW16 and with exposures greater than 1000ms. The failure happens in the following function. As you can see, it never reaches the function which actually downloads the buffer.

while (status == ASI_EXP_WORKING)
{
ASIGetExpStatus(0, &status);
}

When the loop ends, the variable "status" is, for the best part, equal to ASI_EXPOSURE_FAILED

Nikki


I've just been looking at the INDI ASI driver for ccds. In that code, they trap this error and restart the image up to 3 times - might be worth restarting the image on failure and seeing if that works - or do you get failure every time you try and take an image?

Colin


Top 
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
 
Post new topic Reply to topic  [ 14 posts ]  Go to page 1, 2  Next

Board index » Software » Linux


Who is online

Users browsing this forum: No registered users and 2 guests

 
 

 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: