1) Overview

With shutterless operation, the derivation of the IRAC flat-field is problematic. Previously, we would have taken concurrent dark images and subtracted these from images of the hi-zody sky. Once dark-subtracted, it is relatively easy to then derive the flat either through traditional means (median stacking of sky frames) or via more advanced methods such as the Fixsen algorithm. Without a true dark, we are forced to fall back on either deriving the dark dorectly from the data or using a substitute, such as a "dark" region of sky.

It is important to keep in mind that these simulated exercises must produce the flat-field images entirely from what can be derived in flight. It is insufficient to show that derived results are consistent with the input calibration data, they must reproduce the calibration data directly as output.

2) Input Data

The input data was a set of "truth images" made by me and representing the noiseless infrared sky. For the low ecliptic (high background) these consisted of three components:

• Initially, a zodiacal background model generated by Bill Reach, which was normalized to the background estimate given in SPOT for ecliptic 110,0. The gradient in this field is probably larger than that of the real sky. Later this was replaced with a lower-level gradient. This had an intensity of 0.25MJy/sr at band 1, and 17 MJy/sr at band 4.
• A stellar model consisting of stars and their magnitudes taken from the 2MASS catalog for the above position. Since 2MASS isn't very deep, the stellar luminosity function was extended down several more magnitudes and stars were added with random positions in the field. The stars were all treated as colorless, i.e. they were all treated as having the same magnitude as the 2MASS stars, and their zeropoints were set using the Vega zeropoints given by Tokunaga.
• An extragalactic background model created by the SWIRE team.

For the low background case these consisted of only the extragalactic background and a zodiacal sky level corresponding to the "low" level found in the IRAC user's manual.

The simulated data were produced using the ISDS. For the experiment in section 3, I wanted to produce many frames to see how the acheived noise scaled with total exposure time. I chose a 3x3 (30" throw) mapping option with an 18-point releaux dither at each map point. Thus, every frame is dithered, with a variety of spatial scales, although this dithering was hardly optimized. 162 100-second frames are produced per band. This AOR is roughly the longest that an AOR is allowed to be (5.2 hours). The ISDS was run with most instrumental effects turned off, with the exception of the gain and dark. This will produce a product similar to what we will have in flight, where the pipeline will preprocess the data before special handling for producing calibrations.

3) Mask, Coadd, and Difference

Here I present flat-fields derived using traditional IR observing techniques. Images are taken of a "target" (in this case, a region of high zody in the ecliptic plane) and from these are subtracted images of "sky" (a high ecliptic field). The result is something akin to a median sky flat.

The data was reduced by first flagging all the objects (stars, galaxies, etc.) in the data. This was done with a simple median filter detection algorithm (notably, the CRUTIL.CRMEDIAN task in IRAF) which produces a mask image (one per data frame) with all the pixels containing "pointy" objects flagged. The data was then combined using the IMCOMBINE task in IRAF, using the median option and a priori rejection of masked pixels.

Fig. 1 - High zody images in bands 1 (top) and 4 (bottom), showing the mask generated by CRMEDIAN.

Results

Band 1

Band 1 is the hardest band to reduce via this method because the sky is very faint relative to the stars, and rejection of the high stellar density in the ecliptic is fairly difficult. Shown below is the result of such a reduction. The resulting flat is quite good on small spatial scales with a pixel-to-pixel error of 0.7%. There are, however, low spatial frequency residual errors with an amplitude of 1.5%. These are errors left over from wings of the PSF from bright stars. The dither and mapping pattern selected were too small to fully dither these out.

Fig. 2 - Derived flat-field (left), true flat-field (center), and percent difference (right). There are obvious large-scale residuals of order 1.5% left over from inadequately rejected stars. Photometric noise on small scales is 0.7%.

Fig. 3 - Derived flat-field (left), true flat-field (center), and percent difference (right), derived from dataset with larger random dither pattern. 1-sigma error is 0.6%, the maximum large scale deviation (at bottom right) is about 0.5%, the rest of the image is flat to 0.2%.

In an attempt to fix this problem I repeated the experiment, only using the cycling dither pattern and a 100" mapping throw. The results are shown above. Now the large spatial frequency errors are gone. The derived flat is the same as the input flat with a 1-sigma error of 0.6%. One thing that this result points out is that given the potential problems caused by bright stars, it would be better if we plan the dither patterns and positions used for the flat-fielding data collection based on our actual knowledge of the IR sky, rather than attempting to produce a generalized solution to the problem. A more sophisticated star rejection scheme would do better than what was used here. The ideal system would use a better source detector (SExtractor or DAOPHOT) and then apply a mask of variable size based on the extracted magnitude of the source.

The total time needed to collect the flat-field data is driven by band 1, where the sky is the faintest. Since data in all 4 bands is taken at the same time, we are always limted by the band 1 result. The above results took a total of 10.4 hours of wall-clock time to acquire the data.

Band 4

Band 4 is relatively easy. Here the sky is very bright relative to the stars, and an extremely high S/N for the sky can be quickly acheived with minimal difficulties rejecting the stars.

Fig. 4 - Derived flat-field (left), true flat-field (center), and percent difference (right). Pixel-to-pixel error in the derived flat-field is 0.3%, dominated by the vertical striping. Max deviation is about 0.4%.

The vertical striping in the derived flat is probably a result of erroneous dark subtraction . The peak-to-peak amplitude of these stripes is roughly 0.5%, and is the dominant contributor to the flat-field error. Within a given stripe the pixel-to-pixel scatter in error is only 0.05%. One approach to fixing this is to fit a constant line-by-line and subtract this. This is shown below.

Fig. 5 - Destriped flat-field (left), true flat-field (center), and percent difference (right). One-sigma deviation over the whole image is 0.3%. Max deviation is about 1%, but is due to erroneous structures in the true flat which are removed by vertical destriping. Within a small region the photometric noise is below 0.05%.

The resulting destriped flat is shown at left, and is very similar to the true flat. Unfortunately, because of the way the true flat was originally derived by Rick, it contains (non-real) vertical structure which the destriping has removed. Even so, the resulting destriped flat deviates, pixel-to-pixel, by only 0.3% from the true flat.

Update (01/18/02): The striping is now believed to be likely not due to bias offsets, but to how the individual gains are applied to the independent readout channels in the IRAC detectors (which may or may not be a bug in how the simulator is working when it applies the calibration flat). Shown below are the results of destriping based on forcing the median value in each detector readout channel to be the same.

Fig. 5b - Results of destriping based on separate gain values per readout channel.

4) Fixsen Reduction

I present the results of attempting to flat-field using the Fixsen algorithm. The code is what Rick sent me, with a new front end written for escargot in order to more easily read in the simulator output.

Simultaneous Solution for Flat and Dark

For this I fed in the results of high and low zodiacal background simulations. I had to set the positions of the low zodiacal background data to something relatively near the high background frames so that the generated skymap would be finite in size.

There were 72 30-second images in the high background, and 72 in the low. They were taken with a 36-point releaux pattern. This test should probably be rerun with a larger dataset with more exposure time, but this has already taxed the memory resources of my available computers.

Fig. 6 - Fixsen output for blank sky regions in the high and low background cases. Gains are on left, offsets are on right, band 1 is on top, band 4 is on the bottom. Typical errors are factors of 2.

Results were not very encouraging. There seems to a be a lot of degeneracy between the solutions for the gain and the offset. This doesn't surprise me given the expected uniformity of the celestial background. If the background model is to be believed, the total contrast beween high and low background is typically only a factor of a few, and I am not sure if this is sufficiently high dynamic range. Also, I am using the default weights, perhaps others would work better. Note that the sky reconstructions look pretty good (the errors in channel 4 are due to masking errors). Also note, however, that the algorithm has reconstructed the sky as having identical backgrounds in both locations, when the data on the left should have been several times higher than the right.

Fig 7. - Gains and offsets derived simultaneously via Fixsen.

Gain-Only

Using Fixsen to derive just the gain essentially reduces it to a fancy way of separating the structure in the sky from the gain. This requires that one supply a "dark", if not, the derived gain will have a strong component from the dark, as seen below.

Fig. 8 - Derived gainmap resulting from processing band 1 high background data with no dark estimate. The gainmap has a strong component from the dark, as evidenced by the broad glow in the bottom left of the image. Since this dark component appears in all the data, it is interpreted as a region of high gain producing an elevated background in the input.

Where, then, does one get the dark? One could imagine that one could process the high and low background cases separately in much the same way as was done for the coaddition method above. This doesn't actually work, however, because what Fixsen is producing for the gainmaps isn't in absolute counts but rather in gain units, and hence they can't be subtracted from each other. For the sake of argument I consider the case where a "dark" already exists. In particular, I use the exact same high background datasets as were used to produce the best case coadd results above, namely 162 frames taken with the big cycling dither pattern. For a "dark" I have subtracted off the coadded low background image from each individual high background image. Personally I think is something of a cheat since it requires prior processing of a different dataset with a different technique, but then this is just for comparison.

Results are shown below. In band 1, the flat deviates by roughly 0.5% from the true flat. No doubt some of this is due to noise in the sky subtraction. There is some spatial structure (the small blocks are due to errors in the bad pixel masking). This structure looks essentially the same as that seen in the coadd method above, and it's amplitude is about 0.5%. The similarity between the results of the two methods strongly hints that the error is introduced by the dark subtraction. For band 4, the flat is again dominated by vertical striping whose total amplitude is roughly 0.6%. One-sigma error is around 0.3%. The reconstructed sky maps look terrific.

Fig. 9 - Fixsen gain-only processing. Derived flat (left), true flat (middle), percent deviation (left).

Fig. 10 - Fixsen gain-only processing for band 4. Derived flat (left), true flat (middle), percent deviation (left).

Fig. 11 - Fixsen derived skymaps for the high background case (top) and low (bottom), for bands 1 (left) and band 4 (right).

From this somewhat limited exercise, several conclusions can be drawn:

• IRAC can be flat-fielded in all bands at the 0.5% level using roughly one PAO worth of data. 1% flats could probably be acheived in just a few hours. Further simulations will be needed to explore this additional parameter space.
• Either of the two methods outlined above will work roughly equally well. Fixsen is probably less sensitive to the exact details of the dithering relative to bright stars.
• Given that we know what the IR sky looks like at such high flux levels, it would be prudent to model the sky and derive a set of customized flat-field observations whose dithering and depth achieve the required goal. Although more labor-intensive than a generalized solution, it will produce better flats. Restriction to a relatively small number of designated flat-field targets would help limit the scope of the problem.