Xref: utzoo comp.compression:14 sci.astro:12166 Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!elroy.jpl.nasa.gov!decwrl!mcnc!uvaarpa!murdoch!murdoch.acc.virginia.edu!bglenden From: bglenden@colobus.cv.nrao.edu (Brian Glendenning) Newsgroups: comp.compression,sci.astro Subject: Re: Astronomical data compression Message-ID: Date: 24 Mar 91 21:08:17 GMT References: <1991Mar23.013557.28151@nntp-server.caltech.edu> Sender: usenet@murdoch.acc.Virginia.EDU Organization: National Radio Astronomy Observatory Lines: 41 In-Reply-To: sns@lo-fan.caltech.edu's message of 23 Mar 91 01:35:57 GMT [I have cross-posted to sci.astro] In article <1991Mar23.013557.28151@nntp-server.caltech.edu> sns@lo-fan.caltech.edu (Sam Southard Jr.) writes: >I have a topic/question that should be suitable for this newsgroup. I am a >software engineer working on the software for the Keck Telescope (the new 10m >in Hawaii). One of the things I am doing is taking the data from the CCD to >a VMEbus controller crate (Sun 1E running VxWorks) and from there to a >Sun 4/470 over the ethernet. > >Each image can be up to 4096x4096 pixels (a 2x2 mosaic of 2048x2048 CCDs), each >pixel being 16 bits. Obviously, if this kind of data is going over the >Ethernet, we want to compress it as much as possible. > [...] >Sam Southard, Jr. >{sns@deimos.caltech.edu|{backbone}!cit-vax!deimos!sns} Whatever you do, you don't want a lossy technique for transferring actual data (as opposed to guiding images or something similar). Since most of the sky is "blank" (bias +/- a few noise bits), and since those parts that aren't blank have redundant information (i.e. the psf covers several to many pixels) I have often thought you could do well by sending a model (bias plus source list plus psf approximation) and then encode the image as (actual - model). Hopefully most pixels could be represented by a few bits (obviously you'd have to come up with an efficient scheme for encoding variable width bit patterns). Of course your model could also include things like saturated columns, extended emission surfaces, cosmic ray hits etc., although if your model gets too complicated you start to lose in transmitting it. Another advantage of a scheme like this is that you can transmit the model first and send the residuals later, to let the image "build up" for the impatient "observer." Anyway, this is a pretty simple concept, better probably exist. Let us know what you find out! Brian -- Brian Glendenning - National Radio Astronomy Observatory bglenden@nrao.edu bglenden@nrao.bitnet (804) 296-0286