When a Hyperspectral Image is a Cube

When I was interning at Carnegie Mellon Vision Science Labs the summer after my junior year, I was having difficulty running the code in a way that did not cause my small computer to crash. So, Professor Sankaranarayanan told me to ask one of his graduate students for ways to make more efficient pipelines. I told her I was working with hyperspectral images. She paused for a second and asked, "Are those cubes?" I responded that they were. We eventually figured out a solution to the problem.

If you are at all familiar with spectroscopy, or or vision science in general, you are probably wondering why they referred to hyperspectral images as 'cubes'. I will explain why. Let's say you want to have a black and white image stored on a computer. The data looks like a square. Each value in the square is a value from 0 to 1, with 0 being black and 1 being white. It looks kind of like this:

What about color images? It really doesn't take much to make the human eye perceive color. We call red, green, and blue the 'primary colors,' but that doesn't mean that all colors are made up of red, green, and blue. If you create light with a wavelength of 570–580 nanometers, you are going to see yellow. Red, green, and blue are not present. However, if you have red light (620 to 750 nm) and green light (490–580 nm), your brain will think you are seeing yellow light. You are not! But our eyes are designed for fairly basic tasks, so it's all we need. As a result, most computer screens have only red, green, and blue diodes. That means that colored images are stored in computers like this:

With our limited eyes, we have a pretty good idea of the world around us. But what if we could get a computer eye that could differentiate much narrower shades of color than a human eye can? There would be one matrix for 490 to 495 nm, another for 495 to 500 nm, etc. That is what a hyperspectral image is! You can't really display it on a standard screen because that isn't what computers are designed to do, but you can get lots of data on what the substance you are observing is. And, since there will be hundreds or thousands of matrices all stacked on top of each other, it looks kind of like a cube! And that is why the graduate student called it a cube.

A year after this, I was working in Specere Labs, which studies light and spectroscopy. I was explaining to some other researchers what I had been studying at CMU. But they didn't see hyperspectral images as cubes; they saw them as groups of spectra.

Generally speaking, most spectroscopy labs don't study hyperspectral images. When you have a hyperspectral image, there has to be a way of measuring the magnitude of the signal from 490 to 495 nm, etc. This is extremely hard! Specere Labs covers a wide range of research focuses, but this is a major one.

When a spectrometer is used, it generally doesn't return a hyperspectral image; it returns a spectra. A spectra is typically viewed like this:

The above plot could be seen as a cross-section of a hyperspectral image, but almost nobody thinks about it like that because that is not how the spectral data is typically obtained. So in Specere Labs, hyperspectral images were not cubes. They were groups of spectra taken from regions nearby each other.

So while I don't think this information is earth-shattering, I think it does illustrate why I really enjoy cross-disciplinary work and learning from multiple professors in multiple labs.