In my original post, I asked how a robot would be able to recognize itself, when presented with its image, covered in spaghetti. In other words: how can a robot robustly and reliably recognize itself, based on images? Simple enough, right?
Well, how would a robot ‘see’ to begin with? What’s the ‘vision’, of computer vision? Gabuglio wondered last time, in this comment whether the PR2 could be like Rosey from the Jetsons. Unfortunatley: no. Or at least, not at this point. Using only computer vision however, it could do some other jobs.
Right now, our robot could stand in a factory, matching label colors against the desired color for a paint job . A factory is a highly controlled environment, so you might get away with just using thresholding. For red paint for instance, if your image is made up of levels of red, green and blue: check to see if there’s a uniform patch in the image that’s more than 90% red, but less than 10% blue or green. He could do something more advanced as a factory worker, and be a bottle level inspector. He would probably use an edge detector for this, like you could in Photoshop or any other image editing program. These are some of the simpler operations. Generally speaking, they’re very easy to understand, and use. Like the circuit laws, or the ideal gas law…
If our robot went to school a little bit longer, it might be working for the TSA, where it would be in high demand right now. As you may know, they use so-called full-body scanners over there. That used to mean someone 30 feet down is supposed to literally, but might be figuratively, looking under your clothes, ‘checking you’. Some people were offended… Our robot could do a more acceptable job, and these days, they do.
Obviously there’s a lot more involved here. A lot of it though, would have to do with image segmentation: partitioning the image into more meaningful, analyzable regions. Once that’s taken care of, a computer get rids of the areas that are definitely not of concern. What remains is marked, to be inspected by a human. It could do more meaningful things too, like finding tumors in fMRIs (computer vision right now has a lot of applications in medical imaging).
How would the PR2’s computer know what to mark, and what not? To make the problem clearer, I’ll give our robot yet another job: to check my fingerprints at the border control (this is the part I hate, but anyhow…). Suppose I was a criminal, how would you compare my fingerprints against the millions of fingerprints of known criminals. This is where you need feature extraction; you need some way to extract a small amount of information from the image, that still represents its content, and can be compared with similar information. Possibly through something like the aforementioned methods, sometimes something more advanced, like in the fingerprints below. Sometimes, the features might not have a clear meaning to us anymore, and sometimes they simply don’t… The measurements in this photo of a fingerprint for instance, make total sense:
At least when compared to the features found in these faces:
These are the kind of tools a computer can use to transform images into something it can make sense of. But, it still doesn’t explain how he’d be able to recognize himself. Or spaghetti for that matter… I’ll explain how it can, in my next post.