DAVID structured light scanners

Problems, solutions and discussion about scanning with a video projector and stripe patterns.

structued light/phase shift homebrew

Postby mm_alex » Sat Nov 24, 2007 7:18 pm

hello,
this is my first post as i only discovered the david scanner and community today :) in fact, i haven't even tried david out yet! so forgive in advance my ignorance...
anyway, I was wondering about 3d scanning for a while and some web-searches brought me here. I was interested to read the david technical paper, and then found this active community, which was a nice surprise.
via this forum, I found a link and youtube video to the work of
Song Zhang - http://www.math.harvard.edu/~songzhang/ - and I spent the afternoon reading his phd thesis on phase-shift imaging.
I don't know anything about camera calibration, computer vision, etc, but I am a good programmer (from the games industry) and after reading about his thesis, it seems that phase shift technique (which in his case is capable of realtime capture in 3d at 120 frames per second! see http://www.youtube.com/watch?v=DiY45jALWjE
and in fact the technique seems to me to be very easy to program -at least in an approximate, rubbish, 'homebrew' style. a simple implementation would not be attempting to solve all the interesting and hard parts that for example david solves, eg shape matching, absolute geometry scale, etc. however basic 'heightfield' type output seems possible as a short term project.
has anyone here tried this, and if not, I think I might give it a go (free time permitting - and lack of a working projector not withstanding)! any input or thoughts on pitfalls / any already existing successful or failed attempts at things like this / etc appreciated.
mm_alex
 
Posts: 7
Joined: Sat Nov 24, 2007 7:02 pm
Location: london

hi

Postby hal » Sun Nov 25, 2007 5:18 pm

salve,

WOW: real time scanning! incredible... :shock:
Simon, Sven! when david can do this?! :wink:
bye
PC: DELL M90, Intel T7200, 4 Gb Ram, nVidia Quadro 3500FX.
SO: Win 7 64Bit.
CAMs: USB CCD Color (1024x768, 30 fps) & Canon EOS 1000D.
LENS: 12 mm & VariFocal 3,5-8 mm.
PROJECTOR: ACER K11 & DELL M110.
LASERS: 5mW and 16 mW.
User avatar
hal
Moderator
 
Posts: 1468
Joined: Thu Mar 01, 2007 4:08 pm
Location: Cavaria con Premezzo (VA), Italy.

Postby johnrpm » Mon Nov 26, 2007 9:17 am

Welcome mm_alex and thorn n vine
This 3d scanning thing seems to get under the skin, once you play around with it you can never really forget about it, wouldn't it be great if David had the option to switch from laser scanner to structured light... none of the others do that as far as I know...
mm_alex, if you are willing to have a go at structured light then I wish you every success, please keep us informed of your journey, if you do decide to do it what about doing a blog on the wiki, we could follow your learning journey and maybe other coders may join in to help????????.
User avatar
johnrpm
 
Posts: 77
Joined: Tue Apr 24, 2007 8:31 am
Location: UK

Re: structued light/phase shift homebrew

Postby 3D » Tue Nov 27, 2007 3:04 am

mm_alex wrote:hello,
this is my first post as i only discovered the david scanner and community today :) in fact, i haven't even tried david out yet! so forgive in advance my ignorance...
anyway, I was wondering about 3d scanning for a while and some web-searches brought me here. I was interested to read the david technical paper, and then found this active community, which was a nice surprise.
via this forum, I found a link and youtube video to the work of
Song Zhang - http://www.math.harvard.edu/~songzhang/ - and I spent the afternoon reading his phd thesis on phase-shift imaging.
I don't know anything about camera calibration, computer vision, etc, but I am a good programmer (from the games industry) and after reading about his thesis, it seems that phase shift technique (which in his case is capable of realtime capture in 3d at 120 frames per second! see http://www.youtube.com/watch?v=DiY45jALWjE
and in fact the technique seems to me to be very easy to program -at least in an approximate, rubbish, 'homebrew' style. a simple implementation would not be attempting to solve all the interesting and hard parts that for example david solves, eg shape matching, absolute geometry scale, etc. however basic 'heightfield' type output seems possible as a short term project.
has anyone here tried this, and if not, I think I might give it a go (free time permitting - and lack of a working projector not withstanding)! any input or thoughts on pitfalls / any already existing successful or failed attempts at things like this / etc appreciated.
:!:
User avatar
3D
 
Posts: 27
Joined: Thu Oct 25, 2007 7:20 am
Location: Belgium

structured light project

Postby mm_alex » Wed Nov 28, 2007 11:45 pm

I spent a bit of time this evening thinking more about the structured light project. thought I'd post my thoughts here. I was thinking of making a new topic, since this isn't really a david feature request, but wasn't sure where to put it (any ideas?)
anyway, the attached image (setup.jpg) shows my idea with details. it seems to me the simplest setup I can use to test the ideas of zhang's phd thesis. in the end, only 3 'fringe' patterns are enough to capture 3d meshes with this setup, although I wrote some tiny piece of code to generate a bunch (53!) of test-card images which I'd project over the scene and photograph - because I don't really know what will work and what won't, it seems sensible to overkill at first - have a nice photoshoot session and then spend some time writing software to process the images. only problem for me is... I don't have a camera with remote shutter control or a tripod :( (tho I have found myself a bad old lcd projector...)
do any more experienced scanners here have any (useful) thoughts? am I doomed!? :shock:
Attachments
setup.jpg
proposed setup + details
test_slides.zip
proposed test slides to be photographed both with and without object present in scene (for calibration and test!)
(489.17 KiB) Downloaded 1070 times
mm_alex
 
Posts: 7
Joined: Sat Nov 24, 2007 7:02 pm
Location: london

Postby johnrpm » Thu Nov 29, 2007 9:21 am

From what I know, it looks to me that you are precisely on the right track, if the finished system could scan quickly to capture humans that tend to move and then if the proccesing takes some time it would be a good result, if you could do realtime scanning then that would be a real bonus.
User avatar
johnrpm
 
Posts: 77
Joined: Tue Apr 24, 2007 8:31 am
Location: UK

Postby mm_alex » Mon Dec 10, 2007 6:48 pm

I spent some time this saturday coding, after borrowing a video camera and a projector last week :)
here is my first attempt. it's from 3 640x480 greyscale video frames of 3 fringe patterns, taken from a single front view. (the model is shown here from the side so you can see where the scan 'ends' due to shadowing)
there are large systematic errors leading to the 'ridges' on the model, which should be removable by more careful calibration of the camera's gamma curve. and I'm doing no filtering for noise or outliers at all. still, it's promising for only a few hundred lines of (disgusting, horrible, magic-number-ful) c code. :)
Attachments
test.jpg
early test of a structured light 3d scan
mm_alex
 
Posts: 7
Joined: Sat Nov 24, 2007 7:02 pm
Location: london

Postby Bongobat » Mon Dec 10, 2007 7:35 pm

Congratulations on your results. That is absolutely marvelous! Do you think in the future you could use a static projector such as a slide projector or is that impossible? I am very impressed :)
DealExtreme red line laser, Microsoft Lifecam HD-5000
User avatar
Bongobat
Moderator
 
Posts: 1002
Joined: Mon Apr 02, 2007 10:02 am
Location: Ohio, USA

Postby johnrpm » Tue Dec 11, 2007 8:58 am

Wow, that is a brilliant start and in such a short time, well done.
User avatar
johnrpm
 
Posts: 77
Joined: Tue Apr 24, 2007 8:31 am
Location: UK

Postby 3D » Wed Dec 12, 2007 10:56 pm

mm_alex wrote:I spent some time this saturday coding, after borrowing a video camera and a projector last week :)
here is my first attempt. it's from 3 640x480 greyscale video frames of 3 fringe patterns, taken from a single front view. (the model is shown here from the side so you can see where the scan 'ends' due to shadowing)
there are large systematic errors leading to the 'ridges' on the model, which should be removable by more careful calibration of the camera's gamma curve. and I'm doing no filtering for noise or outliers at all. still, it's promising for only a few hundred lines of (disgusting, horrible, magic-number-ful) c code. :)

You're our hero. :!:

How many times for the scan of this face plz ?
User avatar
3D
 
Posts: 27
Joined: Thu Oct 25, 2007 7:20 am
Location: Belgium

structured light scanner images & example code

Postby mm_alex » Thu Dec 13, 2007 12:40 am

bongobat, 3d, johnrpm: thanks. I'm not ready for hero status with scans that noisy though!
:)
Bongobat: yes, pre-made slides would absolutely be possible. I've made all my test scans (all 3!) so far with just 3 projected images, see attachments. the images consist of simply 32 horizontal 'stripes' that are in the form of sinewaves, 'phase shifted' 120 degrees from each other. the projector is mounted 10-20 degrees below the camera, tilting slightly up towards the face, and the camera which is front-on to the face. (a bit different from my original setup)

there is no camera calibration or geometry calibration step at the moment, I'm making an assumption of linearity of the camera/projector brightness curve which is pretty naughty! (and causes the ridges). but I can't yet figure out a way yet to compensate for them. and there are magic numbers in the code related to the angle between projector and camera.

so far I've tried 3 camera setups (all using a bad old 1024x768 lcd projector with HORRIBLE banded output) - 1, an HDV video camera which had too many jpeg artefacts :( 2, a 640x480 logitech messenger cam which I foolishly passed thru an indeo avi compressor before reaching my code, quite amazingly ok results from it, and finally a 6megapixel still camera on a tripod (though it shook a bit each time I pressed the shutter). none of these are really a 'good' quality setup. :( but I don't really have the time to make a good job of it. still, it's been fun learning :) surprisingly the quality of the camera isnt the biggest issue for my scans ATM, because the noise of the ridges far outweighs the noise caused by bad camera/projector!

I've attached 3 stills from the latest photoshoot. the camera shook between exposures so I had to line them up by hand and it's all a bit of a mess. resulting scan is recognisable but not great :(
I've attached code too for those who are good at C++ :) note this is NOT ready for general consumption so unless you have libpng, zlib, directx, a copy of devstudio and a lot of time, I don't recommend worrying too much. on the plus side, its very very short and more-or-less platform agnostic. it eats 3 png files (eg the ones attached to this post) and spits out various test png files of depth maps and phase maps and etc, as well as an OBJ file of the mesh (as per my last post). that you can load into a 3d program.
the algorithm is the fast phase shifting algorithm, roughly after zhang, and then I unwrap the phase using a made-up variance guided floodfilling unwrapper, which I think is OK but I cant get access to most of the relevant papers without paying, so I decided to just cook it up :)
enjoy!
Attachments
dave1.png
dave2.png
dave3.png
mm_alex
 
Posts: 7
Joined: Sat Nov 24, 2007 7:02 pm
Location: london

Postby mm_alex » Thu Dec 13, 2007 12:43 am

hmm. the forum ate my 8k zip file! maybe I'll try again. sorry for double post. hopefully this has the code.
mm_alex
 
Posts: 7
Joined: Sat Nov 24, 2007 7:02 pm
Location: london

Postby johnrpm » Thu Dec 13, 2007 9:17 am

Well I think you have done a wonderful job, to go from a standing start and get results in such a short time is remarkable, and although I am as thick as two planks I do know people who are working in this area, I could ask for papers if you let me know what to ask for.
User avatar
johnrpm
 
Posts: 77
Joined: Tue Apr 24, 2007 8:31 am
Location: UK

Postby azkuene » Fri Dec 14, 2007 8:28 am

Wow, great job!! :shock:
How are you going to synchronize the projector and camera??
azkuene
 
Posts: 5
Joined: Tue Nov 27, 2007 10:18 am

Postby Bongobat » Fri Dec 14, 2007 8:56 pm

Hi again mm.alex

Your quick work has turned me from a skeptic into a believer. I was hoping that perhaps you could do me a favor and briefly explain the whole process you used in laymen's terms. I read the paper's available and even read up on DLP technology and still cannot figure out how exactly 3d data is being derrived from the fringe patterns. Does phase refer to the phase of a light wave? What exactly is a fringe pattern? and etc. Basically I need you to avoid using too much math lol. Thanks again for your terrific work and I hope you continue the development.
DealExtreme red line laser, Microsoft Lifecam HD-5000
User avatar
Bongobat
Moderator
 
Posts: 1002
Joined: Mon Apr 02, 2007 10:02 am
Location: Ohio, USA

PreviousNext

Return to Structured Light Scanning

Who is online

Users browsing this forum: No registered users and 0 guests