Team:TU Darmstadt/tech/software

MainPage

Software HoloPyGuy


Here, we present HoloPyGuy, an universal software solution created for Digital Inline Holographic Microscopy (DIHM). HoloPyGuy is an easy-to-use hologram reconstruction suite for self-made DIHM. We employed the open-source framework HoloPy, extended the existing solution with a graphical user interface, which greatly ease the whole process of parameter tuning, visualization of the holograms and reconstructed images. The software also supports 3D imaging from the reconstructed stack of images, by invoking the open source software Fiji. The graphical user interface relies on the Qt5 framework and is written in Python. The project is hosted on GitHub under MIT License and is also available for download. A complete user manual is provided in the following section.


HoloPyGuy: User Manual


HoloPyGuy functions simply by uploading the reference picture, the recorded sample hologram, adjusting parameters, and hit “Reconstruct”. As demonstrated in the video, we first adjusted the parameters according to our experimental setup, whereas the settings in “Point Source” tab are explained in the table below. Note that HoloPyGuy also supports collimated source calculation, if an collimated source instead of a pinhole is used.


As HoloPy only supports square images when it comes to point source, HoloPyGuy automatically cuts the images into square images stemming from the center, with equal linear pixel size, once the images are imported. In addition, HoloPyGuy also converts the images to gray scale, this is due to the fact that for a coherent light source only a single wavelength is considered in the algorithm, thus color information is discarded for a simpler and better reconstruction.


Parameters Description
Spacing Distance between the center of two pixels, can be arbitrarily specified in point source setup, relative to “Magnification”
Wavelength Wavelength of the used light source in nm, in our case the cell phone flashlight has a wavelength of approx. 450 nm
Medium Refractive index of the medium, HoloPy currently only supports index = 1 for point source setup
Magnification Magnification factor from the sample object to the reconstructed wavefronts, can be arbitrarily specified, relative to “Spacing”
Distance The distance between image sensor to light source in mm (don’t forget the distance from image sensor to cam!)
Z min Minimum distance from the camera to the xy plane near the object for wave front reconstruction
Z max Maximum distance from the camera to the xy plane near the object for wave front reconstruction
Z steps Numer of wavefronts at various distance near the object to be reconstructed
Outpixel [#] Size of the output square pictures, can be used to cut the center part of the reconstructed images

In our experiment we used cell phone flash light attached with a 5 micrometer pinhole, which makes it a point source. Unlike the collimated source, the pixel spacing at the reconstructed wavefront can be chosen arbitrarily due to the magnification of the emanating spherical wave. The “Spacing” and “Magnification” parameter are relative and reversely proportional, the higher the magnification, the smaller the pixel spacing, thus the higher the resolution in the reconstructed images. When the reconstructed image has less pixels than the specified “Outpixel[#]”, the whole images will be displayed in “Reconstructed Images”, otherwise the images will be cut from the center [15].
Next, select “Point Source” tab, “Open Background” to import the background image, such that the recorded hologram with the sample object and the same background, would be able to subtract the background noise for a better reconstructed visualization. In addition, more than one background image can be imported, whereas the average is taken for better estimation. Then, “Load Sample” loads the recorded hologram of the sample object, then “Reconstruct” ignites the calculation, a stack of “Reconstructed Images” will be shown in the window, where you can move the slide bar through the stack, as well as zoom in by rolling your mouse.
Essentially the algorithm in HoloPy functions as a digital lens, taking pictures at different depths, or cross-section slicing images of the sample object, which contains the 3D information of the object. Next “Save holograms as”, where all reconstructed images will be named and saved in sequence in the specified folder, clicking now on “Show 3D Object”, a subprocess in python would trigger the software Fiji, passing command arguments converting images to stack, invoking the 3D Viewer plug-in, thus visualizing the stack images as one 3D object.
If the reconstructed images is not yet optimal due to parameters tuning, or when we simply want to compare different reconstructed images, we can “Set Reference Pic”, so that the current displayed image in “Reconstructed Images” window will be shown in the “Reference Reconstruction” window. Further adjustment of the parameters and the resulting updated “Reconstructed Images” will be shown for comparison.

Achievements

After taking pictures using our hardware setup, we ran HoloPyGuy software to reconstruct the sample object images. We tested three different sample objects: microscope resolution slide, bubbles, and blood cells respectively.

A reference image for a laser system.
A reconstructed hologram with HoloPyGuy.
Figure 1. From left to right: average background image, recorded hologram with microscope resolution slide, moving through reconstructed stack images

According to the above figure, HoloPyGuy first takes the average of the import background images, subtracts the average background from the recorded hologram to remove noise, and then computes the reconstructed images. We can see from the reconstruction of the microscope resolution slide, that our hardware setup and HoloPyGuy software solution can resolve up to 2 micrometer resolution. However we also observe some dust within the resolution slide, which can't be cleaned and results in excessive inteference patterns in the reconstructed images. The excessive interference is also observed in the following figure, where the sample object is a bubble.

A reference image for a laser system.
A reconstructed hologram with HoloPyGuy.
Figure 2. From left to right: average background image, recorded hologram with a bubble, moving through reconstructed stack images

In the following figure the sample object are blood cells, however there're too many cells in the field of view, resulting in quite noisy interference patterns in the reconstruction.

A reference image for a laser system.
A raw hologram on the background.
A reconstructed hologram with HoloPyGuy.
Figure 3. From left to right: average image, recorded hologram with blood cells, moving through reconstructed stack images

Multiple objects in the field of view and excessive interference patterns partly destroy the 3D information. Thus with out current hardware setup and alignment, 3D imaging using Fiji is not yet feasible. However, we have successfully tested the sample images from HoloPy documentation in the point source set up. The resampling is set to 1, and threshold set to 74 to remove remaining background noise. The resulting 3D object is shown below:

holopygram
holorecon
3D object testing using HoloPy doc point source sample images
Figure 4. From left to right: the recordered hologram from holopy documentation, moving through the reconstructed images, 3D object in Fiji

Download and further documentation

Setting up your system for using HoloPyGuy

To set it up for running, we follow the getting started section from holopy. If you use a unix based operating system, please run the following lines in command-line:
sudo apt install conda
conda install -c conda-forge holopy

This should set up the production branch of HoloPy. For Windows, the installation process is quite similar. Install Anaconda3 for your OS. Be aware that if you let the box checked in the installer, Anaconda overwirtes the default python interpreter. Then, run the following commands in the anaconda prompt:
anaconda search holopy
anaconda show ralex0/holopy
conda install -c conda-forge emcee
conda install -c conda-forge h5netcdf
conda install --channel https://conda.anaconda.org/ralex0 holopy
Install hologuy requirements (a converter from QImage to NdArray)
conda install -c conda-forge qimage2ndarray
For 3D model support, you have to install the latest version of FiJi from their website. Follow the instructions provided. Then, you have to edit the paths in call_fiji.py and testfiji.py.

Get a copy of HoloPyGuy

Download the latest version from hologuy and extract the files either from our GitHub repo or download the .zip file. To get the latest version, it is recommended to check our git. You can get a copy of our repository by installing git and run in a terminal:
git clone https://www.github.com/iGEMDarmstadt/holopyguy.git

python hologuy.py

Outlook: Future Development

There's still room for improvement in our software solution, as well as in coordination with the hardware setup:

  • As we mentioned earlier in the Achievements section, too many objects in the field of view would introduce excessive interference patterns which can't be effectively removed after reconstruction. This can be solved with a precisely controlled hardware setup, careful selection and alignment such that there's one or only a few sample objects in the field of view.

  • A bug in software solution: adjusting the parameter "Spacing" needs to be prior to importing any pictures to HoloPyGuy, otherwise the calculation would raise memory error. This is due to source code of HoloPy, as 'cam spacing" is not updated in the holopy.core.update_metadata() function. In collimated source setup, "Spacing" is a parameter that doesn't need frequent update. However in point source setup, "Spacing", in relation to "Magnification", can be arbitrarily set for better reconstruction. Thus future improvement can be done in updating the source code for the point source setup.

  • Integrate the pi-cam webserver into the user interface of HoloPyGuy, where an end-to-end process from taking pictures upto 3D imaging can be realized.


References

[1] Shiraki, A., Taniguchi, Y., Shimobaba, T., Masuda, N., Ito, T. (2012) Handheld and low-cost digital holographic microscopy.
arXiv:1211.0336
[2] Cotte, Y., Toy, F., Jourdain, P., Pavillon, N., Boss, D., Magistretti, P., Marquet, P., Depeursinge (2013) Marker-free phase nanoscopy Nature Photonics, 7 (2):113
DOI: 10.1038/nphoton.2012.329
[3] Giuliano, C. B., Zhang, R., Wilson, L. G. (2014) Digital Inline Microscopy (DIHM) of Weakly-scattering Subjects Journal of Visualized Experiments, DOI:10.3791/50488
[4] Molaei, M., Sheng, J. (2014) Imaging bacterial 3D motion using digital inline holographic microscopy and correlation-based de-noising algorithm Optics Express, DOI: 10.1364/OE.22.032119
[5] Braat, J., Dirksen, P., Janssen, A. J. E. M. (2003) Diffractive Read-Out of Optical Discs, Optical Imaging
Springer Verlag
[6] Deng, Y., Chu, D., (2017) Coherence properties of different light sources and their effect on the image sharpness and speckle of holographic displays, Scientific Report,
DOI: 10.1038/s41598-017-06215-x
[7] Jericho, M. H., Kreuzer, H.J., (2011), Point Source Digital In-Line Holographic Microscopy, Chapter 1, Coherent Light Microscopy, Springer Series in Surface Sciences 46, 46
DOI: 10.1007/978-3-642-15813-1_1
[8] Rostykus, M., Moser, C. (2017) Compact lensless off-axis transmission digital holographic microscope, Optics Express, DOI: 10.1364/OE.25.016652
[9] Reichert, C. C., Herkommer, A., Claus, D. (2016) Das Smartphone als Mikroskop, AT-Fachverlag GmbH,
www.biophotonik.de
[10] Moon, I., Daneshpanah, M., Anand, A., Javidi, B. (2011) Cell Identification Computational 3-D Holographic Microscopy, Optics & Photonics, 22 (6),
[11] Greenbaum, A., Luo, W., Su, T., Göröcs, Z., Xue, L., Isikman S., Coskun, A., Mudanyali, O., Ozcan, A. (2012) Imaging without lenses: achievments and remaining challenges of wide-field on-chip microscopy, Nature America, DOI:10.1038/nmeth.2114
[12] beniroquai (2017) Blog, https://beniroquai.wordpress.com/2016/01/20/holoscope-linsenloses-holographisches-mikroskop/, last visited: 10/15/2017
[13] BDan (2015) micromanipulator, Thingiverse, https://www.thingiverse.com/thing:923865/#files, last visited: 10/15/2017
[14] "Do-it-yourself" project for steering HD-DVD pickup homepage: http://www.diyouware.com/ last visited: 10/15/2017
[15] "HoloPy, a python framework for analyzing digital holographs, manoharan lab, Harvard https://github.com/manoharan-lab/holopy last visited: 10/18/2017
[16] "HoloPy, documentation, https://holopy.readthedocs.io/en/latest/users/index.html last visited: 10/18/2017
[17] User:Egelberg, OpticalSetupDHM, CC BY-SA 3.0
[18] Schmid B., Schindelin, J., Cardona, A. et al. (2010) A high-level 3D visualization API for Java and ImageJ, BMC Bioinformatics 11(1): 1, DOI: 10.1038/nmeth.3392
[19] Schindelin, J., Arganda-Carreras, I., Frise, E. et al. (2012) Fiji: an open-source platform for biological-image analysis, Nature methods 9(7): 676-682, DOI: 10.1038/nmeth.2019