Running DeepCell on Cluster GPU

Setting up the environment (you should only need to do this once)

1. If you don’t already have it I recommend you download cyberduck (https://cyberduck.io) to easily interface with the cluster
2. Once cyberduck is installed, open it, hit open connection, in the top drop-down menu select "SFTP(SSH File Transfer Protocol)", for server type in "mercury.pmacs.upenn.edu", enter your PMACS username and password, and hit connect.
3. Create a file in your home directory on the cluster Called "DeepCell" (in cyberduck your home directory should appear as "/home/" in the dropdown menu)
4. Download all the files in this GitHub repository by hitting the green "Code" button above on this page, and select "Download ZIP"
5. Drag and drop the downloaded file "DeepCellHelper-main" into the "DeepCell" folder created in step 3
6. Change the file name from "DeepCellHelper-main" to simply "DeepCellHelper"
7. To set up the virtual environment containing the necessary dependencies, open up terminal and type in ssh <username>@consign.pmacs.upenn.edu replacing <username> with your PMACS username, and hit enter. Then enter your PMACS password and hit enter
8. You should now be in a session on the terminal.Enter bsub -Is bash to start up a node
9. We will now make sure we are in the correct python version by entering the command module load python/3.6.3. If this worked you should be able to enter python --version and see the output Python 3.6.3
10. Enter cd /home/<username>/DeepCell/ replacing <username> with your PMACS username. (tip: to get paths of folders I usually right click on them in cyberduck, select "copy URL", and select either choice, and keep everything after ".edu")
11. Now set up your virtual environment by entering the line python -m virtualenv DeepCellEnv
12. Next activate your virtual environment by entering source /home/<username>/DeepCell/DeepCellEnv/bin/activate, replacing with your PMACS username
13. We can now install the necessary packages in this environment by running: pip install -r /home/<username>/DeepCell/DeepCellHelper/DeepCellRequirements.txt, replacing <username> with your PMACS username
14. Your virtual environment is now ready. If you ever want to leave the virtual environment simply enter deactivate, every time you want to active the environment enter: source /home/<username>/DeepCell/DeepCellEnv/bin/activate
15. To use the GPU on the cluster we also need to get some software from NVIDIA called cuDNN. Specifically we need to get cuDNN version 8.2 which can be downloaded here: https://developer.nvidia.com/cudnn
16. Once downloaded place the folder in the "DeepCell" folder on the cluster. For the files provided here to run with the least amount of modifications this file should be named "cuda"
17. at this point you should have all the files you need to run DeepCell, and your directory should look like this:

 /home/gharm/DeepCell
 ├── cuda
 │   ├── include
 │   ├── lib64
 │   └── NVIDIA_SLA_cuDNN_Support.txt
 ├── DeepCellEnv
 │   ├── bin
 │   ├── etc
 │   ├── include
 │   ├── lib
 │   ├── lib64 -> lib
 │   ├── pip-selfcheck.json
 │   └── share
 └── DeepCellHelper
     ├── Cyto_Nuc_DeepCell.py
     ├── dcHelper.py
     ├── DeepCellRequirements.txt
     ├── DownsizeScan.py
     ├── README.md
     ├── RunDeepCell.py
     └── SubmitDeepCellJob.sh>

Running DeepCell

1. The main file that will be edited each time you run DeepCell is "RunDeepcell.py" which can be found in the DeepCellHelper folder. This file contains the main parameters to change during your run. I suggest you make a copy of this file and save it with the parameters you used each time to maintain a record of what was run.
2. The first parameter is entering what file to run your analysis on. The input here has to be a tif file. You can either enter a path directly to a tif file, or use glob to return a list of tif files you would like to be run together with the same parameters.
3. For the NucleusChannel enter the channel that contains the nuclear stain you would like DeepCell to use to segment the nucleus. If you don't want to segment the nucleus and/or you have no nuclear stain enter 0 here.
4. For the CytoplasmChannel enter the channel that contains the cytoplasm stain you would like DeepCell to use to segment the cytoplasm. If you don't want to segment the cytoplasm and/or you have no cytoplasm stain enter 0 here.
5. For the objective enter the magnification you used to take the images being segmented. The purpose of this parameter is to give the model information about um/pixel. Therefor if you used the Shaffer lab scope you can just enter the objective and the correct um/pixel value will be chosen. If you are using another scope you will need to go into the Cyto_Nuc_DeepCell.py file and set the correct um/pixel value for each objective.
6. For the type either enter TC or tissue. Depending on what you enter here it will select model that was trained on cells plated in a dish or cells from tissue slices respectively.
7. Finally you will need to change my user name to your username in the path on line 24 so that the script can find the RunDeepCell function. Save all these changes and close out of the file!
8. Now open up the File SubmitDeepCellJob.sh. Here you will need to adjust all the paths so that they point to the correct files.
9. For line 12 you should only need to change my username to yours if your directory is set up the same as mine, and that is also true for line 14
10. On line 14 enter the path to the "RunDeepcell.py" file you just finished editing in step 7 above. Save the file and close out.
11. You can now run DeepCell by running: bsub < /path/to/SubmitDeepCellJob.sh (NOTE: you must be in an active DeepCell virtual environment when you run this)
12. If everything works properly the output should be tif files containing masks of the nuclei and/or cytoplasms depending on what you specified