Learning to use AI to detect cancer


Recently, I was given an amazing opportunity to work on a project in biomedical image analytics in collaboration with a large university medical center. The goal of the project was to develop a computer vision system that identifies tumors in CT scans of livers.

I have always loved applying technology to meaningful causes, especially in the medical imaging space. It’s mind-blowing and inspiring to me that the software I build could play even a small part in saving someone’s life.

I worked on this project in the Artificial Intelligence and Machine Learning Division of SAS R&D. Previously, I built a skin cancer assessment tool using statistical modeling and image processing. I’ve also developed an automated add-in builder for JMP Scripting Language and R. My knowledge from these projects carried through to this project where I built an end-to-end, automated data pipeline for liver tumor segmentation in CT scans with deep learning and computer vision.

I’ve documented the entire project in a series of blog posts that will publish over the next few weeks. First, I’ll tell you about my research for the project, and then you’ll learn how I applied that research to the final project.

Getting started with AI

I spent the first few weeks of this project getting familiar with SAS® Viya®, the constraints and the goals of the project, the capabilities of SAS Cloud Analytics Services (CAS), the current literature in medical image segmentation, and different convolutional neural network (CNN) architectures and their strengths. I also spent time developing a plan of attack for the project, running various scripts and toy neural networks, and testing those on my skin cancer data from my previous project. Finally, I  installed the necessary python packages and started testing their capabilities too.

I became instantly interested in watching lectures on the internal workings of CNNs and loved delving into the literature on medical image segmentation. I also learned a lot by observing how neural networks assessed the skin cancer image data so differently from how I had handled it in the skin cancer project using statistical modeling. Rather than implementing programs for extracting features myself, I handed the reins over to the network and fed it the best data and parameters I could to let it do its work.

Right off the bat, I began to understand the AI hype and knew I was working with some powerful tools.

During this process, my mentor was indispensable. He showed me the ropes, from the nitty-gritty details to the overview of current CAS capabilities and the most recent literature in the image segmentation space. From sending me sample scripts for starting a server to pointing me to the necessary online resources for syntax and available action sets, he helped me through navigating the learning curve that comes with complex software and database systems.

Before we get any further, it’s a good time to introduce the entire data pipeline that I ended up implementing on a high level. Below, you’ll find a flow diagram of the pipeline I created for taking in a raw input image (straight out of the CT scanner) and creating a 3D black-and-white segmentation mask that identifies the location of lesions in that scan.

medical image segmentation project flow
Medical image segmentation project flow for detecting tumors in CT scans of livers.

If you’re excited to learn more about the details in that diagram, check back soon to learn how we applied what I learned and redefined the plan for the project.

Image Citations

[1] HellerhoffSarkoidose der Milz CT axial, Extended background for square shape, added into flow diagram in entirety and in cropped and distorted pieces to illustrate various data augmentation steps of project pipeline by Julia Gong, CC BY-SA 3.0
[2] HellerhoffLeberabszess - CT axial PV, Extended background for square shape, added into flow diagram to illustrate project pipeline by Julia Gong, CC BY-SA 3.0

About Author

Julia Gong

Julia Gong is a sophomore at Stanford University majoring in Mathematical and Computational Science and minoring in Linguistics. She started at SAS in the summer of 2016, when she created a skin cancer detection software at JMP using image analysis and statistical modeling techniques. In the summer of 2017, she used JMP Scripting Language to build an interactive custom R add-in builder for JMP. In the summer of 2018, she built an end-to-end, automated data pipeline for liver tumor segmentation in 3D CT scans using deep learning and computer vision for biomedical image analytics in SAS Viya and CAS. She has been recognized in international technology competitions, loves public speaking and puns, and enjoys seeking novel solutions in overlooked intersections of AI, machine learning, language, environmental conservation, medicine, service, and art. Julia hopes to pursue a career that unites her many interests in computer vision, artificial intelligence, medicine, natural language processing, social good, education, and sustainability.

Related Posts

Back to Top