This is the second installment of Research Revelations: Conversations with Our Student Researchers, a podcast where Aquila staff members talk to student researchers about their projects and research goals. In this episode, Aquila reporters Selina and Lindsey meet with Nidhya Shivakumar (12) to discuss her project using neural networks to categorize damage from climate change.

Lindsey: Hi, I’m Lindsey.

Selina: I’m Selina.

Lindsey: And welcome to Research Revelations: Conversations with our student researchers.

Selina: Today we’re with senior Nidhya Shivakumar to discuss her research on climate change and assessing damage to buildings using neural networks.

Selina: Thank you so much, Nidhya, for coming to join us. So our first question is just how long have you been doing research, and how did you get started?

Nidhya: So I’ve been doing research since sixth grade, and I kind of got started by reading some magazines. And I just saw some interesting facts there and I thought I could research them. That’s how I got started.

Lindsey: What did you see in the magazines that inspired you to do the research?

Nidhya: So actually, my first project, this was in sixth grade, was looking at plants that are able to grow in very salty areas. It was kind of looking at how that could be used to promote agriculture, so I saw the effects of the wastelands on the environment, so I kind of researched on that.

Selina: And then could you just talk in general, what is your research about and kind of explaining it to someone who has no background in research?

Nidhya: Sure. So currently, climate change, has made disasters a lot more powerful, and as a result, those disasters affect a large area of land. But only some places in that large area are really affected by the disasters, and that’s where there’s high populations, where buildings and infrastructure is damaged. That’s where really the first responders would need to go immediately to get assistance. And right now, that’s pretty hard, it’s pretty hard for a first responder to determine where in that disaster it to go to. So my project is using drone footage that’s taken right after a disaster, and I trained a neural network to basically predict buildings in those images, and also classify their damage. And in addition, actually putting roads in those images to see if those areas are accessible by using like a fire truck or something like that.

Lindsey: Can you describe your procedure for this project?

Nidhya: Yeah. So with this project, I first found my data set, which basically consisted of a lot of low-altitude disaster images taken from drones. And then, after I found that data set, I labeled a few of those images and then I actually used a technique called weak supervision. Basically, I trained a model on those few images and then I had that model produce more training data for me for later stuff. So in that data set, all the images were labeled as either there’s damage here, or there’s no damage here, and also the type of damage. So based on that, I trained the model, and then I labeled each of the buildings in the image with that label. So after I had my training data, I first started out small just by predicting either the building is damaged or on damaged in the image.

Nidhya: After that, I moved on to classifying different types of damage. I focused on mainly three types. Undamaged, rubble and flood damage. Those are the most common types of damage you see in really any disaster. And then I did a few things to try to improve the accuracy of the model. I actually also took drone footage from one of the tornado disasters in Kentucky, which was in 2021, that was just from a local news station. And I actually cut up that video into frames, and I predicted on each frame, and I searched the video back together. So basically, that produced like an annotated video of that whole town. And actually, if you look in the video, you’ll see the path that the tornado takes through the town because the buildings in its path would be damaged, and the buildings not in its path, would be undamaged.

Selina: And in general, how long does this whole process take you?

Nidhya: So I started this project in last year, around August or September, and then the preliminary results, were probably February or March. And now I’m expanding on it, slowly, just improving it little by little.

Selina: Yeah. And then could you also talk about just what inspired you to kind of focus on this topic of research? Why do you think this field is so crucial?

Nidhya: Well, climate change is affecting all of us, no matter where you live, and I think a lot of first responders do have difficulty locating those specific places that are particularly damaged and need immediate assistance. So I think that’s kind of how I got started and with this project and then I kept on going and found the data set.

Lindsey: And so what’s your favorite part of the research process in general?

Nidhya: I think my favorite part would probably be getting those first results back. That kind of is an indication that, okay, this project has some merit, it has the potential to go on and become something that’s really useful. So I think that was probably my favorite moment.

Selina: Yeah. And on the flip side, I guess what would you say was the most challenging part of doing this research?

Nidhya: The most challenging part, probably was getting all of that training data, because even though I labeled a small part of the data, it took quite a while and I knew I would need a lot more data in order to actually train a good model to predict the building’s themselves, and also their damage status. So I think just trying to figure out how to efficiently make a lot more training data, that was the hardest part.

Lindsey: Can you describe how you felt when you got your initial results back?

Nidhya: Yeah, so like I said, I felt like that was the moment when I felt like this project could actually be used in the future for first responders to actually use some sort of annotated video or pictures to see where they should go first and help out and provide their system.

Selina: Yeah, and would you say there were any like surprising or unexpected results?

Nidhya: One unexpected result would be some of the images where there was a flooded area, it was actually pretty difficult to predict the buildings, and especially the roads in those areas, because some flooded roads from above look kind of like normal roads, especially if you convert the image to black and white. So I think that was something that was surprising when it predicted like a flooded road as just a normal road, so that’s actually also one of my challenges.

Selina: Oh, yeah. And what would you say sets this kind of research apart from like other projects you’ve done or other types of research projects on climate change?

Nidhya: I think this project is kind of useful, in the here and now. Because these disasters, they come and go, like recently Hurrican Ian in Florida. Like, this is something that’s going to be affecting us, and it’s affecting us right now, and it’s going to be affecting us a lot more in the future, especially as climate change increases. So I think, just the usefulness of it right now, is something that sets it apart.

Lindsey: What’s the most important thing you learned from research in general?

Nidhya: Yeah, so the most important thing was that I learned to actually make the computer help me do work. So I kind of learned techniques to have the computer, like I said, with the weak supervision, have it label data for me, training models and things like that. I think just making the computer work for me was one of the biggest things I learned.

Selina: And what would you say is the most important thing that you’ve learned from doing this project and this research specifically?

Nidhya: Yeah, I guess from this project. There’s a lot of minor details that if they’re overlooked, it kind of propagates into a big error. So I think just focusing on the minor details was something that I learned from this project.

Selina: Yeah. What kinds of minor details do you mean?

Nidhya: For example, with the flooding, flooded roads, I mentioned, if I had overlooked the flooded roads, and just had it predict a normal road, that would propagate into quite a bit of error. And especially if that got into the hands of a first responder, and they would say that, okay, this route is usable, but then they get there, and it’s actually flooded. That would be pretty disastrous. So I think those minor details are definitely important.

Selina: Yeah, definitely.

Lindsey: Do you want to continue researching this topic in the future?

Nidhya: Yeah, definitely. So some things I want to do is actually implement this hardware onboard a drone so that you can provide real time feedback to a first responder. I’d also probably like to — this was actually part of the onboard drone, but map the images to actual GPS coordinates, because that would make it a lot easier for first responders to know where to go when they receive the image where there’s a lot of damaged buildings. So I think those are some of the directions I want to take in the future.

Selina: This is kind of a similar question, but do you think there’s any improvements that you could have made from the research that you’ve done so far?

Nidhya: Yeah, I think mainly, it’s the usefulness, or the practicality of it. Right now, just feeding images and having it output images, that works. But in a real disaster situation, it may not be the most efficient. So I think having a first responder — I just kind of have to place myself in the situation of a first responder and think what would they need? So I think that’s a direction that I’m looking to continue in.

Selina: Yeah, for sure.

Lindsey: You mentioned that you wanted to make your drones with the program on board so that you could have a real time video of it, what kind of steps which you need to take in order to achieve this?

Nidhya: Yes, so I haven’t really looked into that yet. But I would have to find some sort of hardware that could, that has the capability of storing a model in it. And I would have to take in the pictures from the drone and feed it directly into that hardware, and into the model. And the model should be able to output and send to maybe a first responder on the ground. So there’s a lot of things that need to be done.

Selina: I know that your research is very thorough, and it requires like a very meticulous process. So do you have any tips for student researchers who are maybe curious about research, or want to start a project but don’t really know where to start?

Nidhya: Yeah, I think one main advice that I would give is focus on problems that are plaguing society currently, because there’s going to be a lot of information about those already. And you can sort of take that and take the information that’s already there, and then you can formulate a question, and hypothesis and so on.

Selina: Yeah. And then I think you mentioned before that, like one of the challenges was that was just accumulating all of the data. So could you talk about how you kind of come back from facing any challenge in your research project?

Nidhya: Well, actually, the most important thing is to step away from it for a little bit. Because I know that in the back of my mind, I’m still thinking about it. So I think that definitely helps. But I guess in general, facing a problem in research is definitely very normal, and if you’re facing that problem, it’s more than likely that other people have faced a problem. So one thing that I recommend is just Googling any problems you have, and they’re likely going to be solutions for them.

Selina: Yeah, that’s really good advice. All right. I think that’s it. So yeah, thank you so much, again for joining us. Yeah, this is really fascinating.

Selina: Thank you for listening to today’s episode of Research Revelations: Conversations with our student researchers. We hope you enjoyed hearing about Nidhya Shivakumar’s project on categorizing building damage with neural networks.

Lindsey: If you are a student researcher and would like to be featured next, please feel free to email us at [email protected].

Selina: This is Selina.

Lindsey: This is Lindsey.

Selina: And we’ll see you next time.