Stephen Deems, PI for the NSF-funded ACCESS Allocations at PSC
I sat down recently with Stephen Deems, the PI for the NSF-funded ACCESS Allocations program, to chat about his recent Mellon College of Science Outstanding Achievement Award and his time at PSC.
ACCESS ALLOCATIONS
CC Stephen, congratulations on your 13 years here at PSC and your recent award. You’ve held several jobs here at the Center. Thinking back to your work in allocations and what you’re doing now with ACCESS, how would you compare them?
SD Well, it’s like growing up in a family business. If your parents had a grocery store, you might start by stocking the shelves, then move to some other duties: cashier, ordering, accounting, etc. I feel like I’ve certainly had a career arc with learning more of the processes important to the HPC community. [Such as] what types of work are done on large-scale computers? What methods do they use? Is it memory- or IO-intensive? Is it GPU- or MPI-enabled? Over time, I built a list of items and key terms that I had to research on my own. After understanding some of those topics I moved on to thinking about infrastructure: Why do fluid dynamics people want to run here, but AI projects run here, and bioinformaticians run here? Then connecting all of it together with the people who make it all possible.
There’s a cycle in the allocations process. You come in, you get something, you consume it, renew it, extend it. What I liked early on is that most of the activity is a pretty finite box. There’s only so many things you can do: I need a new one, I need replenishment, I need to try another machine, I need more time. That stuff was easy, to me. It’s like cooking: there’s a bunch of different spices and ingredients and people use them in different ways. That’s when I started to get hooked. I get to work with so many different people. From working in the allocations process to now leading a project that carries out allocations for the national community, it’s akin to that family business analogy. Eventually you’re at the helm, but you never think you’re ready to take over. When Shawn Brown, Director of PSC at the time, took a new opportunity, it was more like, “here you go, whether or not you’re ready.” We had just received approval of the $7.5 million grant that he was planning to drop in my lap [laughs].
CC In addition to the main $7.5 million ACCESS award, PSC also recently received $3.5 million in supplemental funding. How did that come about?
SD Without the team that I’m a part of, nothing would have happened in the first place. If it weren’t for the support of our NSF program officers who work so diligently with us, we may not have been able to expand our scope of work. Our team had taken on a significant amount of additional tasks to better serve the research and educational community along with our other peers. NSF and the reviewers of our supplemental request recognized that we went above and beyond and that we were delivering on what we had initially promised and more. The request was to get more staffing to better serve the communities and cover some of those things that we were doing without further financial assistance. There was also talk about some emerging programs at NSF. One is the National Discovery Cloud for Climate (NDC-C) research. The aim was to take resources that already exist in the NSF community and unite them together toward a climate research focus. How can we integrate climate software sets and data tools, domain expertise, and maybe some resources that aren’t directly available through ACCESS? How can we pull all of that into a catalog of rich resources?
On top of that effort, the National Artificial Intelligence Research Resource (NAIRR) Final Report was recently released and had similar aims but for AI. In October 2023 an Executive Order was released from the White House on safe and trustworthy AI that ordered agencies to rapidly respond. A pilot program to support the allocation of AI resources to U.S.-based researchers and educators was one of the orders. The software that our team develops under ACCESS Allocations is now being utilized to accept proposals through the NAIRR Pilot Program, review them, and administer creation of accounts. The NAIRR pilot will be a big focus for us and the nation’s AI research and educational communities in the coming years. We’re essentially providing one software solution and customizing it for multiple programs.
EQUITABLE ACCESS TO HPC
CC It must be gratifying that they recognized your efforts and approach with the extra funding?
SD It was a great feeling when the email notification came through. There’s been a lot of growing pains I’ve had to deal with in an extremely short amount of time and that notification gave me a lot of assurance in myself and our team’s efforts. The hardest thing that I’ve done in my professional life is being the principal investigator (PI) of this grant. I owe so much gratitude to Ken Hackworth, my first boss and mentor at PSC, and Dave Hart, who led this area during the previous program and continues to be a leader I look up to and learn from. Dave’s a co-PI on ACCESS Allocations and is located at the National Center for Atmospheric Research (NCAR). He is the architect of our software. He’s had the vision the whole time and our development team has been building and refining the software for over ten years. The reality is Ken and Dave used to be my bosses. I wouldn’t be here without those two guys. They imparted a lot of skills and knowledge to me and Shawn saw that I would be up for the task.
I’d like to clarify that Shawn didn’t necessarily force my hand. I thought to myself, “who’s going to approve a kid with a degree in Marketing to be the lead of a $7.5M federal grant?” It turns out, Carnegie Mellon University and the National Science Foundation were alright with it [ha ha]. So, it’s been a wild past few years, and it’s brought me to tears at times because I was so unsure of whether I was doing a good job or not. I looked to these people in the past for guidance and now they look to me for input. That’s a lot of change to go through quickly.
CC It sounds like a lot of people have confidence in the job you’re doing. You are also a big promoter of equitable access to HPC resources. How do you promote and ensure that?
SD Let’s start with a look at Pitt and CMU. The institutions we’re a part of are very well-resourced universities and students that go here have incredible opportunities and likely already have backgrounds well-suited for their pursuits. When you shift the lens away from major research institutions to community colleges, minority serving institutions, or tribal nations, you rapidly realize that we take things for granted here on our campuses and in our respective lives. We complain about our Internet if it drops, about the dining options near our dorms, or lack of certain services. Well some institutions don’t even have broadband connections. Large portions of their student population commute to campus, work full time, raise families, and have many more obstacles to navigate in life.
You have to start thinking about “Where are the gaps? How can we help fill those gaps with ACCESS?” The simplest thing that we did was change the model we were running for the allocations process in the previous program. If you had a need for a modest amount of resources you needed to craft a 10-page proposal and get past a diligent review panel. Now, you are asked to provide us with three to four sentences describing what kind of activity you’re doing; a little bit about your discipline; whether you know some code and software that you want to use; and if you know what types of resources you want to run on. This makes it much easier for people to get in the door and get on the system.
CMU ROBOTICS INSTITUE PARTNERSHIP
CC You also helped to broker a partnership between PSC and the CMU Robotics Institute?
SD This was an effort I’m extremely proud of. I had been working with this group for a while: getting them on the systems, helping them write those 10-page proposals I was talking about. The Air Lab, run by Dr. Sebastian (“Basti”) Scherer, works on a number of initiatives related to autonomous aerial, ground, and subterranean vehicles. They’re running 4X4 ATVs, completely autonomously, through the woods at crazy speeds. At one point the group wanted to procure a node or two and have PSC operate them. I explained to them what costs associated with those services would be and it turned out to be cost prohibitive. A few months later Basti identified a solicitation to pay for hardware and we crafted a successful pitch.
What I brought to the collaboration was thinking differently about our approach. I think like a business guy, I wanted to pitch a proposal to the sponsor that they couldn’t turn down. It’s been terrific working with Basti and his lab. They’re kind, brilliant, and thorough researchers. PSC had never done a collaboration with the Computer Science department before. It’s been a really smooth collaboration, and I’m hoping to brew up some larger opportunities. Those are the types of projects we need to pursue; we’re stronger together. It’s still a bit weird that I started out answering the phones at PSC and ultimately crafted the first collaboration with the Computer Science department in the center’s 37 year history.
With our new Director Barr von Ohesen, and some new staff coming on at CMU, we’re working on some higher-level vision projects to really bring the campus together. But we still have to offer services if people want to operate their own stuff. In general we all have a lot of opportunities for growth.
THE PSC EXPERIENCE
CC Thinking about your time at PSC, what kinds of things stand out in your mind the most?
SD It’s hard to not answer in a cliche. It’s not the hardware, not the science. It’s the people. At the Awards ceremony we saw PSC staff with 30, 35, 40 years of experience. It brings a lot of good feelings to our staff. I don’t mean to switch to negativity, but I’ve been here 13 years as a full-time employee and it wasn’t always so sweet. There was a lot of pain, uncertainty, and questioning. Am I doing the best work that I can do? Am I in the right space? A lot of those years were spent wondering about that.
I will always praise the wonderful staff I’ve been able to work with, but the truth is you’re going to encounter people that aren’t easy to work with. Sometimes you’re stuck, in a sense. I’m fortunate that most people have been stellar humans. I haven’t had the sweetest time here, there were a couple of years that I was not in good mental health. Ultimately the challenges I encountered kept me coming back. Things like, I don’t understand these acronyms or these terms. I felt stupid asking some questions. You have to push yourself through that. The people who have been here 30 or 40 years have done so by persevering. They have gone from project to project through uncertainty in funding. Other departments we work with aren’t always familiar with working on “soft money” that isn’t backed by tuition dollars.
I’ve never had another opportunity where I’ve been able to learn as much or be around such brilliant people. I aspire to work with others who make me love what I’m doing even more. I never thought I’d be here, but I’m so glad I am.
CC That’s a great observation. If you talk to any of those people up on that stage with more than 10 years at PSC, I’d wager that they all have stories like that they could share.
SD I used to be a cook, and people would ask, where are all your burns and scars? Well, I actively tried to avoid doing that, ha ha. Cooks don’t seek it out, but it does happen. Everybody has their stories – their burns and scars.
DO TELL!
CC What is something about you that isn’t generally known that you’d be willing to share?
SD Secretly, [well, not any more I suppose] I’m a nervous wreck. I try to channel it into productivity. It doesn’t always work, and sometimes it really does consume me. I care too much about people around me and the work, and it’s been a struggle of mine because as a young investigator you feel way more pressure. I may come across externally as very confident, fun, and energetic, but on the inside I’ve been really nervous, but I try to channel it into positive things.
Shawn was a big advocate for me, but he was also the one to say, “you need to advocate for yourself because I’m not always going to be here”. And then he wasn’t. I’ve always felt like an outsider here because of my background. Now is the time when I should feel like I finally belong here. But I think as soon as you feel that way, you start down a bad path. I struggle a lot with “what do I do next?” and for the past 18 months, I was doing something every day that I had never done before. Every day I showed up something would pop into my inbox, or the business office would ask me about a particular thing, or they’d ask me why I didn’t do something. Sometimes the answer was,”I had no clue I was supposed to do that,” or “I’ve never done this before,” but our staff would take the time to walk me through the steps.
I care for the people that work here and even the users that I’ll likely never get to meet. I come across as a people person, I’m fortunate to have good people skills. I thank my Mom for everything I have in my life. When I received that award from MCS it gave me some assurance that I’m doing well in my professional roles.
CC You hide your anxiety very well. Part of that comes from the nature of being in the position that you’re in, as a young person, I mean, that’s a lot of responsibility in some ways.
SD I think some of that comes with the territory. Other people who are in my position as a PI in the ACCESS program, most of them are 15+ years my senior and they’ve already made their marks in their career. They don’t have to prove much. Meanwhile, I’m in a position where I never set out to prove anything. I’ve joked that I always wanted to be in a rock band, but I didn’t want to play lead guitar or sing – I wanted to hide behind the drum set or play rhythm guitar and be a part of it, but not in the spotlight. When I used to cook for a living, I liked being in the back of the house. I didn’t want to go out and talk to people at tables and ask about how their wine was tasting that night. I want to be a part of the system but shy away from the spotlight.
Sometimes I think, “well here I am, get used to it,” but the truth is it’s taking me a while. Thanks to great people like Tom, Ken H., Robin, Megan, Ken G., Kapcin, Dave Moses, Erica, and so many others that have been there for me. No one’s left me out to dry and I couldn’t do it without them. Sometimes when I talk at PSC meetings and thank people for their efforts I have to hold back a lot of emotions because I couldn’t do anything without all these folks. There is no more grant money to pursue if there’s no Rob Light or Matt Yoder churning out software features. We work with so many brilliant people and some are similar to me in the sense that they don’t want attention, but they like what they do. That can be an infectious feeling. And I’m just elated to be here and excited for what’s next.
CC I really appreciate you taking the time to sit down and talk to me. Thank you!
Learn more about ACCESS Allocations.