Career Paths for Data Scientists, Data Infrastructure to produce results

We spoke to Stefania Bonà, Lead Product Manager at Trustly, on the challenges associated with hiring talent, diversity in data teams, preferred career paths for data scientists, and the infrastructure that needs to be in place to produce successful results.

Quotations have been lightly edited for concision and readability.

Give us your thoughts on this issue:

What are the challenges that you find yourself facing on a day to day basis In your role as a data science leader?

“So bear in mind that I'm still working in a product role which is really data science and machine learning engineering and back end focused. I would say that in general now there's a huge buzz going on how everyone wants to do AI, especially after ChatGPT. And to be honest, the companies that can do that at the same scale, you can count them on 5 fingers. It's Google, it's Open AI, it's Microsoft. It's Amazon and Netflix. So everyone wants to jump on the wagon and do AI”

Adopting AI can be an expensive proposition.

“But, it's really hard and super expensive to have machine learning at those levels. So that's definitely a bit of a hangover going on there.”

Hiring (and making the case to hire talent) is also a challenge:

“The challenges are really putting together the right team. Data science recruitment has gotten easier in the past few years, but where [I'm] struggling with is having the right talent for machine learning engineers. And then really trying to explain to leadership that we need both. We need really good data engineers, and we need really good data quality. Good machine learning outputs, so it's not just getting data science and then getting it done. It's the entire infrastructure [required] around data products that is sometimes a hard message and an expensive one to get to leadership.”

What are the major expenses you see when it comes to adopting AI?

“It depends what you're trying to do, because you can always buy machine learning as a service, whether you are scoring transactions for fraud or you need a classifier to classify images. Whatever you're trying to do, you can always buy it because there might be, and usually is, someone else out there doing that for you.”

Building a proprietary solution can be costly:

“But if you want to have your proprietary in-house solution, that expenses are on the cloud of course, because the computing power is costly, the infrastructure, but also people, like the actual headcount is very expensive especially for data engineering and machine learning engineering roles. So it's not cheap.”

What do (executives) they look for from you to actually make the decision to hire people?

“[I wouldn’t say exactly,] but data scientists have gotten easier because you know, there's the buzz around it. But they're also many courses where people are carrying out learning exchanges they can go to and learn how to do data science.”

On the most difficult roles to fill:

“The hardest hires are, I would say machine learning engineers because there are not many out there and there are not many out there that have [experience] creating a startup and scaling it’

It’s hard to find qualified machine learning engineers at an affordable price, and even harder to find diverse candidates:

“And of course, all the major companies [like Facebook] are the ones that can pay those salaries. So there are really not many around and then when it comes to diversity, even less, because it's usually all male machine learning engineers and data scientists. Again, it's getting better, especially in London that I'm familiar with the landscape, but machine learning engineers [are] super hard to recruit and there are not many women around doing that.”

In your personal experience, why is it difficult to find women in STEM and data science and machine learning specifically?

“To me it really comes [down] to your childhood. Like, what have you been exposed to? And then, making that choice that you can do whatever you want. It's not that there's a female or male subject. It's really [just about] breaking that pattern. But it's also true that if you look at universities and those diversity stats, they're probably the same, that there are not many girls going into STEM and then you have even less in the workforce.”

A survey by Boston Consulting Group found that only 15% — 22% of data scientists are women and only 18% of women in the field hold leadership positions at tech companies. Things seem to be changing for the better, but there is still a big divide to cover:

“Those things are changing, of course, in the right direction, but there's still a huge gap that I feel goes back to what you're presented as a child. How you're growing and then opening up those possibilities.”

How do you make sure that your data scientists are growing in their career? What do you measure in terms of their progress?

“So at Checkout.com where I used to work, I sat down with the data science manager and we pretty much created those competency profiles. Making sure that you need it is [sometimes] very hard in a startup. [But if] you create those competency profiles, then you have at least two career paths.”

The responsibilities in these two broad career paths may overlap:

“One is an individual contributor, one is the management path and then your responsibilities would just grow in scope for an individual contributor role. I believe that if you want to get to Principle, you would pretty much almost have to create your own algorithm. It gets very much into super technical, while the management one it's more about ‘Yes, you have the technical knowledge, yes, you can review pull requests,’ but also there's soft skills of mentoring and managing others.”

Do you find that people tend towards one over the other in terms of an individual contributor or manager role?

“Just from my experience, I would say there's a tendency towards the individual contributor role [for data scientists]. [Depending on] the company direction as well, along with the IC, then the management path [may] become more appealing, but it definitely depends on the individual goals of course. But usually, yeah, I've seen the individual contributor being preferred to management.”

Data scientists may be naturally drawn to developing their skills in in individual contributor roles:

“Both have their appeal, but I’ve just found that people that work within the engineering or data scientist world where we really like their coding and data and solving those problems instead of managing other people or spending time mentoring someone. But again, it's just a shift of perspective and maybe comes also with maturity.”

A common refrain heard in the data science space is that there are big expectations for data science, without the resources to back it up. What is your experience with that?

“[I think] especially because data science and machine learning, it's a relatively new discipline, people, especially in executive leadership roles think that OK, bring in a data scientist, they're going to do their magic and we're going to solve everything.”

There need to be a lot of key things in place to support data science science efforts and improve the chances of success:

“All the problems we have, it's very far from that and there needs to be an understanding that, yes, data science is going to do the work as long as you have everything else, which is not only enough data, it has to be of good data quality. You have to have a data infrastructure. We have to have a way to retrain and redeploy. There has to be so much in place that if you have a data scientist only and everything else is lacking, they're going to be bored and leave the company. Also, you won't be able to achieve your goals. Yeah, it's a story that I heard so many times unfortunately.”

If your data team has everything they need to get started and get insights, how likely is it that the insights they get will be actionable? What other factors are in play to actually get successful results from data science projects?

“I think having data product managers is fundamental because sometimes you can give projects to data scientists and they can do amazing stuff, but then what? Is this actually helping us achieve this goal or not? Having the product figure in between that understands enough data science but also knows what are the company goals where we're going [is necessary].”

While in the past, there haven’t been many data product managers around to guide teams, she notes that things have changed.

“Nowadays there's a lot, at least many companies looking for someone, because data science tests alone or machine learning engineering alone is good, but then you still need someone to give guidance and the vision. And that's where product comes in, making sure that it's not just research and experiments, but it's something that can be used. The two go hand in hand.”

You mentioned that data science might get bored if they don't have anything to do. How do you keep your team motivated?

“First thing, making sure they're set up for success, which is having the right access to the right stack. Career wise, having a good and clear career path ahead of them. [Also] but this is, at least in my opinion, true for every role, making sure that they can look back like ‘OK, this is how you contributed to reaching this goal. This is thanks to the work we did as a team.’”

Data scientists need to know whether their efforts are leading to real results for the business.

“Just making sure to give feedback that is not just [about] time spent coding or creating models, but we are actually achieving tangible results and showing those results. So having that feedback loop that I have seen in some companies is missing.”

You also touched on soft skills for data scientists who are going into the management path. Is it also important for people [who aren’t]?

[Stakeholder management is] another one that is fundamental even if you are an individual contributor. [If] you are looped into a meeting, especially the more senior you go, the more you have to be able to not speak scientifically, but making sure that everyone that it's not coming from your same background understands what you're talking about, that's definitely true as a manager, but also [in] an individual contributor role.”

Soft skills also come into play when being a mentor to teammates:

[And mentoring,] even if you are a principal data scientist, making sure you can still share the knowledge, still being a team player, and can still make someone in your team grow within the next step even if you don't have direct management responsibilities. I think those two are really, really important.”

When you're hiring someone, which do you give more importance to (soft or technical skills)? Which do you think would be easier to develop once after you hire?

“Yeah, that's a good question. It depends on the role you're looking for. So [in some cases] you might swing on one instead of the other, but it really depends on which role you have that is open. You can always come in at an entry level and develop both.”

See you next week.

Do you have a unique perspective on developing and managing data science and AI talent? We want to hear from you! Reach out to us by replying to this email.