Speaker 1: 0:25

Welcome to this brand new episode of TechScene Talks On Tour. I'm at the offices of SAS Institute, or should I call it SAS nowadays?

Speaker 2: 0:33

SAS is great.

Speaker 1: 0:35

And I'm here with Reggie Townsend. You're the VP Data Ethics at SAS, that is correct. So what does that mean?

Speaker 2: 0:45

Great question. I'm still trying to figure it out myself. The way I describe it is that we are the part of the company that is really focused on where our technology intersects with society. As a matter of shorthand, I tell people, my job is to make sure that, wherever our software shows up, that we don't hurt people or, moreover, that we make sure that when our software shows up, we help people to thrive, and that's probably a better way of kind of thinking about it. We are a team that was formed about four years ago in the company with the mandate to make sure that we had global consistency and coordination specific to how we use our technology again as an intersectoral society, of course, with today's world.

Speaker 1: 1:32

Yeah, just to be sure, before four years ago there was already.

Speaker 2: 1:37

Yeah, we were already doing it right and that's why I said you know, next year we'll be 50 years old as a company and so, while a lot of what my team is focused on is part of what has become normal business practice for SAS, we were doing it intuitively and we were doing a lot of this work and a guided by local individuals who were kind of driving the mission of the company and that sort of thing. So, as we are centralizing a lot as a company, kind of globalizing our business practices as a company more broadly, this is a part of that portfolio. At least that's how I see it.

Speaker 1: 2:20

So what was the specific reason to set up this specific branch?

Speaker 2: 2:24

Yeah, well, the idea here is that we've got a measure of social responsibility that we take very seriously, and, quite honestly, it was after kind of the racial reckoning in the US that, like a lot of folks, we started to think somewhat differently about how, how AI is going to show up in the world, and, given that we are a significant part of that conversation, we thought it important to establish the practice that we did to address some of the issues that might come up as AI begins to intersect with society, and so that was probably the genesis. Now, it'd be unfair to say that we were the first ones to think about it. Even at the company, there were folks prior to us at the company who had been thinking about kind of responsible AI.

Speaker 1: 3:17

That's always a very hard claim to make, right? Yeah, yeah.

Speaker 2: 3:20

You know I never want to kind of come across as kind of the only in this conversation, but certainly we were the folks who you know, my small team at the time who kind of marshaled the effort you know, built the business case, if you will, and we've been running and running hard and growing for the last four years now.

Speaker 1: 3:39

So what does it mean in the development of products and services for the company? In the development of products and services for the company, do you have a veto on whether or not a product or service goes into full deployment, or how does that work?

Speaker 2: 3:56

Yeah, so I never wanted to show up as the police, right? So that's not our job. But what we have done is we've tried to institutionalize this idea of kind of data ethics, and I'll give you an example or two. One is we've helped to stand up our AI oversight committee. That's a committee that has never existed in the history of the company and the idea there is there's a cross-functional group of executives who are focused on matters associated with everything that we buy related to AI or things that we sell related to AI, and we're there to kind of act as a buffer and a recommendation engine, if you will, for our senior, most leadership team, and so that's one example kind of driving that institutional decision-making Separately.

Speaker 2: 4:46

We're focused on a lot of internal literacy and training activities to make sure that not just our products but our people are reflective of this ethos. But we're also focused on the product side to kind of answer your question from a more tangible perspective. So my team has been behind a lot of the features that are showing up in our platform related to bias reduction, as an example, and things like synthetic data. We helped drive the creation of our very first model car so we can model or, I'm sorry, report on the process of developing our very first AI governance product as well, which will allow us to have a full AI lifecycle ability to govern AI.

Speaker 1: 5:34

How much trial and error is in that kind of thing that you do? Another way to phrase the same question is how much hard science is there behind what you're doing in order to achieve this? So how tangible is what you do?

Speaker 2: 5:52

Yeah well, we aren't the first to come up with things like bias detection, right, so that exists in industry. So what we're doing is taking some of those ideas, putting our own little special twist on it where required. Ideas putting our own little special twist on it where required and then making it available to our users of our platform in the ways that are most conducive for them to use it. What we found is, as we were digging through a lot of the things that we already had, is that there were capabilities related to, say, explainability and transparency. They were just buried somewhere in the stack, and so, to be able to elevate those things and shine a little light on them, let the sales team know hey, this is there and you've got customers who are going to be concerned about this particular feature as they start to adopt things like LLMs. It's that measure of work that I think has proven to be very tangible and very useful.

Speaker 1: 6:50

Yeah, but especially when it comes to Gen AI, the recent involvement of the AI stuff that's going on, explainability is a bit of a tricky one, right.

Speaker 2: 7:05

Yeah, it's huge Because it's non-deterministic.

Speaker 1: 7:07

It's quite hard to determine exactly how it's going to react, what it's going to do, but also, by default, how to explain it. That must have had quite a big impact on your side of the argument.

Speaker 2: 7:22

To your point that architecture is, by design, going to create a non-deterministic output. So, while our team hasn't been explicitly involved from a data ethics standpoint, certainly SAS has been involved with making sure that we can say deploy RAG techniques as an example so we can try to reduce some of the variability with the outcomes example, so we can try to reduce some of the variability with the outcomes. It's important for us to see Gen-AI more as a feature other than a product itself. So what we're doing is trying to pull some of those capabilities into our platform, use them where it's important to use them, integrate where we see our customers needing them.

Speaker 1: 8:02

So you don't see it as a separate discussion? Data ethics in Gen-AI compared to.

Speaker 2: 8:07

No, it's a part of the broader discussion. It's a new set of capabilities. Does it have its own nuances? Yeah, if it's non-deterministic and completely probabilistic, what does that mean in terms of impact on society? So, absolutely, we got to have that conversation. Importantly, we also see the synthetic data conversation as being a part of the generative AI kind of subset, and so what does it mean to use synthetic data in ways that are responsible? Right, it's great to use it in ways to increase representation of a particular data cohort. It's not so great to use it in ways that might produce deepfake images of, you know, political figures saying things that they actually didn't say. So what we're doing is living in that space of that tension where we're providing technology and then trying to advise and counsel on how to use it in the most appropriate way.

Speaker 1: 9:00

Well, you're actually ahead of me, because one of my next questions was about synthetic data. But also, is there such a thing as an unbiased data set? That's extremely hard. I mean, I sort of always say that I don't really believe in unbiased data, because there's always bias somewhere in data.

Speaker 2: 9:19

That's right.

Speaker 1: 9:21

But what's your opinion on that?

Speaker 2: 9:23

I couldn't agree with you more. You know, bias exists because humans exist. Like human beings are biased, that's not necessarily a bad thing, right? I know bias has kind of this negative connotation in today's language and we can talk about why. But you might be biased toward one type of fabric or another, one type of car or another, and that's perfectly fine. I think we're and this requires a little bit of nuance, right? Where this conversation gets lost is that we automatically see bias as bad. And, to your point, there's a lot of companies out there saying we're going to give you bias free, and I'm always like no, you're not, because you're not going to get rid of human beings. That's part of our cognition. Now, that said, we do have to be able to detect and mitigate against harmful biases. But what's a harmful bias, right?

Speaker 1: 10:18

so you find that one right well it's.

Speaker 2: 10:21

it's important that we understand context there, right, and I like to use this example.

Speaker 2: 10:25

All the time, you know, you go around the world and you deal with banks who have an interest in providing more capital to certain communities say, we say women as an example, right, and and? And there might be very good reason why they want to do that because there's been an under-representation in women business owners or what have you women homeowners. And so by identifying gender in their data set, we can say we're going to extend capital, right, is that a bias? Yeah, right, but we would suggest that that's a positive bias because we're going to extend capital. Is that a bias? Yeah, but we would suggest that that's a positive bias because we're using it for a positive purpose. In this particular example, conversely, you can use that same gender variable and say, no, we're not going to extend, and so bias exists because humans exist. But what we want to do is make sure we're using bias for proper purpose, right, and capturing biases that we don't want, flagging them and then mitigating against those for particular use cases. Maybe another way to put it is ethical use of bias yes.

Speaker 2: 11:28

We can use that term.

Speaker 1: 11:30

Frame it in your job description maybe.

Speaker 2: 11:32

Thank you, Maybe that works.

Speaker 1: 11:40

And then the follow-up question, obviously and you alluded to it earlier as well is the concept of synthetic data. I mean that has the by some that's being presented as the data that will solve a lot of bias and a lot of other things that are bad about models at the moment. What's your stance on the value of synthetic data?

Speaker 2: 12:06

So I'm going to look at it in terms of to use our prior part of the conversation I'm going to look at it in terms of ethical use, right? I think that there are cases where we don't have enough actual examples of data to be able to be useful, and filling in gaps help us to simulate in ways that we may not be able to. So let me try to ground that a bit. If we are looking at a clinical trial, the way clinical trials work, once you get into the later stages, is you actually have to have people and you're going to get some sort of I'm going to derive some sort of readings from those people, their blood or their condition, whatever. Well, you gotta be able to identify people with the condition that you're trying to solve for, commonly in a particular locale, right, which becomes very difficult.

Speaker 2: 13:07

So you wanna know why we don't get a lot of therapies out of the final gates? It's because you're trying to find the right number of people to be statistically relevant. Well, imagine if you don't have to be in a particular locale because you can statistically represent that particular condition. Now, whether or not that passes muster in the US through the FDA or your equivalent here in the Netherlands is another question. But as a matter of science, we should be able to simulate those conditions to be able to demonstrate the efficacy of a therapy. And if we can use synthetic data for that simulation, why wouldn't we? Right now, of course, you got to prove it out over time and make sure that that you know is efficacious. But these are the sorts of capabilities that exist for us now, with synthetic data.

Speaker 1: 13:51

That didn't exist, you know, five years ago yeah, but maybe shouldn't we shouldn't make it too important, since it's synthetic data.

Speaker 2: 13:59

I got the impression that a lot of companies are a lot of discussions in the market are making it a little bit too important right it you might be referring to those that want to use synthetic data to kind of build data sets, to create stronger models, and that, yeah, because at some point there's kind of this self-fulfilling prophecy, right, and it starts to, um, it starts to build on itself and and if you're building something out of nothing, right, if you will, synthetic data being that nothing, um, it's questionable. It's questionable to be.

Speaker 1: 14:31

You might as well be actually creating bias by using this. You never know.

Speaker 2: 14:34

Yeah, you just never know and I think I think the jury is still out, as they say, on that one.

Speaker 1: 14:40

I remember I had a discussion with one of your colleagues last year at. Innovate as well about this. This is quite a hard thing to do.

Speaker 2: 14:50

In theory I get the case, but it's just something that would have to be proven out over time. And that's what science is right. You Gotta prove the stuff out well, I'm a scientist by nature.

Speaker 1: 15:00

I like, I like doing the year, the old scientific method, the scientific method. That's right. But you, you just alluded to Sort of geographical things as well. Maybe something will be Found good in one geography and not in the other, right? So if you look at Europe versus the US, what's the status of ethics and AI in this kind of world, on the world stage basically? So there's lots of ambitious projects at the moment, at least by way of what they say in Europe. There was an AI summit in Paris, I think about a month ago or a couple of months ago, at least by way of what they say in Europe. There was an AI summit in Paris, I think about a month ago or a couple of months ago, and I think they pledged a lot of hundreds of millions or whatever to actually do stuff. And honestly, how do you see that? Do you see ethics being different in different parts of the world when it comes to AI? And how do all these things have an impact on that.

Speaker 2: 16:01

So, you know, I have the great fortune of being able to have these kinds of conversations throughout the world, and so I speak to you with a measure of credibility on this one. I think there is a tremendous amount of commonality around the world when it comes to doing what's in the best interest of human beings with AI. Now, I think there are different degrees of risk tolerance, and that should not be conflated with one's ethical standing, and I think commonly they are. You know, I think there's also kind of a conflation when it comes to the regulatory conversation right now, and whether we should or we can't, whether it's too much or enough.

Speaker 2: 16:46

Well, maybe at the end of the day.

Speaker 1: 16:46

The question is does the creation of the best AI technology can that coincide with being also the most ethical AI technology? Because things like the AI Act are trying to regulate stuff and you might make a better model if you didn't comply with it, but you're not allowed to.

Speaker 2: 17:09

Well, I'm not sure that I agree with that premise. I don't think that ethics and quality and effectiveness are mutually exclusive. You can create human-centered technology, meaning that it's technology that's going to be put in place for the greatest benefit of us, and still comply with law if laws exist and be effective, but that's based on the assumption that you're actually making it to actually benefit the humans, Not benefit yourself as a company.

Speaker 1: 17:48

Yeah, yeah, yeah.

Speaker 2: 17:49

I get that With respect. I think that's a bit of a cynical view, but a lot of people have that perspective. Let's kind of follow that out. I think that those who operate fully in self-interest have relatively limited markets, right, because who wants to buy something that's not in their interest as well? Like, I'm not going to consume your podcast if you never give anything to me in the podcast, right, I'll just go find another podcast, and so I think that's just kind of a bit of a straw man Now. And so I think that's just kind of a bit of a straw man Now.

Speaker 2: 18:33

I do think there's something to the sentiment, though, because I think what you're really getting to the heart of is this trust gap that exists right now, and so when you look at the studies, most studies are showing that people, when you just say AI, most people either fear or distrust, and I know that anecdotally, just kind of having conversations, so that's something real there. I do think that there is an element of inaccurate information or insufficient information in some cases, so I do think we have to raise our overall kind of global literacy around the topic.

Speaker 1: 19:04

That's quite hard to do.

Speaker 2: 19:06

100%, yeah 100% difficult, but I do think that, just like we all have a general sense for how electricity works like there are some people who are engineers and physicists but then most of us know how to plug something in the wall we have to get to the plug something in the wall version of AI literacy and we're not there yet Especially of the main people who start working on it. But I do think that that becomes really important as it relates to people understanding how they apply AI to their lives.

Speaker 2: 19:36

We don't need everyone to become a data scientist or a developer or something, but what we do need people to understand is how AI might or might not apply to their lives, and to be able to make an informed choice. I think once they get to that level of literacy, maybe we'll see some of the trust measures improve. But here's one thing that is true If, as an industry, we don't figure out a way to get across this gap in trust all of the wild dreams about people adopting AI you may as well throw them out the window. No one's going to adopt something that they don't trust.

Speaker 1: 20:13

Would it help, for example, if financial institutions and the kind that get out loans or I think you alluded to that example earlier- as well. Would it help if they were more open about how the decision-making process went, because they use a lot of AI. They have been using a lot of AI for I don't know how long For years? Yeah, do you think those kinds of initiatives would make sense to do?

Speaker 2: 20:42

I think it's worth trying. So the supposition there is more information and creates some measure of trust. Maybe I don't know that we have an information deficit, though you can pretty much go if you have an interest. You can go and figure out how banks make loans.

Speaker 1: 21:06

Yeah, you can. They're usually not very open about it.

Speaker 2: 21:08

Right, right, and so what's the right measure of transparency? Right? And this is where things like in the AI space, where model cards come into play. As an example, I use the example of a model card being like a nutrition label you can look at it on your food and see how much sugar or salt you're going to consume. Well, the same should be true with your AI models. Is this thing drifting or not? Is it fair or biased or what have you? We should be able to know that or at least make that information available.

Speaker 2: 21:35

Now, we can't force people to use the information, but we, I think, have an industry obligation to make it available at least. But over time, we do have to figure out ways and I'm using the larger we society ways of getting people more tapped into the conversation. So, is that through force of government? I don't know if that makes the most sense. Is that through the academy? You know, are there some things we can do at home, learning Like? There's a variety of ways that we inform society, and I think AI just has to become a part of the content that we inform society about, since it's going to be such a critical part of our lives going forward.

Speaker 1: 22:15

Yeah, and part of the problem obviously also is that we're still in the midst of where is everything going when it comes to, especially with the latest installment, the Gen AI. We have no clue where we're going to be in about six months time anyway.

Speaker 2: 22:27

It's a borrow for that matter, so that doesn't make it easier right. Not at all.

Speaker 1: 22:33

And I think that's. If you want people to trust AI more, then you first need to define what it exactly is, and that changes by the day and by the month and by the year.

Speaker 2: 22:46

Yes, I cannot agree with you more. Yeah, I think that, in addition to that kind of frequency of change that you're speaking about, establishing some sense for compelling vision that justifies the change I think is really important, and and around the world, I don't think we've done a good enough job and again I'm using the big we in terms of defining what that compelling vision for tomorrow is that justifies why we're doing all these things today.

Speaker 1: 23:18

That may also be because we everybody has no clue where we're going. Yeah, yeah.

Speaker 2: 23:27

So I think the opportunity in that, though, is for particularly those of us, business leaders, to stake a claim right To say, hey, here's where we see things going. It would be great if we could use some of these great capabilities to address some of the ills biologically with the species. So let's do a moonshot as it relates to solving for cancer around the globe. Or let's figure out how we can deal with depression, schizophrenia around the globe. Or let's figure out how we can equalize societies so that we don't have other humans who are starving Now. Will a chatbot solve for that? No, but maybe there are ways that we can use these new capabilities to help us in the real world, in the physical world, differently. Right, you don't know what those are yet, but I think you know.

Speaker 1: 24:27

That's the kind of vision casting that that we lack right now is that also what you would say to two companies or two people listening and who want to know what? What can I do? What can we do to actually make this more successful, or land better at our clients, or whatever? Is it also so you sort of do it, formulating a moonshot vision? Yeah, you know.

Speaker 2: 24:53

I say start bold, start bold. Now you can't throw AI at everything, right. Most of the problems that we see in our society today aren't a function of technology's existence or lack thereof. Most of what we see today is a function of us, and AI is going to mirror us. So if we just take the real world and bake it into AI, guess what we're going to get back the real world, right.

Speaker 1: 25:17

We're not going to like it, probably Right.

Speaker 2: 25:20

And so you know. But that's the opportunity, right, is to say okay, what's not the way we would like to see it in society today, and as we are in the process of digitizing society for tomorrow, let's do a better job, right? I think we owe that to not only ourselves, but to our children as well.

Speaker 1: 25:40

So you should start with more generic kind of visions towards where we want to go.

Speaker 2: 25:47

That's going to be a very different discussion. I think, yeah, it's a big leap, I get it. I'm a dreamer man. What can I say? No, but I mean.

Speaker 1: 25:54

I think for companies, it's very important to understand what they're actually doing, so what they can do to actually make this more successful. That's right, that's right. So defining as a company, so defining what you want to achieve for society, is a nice first start, and if AI can help you do that, you're getting somewhere.

Speaker 1: 26:13

Yeah, and because you can't do everything, maybe you find like-minded companies and leaders to work with in pursuit of these broader goals companies and leaders to work with in pursuit of these broader goals, and maybe the AI industry as a whole should also collaborate more or do more, or are they doing that already? I?

Speaker 2: 26:31

think there's a fair amount of collaboration in areas where you would expect collaboration. So you brought up the Paris AI, something like. Out of that was the current AI. So there's collaboration as it relates to creating safe and trustworthy AI and those sorts of things. There are companies with whom SaaS competes, but I can pick up the phone and call some of the people over there that work in the same space as I do when it comes to trustworthy AI and ethics, and so I think there is a fair amount of collaboration. And so I think there is a fair amount of collaboration.

Speaker 2: 27:02

I think we have to acknowledge some of the kind of the larger forces that we work within. Like not to be too abstract, but you know, we are all at least in the West kind of working in capitalistic environments that have a natural incentive right to be profitable and to grow and all of that and that's all fantastic. And at the same time, we also have to find ways of working with our governments, because our governments can't keep up with the pace of industry, I would argue, nor should they try. But what we do have to make sure they are aware of is how these things are moving, how quickly they're moving, what the potential impacts might be to society, so the government can play its proper role in all this. And and this will be the final point when government says, okay, guys, I hear you, you know.

Speaker 2: 27:54

So this is, you know, the conversation of 18 months ago. We're showing up with something really powerful, potential existential threat, help. And then here in Europe, government says, ok, we hear you, we're going to put this law in place. And then all the people who were saying that said, no, wait a second, we didn't mean it. Like, when government shows up to do its job, allow government to do its job right. And of course there's always going to be attention and kind of a give and take, but that's a part of the process and we should not kind of overplay, irrespective of the power that we have. We shouldn't overplay our hands when we know full well that we can't do it all.

Speaker 1: 28:38

And just in closing, is it going to the process that you're talking about? Is that going in the right direction in terms of there's always a give and take.

Speaker 2: 28:49

If you look more tangibly here in Europe, you've got the EU AI Act. A lot of us were bullish about it. Did the government overplay its hand? Maybe there's a lot that came with the act. There's a lot that wasn't defined.

Speaker 1: 29:09

There's also a lack of knowledge by the people who actually set it up.

Speaker 2: 29:13

I think Right, so all of those things. So if we're going to measure potential harm based in flops, is that the right measure even now? And so this is what I mean. We've got kind of this emerging technology and it's very difficult to regulate something that's actively emerging. Commonly, regulations make more sense when you have something stable. Now should we allow something to emerge that is being positioned as existential threat? Well, probably not the best idea. And so what's the proper give and take? And I think it's important for all stakeholders involved to recognize their place and not overplay their hands. But again, when we're talking about some of these macro forces kind of pushing us on what our incentives are, I understand it and I'll leave it there. Yeah, that's fine.

Speaker 1: 30:03

I promise you I wouldn't ask you yes or no questions. I think I already asked you one, but I mean, that's fine. I didn't say no, but just maybe one to just finish it off with. Are you optimistic about where ethics in AI is going?

Speaker 2: 30:24

I am. I'm optimistic that it is integrated and embedded from the very beginning. I think there are a lot of players out in the space right now who, if only vocally, even if they aren't doing something operationally, if only vocally, they're saying they're doing the right thing, and what that allows me to do is to hold them to account. So at least we've got that part. Now do I want to see greater operational output? Absolutely, greater operational output, absolutely. But I think to be fair and to be objective, we have seen, you know, just look at the foundation model providers. You know some, we would argue, more responsible in the spirit of responsible way than others. But I think that's reflective of risk tolerance more than anything else. And the same would be true us old heads who have been doing this for a really long time, and so we want to be kind of the wise voice in the room, because we've been at this thing for 50 years and I think if you look at our track record, it would be reflective of responsibility as well.

Speaker 1: 31:43

I think you shared quite a bit of wisdom today. Thank you for that and I think with that, I think we're done. Very well Thank you Thanks for joining.