Eric Wallace 0:00
Welcome to the Argyle AI/ML Technology Leadership Summit. My name is Eric Wallace with Argyle. And it’s great to have everyone joining us today. A couple of notes before we begin. First, a quick reminder to stop by our sponsors virtual boots at any time during today’s event, and for the following week. Our partners are committed to providing you with valuable content and a great overall experience. At any time during today’s event, you can visit their virtual booths from the main agenda page. And these include complimentary materials, information and meet and greet opportunities. To ask questions during the session, simply type into the q&a chat, and we’ll address your questions at the end of the session. Now, let’s begin with introductions. First, we’ll just go around the virtual table and introduce ourselves for our panel discussion today. Norm Can you please start? Yes. Thanks,

Norm Trujillo 0:46
Eric. I am a former vice president of data and analytics at NCR, a lifetime consultant specializing in retail consumer products and financial services.

Eric Wallace 1:00
Right Thank you deeply. Can we continue introductions please?

Deepti Kunupudi 1:03
Yeah, my name is Deepti. I lead the IML practice at MoneyGram International. And my primary role is around like establishing the AML governance standards and also like leading initiatives which are primarily focused on optimizing the effectiveness and efficiency and build efficiencies around.

Eric Wallace 1:23
Let’s conclude introductions with the place.

Greg Thompson 1:25
Sure. Hey, good afternoon, everybody. My name is Greg Thompson, I work for Eightfold AI. We are a talent intelligence platform. So basically a data platform on which we have built talent intelligence, longtime practitioner and product manager in the talent space. Having worked at workday, SuccessFactors and Oracle in the past, so very happy to be here today.

Eric Wallace 1:51
Thank you Welcome, panelists. Let’s begin with our first question. And that is, what do we mean, when we talk about workforce optimization? Greg, can we start off with you, please?

Greg Thompson 2:01
Sure, sure. So I think of workforce optimization, and how do you effectively run your business better than you have in the past? I think the tools and techniques have, obviously, very quickly over the last five years. And so we’re seeing, you know, the focus on workforce optimization, really starting with defining what do we have? And what do we need? But, you know, that goes into all parts of the company, from, you know, what are the responsibilities of individuals to? How are we going to pay for those individuals as well?

Eric Wallace 2:40
Norm, what do you think? What do we mean when workforce optimization mean in your context?

Norm Trujillo 2:45
and I think of it from a data and analytics perspective and working in this space for some long period of time, I think of it as productivity and proficiency of an employee. So if you put the CX of you on it, it’s the experience of the employee. I also think it has four elements that you need to examine, through data, through analytics, and even AI. Now, you look at it from a workflow perspective, you look at it from a process perspective, you look at it from a decision support. And then also you look at it from a skills and knowledge perspective, all these four areas, in my opinion, helps you optimize your workforce. And we can give several examples through the course of this panel discussion.

Eric Wallace 3:37
Great, thank you deeply, what does workforce optimization mean when you talk about it?

Deepti Kunupudi 3:41
So I see workforce optimization as a kind of a strategy for the organization to build the effectiveness and efficiencies around that. And I see that having the right skills and having the right talent and being in the right place is what we are trying to achieve better workforce optimization. So they are like I see that people see as employees as a cost. But actually employees are the resources are nothing but the drivers for the company’s strength and revenue. So if I look into it, like a kind of a high-level overview, I see workforce optimization into the management of the talent, training and development of the resources, and at the same time, making sure that we look into the performance metrics around the resources and optimize them.

Eric Wallace 4:30
Thank you, as long as we’re defining terms, let’s continue. What do we mean when we talk about workforce intelligence norm? Can you help us out with that to start, please?

Norm Trujillo 4:39
Yes, simply put, it’s the ability to collect information and analyze it around work patterns. So an example of that. You can look at how your employees are interacting with an application. There’s tools out there a lot by hot jar that not only is it used to look at the experience of a customer, but you could also actually look at it as using that tool to look at how your employees interact with applications and systems. Deeply What do you think?

Deepti Kunupudi 5:18
So I see that workforce, like the previous question is like primarily around workforce management, I see that as a strategy and workforce intelligence, also, I see as a strategy. So if you look into the difference between both of them, I see that for workforce intelligence, we use the data and analytics to drive insights, and do a long-term planning and workforce intelligence. Whereas in workforce management, what we are doing is like we are planning it and executing it so that we have we have the right opportunities to build those efficiencies. So when it comes to workforce intelligence, we are using that data to optimize it for a longer term. So I see those two as a kind of like two pillars for the complete workforce optimization.

Eric Wallace 6:03
Right, thank you, Greg, what do you what do you offer us in conclusion on this question,

Greg Thompson 6:07
yeah, I love what Deepti and Norm are saying. I would add that I think intelligence now means having data that not necessarily was generated by your own organization, right? So being able to compare yourself from a benchmark standpoint to other organizations, and really get intelligence about what are the skills that are increasing in demand? What are those skills that are declining in command? Certainly, we can look at a lot of examples right now, where machine learning is changing people’s responsibilities. A resume five years ago, CV may include Microsoft Word for an executive assistant. Now, I think you’ll see a lot more chat GPT type of, you know, skill sets. So workforce intelligence, for me, really means a constant review, and adjustment of what you’re doing within your workforce with a keen eye on, are we engaging the people retaining the people that we need? And if we don’t? How are we going to get them from the market? Are they readily available or not? as well? So the intelligence aspect really goes to strategy operations, financial controls, but for me, the intelligence is being able to compare yourself to

Eric Wallace 7:30
others. To follow up, how can artificial intelligence and machine learning technologies help with workforce intelligence? You’ve taken your take the lead on that one, please?

Deepti Kunupudi 7:40
Yeah, sure. So like Greg mentioned that workforce intelligence is primarily like using the data and analytics and driving initiatives around AI ml. So when you say when we are using AI ml, the first thing we see is like the predictive capabilities of the machine learning that means that we are mining the data, which is already we have saved for a long time and then using that capabilities on top of it to build the AI ML Engine. So in this case, like for workforce intelligence using AI ML is like primarily to drive the using NLP or like charge up the to optimize the person and make it more personalized, or when you’re sending the emails around, like when you’re hiring or something along those lines. And predictive capabilities is like to understand the employee engagement, how we can optimize the engagement and making sure that they we try to retain those employees within the organization. So there are multiple capabilities, which can which we can enable based on like, what kind of use case you’re working on from AI and ML standpoint. And to the grid point like previously, five years back, it was word, but now we need to be a little more creative using Gen AI, generative AI capabilities, and how do we optimize our complete process to make sure that we are building those efficiencies and effectiveness within that particular department? Right, what do you think?

Greg Thompson 9:06
I think what’s key here is, you know, the visibility and translate ability of the machine learning and what I mean by that is, do we see what it’s doing? Do we know why these suggestions and recommendations are being presented to us? One thing I would caution against is the terminology of machine learning and AI is thrown around quite a bit. We see that nearly every industry has adopted machine learning to some degree over 50% of the organizations out there are already using machine learning, but the visibility, the insights that are being provided to you, are they being generated based on language? Are they being generated on other data right So I think Norman brought up a great point, you can be looking at data, for example, not just how many people have come to your site, or your corporate site, but what were the outcomes? Did we actually schedule interviews with those people that came to our career site? Did we actually hire them? Those can be self-referential, in that, you know, you can literally start to target should we get employees from our competitors? Yes, or no? Maybe it’s a different culture. They don’t typically work out well, if they’re in the sales organization, and join our company. Other types of insights, like, you know, data scientist are now embedded in every business, right? So it’s not necessarily a technology company that you’re competing against. Really, everybody’s looking for learning and AI capabilities. And so the ability for you to target going after those specific skills. That’s really where the intelligence needs to help you, not necessarily creating a better CV, but really saying, hey, to limit your efforts, here’s where you should start. And here’s the expected outcomes.

Eric Wallace 11:18
Thank you. Norm. What do you think? What, how can AI NML help with the workforce intelligence in your experience?

Norm Trujillo 11:25
Let me give two example, maybe I’ll even give a third if I can go through it fast enough. So the first example and these are all examples I’ve personally been involved in, is pick your favorite grocery store, you go there on the evening, and you’re trying to go to the front of the store to check out at a grocery store. And there’s not enough folks manning the lane, or the self-checkouts are only card only. So you can use the intelligence of the traffic in the store, you can use a computer vision to look at the queuing, you can use all the data points to basically optimize what your labor schedule should be during the morning shift the afternoon shift in the evening shift. And that companies are doing this today. That’s example number one. Example number two with Jana AI. And again, this is zin use. So it’s folks that have done it will know what I’m talking about isn’t a call center, there’s two ways of thinking of it, a passive AI approach or a real time AI approach. So the passive takes the call recordings, and basically uses it to improve how the caller made its way into an agent to help them on whatever it may be. or direct them to a specialist that can really answer that question based on how they’ve been bouncing through the menu. That’s more passive. And you can make changes by looking at the, from a design and implementation from how they navigated their way through the call center and finally got things resolved. For but more Gen AI is going as a real time, age, and that’s listening into the call and serving up content for you searching for it, serving it up and even whispering in your ear, what to say or ask Max, an example that Best Buy’s trying to be using that for its for its Geek Squad, troubleshooting, and so on. So several examples around that. And then my last example, just quickly here, think of the carriers that pick up our package and deliver it to our household. When a sort of facility goes down, or a truck is late into a sort of facility, you got to you have to redirect the packages on the next sort of window. And that’s all done behind the scenes without, you know, people getting involved. And again, it’s optimizing your workflow, your process. And if you can do it with less labor, you do it right. So these are three examples that I personally have been involved in.

Eric Wallace 14:20
Hey, thank you. Let’s continue with, again with another definitional question. What do we mean when we talk about talent mapping? Greg, can you lead us off on that one, please?

Greg Thompson 14:28
Yeah, talent mapping goes back to what I said before, understanding what you have today and where you would like to go in the future. So talent mapping, you know, for some industries, really means Hey, we need to prepare for changes within our organization. Oil and gas, right? If you look at the type of data that they’ve got, they’ve hired people from the oil and gas industry, their job descriptions are all about Oh boy oil and gas, their outcomes of promotions are attributable to oil and gas. But yet they’re being disrupted. And they really need now to have a brand-new vocabulary around solar renewable energy. Right? So a lot of organizations are in this, you know, point in time where they’ve dedicated a lot of humans to trying to describe jobs. And those descriptions are not very accurate or have up to date. And so they become stale. I think where organizations are looking now is not only what do we have today, but what’s the intention of this particular role in the future? And I think that’s a good thing. Because whether it’s the organization looking down and saying, What is the career pathing that we will provide? And where do we put development resources? Or the employee looking up saying, hey, is there an alternative path for me, I joined this organization, maybe I feel like it’s a very short runway for my career. But I’ve got all these skills, how might I go and use those skills in a different manner. Personally, I’ve been a product manager in the HR software space for about 30 years, I joined this organization, and they have convinced me to be part of the services and the sales organization, things that I never expected that I could do, but because of my experience, and my skills, you know, customer satisfaction, technical acumen, you know, a desire to, to self-educate. These are very popular in each of those organizations in our company. So it was a very easy transition for me personally, even though I didn’t think that I had the vocabulary to be in the sales organization, I saw that I did have the skills that contribute to a positive outcome. So I’ve made a midlife career change, simply because I was excited about a new role not stuck with an old job title. Thank you, norm. What do you think? What do you mean, when we talk about talent mapping? Well,

Norm Trujillo 17:20
obviously, it’s in anticipating the future needs of the business. And the two examples that I would give is, again, in retail, it’s arming your workforce, with the product knowledge in the department or all departments that are that they’re in. So if you go into the grocery store, you are asking about to produce, they help you. If you ask about another product in a grocery store, they also help you so product knowledge, arming them with that knowledge to help a customer is pretty important. But as of probably recently, think of the knowledge and skill set needed for people in the data field, or the data science field or the analytics or AI field. So there’s efforts going on today to educate, train, create courses, for corporations to drive, more knowledge, more skills, to take advantage of data, AI, analytics, and so on. So I’m seeing these two things really take off in the last 10 years.

Eric Wallace 18:31
Right, thank you deeply, what is talent mapping mean in your place of work?

Deepti Kunupudi 18:35
So when you say talent mapping justify the two words, it’s like mapping the talent to the right skill set. So I see that in talent mapping, what we try to do is like identify what your current skill set is of a particular employee or resource and what are the future needs for a particular department, each department works differently. HR has a different needs. Finance has a different needs. And like data and analytics has a different needs, like you have data business intelligence reporting, and then we have data science. So it’s like understanding the requirements and mapping it to the future, like you need to also make sure that what are the current runs which are going on in the industry and mapping so that the employees would be more resourceful. And I see that AI ml in the space as like primarily is like kind of optimizing the efficiencies around it, then then, like cutting on the job side or something on those lines. So I see that it’s like just a mapping so that the resources are well prepared for the future, and also like making sure that they are well integrated within the organization. So talent mapping is like primarily around those four pillars, which I mentioned.

Eric Wallace 19:47
And let’s, let’s have you take the lead on this next question. And that is, how can artificial intelligence and machine learning make talent mapping easier and more effective?

Deepti Kunupudi 19:56
So pollen mapping is like I put it into two category Phase one is internal resources and other is in getting the external resources. So, when you say within internal resources with the rapid space in the AI ml space and how quickly the Gen ai has took over in last couple of months it you can see the space base in which it is going. So, in talent mapping, we can use AI ml, especially in the sourcing side of it, if you wanted to bring in new resources. And we need to we can use it in the sourcing, then we can use it into an automating those messages, which we send it to them, personally send it to the resources who are applying for a particular job, or maybe we can use them in the onboarding side of it. So the talent, AI can be used in multiple different scenarios within the HR department. And also, we can we can use it within each department of the individual department, whether it is finance, or data and analytics space.

Eric Wallace 20:57
Greg in the norm, can you please respond to this one?

Greg Thompson 21:02
Yeah, I would say that there’s a couple of things to keep in mind. explain ability is going to be much more important going forward. Right? I don’t think that machine learning is always going to be a black box, I think there is compliance and regulatory aspects that are coming into the space. So being able to retrace, why is this suggestion? Why is this recommendation being given to me? And with that transparency, I think, is explainability. And it should be explainable both to the C suite all the way to the jewel, you know, full time employee. Why do we think that your job or what are those deficiencies that you need to work on so that you can progress into this particular position? that transparency is really going to revolutionize, if you will, the whole organization going forward, it’s no longer a back office? Hey, I wrote down what your job is. And once a year, we’re going to have a performance review where we look at those bullet points and I grade you. Jobs are much more dynamic jobs are much more project based today. And so how do you assemble a team of people together to accomplish something, as opposed to us hired you for this job, and for the next 10 years, you’re going to do exactly the same thing. So I do think that explain ability meets compliance is really where machine learning has to go. Great, thank you, norm. Yeah,

Norm Trujillo 22:43
I’m aware of some key initiatives in the marketplace around creating bots to evaluate the knowledge of an employee. And these efforts are tied to universities with some of the top professors like Stanford, Harvard, MIT, and so on. It’s focused on an interactive dialogue between the bot in the individual and based on those set of questions that the bot comes up with and how the employee or person responds to it will gauge their level of proficiency in that area. And why this is so important is you got to make decisions whether an employee is ready for the front line and critical jobs, you want to be able to go through something other than a pass-fail type of testing or certification to gauge whether they’re really equipped to do with the various exceptions that surface out in the field. So that’s my short answer on what I believe is happening in the space. Thank you. Let’s

Eric Wallace 23:56
let’s go into a little more detail on the specific stages of the employee lifecycle. The next question is, what are some examples of how AI and ML can help with talent sourcing and recruitment? Greg, can we start with you? And then we’ll continue with Norm?

Greg Thompson 24:11
Yeah, yeah, happy to. That’s really the essence of the organization that I work for. So we start with a global dataset that includes publicly available information from around the world. So think of it as mapping about half of the career directories of everybody on the planet. So 1.5 billion career trajectories is the data that we start with the question of how can it help many, many different ways but in machine learning and AI? One thing that we use that we talked about is a feature and a feature is a very specific answer to a question of the data. Right. So the day data itself can answer many, many different questions. But you do have to focus the model on specific data that is important or should be excluded. And let me give you an example of that. When trying to match people to jobs, one thing that you must consider is bias. And you must remove that bias, right. So, in our models, one of the things that we do is, we use a golden dataset that does not include any personally identifiable information. So we extract out people’s names, university names, the year that you graduated, right. And we train our model. After we train the model, we go through a QA process to say, Okay, let’s see if bias came into the model. And so we use the golden data, not have bias information against a very biased data set. So think of 10,000, clean, CVS, and 10,000 CVS that have lots of dates and pronouns. In this way, we’re able to statistically prove that the model is not suggesting based on gender not suggesting based on age, or these other PII factors. And so we do have the ability to have a feature that is purpose built, the result of that is we see in our customer base, a 60% increase in application completions, because the bot the AI is not bias and it’s saying here are some jobs that you’re a strong match and a good match for we also see about 40% increase in female applicants, it studies that men tend to apply to any job be considered for where women tend to really look at the responsibilities and judge themselves a little harder than men. So, with the bot making the recommendation to a person, Hey, these are the jobs that you should apply for based on your CV, it overcomes that bias it overcomes the candidate maybe not knowing what terminology you use within your organization. So, these features are really important because they are based on data, they are statistically valid. And you can also as Norman is saying look at the outcomes, who made it to the interview, who got an offer who was actually hired, well hiring is a great signal but so is silver medalist those people that made it multiple steps through the process right. So, if we want to go back and look at somebody we want to hire, why not look at the so the silver medalist to begin with AI can do that that out of the box, it can make the recommendations of this versus that in a very transparent sort of manner.

Eric Wallace 28:08
Thank you deeply can continue with you please.

Deepti Kunupudi 28:12
Yeah. So on the on the complete lifecycle you see that we have sourcing is one part of it, the pre-employee engagement is also one part of it, where we can leverage AI ml, I see that we get pools of resumes like which are like start we apply, and you don’t get any response from it. But then you are sourcing it and identifying like what the job what we are looking for and mapping the resources and the pre-employee to the mapping, I think it is very critical. And AI ML is like using the NLP technologies. We can map those the unstructured data to a structure format and market with the job what you’re looking for, that is one particular use case where we can optimize using AI and then and the second thing is like having the kind of the resource chatbot which Greg mentioned about like having those engagement early on to understand a little bit more about the talent and removing the bias in the data and then mapping back to the job description what we are looking for, I think it is very critical because biases is there and it is a thing which we have to address right now. And also making sure that we are in compliance but all the other AR norms within the within that AI ml space. I think that is critical for the end-to-end implementation of AI ml solution

Eric Wallace 29:42
to a norm can we conclude with you please?

Norm Trujillo 29:45
I don’t have any more to add these guys did a great job.

Eric Wallace 29:48
Well, let’s have you lead off on the next question, which is what are some examples of how AI and ML can help with employee onboarding? Oh,

Norm Trujillo 29:58
I think first of all IT knowledge acquisition is the most important part of it. So as part of onboarding, you could have ways of accelerating not only the knowledge acquisition of an employee, but also any formal training that they need to get certified on. It’s, it’s in line with what I said earlier about eight interacting with a bot, and it is evaluating your knowledge and your proficiency in a given area.

Eric Wallace 30:35
Thank you deeply, what do you think?

Deepti Kunupudi 30:37
So on the onboarding experience, I think we need to have like the trainings, which are currently being manual that can be automated using AI and having a chat bot with pre information around like, what to look for, where to where the accesses where the folders are, where the knowledge repository, those things are currently manual, and it takes like long time to onboard a resource, I think those stuff can be automated. Along with that we can automate few of the pre administrative stuff which, which will help the employee to be engaged with personal messages and, and busy Gen AI, we can have those personalized content created for the onboarding process that would that would help introduce a lot of manual effort and increase the efficiencies around that process.

Eric Wallace 31:26
Great, thank you, Greg, what do you make of this?

Greg Thompson 31:29
So for onboarding, I think a couple of things. Number one, does the organization understand me? Right? I think a huge failure in the past has been, we’ve focused on, you know, compliance forms, like an I nine fill that out, and we’ve called that onboarding. I think the new method is, does the organization understand me, and my particularities, right, that my strengths, my weaknesses, where I want to focus? So onboarding night, I think, you know, the traditional HR data systems don’t really let you describe yourself in multi-dimensional ways. We are seeing much more comprehensive CVS you know, here are, here is my work, my portfolio of things that we I’ve worked on in the past. So I’m bringing my whole self to this organization. That’s one. The second part is what happens after I provide this information. And again, things like I nine forms just go off into the ether. And we really don’t feel the impact of filling out that form and what it does for me, so I think, onboarding, in addition to describe yourself, the organization needs to tell you, Okay, did you know here are immediate learning opportunities for you, right? The bot has been able to identify that there may be a skill gap, maybe are 99%, perfect for this job. But there’s just one area you don’t have experience and the bot can close that gap by recommending the class, that class doesn’t necessarily have to be tagged by The Learning Department to say that it’s a relevant class, the bot may be looking at the content, may also be looking at people in your organization, and saying, I don’t know what reason, but it seems like everybody on your team has taken this Agile methodology. And I don’t see Agile methodology on your CV or your profile. Maybe this is something that you need to take. So I think those two things of bring yourself and let the organization in the system reflect back to you what your opportunities are. I think that’s tremendously transformational. We’ve talked about it in the HR industry for a long time. But now we actually have that engagement, that experience come immediately for the employees. And again, it shows up in a variety of ways, such as enrolling into a learning class increases by 65%, if you have a profile that has been completed, right, so these things are connected to each other, and it really comes down to does the employee feel engaged? Do they feel like they’re part of the, you know, the culture in the tribe and DEP I think you’re spot on? Folks like ServiceNow are investing a lot into AI to predict what are those business processes or workflows that you need to complete in week one, week three, all the way down to you know, at your job title, not just your business unit. So lots of recommendations. You know, immediate outcome and engagement I think are important.

Eric Wallace 34:58
Thank you. Let’s change the channel. A little bit and take some questions that are coming in from the audience. The first of which is an interesting one. And its Hello, I’m curious to know how you see AI and ML technologies impacting space travel? In what ways do you believe these technologies can improve predictions and strengthen decision making in this field of space exploration? Anybody want to take the first one?

Greg Thompson 35:27
I think again, there are lots and lots of tests that are run on astronauts today, because there are so few astronauts. But as we see with, you know, recreational space travel now, you don’t have to necessarily go through a five-year program. So space travel is going to benefit by again, artificial intelligence, knowing me very personally, and how I am different from the astronaut next to me, right, so my sleep patterns, my, you know, my mood swings, am I a morning person, afternoon person in space? Maybe that gets very confusing. But I do think that artificial intelligence is going to now include not just the technical aspects of the rocket, but you know, the soft aspects of the human as well.

Eric Wallace 36:19
Norm, any thoughts on the impact of AI NML on space travel?

Norm Trujillo 36:23
That’s way beyond me, I’m focused on the graph.

Eric Wallace 36:31
Let’s move on to next question that’s, have you noticed any false positives in your results?

Deepti Kunupudi 36:37
So I can take that question. So when you’re building a model, and especially, you have like data, which is like, so wide variety that you end up being, you end up with the false positives. But the idea is, like, when you’re building a model, and you see a false positive, you need to fine tune your model, and understand like, what those false positives are, and whether we are using the right metrics around like to evaluating that particular model. And then also making sure that we are not introducing bias into the model so that it is like only bias towards certain aspects of it. So you need to completely evaluate the model from all the perspective and then say that, okay, everything is evaluated. And having that AI ml governance around it would help you to streamline it and standardize the ML model. So yeah, these are some of the caveats, which we have to look into it and make sure that the results are aligned with what your business goal is. And for fraud reduction, you need to look into it from certain aspects of it, whether it is a true positive or false positive. So similarly, based on the business use case, you can, you can evaluate a which metric with completely aligned to your model behavior.

Eric Wallace 37:56
Thank you, Norman. Greg, do you have any thoughts on when have you noticed false positives in your in your results?

Norm Trujillo 38:02
Well, let me just go quickly, I’ve run into false positive throughout my entire career. And normally when I run into it is works well, in one country, when I go into another country, it’s flipped upside down. So you just got to know about, you know, how are you using? You know, your models as you go from country to country and just realize there’s going to be differences, flip, flop the differences, and you’ll have false positives.

Greg Thompson 38:35
I would add that, you know, what Deepti is saying is, the features need to understand what their answers are, right? And so if you’re consistently getting false positives, then you got to go back to your model, and reassess how you’re looking at that data. But there’s also the case where you, you might think there is a pot of false positive, and you need to investigate. And I think the important point here is, be involved. Right? test that theory. Find out the underlying reasons. And I’ll give you an example. We use artificial intelligence to infer genders of individuals, very famously, only about 10% of the people filling out an application will, necessity and their gender. So a lot of organizations struggle because they don’t have enough data. We’re using AI in two different ways. One of those is to identify based on large dataset, the US Census, look at somebody’s first name. And does the US Census say that over 95% of the people are versus another. So Susan, yes, one gender, Chris. Mm. Not so much we can’t really discern which gender somebody is right. But the inference is there to help us when we don’t have data. And the false positive aspect is this. We could be hiring the best people possible all five stars, and then discover that we have a hiring manager that has hired three people in a row that only have two stars. Okay, well, there’s a false positive, right? It’s, it’s something’s going on. Maybe it’s a brand-new job title. The companies never had before. And our job description is not very good for that particular manager. But we have to ask ourselves, why did that manager overlook maybe 1000 other candidates that were five and four and three stars? To always pick two stars, right? So the false positives can also be an indicator, right? When a human looks at it to say, let me find out why this is so radically different. And again, if the model is doing the right thing, you’re going to find out that it’s human behavior that is influenced those anomalies?

Eric Wallace 41:13
Do you have ethical concerns at this stage? If so, are

Greg Thompson 41:20
they just like any tool? Um, I think that bias and compliance are super, super important. I don’t think that we’ve caught up to the reality yet. I do think there is some good starts. So New York City now has their bias audit requirements. So if you’re collecting information from public people, in New York City’s for marketing, or for job matching, you need to go to a third party and be audited to ensure that you do not have bias in your recommendations. I think we’re going to see a lot more of those types of compliance. Certainly, our German customers are very much involved in wanting to understand what is it doing? And how is it doing? So the automated aspect of self-driving cars may be a little farther out for the organization. But, you know, if we start focusing on bias right now, and outcomes, I do think that we can mitigate that risk, because we are, you know, acknowledging that it may be there and taking proactive steps to ensure that it doesn’t influence.

Eric Wallace 42:39
You have ethical concerns, and what are they?

Deepti Kunupudi 42:46
So I see that there are a few like, things like at least the three things which I normally consider when I’m like, when, when organization is a building, a model is like, the data and privacy concerns around like, what data are we using to build the model? And the second thing is the bias in AI, like, are we introducing any bias in the data? And the third one is the black box problem, because ml generates some results, you give a data, and it will provide good result. But is the result valid? And is it aligned with your business use case? And can you do some explain ability around it, I think a few years back, it was not the case, like where you’re using it. And you can use some of the models where it gives you a little predictability. But I see that the future of AI is around these three concerns, like the AI and by us the data and privacy concern and the black box. And I think as per the regulatory and the compliance side, we need to have some explain ability integrated within the model so that we can address the ethical concerns at each stage and make sure that whatever the result is throwing we have an explanation to it. I think I feel like there are more regulations coming towards the black box problem going forward.

Eric Wallace 44:01
Right, thank you. Norm, Can you wrap us up? What are your what do you have ethical concerns in your AI NML practice?

Norm Trujillo 44:06
No, not from an employee perspective. If he and if you look at the big four accounting firms, if you’re an employee of those type of companies, you have to disclose everything, and they track everything about you. So if I was a customer, yes, if I’m an employee, I would say no.

Eric Wallace 44:30
Thank you. Unfortunately, that’s all the time we have for questions today. Thank you, panelists, for this insightful discussion. And I want to thank everyone in the audience for joining us today. Thanks very much.