Data Driven Product Development

00:26

Okay, well welcome everyone. Good afternoon or good evening, wherever you’re tuning in from, to welcome to the data driven product development webinar. This is being hosted by the i Four group and build wells labs. And we hope that you’re gonna have a very informative session this afternoon. I’m going to be your facilitator, Charles Maddox, and I have my colleague, Imran Khan, who was a data data analytics engineer. And we’re going to give you an overview on how you can take a data driven approach to product development. So I’m going to kick off and give a brief introduction to ourselves and give a little bit of time for those that haven’t joined yet to get a chance to join in by give you a brief overview of in background of myself and Imran. I’m Charles Maddox, with the i Four group, I’m the principal and founder. We are a consulting and training company have been doing Scaled Agile training, Lean and Agile training for the past 10 years, based in the Dallas Fort Worth area, and I’ll turn it over to Emraan. Want to give a brief introduction to yourself.

01:45

Thanks, Charles. Good to be here. Nice to see everyone. My name is Imran I am originally trained as a materials engineer, but stepped away from that space, spent some time in government working in project management for some years. And then over the past, I would say five years or so have been involved with data science and data analytics, both in a training and education capacity. I teach various courses in the space and then also work as a practicing data scientist as well. Taking on projects from the public sector or private sector, a number of different projects through an analytics consultancy, build well labs. So I’m happy to be here.

02:34

All right, thanks. I’m Ron. And for those that might have to jump a little bit early, you know this, this actual webinar is recorded. So you will get a link sent out to the recording so you can join later. But one good thing that being live here, you can actually give us some some questions in the chat. And we can address those in the last 15 minutes of the session here. So we’d definitely like to hear your feedback on any of the questions or challenges or opportunities that you might have going on. And we can help address some of those aspects. All right. Well, without further ado, we’re going to jump right into it. And just to give you an overview, kind of a plain and simple approach that we’re going to take with this webinar is that we know that there’s a traditional approach to project I’ll say project product development. And it’s not necessarily traditional, we’re going to talk about an agile approach or you know, and that’s one of the kind of the newer ways of doing development work. And so what is the Lean agile way of approaching any any product development initiative? And then we’re also going to give How can you make that data driven? How can you make that data driven, and I love this, I love this. This slide here, by Edward Deming about In God We Trust, but everyone else brings data everybody else has to bring data is really meaningful and hits, you know, hits the nail on the head, because, you know, if we’re not using data driven decisions, we you know, are we just making gut feel decisions are called the hippo effect, the highest paid person’s opinion and how we decide which feature to go for next, we should be making data driven decisions at all times, and especially when we’re running our businesses, you know, so a product development approach using data, you know, how could you know, that definitely is should give you the best return on investment, and the best opportunities for success versus just a shoot in the dark type of approach. All right, any comments or feedback on that? Imran before we jump right into the material?

04:36

Nope. Let’s move on. I

04:37

love that. All right, great. All right. So first, first item on the list and kind of the way that Imran and I are going to kind of walk through this process with you again, are i Four group process is that we kind of use this lean, agile approach to doing product development. We’re going to give you that kind of overview from that approach, and how can you make that data driven on top of it? Alright, so In any product development space, a lot of the the aspects that we like to take from a lean agile perspective is we need to take a design thinking approach, you know, we need to understand what the customers are struggling with, we typically may have brainstorming sessions, we call them empathy sessions, things of that nature, we sit with our customers, we understand their problems, we map them out, we get alignment on those. And we call this as a divergent thinking process. We’re open for many ideas in that space to get as many ideas in the funnel that we can to start to try to construct a hypothesis of what might be happening, what might be going wrong, or what benefit we might be able to give to the customer and in the certain aspects. So it’s the discovery process that we’re at the beginning of the funnel that we’re looking at, and how do we make sure that we’re engaging with our customers in the right way to make sure that we’re, you know, getting the next features and products that they’re wanting? And so that we can actually start defining what that is, so, Imran, how would we actually take a data driven approach to getting some of these hypotheses in place, so that we can actually, you know, start building the right features for our customers?

06:16

Yeah, I think one of the things to keep in mind is that it is really important to have data in mind from the very beginning. I feel like processes, anywhere you are in the product development lifecycle, wherever you are, might get a little bit convoluted and confused, if you attempt to introduce some type of data driven process or data driven approach. At some point, else other than the beginning of your process, it works much more smoothly, if you go in to any project from the very beginning, thinking that you’re going to use data in some way, shape, or form to inform how are you going to kind of go through every step of your product development lifestyle, so that I think more than anything else, is probably the most important thing that does depend on a lot that depends on every unit, any team that is involved with this particular product to be conscious of the way that data might be used, it’s important to get all teams functional, and otherwise on the same page, in terms of how data will be used, it’s important to make sure that people on these various teams are educated. Oftentimes, feedback is typically taken in by some sales team or some team that has interaction with customers. So it’s important for it’s especially important for teams like that, to be aware that, you know, we are going to be using data to drive decisions. And I think that’s something that needs to be taken into account. Again, from the very beginning, I’ve been involved with a number of projects that maybe not maybe might not start out with a particularly data driven approach, and might look to implement some type of data driven strategy somewhere further downstream in the product development lifecycle. Sometimes that can be a little bit confusing, not as straightforward, there’s a ton of communication, then that needs to be that needs to be handled, in terms of communicating with various teams that might be developed Don’t be involved with, with product development. So it just works out much more smoothly, if goals and intentions of being data driven from the very beginning are implemented.

08:39

Got a question? I? So practically, what does that look like? If you’re taking that, you know, data driven approach? Right off the bat? Are we constructing data models? Are we you know, is it simply just defining the data sets that we’re going to be tracking? I mean, how does that actually get implemented on a product development initiative?

08:59

Yeah, I mean, it’s all of that, right? It’s making sure that all the systems are in place to actually intake in data, it’s making sure that systems are in place to clean data, if it comes in in a specific form, making that data usable and workable. That is a whole, that’s a whole project in and of itself, you know, typically handled by, you know, data scientists on a particular team or data engineers on a particular team, making sure that pipelines are robust and open. And all the data all the backend kind of data engineering work has been done to make sure that those pipelines are available for data to actually come through and make its way from customers back through sales teams, back to other teams in in that that are involved in the project or in the product development lifecycle. Making sure that that data is communicated in some type of summarized form to executives that might be looking at how data might be used to drive the data. aren’t men have this product. So it really does involve systems and processes. And again, those systems and processes work best when they’re developed from the very beginning with the product in mind, right. And so making sure that your data is coming in in a certain way that is specific and that is relevant to your particular product can only happen if, if a data driven approach is intentional from the very beginning. Oftentimes, like I mentioned before, if that data driven approach is attempted somewhere further downstream in the product development lifecycle, it’s a little bit tough to play catch up and like develop those systems, retro actively to be able to catch data that that has come in, you might be missing a lot. And if you aren’t able to catch everything, it still might be coming in at some type of convoluted way that might not be easily workable.

10:53

Let’s go to the next slide in and talk about the next phase of the product development lifecycle, which is, you know, now that we have an hypothesis, we need to go on validate that, right, we can’t just stay in a vacuum with our ideas and not, you know, go test them out or share them with our customers, we need to engage with our customers, we need to, you know, in traditional ways, you know, we may send out customer surveys, and we may send out, we may have focus groups or you know, have test user groups that we go and engage with and look at, you know, some of the data that might be coming from those groups to determine like, what are we actually going to be building next? Or what are the next problem pain points that we might be solving for? So, so that that collection of, of ideas and activities? How can that be a more data driven approach? From your experience? And right?

11:46

Yeah, so I think, again, here, I think it’s really important to make sure that you create channels for input, right, so your primary data source, honestly, at the end of the day is going to be your users, it’s going to be your customers, it’s gonna be people that are that are continuously and constantly using your product, they’re going to be your number one data source, right, of course, they’re going to be, you know, internal, you know, there’s gonna be a lot of internal testing that happens and all that. But oftentimes, like, you know, if you’ve spent time in the product development space, you know that sometimes information that comes from any internal source, you know, you’re a little bit too close to the product, you might be a little bit biased in terms of the way that you might be interacting with or like using a product. So typically, the most valuable data will come from regular users or customers. So it’s important for you to create channels for them to actually speak to you. That means, again, touching on this idea of, you know, making sure that pipelines are in place for data to actually come in and some streamlined way that there is a system to clean that data that does come in, and then subsequently work with them, like analyze that data, for any type of decision that needs to be made on a particular feature, should it should a particular feature that that has, you know, that that has been received positively? Should we maintain that feature? Should we eliminate a feature that has maybe a bit more of like a negative reception? So it’s important to create the channels first and foremost, to be able to, like take that data in? Only then can you actually work with that data. One thing else that I think should be mentioned here is that oftentimes, when we think of data, we think of like numbers, we think of quantitative approaches, things of that nature. While that is important to remember that data can also take a qualitative form, right? So feedback from focus groups, from user groups that all informs that all typically informs the why of why you might be looking to maintain a certain feature, like eliminate a certain feature, the qualitative stuff gets more at the why I think the quantitative stuff gets more at like the, the extent to which a feature is being pursued or not pursued. And so that also is really important. And so your kind of collective set of data should include both qualitative and quantitative approaches should include both qualitative and quantitative data, because that’s, that’s going to inform how your product is being received, that’s best going to inform to the extent to which your product is has has good or bad uptake.

14:19

Alright, great. All right, cool. Let’s go on to the next slide here. So when when we’re actually so we got all this data, we got a lot of good insights from our customers, from the focus groups and some of the, you know, the things that are happening in the field. And we could actually get those insights. And we had a hypothesis and so we have some ideas, and we got some good feedback. Now it’s time to, you know, focus on the next round of features that we’re going to build, right. So we’re defining and basically, you know, building some conclusions on what’s next. And like this quote by Steve Jobs that says you know, if you Define, if you’re defining the problem correctly, you pretty much almost have the solution itself. So, you know, how do we how do we use data for your experience on forming these conclusions? And what does it actually look like in terms of, you know, what practically, what does it look like when you’re using this data and you’re building, you’re using this data to build your next round of features, or you’re using this data to actually determine and define what these, these next round of features should look like?

15:30

Yeah, the first thing is, your data should like let your data actually help, like, let your data actually speak to you let your data do its job, right in that, like, you’ve put in all this work of making sure that you’ve developed a good data driven culture within your organization or within your company or within your team, you’ve done all this work and making sure that all of your data pipelines are set to where they’re able to take in information efficiently and send information out to various teams that might be interested in that information that are looking to make data driven decisions based on that data. Let all of that happen. But then also, like work, work with your results, like work with the actual data that you have, don’t just ignore it. Remember, data can come in and various forms, it can come in qualitative ways, it can come in quantitative ways. Let’s let that let that actually let that data actually come to you. And once you have it in some workable form, actually use it and work with it. I’ve spent time on teams that collect data for the sake of collecting data. And that data might even be analyzed in some type of usable form, but the result might be ignored. And oftentimes, this happens when a particular team or a particular developer might be like really married to a particular feature, right. So like, I really liked this feature, I don’t care what the data says, I have this instinctual thing inside of me, that’s just telling that like, I think this is going to benefit customers well, so I’m just going to stick with it. I think there is some place for having that instinct, you know, and sometimes there’s a lot of innovation that comes out of maintaining an instinct that you have about a particular feature for a product. But in terms of usership. And in terms of customer experience, and the way that people are working with your products, information that comes back from customers is again, like we mentioned before, like the most valuable information that you can take in that data is the most valuable data. And so let that data actually work for you. Let that data actually speak to you and inform you on what your customers actually think about your product or your feature.

17:41

Question for you that, you know, what, what what would you do? Or kind of what have you ever seen this example where the qualitative data and the quantitative, you know, may seem at odds with one another? And what do you do in those cases? How do you how do you how do you move forward?

17:59

Yeah, yeah, no, that’s a great question. I think that happens, probably more often than then one might think you have the numbers that kind of speak one way, all every quantitative analysis point to some type of decision that should be made just based on numbers itself on a particular feature. But then you have qualitative feedback, saying that maybe this this feature might need to be modified in like some way. I think it’s important then to like design, a B testing or whatever testing you’re deciding to pursue, on on a particular feature, but construct your A B testing, or what other form of testing you decide to do in a very, like, treat it in a very smart way. Right, like, like treat it in a way where you are not necessarily providing, you know, one or two options. I know this whole, like a B testing idea has become very popular in like the product development space. And, and for good reason. It allows you to be really analytical, it’s binary, choosing one feature of another choosing one approach over another. You can you can really work with that. But I would really encourage people to think about a way to test products in a way that isn’t so binary. And that then I think gets into a little bit more of like, the qualitative side of things, the more options you provide, in terms of how a feature should be adopted, how feature might be changed, different variations of a feature, the more you’re getting at more of a qualitative response. And so you’re still you’re, you’re still kind of keeping that quantitative approach. That’s still kind of informing everything. But because you’re providing a few more variations and a few more options, you’re able to kind of break into a little bit more of like a nuanced kind of detailed understanding of what feature needs to be taken up.

19:48

Got it? Yeah, thanks for that response. One additional question on on this topic before we move to the actual the next topic, up to this point in the process, the product development process US there are, you know, there are tools that we typically use to, you know, collaborate with our customers, there’s these, there’s tools like Miro, there’s online collaboration tools. There’s the, the, you know, the actual tracking tools that we track user stories and, and features in like the JIRAs. And the Azure DevOps environments and such, from a data analytics perspective, up to this point up to kind of collecting data from, you know, hypothesis, you know, creation to collecting user feedback to actually defining and building these conclusions. What are some of the tools that that are used in the data analytics space to supplement the product development process up to this point?

20:44

Yes, I think there are a couple of different there there are, well, there are several different tools that have become a little bit more popular and ubiquitous in this space. One thing to consider is that oftentimes, when data comes in, it’s coming into some type of like database. And again, this all is very dependent on how your data engineers have like structured these various pipelines and how information is moving through through your data system. But oftentimes, what ends up happening is your data gets input in to some type of database. Database query languages, like SQL are very popular, like querying tools for not just pulling data in organized ways, but also like cleaning data and making data a little bit more usable and workable, independent of the way that data might have actually come in. And so SQL can be used as a data cleaning data munging tool. It’s a great database extraction language, it’s become very popular. So that oftentimes is used on very, you know, that oftentimes is used on or used by data driven product teams. Once that data has been extracted, and in some type of usable form. tools, like Python, languages, like Python have become really popular now, for working with data that’s taken out of some database. There are many packages within Python that have become really popular and like ubiquitous, there are many machine learning packages that have become popular within the kind of the Python world for analyzing customer data for analyzing any type of data, really. And so these tools, I think, have become a little bit more popular, like ubiquitous, whether it’s like an actual, like software product, whether it’s a digital product, whether it’s a you know, whether it’s a textile product, a physical product. So regardless of the nature of the product, these tools have become quite popular in helping analysts and data scientists work with work with data within an organization.

22:49

Got it? Yeah, thanks for that. Alright, let’s move on to the next topic. We touched on this a little bit previously about the beta testing, this this funny, this comic here, about the, you know, Guy building a, a beta version and an alpha version. He says, maybe, you know, I’m just thinking about it right now, just my mental exercise. But but but yeah, but but beta testing, and alpha testing is a real thing here. And so, you know, the whole idea, you know, from a traditional product development approach, and what we say from an agile development approach is we say, assume variability, preserve options, or, you know, you know, you defer your decisions to the last responsible moment that you want to, you want to have, you know, you want to have options, you want to want to test various options, you want to make sure that you’re not locking yourself into a single design. So agile development is all about, you know, building flexibility into your, your architecture and the ways that you look at things. So be the, you know, detrimental to your product designed to be to, to be too locked into only one way of looking at things. So the whole idea from an agile perspective is you want to learn faster about what you’re you’re building so that you can move in the right direction. And so the A B testing, or the alpha beta testing is a way to accomplish that. How can you make that more of a data driven approach? I like what it what is the data analytics approach to AB testing that you’ve seen in the past? Mr. Yeah.

24:23

So I think it’s really important to remember that any data driven approach is going to be an approach that’s dynamic. It’s going to be an approach that’s fluid. It’s going to be an approach more than anything else. That’s iterative, right? That, you know, long gone are the days that we develop products in kind of like a uniform way where our products just get pushed out. Without further feedback. I think one of the things that a data driven approach allows you to do is that it allows you to create that feedback loop meaning that like you are getting data based on how a certain feature is being taken up or not taken up, you’re then using that to provide quick changes, quick regular changes to the way that that feature was being put out, that new feature, then gets put out, gets tested, get gets tested, feedback from that, then is fed back into the loop. And so it’s that constant cycle. And that’s what we refer to when we say that data driven processes in the product development lifecycle are oftentimes iterative processes, right? That, that you have data coming in, that’s constantly informing the way that a feature is evolving. And that I think more than anything else is the benefit of, of using a data driven approach to product life cycles, it saves on costs, it saves on time, it saves on it saves, they’re less errors that happen oftentimes, when, when like communicating ideas, or, you know, updates to a feature to all the teams that might be involved. So whenever you have constant feedback loops and communication channels are being opened, oftentimes, that leads to a very optimal feature, or very optimal outcome in a much shorter period of time. And so again, for any profit driven company, that’s huge, right? saving costs on product development, just by undergoing a specific process. And foregoing a less efficient process, in and of itself can lead to savings.

26:27

from a tactical standpoint, in some examples, you know, I, you know, I’m familiar with like some product telemetry that you can kind of build in, you know, measures and ways of kind of measuring feature usage on certain products or, you know, performance in certain areas of the system. And you can have, you know, you know, real time health monitors report out how certain aspects of the system are doing, you know, Are these some of the data collector opportunities that you’re referring to when you have these different ad versions out there, and anything else that you could give as an example of how you would actually collect real date real time data from the products that are out in the field?

27:07

Yeah, so yeah, I mean, that’s, that’s a very popular approach to date. Right, AV testing is, you know, oftentimes, like digital products, software, that that is a good approach. You know, we had talked about this idea of making sure that you embed qualitative testing to into your, into your product development cycle, that also is really, really important, because like we had mentioned before, that it’s the qualitative information that oftentimes gets at the Y, right, which is, again, very useful, very informative. But remember, also that a lot of this develop a lot of this feedback and the nature of the data that you get back, whether it’s quantitative data, whether it’s qualitative data, really depends on building relationships with users really depends on building relationships with relationships with customers. And so good, trustworthy relationships are essential to all of this. Because if you don’t have relationships with customers, you’re not getting any data back, regardless of how strong your data pipeline is, regardless of the all the great, you know, testing tools that you’re putting out there all the great service that’s putting out there, if you’re not getting response back, and feedback from your responses back and feedback from your customers, then all this kind of goes to waste. So it’s really, really important that you have a team, or set of teams that is working specifically on creating buy in right, creating a good customer base, where you have a good relationship with your customer, that is really, really important. New Balance does really well and the shoe company, new balance has, they have product testers, you can sign up to be a product tester. In that way. They’re they’re trying to like create relationships with customers, right? They’re they’re trying to create a cadre of people that are interested in providing regular real time feedback. For for apparel, again, this is not these these aren’t digital products. These aren’t software products. But the idea is to say, right, that it’s important to create a community of users that you can depend on. And that relies heavily on developing good relationships with customers.

29:18

Good example. Any, any Have you heard of any experience where you have bias and these data coming back? So how do you how do you get around that there’s just you know, maybe like you said that you have a customer, they you know, that you got this good relationship with, but they’re not giving you honest feedback. Because it’s such a good relationship. How do you how do you navigate bias and some of these data sets that you’re getting coming back from from some of these products in the field?

29:51

Yeah. So you know, whether you have five inputs or whether you have a million inputs, you know, a million records in your data set. There’s always going to Do some bias, right to eliminate bias completely is impossible, right? And those of you that that have an appreciation for the way that analytics is used to drive decisions, through hypothesis testing, and through other statistical techniques know that eliminating bias 100% is absolutely impossible, right? There are techniques that you can do to reduce bias, that’s always possible, right? And, you know, again, the more data that you collect, the more random your data is, the more the more your sample population mirrors your or the more your sample mirrors your population, the more the more bias, you reduce it, right. So again, yeah, we can, we can talk about the numbers, but, you know, the more the more data you have coming in, and the more random that data is, and the more representative that data is of your population, the more you’re reducing bias. So sometimes bias may not be easily eliminated from qualitative data input. But oftentimes, through the use of various statistical techniques can be very much reduced through through quantitative data intake.

31:09

In this process, when you’re evaluating some of the A B testing results, any tool or tools that you’ve used, or you know that that’s popular tools that in the market that are being used to analyze some of these, you know, maybe it’s big, these are big data sets here that we’re talking about, when you got a lot of, you know, you have these, this stuff, you got these products that are in production now generating a lot a lot of data, what are some of the popular tools that are being used in industry to kind of crunch this data, clean it, and to make sense

31:40

out of it? Yeah, so, you know, I had mentioned, I’m really biased, because I program almost exclusively in Python, but, you know, Python has become one of those tools. Again, like I mentioned before, that’s become really ubiquitous, very universal, you know, it’s, it’s an it’s object oriented programming language. It’s very robust. There are many libraries that have been developed out to handle various, various types of projects, it can be used as a front end language, you can use as a back end language, it’s used a lot in the data analytics and data science space, I would, again, I might be biased, but I would really recommend using a tool like Python to be able to kind of manage your data, clean data, analyze data, that has become a really, really popular tool, and is used, I think, in a lot of different types of companies. You know, the, the, the title of like a Python programmer, or a data scientist, you know, these, these, these, these job titles have become a little bit more prevalent in the last five or six years, although, you know, tasks and functions that data scientists might take on, like, that’s always kind of been around. But these, this work has been a bit more formalized now. And that just goes to show you the extent to which the extent to which data is now being used, and the extent to which data is being taken seriously, to make decisions. A lot of the work of the data analytics and data science space, has become much more formalized now. Whereas before, a lot of these functions might have been disparate in terms like living within various functional units or living within various teams, now you have entire D teams that are dedicated to the analysis of data to drive decisions. And so that’s a really important, I think, evolution that’s happened. And that’s, of course, being applied to the product development space, you know, in the last several years.

33:36

Got it. All right, good discussion here. So let’s talk about the continuous nature of this this development cycle. And which kind of this image here kind of, you know, kind of alludes to that, you know, it’s never ending the infinite loop, if you’re getting customer feedback is going back into the pipeline for you know, new hypothesis testing, new user story creation improvements, new ideas, new planning, new A B tests happening. In agile development, we kind of call this the continuous delivery cycle or to continuous, like the DevOps cycle that, you know, you got a production environment, that you’re continually getting feedback from your, what your products are actually doing in production in the customers, hands, and also to hearing it from your customers themselves from like that qualitative data that you talked about. So all this data is coming back from the field, and it’s going right back into the development cycle. And it’s just never ending, from a data, a data analytics standpoint. How do you how do you make sense of all this? How do you is there ways of measuring this? How effective that you are? And in this cycle here of product development, this feedback cycle?

34:49

Absolutely. So you know, as I look through this, this product feedback cycle, I’m thinking about all the ways that data can inform every single one of these processes every single one of these steps, right? So again, even at the planning stage, right, again, like we had mentioned at the very outset of this, of this webinar, this idea of making sure that processes are being data driven, that needs to go into the very planning of your, of your product development, right? To the, to the extent to where, or when you’re at the part where you are performing continuous testing, that then gets back to this idea of like, iterating and iterating, constantly, right, this idea that you are putting out a feature, collecting data on it, using that to inform some like, next version of your feature, getting data from that, plugging it back in, coming back through this feedback loop. That’s all really important. And so, you know, the iteration that is, that is defining, like, that’s, that’s that, it’s the iteration, that’s kind of the defining element of any data driven process, right. And that allows you to, again, like I mentioned before, create a more efficient process, get get a more optimal product out quicker. All that happens when, when data is layered on top of all these steps. And that’s really, at the end of the day, what it means to take a data driven approach that, that, you know, keeping in mind that none of this stuff is isolated. All all steps relate back to other steps. They’re all dependent on other steps. If a step was data driven before, then more than likely that, that following steps will also be data driven, though, they’ll feed back into previous steps seamlessly. And so it really is important to make sure that there isn’t a single one of these steps that is more data driven than any other step, it’s important to make sure that you take an overall data driven approach that data is really driving each and every single one of these steps in the development cycle.

36:55

Yeah, that’s that actually, you know, gives me a, you know, a point that I want to emphasize here as well, that, you know, one of the DevOps principles is to measure to use metrics to measure and to measure in all of these areas, to have measures that that tell you that you’re doing things well, that you’re, you know, you’re delivering high quality products to your customer, that your development is high quality that you’re acting on, you know, customer feedback. And so the measures are key. And so, you know, you know, what I’m, what I’m hearing from you too, is that, from a data driven approach to product development, you know, and data analytics should, you know, it should supplement this whole process. So, quick question for you, in this cycle, you know, knowing that this is a continuous cycle, and you know, all these aspects have to work together. Have you seen any data analytics approaches to helping this entire cycle be better performing? I mean, so it’s not, you know, I obviously, at the end of the day, we got to build better we want our products need to be good, but, but also to just the way that we work in terms of how this whole cycle works together. Any analytics around that space that you’ve seen or heard about in the past that, you know, companies have used to make the organization function better?

38:18

Yeah, I mean, you don’t there’s, I think, this is where like education and training, like really, really comes in, right. Like, it’s really, really important to make sure that you have a proper understanding of not just like the more technical like nuances of like working with data and understanding the statistical techniques that are like, that are happening under the hood of any like, Python algorithm or anything like that, like, yes, it’s important to understand those technical things. But I think it’s also really important for people to understand data, meaning, like, understand results, and then more than just understanding results, how did they implement decisions or make decisions based on those results? And so, you know, in a lot of the classes that I teach, I focus on kind of technical aspects of like programming, like, is it important to understand, like, how to program well, in Python, is it important to understand some of the more statistical nuances and frameworks that kind of underlying some of these algorithms that we work in, like machine learning libraries and things like that in Python, is important to understand the more like technical programming or like coding aspect of that work? Absolutely. But I think what’s just as important is being able to communicate your data effectively, like being able to communicate results, effectively being able to look at a spreadsheet or a data frame or some type of chart or visualization, and be able to draw conclusions from that and be able to be able to relay that conclusion be able to explain off that conclusion that you’re making based on data to a team that may not be as technologically sound advanced, that may not be as as as as knowledgeable about some of these techniques right? Being able to explain with being able to explain your data, being able to explain the results of your analysis, I think that is just as critical. And the extent to which a non technical team, like you’re only as strong as your like weakest user, right. And so anywhere in this lifecycle, if there’s a gap in understanding data and how it’s being used, then oftentimes, this cycle doesn’t really work as efficient, right. And that’s why education training is so important, it’s important to if even if you’re not working with the more like technical aspects of developing a product, or the technical aspects of like, analyzing data, even if you’re not working with that, I think it’s incumbent upon upon all people, all teams that are part of an organization that claims to be data driven, that you are at least knowledgeable about the way that data is being used.

40:52

Yes, that’s an excellent point. Yeah, you know, the communication aspect of it is so key. Before we kind of finalize up to, you know, one point I want to touch on on that actual aspect of it, data visualization techniques, because presenting, you know, data may be in pictures or in certain formats, is more conducive for users to be able to, you know, grasp what’s happening, any any experience in that space in terms of some tools that can help, you know, end users to to better grasp the data that is being produced for them.

41:29

Yeah, I mean, there are a lot of tools out there now, for, for data visualization. There are, you know, libraries within Python, of course, that would involve, you know, knowledge of coding and like working with those particular libraries that that you can use to create like, really, really compelling visualizations. There are code free, or essentially code free software platforms like Tableau, that have become really, really popular in communicating data in the form of dashboards and, and other types of visualizations. You know, Tableau has done really well for itself as an organization, because it has, it has really done a really good job of kind of developing a very, like code free platform for putting out compelling visualizations. And so again, no matter how you’re putting out your data visualization, I think more than anything else, it’s important to understand how that data is being used, how it should be used, how it should inform any, any decision moving forward. But you know, there are a number of tools out there already that have become quite popular in, in kind of the data visualization space.

42:41

All right, great. All right. Um, so that pretty much finishes the topic of that we want to cover kind of the full lifecycle all the way to the end of data visualization, and how the customers would end up using the data. So we’re going to actually check the chat and actually address some of the questions that came into the chat. Um, any any feedback from the chat and check in the chat right now? Yeah, this is a good opportunity for if anybody is on the call right now, you can type something into the chat. We can address any particular questions or any particular you know, feedback that you want to get, you know, from Enron or AI or about about a data driven product product development approach. Alright. I will wait a minute here before we then transition.

43:51

What I’ll do is I’ll go to the next slide, though, just to get ready to kind of close out in the meantime, one one aspect that we did want to let you guys know about is that there are some we have a data analytics Bootcamp on the website on the AI for group the eye for group.com website. And we take an agile approach to this and some of the the aspects that we covered in the actual webinar are in our in this boot camp, the boot camp is broken down into four distinct classes, which Python development is a key part of it, you know, using Tableau, which Imran mentioned also to some advanced Python scripting with machine learning and also to from the front end aspect, getting that data out from a database from SQL or the Mr. Do you want to expand on any of the course offerings to you? You’re obviously the lead instructor on the journey. Any feedback on this?

44:47

Yeah. The data analytics with Python we cover most of the most of like the more popular packages, pandas NumPy you know, visualization packages like Seabourn And in matplotlib, the ones that have just become like really, really popular, and we’ll cover several of those in data analytics with Python. With in the tableau class, we’ll talk a lot about like data visualization philosophy, what makes for good visualizations, what makes for bad visualizations. Turns out in the world, there are a lot of visualizations that are out there, they’re just not so good. So we’ll talk about what makes for good visualization in a more philosophical way. And then we’ll talk then about specifically applying those good data visualization practices to Tableau into creating visualizations and dashboards, dynamic dashboards within Tableau, data and analytics with machine learning focuses on modern predictive analytics techniques, we use Scikit, learn SK learn is kind of the main package, that’s probably the most popular machine learning package in in Python. So we spend a lot of time with that. And then in SQL, we go through some of the more popular fundamental query commands. And so will we spend some time going through that we talk a lot about how to export out clean files from a database, and how to get you know, and even an even clean files using using various SQL queries within the database itself. So we talk a lot about various functions, commands within SQL that that help you do all of that. So from beginning to end from like, from like database, creation, to database cleaning, to database extraction, to extracting into extracting data into Python, so you can work with it in Python, from cleaning, and python two, analyzing python two, creating visualization in Python, two, then exporting out of Python into Tableau to create visualizations, that whole kind of data analytics cycle that that project lifecycle is intended to be covered through these through these various classes. All right,

47:07

great. So yeah, and you also get a certificate of completion for the boot camp. So that’s, you know, good, you know, good good certificate to reference that you’ve gone to this full boot camp from qualified, you know, trainers such as Enron, and yeah, just definitely might be a something that you would be interested in. So Well, with that, I still don’t see any other questions that have come in. No other questions that have come in. But just to kind of give you to give you kind of a summary of the webinar, you can see the URLs to the i Four group.com website, which we mentioned, we have the data analytics bootcamp courses up there, as well as the build Well, labs link to to Emraan site and some get a GitHub repository as well, for some development snippets, I assume if anyone wants any, some examples, so But thank you, everyone, it was definitely enjoyable. We’ll, we’ll end it here if there’s no other questions, um, but yet, it’s been a pleasure. Thank you, Emraan, for your time today, and thank you everyone for joining. Again, this will be out on a recording as well. And we’ll send it out to you all, and you also too, guys will be included in a just keeping you informed at some of our other webinars coming up and some of the any classes that we might have coming up on the horizon as well. So again, once again, thanks, everyone, and everyone, have a great day. Thanks a lot. Thanks, everyone. Yep, thank you all. Have a good one. Bye.