Scott Rayburn: Hello. Welcome to today's webinar, presented by DataArt and Microsoft. My name is Scott Rayburn, and I'll be your host for this exciting journey into the world of building generative AI solutions on Microsoft Azure. Before introducing our fantastic speakers, Dmytro and Amanda, let me take a moment to introduce our topic.
Today is ChatGPT's one-year anniversary, which makes it the perfect day for a webinar about the power of generative AI and the ecosystem of technologies like Microsoft Azure and service providers like DataArt that are enabling its implementation.
Here are a couple of statistics on the economic opportunity: IDC projects that generative AI will add nearly $10 trillion to global GDP over the next ten years, and according to a study sponsored by Microsoft released earlier this month, companies are already realizing a 3.5x return on investment for every dollar they spend. However, according to an AI survey, 68% of UK business leaders face challenges in adopting generative AI due to uncertainty around the technology.
Indeed, it's challenging for many businesses to figure out where to start on their transformative journey and how to use generative AI to effectively drive growth, enhance productivity, and stay ahead of the competition. Today, we're here to bridge that gap. DataArt and Microsoft have partnered for over 20 years, developing innovative technology solutions for leading companies.
Now we've joined forces to present you with insights, strategies, and practical approaches to harness the power of generative AI on Azure. Without further ado, I'm pleased to introduce our distinguished speakers: Amanda Wong, AI/ML Cloud Solutions Architect at Microsoft, and Dmytro Baikov, Technical Director of AI/ML from DataArt. Let's dive into this exciting topic. Amanda, the floor is yours to tell us about all things Azure AI. Thank you.
Amanda Wong: Thank you, Scott. First, I'm going to set the scene a bit. You mentioned that ChatGPT entered the headlines a year ago from today, and since then, we've seen quite a storm in generative AI headlines and news reports. It's unavoidable right now. We're seeing that it's not only about hype but also real-world applications with customers applying it to end users.
I have a couple of examples here on the screen. CarMax used content generation with GPT models to drastically reduce the amount of time it took to generate summaries and descriptions for used cars for their customers in a matter of days. The second example I want to highlight is Progressive Insurance, which is helping to save $10 million annually with AI-powered chatbots to help users answer questions about their insurance coverage and customer scenarios.
Before I get into the details of how this has happened and then hand it over to Dmytro to talk about practical implementation, I want to give some background about Microsoft Cloud and some of the core pillars of our mission, and what goes into how we develop our services. Our mission is to empower every person and organization on the planet. With the era of AI, this mission is more important than ever. We are committed to working with partners like DataArt to deliver this to customers and more people.
One of our core partnerships that is of particular relevance to this talk is with OpenAI. We've been working with them to advance AI research, develop AI computing platforms, and make this technology more accessible to all types of developers, whether they are new to coding or more sophisticated data scientists.
I'll be talking about some of the tools and services that we've built to integrate natively with existing services and third-party frameworks to deliver new applications. We have our OpenAI partnership and several other partnerships across different technology sectors to enable customer choice and ensure that we stay at the edge of the most innovative technology.
This includes the popular Hugging Face framework, which delivers open-source models and datasets on our Azure platform for development. For example, we are also working with Meta to offer the Llama 2 model for fine-tuning. Most recently announced at our Ignite conference a few weeks ago, we're offering Llama 2 as a pay-as-you-go inference service.
We understand that customers may want to enable the choice of multiple generative AI models aside from GPT. Meta Llama 2 is one example of those. We also partner with Nvidia to offer specialized AI computing platforms to train models and make inferences. But we know we can't do all of this alone, which is why we partner with folks like DataArt, a Microsoft partner, and an Azure Consulting partner for data protection and data management services.
Dmytro will discuss our collaboration and some real-world scenarios and use cases. But I want to discuss some of the technology that is most applicable here. Our location in the landscape is in the bottom right corner, where you see the highlighted purple Azure OpenAI service.
This sits in our customizable AI models. You'll notice that in this stack, a lot of this technology is embedded into existing tools and services across Azure, across our application platform for business users and analytics folks, as well as our scenario-based services for specific end-to-end scenarios, such as bots or extracting information from documents such as PDFs or Word documents, or analyzing chapters and transcripts from videos through Video Indexer and analyzing content and helping to determine decisions or detect anomalies through services like Metrics Advisor as well as our Decision Cognitive Services.
The machine learning platform is the base layer of this stack, and it is one of the focuses of today's talk. How can we customize this technology for your company data or a specific use case?
How have folks already implemented some of this technology? I gave two very brief examples earlier, but I also wanted to make this feel more real with a use case from H&R Block. H&R Block uses technologies such as Document Intelligence to extract tables, paragraphs, and other types of key information from massive amounts of forms for classification.
Using Cognitive Search to enable customer queries in a more contextualized understanding delivers more relevant and effective results. You can see here that the amount of tax documents they were able to classify was 30 million per year, which is a drastic difference from what they were able to do manually. We published this full story on the Microsoft blog.
I highly encourage you to check that out if you're interested in understanding how this was happening with Cognitive Search and OpenAI models. Still, please stick around for some of the scenarios we'll also discuss with DataArt. Those are some examples of what has already been built, but the focus is on what you will build and what you will contribute to the ever-evolving generative AI landscape.
Some of the focuses that I want to highlight are contextual interactions. How can you support your customers in ways that can both acquire and retain talent and engagement? Many customers are changing their expectations to expect more personalized recommendations, more effective results when interacting with customer service or chatbots, or even a simpler website for your company, as well as amplified automation.
What kinds of tasks can be streamlined and delivered at scale to make jobs easier and customers more excited about engaging with your product? Finally, I wanted to highlight intuitive discovery. In what ways can we utilize search capabilities to have your customers figure out what they're going to discover next, and how they can use information to help develop their own goals or technologies and services as well?
I provided some background about the Azure AI platform and what some companies have been doing with this technology, but I'm going to talk a bit more about what concrete technology looks like and what you can do. At a high level with the Azure OpenAI service, you see that we have several models here, and these are constantly changing.
Even earlier this month, OpenAI Day announced some more recent models. Those models have since been included in Azure OpenAI and are now available for testing and use. We're going to talk about the GPT models more in-depth in a bit. This is a high-level view of some of those chat models that are underlying some of the more popular interactions you may have had with a ChatGPT interface.
These are the same models within an Azure context and can be utilized for similar applications in a chatbot or text completion scenario. On the right, not only do we have text completion, but we also have generative AI image generation and understanding of audio transcripts and recordings for translation with the Whisper model.
I also wanted to highlight that with all of these models, Prompt Flow on Azure has recently been made generally available to help scale workflows and prompt engineering to iterate on the development of these models to connect them to your broader application development services, as well as third-party plugins with functions.
I want to take a moment to recognize GPT-4 and discuss how it is a shift from some of the previous GPT models we may have seen on Azure. With GPT-4, this is really the next level in text generation. There are several reasons for this. First, it is trained to generate more complex documents and accept much longer inputs.
With prompt engineering, it can take in more nuanced instructions to deliver results that feel more specific, relevant, and personalized, as well as translate it into multiple languages and understand contextualized everyday language from customers. But GPT models are made even more powerful when they are paired with services to extract information from existing types of documents.
Take, for example, an unstructured document, a PDF, or even a presentation. How can we pair some of the amazing capabilities of these Azure OpenAI models with search capabilities to traverse all of that information, turn it into knowledge, and generate summaries to answer questions? All of this can be done with the broader landscape of connecting these different technologies here.
How can we make this feel more real for industries, for example? Let's speak to financial industries, what we're seeing with banking, and some inspiration for where you can take some of this technology. For example, there's client engagement. How can we help enable advisors to offer recommendations that feel right to their customers regarding timing and context?
There are also opportunities to understand markets better and to enable predictions that inform present actions, as well as supplement investor report summarization and making this more accessible to all types of customers through translation and other forms of visual accessibility.
In the retail space, companies are taking GPT models and using them for product innovation, such as analyzing market trends, similarly to what we saw in banking. But with customer and retail trends, there are also opportunities to identify anomalies in manufacturing at the front line, such as production errors or defects, using computer vision models and analysis.
There are also ways to generate more evergreen content to deliver fresh, relevant content throughout the year, regardless of season, and flexible to different product offerings. At the frontline with field sellers, you can help them customize scenarios when working with potential leads and converting them into sales.
Finally, the last industry that I want to highlight before passing it over to Dmytro is insurance use cases. I spoke to H&R Block and Progressive, and similarly, here at a high level, what we can automate is claims processing with Document Intelligence, the PDF extractor and document extractor I mentioned earlier, as well as search abilities.
You can also leverage GPT models for fraud detection, similar to anomaly detection, to ensure you are processing insurance documents and not losing money. You can also increase customer satisfaction and improve the experience of interacting with insurance documents with your insurance provider by accurately answering customer queries.
But that's not all. I want to turn it back to your scenario and your data. Here's a high-level diagram of how this works with taking data sources, regardless of where they sit in SQL, in Cosmos DB, or additional third-party data sources. We want to enable you to bring your data wherever it sits, wherever you feel comfortable keeping it secure and pairing it with the Azure OpenAI service on your data to customize it even further than what it's like out of the box for your application.
Where can you do this? We have made Azure AI Studio generally available for everyone to build and train their own models to perform customization. I just spoke to some of the models in our model catalog, whether GPT models or Llama 2 or models from Hugging Face, for example. You can pair this with built-in vector indexing or other native tools to optimize your workflow for data science, whether it's low-code or no-code.
I can't speak about this without acknowledging that there will be so much content in this generative AI era, whether it's input or output. With AI, we want to make sure that we are creating content and contributing content to the world that is responsible and can have moderation tools to traverse all of the content that's being built effectively.
The Azure Content Safety Tool, which is generally available now, moderates across four different categories of harm with different thresholds. This can help you customize it for your use case to prevent misintended technology applications. We're doing this because our responsible AI principles are at the core of all of our AI development.
We want to make sure that Microsoft Cloud is a partner that you can trust with your data. Your data is not used to train OpenAI foundation models without your permission, and it is not going to be used for our own internal services. Your data is protected in your tenant with some of the most comprehensive enterprise compliance and security controls, and we want to make sure that responsible AI is implemented at every step of model development and deployment to your end users.
That's all I have for now. I'm going to pass it back to Scott. That is at a high level, Azure OpenAI.
Scott Rayburn: Amazing. Thank you, Amanda, for that robust tour through all things Azure AI, including the newest developments, use cases, and general overview. While we transition to Dmytro, I wanted to ask you personally about your experience working with DataArt as a partner to implement some of the things you just discussed.
Amanda Wong: Yeah. I've worked with DataArt to develop offerings for generative AI, broader AI, LLMs, Document Intelligence, and various industries. I'm excited to speak with Dmytro a bit more about how we can scale proofs of concept, for example, for customers to get a better feel for how this technology can work on their specific data.
Scott Rayburn: Awesome. DataArt is all about making this stuff real for our customers. Speaking of proofs of concept, I think Dmytro has some of those to tell us about. A demo, all sorts of good stuff. Without further ado, let's hand it over to Dmytro.
Dmytro Baikov: Thank you, Scott, and thank you, Amanda. Hello, everyone. Before we dive into the solutions, I think it's very important to highlight that things like privacy, bringing your data, and responsible AI are three top pillars of what we see our clients ask for. That's what we do when we start developing the AI journey AI project.
We always think about the data where it's stored. Is it private or not private? What are the data controls, and how will these models be used? Are they secure and private? I will speak about all these projects that are working in this domain, so it's quite safe to develop solutions there. There are so many applications, which Amanda also mentioned.
I will be speaking regarding our clients and the feedback we got from the markets. We operate in different kinds of industries. That's why I will speak more generally in a horizontal direction. I think the main key areas where we find our clients’ requests are chatbots, document processing or document intelligence, different types of smart search, coding assistance, and of course, other ways which may be similar to these or maybe as a different category.
A couple of examples of chatbots. Chatbots are a huge tool. They have already been with us for years. But now, the generative AI approach, we mentioned that. We started to think differently about how these chatbots may reply and what they can do. Starting from support, from helpdesk teams for classification of the tickets, understanding what this ticket should do, redirection of the ticket, triaging, and all the way through knowledge base, and how to build the chatbots for easy access to your knowledge base.
What do we have? The call center. What if we want to bring this data closer to our support agents? All of these use cases are powered by AI, and they really give speed, a performance boost, and accuracy in replies. They upskill your employees, and they upskill you as well. If you don't know the hidden gems inside your company or your organization, you can find out if they are in the chatbots.
I would say this is a straightforward use case, but it really gives value. You can tune this chatbot, you can work with different personalities, internal types, and external types. This is the whole range of systems you can build there. Of course, document processing is a crucial part of the chatbots. However, these documents are not the only building blocks of some organizations. They work with these documents the whole time. The big question years ago was how to get the data out of these documents.
There were some advancements in NLP, which got us from document classification to some data extraction. But right now, with OpenAI models and generative AI, we can do it much faster. We can extend it to different types of formats. The same model can be used for emails, documents, dialogs, and chats. We can classify them. We can generate the emails and extract the complex data structures we need to train the previous models. We need to label this data. But now it's very easy to start, and very easy to integrate and see the real impact.
Speaking about the physical model, how can we digitalize the physical model, and what can we do with generative AI? Imagine you scan your email and then digitize it. You get the text, you generate the reply automatically, print it, and you can send it right away. It influences the physical world as well with fewer human efforts. That's what we observed in banks and insurance organizations across the globe.
Search is also the core part. If you have good search algorithms, if you have them, you have a good recommendation engine, and you can match people better with each other. You can match the content to the people; you can match the people with each other. You can explain this match as well. These are all the use cases we observed as well in different industries, starting from healthcare. How can we match the patients and doctors better, how can we explain the query from the patient to the doctor, and vice versa?
How to search across different spaces, including the big spaces of documents. Basically, you can think about it. Imagine you have shared drives on your laptop where you can access everything you have there, including your presentation and your documents. This is what can be done on the organization level, on the product level, and with the organizational data in your profile data as well.
This is also a big direction in which we find the interest of our clients. Embedding models also opens huge opportunities for working with text data in search terms. One unusual way of working with generative models is the code assistance tasks. Starting from GitHub Copilot, which helps a lot to boost a lot of things here, we can think about tiny improvements as well.
Think about security scanning, small, new security scanning tools, for instance, for languages that are not supported by GitHub Copilot, or explainability of the code. Again, very specific code, documentation generation, and even legacy code migration are generated in a way that generates a file or several files. It splits the files; it optimizes the files. All of that is starting to roll out in different tools as well. But even now, you can do it in Jupyter Notebooks, all of that. Again with the Azure, deploy OpenAI models. It can be securely integrated into your CI/CD pipelines, to different repositories, and with your specific needs, in case the tools on the market do not address this need.
This is a pretty in-depth explanation. The question may arise about building these chatbots and documents, and how to do this smart search. What we suggest and how we do that. We have four stages. Basically, as soon as you know your use case, imagine you want to build a document processing solution. You define this use case. Then we go to the prototyping mode. AI has uncertainties sometimes, and we need to prove that it works. Test it on a lot of data. Test how responsive it is. Then we want to scale that.
In 6 to 8 weeks, we will do the prototyping part. We evaluate different models. We try different approaches. We do prompt engineering. Then, we build a clickable prototype, which we show to the business and the clients and ask for feedback. In some feedback iterations, we go to the MVP development. We now understand that this chatbot or document processing part is working. We bring it to millions of daily requests, different clients, organizations, hospitals, and patients.
All that is starting to be the engineering part of the AI solution, and we call it MVP when we move something to production and develop it as scalable. We test different datasets, we improve, and the research continues. But now, we focus on this part: how to bring it closer to the customer, do A/B testing, assess the results, and then roll out to the whole system eventually.
That's where we go live, actually. We collect feedback now from the real users. We understand their needs and tune the model and approach later in the support phases. This is our journey flow and journey. This is only for the first use case. What if you have several use cases and want to power your organization with AI, and you have just startedY? You have the first great results, and how do you build it on scale?
Of course, you need to scale your data science team, but from the engineering perspective, we think about that as a platform. You need to have one single place, one data lake. They bring a data engineering, scalable approach to building projects. Again, it's not only about generative AI, it's about AI overall. How to have access to the data, how to build these use cases on the scale and how to move from one use case in production to ten use cases in production.
That's what we call the AI platform. The next level after that is what we call the AI factory. Imagine you have already developed one document processing solution and have several departments. You can reuse this accelerator. You can reuse that approach for different types of data and tune it inside your organization, basically without integration and without implementing the new code.
You build your new AI use cases so they are also scalable to different data. That's what we call the AI factory, which immediately lets you scale from five use cases to tens of use cases across different departments. With that, I want to show our AI solution and how we move to production, basically in this 1 to 3 months timeframe.
This is DataArt chatbot. If you go to our website, dataart.com, it is public. You can do it right now. I'm sure that OpenAI and Azure OpenAI will be ready for that. This is live. You can update the page, and this is working. You can see it in the chatbot button at the bottom right corner. This is our chatbot, which we basically developed, and it's trained on DataArt data. It knows about DataArt. It has information up to the summer and it knows everything that happens. It knows our website.
I prepared a couple of questions for this chatbot so we can understand how it works. I want to ask what you can tell me about the Microsoft and DataArt partnership. Let's see what it can come up with. You can try your own questions. We have a lot of interesting things included here, like guardrails and responsible AI. It can reflect your questions as well.
But first, the idea is to show what DataArt is. You can see basically what Scott mentioned in the beginning: that we have a 20-year collaboration history. We are different types of partners and we develop across the globe, etc. You have this overview of basically some pages of our website available. This is the best knowledge base you can have.
Let's try something else. Let's try the other question. Do you know how DataArt does generative AI projects? Let's try something more related to our topic today. This is on our website. We have AI services. You can go there. You can check the content. But that's why you need that. If you have this chatbot, which will think a bit and then do the summary for you based on the same documents that you will try to find and look for.
It thinks a bit and then returns a pretty big understanding. As you can see, we approach you in AI with this type of thing. We have an AI solution, an AI platform, and an AI factory. Basically, that's what we described. We have discovery workshops, we can prototype in six weeks, etc. This is right there, and this is available.
I will ask something more tricky. Do you know anything about the DataArt AI webinar and Azure in November? Let's try something like that. This is again to show that it's not ideal. Of course, it will not respond to everything. It will be, you know, it still needs to be up to date. It still needs to understand what you are doing. You can try to ask the questions, and it may involve thinking of a few different topics.
It's still sometimes random, but overall, as you see, it still gives you some response. In this case, it doesn't have any specific information. But you can contact us and you can ask for any information you need, and also about Azure and Microsoft. This is how it is designed to give you the real link to your email addresses; it doesn't harm anyone if it doesn't know the information.
It's based on responsible AI techniques. I encourage everyone to try it on their own. This is a small demo from my side. I think it's back to Scott. We are ready for the questions.
Scott Rayburn: Great. As a representative of the DataArt marketing team, I must say I love the chatbot. It's trained with all of the materials we work so hard on creating, and it actually uses them in a new and innovative way. Please do check that out as we get organized for the Q&A. Actually, Dmytro, let me ask you about this QR code here. Something about a six-week prototype. What's that all about?
Dmytro Baikov: Yeah, exactly. That's what I mentioned as well. We actually prepared this Azure AI marketplace offering for you. You can reach out to us through the website. It's deployed on Azure Marketplace. We offer this six-week generative AI solution if you're interested in something I already showed or any use case we described.
You can apply through the form, and we will reach out to you and speak about your use case, your organization, and how to build it for you. This is our small offering here. I encourage everyone interested to try it out, back to you.
Scott Rayburn: Perfect. Thank you, Dmytro. Let's get started with the Q&A here. Amanda, I have a question for you. As the moderator, I know you have a front-row seat to all this generative AI craziness, and I was just curious how you separate the hype from reality. How do you think about all this stuff that's happened in the last year?
Amanda Wong: That's a great question, especially since there have been so many important developments within the past year. I do my best to stay up to date with reading about industry trends. But also I do have a front-row seat to the internal developments with our generative AI technology, but what helps make it feel more real and less like hype is when folks like DataArt bring customers to us and we get to talk about them, we get to think through what are some trade-offs of different technologies? What is the best path to delivering a solution to a problem? Talking to customers, thinking through problems like that, testing different technologies to see how they work, and comparing results are really exciting. That's what makes it feel a lot more real for me.
Scott Rayburn: Nice. You have the toolbox at your disposal. I just have to pick the right tools, the right partners, and the right use cases. Sounds really interesting. Another question here. I think maybe both of you might have thought about this. How would fraud detection work for an insurer? I guess that means fraud detection with generative AI. Amanda, do you want to start?
Amanda Wong: I can speak to that at a high level. Dmytro, I'm unsure if you want to speak to some of the more concrete examples you worked with, with proof of concept. Still, on the technology side, with insurance fraud detection, for example, you can extract information from large amounts of forms from, for example, a user's history, or you can aggregate findings of forms or cases that have been labeled as fraud, for example, and that can help train the model to identify through a classification system.
Is that similar to previous training examples or not? If there is an anomaly in a user's history, or if it flags it as very similar to labeled cases of fraud, then that's how it can surface at scale what may be different and worthy of further human review.
Dmytro Baikov: I can also add to this one. Basically, what we observe is that a lot of companies already have fraud detection systems working. We have been building these systems for years already. We see the trend where generative AI can detect fraud and explain why it's fraud, and it can explain in a human-readable manner.
Also, it can analyze big amounts of decisions made by some classical model and again, explain the bunch of decisions so it can do the summary about these decisions, about one specific organization, or analyze the whole transactions about one specific client for the last three months and do some statistics and do some explainability. That's where we see the power of generative AI. That's where we see it in conjunction with the classical AI approaches, which is also an interesting part of it.
Scott Rayburn: That explainability sounds great to me. I tried to cash a check, and my bank froze my account for a whole week, and no one could explain it, but they just said, “Nope, you're done for a week”. So that was interesting to me. Now, for my next question. Actually, this is more of a comment, but I think it's a really interesting one. If AI becomes the norm to generate email responses and other content, what if we all start to sound the same? Is it going to change the way we think in society drastically?
Dmytro Baikov: I love this kind of question. I can try to answer this one. It depends on how far in the future you want to go. We can go one month, one year, and ten years, which will all vary. I think that in ten years, you can train AI to speak in your terms. You take your Facebook, you take your messengers, and it will speak as you write in a more, closer perspective. You can tune prompts and the temperature, depending on how you want to react to your responses.
I just say sounds the same in different levels but differently. You will tune your AI as you want. In the distant future, AI can speak instead of you as well. With voice exactly and with text exactly as well. There are always some models that can be tuned to your voice and speak in your words. This is not in the years that you thought, maybe three years from now.
Amanda Wong: I would add to Dmytro's point that I think we are seeing a shift in attitudes towards AI with the rise of different types of copilots or assistants to help you do work that can help you save time overall. Often, AI, generative AI in particular, can be used to augment daily tasks rather than replace people or jobs.
It really depends on how they are used by end users, not necessarily to replace the way that people speak, for example, but to start, to provide a starting point for how to draft an email, for example, or create a proposal. From there, it can work with the human review or the end user to determine what feels right and appropriate for the use case.
To Dmytro's point, you can help streamline that connection and that collaboration between AI and humans through your own data. But that is the dynamic. We're seeing a shift in that direction as well.
Scott Rayburn: Great. Thank you both for your answers to that question. Moving on. Someone asks what type of knowledge-based document can be used for chatbots? Do they have to be SharePoint PDF documents? I guess at this point, you can use many different types of documents. Maybe another side of the question is, where are the limitations? What can't be used as a knowledge base?
Dmytro Baikov: I think there are specifics from my standpoint, for instance, wanting to explore programmatic data types like JSON or XML or the code, there may be some issues and differences in how you index those versus PDFs and doc files. The same with tables, the same with images right now. But I think it will all be the algorithm for indexing everything in the future.
Basically, you will not think about that at all. But now, there are differences depending on data structure and data type. But if you speak about the data, I think it can be anything that represents the data, even the picture of your check, your invoice converted to text, or your call, which can also be converted to text.
Any type of input can be converted to text, including video. But yeah, some nuances are depending on complex data structures.
Scott Rayburn: Okay. Thank you. I have a question for you, Amanda. Could you tell us a little bit more about the customer intent identification service in Azure?
Amanda Wong: I can speak to that. Customer intent identification can be approached in a few ways. I briefly mentioned some cognitive services. There is a family of services under the decision umbrella. This helps in scenarios like recommending certain products based on user behavior on a shopping website. That can help surface actions or next steps for a user to take that are personalized to them.
However, there are also opportunities to understand user intent through natural language processing and some of our text analytics services in cognitive services. I mentioned earlier that a lot of this technology is very similar or uses the same types of models and research. You may see different types of performance. That's where we do a lot of testing and proof of concept. DataArt is an example of helping configure and set up the different examples and scenarios, and performance with different services.
Those are some examples of how to understand a customer query, using large amounts of training data and augmenting it with other types of input signals to determine the customer's intent, not only to understand it but also to present action items and next steps.
Scott Rayburn: Thank you, Amanda. Our next question concerns the visual side of generative AI and some lawsuits against artists for stealing styles and imagery. If these artists actually win the lawsuits, what do you think that will mean for the future of generative imagery?
Dmytro Baikov: I think I need to be an artist to answer that. But I believe there are big shifts in how we understand the legal aspects of content right now, for both text and images. Sometimes, companies that train models exclude certain artists from their generation, and some real people from their generation.
I think this is a fairly new field, and some corner cases should be governed and discussed. I think we will still be able to generate images and videos. There will be different types of models. Maybe some models will have agreements where companies will have agreements with specific artists. Paying them some royalty fees will also allow them to generate their work. It's the same as how different companies can use actors' faces to play in their generated movies and swap their faces.
I think there will be exciting opportunities for new agreements between creativity and these big companies. But I don't think I can tell you more about that.
Scott Rayburn: Yeah, that's a big can of worms to open up on our AI webinar here. Let's go to a more technical question. Is it mandatory to use Cosmos DB to confine AI models, or the data boundaries of AI models, or can blob storage perform the job of Cosmos DB? What's the difference between data in Cosmos DB and blob storage for an AI model?
Amanda Wong: Yeah, I can take this one. It doesn't matter at the end of the day to the GPT model, but the decision is really up to you, what your data looks like, and how you are harvesting and transforming your data in existing workflows or even new workflows. Cosmos DB and blob storage differ in how often your data changes. For example, how long do you need to archive your data? How quickly do you need the response and inference? And how quickly do you need to update the training data that your chat model, for example, may understand? Those are some of the differences that help clear the air and distinguish when you should use Cosmos DB versus blob storage.
To discuss these in more detail or to have that conversation for your specific data management scenario – for the GPT model, or AI model, or fine-tuning – it really doesn't matter because it can take in different sources of data, whether it sits in Azure or another cloud or on-premises. For example, different connectors can bring your data to models.
Scott Rayburn: Great. Yeah, I guess there are many methods to get there, right? The next question is about training. How long does it take to train? What is the validity and reliability of the response? Maybe in the context of the chatbot you demoed, Dmytro, what was training like for that?
Dmytro Baikov: Yeah, the training process is basically what Amanda described. It's about bringing your data to storage and connecting it to models, and it could be any data storage. We were not training the model for that. We are using what's called Retrieval Augmented Generation to build that chatbot together with your documents, and we validate it with a number of different metrics.
There are some automatic digital storage metrics for checking how good your model is for specific responses. We also checked it with different trigger questions. It was a manual check by people. We tried to validate with the end users and see how they reacted.
The length of the context is also limited, which is maybe around 100K in the latest GPT-4 version and around 16K in GPT-3.5 Turbo. So "K" means tokens. It's basically 14,000 words and 80,000 words, approximately. So it's quite a long context.
That's what we are building. Within the same context, we are adding more data dynamically based on our data sources, and we validate it with human effort before the release by testing corner cases. There is no fully autonomous way to test that for now because the text might differ.
We also keep it a bit different because we are testing in production. That's why we don't limit it to one single response each time - we want to behave differently to check how it reacts to the same queries. We can review these logs and the responses. That's how we will evolve later based on real-world feedback and questions.
Scott Rayburn: Great. All right. Well, Dmytro, it looks like you also answered one of our other questions about how you're testing and validating it in an automated way. So perfect. That was two for one, guys. Nice job.
Well, I think that's all the time we have today. I want to thank everyone, especially our speakers, Dmytro Baikov and Amanda Wong, for their time today and for sharing all of their insights with you. If your question was not answered from the chat or you have another question, you are welcome to reach out to the email on the screen. With that, I wish you all well. Have a great day, and we'll see you on Friday. Thank you.
Dmytro Baikov: Thank you for having me.
