Generative AI - The Future of Enterprise Q&A Systems

Hello and welcome today .

I want to present uh generative A I , the future of enterprise QN A system and how we at neurons leverage transformative power of generative A I to shape company knowledge base .

My name is I'm head of Cloud uh practice at neurons .

I am a consulting company and a the partner .

And today , I'm thrilled to guide you through how technology is enhances the way we approach question and answer system within the business today .

And we will identify a key uh problems many companies face uh dive into how solution can be implemented with generative A I .

And finally , I will walk you through uh live Dema and showcase on how you can quickly prototype uh such systems .

And um moving to the next slide and this time is the problem .

And first , let's get to the root of the challenge in my experience .

In over the 10 years , I have seen many companies struggle with similar issues concerning the documentation , for example , or overwhelmed support teams that struggle to handle the volume of customer request .

And the second one , it's underutilized company documentation .

It's hard to find answer in the company and different business unions create own artifacts and missed opportunities to get insights uh from the documentation and data for decision making .

And the issue is especially critical for newcomers and uh for on boarding process .

And so today , we are going to address these areas where I believe we have the power for innovation .

And next slide , it is technical walkthrough uh the solution .

And in my journey , I have discovered transformative power or generative A I and how it can help to build knowledge system of the future .

Let's do walk through on how it works .

The solution contains three components .

Uh First we have a vectorized database .

It is knowledge based , but I can quickly find relevant information .

Here .

We use Amazon camera .

It is uh a the best managed service that utilize uh machine learning under the hood and provides advanced search capabilities .

Next one , we utilize the visual language models such as open I GP T four , Anthropic cloud or cutting edge open source uh models like 25 or falcon with uh 4 billion parameters for text generation .

And lastly , we use open source framework long chain that is like a glue join all the species together .

And next slide is a reference architecture on the site slide , you will see you see a visual representation of a services that show our solution and provide a clear understanding of how Amazon Camera and I models and the best land launch chain framework interact to create Q and a system .

And you can see this offline arrow .

It is we can ingest data from different data sources to Candra and build our Q and A solution .

And Kindra supports many different data injection capabilities like con github , SharePoint service .

Now , Google Drive , Slack , et cetera and web crawler as well .

And , uh , yeah , and uh , Mo moving uh and moving to the to the actual demo .

So let me share the at the best console and let's start with the camera .

First of all , we need to create indexes and ingest data to this database .

Let's click on create indexes .

Click on demo four , no type index name de in my case , gamma four , you need to create a new role as recommended .

And for this indexes , it will be created automatically and you just type the name for this , uh uh for the same role , you also can enable him as a key encryption .

Move into the next page .

Uh You has capability to for token , uh , to use token for access control .

We will not use for the Dema purposes , but it will give you capability to provide more granular access to your indexes .

And the second one , it is user group expansion and you can integrate the camera with , uh , uh using uh IM and center and uh different providers that is integrated with , uh , uh I I saw something like active directory or Google identity provider , etcetera .

You can , you can use it with integration with schedule and to authorize to a and a system in moving to next .

And here we have two different capabilities , development edition and enterprise edition .

Both of them are uh I mean first provide 100 So it's 10,000 documents was second one provides 100,000 documents for this demo purposes .

We are good with uh development edition .

So let's click on create .

I will skip this step because I already created several indexes .

And when you it will take some time for creation , my indexes is already created and active .

So I click on this and second what you need to do it is add sources here .

It will uh kind of provide a tip at uh on data sources .

I will go here on data sources and hit a button , add data sources and let's let's review what capabilities we have .

We can stop some sample sample documents and also we have uh uh available different platforms uh R DS connector S3 connector box confluence .

We can create custom one uh dropbox github , gmail , Google drive , Jira Microsoft exchange connector teams , one drive sales force service now , sharp point web crawler , et cetera .

So plenty of capabilities for data injection and let's select a stream .

Uh And before moving through II , I will show you that I already created uh a bucket neuros handbook .

And uh I extracted from uh we use notion so I extracted uh people handbook from it's Neo handbook , uh I extracted it as uh MD N CS V uh mark down and CS V files and I enable sub pages as well and click on export .

It's also support PDF kra support PDF as well .

So any text documentation can be parsed by , by camera .

So you can upload them to a three bucket and they will be parsed and added to indexes .

So I already exported , uploaded to a three bucket .

And you can see here if you go , you see this home page with MD format and moving further its folder and there are several folders and files and in MD and CC format .

OK ?

We already did this and let's uh add the data sources , data source first demo uh for a stream , you need to type name as a direct language .

Next , you need to create IM row .

It is uh you cannot use previous roles that was created during indexes creation .

You need uh im role in this im should have permission to three bucket .

So if you click on create new role , they automatically provide permission to three bucket and you don't need to do anything .

So we we use create new role .

I will type them off for .

You also can uh integrate with VPC .

I will not do this for demo .

Next .

And uh here we need to provide path to our indexes .

Sorry , our handbook documentation files .

Uh It's done next one .

You can provide metadata for these files .

For example , you can introduce uh departments and you can filter with these departments and search uh of particular department documents and that , that can provide you better quality and better experience in your search capabilities .

And as your meta data is just just on files that show you different categories for particular , I mean , identify different categories for particular files .

So you I will not do this .

And next one , it is access control list .

We didn't enable it so we can skip it as well .

And the last one it is schedule , I will on demand because my files doesn't change quick uh by schedule .

Um but based on your business needs and base it on your , I mean data source .

If it change frequently or not , you can select different uh different schedule .

Click on next .

Yeah , you just uh can skip this page next .

And here it is review of all the configurations that we did .

I will click cancel because I already created the data source .

You need to click on add data source and let's open the data source that we have just created and this one and you need to click on sync .

Now , if you select that option on demand and it will take some time for , to index all these files .

And you can see that the 196 documents was added .

And for example , for failure , it is my fail is because of um my files can contain some images in video files that are not processed .

And I mean , they process this failure uh actually skipped .

And uh yeah , you can see how much documents were scanned , uh how much it takes time , etcetera .

The next step I can create just a moment .

Uh You can uh search uh in this indexes , for example , uh what uh interview questions .

So I'm looking for interior questions and it provides me a link to different documents and highlight some text that is part of the document .

So it doesn't generate new , new text for me and it provides just the link .

So you can see this question validation through you , question through you , etcetera .

All these questions , I mean do do on different stages document .

OK .

That , that , that looks good but it looks like a standard Q and A system .

And I would like to integrate it with LLM and moving further .

It is uh stream uh application for prototyping .

And this stream application is use um this multiplication , use uh one chain and integrated with Kendra and uh uh open the I GP T 3.5 .

So I can answer here as the same question .

And it should provide me more like uh uh kind of like a chat capabilities and provide me like uh I'm talking with a human or with the bot .

So it's uh you can see my question it it provided .

And the answer is , I mean , I asked whether and I mean , wide open ended question .

And it's provide me just general information and says that question includes range , range of topics .

And this is background , technical knowledge , problem solving skills , team , your skills , behavior trait .

So this answer is good , but I would like to get some improvements and I will show you uh So this one is uh here at this qualification interview and , and you can see different categories of the question .

And I would like to specify one of the categories of the questions that I would like to get .

So let's say I would like to um iiml questions .

Let's see if it will find this particular questions in my document and provide me the list .

OK .

What , what do we have uh artificial intelligence , machine learning question or topic ?

I ask during an interview example includes such a difference between supervision and super learning benefits using our traditional approach .

Different uh my back again , what a list of questions .

Let's try this one .

See if it will rock .

It's just a prototype .

I have never tested all this really deeply .

OK .

OK .

That , that sounds good .

And uh let's back to uh our side and my father and um this slide is uh about solution , enterprise and production , ready consideration .

Even though we develop Q and A prototype today , the prototype is far from enterprise ready solution .

And uh so I would like to highlight several points that companies should consider before adoption generative i into the environment .

And the first one , it is the data security and privacy uh concern with third party services .

Second one , it is ownership of A I models .

Third one , it is compliance with enterprise uh reg regulatory standards .

And the last one is ensuring scalability and performance .

And I would like to give you one example uh open A I GP T four , an Anthropic cloud that are available through API can use your data for further the model training and your data can be actually unveiled by the personnel .

And second one , you don't own the I model .

So to address this challenge challenges , we can use it the best services like Amazon bedrock and Amazon stage maker .

This allow you to run commercial and open source generative model in private secure and compliance environment .

And I would like to uh finish uh with the next statement as a conclusion .

So seize the opportunity to benefit from a generative i before your competitors rapidly develop .

Uh uh rapidly prototype is uh uh readily available tools uh prepare to scale and secure your solution for enterprise level and reach out for expert to to get guidance and support on your journey .

So that's all .

Thank you very much .

See you .

Bye .