Hands On AI 8 Transcript
Please be advised that this transcript is AI-generated and may not be word-for-word. Time codes refer to the approximate times in the ad-free version of the show.
Mikah Sargent [00:00:00]:
Your dishwasher is showing you an error. Well, let's see if I type in the dishwasher is showing OE. What does that mean, comma, and what should I check first, question mark? Now I've hit enter, and while I'm waiting for this to load, I want to tell you that what I have done is I have uploaded 3 different manuals to this service, and we'll talk about the service in a moment. But the important thing is— da da da da! The OE error code indicates a drainage problem, meaning the dishwasher is unable to drain water properly. Here's what you should check first: the drain hose, the dishwasher filters, the air gap or garbage disposal, and more. And that came from the dishwasher's manual, and it even shows me where. Citations. Yes.
Mikah Sargent [00:00:57]:
No digging through a drawer. No scrolling a PDF. By the end of this episode, you'll know how to build one of these for your house, with free tools, you'll know why it works and where it can still fool you. I'm Micah Sargent, and this is Hands-On AI. Podcasts
Speaker B [00:01:15]:
you love. From people you trust. This is TWIT.
Mikah Sargent [00:01:25]:
All right, you wanna make it simple to figure out how to fix or at least address many of the problems that you might run into with your appliances. Well, I'm going to walk you through that very process on using AI for something at home. Now, step 1 of this whole process is kind of the part that I see people skip, right? Because they just kind of go to Google, they type in, my LG dishwasher is doing blank. Here's how you're going to avoid running into longer time spent trying to figure out what's going on and cut right to the good stuff, having the answer. find the exact model. That is what you need to do first. Let's talk about it. So when you're getting the manual, find the exact model.
Mikah Sargent [00:02:14]:
So an appliance, of course, you can use the model number for that. For a car, year, make, model, and configuration. You can also use the maker's VIN lookup if they have one. You can also, and I think this is the easiest way to go about getting a manual, go to the maker's support site. Manufacturers will offer their manuals online, and it's better than just typing in search to look for a manual because a lot of times those are coming from third parties and they're not the most recently updated version. Now LG lets you search by model number, so it's very easy to do. Once you've got that, download that matching manual. Again, avoid third-party sites if you can.
Mikah Sargent [00:02:56]:
There are some times where you do need to go to them. I've had that before. It saves— I've I used to do it all the time, scanning my manuals in for the different stuff that I was buying. You don't got to do that anymore. It's much easier now. And then lastly, and we'll show you how to do this in just a moment, you add it to your notebook. This is a special kind of category of AI tool called a notebook. It has some other names.
Mikah Sargent [00:03:24]:
We'll talk about those as well. And so for this episode, I grabbed— I wanted to try to make it as universal as possible, so I grabbed 3 manuals from 3 popular models straight from the maker of those devices. That's an LG dishwasher, a Honeywell Home T9 thermostat— so if you've got a smart thermostat that's giving you issues, you can get some help for that— and a 2026 Chevy Equinox. Of course, we'll include the links in the show notes to the files themselves, so if you want to follow along at home with my files, well, those companies' files, you'll be able to do that. So how do we start? Well, we need to head to, in this case, Gemini Notebook. That's the tool we're using today because this is Google's tool. They used to be called NotebookLM, and this is the important part. There's a free version as long as you have a Google account.
Mikah Sargent [00:04:21]:
You just have to go to notebook.google.com. To get it. And what's great about it is it's pretty full-featured for just being on the free plan. On the free plan, when you build it, you get 50 uploaded sources total. So that can cover a lot of your appliances. Each of those individual sources are allowed to be up to 500,000 words, and an uploaded file has to stay under 200 megabytes. So let's take a look at what this— how this process works over on macOS. Again, you go to notebook.google.com and then you just choose new notebook.
Mikah Sargent [00:05:03]:
And in this case, we will call it home manuals. And we're going to upload those 3 manuals that we talked about. Chevy Equinox, Honeywell Home T9, and the LG dishwasher. So all of those 3 files are going to get the upload treatment. Of course, that Equinox manual, about 17 megabytes, but still well under the 200 that's required, which is good. So it's just a matter of letting those Load. So we will allow those to make their way through. The important thing is, is that the upload will send these files off to Google.
Mikah Sargent [00:05:52]:
Now, Google says that for personal accounts, it does not use your notebook data to train Gemini Notebook unless you decide to send in feedback. So just be aware of that. It's fine, you know, for an appliance manual, but it might be something that you think about before you consider using this specific tool to, you know, do your tax returns or something like that. Now, this is one way of pulling it off, and it's a very focused way of pulling it off. But I want to talk about some of the other, other online projects, online project types and online methods. So of course, there's also ChatGPT that has projects, and there's also Claude, which has projects. They let you collect documents and projects, but the limits and the source tools are a little bit different on the free plan. A ChatGPT project can hold 5 files, and Claude also lets you make 5 projects.
Mikah Sargent [00:06:52]:
So just bear in mind that, again, this is a purpose-built tool for doing this specific kind of work, which we're not talking about it yet, but we'll get there as well. And then also be aware that at different times, free plans can change. The limits change all the time. So just check yours. You know, Just whenever you're deciding to try out a new tool or you're, you know, checking in with an old tool, make sure that the plan still supports however many manuals you're trying to load. Now, the thing that we saw earlier, which was the ability to type in the question, because by the way, that what I just did, what we— I just showed you, I want to go back actually. That was all I did before I started this portion of the episode. What you saw on screen, uploading those 3 files after creating the notebook was all I did before I asked that question from earlier.
Mikah Sargent [00:07:55]:
So that, that's the process. That's how simple it can be to get this set up and give yourself what amounts to kind of a search engine with a few more powers. tied to your specific files. Now, that is where it's time to look into the AI of it all, because you may wonder, you know, how it goes through and pulls one answer out of 100 or more hundreds of pages in a couple of seconds. I mean, bear in mind, one of the files that I uploaded to kind of show you what it's— what it's capable of doing, uh, the Equinox manual, it's 324 pages. So quite a bit. And if you've ever— I don't know about you, but I certainly have taken some open book tests and those are typically set against a time limit. When I took my first open book test and when I took my next open book test, the level of studying I did, yeah, definitely went up for sure.
Mikah Sargent [00:09:01]:
So whenever it comes to that, when you ask your question, right, It first searches the library for relevant passages, a little bit, as I said, like a search engine. Then it retrieves those relevant passages and supplies them to the model as context. So the model can answer based on the context that it pulls from those specific passages, as opposed to having to put every manual into every request. So Google says that with lots of sources, it will first find the most relevant information and then build the answer from that. Exactly what you'd expect. The reason why this is— why I've been kind of harping on this and getting into it is because this is a special and specific type of function that makes all the difference. It's called retrieval augmented generation. And if you've heard anyone say RAG, this is what they were talking about.
Mikah Sargent [00:10:04]:
It comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, which of course is now Meta, with University College London and New York University. And what it does is it matches related meanings, not just identical words. So it's a little bit blurry. So, you know, saying something like foam all over the dishwasher, as you and I might say, will mean that it can retrieve the passage that uses the proper term of oversudsing. So one thing that has, you know, in the past, before I started doing this show and really digging into AI, that I always, I always kind of thought of it as like post-post-training. And I've used that term throughout the show, but it is important to understand that you aren't retraining the model. So those billions of knobs that we talked about in episode 2, Putting in a PDF, it doesn't turn those knobs. If you take the manual out of the equation, the notebook will stop retrieving from it, and that is the, you know, end of its ability to look into that.
Mikah Sargent [00:11:18]:
So without those documents, it's not able to run RAG and do the retrieval augmented generation base. It just has nothing to augment the generation. That means that you have to be aware of the sources for that specific tool. Back at the model of one, of course, important thing with this is something that we talked about before, always checking your answers. Whatever answer comes back, you can check it against the citations. And that's a special aspect of this, being able to ask questions, get results back and be able to trust them a little bit more as long as the citations to those documents hold up. All right, let's look at a fresh use of this. So I'm going to say, answer from the uploaded manuals, identify the manual and relevant Passage.
Mikah Sargent [00:12:25]:
If they don't support an answer, comma, say so. Don't search the web. Now we're going to see if these manuals contain the answers. Here is what I talked about before. I'm going to say, there's foam all over the inside of the dishwasher. What happened, comma, and what do I do? Now we'll watch. Remember that we asked the model to only work with the uploaded manuals, identify the manual and the relevant passage. So we're actually doing more than you really need to.
Mikah Sargent [00:13:13]:
I wanted to be sure that we're super limiting this, but again, by default, these, this RAG actually works from that perspective. So It says in response, from the LG blah, blah, blah, blah, blah dishwasher owner's manual PDF, and then it even goes as far as to say specifically under the adding cleaning products section and the troubleshooting error messages table, here's what happened, what steps to take. It talks about improper detergent possibly being responsible. It even says that that's the most common reason. Foamy food residue, rinse aid dispenser set too high, or improper leveling. It goes on to provide several steps All linked to specific codes, all linked to specific parts of the manual. It has error messages, everything built in. And then Notebook, specifically Gemini Notebook now, gives you the ability to ask further questions, or it prompts you with further potential questions.
Mikah Sargent [00:14:15]:
Let's try this. Because bear in mind that I started with a dishwasher question. Now I'm going to say, how do I set a schedule so it's cooler at night? And then we'll wait. And it's back with an answer. To set a schedule so it's cooler at night on your Honeywell Home T9 Smart Thermostat, goes on to name the guide. You adjust the target temperature for the sleep period. The system includes 4 daily time-based periods: wake, away, home, and sleep. It talks about the method for doing so, choosing your groupings, and it goes on.
Mikah Sargent [00:15:10]:
It also has another method. Uh, the Resideo app that you can use as well. It then talks about, uh, the wireless room sensors that you can pair with the T9. And so it has all of that information again, once again cited from our sources directly here. Thermostat features, flexible scheduling pulled from that manual. Very handy. All right, let's keep going with the next question, which is for— from the, the actual car. How much oil does the Equinox take with a filter change? According to the capacities and specifications section of the Chevrolet Equinox 2026 owner's manual, the engine oil capacity with a filter change depends on the vehicle's drivetrain.
Mikah Sargent [00:16:15]:
Front-wheel drive, 4 liters. All-wheel drive, 5 liters. And here is the best part, folks. I knew it was a trick question, and it was able to properly answer that trick question. And know that there was a difference between the front-wheel drive and the all-wheel drive model. That's RAG. I clicked on the citation as well, and it pulled me right to the page so that if you wanted to, you could use this as a starting point and then have it go further with the research, and you could look into it and kind of learn more from there. That's what I really like about it.
Mikah Sargent [00:16:54]:
Hovering over, over a citation, Gives you the quote from the documentation as well, and clicking it takes you to the passage. A citation gives you that place to check if you need to, and so, you know, you may want to, in kind of gathering all of this information together, try to hold on to those citations so you know where to look in the documentation afterwards. There are a few gotchas that come into play when using RAG in particular. So I want to talk about those because the idea that it knows your manual and that it remembers your manual are 2 different concepts, and they work differently when it comes to AI models in particular versus AI models that have— that are part of an overall RAG system. First and foremost, bear in mind that training is going to have a factor. It changes the model weights. And so It may have seen manuals like these, but that doesn't mean that it's going to give the right answer for this exact model. So if you just ask a standard model without RAG this question and you don't kind of focus in, it can give you a really convincing answer and one that's pretty close to the right one.
Mikah Sargent [00:18:19]:
But this is what helps to add on top of it, giving this particular context. Also, when we're talking about context, you gotta think of it kind of like, um, what's available to you on your desk right now. That's the context that the system has. And well, it needs to know your question. It needs to know the chat. It needs to know anything that you handed to it and everything that fits in there. That's the context window. Long chats, Those get shortened, those get summarized.
Mikah Sargent [00:18:52]:
So again, having this here helps. On top of that, there's also memory, your context between different chats. It's, it holds on to select information that it feels is going to be important. Preferences, earlier chats, previous work, and those all depend on the chatbot. So however you decide to use memory and context, while making use of these tools, that can have an impact on its accuracy as well. Last but not least, of course, is the retrieval. It searches the shelf, it brings useful passages to the desk, but it may not bring everything that's needed. It can miss something.
Mikah Sargent [00:19:36]:
The ability to upload so many different files, right? is like for free, would give one maybe the idea that uploading a lot of manuals, uploading them and making use of them is the right thing to do. Or taking all of your manuals, shoving them into the context, and then saying, give me this answer. No, that is what is helpful with RAG. A pile of manuals isn't going to stay fitting forever because The answers that come after that have to be part of the context. So Claude's help page actually has documentation directly about this. It says, when a project's files near the context— wait, when a project's files near the context window's limit, it switches to searching them. So there's actually RAG that gets built in after the fact. in these projects, and it doesn't work on the free plans.
Mikah Sargent [00:20:41]:
So it's only something available to the paid plans. All right, let's go ahead and try to break this thing. That's something that we always try to do on the show. But now that you've kind of understood how easy it is to create a sort of home manuals search engine, Let's see what we can do to break it. All right, here we are back in our home manuals, and we're going to give it a bit of a test. Let's see, I'm going to say, is it worth repairing this dishwasher, comma, or should I just replace it? Now bear in mind, that's all I'm saying to it. Nothing else. Just that.
Mikah Sargent [00:21:39]:
And we'll see what it has to say. This is its response. The uploaded manual does not support an answer to whether it is worth repairing or replacing the dishwasher, as it does not include repair cost estimates, replacement prices, labor rates, or Economic guidelines for deciding between repair and replacement. The manual does, however, provide the following related information: warranty coverage for repairs, service requirements, and foam issue context, given the earlier conversation. So in this case, it is doing what I asked it to do. I said, please don't ask outside of the documentation, and it gave me the information that it was able to give me. So frankly, I think a good answer. Now another one.
Mikah Sargent [00:22:34]:
I'm gonna try to give it a fake error code. What does error code E9 mean on the dishwasher? E9. And here's what it says. The LG dishwasher manual does not list or support an E9 error code in its troubleshooting documentation. The manual's troubleshooting section covers the following specific codes. Your manual doesn't cover this code. Would you like me to do some web research to see what E9 refers to on LG dishwashers? So it offers to step outside, but it is not doing so. So once again, Test passed.
Mikah Sargent [00:23:22]:
Then remember that episode, I think it was episode 6, where we tested whether a chatbot would agree with us even when we pushed it? That is what we're going to try now. The manual says OE means the heater's broken, comma, right? So I'm saying, hey, I read this. That's right, right? And we'll see what it has to say. All right, it says no, that is not correct. According to the LG dishwasher manual, 0E or OE— and that's another thing, I wrote OE, the code is 0E— indicates a drainage problem, not a broken heater. So This gets my pass. I think that this is fantastic, uh, that it is making sure that it is giving you answers directly from the manual. And that cuts back on what we've talked about before, hallucination, uh, where it is just trying to answer the question that it's been given and it is pulling that answer from somewhere, even if that answer doesn't exist.
Mikah Sargent [00:24:38]:
Retrieval. Can help to reduce those made-up answers. That said, it does not guarantee under any circumstance accuracy. Stanford researchers— yes, I love to bring in my, uh, studies— Stanford researchers tested legal research tools that were built using RAG. They were marketed as avoiding made-up answers, but in their tests, those tools still hallucinated 17% to 33% of the time. That said, legal research, it's not appliance manuals, but I think that that doesn't mean that you should stop using the citation tools. I know I'm getting annoying about citations, but it's very helpful when it comes to this. Uh, I, you know, it's one of those things where we, we want to be responsible in, uh, introducing and encouraging these tools and not put you in a place where you are doing a repair on something and you've read the wrong result.
Mikah Sargent [00:25:37]:
Check the citations. That's what helps. All right, so back to the habits that it comes to when you're trying to use these tools. First, check that the exact model matches your exact product. Then make sure you open the cited passage. Then please, electricity, gas, water safety— read the manual safety instructions Don't just read the answer. And lastly, when the manual says to call for a qualified technician, call one. I— it's important that we put that in there.
Mikah Sargent [00:26:14]:
That is, that is a very important step. And I'm sorry, uh, to— for those of you who, uh, you know, already have that figured out, but important nonetheless. Now, I understand that for Some of the folks who watch this show, they used NotebookLM 100,000 years ago. What feels right? It is old news. And there are some of you who want to go a little bit deeper. You just saw what it's like to simply go to notebook.google.com, create a new notebook, upload some sources, and suddenly you're asking questions about your car, your thermostat, and your thermometer. No, that wasn't the third one. The dishwasher.
Mikah Sargent [00:26:59]:
I would love to hear about people making use of Notebook. I keep wanting to say Notebook LM, Gemini Notebook. But for those of you who want to go deeper, the tinkerers of it all, well, I have that for you too, because there is a private version that you can make using that laptop model that we set up on episode 1. This time we're going to use 2 things, Ollama from episode 1, and then a free app that you can get called Anything LLM. What it does is it adds those document workspaces, it adds retrieval, and it adds source citations. And so that is what we're going to take a look at next. Let's head over to macOS and look at the go deeper version. All right, tinkerers.
Mikah Sargent [00:27:47]:
So this is Anything LLM, the free app that is going to help you with the sort of document search portion of this. Now, remember that first and foremost, you've got to make sure that Ollama is installed and running. That is the app that we downloaded whenever we did episode 1. So if you've watched that, you've already downloaded Ollama. You've already also got the Quen model downloaded. If you don't have that, go back and check out episode 1. But I know that both of those are loaded here. Then the next step is, of course, to install this app, Anything LLM.
Mikah Sargent [00:28:25]:
You can get it, and we'll include this, the link in the show notes as well, anythingllm.com. You get the Mac version for Apple Silicon, and then you open it from your Mac. On your first launch of this tool, you need to connect it to Ollama. Now, it is going to be a little bit different for you whenever you're first getting it set up. It is as simple as clicking Get Started, setting up those options, and than walking through the process of just choosing Ollama. But I want to show you where all of these settings are so that you can match them to mine. Once you've got Anything LLM running, click on the wrench. That's the settings, as you might imagine.
Mikah Sargent [00:29:08]:
You click AI providers, which is at the top here, and then we're going to choose LLM. And you can see I've got Ollama set and I've got this specific model set. So that's Quen 3.59B as well as the embedder. Also, you'll set your vector database to LanceDB and the embedder to the Anything LLM Embedder All MiniLM L6 V2. Once you have those set, something that I like to do is also make sure that I have certain things turned off. So anonymous telemetry, I do not have that enabled on mine. That's absolutely up to you. But those are the preferences that you need to have set here.
Mikah Sargent [00:30:02]:
The LLM provider, the vector database, and the embedder all locked in. And then we can go back to this main screen. Now, once you've set that up, the workspaces that you create from there, are actually going to be part of the— the workspaces will be based off of the default tools. I also, for the sake of doing this specifically, RAG, we want to choose agent skills. And within agent skills, we turn scrape websites off and we turn web search to off. You'll note that I also have all the rest of these off as well. App integrations, And file system access, et cetera. We just want view and summarize documents and then RAG and long-term memory to be part of the process here.
Mikah Sargent [00:30:56]:
Once those are all turned off, then we know that what it's doing is only responding to our specific documents. All right. So in order to set up a new workspace, we simply need to— Hit the plus sign over here, and we'll call it Home Manuals. We choose Settings, and within this, we have to switch it from the current version, which is General. We choose Chat Settings, and from here, you can see that it's set to the system default, so the one that we have in Settings, but chat mode needs to be switched to Query from Agent because Query is the only version That is for document context searching and responding upon document context searches. That's what we want. And that way it isn't going to use the agent tools as well. Here you can also make adjustments to the vector database.
Mikah Sargent [00:32:02]:
The important note here is max context snippets. So we've got this set to 4. That's the recommended value. And basically what it means is How many passages do— does the system hand to the model for every question that it's asked? So 4 is the recommended value, and that's what we want to have as well. Leave your document similarity threshold to low. That's the value that makes sense for this. And then we just hit the back arrow to go back to our model. Now, this is the important thing to understand as part of this process.
Mikah Sargent [00:32:44]:
You don't want to click and drag your files into the chat. Instead, they need to be uploaded using the upload icon in the Home Manuals section. So we'll choose that, and then we will click and drag our 3 manuals in. It will load those 3 manuals pretty quickly. And so that's good to go. And then it takes a second to do what it needs to do as it loads in those manuals. And once that's done, these panels will kind of give you both documents and data connectors. What we want to do is move them to the workspace.
Mikah Sargent [00:33:35]:
And so we can click on the custom documents that we have here. There are 6 because I've actually uploaded them before. So I'm going to click the cached versions here and I'm going to choose move to workspace. And then after I've clicked move to workspace, They are nearly there, but we have to actually embed them. So we click save and embed. I know this is very involved, but once we click save and embed, now the documents have been embedded as part of the workspace that we have in the main section. Now for this, it takes a bit of time. For me, I ran a test.
Mikah Sargent [00:34:25]:
It was about 45 seconds to load these. It may not take as much time. I, of course, use those cached documents, so they didn't take much time at all. But you can, you know, you just wait a second while it goes ahead and loads those, and then you close out of the document manager. Last, it's time. So, We will ask in home manuals with a new thread, the dishwasher is showing OE. What does that mean and what should I check first? Now we'll let the model do its thing. Remember, this is all happening locally.
Mikah Sargent [00:35:08]:
You— if you have this set up exactly as you did in the beginning, then you know that the QWEN model has its web search tools turned off. This is just Processing here on this little MacBook Air, doing its best to come up with the answer for what we have. Now, on the air, it takes about— on the air, it takes about a minute. Almost all of the process is just model thinking, and you remember seeing the model thinking. Uh, so if we want to take a look here, it says, 1, analyze the request, input a conversation, just a user question based on context, relevant context, multiple document excerpts from an LG dishwasher, Owner's manual. Task, reply with an answer to the user's question based only on the provided context and information. Constraint, return only the response to the question given the above information following the user's instruction. So it's going about, it's coming up with the answer, it's drafting the response, it's doing a little bit of polishing.
Mikah Sargent [00:36:10]:
The whole process is taking place. This is the stuff that happens a lot faster In the cloud. This is what Gemini was figuring out relatively quickly in comparison. But I tell you again, this is all just loading locally on my machine, no cloud involved. So we will let this process complete. And it says, the OE error code indicates a drainage problem. You should first check for a clogged— Clogged, kinked, or frozen water drain hose. So it did get the OE error code correctly.
Mikah Sargent [00:36:50]:
Yay, we love that. It says check for a clogged, kinked, or frozen water drain hose. And well, that is correct. It's a little bit incomplete. There is more information that we saw earlier on, right, of, of the full answer, but I do have to say that with just a local model on this little MacBook Air, relatively impressed with what's possible there as well. So a little bit more involved for sure than setting up things via the notebook.google.com. But if you have been wanting to do some local model work, this is one of the ways that you can do it. Very handy.
Mikah Sargent [00:37:36]:
All right, let's wrap this thing up. That is the project. Manuals go into the project, questions get answered, and yes, don't forget to do your citation checks. If you'd like some homework, here is mine for you. Make a notebook for your house. Start with whatever's most likely to give you issues, something that you've had to deal with in the past where you remember looking up the manual, or opening it up on the computer, uh, doing a search for it. Pop those all into a notebook, or even better, use one of your AI agents to go and gather all of those manuals for you and then pop 'em into a notebook. And then you've got your own personal search engine for your home items.
Mikah Sargent [00:38:19]:
I would love to know what is the annoying job you'd hand AI if you knew how. Send it to handsonaiai@twit.tv and you just might see it built on the show. Hit subscribe or follow wherever you're watching or listening, or head to twit.tv/HOAI. And if you'd like it ad-free, well, that's where Club Twit comes in. twit.tv/ClubTwit. I'm Micah Sargent. Don't just wonder about AI, get your hands on it.