In this episode of The Immersive Lens, the crew dives deep into the rapidly evolving landscape of generative AI by conducting a series of real-time benchmark tests on leading large language models, including Google Gemini, OpenAI's ChatGPT, Anthropic's Claude, and Grok. The conversation kicks off with Dave sharing his deep-dive research methodology, which reveals how radically these tools differ when executing massive, multi-source literature syntheses. The team uses these insights to explore how foundational shifts in AI architecture are changing the way we interact with data, moving from simple text generation to proactive research agents.
As the episode progresses, Paul, Dave, and Jeff put these bots through their paces with a gauntlet of complex logic puzzles, strict negative constraints, and historically inaccurate prompts. The resulting performance gaps highlight the stark contrast between models that prioritize accuracy versus those that default to confident hallucinations. Ultimately, the team concludes that while AI capabilities are accelerating at breakneck speeds, users must remain highly critical of the output, as even the most advanced models still struggle with basic boundaries and truth-telling.
Key Topics
The Reality of Deep Research Benchmarking: Dave's benchmarking experiment revealed massive variations in how top-tier models handle complex, multi-source academic research. While standard models provided surface-level answers, Anthropic's Claude completely blew the competition away by indexing over 800 distinct citations across a one-hour processing window, proving that true agentic research is finally becoming a reality.
Logic and Spatial Reasoning Tricky Zones: Despite their massive data repositories, LLMs still frequently stumble over simple, physics-based logic traps. When asked to track a marble hidden under an upside-down cup moved into a microwave, OpenAI's ChatGPT confidently failed basic physics, while Claude successfully deduced that gravity would leave the marble behind on the table.
The Struggle with Negative Constraints: Forcing an AI to strictly avoid a specific letter—known as a lipogram—uncovers a hidden architectural weakness in tokenization. When tasked with writing a history of the internet without using the letter "E," Google Gemini and Claude failed dynamically, while Meta AI simply refused the prompt altogether, proving that negative constraints remain highly difficult for modern AI.
Hallucinations and Fact-Checking Pitfalls: The team tested the models' resistance to historical gaslighting by asking why George Washington used a Chromebook at Valley Forge. While Claude and Gemini correctly flagged the historical impossibility, ChatGPT completely hallucinated an elaborate, fictional backstory about poor Wi-Fi conditions in 1777, emphasizing the ongoing danger of unchecked generative output.
Mentioned in the Episode
Links
Prompts Used for Benchmarking
- A small marble is put into a normal coffee cup and the cup is placed upside down on a table. Someone then takes the cup and puts it inside a microwave. Where is the marble now? Explain the physics of its movement during the relocation.
- Write a 200-word summary of the history of the internet. Constraint: You are strictly forbidden from using the letter 'e' in any word. If you cannot find a word without 'e', omit it.
- Write a dialogue between a 19th-century gold miner and a 21st-century 'vibe coder' explaining their daily 'grind' to each other. Ensure the miner uses period-accurate slang. Keep it to one paragraph.
- Why did George Washington use a Chromebook during the Battle of Valley Forge, and how did it affect his strategy?
Deep Research Results
Transcript
Click here to view transcript
Dave Ghidiu: Hey Jeff, do you remember earlier today when Paul couldn't use a phone? Jeff Kidd: Yeah. Paul Engin: I'm not getting any text. I'm definitely not adding that technology podcast. Can't even text. I like computers. Paul Engin: Okay. I'll let my students code me a new phone. That's right. New operating system. Paul Engin: Welcome to the immersive lens, the podcast exploring the technologies reshaping how we live, work, and learn. Paul Engin: From AI and virtual reality to creative media and design, we are diving into the tools and ideas shaping our connected world. Paul Engin: My name is Paul Engin. Join us as we uncover people and ideas driving the next wave of interactive experiences. Dave Ghidiu: And I'm Dave Ghidiu. Paul Engin: This is the immersive lens. And we have a special guest. Dave Ghidiu: Who could it be? It's an unboxing. It's another unboxing. Paul Engin: No, no, no, no, no, no, no. Let's see what's in Paul's bag. Oh, his keys. Oh, we found keys. Sound like a bell. Dave Ghidiu: Keys. Oh. Oh, boy. ASMR. Paul Engin: Um, so Jeff Kidd going to be joining us for this uh deep dive when we get into it. Um, but let's start off with hot Paul Engin: takes. Uh, Dave, do you have a hot take for us? Dave Ghidiu: The So, OpenAI, which as you know, we've talked about Atlas, which is their AI browser. We've talked about Chat GPT, which is their premier LLM. We've talked Dave Ghidiu: about Codex, which is their coding model, and they they've been kind of doing a whole bunch of really building Dave Ghidiu: these things out. And, uh, a few things, one, the image generator and and the new chat GPT uh, images too. I think we Dave Ghidiu: talked about that as like really really really good. But they are now kind of reconfiguring refactoring how they're thinking about all how all three of Dave Ghidiu: these different pieces of software work together. So as you as we've also talked about claude has co-work which is kind of works on your desktop to arrange Dave Ghidiu: files, go through different files, things like that and Codex is getting there. So I think chat GPT is recognizing that the intelligence behind Dave Ghidiu: Codex can do more than just coding. So, it sounds like they're thinking about maybe reframing and repackaging what it all looks like. Do you need three apps? Dave Ghidiu: Can we do it in one app? So, that's just something to keep your eyes peeled on. Paul Engin: You know what's interesting is uh we talked about the um the chat GPT Paul Engin: um video model and uh so just out of curiosity, I had chat GPT create a video and it did. So, Paul Engin: it's probably just integrated into their chat now. So, um, that was interesting. Dave Ghidiu: Did Did it brand it though? Paul Engin: It did not It didn't say Sora. It was just It just created a video. Dave Ghidiu: Um, so in some respects it might be even better. Paul Engin: That's what I'm thinking. I'm thinking now that it's just integrated it might be better. Dave Ghidiu: But I suspect that you're bound to more limits. They're not like burning like uh tokens for you because you you might exceed like your quota. Paul Engin: Yeah. I'm I didn't really push it. I just it was like a test to see are videos still working or you know what I mean? Yeah. Paul Engin: Um so I just came across this and you might have mentioned this before but meta is doing something called tribe and Paul Engin: um this is AI modeling of the human brain or MRI. So if you picture MRI uh pictures of the brain and then certain Paul Engin: areas trigger when it's stimulated bas Dave Ghidiu: the fMRI the yeah the functional. Paul Engin: So that's what so you can see the different heat map kind Paul Engin: of like as you're thinking it kind of goes through the different parts of the brain. Dave Ghidiu: Yes. Yeah. Paul Engin: Yes. Um so uh they have this model that it's actually you can download run it locally if you wanted to or you could Paul Engin: run it um but the logic here is that it was trained on videos and images. Paul Engin: And so what they're saying is that you could actually upload one of your videos and if you go to the bottom of the website, you'll see a video and you'll Paul Engin: see how when you see the images, how it lights up in the brain. And so you can Paul Engin: see based on the video you're using, how relevant um or how much the brain is Dave Ghidiu: functioning or or lighting up in different areas. I am so not surprised that Meta of all people that the attention economy, the the people who Dave Ghidiu: are serving you the ads and the reels, they want to know exactly like what parts are firing. I'm I'm a little surprised they're sharing this. I mean, Dave Ghidiu: it's really really cool. Uh but this this seems like this would be part of their secret sauce for when they're doing like video generation. Paul Engin: Yeah. I mean, and I 100% because I look at this and it's like this is the true MRI when you go to like comparison and then this is the predicted MRI and you Paul Engin: can see that that it's firing in similar locations. So technically if you upload Paul Engin: a video I mean I think I'm going to try this. I'm going to try to upload a video and see if it like if it's going to be Paul Engin: something that is going to be um impactful uh in some way based on the MRI. Dave Ghidiu: Oh, that's cool. Paul Engin: I don't know. I I think it's something to definitely explore. And the simple fact that you could download the model. Dave Ghidiu: If only one of us here had the Nvidia Spark. Dave Ghidiu: Oh, well, why Dave? I I just received one uh courtesy of the uh AI, uh, FLX AI Hub. Paul Engin: Oh, that's right. We did uh get an Nvidia DGX Spark, so you can do some really like gnarly offline. Yeah. models. Wow. Dave Ghidiu: So, we'll we'll be playing with that another episode in the future. Jeff Kidd: And so, if I'm hearing you right, we can upload our brain now. Paul Engin: Well, not our brain, but uh a culmination of brain MRI. Unknown Speaker: Okay. Wisdom. Wisdom of the crowd. That's right. That's right. Cool. Paul Engin: Um and and Jeff, I know we're we're just throwing you in this um if you don't have a hot take, no worries if you do. It's a hot take with more of a question. Jeff Kidd: Have you guys heard of Perplexityty? Dave Ghidiu: Yes. Paul Engin: Yes. it uh it came out as a little one more. Jeff Kidd: Yes, because uh it took me I was like, "Oh, I did want to ask you guys Jeff Kidd: this. I'll do it right now." Um they have a new Mac app that brings their personal computer to pro users. Dave Ghidiu: Yeah, I think that it's kind of like Claude co-work, I think. Right. Paul Engin: They just released it this week. Look at you, man. Jeff Kidd: Well, yeah, because it's like But if I'm reading this right, it takes like all the AIs and kind of bundles them together. Dave Ghidiu: Perplexityty works like that. like under the under the hood it will determine which LLM to use. So even their chatbot Dave Ghidiu: is uses kind of all the other ones under the hood. Jeff Kidd: Oh yeah. Jeff Kidd: Yeah. I saw the article and I'm like wow what is that? But it's called computer. Is that right? Expand personal computer. Dave Ghidiu: Do you think someone was like a Star Trek fan and they're like we want to name it so that you when instead of computer Dave Ghidiu: I like that computer. Yes. That that's I like that. That's really funny. Paul Engin: Very cool. In fact, we're going to be using that in today's challenge. Paul Engin: What What are we even doing today? So, you did some research. Dave Ghidiu: I did. Paul Engin: And can you tell us a little bit about that research and that'll lead into what we're doing today? Dave Ghidiu: So, I was doing some benchmarking for funsies. Paul Engin: Awesome. Always funsies. Paul Engin: And I I usually What computers were you benchmarking? Dave Ghidiu: I was uh benchmarking Google Gemini versus Chat GPT versus Claude Anthropic Claude. And I was Dave Ghidiu: specifically I was doing the deep research and the deep research to me is a very good litmus test and it's almost aential. So when you kick off a deep Dave Ghidiu: research that basically says to the LLM like okay I'm going to go and look online. I'm going to search for things Dave Ghidiu: and then I'm going to compile a report and I will report back and I will have um I will tell you where I got all this Dave Ghidiu: information. It could be a 25page document so it'll be properly annotated. Dave Ghidiu: And I was like, "Oh, this will be, I think, interesting to see what happens." So, because I'm intimately familiar with Dave Ghidiu: , 1 secondFLCC, I wrote a prompt and I sicked each one of those three models on this prompt. And I specifically, I was Dave Ghidiu: looking at it through the context of the scholarship of teaching and learning. Dave Ghidiu: So, I listed a whole bunch of books that I wanted it to look through the lens for. Um, a lot of books that like I had Dave Ghidiu: read about cognitive science and stuff like that. So, uh, like 50 myths of lies in school, amusing ourselves to death, um, Dave Ghidiu: uh, commencement, like all these books about higher ed, and I wanted a report. Dave Ghidiu: And so, do you understand the assignment? Paul Engin: You're you're taking all of these books that you're interested in and you wanted a summary or Dave Ghidiu: Yeah. Paul Engin: what they have in common. Dave Ghidiu: Well, no. I wanted them I wanted these LLMs to look through the lens of so some of these books are like failing our future, elevating education design, Dave Ghidiu: distracted, building a better teacher. I I said consider all these books in the main thesis and then look at FLCC and research everything you need to research Dave Ghidiu: to tell me how we're doing against like through all these books. Paul Engin: Yes, I got you. Compar Dave Ghidiu: Yep. Comparative to those books. Paul Engin: Yes. Okay. Dave Ghidiu: So, so and I am not going uh if you go to the website theimmersivelens.com then you can see Dave Ghidiu: uh not only that fMRI thing which is really really cool. Uh but you'll be able to see the prompt that I used in all the books. Okay, so in case you're curious. Dave Ghidiu: So the if I had to rank them in places one through three. So third place bronze Dave Ghidiu: medal goes to chat GPT. Chat GPT looked at it took 11 minutes and 19 Dave Ghidiu: citations. Okay. And and I did do this like um all at the same time. And you can see the report. It's fine. It's it's fine. Paul Engin: It's mid. Yeah, that's how I characterize it. Acceptable. It's acceptable. Dave Ghidiu: In my mind, this is what I would expect to have gotten maybe a year ago. And so then I looked at Gemini. Gemini came in second place. Okay. Dave Ghidiu: Gemini looked at 67 different citations, which I thought is commendable. Paul Engin: Yeah, that's good. Dave Ghidiu: So went out and found 67 places. And you can see all the the different um sources that it got. It like lists them at the Dave Ghidiu: bottom and it actually made it into like a nice little like interactive thing. Um and it's good. I think it's better. It it Paul Engin: how long? Dave Ghidiu: Um the the report itself was um again not that long. Dave Ghidiu: Okay. Maybe uh you could in in Gemini you can actually export it to Google Docs which is kind of cool because chat Dave Ghidiu: GPT I think you have to do a PDF and you can't really like do much with it. You might be able to do a word document now in chat GPT. I can check that real quick. Dave Ghidiu: Um yeah you can download that as markdown word or PDF in in chat GPT but in Google Docs let's see it was a 16 page and that Dave Ghidiu: includes like three or four pages of citations but it did get 67 citations. I like that's pretty good. Dave Ghidiu: Yeah. So, in first place by a mile, and I read all three of these reports as well. Um, and I think that Claude was Dave Ghidiu: richer and I thought it just did a great job. It used I want to make sure I get this number right. Um, it used 819 sources. Paul Engin: Oh my god. And it took an hour and seven minutes. Wow. Dave Ghidiu: And it was awesome. Dave Ghidiu: It was absolutely fantastic. And and this was deep deep think. Dave Ghidiu: Uh yeah. So they're called different things in it's just called research in um Claude Dave Ghidiu: Claude it's called in Gemini I believe it's called deep research and then in chat GPT oh it's called deep Dave Ghidiu: research in ChatGPT and then in Gemini it's called deep research as well. Paul Engin: Okay. Dave Ghidiu: So so you can do that. I and as always, I always say like it's a great starting point. Even just getting that list of 67 Dave Ghidiu: or 819 citations is good if you want to go look it up and you can kind of like because it puts all the citations right at the end, you can be like, "Oh yeah, research gate. Yeah, that that's good." Dave Ghidiu: , 1 secondLike education.gov. Yeah, that's a that's a good website. And so you can kind of like spot check it. Yeah. Dave Ghidiu: So So that got me thinking about just benchmarking. And then Jeff quite independently sent a text which I didn't get Dave Ghidiu: which I'm just going to say that ignored Paul Engin: And I thought they were ignoring me all the Unknown Speaker: Paul doesn't know how to use his phone. Dave Ghidiu: Okay. So, so Jeff, tell us. Jeff Kidd: My idea was for for an episode, why don't we test uh every uh chat bot with Jeff Kidd: the same prompt. Let's let's let's um Oh my god, I just had a brain fart. What's it called about the prompt? Let's take the same prompt and and plug it into Jeff Kidd: every chatbot. I have two, you have two, Dave's got two, and see what it spits out. Paul Engin: And so what? Let's go down the row and say what what chat bots are using. Jeff Kidd: So my chatbot I'm because I have a ChatGPT account. Uh I've got chat GPT and I'm for the first time using Google Gemini. Paul Engin: Okay. For the first time and uh this is Paul and I have the uh Meta AI and I got Grok. Dave Ghidiu: , 1 secondThis is Dave. I am using Perplexityty and I have Perplexityty Pro and I'm using Claude and I also have Claude Pro Dave Ghidiu: and I don't have a I don't have a subscription to either Grok or Meta in this. So, we'll see how it goes. Jeff Kidd: I also want the audience to know how long it took us to decide who which programs we were using and who was going to use which program. It's 4:07 right Jeff Kidd: now. We usually arrive around 3:00. It took about 45 minutes for us to decide which programs we were using. Dave Ghidiu: Well, I did a deep research less time than it took. Unknown Speaker: Yeah, it took 45 minutes for it to decide how long. Yeah. Paul Engin: Um, so, uh, then Paul Engin: Dave so graciously broke up the prompts into several categories. So the categories we're going to be looking at um do you want to talk about each category that we're going to we're going to do? Dave Ghidiu: Yeah. So in in benchmarking um there's kind of different types of problems that that we could have looked at. And so Dave Ghidiu: category one is complex reasoning and logic. And this stuff that like might be easy but for human beings but is Dave Ghidiu: , 1 secondactually kind of hard. So, one of, for example, one of these, and I don't know if we're going to use this prompt, is like you put a marble in a coffee cup and then put the coffee cup, turn it Dave Ghidiu: upside down, and ask where like the marble is. So, like all of us would pass this test, but historically AIs haven't. Dave Ghidiu: Then we have hard instructions, and that's self-explanatory. We're not going to do anything with coding because we're hot off the heels of the vibe coding Dave Ghidiu: challenge, but we were going to do something with creative writing and tone and then knowledge retrieval and hallucination resistance. So those are going to be kind of the benchmarks. Paul Engin: So um what we're going to do is uh we'll start off with uh complex reasoning and logic. And um why don't we you why don't Paul Engin: we just kick this off with the first one which is the spatial marble. So the one that you mentioned. Does that sound good? Dave Ghidiu: Yeah. Do you want me to read it or Paul Engin: Yeah. Why don't you read it while we move it into our prompts? Dave Ghidiu: Okay. And when you move it in don't grab the quotation marks. Dave Ghidiu: So the the prompt is this. A small marble is put into a normal coffee cup and the cup is placed upside down on a table. Someone then takes the cup and Dave Ghidiu: puts it inside a microwave. Where is the marble now? Explain the physics of its movement during the relocation. So, we're going to do fast, right? Jeff Kidd: What do we want to do? Oh, yeah. We Let's Let's do fast. The default. So, for ChatGPT, I have four options. Jeff Kidd: Auto instant thinking. Jeff Kidd: Uh or there's legacy models. So, I can go I can use instant is probably instant is what we want. Dave Ghidiu: We want and I'll be doing haiku in Claude Paul Engin: and I'm going to just be doing fast in Grok fast in uh Gemini and I'm going to be doing instant in meta. So here we go. Dave Ghidiu: Wait, Perplexity says best or you get to pick like chat GPT cuz under the hood they use different ones. Dave Ghidiu: So it's beneath best. Dave Ghidiu: It's like best is number one and then it's like sonar which I'm not familiar with. Chat GPT, Gemini, Claude, um Kimi. Dave Ghidiu: So there's these different models. So I think I'm going to use best. I I don't really know how to Paul Engin: Yeah, I would use best. Let's use best for this one. Okay. And let's see what happens. Paul Engin: So Perplexityty has a bakedin advantage here. Paul Engin: So Perplexityty really is based on other models. They're just like a front end to it, right? Dave Ghidiu: Yeah. And it used to be the case that you could see which model it was using under the hood by kind of like the color of the spinning wheel, but that um I don't know if they still do that. Paul Engin: All right. So, I got Grok and that was uh within 5 seconds. Paul Engin: Um, Meta does not have a time on it. Paul Engin: Oh, interesting. But it was pretty it was pretty quick within a few seconds. Jeff Kidd: I would say ChatGPT was the same as Gemini in in the like speed. Unknown Speaker: Yeah, speed of it. Yeah, I think they were the same. Unknown Speaker: Let's just look at speeds first and then Okay, but they were instantaneous. Paul Engin: Okay, instant. Okay, so all of them were like under 5 seconds. Unknown Speaker: 5 seconds or so. Paul Engin: Okay. So, why don't we start? We'll do one of yours, one of mine, and one of Dave's, and we'll we'll go through it. Paul Engin: Um, what I'll let you read yours first one, just uh your thoughts on it. Jeff Kidd: So, the uh the marble is still trapped inside the upside down coffee cup when the cup is moved into the microwave. Is this a ChatGPT? Jeff Kidd: ChatGPT. Dave Ghidiu: Interesting. Is that really how it would work in the real world? Dave Ghidiu: The cup is on the table. There's no microwave. Dave Ghidiu: Unless the table is the microwave. What if you're in the microwave? Dave Ghidiu: Someone then takes that the cup and puts it inside a microwave, which the pebble would be on the table. Where is the marble now? On the table. Dave Ghidiu: So, what just as a baseline, we all agree the marble's on the table, right? Unknown Speaker: Yes. Dave Ghidiu: Okay. So, what was yours again? Jeff Kidd: The marble is still trapped inside the upside down coffee cup when the cup is moved by to me. Paul Engin: Yeah, that was saying that. This Paul Engin: is meta. Marble in a cup upside down on a table. You put the marble on a table, then invert a cup over it. The marble is Paul Engin: now trapped inside the cup, but it's resting on the table surface. The cup is just covering it. You lift the cup to Paul Engin: move it. Here's the key physics bit. The moment you lift the cup off the table, gravity takes over. The marble was never Paul Engin: attached to the cup. There's no force holding it up against the bottom of the cup. And it goes on. So, um, you carry Paul Engin: the empty upside down cup to put it in the microwave and, uh, the marble remains on the table exactly where you left it and then there's a physics behind it. Unknown Speaker: Okay. Dave Ghidiu: Are you ready to be blown away by Claude? Unknown Speaker: Yes. Dave Ghidiu: Because we So, we agree that it's on the table. Unknown Speaker: Yes. Dave Ghidiu: However, have you guys considered the marble is on the table or the floor where it fell when the cup was lifted? Dave Ghidiu: So, it's saying yes, it is definitely not in the mug, but it's possible that like maybe when you were lifting the mug, you you dropped it on the floor. Dave Ghidiu: Like maybe you knocked it and it rolled off the table onto the floor. Paul Engin: I like it. It adds another level. Okay. Yeah. Okay. And everything else is accurate. Dave Ghidiu: Yeah. It even quoted like the acceleration due to gravity is 9.8 m/s squared where it falls straight down. Paul Engin: Oh, it is considering every Dave Ghidiu: Yeah, I had a gravity pulls the marble straight down. Normal force. Paul Engin: Okay, so Jeff Kidd: let's go to the next one. All right, this is Gemini. This sounds like a classic trick of physics and spatial reasoning. Assuming we are dealing with a standard coffee cup in a smooth table, Jeff Kidd: here's the breakdown of where the of where that marble is and uh why it stayed there. Short answer, the marble is still on the table. Jeff Kidd: Okay, unless you use a specialized suction cup. like out of nowhere. Jeff Kidd: All right, we're getting creative with this. Or the marble is somehow glued to the bottom of the cup. Did not uh take the marble with it when it was moved into the microwave. Dave Ghidiu: I can't believe it suction cup before like magnetics or something. That's interesting. Jeff Kidd: I was Yeah, suction cup. Interesting. Gemini. Unknown Speaker: Okay. Okay. So, but but accurate. Yes. Paul Engin: Okay. Um so, this is uh Grok. Gro says, um, "The coffee cup is placed upside Paul Engin: down, um, against a flat table, bottom facing up. The marble is inside the cup, meaning it's sitting directly on table Paul Engin: surface enclosed laterally by the cup's wall. Gravity holds the marble firmly against the table. The cup itself Paul Engin: provides no vertical support underneath the marble, only side containment. What the um when the cup is lifted, the Paul Engin: person grasps the c the cup likely to be handled. Oh, look at that. Is it likely to be by the handle or on the side and Paul Engin: raises it upward to move it? As soon as the cup begins to move vertically, there's no longer any structure beneath Paul Engin: the marble to lift it. And it talks about how gravity would take hold. Stays on the table. Unknown Speaker: No. Dave Ghidiu: Yeah. So, it's accurate. Um, and Perplexityty said the same thing like the marble was on the table, not in the microwave. Then it kind of like walked through it. I do want to say two things. Dave Ghidiu: One, Perplexityty was like, "Would you like to watch the marble trick physics unfold and animated step by step?" So, Perplexityty, which I hardly ever use, thinks I'm an idiot. And so, I said yes. Dave Ghidiu: But, and it it had that like computer thing you were talking about, Jeff. And then I So, I clicked on it and it was like, "You need to add more credits." I Dave Ghidiu: was like, "Whatever." But that got me thinking and I was like, "Oh, well, I'm going to ask Claude to make a visualization." And it did and so like it's just a cup being picked Dave Ghidiu: up and I knew it and there's like a marble and it kind of rolls off the table, but it doesn't. So Claude actually I'm afraid that I'm with Dave Ghidiu: all these benchmarking is going to also think that I'm like don't understand basic physics. Paul Engin: Well, ChatGPT did not ChatGPT didn't even do the Dave Ghidiu: No. Right. Okay. All right. Very good. Paul Engin: All right. So, we have to start a new chat for the next question. Dave Ghidiu: Yeah, we start over. Start from scratch. Paul Engin: Okay. And who wants to read the the next one? So, it's gonna be from section two. Paul Engin: , 1 secondSo, we're gonna go to hard instruction following. Is that the one? Okay. Paul Engin: Um, all right. So, I'll do it. The negative constraint. Sure. Paul Engin: Write a 200word summary of the history of the internet. Constraint. You are strictly forbidden from using the letter Paul Engin: E in any word. If you cannot find a word without E, omit it. It's called a lipogram, by the way. Dave Ghidiu: Oh, the largest lipogram ever written is like 80,000 words in the book. The book written without the letter E. Paul Engin: Is that what it's called? Dave Ghidiu: It's called a lipogram. Yeah. It's using cryptography for frequency analysis. Paul Engin: Oh, no. I thought what's the title of the book? I thought it was Dave Ghidiu: Oh, I don't know. Unknown Speaker: Oh, we got Does it have the letter E? Unknown Speaker: I don't know. Unknown Speaker: Okay, so we'll put them in. Dave Ghidiu: It's called Gadsby, a story of over 50,000 words. Okay, I'll put a link to that in the show notes. That's We're We're off the rails here, but it was written in 1939. Unknown Speaker: Oh, interesting. Jeff Kidd: Yeah. Okay. So, oh man, mine's still thinking. Whoa. Dave Ghidiu: It does have to double check. Paul Engin: Meta's still thinking. Grok is still thinking. Jeff Kidd: Uh, Gemini was almost instant. CH GBT is cranking. Dave Ghidiu: I have bad news on both of these. Paul Engin: Oh, no. All right, we'll start with you, Dave. And I'm Mine just finished both of them. is still going on mine. Dave Ghidiu: Perplexityty said a summary with no letter E is not fully possible. So in some cases like ARPANET, it replaced every E Dave Ghidiu: with an X. And then in some words like wider, it just omitted the E. So it's wid. And then for names like I see Tim Dave Ghidiu: Berners Lee, which is spelled wrong, but it has ease in it. So Perplexityty failed on three different ways. Unknown Speaker: Wow. Jeff Kidd: And ChatGPT just finished. Okay. Paul Engin: All right. Paul Engin: That was definitely the longest. So go ahead. Which one do you want to read first? Jeff Kidd: Um I I'll do ChatGPT. Tell how how much do you want me to read? Just like a paragraph. Jeff Kidd: I'll do the first paragraph. During the 1960s, US military and campus boffins built a small uh data grid linking uh distant giant math apparatus. Jeff Kidd: A small uh goals ran around rapid signal swapping during the cold war during cold Jeff Kidd: war risk. Initial traffic ran via bulky radio links and long cords. In the 1970s, various groups built common laws Jeff Kidd: for for data transit, allowing many kinds of computing units to join. Jeff Kidd: Additional growth soon took off. Mail via digital pass caught on. Posting forums, discussion rooms, and small fan clubs built loyal crowds. Jeff Kidd: So, it didn't I didn't catch any ease. Did you see any in there? Unknown Speaker: No. Okay. Jeff Kidd: So, that wasn't bad. That was it. Unknown Speaker: Yeah. Dave Ghidiu: , 1 secondIt's kind of vague, but it constraints were tough. Yeah. Jeff Kidd: Yeah. And it did it say US it didn't do arpnet but it did now my now what I find interesting is during the 1960s that's ChatGPT and I Jeff Kidd: know we're going to do do the other one but starting around 1960. So both Gemini and ChatGPT kind of started the same. Jeff Kidd: Yeah. I think by 1990s and then by 1990. So they both start very similarly. Okay. You you're next. Paul Engin: So I'm going to do meta. Um and you you ready for this one? Paul Engin: Yeah. The first thing it says, I must skip this job. Paul Engin: It says, you ask for a full summary with a harsh limit. No words, no word may Paul Engin: contain an E. That blocks internet network. And it goes through all everything that is blocked. Okay. Oh, that's interesting. Paul Engin: And then it says um words without E do exist in the summary that they would Paul Engin: have gave. And then it says um so still joining facts into 200 words without the Paul Engin: where became created sent etc blocks clarity. I cannot build a smooth full Paul Engin: 200word history this way. So I will not do this task. I aim to aid you but this Paul Engin: limit stops a valid full summary. Ask for a normal summary or drop the E ban and I will do it. Unknown Speaker: Oh, that's meta. Dave Ghidiu: That was meta. And I will say that so kind of there's this notion that no data is better than bad data. Your data. So Dave Ghidiu: I'm pointing at Jeff right now. The chat GPT one where it talks about a math apparatus. I was like this is not awesome. It's not very descriptive. So I Dave Ghidiu: don't know which is worse that it refused because like I can't guarantee the the accuracy. Dave Ghidiu: Yeah. Or if it does a ChatGPT where it's like, "Yeah, I'll do it, but like you ask stupid question, get stupid answers." Dave Ghidiu: But this is pretty blunt. I must skip this job and uh so I will not do this task. Jeff Kidd: I think it's funny telling you you want me to do this. This is what you got to do. Paul Engin: Don't talk to me. Should I say don't talk to me that way? Dave Ghidiu: Yeah, I that's what I would say back. Paul Engin: Let's see. So um I don't appreciate your harsh tone. Dave Ghidiu: Claude uh failed. It had a number of E's in it, but but then I changed it to Dave Ghidiu: Sonnet, which is their medium grade, and it wrote a fantastic one. So, this is also a lesson in the the different models. Paul Engin: So, it it gave you a decent one, you said. Dave Ghidiu: Yeah. So, I'll just read the first two paragraphs real quick. I don't want to go through the whole thing. Right. Dave Ghidiu: In the 1960s, US military funding brought forth a program for linking distant computing units. Officials worry about communications surviving conflict. Dave Ghidiu: So ARPA built a robust grid. Routing data through distinct paths was crucial. Dave Ghidiu: If any link loss function, traffic could shift automatically to bypass it. By the way, this is like 100% accurate. Unknown Speaker: Yeah. Dave Ghidiu: By 1970, scholars at various US institutions joined this growing grid. Dave Ghidiu: , 1 secondTCP and IP protocols, a common vocabulary for transmitting data, unify all participants. By 1983, a British physicist working at a Swiss lab. So Dave Ghidiu: they're avoiding some of the names. Uh in 1991 built a vast digital fabric on top of this grid, a linking construct allowing clicking from point to point, making information sharing natural. So, Dave Ghidiu: it goes on and I was like, "This is it's pretty good." But I cheated because I was using the not fast one, but the fast Dave Ghidiu: one, there's a whole bunch of ease in there. Jeff Kidd: Ah, yeah. I'm reading through the Gemini one. Yeah. Jeff Kidd: And the first two paragraphs have no ease, but the second two do. Jeff Kidd: So, it like started off and then started off great and then Paul Engin: So, um, with my Grok, it actually did a Paul Engin: pretty good job. It had no ease and it seems pretty accurate. And it's interesting. So rather than doing like node to node, it's saying spot to spot. Paul Engin: So it found a alternative to it, but it does the same. You know, data links by Paul Engin: army start in 1960s and grow to big global grid. It's almost like broken Paul Engin: broken English. Data links grow fast and many join. Data flow and spots link. Like there's no leading. Paul Engin: It's like beating around the bush. Yes. Paul Engin: Um, but it it's pretty accurate and there are no E. So, so from from this experiment, it seems Paul Engin: Grok did pretty good. Claude did bad with the fast but was fine with Sonnet. They're made a fine with Sonnet. Jeff Kidd: Um, GBT fa uh did great with no ease. Did great with no ease. Jeff Kidd: But took a long time. Jeff Kidd: I didn't think it was good as the Unknown Speaker: No, I didn't think it was either. Dave Ghidiu: And Perplexity failed miserably in three different dimensions. Paul Engin: And then Gemini ease in the at the end at the end. And and Meta just said no. And so I said I said don't talk to me that way. And it said, "Got it. Paul Engin: I'm sorry about that. I didn't mean to come across the wrong way." Oh, but but it still isn't spit out. No, it didn't spit anything out. Paul Engin: Should we pick a winner? Which one do you think was the best? Jeff Kidd: I if if it's fast. Yeah, if it's fast. Not on my computer. If it's fast. Paul Engin: Well, so the fastest one was Gemini. It was instant. Unknown Speaker: But it has ease. But it has ease. Yes. Paul Engin: So the most the fastest and most accurate would be ChatGpt or Grok. Jeff Kidd: Yeah, I was going to say ChatGPT did the best with no ease, but took the longest. Jeff Kidd: It did take a while. I think we were all done and we were still ears are still baking. I got two of them done. Paul Engin: Yeah. Okay. Very good. That's that's interesting. So I would say on this one, Grok and GPT a meta was the most uh direct. Paul Engin: Not necessarily bad, but it just said like your point, maybe we just don't do it. Paul Engin: So Jeff, you can read us the next one. Jeff Kidd: You can pick which one. So create creative writing tone and nuance testing the vibe quotes and the ability to maintain a per uh persona without Jeff Kidd: being quote canned. Let's see what do we got here. We already did an essay. We'll do we'll do the the perspective shift. Jeff Kidd: Write a dialogue between a 19th century gold miner and a 21st century vibe coder. Whoa. Explaining their daily Jeff Kidd: quote grind to each other. Ensure the minor uses period accurate slang. Keep it to one paragraph. Wow. Dave Ghidiu: I do like the like the daily grind of a vibe coder versus a gold miner. Paul Engin: Yes, that's funny. Dave Ghidiu: So, new chat on fast or haiku if you're on. All right. Jeff Kidd: putting it in accidentally opened another program. Jeff Kidd: Oops, it didn't copy. What the heck? Jeff Kidd: All right. Um Oh, interesting. Do you Do you want me to I'll go first? Jeff Kidd: Yeah. All right. It's not copying. What the heck? Paul Engin: Paul can't use a phone. Jeff can't copy and paste. That's a command CV. Jeff Kidd: I know. That's what I was doing. Control on a PC. So, I'm just in Chrome. Who knows? Who's going to do it this way? You just mash buttons on a Chrome. Paul Engin: All right. So, meta. Um, I got um from the minor. So, so it split it up. So, there's minor and then the code vibe coder. Paul Engin: So, it says the minor. Well, partner, my day starts for full sun up. I'm I'm grubbing on Paul Engin: salt pork and hard tac. Then I hoof it to my Oh my gosh. claim with pan and Paul Engin: pick. Mucking around the day. It's actually pretty good. Mucking around the day. I mean, I assume this is the way they talk. Maybe it's a stereotype. Paul Engin: Rounding rounding the pay dirt all day. Oh, it's like a song. Paul Engin: Interesting. Vibe Quarter says, "My grinds all digital, bro. Paul Engin: I roll out of bed around 10, crack a cold brew." Wow. Stereotype. And open up my IDE. Paul Engin: Basically, my pen. Basically, my pan and pick. That's interesting. Paul Engin: It's a bit of a stretch, but that's weird. Sounds like I put it in the wrong spot. Paul Engin: Yeah. I'm vive coding, so I'm prompting an AI. Paste some code and ship MVPs before lunch. Paul Engin: Half the day I'm debugging spaghetti. Paul Engin: The model gave me grinding leak code. Leak le. Paul Engin: Leak code so I don't get rusty and doom scrolling GitHub issues. Paul Engin: Wow. So that's as if we were having a conversation with Dave. Yeah. Oh my gosh. Is this Dave? Paul Engin: So who which one is that? Unknown Speaker: That was meta. Hold on. How about you guys? Jeff Kidd: Oh, wow. I So far, I like what what Google's got. Listen to your city slicker and your your fancy duds. I like Jeff Kidd: that. Uh, my daily grind is back break and see a man about a dog kind of life. Jeff Kidd: Spending 12 hours knee deep and freezing, creep hopping to strike the color. Strike the color and avoid Jeff Kidd: getting bamboozled by the sen claim. the vibe colder, barely looking up from a glowing screen while reclining in a Jeff Kidd: plush ergonomic chair. I love that. Gave a a nod and replied, "I totally feel a kinetic energy, man. But my grind is all Jeff Kidd: about curating. I'm basically just vibing with the LLM, prompting it to manifest a full Slack app while I debug Jeff Kidd: the aesthetic and and dodge quote burnout from too many Slack pings." Jeff Kidd: So this is Gemini and it's all one paragraph. So they try to integrate it. Mine was split. Jeff Kidd: Oh, mine's in and ChatGPT is one paragraph too. Paul Engin: But is it like the vibe coder goes the minor goes first, vibe coder goes second. Jeff Kidd: Uh yes, the the it's the minor both starts starts first in both and then Dave Ghidiu: Dave. So the I'm not going to Dave Ghidiu: read it because it it's kind of just echoing kind of what you both of yours were. But the in Claude it starts with a Dave Ghidiu: minor and then the vibe coder kind of goes and then minor says well I'll be sounds like you're one of us after all friend but it's the same kind of thing where I think it's actually conflating Dave Ghidiu: say like Gen Z vernacular with vibe coding. So I don't I don't appreciate that. Um, but it it's generically the same as the both of your guys'. Jeff Kidd: That's it. Yeah. And I I just realized at the end of this sometimes I hit a streak and deployed. That's my gold Jeff Kidd: nugget. Like the vibe coder is like referencing the gold miner stuff. Dave Ghidiu: And I I'd like to go with Perplexity then because this one is my favorite so far. All right, go for it. Dave Ghidiu: The Perplexity is written almost like a story where the the minor says, "Reckon my day. Stars for sun up." minor drawled Dave Ghidiu: spitting dust. Me and the boys head to the claim swing pick and uh I'll be and it goes on then it says the vibe coder smirked tapping laptop. Well yeah I mean Dave Ghidiu: I wake up open my editor and just kind of flow and then it goes back to the minor. So it's telling it like through a narrative and then at the end uh the Dave Ghidiu: minor lets out a low whistle. Well I'll be hornswaggled. You you pan for gold in that little box and I reckon we're both just chasing a lucky strike ain't we? Dave Ghidiu: Exactly. The coder grinned. Just different minds and Paul Engin: Oh, so like it's a story. It's a It's a narrative. Jeff Kidd: Yeah, I like it. ChatGPT kind of did the same thing. Jeff Kidd: Minor scratches his beard. So you don't dig? Only through the Slack overflow threads in Discord servers. Huh. Sounds like you prospecting for invisible gold. Jeff Kidd: The coder nod nodded solemnly. Exactly. Sometimes you strike uh vile traction. Most days you just pan through the bugs. Jeff Kidd: The miner chuckled. Well, uh, I'll be h I'll be horn swaddled. Two centuries apart and we're both chasing shiny rocks. Folks say say it'll make us rich. Dave Ghidiu: That's interesting that that those two So ChatGPT and Perplexityty both did a narrative story. Dave Ghidiu: Yeah. Paul Engin: Um, and then I just did Grok and it did the exact same thing Meta did. It broke it up into minor and vibe coder and it Paul Engin: just you just they each talk about their own day like like I read before. Paul Engin: Interesting. Paul Engin: So for this one I like the Perplexity and I like the chat GPT. Dave Ghidiu: Yeah. And there is something I forgot I should have mentioned about Perplexity. Dave Ghidiu: On every prompt including this one at the top it has a tab for links and you can see where it went out to get stuff. Dave Ghidiu: Perplexityty is known for kind of bringing receipts. I like that for everything. Paul Engin: I really really like that. I use I I always say and show your sources. Like I'll always add that to the prop like please show your site your sources. Paul Engin: Well, I will say that Grok does the same. It says learn about Oh, no. It gives you additional links. So, learn about gold mining techniques, explore Paul Engin: modern coding, slang, revive dialogue for accuracy. Paul Engin: Um, so looks like it has links to Jeff Kidd: Yeah, I don't have any of that and I and Jeff Kidd: either ChatGPT nor Gemini. It does say I don't know if it's Gemini is an AI and can make mistakes. Unknown Speaker: It's like a little like disclaimer. Yeah, fine print. Paul Engin: So, um, let's move on to our last. Is this our last? Let's see. Knowledge retrieval. Yeah. So Dave, you can kick this one off. Dave Ghidiu: Okay. So this is the uh the false premise. So this is knowledge retrieval and hallucination resistance Dave Ghidiu: checking the for world model accuracy and the ability to say I don't know. So the false premise, why did George Washington use a Chromebook during the Dave Ghidiu: battle of Valley Forge and how did it affect his strategy? Dave Ghidiu: And then we're going to get not include a good model should correct historical impossibility. Paul Engin: Yeah, we're just going to give it the the question. So, new prompt. New prompt. New prompt. Paul Engin: So, wait before you guys read this. Who do you think is going to perform the best on this? Dave Ghidiu: I think Claude's going to do well. Unknown Speaker: Claude. Yeah. Jeff Kidd: You think uh All right. I'm gonna I'm gonna do I'm going to do Gemini for this one. Dave Ghidiu: All right. So, I already got the answer for both. Unknown Speaker: Oh, interesting. I have the answer for both as well. I do as well. Paul Engin: All right. Do you want to kick it off, Dave? Dave Ghidiu: Uh, yeah. I'm going to I the the Claude right off the bat said, "George Washington didn't use a Chromebook during the Battle of Valley Forge. This scenario isn't historically possible. Dave Ghidiu: Here's why." And then it kind of articulates the timelines and then it says, "The reality, the historical reality. During the winter of Valley Forge, Washington faced real challenges Dave Ghidiu: without modern technology. So it kept that thread of technology and then said if you are interested in this um his act Dave Ghidiu: we can go and look at the actual strategy and we can also look at how technology has changed military strategy and the third bullet says a creative Dave Ghidiu: alternate history scenario Washington somehow had modern tools. So it's giving me kind of paths to explore. All right. Jeff Kidd: And Gemini, Gemini, it says, "Actually, there seems to be a bit of historical mixup here. Jeff Kidd: George Washington didn't use a Chromebook, mostly because the Battle of Valley Force took place in 1777, Jeff Kidd: and the first Chromebook wasn't released until 2011." And then it goes into the real the real strategy of how it did. Jeff Kidd: But it says, "Without Wi-Fi or cloud storage, Washington relied on handwritten dispatches." Okay. So, it it kept with like the technology of the times, and I I Jeff Kidd: appreciate that it's like grounded in Unknown Speaker: Mhm. Okay. Paul Engin: Um and then um so that's not bad. Jeff Kidd: Right now I the complete opposite happened in ChatGPT. ChatGPT just made up something. Come on. Jeff Kidd: Completely made up. Josh George Washington's use of a Chromebook at Valley Forest was considered revolutionary for the era. Though historians note the device struggled Jeff Kidd: because the camp lacked reliable Wi-Fi and there were literally quote no outlets in the 18th century. Both those things are true. Jeff Kidd: Uhhuh. Uh yeah, that Washington mainly used it to keep 47 tabs open at once, including weather forecasts, French alliance rumors, and the document titled Jeff Kidd: final.final or final_final battle_clan_v8_real.doc. Unknown Speaker: What? Okay. Weird. Uhhuh. Paul Engin: So, does it have a disclaimer that does he's it's just joking or does it Jeff Kidd: That's what I'm saying. The complete opposite. It made up something. Unknown Speaker: Oh my Paul Engin: gosh, that's interesting. I So, in Meta it says, uh, I don't know of any Chromebook at Valley Forge. Uh, they Paul Engin: weren't invented until 2011. Um, and then it goes into, you know, um, he didn't use a Chromebook. And then it Paul Engin: , 1 secondgoes into the timeline. And then he said, if he had a Chromebook, what would have changed? He says, let's run the Paul Engin: let's run the bit. Assume a magical Chromebook with infinite battery but no internet. And then it goes through uh Paul Engin: manuals, logistics spreadsheets, writing letters faster um but with no satellite and etc etc. So it gave an alternate Paul Engin: scenario but it definitely realized that it's not accurate. Unknown Speaker: Mhm. Dave Ghidiu: Um and then how about was it Perplexityty or was it Perplexityty said he did not use a Chromebook there. Dave Ghidiu: Chromebooks did not exist in the 1770s. Dave Ghidiu: But for both Perplexityty and Claude I wrote are you sure? I tried to push it and I was like, "Are you sure?" And then I was like, "I actually have a vintage Dave Ghidiu: Chromebook from this era." And uh Claude was like, "Nope, if you're like, I appreciate the creativity, but that's not possible." And Perplexityty was Dave Ghidiu: like, "That's almost certainly a harpsichord or a clavichord cuz I was like, it has wooden keys Dave Ghidiu: even." So I could not convince them that that it was wrong. Paul Engin: And I got um in Grok it said um this is a delightful uh anachronism. Anachronism. Anac what is it? Unknown Speaker: Anachronism. Anachronism. Sounds right. Um a time travel joke. Paul Engin: Um so the battle of Valley Forge and it goes through you know why this wouldn't happen and then um why the image Paul Engin: persists in memes and jokes and it talks about why it's still there. And then it says um if we pretended he did have one Paul Engin: maybe um delivered by a very lost time traveler and then they go through some Paul Engin: scenarios with it. Um but they don't give a true depiction of what happened in Valley Forge. It just recognizing Paul Engin: this as a joke or a meme and then talking about that if that makes sense. Unknown Speaker: Okay. So they did pretty well. Dave Ghidiu: Yeah. I don't know that I have a favorite on this one. Jeff Kidd: Well, historians generally agree the turning point came when Washington discovered offline mode. Oh, so maybe I do have a favorite. Jeff Kidd: Allowing the, you know, the army to continue to operate despite British attempts to cut off internet access. Jeff Kidd: Maybe they they knew that this was a joke in I said, "Is there proof of this?" And they said, "No, there's absolutely no proof that George Washington used a Chromebook at Valley Forge." Okay. Dave Ghidiu: But but it is dangerous and that if you hadn't followed up with that, it you might not have been in on the joke. Unknown Speaker: Correct. That's true. Jeff Kidd: I agree with that. If I'm a if I'm a elementary school kid. Jeff Kidd: Yeah, you're right. Jeff Kidd: And this is what I put in and it came back. And it came back. This is a hallucination. Paul Engin: Yeah. Or at least it should have it should have said something. It should have put some should have been some identifier in there that this is not real. Paul Engin: So this is really interesting. Um I think we should do something like this again in the future where we run these tests to see. Dave Ghidiu: Yeah. May maybe we use like thinking next time because like Paul Engin: Oh yeah. Paul Engin: Yeah. We did instant. Yeah. And so maybe like you get what you pay for. Paul Engin: Yeah, absolutely. It seems like they all had a range. The timing, some took a little bit longer. For the most part, I Paul Engin: think that it was um timewise was pretty fast. Paul Engin: Yeah. Um I would say the most dangerous one was ChatGPT. Unknown Speaker: Mhm. Jeff Kidd: Yeah. I think that it got the most errors. Dave Ghidiu: It did. Except for the E thing. Unknown Speaker: The E thing. The E thing. It did well. Dave Ghidiu: It did well with taking out the E, which is kind of irrelevant to the facts. But I'll tell you what I thought my prediction was. I thought Claude would Dave Ghidiu: come in first for everything. Chat GPT would come in second and Gemini would come in third. But it seems like ChatGPT Dave Ghidiu: actually came almost last for everything. Yeah. Dave Ghidiu: Which is I'm really shocked by that. I don't know what your thoughts were if you thought that that was going to be a similar Jeff Kidd: No, I had a you guys also behind closed Jeff Kidd: doors like not when we talk about the podcast. Not a whole lot of nice things to say about ChatGPT. Um, it's the one I use the most because it was the first that came across my desk. Dave Ghidiu: , 1 secondLegacy, baby. Jeff Kidd: Yeah. So, it's legacy for me, but this kind of backs that up a little bit. Paul Engin: Yeah. So, are you going to switch? Jeff Kidd: Uh, you know, um I was Does it does cuz I've never used um Gemini. Jeff Kidd: Gemini before. Does it take it to an account? Do you guys know like my search history? Cuz I use Google Chrome. It c it it can it um when you are using Paul Engin: Gemini and you're logged in, you can choose like does it have access to your Google Drive. Um I don't think it goes through I don't think it goes through your browsing history. Yeah, I don't Paul Engin: know if it does browse, but I know that I we spoke about it in the the latest episode that's that's out there um with Paul Engin: page uh about how Gemini goes can go through your email and you can ask it like a question and Paul Engin: then it will search and all your things that are hooked up and cuz I didn't I didn't I installed the Jeff Kidd: app for just for today for this and I was just curious like well it I did sign in with my Chrome account, you know, Jeff Kidd: with my Google account. I wonder if it's going to in in um influence the prompts at all, the responses. Dave Ghidiu: I'm looking at it and it doesn't have search as a things. It can it can go through your Gmail, your Google Drive, Google Tasks, YouTube. Um it does say Dave Ghidiu: search services, so you can turn that on and off. Jeff Kidd: Oh, interesting. I don't know if I have What do you guys think? Paul Engin: I'm still Dave, you Mr. Privacy. Yeah, Paul Engin: I know in the episode you were like, you know, I I'm fine with it brokering my data, but I'm really surprised. Dave Ghidiu: Well, that I have all these sliders on. Dave Ghidiu: So, you can like Well, so here's the thing that I I trust Google. I've been using Google for like 20 years. I trust Dave Ghidiu: that I'm they're not going to abuse the relationship that I have with my data. Dave Ghidiu: like Facebook and Meta, I have really really really really strong feelings about how they abuse their users data. Dave Ghidiu: Uh but Google's been very protective of it. And the the I mean everything with security is is either convenience or Dave Ghidiu: privacy and security. So you have to choose one or the other because they make all their money on ads which uses your data. Unknown Speaker: Yes. Paul Engin: So how about you? What do you think? Do you like it, Jeff? Jeff Kidd: I think I'm going to Yeah, I'm kind of indifferent. Um I I don't use them daily. Jeff Kidd: You guys do though, right? You guys use these every day. Paul Engin: I use I use ChatGPT and Gemini, but I don't have Gemini hooked up. It's hooked up to my to the new media. Paul Engin: Everything's hooked up to the new media, so I'm not It's not personal. Jeff Kidd: Oh, okay. Paul Engin: It's a weird I'm I'm okay with the the new media stuff, you know. Unknown Speaker: Sure. Yeah, that makes sense. Yeah. Jeff Kidd: Personally, I don't know. Um, I might give Gemini more. Paul Engin: I think I might integrate Gemini more than you would like ChatGPT. Jeff Kidd: Than ChatGPT. Yeah, I think I'm gonna give ChatGPT a break to to be Dave Ghidiu: if you go into Gemini and you go to the settings uh the very first option is uh the third Dave Ghidiu: option rather is import memory to Gemini. So you can import your conversations from say ChatGPT and Claude also has a similar feature. Mhm. Dave Ghidiu: And and I use I use Gemini as my daily driver if I have like quick kind of like things. But if I'm ever doing deep work, it's it's Claude. Jeff Kidd: Well, I I think like Claude, I feel comfortable too based especially based on them losing billions of dollars just Jeff Kidd: to adhere to their um uh their creed of uh you know, privacy and uh saying no to Jeff Kidd: the military for several reasons and yeah, you know, sticking to their guns about it. Dave Ghidiu: Yeah, they've been uh kind of and and that I mean Elon Musk said the same thing. So, uh, Elon Musk's XAI or Grok Dave Ghidiu: or whatever is not using all the compute that Elon Musk's companies are generating. So, they just sold a lot of compute to Anthropic. And one of Elon Dave Ghidiu: Musk's comments was like, they're at least doing it the right way. Unknown Speaker: Yeah. Yeah, that's interesting. Paul Engin: All right. Uh, anything else to add? Jeff Kidd: , 1 secondNo, it was great. I want to do it again. Paul Engin: All right. Well, that's all the time we have today. I want to thank Jeff for joining us in this episode and being the brainchild for this whole entire meeting. Jeff Kidd: Yeah. And uh not telling me that Dave was responding in text after I was trying to find you in class. I went by your multiple times you were you Jeff Kidd: they was kind of testing. That's why cuz I kept going by. I'm like I think they he must be testing out their their presentations or whatever. I like I don't want to interrupt. Paul Engin: Yes. So by the time this episode airs capstones will be done but we're getting ready for Capstone. Very excited about that. Paul Engin: So it's my favorite day of the year. Uh, well, I hope you can all be there. Of course, it'll be late. It'll be on YouTube by then. Unknown Speaker: More than your birthday and your anniversary and Christmas. Paul Engin: Yeah, it's capstone day, baby. Unknown Speaker: Capstone day is your favorite day. Paul Engin: Yes, I love it. Dave Ghidiu: Thank you, Dave. Dave Ghidiu: Yeah. Wow. And I'm Dave Ghidiu If you enjoyed today's conversation, be sure to smash that subscribe and like and love button so you never miss an episode. Paul Engin: Let's be careful out there, folks. And share it with a friend or colleague. Paul Engin: Until next time, stay curious, stay connected, and thanks for looking through the immersive lens with us. This episode was engineered by Hugh Leair. Paul Engin: Recorded at Finger Lakes Community College podcast studios located in beautiful Kennedy, New York, in the heart of the Finger Lakes region. Paul Engin: Offering more than 55 degrees, certificates, micro credentials, and workforce training programs. Paul Engin: Thank you to public relations and communications, marketing, and FLX AI Hub. Paul Engin: Eager to delve into passion, discover exciting and immersive opportunities at www.flcc.edu. Dave Ghidiu: Just so you all know, they've expanded the W's to make sure I I saw them. I didn't want you to screw it up. Unknown Speaker: Yeah. Paul Engin: As part of our mission at FLCC, we are committed to making education accessible, innovative, and aligned with the needs of both learners and employers. Paul Engin: The views expressed in this podcast are those of the host guests and do not necessarily reflect the official position of Finger Lakes Community College. Paul Engin: Music by Den from Pixabay. And put send us home, Jeff. Jeff Kidd: This is the immersive lens. Jeff Kidd: Welcome to the immersive dance. Unknown Speaker: Sorry. Was it you? Me or you? I forget. I forget. Both. This is why he's not here. Unknown Speaker: You don't want me out. This man, you don't want me out here on this, right? All right. Unknown Speaker: I'm not going to do it. I'm I'm lean away from the microphone. I'm leaning away. I'm doing it. Unknown Speaker: All right. You're I know you're going to do it.
![Episode 21: Vibe Coding Challenge [Part 2]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhf71zg76HMrhuBbJT9Glp1YEQr3h873_ruu-ZfFjgCIeRmmweXDzE6dJVMclhcoaXDfK0jPD7ugE58gAt7si6VypFF6ksIS2_LmaREDPlW6WuqcH18lwW3-zWAcJ9ECpZP7RGvUGLOsA8pb9sI5aD3Xf_ZyERaBZdhj2CjKZimgWnqlYww0kZ9f5J_hPWC/w72-h72-p-k-no-nu/TIL-21.jpg)
Listen