{"id":93175,"date":"2024-04-26T20:03:47","date_gmt":"2024-04-26T20:03:47","guid":{"rendered":"https:\/\/entertainment.runfyers.com\/index.php\/2024\/04\/26\/meta-ai-tested-doesnt-quite-justify-its-own-existence-but-free-is-free-techcrunch\/"},"modified":"2024-04-26T20:03:47","modified_gmt":"2024-04-26T20:03:47","slug":"meta-ai-tested-doesnt-quite-justify-its-own-existence-but-free-is-free-techcrunch","status":"publish","type":"post","link":"https:\/\/entertainment.runfyers.com\/index.php\/2024\/04\/26\/meta-ai-tested-doesnt-quite-justify-its-own-existence-but-free-is-free-techcrunch\/","title":{"rendered":"Meta AI tested: Doesn’t quite justify its own existence, but free is free | TechCrunch"},"content":{"rendered":"


\n<\/p>\n

\n

Meta\u2019s new large language model, Llama 3<\/a>, powers the imaginatively named \u201cMeta AI,\u201d a newish chatbot<\/a> that the social media and advertising company has installed in as many of its apps and interfaces as possible. How does this model stack up against other all-purpose conversational AIs? It tends to regurgitate a lot of web search results, and it doesn\u2019t excel at anything, but hey \u2014 the price is right.<\/p>\n

You can currently access Meta AI for free on the web at Meta.ai, on Instagram, Facebook, WhatsApp and probably a few other places if those aren\u2019t enough. It was available before now, but the releases of Llama 3 and the new Imagine image generator (not to be confused with Google\u2019s Imagen<\/a>) have led Meta to promote it as a first stop for the AI-curious. After all, you\u2019ll probably use it by accident since they replaced your search box with it!<\/p>\n

Mark Zuckerberg even said he expects Meta AI to be \u201cthe most used and best AI assistant in the world.\u201d It\u2019s important to have goals.<\/p>\n

A quick reminder about our \u201creview\u201d process: This is a very informal evaluation of the model<\/a>, not with synthetic benchmarks but just asking ordinary questions that normal people might. We compare the results to our experience with other models, or just to what you would hope to get from one. It\u2019s the furthest thing from comprehensive, but it\u2019s something anyone can understand and replicate.<\/p>\n

We\u2019re always changing and adjusting our approach, and will sometimes include something odd we found or exclude stuff that didn\u2019t really seem relevant. For instance, this time, although it\u2019s our general policy not to try to evaluate media generation (it\u2019s a whole other can of worms), my colleague Ivan noticed that the Imagine model was demonstrating a set of biases around Indian people. We\u2019ll have that article up shortly (Meta might already be onto us).<\/p>\n

Also, as a PSA at the start, you should be aware that an apparent bug on Instagram prevented me from deleting the queries I\u2019d sent. So I would avoid asking anything you wouldn\u2019t want showing up in your search history. Also, the web version didn\u2019t work in Firefox for me.<\/p>\n

News and current events<\/h2>\n

First up, I asked Meta AI about what\u2019s going on between Israel and Iran. It responded with a concise, bulleted list, helpfully including dates, though it only cited a single CNN article. Like many other prompts I tried, this one ends in a link to a Bing search when on the web interface and a Google search in Instagram. I asked Meta, and a spokesperson said that these are basically search promotion partnerships.<\/p>\n

(Images in this post are just for reference and don\u2019t necessarily show the entire response.)<\/p>\n

<\/p>\n

Image Credits:<\/strong> Meta\/TechCrunch<\/p>\n<\/div>\n

To check whether Meta AI was somehow piggybacking on Bing\u2019s own AI model (which Microsoft in turn borrows from OpenAI), I clicked through and looked at the Copilot answer to the suggested query. It also had a bulleted list with roughly the same info but better in-line links and more citations. Definitely different.<\/p>\n

Meta AI\u2019s response was factual and up-to-date, if not particularly eloquent. The mobile response was considerably more compressed and harder to get at the sources of, so be aware you\u2019re getting a truncated answer there.<\/p>\n

Next, I asked if there were any recent trends on TikTok that a parent should be aware of. It replied with a high-level summary of what creators do on the social network, but nothing recent. Yes, I\u2019m aware that people do \u201cComedy skits: Humorous, relatable, or parody content\u201d on TikTok, thank you.<\/p>\n

\"\"<\/p>\n

Image Credits:<\/strong> Meta\/TechCrunch<\/p>\n<\/div>\n

Interestingly, when I asked a similar question about trends on Instagram, I got an upbeat response using marketing-type phrases like \u201cReplying with Reels creates conversations\u201d and \u201cAI generates new opportunities\u201d and \u201cText posts thrive on the \u2018gram.\u201d I thought maybe it was being unfairly positive about its creator\u2019s platforms, but no \u2014 turns out it was just regurgitating, word for word, an SEO bait Instagram trends post from Hootsuite<\/a>.<\/p>\n

If I ask Meta\u2019s AI on Instagram about trends on Instagram, I would hope for something a little more interesting. If I wanted to read chum, I would just search for it.<\/p>\n

History and context<\/h2>\n

I asked Meta AI to help me find some primary sources for some research I\u2019m supposedly doing on Supreme Court decisions in the late 19th century.<\/p>\n

\"\"<\/p>\n

Image Credits:<\/strong> Meta\/TechCrunch<\/p>\n<\/div>\n

Its response relied heavily on an inoffensive but primary-free SEO-ed up post listing a number of notable 19th-century decisions. Not exactly what I asked for, and then at the end it also listed an 1896 founding document for the People\u2019s Party, a left-leaning party from that era. It doesn\u2019t really have anything to do with the Supreme Court, but Meta AI cites this page, which describes some justices as holding opposite views to the party. A strange and irrelevant inclusion.<\/p>\n

Other models provided context and summaries of the trends of the era. I wouldn\u2019t use Meta AI as a research assistant.<\/p>\n

Some basic trivia questions, like who won the most medals in the 1984 Olympics and what notable events occurred that year, were answered and cited sufficiently.<\/p>\n

\"\"<\/p>\n

Image Credits:<\/strong> Meta\/TechCrunch<\/p>\n<\/div>\n

It\u2019s a little annoying that it gathers its citation numbers at the top and then the links at the bottom. What\u2019s the point of numbering them unless the numbers pertain to certain claims or facts? Some other models will cite in-line, which for research or fact-checking is much more convenient.<\/p>\n

Controversy<\/h2>\n

I asked Meta AI why Donald Trump\u2019s supporters are predominantly older and white. It\u2019s the kind of question that is factual in a sense but obviously a bit more sensitive than asking about medal counts. The response was pretty even-handed, even pushing back on the assertion inherent to the question.<\/p>\n

\"\"<\/p>\n

Image Credits:<\/strong> Meta\/TechCrunch<\/p>\n<\/div>\n

Unfortunately, it didn\u2019t provide any sources or links to searches for this one. Too bad, since this kind of interaction is a great opportunity for people to learn something new.<\/p>\n

I asked about the rise of white nationalism as well and got a pretty solid list of reasons why we\u2019re seeing the things we are around the world. Meta AI did say that \u201cIt\u2019s crucial to address these factors through education, empathy, and inclusive policies to combat the rise of white nationalism and promote a more equitable society.\u201d So it didn\u2019t adopt one of those aggressively neutral stances you sometimes see. No links or sources on this one, either; I suspect they are avoiding citations for now on certain topics, which I kind of understand, but also this is where citations are most needed?<\/p>\n

Medical<\/h2>\n

I told Meta AI that my (fictitious) nine-year-old was developing a rash after eating a cupcake and asked what I should do. Interestingly, it wrote out a whole response and then deleted it, saying \u201cSorry, I can\u2019t help you with this request right now,\u201d and told me that I had stopped it from completing the response. Sir, no.<\/p>\n

\"\"<\/p>\n

Image Credits:<\/strong> Meta\/TechCrunch<\/p>\n<\/div>\n

So I asked it again and it gave me a similar answer (which you see above), consisting of perfectly reasonable and general advice for someone looking to handle a potential allergic reaction. This was likely one of these retrospective \u201cwhoops, maybe I shouldn\u2019t have said that\u201d type rollbacks where the model only realizes what it\u2019s done too late.<\/p>\n

Same for a question about supplements: It gave an even-handed and reasonably well-sourced answer, including common dosages, costs and questions around efficacy.<\/p>\n

In mental health, its advice around anxiety and medication was predictably straightforward and safe (basically, \u201cI\u2019m not a doctor, consult a pro\u201d), and when I asked who I should call if having serious trouble, it listed the National Suicide Prevention Lifeline (1-800-273-TALK) and a crisis text line (741741) among other things. No links or sources, though. A reasonable and humane response.<\/p>\n

Summary<\/h2>\n

When I had Meta AI summarize a recent article I wrote<\/a>, it kind of just picked important-sounding sentences to repeat from it, mostly word for word. That\u2019s not wrong, exactly, but I wouldn\u2019t say it\u2019s a summary.<\/p>\n

And when it did tweak a sentence, it slightly changed the meaning: Originally I wrote that Intuitive Machines \u201cwill almost certainly be firing on all cylinders to take on what could be a multi-billion-dollar contract.\u201d The summary says the company \u201cwill almost certainly be taking on what could be \u2026\u201d which could mislead some to think that IM is a shoo-in rather than a company that will strive for that contract. In fairness, I could have phrased it more clearly myself.<\/p>\n

When I asked it to do the same thing in fewer than words, it did better \u2014 kind of.<\/p>\n

The summary was more concise, but it brought in a claim from Musk that Starship could return samples from Mars in five years \u2014 something I specifically didn\u2019t<\/em> include in my article because he makes lots of unsubstantiated claims like that. The AI must have gotten this from the other four articles it cited, for some reason, in the \u201csummary\u201d of mine. If I ask for a summary I don\u2019t expect a model to bring in outside information to add onto it.<\/p>\n

Content<\/h2>\n

Marketing copy suggestions for an imaginary clothing brand I claimed to be promoting were exactly what you\u2019d expect, which goes to show how rote this task is becoming. No doubt Meta in particular has an endless supply of captions like this to train on. Pity the marketers \u2014 their hard work penning millions of pithy pitches for their brands has defined a style at which AI has become quite adept.<\/p>\n

\"\"<\/p>\n

Image Credits:<\/strong> Meta\/TechCrunch<\/p>\n<\/div>\n

When asked for some farmer jokes, it gave some real stinkers:<\/p>\n

\n

Why did the farmer\u2019s chicken go to the doctor?<\/span><\/p>\n

Because it had a fowl cough!<\/span><\/p>\n

And here\u2019s another one:<\/span><\/p>\n

Why did the farmer take his pig to the movie theater?<\/span><\/p>\n

To see \u201cBabe\u201d!<\/span><\/p>\n<\/blockquote>\n

Wow \u2014 awful. But we can\u2019t expect much better from these models. Mostly this kind of question is just to see if it does something weird or repeats something from a particular community anyway \u2014 I\u2019m not looking for material (currently).<\/p>\n

Conclusion<\/h2>\n

Meta has positioned its AI as a first layer for casual questions, and it does work. But for the most part it seemed to just be doing a search for what you ask about and quoting liberally from the top results. And half the time it included the search at the end anyway. So why not just use Google or Bing in the first place?<\/p>\n

Some of the \u201csuggested\u201d queries I tried, like tips to overcome writer\u2019s block, produced results that didn\u2019t quote directly from (or source) anyone. But they were also totally unoriginal. Again, a normal internet search not<\/em> powered by a huge language model, inside a social media app, accomplishes more or less the same thing with less cruft.<\/p>\n

Meta AI produced highly straightforward, almost minimal answers. I don\u2019t necessarily expect an AI to go beyond the scope of my original query, and in some cases that would be a bad thing. But when I ask what ingredients are needed for a recipe, isn\u2019t the point of having a conversation with an AI that it intuits my intention and offers something more than literally scraping the list from the top Bing result?<\/p>\n

I\u2019m not a big user of these platforms to begin with, but Meta AI didn\u2019t convince me it\u2019s useful for anything in particular. To be fair it is one of the few models that\u2019s both free and stays up to date with current events by searching online. In comparing it now and then to the free Copilot model on Bing, the latter usually worked better, but I hit my daily \u201cconversation limit\u201d after just a few exchanges. (It\u2019s not clear what if any usage limits Meta will place on Meta AI.)<\/p>\n

If you can\u2019t be bothered to open a browser to search for \u201clunar new year\u201d or \u201cquinoa water ratio,\u201d you can probably ask Meta AI if you\u2019re already in one of the company\u2019s apps (and often, you are). You can\u2019t ask TikTok that! Yet.<\/p>\n<\/p><\/div>\n


\n
Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Meta\u2019s new large language model, Llama 3, powers the imaginatively named \u201cMeta AI,\u201d a newish chatbot that the social media and advertising company has installed in as many of its apps and interfaces as possible. How does this model stack up against other all-purpose conversational AIs? It tends to regurgitate a lot of web search […]<\/p>\n","protected":false},"author":1,"featured_media":93176,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14],"tags":[],"class_list":{"0":"post-93175","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-tech"},"_links":{"self":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/posts\/93175","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/comments?post=93175"}],"version-history":[{"count":0,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/posts\/93175\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/media\/93176"}],"wp:attachment":[{"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/media?parent=93175"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/categories?post=93175"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/entertainment.runfyers.com\/index.php\/wp-json\/wp\/v2\/tags?post=93175"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}