\n<\/p>\n
How much control are you willing to give an LLM over your digital life?\u00a0<\/p>\n
Getting the most value from a model means giving it the keys. For a control freak or the AI hesitant, it seems like a lot. For Andrew Ambrosino, the lead engineer for OpenAI\u2019s desktop app, it\u2019s the only way to test the future, which is why that app now has access to, and control over, his inbox, his Slack account, his phone, apps like Notion and Figma, and more.<\/p>\n
\u201cIf I\u2019m asking it to write a document, is there a possibility that it\u2019s going to pull from a private DM on that subject and not know that it\u2019s not supposed to share some info? Yes,\u201d Ambrosino told TechCrunch. \u201cI\u2019ll do it for the job. I will take the personal hit here and there if I have to. And I haven\u2019t had to.\u201d<\/p>\n
Ambrosino works on OpenAI\u2019s biggest bet, ChatGPT Work, which was released last month and is available on the company\u2019s lowest subscription tier, for $20 a month. The product is intended to allow white collar workers to field AI agents \u2013 hooking LLMs up to the digital workflows used by accountants, investors, doctors and everyone else whose day-to-day is dominated by their computer.\u00a0<\/p>\n
OpenAI\u2019s marketing copy<\/a> puts the goal succinctly: A world where \u201cwhere [artificial] intelligence goes beyond answering questions to helping everyone turn their biggest ideas into reality.\u201d<\/p>\n For software developers, that shift is already happening, but it\u2019s been slow to spread to other departments. ChatGPT Work is a modified version of the company\u2019s Codex coding tool. It\u2019s meant to give non-engineers a version of the same functionality that software engineers already get from agents: an AI tool that doesn\u2019t just answer questions, but completes multistep projects on its own.<\/p>\n \u201cIn this new factor, ChatGPT can actually do entire, very complicated tasks for you all autonomously in a way that is delightful and safe,\u201d Tibault Sottiaux, who leads OpenAI\u2019s core product work, including Work, told TechCrunch. \u201cIt\u2019s the very mission of OpenAI\u2014to bring everyone along.\u201d<\/p>\n Commercially, that matters a lot. Agents that work for longer stretches burn through more tokens, which makes them more lucrative for OpenAI on a per-user basis. Reaching new professions is crucial \u2013 not just for OpenAI, but for the industry at large. If coding has proven lucrative territory for AI labs, it\u2019s still a tiny subset of the professional work AI tools need to enable if these companies are to justify their massive investment in training and computation. While labs have been focused on software engineers, vertical-specific competitors like Harvey (for law) and Clay (for sales) have been chasing those customers with a model-agnostic approach, meaning they\u2019ll plug in whatever AI works best at the time.<\/p>\n Industry analysts see this as one of the major challenges facing OpenAI and its competitors. \u201cIf the labs cannot rapidly get ahold of the key complementary assets needed to scale AI in the market, value will accrue elsewhere,\u201d Christian Catalini wrote<\/a> on a16z\u2019s Time to Build blog.\u00a0<\/p>\n Making the AI apps work for people who aren\u2019t software engineers requires more hand-holding. OpenAI\u2019s non-engineering workforce, like the communications and finance teams, started using Codex \u201cat a time that it was actively hostile to them\u2014asking them about code and showing them, \u2018oh, you have an empty diff for this thing,\u2019\u201d Ambrosino said, referring to a technical readout meant for software changes. \u201cSo, we started to make it more general purpose between February and now.\u201d<\/p>\n An OpenAI-backed study found that in June, 98% of OpenAI employees were using Codex, but just 17% of organizational subscribers and less than 1% of individual subscribers were using the agentic coding tool. That difference between near total adoption inside the company and negligible adoption outside it is the challenge and opportunity for the company.<\/p>\n \u201cThe more value and the more utility that we generate for users, the more they will be willing to also pay for some part of that utility, and that\u2019s how we\u2019ve always seen ChatGPT as well,\u201d Sottiaux said. \u201cYou sit there and you\u2019re like, \u2019of course I want to pay $20 bucks a month for this,\u2019 because the value that you get is so much more.\u201d<\/p>\n To understand that disconnect, it helps to understand what OpenAI\u2019s engineers are building. Every LLM requires what engineers call a \u201charness\u201d\u2014the software wrapped around a model that decides what information it sees, what tools it can use, and how it presents its answers back to you.\u00a0<\/p>\n If you want that model to do stuff\u2014to become an agent\u2014the harness gives it tools and instructions for using them on long-term tasks. For developers, a command-line interface (CLI) that enabled LLMs to code was enough to change the way software was built and deployed. But most people aren\u2019t using CLIs; there\u2019s a reason Windows replaced DOS.\u00a0<\/p>\n An agentic product that goes beyond software engineering is \u201cgoing to be something that plays with the messy world of your life and your tools and websites that were built in 1995 and never updated,\u201d Ambrosino told TechCrunch, explaining that the experiences his team is building are vital to expanding access to useful AI.<\/p>\n Consider apps like Claude Code and Codex: They unleashed \u201cvibecoding\u201d by abstracting away all the actual software writing, and letting users just tell the model what they want in a program. Now, OpenAI wants to make functionality found in tools like OpenClaw, which coders use to put LLMs to work, as easy as prompting.\u00a0<\/p>\n \u201cWithout these products in front of the model, experts would know how to get the same results, but you wouldn\u2019t get to a billion people using the thing,\u201d Ambrosino said. That trade-off between what power users need and what mainstream adoption requires plays out in internal debates at OpenAI, where some employees argue that a button is unnecessary if users can just ask the model directly.<\/p>\n \u201cWe push back on [that]\u2014 because it\u2019s very early,\u201d Ambrosino said. \u201cDiscoverability matters in this phase, and at some point we won\u2019t have the button.\u201d\u00a0<\/p>\n Work has a few more buttons for selecting projects and plugins, but it aims for the same magic box interface as other OpenAI products.He compares it to skeuomorphism, the fading practice of making digital tools look like the physical objects they replaced, like a calculator app made to look like a pocket calculator. \u201cThat stuff wasn\u2019t just cringe design. That actually helped get people into this [and] make the transition,\u201d Ambrosino said.<\/p>\n \u00a0OpenAI wouldn\u2019t say how many people used Work versus Codex, but the joint app is used by just 20 million people, compared to\u00a0 more than a billion users the company says are prompting ChatGPT online.\u00a0<\/p>\n For now, OpenAI is pitching this tool as best suited for routine, data-intensive coordination tasks. Its employees are setting up weekly metrics reports<\/a>, for example, and making spreadsheets into planning tools<\/a>.<\/p>\n I\u2019ve spoken to VCs using agents to assemble relevant communications and analysis about companies into investment memos, and ops teams spinning up bespoke dashboards and data visualizations. Sam Altman is using it to plan his vacations<\/a>. One OpenAI engineer described asking the program to look at a Slack conversation about an engineering problem and \u201cmake some charts,\u201d then receiving back a series of insightful plots.<\/p>\n \u201cThere is a deluge of information for the average worker or employee of any of these companies, including myself,\u201d Akshay Nathan, who leads the product engineering team at OpenAI, said. \u201cWe\u2019re actually quite limited by our ability to parse everything that\u2019s available to us, and then take action on it. That information lives in all these system records tools [like, Salesforce\u2026the value of ChatGPT is you already have access to this, but now you truly<\/em> have access to it.\u201d<\/p>\n This, then, could be the digital personal assistant that AI evangelists dream about. As with Claude Cowork or Perplexity AI browsing agent, ChatGPT Work links agents to your existing workspace \u2014 email, web browser, a slew of SaaS platforms \u2014 and puts that context to work for you.<\/p>\n When the system works, it can be impressive: I asked ChatGPT Work to get my son\u2019s weirdly-formatted preschool school calendar out of my email and put it into my Google Calendar, and it did, saving me a lot of repetitive data entry. Hopefully now I won\u2019t forget the school potluck or fail to arrange vacation childcare.<\/p>\n I didn\u2019t trust OpenAI with access to my inbox, source interviews or story drafts (fear not, AI haters) and wouldn\u2019t let it have access to my bank account, but I believe it would be more useful had I the faith. I tasked it to do financial analysis on publicly-traded companies that I cover, and it delivered an auto-updating dashboard of metrics for me; it made a queryable database of space launches, a task I\u2019d previously had to accomplish by writing python scripts. It also sends me a weekly email about new AI research posted at academic clearinghouses. I\u2019ll keep experimenting with it.<\/p>\n While asking the model for something is intuitive, giving it what it needs to take action isn\u2019t as simple. Setting up the permissions for agents to access, say, a cloud drive was confusing and circular \u2014 I tried multiple times to give it just \u201cread\u201d access and received error messages. The model itself wasn\u2019t too helpful, but eventually on the mobile app, a dialog box popped up to tell me that only complete access would make it work.\u00a0<\/p>\n Many important settings are only available on the web app, so I frequently found myself working in both at the same time. Sometimes ChatGPT Work\u2019s limitations are baffling\u2014link it to your Google calendar and it can create events, but not new calendars. And don\u2019t bother trying to do anything unless the effort level is high, otherwise you\u2019ve got the worst intern you\u2019ve ever worked with.<\/p>\n That\u2019s common advice from AI early adopters, who fear that frustrated newbies will give up. Joe Gershenson, the engineering lead for OpenAI\u2019s harness, admitted that effort settings aren\u2019t intuitive for new users yet\u2014\u201dthere are things that we can do better to help them get the right level of reasoning\u2026\u201d he said, adding, \u201cWatch this space.\u201d<\/p>\n OpenAI faces another important challenge breaking into normie white-collar work: Most workflows aren\u2019t as measurable\u2014or evaluable\u2014as code. Software either works or it doesn\u2019t, and even that distinction reduces the nuance about what makes code good or bad. A good presentation, business strategy, or sales pitch isn\u2019t as easy to evaluate or trace.\u00a0<\/p>\n \u201cOne of the unique challenges with a product like this is just that it can really do anything,\u201d Ambrosino said. When I asked what specific problems the team designs around, and what workflows it targets, the engineers I spoke with demurred, saying that was a question for OpenAI\u2019s research team.\u00a0<\/p>\n OpenAI later provided an answer, telling TechCrunch that it uses its benchmark GDPVal<\/a>, drawn from 44 occupations and hundreds of knowledge work tests, and supplements that with user feedback.A less official answer is that it comes from OpenAI employees themselves. Said Ambrosino, \u201cWe have to always parse out \u2026 are we doing the workflow that everybody else will be doing, or are we weird?\u201d\u00a0<\/p>\n The early adopters of the app itself will create valuable traces with their actual usage\u2014much of the success of coding tools is built on similar data collection\u2014assuming they don\u2019t opt out of making it available for training. (I did).\u00a0<\/p>\n Despite all the attention on the model interface, OpenAI\u2019s engineers were reluctant to answer a fairly simple question: What sets Codex and ChatGPT Work apart from Claude CoWork, or other competing agentic harnesses intended for a mass user base?<\/p>\n \u201cIt\u2019s going to be a really disappointing answer for you, and I\u2019m sorry, but the honest answer is that I really don\u2019t look at the harnesses that they\u2019re building,\u201d Gershenson said in a typical answer. \u201cThe Mad Men \u2018I don\u2019t think about you at all\u2019<\/a> meme comes to mind here.\u201d<\/p>\n Frankly, I don\u2019t believe them, if only based on the extreme similarities between the products\u2019 user interfaces, the need for competitive intelligence at any business, and because the first thing ChatGPT Work asked me to do when I started it up was port over my Claude Cowork data.\u00a0<\/p>\n It\u2019s understandable if Claude Code is a sensitive topic around the OpenAI offices. Their corporate rival defined the market for AI coding and launched a revolution in how software engineers do their jobs. It\u2019s additionally frustrating because OpenAI had the idea first, but didn\u2019t quite harness it correctly.\u00a0<\/p>\n When OpenAI first developed Codex as a web app, the engineers got a bit over their skis\u2014or, as Ambrosino puts it, \u201ca bit more AGI-pilled.\u201d In short, they bet on the model being smart enough to handle a task entirely on its own, with minimal user input.<\/p>\n Built shortly afterward, Claude Code was oriented around a back-and-forth conversation with the user. If you gave it a problem, it would survey the possibilities and give you three or four options for proceeding. Once you chose, it would go a little further and then check back again, giving continual updates and leaving less room for the model and harness to make mistakes.<\/p>\n Anthropic\u2019s approach proved more effective, even if it demanded more work from users. \u201c[Our] product was a little ahead of where the model and harness was at the time,\u201d Ambrosino says now.\u00a0<\/p>\n OpenAI eventually followed suit by adding more opportunities for users to interact with the model. That became the Codex we know today, with desktop and mobile apps. Using download statistics as a proxy for interest in the programs, Claude Code was more in demand until April of this year, but now Codex has taken a slight lead; surveys<\/a> of enterprise use also suggest OpenAI is catching up. <\/p>\n Part of that lead is getting the product-market fit right, and part comes from complaints about safety restrictions on Anthropic\u2019s models and compute shortages. OpenAI\u2019s steps toward more human-centric harness continue with ChatGPT Work, but the engineers I spoke to insisted the key differentiator is the strength of OpenAI\u2019s latest powerful and cost-effective models.\u00a0<\/p>\n \u201cThe frustrating answer is that a lot of times it is the model, and one thing that we have tried to do really well with this app is fully leverage the model,\u201d Ambrosino said.\u00a0<\/p>\n That explanation returns to the \u201cbitter lesson<\/a>\u201d learned by AI researchers that a better general model is more important than specific domain experience. For true believers, the harness is a temporary crutch, not the moat.<\/p>\n \u201cYou could get good results in the short term by adding a whole bunch of extras\u2014if and thens and tools\u2014but like, come on, the next model is going to come out in a couple of months and make that obsolete,\u201d Gershenson told TechCrunch. His team focuses on the simplest ways to expose the model to the tools and context it needs\u2014and no more.\u00a0<\/p>\n \u201cThe goal of good harness engineering is to\u2026 be more precise about what information the model really needs to solve your problem, because the models are getting better and better at doing that if you simply let them do their thing,\u201d Gershenson said.<\/p>\n There is an open question, though, if most people or models are ready for that. Ethan Mollick, the Wharton School of Business professor who studies AI tools in the workplace, still sees Claude as more user-friendly, writing<\/a> that \u201cChatGPT tends to want to do magic & just do it for you, while Claude does comparisons & shows them, repeatedly asking for input & feedback and doing A & B tests.\u201d<\/p>\n Sottiaux, and perhaps OpenAI at large, disagree, arguing that the conversational nature of the app is better than learning how to use an application. \u201cWe definitely see that the world seems to be ready,\u201d he says of the app. \u201dThis is why we\u2019ve had incredible adoption.\u201d<\/p>\n Still, it\u2019s not clear that a model-specific harness is even the right bet for maximizing a model. Comparisons run by companies like Composio<\/a> and Databricks<\/a> show that different harness and model combinations deliver different performance on coding benchmarks. Databricks found that Pi, an open-source harness published by the software company Earendi, outperformed Codex while using the same GPT 5.5 model. Pi has been used to build projects like OpenClaw and CloudflareOS.\u00a0<\/p>\n Pi\u2019s creator, Mario Zechner, says his intentionally minimalist harness is evidence that an AGI-pilled approach can work, at least for software engineers and coding tasks. What it lacks in explicit features, he says, is made up for by its ability to modify itself and build its own interfaces. He sympathizes with the challenge that OpenAI\u2019s engineers face in expanding their user base beyond engineers.<\/p>\n \u201cEverything is coding agent shaped\u2026the reason is that they only have training data for coding agent tasks,\u201d he told TechCrunch. \u201cSay I\u2019m in management, I make a decision today, and the outcome happens months later. You cannot capture that in a simple trace of a user and agent back and forth, so all of these kinds of tasks and anything that you don\u2019t digitize is inaccessible to a model to learn.\u201d<\/p>\n Like other open source providers, he sees the big lab\u2019s effort to push their harnesses as a way to lock-in users; \u201dThey need to own the entire stack; otherwise, they just become a model provider and then need to compete with Chinese models.\u201d\u00a0<\/p>\n He and other engineers TechCrunch spoke to felt that the insight into token spend and agent behavior in frontier labs\u2019 harnesses is too limited. In a sense, that\u2019s less meaningful to non-technical workers, but as with the coding tools, uptake at the scale OpenAI hopes for will eventually force harder conversations about cost.\u00a0<\/p>\n For example, messing around on a $20-a-month subscription, I used more than 80 million tokens in four days, which cost $65, according to the model\u2019s analysis (there\u2019s no dashboard in the app). That\u2019s a subsidy of more than 3x the subscription price for four days of casual use alone.<\/p>\n \u201cWe are working every day to push the frontier on efficiency,\u201d Sottiaux said, pointing to a recent 80% price cut for users of OpenAI\u2019s Luna model. \u201cIf you wake up six months from now, you should be able to do all of the same [tasks] with less spend.\u201d<\/p>\n The other relevant question is whether these apps create a lock-in effect on customers through data retention, or the sheer pain of configuring access to all the plug-ins and their permissions.\u00a0<\/p>\n Inside OpenAI\u2019s wood-panelled, plant-filled headquarters, which I visited in July, the atmosphere was calm but slightly tense; these are people with a lot to do. The engineers I spoke with were constantly monitoring their laptops as we talked, and rushing from meeting room to meeting room.\u00a0<\/p>\n Nathan, the head of the product engineering team, said the focus remains on \u201cthe promise of the magic box, but I still think there\u2019s too much complexity\u2026I\u2019m very optimistic that we can solve it, with the model and in a truly AI native way.\u201d<\/p>\n<\/div>\n When you purchase through links in our articles, we may earn a small commission<\/a>. This doesn\u2019t affect our editorial independence.<\/em><\/p>\nHow to make AI intuitive\u00a0<\/strong><\/h2>\n
Giving ChatGPT a license to skill<\/strong><\/h2>\n
The rivalry that drove OpenAI\u2019s product design<\/strong><\/h2>\n
What makes a good harness, anyway?<\/strong><\/h2>\n