When it comes to generative AI, size definitely matters. The large in large language model is almost an understatement at times.
But many businesses are finding that what they need from AI doesn’t necessarily take a trillion parameters of knowledge. Those are the companies IBM is targeting with its latest Granite family of models, which feature reasoning, vision focused specifically on document understanding, and—like previous IBM releases—open-source availability.
At the HumanX conference in Las Vegas this week, we caught up with David Cox, VP for AI models at IBM Research, about the lane the enterprise giant is seeking to carve out with small language models in a crowded AI race.
Super-smart, giant models that can grapple with an advanced math problem, then turn around and write a poem, are certainly valuable, Cox said. But businesses are more often concerned with narrower, more routine sets of tasks than that.
“There’s a little bit of a fork in the road,” Cox told us. “There’s this AGI push where we’re trying to say, ‘We want to create the all-powerful, all-knowing single model,’ versus another path, which says, ‘Hey, now, all of a sudden, all of the unstructured text in my enterprise, all of the images and everything, all that unstructured data is now suddenly unlocked, and I’m going to write enterprise applications that take advantage of that.’”
Paper pile-up: Take, for instance, documents. The average office tends to have stacks of scanned paper files that are inaccessible to an LLM without computer vision, Cox said. For the latest release, IBM created a small model that homed in on recognizing things like “layouts, fonts, charts, and infographics” rather than natural images, the company said in a blog post.
“There was an unmet need for small, high-quality document open models,” Cox said. “I can’t tell you how many bank CIO, CTO-type people have told me how many old scanned documents they have just locked up [with] all kinds of information.”
Keep up with the innovative tech transforming business
Tech Brew keeps business leaders up-to-date on the latest innovations, automation advances, policy shifts, and more, so they can make informed decisions about tech.
Reasoning within reason: In keeping up with the buzz around chain-of-thought reasoning models like DeepSeek R1 and OpenAI’s o3, IBM has also built the capability into some of its offerings. But Cox said the feature can be toggled on and off so businesses can more easily keep costs down.
That ability also gives developers more control over how the AI operates, Cox said.
“This is actually part of a big theme we have where we want to actually change a little bit how people use LLMs, where you can get more reproducible, controllable behaviors the developer can invoke,” Cox said. “It’s not just like I’m having a conversation with this little homunculus in a box that I can talk to and it does stuff.”
Open season: Cox said that while the frenzy around DeepSeek might have been a bit overblown in certain ways, it did bring more attention to the open-source AI world, though R1’s delays and skill sets aren’t necessarily enterprise-friendly.
“As somebody who’s thinking about, ‘How do you build actual, real solutions,’ spending 50 seconds on every question, being really great at math, that’s not exactly what our customers want,” Cox said. “But the fact that there’s going to be this diversity of ideas, the fact that so much of it’s going to happen in the open, I think is fantastic.”
As businesses become more concerned with ROI, they may consider smaller, more specialized models. “[Businesses] don’t need an HR chatbot that knows how to do advanced physics. That’s just a waste. I’m just wasting energy. I’m wasting money. It’s just not helpful,” Cox said.