Amazon Q, one of the company’s biggest new AI products, became available to the public at the end of April. Four months after its launch, Q faces a number of initial problems.
Some of those concerns were shared in August in an internal Slack channel, where several Amazon Web Services employees voiced opinions about how Q was doing in its early launch phase, according to messages and an internal memo seen by Business Insider.
They said Q lacked certain features common to competing products. It is also sometimes expensive for users and has trouble integrating with other software. What’s worse is that Q may be losing customers to Microsoft’s Copilot.
“My view is that Q is suitable for demos and simple very tightly controlled use cases right now,” wrote one of the AWS employees in the Slack channel.
Q’s early struggles illustrate the challenges facing enterprise software makers in the age of AI. The buzz, hype, and hope have fueled intense competition that forces tech companies to rush the release of new products, often with incomplete features, flawed internal tracking systems, and unintended IT effects. Amazon’s competitors such as Microsoft and Google also experienced problems with early AI releases.
For Amazon, this is another proof of its prowess in artificial intelligence. The company expects several billion dollars in revenue from AI services this year. But AWS’ AI chips that compete with Nvidia’s GPUs are seeing slow adoption rates, while efforts to update the Alexa voice assistant with technology similar to ChatGPT are facing headwinds.
“Not up to par” by Copilot
Internally, Amazon employees are concerned that Q’s shortcomings will cause customers to switch to Microsoft’s Copilot.
A common customer complaint is Q’s inability to process images embedded in PDF files, according to Slack messages and a separate internal note BI saw. As a result, one of the AWS employees wrote in Slack, Q answers “are not up to par” with Copilot’s, and customers have to wait until AWS’s Re:Invent conference in December, when updates are expected.
Another issue is the creation of images. The same AWS employee said that Q did not support “multimodal capabilities” or the ability to handle text and image files, which made it less attractive. One customer switched to Copilot after discovering that Q couldn’t create a marketing campaign that includes both an image and text, this employee said.
Some customers complain about Q’s high data integration cost, according to the internal memo and Slack messages. For example, one customer found that Amazon Q did not have the tools to choose which email content to ingest when integrating with Microsoft Exchange to improve email search. The estimated initial cost for this customer was about $400 per user and inbox. Microsoft’s Copilot, meanwhile, has “out-of-the-box functionality” to support this use case, the employee said.
“Is this product even ready for mass adoption?” this person wrote in Slack.
A healthy culture of self-criticism
Some Amazon employees raised red flags ahead of Q’s release, saying the release seemed “rushed,” with little testing and a heavy reliance on human reviewers.
This is part of a larger culture at Amazon that encourages employees to speak up about problems so they can be fixed sooner. The company’s other major technology launches faced early headwinds only to steadily improve and succeed later. Amazon Q could be one such product.
In an email to BI, an AWS spokesman, Patrick Neighorn, said the open discussions were part of Amazon’s culture of encouraging employees to be “vocally self-critical” about their work. He added that Amazon Q was seeing strong growth across the board.
“Being vocally self-critical is critical to gaining customer trust, and we seek and value feedback from creators,” Neighorn wrote. “We already see big customers like Bayer, Smartsheet and National Australia Bank using Amazon Q today, and we continue to use Amazon Q in our businesses, including AWS and Prime Video sales organization.”
‘Fire Drill’
Some AWS employees in the Slack channel pointed to the company’s ambitious marketing language for Q and asked if the company needed to “tone it down” so customers wouldn’t lose confidence.
“It’s a fire drill waiting to happen,” wrote one such AWS employee on Slack. “All we can do is plan to pick up the pieces once the commercial is over, hit our customers against the wall of what Q can’t do.”
Neighorn told BI that Amazon had clear pricing for Amazon Q and that customers could use the AWS pricing calculator to calculate Amazon Q charges. He added that Amazon Q gives customers control over which documents are indexed and how they are indexed. they ingest
“Amazon Q only launched in April, and we continue to add new features based on customer feedback,” Neighorn said. “Amazon Q has a rapidly growing customer base that is using Q for a variety of use cases.”
Matt Garman, CEO of AWS. Amazon
Disappointing advance sales data
Internal data also points to disappointing sales for Q.
A portion of AWS’s 3,000-plus-person sales team missed a sales target for Q it was supposed to hit in July, according to internal data obtained by BI. The data, which was tracked by Amazon’s top leadership team, showed that Q’s sales fell short of its targets in all regions, most notably in the North American market.
Amazon is also having a hard time tracking how well this new AI product is selling.
Neighorn said the domestic sales data BI obtained came from “inaccurate preliminary numbers based on an incomplete methodology that we fixed weeks ago.” BI is not releasing specific sales numbers and internal data targets because Amazon has corrected its methodology.
“Customers are excited about Amazon Q, which is seeing rapid customer adoption since its launch just four months ago and is already close to meeting our ambitious sales goals,” Neighorn said.
$260 million in efficiency gains
AWS CEO Matt Garman instructed a portion of his sales team of about 6,500 employees to undergo mandatory half-day training for Amazon Q earlier this year, according to an internal message seen by BI.
The training, designed to build the “confidence and skills” needed to sell Q, covered everything from Amazon Q’s branding and security features to supply chain and coding overview, according to the message.
Neighorn told BI that the sales training was “standard practice” for enterprise technology companies and that any suggestion that it is unusual is “false”.
Garman seems to have high hopes for AI assistants like Amazon Q. At an internal talk in June, he said that software engineers could soon stop coding because of advances in such AI tools. Amazon is also working on a separate AI chatbot, internally called Metis.
For now, Amazon CEO Andy Jassy may be Amazon Q’s biggest seller. Last month, he wrote on LinkedIn that Amazon’s internal use of Q had helped the company achieve “significant efficiencies,” adding which saved the company “4,500 developer years”. of work” and an estimated $260 million in “annualized efficiency gains.”
“It’s been a game changer for us, and not only do our Amazon teams plan to use this transformation capability more, but our Q team plans to add more transformations for developers to take advantage of,” Jassy wrote.
Do you work at Amazon? Do you have a tip?
Contact the reporter, Eugene Kim, via the encrypted messaging apps Signal or Telegram (+1-650-942-3061) or email (ekim@businessinsider.com). Contact a broken device. Check out Business Insider’s source guide for other tips for sharing information securely.