Jan 31 • 3 min read

Fox School of Business professor analyzes chatbot effectiveness,  depending on whether the chatbot discloses that it is a robot.

People reading and shopping on phones

PHILADELPHIA, Jan. 31, 2020—Need to order a pizza? There’s a chatbot for that. Need to pay off your credit card? There’s a chatbot for that, too. 

In fact, there could soon be a chatbot for just about anything. Studies have suggested the chatbot market is worth $250 million now and it’s expected to balloon to more than $1.34 billion by 2024. In addition, 21% of U.S adults and more than 80% of Generation Z use voice or text bots for information search and shopping.

The growth comes as no surprise to Xueming Luo, professor of marketing, strategy and management information systems in the Fox School of Business at Temple University. His research has outlined how these chatbots really can work. However, there is a caveat, says Luo. 

“A chatbot can be effective in helping to drive sales, but let me emphasize the term ‘bot.’ If a potential customer realizes it’s just a robot on the line or if that information is disclosed, that can actually drive sales conversions down by nearly 80%,” Luo says.

Together with Siliang Tong, a marketing PhD student in the Fox School, and colleagues from Sichuan University and Fudan University, Luo authored “Machines versus Humans: The Impact of AI Chatbot Disclosure on Customer Purchases,” which was recently published in Marketing Science. 

As part of the study, more than 6,200 potential customers were randomized to receive highly structured outbound sales calls from either a chatbot or human worker. The results were both positive and negative, at least as far as the chatbot was concerned.

“Results suggest that undisclosed chatbots are as effective as proficient workers and four times more effective than inexperienced workers in getting customers to make a purchase,” Luo says. “However, if the chatbot reveals that it’s a machine before the machine-customer conversation, purchase rates plummet by 79.7%.”

Not just that, but potential customers have shown that they have no problem being brash and rude once they know that they’re speaking to a robot. Professional etiquette basically goes out the window.

“Go all the way back to popular movies like the Terminator, and I think history shows that humans have always been wary when it comes to robots. It’s no different when it comes to chatbots,” Luo says. 

As previously noted, chatbot use is projected to rise in the years to come. But is that the most sound strategy, given the tepid response seen by potential consumers when a chatbot’s identity is disclosed?

“Although the popular prediction is that the displacement of some workers by chatbots is an inevitable trend, research and market data paradoxically show that the disclosed chatbots are not going anywhere currently,” Luo says. “So it will be interesting to see how both potential customers and the developers behind these chatbots work to adapt in the years to come.”

About the Fox School of Business

The vision of Temple University’s Fox School of Business is to transform student lives, develop leaders and impact our local and global communities through excellence and innovation in education and research. 

The Fox School’s research institutes and centers and 200+ full-time faculty provide access to market-leading technologies and foster a collaborative and creative learning environment that offers more than curriculum—it offers an experience. Coupled with its leading student services, the Fox School ensures that its graduates are fully prepared to enter the job market.

The school’s knowledge-creating research faculty affords it the flexibility and responsiveness to address the needs of industry and generate courses and programs in emerging fields of study. As a leader in business research, the Fox School values interdisciplinary approaches and translational research that advance actionable insights to solve real-world problems. Our research informs an adaptive curriculum, supports innovation in teaching and prepares students for the changing nature of work.