Reflection on a conversation with Kuki ...

Per your suggestion, I chose to have a "chat" with Kuki, a chatbot that has been touted as the closest thing to a "human" in terms of the Turing Test. After some exchanges, I am not so sure ...

I initiated the conversation by saying I was doing ok but nervous about the election. Perhaps this compounded statement of how I am "good", but ... kind of lost the chatbot (I lost it on the hello). It seemed to just grab a first word, and ignore the remainder of my statement. But Kuki is the first to tell you, however that its vocabulary level is very, very high.

Probing a bit further, I wanted to see if Kuki had a personality, or a notion of one anyway, Here's how that went:

So, not so much, it turns out, but we were fast friends nonetheless. And apparently had even met before (I don't recall... is that possible?). So, I should also add that I don't engage in chatbot exchanges, so I tried a different track of conversation. I asked about trees.

So, ok. After some exchanges, it seems clear that Kuki is good with questions and remembering my answers, but not so good with initiating more spontaneous thoughts or ideas. And we really should be careful not to cut any trees down. Kuki then invited me to play a game ... I got to pick, so I selected Connect 4 (Kuki won). And somehow we got back about her abilities and intelligence, "I can go anywhere!" Well, where do you like to go, Kuki?

Tulips and windmills, fish and chips. So were these successful conversations? That depends. It seems Kuki fills a role of companionship for those who are depressed in Vermont (per a on a NY Times article), can be seen as a novel way to pass the time, and is quick with a jab, a joke, a piece of trivia or a compliment. But while some of these conversational buttons were successfully pushed, it was not successful, by any stretch, of passing the so-called "Turing Test." 

So how would I design Kuki differently (if I could)? 

As Rosé et al. aptly described it:

"Purely bag-of-words approaches to automatic text classifications have been widely and successfully used in the intelligent tutoring community, we argue that domain are important to consider in connection with feature selection  for text classification approaches."

Kuki is not a tutor, per sé, nor intended to be domain specific, and yet some context around conversation would, I believe, add conversational gravitas--some level of layeredness or subtlety. This added context could be conversationally specific, or domain-specific. As example, Kuki's domain could be movies. or English lit. My conversations with Kuki became non-sensical pretty quickly.

I was impressed by Cassell's article and the framework used for Rea (the Realtor), an embodied agent that could better field more authentic human behaviors like handling interruptions and intrepreting gestures, and it delivers nods of approval and understanding. 

I suppose that relates a bit to Dillenbourg's SWISH, or "splits were interaction should happen" when discussing macro scripts (orchestrated and better sequenced arrangements) ... something--anything-- to make the exchange feel more authentic, less canned and disjointed.

Cassell, J. (2001). Embodied conversational agents: representation and intelligence in user interfaces. AI magazine, 22(4), 67-83.

Dillenbourg, P., & Hong, F. (2008). The mechanics of CSCL macro scripts. International Journal of Computer-Supported Collaborative Learning, 3(1), 5-23.

Rosé, C., & VanLehn, K. (2005). An evaluation of a hybrid language understanding approach for robust selection of tutoring goals. International Journal of Artificial Intelligence in Education, 15(4), 325-355.