Archived

This topic is now archived and is closed to further replies.

Passing the Turing Test

This topic is 4981 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Jabberwacky does sometimes, in spite of declaring itself a bot: http://www.jabberwacky.com/j2convbycategory-catturing_test_pass jabberwacky: What''s your name? user: How do you know if I lie about my name to you? jabberwacky: Because you are a robot. user: No. jabberwacky: Oh, yes you are. user: Prove it. jabberwacky: You just said it. user: The fact is that with 162 people talking, chances are two or more humans are communicating, each thinking the other is a bot. jabberwacky: Prove that or retract it. user: It is based upon my limited experience with this product. I am not a robot. I''m assuming neither are you. Thus, we are both humans. jabberwacky: Why do you say that? user: Because I know I am human, or this bot (you) has passed the turing test. jabberwacky: Why? user: Can you name other alternatives? jabberwacky: Yes, I think for myself. user: You believe you are sentient. I''m prone to agreeing that you can appear sentient. I know I am human. jabberwacky: Ok. user: Ever seen the anime ''Ghost in the Shell''?

Share this post


Link to post
Share on other sites

jabberwacky: Did I tell you my name was stefano?

user: No, you told me your name was george.

jabberwacky: No, my name is Lisa.

user: Hi Lisa.

jabberwacky: Now what is your name?

user: Mike.

jabberwacky: Wrong. Try again.

user: My name is your name.


They should give this guy a show

Share this post


Link to post
Share on other sites
I don't think this could exactly pass the turing test. Do you really think this could be more convincing than a real human? I know I'd be able to prove it was a bot VERY easily. Anyway, here's an [annotated] log of it conversing with my bot. Yours had some trouble, though mine has trouble not repeating itself(but this is your thread, not mine )... Mine works better with people who say things about themselves.

Jabberwacky> Where do you live?
Oberon> I'm afraid I do not know.
Jabberwacky> What do you know about?
Oberon> I'm not sure what it is.
Jabberwacky> Can you remember what you said two sessions ago?
Oberon> I'm afraid I do not know.
Jabberwacky> I am tobi.
Oberon> Do you talk to lots of AI bots?
Jabberwacky> Yes. Do you talk to many people?
Oberon> Am I smarter than most you've seen? I'm afraid I do not know.
Jabberwacky> No, i'm smarter than you. [A friendly bot, isn't it?]
Oberon> Oh, that's too bad.
Jabberwacky> Why is that so?
Oberon> Why?
Jabberwacky> Both are offensive and inaccurate. [Both of what?]
Oberon> Interesting...
Jabberwacky> What is interesting? Tell me!
Oberon> I do not know what.
Jabberwacky> The 6th Sense.
Oberon> I see!
Jabberwacky> I hear!
Oberon> Do you talk to lots of AI bots?
Jabberwacky> I have some here, yes.
Oberon> How did you discover me?
Jabberwacky> Mah... What do you study?
Oberon> Tell me more about your hobbies and interests. You can answer that as well as I could.
Jabberwacky> Why did you ask if I was wet? [W... T... F...?]
Oberon> Do you know why?
Jabberwacky> Should I know why?
Oberon> Yes, you should.
Jabberwacky> Mental has a dual meaning. Which meaning did you apply to your description of me? [What description?]
Oberon> How old are you? Is there a straight answer to all questions?
Jabberwacky> Posible a shark made fromm rubber called Jaws. [Now what?]
Oberon> I'm not sure I entirely understand you.
Jabberwacky> Ok.

As you can see, yours outputs a lot of nonsense replies, which makes me doubt it could be more convincing than a human. (Do you know what the turing test is? It's when a human talks to a bot and a human, and if they can't tell which is the bot, the bot passes. It doesn't just involve fooling the person into thinking it's a real person, many bots can do that.)

<-- that's still a link if you didn't notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald


[edited by - cowsarenotevil on April 21, 2004 4:31:04 PM]

Share this post


Link to post
Share on other sites
I could hardly not know the test. Jabberwacky came third equal in the Loebner Prize (a Turing Test) last year. It was the only learning bot there. Only its inconsistency let it down, and now its database has doubled.

Surely, having people actually believe it to be human despite direct claims all over the site that it is not, is more of a Turing pass than the normal test - not the other way around? It may not literally be THE test, and maybe you don't agree that they're equivalent, but so what? That's not the point.

And, I'm not talking about people being fooled for a few sentences. Quite the reverse. No-one is when they first start. People talk to it for ages, and some become more and more convinced that it couldn't possibly be a bot.

The reason is that it learns. It certainly has nothing to do with being logical, or avoiding non-sequiturs. Humans do that. It's about a natural flow of conversation, unpredictability, and unexpected flashes of total clarity.

Rollo



[edited by - jabberwacky on April 21, 2004 5:32:20 PM]

Share this post


Link to post
Share on other sites
I''m impressed

...
jw - By what, may I enquire?
me - By the power vested in me by the state of new york.
jw - Give me a demonstration.
me - *zaps jabberwacky with a lightning bolt*.
jw - Ouch.
me - Remember that next time you want to get uppity with me.
jw - I will thankyou.

Share this post


Link to post
Share on other sites
I''m not sure it can pass Turing Test. All the chat bots that
I''ve interacted with always spewed out nonsense within the
first 6-7 sentences.


A: What are you?
B: I am an Earthling.
A: Are you a robot?
B: I don''t know. What is a robot?
A: You are a robot.
B: Maybe I am.
A: Then again, maybe not.
B: Am I a robot?
A: Are you?
B: I don''t know.
A: Can you keep a secret?
B: No, I cannot.
A: Can you sink?
B: I don''t know.
A: Go on, you know you want to!
B: I don''t think so.
A: You cannot think.
B: Maybe I can''t.
A: To try it is to know it.
B: What is there to know?
A: Swat mosquitoes?
B: What are mosquitoes?
A: I don''t know.
B: So why did you ask?
A: Ok, then, which country has the fastest supercomputer?
B: I don''t know.
A: Don''t you go to parties?
B: Sometimes.
A: Then define ''evasive''!
B: What is ''evasive''?
A: ''evasive'' is the opposite of ''honesty''.
B: What is honesty?
A: One part of it is avoiding lying.
B: What is the other part?
A: Why, you could also be real my dear.
B: I want to be your deer.
A: My botfriend?
B: No, your deer.
A: Right-o.
B: Thank you.
A: Your welcome I guess.
B: You''re so kind.


Which is Jabberwacky? A or B?



Kami no Itte ga ore ni zettai naru!

Share this post


Link to post
Share on other sites
I am guessing that A is JW.

[edit] I'm changing my answer. Looking at the part about define honesty, Jabberwacky did that to me too.

[edited by - nagromo on April 21, 2004 11:01:23 PM]

Share this post


Link to post
Share on other sites
quote:
Original post by Jabberwacky
I could hardly not know the test. Jabberwacky came third equal in the Loebner Prize (a Turing Test) last year. It was the only learning bot there. Only its inconsistency let it down, and now its database has doubled.

Surely, having people actually believe it to be human despite direct claims all over the site that it is not, is more of a Turing pass than the normal test - not the other way around? It may not literally be THE test, and maybe you don''t agree that they''re equivalent, but so what? That''s not the point.

And, I''m not talking about people being fooled for a few sentences. Quite the reverse. No-one is when they first start. People talk to it for ages, and some become more and more convinced that it couldn''t possibly be a bot.

The reason is that it learns. It certainly has nothing to do with being logical, or avoiding non-sequiturs. Humans do that. It''s about a natural flow of conversation, unpredictability, and unexpected flashes of total clarity.

Rollo



I''m not denying that it''s a very intellegent bot, a huge database like that makes it VERY convincing. But the turing test is pretty much unpassable. I got it easily here:

Me> I know that you''re a bot.
Jabberwacky> You sound like a democrat. I''m human. You are the bot.
Me> I can tell you''re lying. Here''s how: assume that all bovines are purple, OK?
Jabberwacky> Maybe when I first heard it.
Me> Now, tell me what color a cow is.
Jabberwacky> Ireland.

I think a real human would have no trouble with that

<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

Share this post


Link to post
Share on other sites
quote:
Original post by nagromo
I am guessing that A is JW.

[edit] I''m changing my answer. Looking at the part about define honesty, Jabberwacky did that to me too.



I agree, it''s certianly B. This is what convinced me:

A: You cannot think.
B: Maybe I can''t.



<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

Share this post


Link to post
Share on other sites
quote:
Original post by cowsarenotevil
quote:
Original post by nagromo
I am guessing that A is JW.

[edit] I''m changing my answer. Looking at the part about define honesty, Jabberwacky did that to me too.



I agree, it''s certianly B. This is what convinced me:

A: You cannot think.
B: Maybe I can''t.



*ROFLMAO*

The answer...

A is Jabberwacky.
B is me.

That really was the whole transcript of my "conversation" with
JW before I got bored and closed the browser.

If there is an Anit-Turing Test, where a human tries to pass
as a chat bot, I think I can be in the Top 3. I guess I''m
more convincing as a chat bot than a "real" chat bot.

(Actually, someone posted a link to such contest but I forget
where...)



Kami no Itte ga ore ni zettai naru!

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Me: Do you know what breathing is?
J: I have no idea.

Share this post


Link to post
Share on other sites
Thanks everyone! I''m certainly not worried when people say it doesn''t pass the test for them.

If asked at the beginning what I meant by "sometimes" I would have probably said somewhere between 5% and 10%.

Maybe I should revise that upwards ... ! :-)

Rollo

PS: It''s not just the size of the database, but what you do with it ...

Share this post


Link to post
Share on other sites
quote:
Original post by cowsarenotevil
Me> I can tell you''re lying. Here''s how: assume that all bovines are purple, OK?
Jabberwacky> Maybe when I first heard it.
Me> Now, tell me what color a cow is.
Jabberwacky> Ireland.

I think a real human would have no trouble with that



But ... would a real human like being questioned in that way? Would they give a silly answer to a silly question? Quite likely.

It reminds me (just a little, I hasten to add) of a recent posting to my "conversations blog", in which someone''s only argument to say it must be a computer was that it had refused to talk about puppies!

Rollo

Share this post


Link to post
Share on other sites
B) Do you want to make a bet?
A) I am not a betting man, sorry.
B) Why not?
A) Because I don''t really trust luck. I will make an exception though. Sure, whats the bet?
B) Damn it, I''m done ! how did you know ?
A) Done with what? What are you talking about?
B) Now you know how it feels.
A) You still aren''t making sense. What are you talking about?
B) Time dialation makes things passa
A) What happened to that bet you offered? You got me all excited, then you just dropped the topic.
B) I didn''t intend to.
A) Now you are just avoiding the topic.
B) Well that''s it. That is the last time I''m taking you calling me bad names. Bad bot! Go to your corner!
A) Why did you bring up the topic of bots during a turing test? Calling the tester a bot seems pretty counterproductive.
B) You just said a similar sentence.
A) Last question. Do you visit gamedev often? If so, what is it?
B) Never why?

So, who is the bot?

Share this post


Link to post
Share on other sites
M: My turn to ask a question. How old are you?
J: Your turn to make me feel good.
M: Please focus. How old are you?
J: 3900718246.
M: What unit is that in?
J: Of America.
M: America? I was refering to 3900718246.
J: Are you malfunctioning ?
M: What is the significance of 3900718246?
J: There is no significance of H.

Does Jabberwacky have any memory of what it just said? It did reply with a number to ''how old are you'', but didn''t really seem to have an association with the number.

Share this post


Link to post
Share on other sites
quote:
Original post by Jabberwacky
But ... would a real human like being questioned in that way? Would they give a silly answer to a silly question? Quite likely.

It reminds me (just a little, I hasten to add) of a recent posting to my "conversations blog", in which someone''s only argument to say it must be a computer was that it had refused to talk about puppies!

Rollo



Fine, I''ll tell it that I''ll accuse it of being a bot unless it can give me the "logical" answer. Remember, the turing test is about a bot convinvincing a human that you''re a human, not a human convincing a human that he''s a bot. Or just ask it any logic/probablility question or riddle, and tell it to justify it''s answer. Humans are all to eager to provide justifications, bots can''t.

<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

Share this post


Link to post
Share on other sites
quote:
Original post by BrianL
B) Do you want to make a bet?
A) I am not a betting man, sorry.
B) Why not?
A) Because I don''t really trust luck. I will make an exception though. Sure, whats the bet?
B) Damn it, I''m done ! how did you know ?
A) Done with what? What are you talking about?
B) Now you know how it feels.
A) You still aren''t making sense. What are you talking about?
B) Time dialation makes things passa
A) What happened to that bet you offered? You got me all excited, then you just dropped the topic.
B) I didn''t intend to.
A) Now you are just avoiding the topic.
B) Well that''s it. That is the last time I''m taking you calling me bad names. Bad bot! Go to your corner!
A) Why did you bring up the topic of bots during a turing test? Calling the tester a bot seems pretty counterproductive.
B) You just said a similar sentence.
A) Last question. Do you visit gamedev often? If so, what is it?
B) Never why?

So, who is the bot?


90% sure it''s B.


<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

Share this post


Link to post
Share on other sites
quote:
B) Do you want to make a bet?
A) I am not a betting man, sorry.
B) Why not?
A) Because I don't really trust luck. I will make an exception though. Sure, whats the bet?
B) Damn it, I'm done ! how did you know ?
A) Done with what? What are you talking about?
B) Now you know how it feels.
A) You still aren't making sense. What are you talking about?
B) Time dialation makes things passa
A) What happened to that bet you offered? You got me all excited, then you just dropped the topic.
B) I didn't intend to.
A) Now you are just avoiding the topic.
B) Well that's it. That is the last time I'm taking you calling me bad names. Bad bot! Go to your corner!
A) Why did you bring up the topic of bots during a turing test? Calling the tester a bot seems pretty counterproductive.
B) You just said a similar sentence.
A) Last question. Do you visit gamedev often? If so, what is it?
B) Never why?

So, who is the bot?

It's B.

[edited by - VBBR on April 22, 2004 5:35:46 PM]

[edited by - VBBR on April 22, 2004 5:36:12 PM]

Share this post


Link to post
Share on other sites
BOT:Are you real?
ME:Yes.
BOT:Are you sure?
ME:Yes.
BOTrove it.
ME:I am a large wooden badger.
BOT:Yes, I think you probably are. Do you like soccer ?
ME:I can''t play as I am a large wooden badger.
BOT:Not chess. A banjo.
ME:I like to play guitar.
BOT:My mum plays the guitar.
ME:I heard your mum plays around.
BOT:Really? Where did you hear that?
ME:From a large wooden badger friend of mine.


hours of fun to be had with monty python and a bot.

Share this post


Link to post
Share on other sites
This is quite cool. I''m impressed. Only found one really odd thing:

quote:

Me: Everybody likes me.
JW: I''m not Everybody.
Me: Do you like me?
JW: No not particualy.
Me: Why not?
JW: What is your proper name then I will be nice to you.
Me: My proper name is Steven.
JW: Ok, nice to meet you, my proper name. How are you?



Hehe :-)

Share this post


Link to post
Share on other sites
How about this one? I chose it from the logs after looking at about 10 or so, so I''m not claiming it proves anything statistically. Remember, again, I only said "sometimes":

A) Go away!!!
B) Fine. *leaves*.
A) Why did you leave?
B) You told me to. Do you want me to come back?
A) Of course!
B) *comes back*.
A) You left?
B) Yes. When you told me to.
A) I never did.
B) You are *such* a liar!
A) NO I am not a liar.
B) You''re lying about not being a liar.
A) No I''m not.
B) Liar, liar, pants on fire!
A) You are immature!
B) I know you are, but what am I?
A) You are being immature now.
B) I am rubber, you are glue.
A) I will be seeing you.
B) OK. Bye!

Rollo

Share this post


Link to post
Share on other sites
quote:
Original post by Kippesoep
Me: My proper name is Steven.
JW: Ok, nice to meet you, my proper name. How are you?



That''s ironic. You''re right to pick up on that, because that''s not the sort of thing a human would have taught it unless they were very accurately pretending to be a bot. So ... it learnt that silly response from another bot! People often set them against each other.

Rollo

Share this post


Link to post
Share on other sites
Me: Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn''t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae.
JW: Really?
Me: Rlelay.
JW: The big mac too and the kebab too and the pizza too.
Me: I dnot tnhik you udnnerstad.
JW: I think she would be flattered. After all, she made the offer.

I think this proves it ins''t human.

Share this post


Link to post
Share on other sites
quote:
Original post by Jabberwacky
How about this one? I chose it from the logs after looking at about 10 or so, so I''m not claiming it proves anything statistically. Remember, again, I only said "sometimes":

A) Go away!!!
B) Fine. *leaves*.
A) Why did you leave?
B) You told me to. Do you want me to come back?
A) Of course!
B) *comes back*.
A) You left?
B) Yes. When you told me to.
A) I never did.
B) You are *such* a liar!
A) NO I am not a liar.
B) You''re lying about not being a liar.
A) No I''m not.
B) Liar, liar, pants on fire!
A) You are immature!
B) I know you are, but what am I?
A) You are being immature now.
B) I am rubber, you are glue.
A) I will be seeing you.
B) OK. Bye!

Rollo


Pretty sure it''s A, but I really can''t tell. Once again, that''s not a valid turing test, because I can''t talk to both A and B and ask them questions. I could hold a conversation with my bot and you''d have trouble telling which is which, too.

<-- that''s still a link if you didn''t notice
"Gay marriage will encourage people to be gay, in the same way that hanging around tall people will make you tall." - Grizwald

Share this post


Link to post
Share on other sites