Archived

This topic is now archived and is closed to further replies.

rcode

Natural Language Processing (NLP) SDK or library?

Recommended Posts

Hi, Does anyone know where I can find/buy an SDK, API, library or something similar that does natural language processing? I want to be able to have my game be more smarter at responding to what the user types in when in chat mode during a heated battle! Thanks in advance. RCode http://progui.tripod.com/codeedu

Share this post


Link to post
Share on other sites
I have a minor in linguistics and if there is one thing a learned those years that is:

NLP: Avoid it!!

The state of the art is not nearly satisfactory enough to be of any real use.

If you just need to parse simple sentences then try and go back and find the source to some old Text Adventure parser. I think there are some open source available.

Jacob Marner

Share this post


Link to post
Share on other sites
There is one with a Java interface at [url="http://www.thinktelligence.com"]www.thinktelligence.com[/url] that I was working on. I would appreciate feedback on which areas to develop more first. For example, I could write a C++ interface if anyone was interested in using it. It comes with a sample java 'game' that demonstrates how the interface would be integrated with a program and the current capabilities. If you load the sample.nli file and look under "Test Cases" you can see the range of input that can be processed.

Share this post


Link to post
Share on other sites
[quote name='felonius' timestamp='972906449' post='1150864']
The state of the art is not nearly satisfactory enough to be of any real use.
[/quote]

QFE. Right now the best things can do are somewhere in the 85% area. And that's with a giant dictionary and tons of pre-compiled statistics and/or time to run the genetic algorithm.

Your best bet (depending on what you really are looking for) is something less than full parsing of language. Keyword matching, or something like the old mud style processing is often 'good enough'.

And even then I'd probably aim to avoid it simply for the internationalization [i]nightmare[/i] it would introduce.

Share this post


Link to post
Share on other sites
[quote name='Telastyn' timestamp='1296854671' post='4769712']
[quote name='felonius' timestamp='972906449' post='1150864']
The state of the art is not nearly satisfactory enough to be of any real use.
[/quote]

QFE. Right now the best things can do are somewhere in the 85% area. And that's with a giant dictionary and tons of pre-compiled statistics and/or time to run the genetic algorithm.

Your best bet (depending on what you really are looking for) is something less than full parsing of language. Keyword matching, or something like the old mud style processing is often 'good enough'.

And even then I'd probably aim to avoid it simply for the internationalization [i]nightmare[/i] it would introduce.
[/quote]

Where did you get 85% from? IBMs Watson excluded, I haven't heard of anything that was remotely close to solving NLP. If you have some links please share. :) :)

I second your suggestion; keyword matching is probably good enough. NLP is a huge problem.

Share this post


Link to post
Share on other sites
[quote name='willh' timestamp='1297006973' post='4770424']
[quote name='Telastyn' timestamp='1296854671' post='4769712']
[quote name='felonius' timestamp='972906449' post='1150864']
The state of the art is not nearly satisfactory enough to be of any real use.
[/quote]

QFE. Right now the best things can do are somewhere in the 85% area. And that's with a giant dictionary and tons of pre-compiled statistics and/or time to run the genetic algorithm.

Your best bet (depending on what you really are looking for) is something less than full parsing of language. Keyword matching, or something like the old mud style processing is often 'good enough'.

And even then I'd probably aim to avoid it simply for the internationalization [i]nightmare[/i] it would introduce.
[/quote]

Where did you get 85% from? IBMs Watson excluded, I haven't heard of anything that was remotely close to solving NLP. If you have some links please share. :) :)

I second your suggestion; keyword matching is probably good enough. NLP is a huge problem.
[/quote]

I had done some quickie research recently and the number stuck in my head. The research was almost entirely around anaphora resolution, so might be constrained towards those algorithms rather than in general understanding. Taking a quick look, I found [url="http://docs.google.com/viewer?a=v&q=cache:V_OK-VpIj8YJ:www.aclweb.org/anthology/J/J05/J05-3004.pdf+anaphora+resolution+percent+correct&hl=en&gl=us&pid=bl&srcid=ADGEESgtOh9YbbYAyvLBzk0z6Vk8cxSHic6-BMbbhajhdRjsFZBF0gNz6_W7we3MrvfCz4y6RkHSregg0abMM5Nu1oPlFTthdtLwDxWlYNbozT7o24lwi6bGvrMOMVTHZ5kr2Q04rHiL&sig=AHIEtbRgkBbIgLmxnqWQoXAFbV6SsBAIAA&pli=1"]this paper[/url] which has ~85% for anaphora resolution algorithms.

Share this post


Link to post
Share on other sites