Getting dictionary addon to work with ref-counted strings

Started by
6 comments, last by dkrusu 8 years, 10 months ago

Hi, I'm trying to get the dictionary add-on to work with ref-counted strings. After I replaced all the uses of std::string with my own most things worked except for the initialization lists and index operators. I was able to initialization lists working with the following change to how the name is extracted:


UString *name = *(UString**)buffer;
buffer += sizeof(UString*);

That fixed that issue, but now I'm getting a crash when attempting to retrieve the data using the index operators


dictionary dict = { {"test", "foo"} };
dictionaryValue val = dict["test"];
stdout << string(val);

That will cause a crash when attempting to call opEquals on the string object. The following will work:


dictionary dict = { {"test", "foo"} };
string val;
dict.get("test", val);
stdout << val;

I'm sure the pointer it's using for the string class is not correct, but any ideas on why that would be? I attached a valgrind log.

Thanks

[attachment=27620:valgrind.txt]

Advertisement

You don't really show much that we can evaluate in order to help you figure out the problem.

But from the valgrind log I'd guess the problem is with how you've registered the Ustring::operator= with AngelScript.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

I've registered it with the following:


r = engine->RegisterObjectMethod("string", "string &opAssign(const string &in)", asMETHODPR(UString, operator=, (const UString&), UString&), asCALL_THISCALL); assert(r>=0);

Here is the operator= method


UString& UString::operator=(const UString& other)
{
    mData = other.mData;
    return *this;
}

Attached is the full UString source.

Thanks

[attachment=27636:UString.cpp.txt]

I decided to test this against the scriptstring add-on from testing, and it fails the same way. Here is my test code:


#include <angelscript.h>
#include <scriptstring.h>
#include <scriptdictionary.h>
#include <scriptarray.h>
#include <scripthelper.h>
#include <stdio.h>

void MessageCallback(const asSMessageInfo *msg, void *param)
{
	const char *type = "ERR ";
	if( msg->type == asMSGTYPE_WARNING ) 
		type = "WARN";
	else if( msg->type == asMSGTYPE_INFORMATION ) 
		type = "INFO";

	printf("%s (%d, %d) : %s : %s\n", msg->section, msg->row, msg->col, type, msg->message);
}

int main()
{
    asIScriptEngine *engine = asCreateScriptEngine(ANGELSCRIPT_VERSION);
	engine->SetMessageCallback(asFUNCTION(MessageCallback), 0, asCALL_CDECL);
    
    RegisterScriptString(engine);
    RegisterScriptArray(engine, true);
    RegisterScriptDictionary(engine);

    int r = ExecuteString(engine, 
            "dictionary dict = { { \"foo\", \"bar\" } };\n"
            "string s = string(dict[\"foo\"]);\n");

    return 0;

}

I made the following change to scriptdictionary to use pointers for the key


// Get the name value pair from the buffer and insert it in the dictionary
string name = **(string**)buffer;
buffer += sizeof(string*);

Here is the GDB backtrace


Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b2c43b in std::string::assign(std::string const&) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff7b2c43b in std::string::assign(std::string const&) ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x0000000000409569 in CScriptString::operator= (this=0x7ba3d0, other=...)
    at /home/droz/trunk/sdk/tests/test_feature/source/scriptstring.cpp:76
#2  0x000000000048c0bc in endstack ()
#3  0x0000000000402950 in std::string::compare(std::string const&) const@plt ()
#4  0x00007fffffffe120 in ?? ()
#5  0x0000000000000000 in ?? ()

Again, here is valgrind log


==14332== Invalid read of size 4
==14332==    at 0x4F0549E: std::string::assign(std::string const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x409568: CScriptString::operator=(CScriptString const&) (scriptstring.cpp:76)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332==    by 0x41549B: ExecuteString(asIScriptEngine*, char const*, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:137)
==14332==    by 0x402B62: main (test.cpp:30)
==14332==  Address 0x5cfd708 is 8 bytes before a block of size 16 alloc'd
==14332==    at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14332==    by 0x40B218: StringDefaultFactory() (scriptstring.cpp:618)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332==    by 0x41549B: ExecuteString(asIScriptEngine*, char const*, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:137)
==14332==    by 0x402B62: main (test.cpp:30)
==14332== 
==14332== Invalid free() / delete / delete[] / realloc()
==14332==    at 0x4C2C2BC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14332==    by 0x4F054B7: std::string::assign(std::string const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x409568: CScriptString::operator=(CScriptString const&) (scriptstring.cpp:76)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332==    by 0x41549B: ExecuteString(asIScriptEngine*, char const*, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:137)
==14332==    by 0x402B62: main (test.cpp:30)
==14332==  Address 0x5cfd6f8 is 24 bytes before a block of size 16 alloc'd
==14332==    at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14332==    by 0x40B218: StringDefaultFactory() (scriptstring.cpp:618)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332==    by 0x41549B: ExecuteString(asIScriptEngine*, char const*, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:137)
==14332==    by 0x402B62: main (test.cpp:30)
==14332== 
==14332== Invalid read of size 8
==14332==    at 0x4F0542B: std::string::assign(std::string const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x409568: CScriptString::operator=(CScriptString const&) (scriptstring.cpp:76)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332==    by 0x41549B: ExecuteString(asIScriptEngine*, char const*, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:137)
==14332==    by 0x402B62: main (test.cpp:30)
==14332==  Address 0x5cfd458 is 24 bytes inside a block of size 28 alloc'd
==14332==    at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14332==    by 0x4F03F68: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x4F0423A: std::string::_M_mutate(unsigned long, unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x4F0485D: std::string::_M_replace_safe(unsigned long, unsigned long, char const*, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x409354: CScriptString::CScriptString(char const*, unsigned int) (scriptstring.cpp:22)
==14332==    by 0x40B161: StringFactory(unsigned int, char const*) (scriptstring.cpp:603)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332== 
==14332== Invalid read of size 4
==14332==    at 0x4F0543B: std::string::assign(std::string const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x409568: CScriptString::operator=(CScriptString const&) (scriptstring.cpp:76)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332==    by 0x41549B: ExecuteString(asIScriptEngine*, char const*, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:137)
==14332==    by 0x402B62: main (test.cpp:30)
==14332==  Address 0x72615a is not stack'd, malloc'd or (recently) free'd
==14332== 
==14332== 
==14332== Process terminating with default action of signal 11 (SIGSEGV)
==14332==  Access not within mapped region at address 0x72615A
==14332==    at 0x4F0543B: std::string::assign(std::string const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)
==14332==    by 0x409568: CScriptString::operator=(CScriptString const&) (scriptstring.cpp:76)
==14332==    by 0x48C0BB: X64_CallFunction(unsigned long const*, int, unsigned long (*)(), unsigned long&, bool) (in /home/droz/test)
==14332==    by 0x48CCE6: CallSystemFunctionNative(asCContext*, asCScriptFunction*, void*, unsigned int*, void*, unsigned long&, void*) (in /home/droz/test)
==14332==    by 0x48BA86: CallSystemFunction(int, asCContext*) (in /home/droz/test)
==14332==    by 0x421820: asCContext::ExecuteNext() (in /home/droz/test)
==14332==    by 0x41E965: asCContext::Execute() (in /home/droz/test)
==14332==    by 0x41571F: ExecuteString(asIScriptEngine*, char const*, void*, int, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:183)
==14332==    by 0x41549B: ExecuteString(asIScriptEngine*, char const*, asIScriptModule*, asIScriptContext*) (scripthelper.cpp:137)
==14332==    by 0x402B62: main (test.cpp:30)
==14332==  If you believe this happened as a result of a stack
==14332==  overflow in your program's main thread (unlikely but
==14332==  possible), you can try to increase the size of the
==14332==  main thread stack using the --main-stacksize= flag.
==14332==  The main thread stack size used in this run was 8388608.

After a quick review of the UString.cpp I noticed a couple of errors.

1. The UString::UString(const char*, size_t) constructor doesn't copy the full buffer to the mData member. It will only copy the buffer up to the first null-character.

2. The UString::substr method returns a UString by value, but it is registered as returning a new UString by pointer. The method needs to return the substr by pointer, or at least by reference, or else you will have trouble with reference counting and memory management.

As I didn't do a thorough review, there might be other problems like this that I didn't notice.

Still, neither of these two problems are likely the cause of the problem you're facing above.

For the problem you're facing, I have a feeling that the script statement "string s = string(dict[\"foo\"]);\n" is causing the problem. It is probably not working as expected due to the string class being a ref type in your case. It is quite possible that there is a bug in the library, or dictionary add-on, when trying to do a value conversion on a ref type. I'll investigate this.

In the meantime you may want to try using a ref cast like this "string @s = cast<string>(dict[\"foo\"]);\n". That should work.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

1. Thanks for pointing that out, eventually it would have caused me a issue.

2. The substr method in ustring does return by value, but that isn't exposed to angelscript. The ref-counting is only really used for the scripting language, internally I use UString as a value type.


r = engine->RegisterObjectMethod("string", "string@ substr(size_t s=0, size_t l=size_t(-1)) const", asFUNCTION(UStringSubstr), asCALL_CDECL_OBJLAST); assert(r>=0);

UString* UStringSubstr(std::size_t pos, std::size_t len, const UString& obj) { return new UString(obj.substr(pos, len)); }

Using the cast<string> does work, thanks for the help.

I've fixed the bug in the compiler that made the value conversion fail. You can find this fix in revision 2180.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

Thanks, works great now.

This topic is closed to new replies.

Advertisement