Sign in to follow this  
njpaul

wifstream and unicode

Recommended Posts

njpaul    367
Ok, I can get Unicode to work with wifstream if I do something like
wifstream fin(_wfopen(L"test.txt",L"rb"));
What I really need to do though is declare the wifstream earlier, like a member of a class, and then pass it the file pointer returned from _wfopen to open the stream. I spent an hour or two looking up various ways to do this, but obviously I failed. Does anyone know how I can achieve this? I'm thinking of creating a class that inherits from wifstream, but I don't know if I can achieve what I want this way. I can't use the normal open function of wifstream because it doesn't take in a unicode string. Any help, appreciated as always.

Share this post


Link to post
Share on other sites
taby    1265
This code shows some basic usage of wide streams. I found that it did not work on files saved in Windows Notepad. Yet, it works great in other programs. Strange.



#include <iostream>
#include <fstream>
#include <string>
using namespace std;


// wide string to basic string
string wsts(const wstring &src_string);

// basic string to wide string
wstring stws(const string &src_string);


int main(void)
{
// open file
// the filename parameter must be in ASCII
wifstream infile("lala.unicode.txt");

wstring line_contents;

while(getline(infile, line_contents))
{
wcout << line_contents << L'\n';
cout << wsts(line_contents) << endl;
}

return 0;
}


string wsts(const wstring &src_string)
{
size_t src_len = src_string.length();

if(0 == src_len)
return "";

char *buf = new(std::nothrow) char[src_len + 1];

if(0 == buf)
return "";

wcstombs(buf, src_string.c_str(), src_len);
buf[src_len] = '\0';

string final_string = buf;

if(0 != buf)
delete [] buf;

return final_string;
}


wstring stws(const string &src_string)
{
size_t src_len = src_string.length();

if(0 == src_len)
return L"";

wchar_t *buf = new(std::nothrow) wchar_t[src_len + 1];

if(0 == buf)
return L"";

mbstowcs(buf, src_string.c_str(), src_len);
buf[src_len] = L'\0';

wstring final_string = buf;

if(0 != buf)
delete [] buf;

return final_string;
}


Share this post


Link to post
Share on other sites
snk_kid    1312
Quote:
Original post by njpaul
Ok, I can get Unicode to work with wifstream if I do something like

wifstream fin(_wfopen(L"test.txt",L"rb"));


What I really need to do though is declare the wifstream earlier, like a member of a class, and then pass it the file pointer returned from _wfopen to open the stream. I spent an hour or two looking up various ways to do this, but obviously I failed. Does anyone know how I can achieve this?


There is no portable way of doing this, besides the point you don't need to do this since C++ iostreams are already internationalized/localized, iostreams maintain "locales" of which contain an assortment of "facets" for formatting/parsing in an internationalized/localized manner.

Just opening a file using a wide-character string literal does not mean you are ready to read in a unicode file, _wfopen behaves identically to fopen except that _wfopen takes wide-character strings arguments.

What you need to be doing is setting up locale object correctly for the external & internal representations and passing it to an instance of a file stream. Typically you create a locale object with a locale name and that is all that needs to be done however sometimes you need modify/add facets.


Quote:
Original post by njpaul
I'm thinking of creating a class that inherits from wifstream, but I don't know if I can achieve what I want this way. I can't use the normal open function of wifstream because it doesn't take in a unicode string. Any help, appreciated as always.


As i mentioned earlier this pretty much pointless but if you are going to have a custom file-stream that works with C file pointers then don't inherit from wifstream.

What you should be doing is create a new stream buffer type by deriving from std::basic_streambuf and overriding the appropriate virtual functions. Then make a new iostream type for your new stream buffer by deriving from std::basic_i/o/iostream. The new stream type should contain your new stream buffer type and on construction of the new stream it should pass the address of the stream buffer to the base stream type on construction. No need for overriding any member functions in the new stream type just add the extra member functions needed (i.e. open/close/is_open etc).

[Edited by - snk_kid on November 10, 2005 3:27:51 AM]

Share this post


Link to post
Share on other sites
njpaul    367
Quote:
Original post by snk_kid
What you need to be doing is setting up locale object correctly for the external & internal representations and passing it to an instance of a file stream. Typically you create a locale object with a locale name and that is all that needs to be done however sometimes you need modify/add facets.


Do you have an example of this? I tried doing this once before but had a hard time finding documentation and examples and it ultimately ended up failing.

Share this post


Link to post
Share on other sites
Grammarye    142
Check out http://www.cantrip.org/locale.html for a reasonable first discussion of locale objects, custom facets, and custom stream operators that use them.

However, the setting of a locale can be summed up in a single line from that article:

std::cout.imbue( std::locale("<insert your favourite locale name here>") );

Voila - cout will now operate in the chosen locale. Facets will allow you to customise behaviour further.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this