Problems with FilesGroupsAndMasks::Save

[Der Meister]

I'm using Code::Blocks Version 1.0 revision 1595 (gcc 3.4.4 Linux/unicode, build: Dec 26 2005 17:16:07).

In the function FilesGroupsAndMasks::Save the each file-group and its mask are stored in that way:

Code Select Expand


        FileGroups* fg = m_Groups[i];
        wxString key;
        key << _("/file_groups/group") << i << _T("/") << _T("name");
conf->Write(key, fg->groupName);
        key.Clear();
        key << _("/file_groups/group") << i << _T("/") << _T("mask");
conf->Write(key, GetStringFromArray(fg->fileMasks, _T(";")));

Although this looks correct it doesn't work (at least for me). The keys look like this for all entries:

Code Select Expand


/file_groups/group/name
/file_groups/group/mask

You see, the indices are missing although they are pushed into the stream. I have no idea why this happens. I don't think that my wxGTK-installation is broken because similar code works well in other projects and probably in Code::Blocks, too. Is anyone able to re-produce this problem?

Anyway, the result of this problem is that only the last file mask is stored in the config file and thus after re-loading Code::Blocks all files are either added to the group "Ressources" (which seems to be the group with the highest index) or to the group "Others" if the mask for ressources doesn't apply to them.

[thomas]

That's a known problem which we have everywhere.

I don't know the cause, but wxString::operator<< fails miserably in Unicode. Using Format() or Printf() solves the problem. We'll have to spend a few days any time soon locating all instances of operator<<.

[Der Meister]

Oh... sorry, seems as I missed this thread.

Anyway, 280Z28's solution with wxString::Printf seems to work. It looks like this now:

Code Select Expand


        FileGroups* fg = m_Groups[i];
        wxString key;
        key.Printf(_T("/file_groups/group%d/name"), i);
        conf->Write(key, fg->groupName);
        key.Clear();
        key.Printf(_T("/file_groups/group%d/mask"), i);
        conf->Write(key, GetStringFromArray(fg->fileMasks, _T(";")));


[grv575]

That's a known problem which we have everywhere.

I don't know the cause, but wxString::operator<< fails miserably in Unicode. Using Format() or Printf() solves the problem. We'll have to spend a few days any time soon locating all instances of operator<<.

This help?  Wide I/O streams:

http://www.codeproject.com/vcpp/stl/upgradingstlappstounicode.asp

[Der Meister]

I'm not sure if this really would help because as far as I know this is not a general unicode problem but a Code::Blocks specific problem. wxString::operator <<(int) *does* work, even with unicode. Take a look at this example:

Code Select Expand


#include <wx/wx.h>

int main()
{
wxString foo;

for (int i = 0; i < 5; ++i)
{
foo << _("/file_groups/group") << i << _("/") << _("name");
wxLogError(foo);
foo.clear();
}
}

And here is the output of wx-config:

Code Select Expand


$ wx-config-2.6 --libs
-pthread   -lwx_gtk2u_xrc-2.6 -lwx_gtk2u_html-2.6 -lwx_gtk2u_adv-2.6 -lwx_gtk2u_core-2.6 -lwx_baseu_xml-2.6 -lwx_baseu_net-2.6 -lwx_baseu-2.6

You can see I really do use unicode and not ANSI.

The output of this sample is what a normal person would expect form the source code:

Code Select Expand


23:35:32: Error: /file_groups/group0/name
23:35:32: Error: /file_groups/group1/name
23:35:32: Error: /file_groups/group2/name
23:35:32: Error: /file_groups/group3/name
23:35:32: Error: /file_groups/group4/name

You see, it works.

But the corresponding source code *in* Code::Blocks (i. e. FilesGroupsAndMasks::Save) doesn't work as it should. It simply drops the index. Thus I think that it *must* be a problem of Code::Blocks and not a general unicode problem. But I have absolutely no idea where the problem really is.

[grv575]

I'm not sure if this really would help because as far as I know this is not a general unicode problem but a Code::Blocks specific problem. wxString::operator <<(int) *does* work, even with unicode. Take a look at this example:
...
You see, it works.

Don't think so.  Build CB unicode.  Run CB unicode and create a new wxWidgets project.  Set paths to use Unicode wx dll.  Edit wx/setup.h to have     #define wxUSE_UNICODE 1   (<-- essential).  Now, using your example:

Code Select Expand


void MyFrame::OnAbout(wxCommandEvent& event)
{
    int i = 50;
    wxString foo;
    foo << _("/file_groups/group") << i << _("/") << _("name");
    wxLogError(foo);
}


And the i does not get handled properly:


[attachment deleted by admin]

[280Z28]

I've posted patches for most instances of << with integers throughout Code::Blocks in individual patches by topic over at SF.

Eventually they'll be committed (as Thomas has time; right now he doesn't have a Unicode build to test on so he can't get to them).

In the meantime, Unicode users can manually apply the following patches:

brings gdb support back to the unicode build
http://sourceforge.net/tracker/index.php?func=detail&aid=1389953&group_id=126998&atid=707418

Syntax highlighting in the unicode build
http://sourceforge.net/tracker/index.php?func=detail&aid=1386025&group_id=126998&atid=707418

http://sourceforge.net/tracker/index.php?func=detail&aid=1385378&group_id=126998&atid=707418

[grv575]

Maybe this illustrates the problem:

Code Select Expand


    int i = 64;
    wxChar ch = i;
    wxString str = _(" ");
    str << ch;
    wxString foo;
    foo << _("/file_groups/group") << str << _("/") << _("name");
    wxLogError(foo);


Produces (see below).

Now from that codeproject link:

Code Select Expand


wchar_t problems

wchar_t is the type that is used for wide characters and is defined like this:

typedef unsigned short wchar_t ;

Unfortunately, because it is a typedef instead of a real C++ type, defining it like this has one serious flaw: you can't overload on it. Look at the following code:

TCHAR ch = _T('A') ;
tcout << ch << endl ;

Using narrow strings, this does what you would expect: print out the letter A. Using wide strings, it prints out 65. The compiler decides that you are streaming out an unsigned short and prints it out as a numeric value instead of a wide character. Aaargh!!! There is no solution for this other than going through your entire code base, looking for instances where you stream out individual characters and fix them. I wrote a little function to make it a little more obvious what was going on:

#ifdef _UNICODE
    // NOTE: Can't stream out wchar_t's - convert to a string first!
    inline std::wstring toStreamTchar( wchar_t ch )
            { return std::wstring(&ch,1) ; }
#else
    // NOTE: It's safe to stream out narrow char's directly.
    inline char toStreamTchar( char ch ) { return ch ; }
#endif // _UNICODE   

TCHAR ch = _T('A') ;
tcout << toStreamTchar(ch) << endl ;


So... it looks like you can't directly stream wxChars... (it thinks they are shorts and just uses the ascii equivalent or somesuch...).


[attachment deleted by admin]

[tiwag]

that's interesting, i tried this on my ubuntu 5.10 breezy with a CodeBlocks SVN r1595 (gcc 4.0.2,  wx-gtk-2.6.1 unicode)
and it doesn't show the problem, which you're discussing here...


[attachment deleted by admin]

[thomas]

This help?  Wide I/O streams

So... it looks like you can't directly stream wxChars... (it thinks they are shorts and just uses the ascii equivalent or somesuch...).

Well, we don't actually stream wide characters. What we do is, we invoke an overloaded operator. True, this happens to be the same operator that is used for streams, but that doesn't mean anything. We are only calling an (arbitrary) overloaded operator, which is just as good as if we were calling any other member function. So it really doesn't have to do with streams, it is a wxString bug.

[grv575]

Was not my point.  Look in string.h in the wx src.  The way wxChar is defined depends on settings in wx/setup.h, version of gcc being used & whether wchar_t is a real type or a typedef to unsigned short (as mentioned in that codeproject link).  Now if wxChar ends up resolving to an unsigned short instead of a real type then it will invoke the operator for an integral value and give you not what you want in Unicode builds:

Code Select Expand


wxTextOutputStream& wxTextOutputStream::operator<<(wxUint16 c)
{
    wxString str;
    str.Printf(wxT("%u"), (unsigned int)c);
    WriteString(str);

    return *this;
}


I'm guessing you can get the same behavior by doing str.Printf(wxt("%u")) to print an unsigned int and not a Unicode character.  Of course if you don't have wx_USEUNICODE defined it will use a different overload and conversion.

Code Select Expand


#if wxUSE_UNICODE && wxWCHAR_T_IS_REAL_TYPE

wxTextOutputStream& wxTextOutputStream::operator<<(wchar_t wc)
{
    WriteString( wxString(&wc, m_conv, 1) );

    return *this;
}

#endif // wxUSE_UNICODE


It looks like << just does a + operation where wxStrings are concatenated.  The source states that no conversion is done - use Printf() instead.  So converting from int / unsigned short to wxString is probably not done properly as you'd expect in Unicode mode.

[grv575]

To clairify:

What

Code Select Expand


  int i = 64;
    wxChar ch = i;
    wxString str = _(" ");
    str << ch;


does is take the narrow integral value 64 and convert this to whatever 64 maps to for wide characters ('@').  It does not convert to 36 00 34 00 or whatever the wide character representation of the string "64" happens to be.

[thomas]

But... make a new wxWidgets project and modify it like this:

Code Select Expand

#include <stdio.h>
#include <typeinfo>
bool MyApp::OnInit()
{
printf("wchar_t %d\n", typeid(wchar_t) == typeid(wxChar));
printf("int %d\n", typeid(int) == typeid(wxChar));
printf("uint %d\n", typeid(unsigned int) == typeid(wxChar));
printf("char %d\n", typeid(char) == typeid(wxChar));
printf("uchar %d\n", typeid(unsigned char) == typeid(wxChar));
// MyFrame* frame = new MyFrame(0L, _("wxWidgets Application Template"));
// frame->Show();
return true;
}


Output non-Unicode:
wchar_t 0
int 0
uint 0
char 1
uchar 0

Output Unicode:
wchar_t 1
int 0
uint 0
char 0
uchar 0

As you can see, wxChar does not map to unsigned short, but to wchar_t.

EDIT:
Furthermore, we are neither using wxTextOutputStream or anything, nor do we have to deal with wchar_t here at all.
What we use are these two overloaded operators:

wxString::wxString& operator<<(const wxChar *psz) { append(psz); return *this; }
wxString::wxString& operator<<(int i) { return (*this) << Format(_T("%d"), i); }
...
wxStringBase::wxStringBase& append(const wxChar *sz) { ConcatSelf(wxStrlen(sz), sz); return *this; }

...and that does not work for some reason.

[thomas]

I modified wx/string.h to narrow the problem.

Look at this:


For some reason, Format() returns an empty string.

EDIT:
The really grotesque points to note are:
1. Format uses PrintfV internally. If you use Printf or PrintfV, it will work fine.
2. If you do someString << Format(_T("%d"), someInt), it will still work fine, but it does not inside string.h

[Der Meister]

Don't think so.  Build CB unicode.  Run CB unicode and create a new wxWidgets project.  Set paths to use Unicode wx dll.  Edit wx/setup.h to have     #define wxUSE_UNICODE 1   (<-- essential).

Well, I have Code::Blocks build as unicode (you can see this in the first post) and I have build wxWidgets with unicode. And my wx/setup.h has already the #define you mentioned. And of course the paths are setup correct, I even checked that twice. This example works on my maschine.

If I did not have the problems within Code::Blocks, too, I would say it is a problem that occures only with some wxWidgets versions (I'm using wxGTK 2.6.1 (unicode build) on a gentoo linux) and not with all versions. And as some of you have problems with my little example it seems that this isn't just a problem of Code::Blocks. I have no idea what the reason for this problems is. Especially thomas' results are confusing. Maybe we should talk to some wxWidgets developers about this problems?