Welcome to test it.
1, I add a state variable to Tokenizer, and this will make the code more clearer. So, we can still use the GetToken() and PeekToken() with out any argument. also, I just re-factor the SkipUnWanted() function.
2, add a attempt to deal variable seperated by commas (http://forums.next.codeblocks.org/index.php/topic,11720.0.html) problem.(these related code is developed by blueshake).
[attachment deleted by admin]
Mmmmh... a few things are not clear to me:
For example: Why did you remove theis code:
#ifdef __WXMSW__ // This is a Windows only bug!
else if (c == 178 || c == 179 || c == 185) // fetch ² and ³
{
str = c;
MoveToNextChar();
}
#endif
???
This re-introduces a critical crash-bug on Windows?!
You've removed quite some portions... could you explain a little more why exactly? Or did you have a difference code base?
@morten:
Sorry, That's my fault. That's becaus In my Chinese Windows XP, I can't see the charactor well, sometimes, there will be shown a rectangle or a question mark. So, that's was a little annoying. So I delete that code. :wink:
Is there other question or comments?
Quote from: ollydbg on January 01, 2010, 03:28:34 PM
Is there other question or comments?
Sure. ;-)
The other thing is: Your re-factored the
SkipUnwanted(bool bGetValue) quite heavily including the removal of handling of preprocessor directives (which makes
m_IsPreprocessor and
m_LastPreprocessor obsolete btw.). Why? This was working great?!
Finally shouldn't statements like:
if ( m_State & tsSkipQuestion )better be:
if ( (m_State & tsSkipQuestion) == tsSkipQuestion)(Need to have a look myself to see how the bits are defined though...)
Quote from: MortenMacFly on January 01, 2010, 07:44:05 PM
The other thing is: Your re-factored the SkipUnwanted(bool bGetValue) quite heavily including the removal of handling of preprocessor directives (which makes m_IsPreprocessor and m_LastPreprocessor obsolete btw.). Why? This was working great?!
Handle preprocessor is quite easy in ParserThread, So, I remove the handling preprocessor related code in Tokenizer, I don't think they are needed any more. :D Maybe, we need more test.