cc-branch crashes on linux

[Jenna]

The cc-branch crashes reproucible on linux, if <wx/hashmap.h> is parsed.
Ther crash happens in wxString Tokenizer::ReadToEOL(bool nestBraces, bool stripUnneeded) when #define _WX_DECLARE_HASHTABLE( [...] is parsed.

After many debugging I found the cause.
Normally if we reach the backslash before the EOL, the space(s) before the backslash and the backslash itself are removed from buffer by decrementing the pointer p and after that moving to the next char:

Code Select Expand

while (*(--p) <= _T(' ') && p > buffer)

In some rare cases we just appended the buffer to the return-string and set the temporary pointer p to the beginning of buffer. If we now decrement p and look if the character it points to is less or equal ' ' we get a segfault.
The same might happen some lines later.

This can only happen if the buffer is smaller than the line and if the next character (after appending buffer to str) is the backslash before the EOL.

I attach a project wher this happens (at least on linux 64-bit), I just copied the define of _WX_DECLARE_HASHTABLE into main.cpp and removed everything else.

The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).
Another approach would be to leave everything as it is, but do it only if p is greater than buffer and if it is not (buffer has already been appended to str), remove the trailing spaces from str.
It's also easy to do, but I am not sure whether it is needed, or if we can leave the spaces.

It would be nice if the cc-gurus can have alook at the patch:

Code Select Expand

Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (Revision 6508)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (Arbeitskopie)
@@ -410,13 +410,13 @@
                 break;
             else
             {
-                while (*(--p) <= _T(' ') && p > buffer)
+                while ((p > buffer) && *(--p) <= _T(' '))
                     ;
                 MoveToNextChar();
             }
         }

-        while (*(p - 1) <= _T(' ') && --p > buffer)
+        while (p > buffer && *(--p) <= _T(' '))
             ;
         str.Append(buffer, p - buffer);



The attached patch is the same as above, but with abuild-fix and a correction of slashes in the unix project-file.

[Loaden]

Thank Jens!

[Loaden]

Hi, Jens, can you give me a favor?
In Ubuntu 10.04, if apply this patch, will lead C::B can not load libcodecompletion.so.
I check it more times, but i can not find the reason.
I need you help. ^_^

[Loaden]

WOW, I got the reason.
THAT IS: We must add "classnavigator.cpp/h" to project!!
Solved now!

[MortenMacFly]

The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).

Ooops - sounds weired. You mean for a single file "Hello World" project?! :shock:

@Loaden: What could be the reason for this?!

[Loaden]

The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).

Ooops - sounds weired. You mean for a single file "Hello World" project?! :shock:

@Loaden: What could be the reason for this?!

I am a little busy, will comment later.

[Jenna]

Another approach would be to leave everything as it is, but do it only if p is greater than buffer and if it is not (buffer has already been appended to str), remove the trailing spaces from str.

Next patch, this time it generates the exactly same tokens-tree, -list, include-dirs and file-list:

Code Select Expand

Index: src/plugins/codecompletion/parser/tokenizer.cpp
===================================================================
--- src/plugins/codecompletion/parser/tokenizer.cpp (Revision 6508)
+++ src/plugins/codecompletion/parser/tokenizer.cpp (Arbeitskopie)
@@ -410,14 +410,34 @@
                 break;
             else
             {
-                while (*(--p) <= _T(' ') && p > buffer)
-                    ;
+                // only work with p if the buffer is not just appended (p==buffer)
+                // otherwise remove the traling spaces from str (avoid segfault on linux in rare cases)
+                if(p > buffer)
+                {
+                    while (*(--p) <= _T(' ') && p > buffer)
+                        ;
+                }
+                else
+                {
+                    while(!str.IsEmpty() && str.Last() <= _T(' '))
+                        str.RemoveLast();
+                }
                 MoveToNextChar();
             }
         }

-        while (*(p - 1) <= _T(' ') && --p > buffer)
-            ;
+        // only work with p if the buffer is not just appended (p==buffer)
+        // otherwise remove the traling spaces from str (avoid segfault on linux in rare cases)
+        if(p > buffer)
+        {
+            while (*(p - 1) <= _T(' ') && --p > buffer)
+                    ;
+        }
+        else
+        {
+            while(!str.IsEmpty() && str.Last() <= _T(' '))
+                str.RemoveLast();
+        }
         str.Append(buffer, p - buffer);

         TRACE(_T("ReadToEOL(): (END) We are now at line %d, CurrentChar='%c', PreviousChar='%c', NextChar='%c'"),


[attachment deleted by admin]

[Loaden]

The following patch avoids that problem, but it seems to change the behaviou alittle bit (for a simple hello-world project one file more is parsed, but less tokens are found).

I'm back.
Need changed to:

-        while (*(p - 1) <= _T(' ') && --p > buffer)
-            ;
+        while (p > buffer && *(p - 1) <= _T(' '))
+            --p;

This patch fixed crash in ReadParentheses too.
And do some simple optimization.

[Jenna]

Tested and confirmed, no crash and identical tokens-list etc. for test-project.

Thank you for the quick fix !

[killerbot]

I don't know if it is related, or already fixed by above mentioned patch, but for me CB is crashing also a lot.
Whenever I have method call for example, and just before the closing ')' of the call I press comma (,) CB dies on me (well never comes back and I have to kill it).

x.foo(1,2,3);  --> before the ) and after the 3 press comma and it crashes.

Other way to reproduce. Have CB create a console application,

Go to the line of void main(), and try to type "int argc," , once the "," has been typed --> crash.

Can the above path be committed to cc branch.

PS : I am on linux

[killerbot]

I just svn updated (cc branch and rev 6522) and the problem still occurs. Anyone else can reproduce this ?

EDIT : I did even a make clean && make : no luck, issue remains.

[Loaden]

I don't know if it is related, or already fixed by above mentioned patch, but for me CB is crashing also a lot.
Whenever I have method call for example, and just before the closing ')' of the call I press comma (,) CB dies on me (well never comes back and I have to kill it).

x.foo(1,2,3);  --> before the ) and after the 3 press comma and it crashes.

Other way to reproduce. Have CB create a console application,

Go to the line of void main(), and try to type "int argc," , once the "," has been typed --> crash.

Can the above path be committed to cc branch.

PS : I am on linux

This is another issue.
I will fix it soon, please waiting.

[Loaden]

Fixed!

[killerbot]

I can confirm this patch fixes the issue. Bring it on the cc branch ;-)

[killerbot]

today, already had a lot of other crashes.
From the log xml, this is the stack part (all crashes resulted in the same dump like this) :

Code Select Expand


 <stack>
   <frame level="0"/>
   <frame level="1" function="NativeParser::OnEditorActivatedTimer(wxTimerEvent&amp;)" offset="0000024d"/>
   <frame level="2" function="wxEvtHandler::ProcessEventIfMatches(wxEventTableEntryBase const&amp;, wxEvtHandler*, wxEvent&amp;)" offset="00000055"/>
   <frame level="3" function="wxEventHashTable::HandleEvent(wxEvent&amp;, wxEvtHandler*)" offset="000000ac"/>
   <frame level="4" function="wxEvtHandler::ProcessEvent(wxEvent&amp;)" offset="000000b4"/>
   <frame level="5" function="wxTimerBase::Notify()" offset="0000006e"/>
   <frame level="6"/>
   <frame level="7"/>
   <frame level="8" function="g_main_context_dispatch" offset="000001f3"/>
   <frame level="9"/>
   <frame level="10" function="g_main_loop_run" offset="00000195"/>
   <frame level="11" function="gtk_main" offset="000000a7"/>
   <frame level="12" function="wxEventLoop::Run()" offset="00000048"/>
   <frame level="13" function="wxAppBase::MainLoop()" offset="0000004b"/>
 </stack>


Hopefully this can be fixed.

I also have the *feeling* that cc works less good then before. Several times nothing comes up, where I think in the past it did. As said : a *feeling*.


EDIT : I found a way to reproduce this in my working environment (didn't try with a small project).
So I have a workspace with several projects, some files of these projects are open. I drag and drop a cpp file to CB (this file doesn't belong to any of the projects). CB shows the file.
Then I select the tab of one of those other open files (which are part of the project). All still is ok, then I select the tab of the "open non project file" --> crash.

EDIT 2 : tried this with a small project (generated by CB), and same recipe as in the EDIT above.