Codecompletion delays fixed in rev 1826 (update: in rev 1840)

[rickg22]

"One extra thread" the answer is. More refinement Code completion needs, indeed. Hmm.

[Ceniza]

I wonder if setting CodeCompletion to use only 1 thread would "help". It's like if instead of using 2 threads it was using 1/2 thread.

[thomas]

As for this I have one question, why are we using threads at all if doesn't run more than one at once?

It does not freeze the GUI for 20 seconds.

I think you mean it only runs one <i>extra</i> thread, counting the main thread if you have one CPU?, or it only one thread total include the main one?

The former is the case (obviously, nothing else makes sense).
You could indeed argue why the "thread pool" is rather a "thread blocker" than a real thread pool, but that is not the cause of the bottlenecks in parsing. Rick initially wrote that code to be simple, and he is certainly aware of the unnecessary thread creation overhead. That, however, is not the thing which causes all the problems which we are facing in code completion. Since we are only creating a dozen or two unnecessary threads, this is manageable, even on such a bad performer like Windows. He might change that one day, but only after the real issues are solved.

The whole point of threads is to run more than the number of processors you have because the CPU is rarely full on, threads of execution spend quite a bit of time waiting.

Look at Ceniza's screenshot. It has a higher resolution than mine, so the issue is better visible there.
His CPU is 75% idle on the average (if you count in hyperthreading). 50% of the time, it uses one CPU, and 50% of the time, it is completely stalled. Thus, he could as well run two more concurrent threads without any penalites.

Also I think the threading model needs a thorough rewrite or going over because it dead locks all the time on linux.

Yes, although the deadlocks do not necessarily come from that, the parser does some heavy locking here and there too, so it might as well be that.

[Ceniza]

I tried revision 1854 last night which creates 4 threads and I have the same results

BTW, those times (working then stalled) are about 3-5 seconds each (haven't timed them though, but they really count in seconds). Doesn't look like waiting for I/O to me.

[thomas]

Uh... seconds? What on earth is taking seconds? (I was thinking you had the task monitor running at higher speed... :lol:)

Rick, you don't happen to trigger the next parser with a wxTimer event, do you?

[Ceniza]

By checking the process' threads you can see all of them are executed in the working time (I got 36 threads with Thomas' workspace: 9 projects, 4 threads per project). When the CPU is stalled... well... you know... they do nothing.

[rickg22]

Rick, you don't happen to trigger the next parser with a wxTimer event, do you?

Nope... this is how it works: The wxTimer works as a "countdown for start parsing." When a new file is requested to be parsed (from the main thread, that is), it resets the countdown. When all files are requested, the countdown begins and finally the parsing is triggered. This is to avoid problems with adding files recursively taking forever.

But I think the part that measures the time is broken I was going to change it, but i was too tired last night.

But don't worry! Everytime we're getting closer to a bug free plugin

[Ceniza]

Rick: could you add the textbox for the concurrent threads again? I'd like to try it with only 1 thread.

[rickg22]

Um... :oops: I don't know how... I didn't touch that code

[Ceniza]

I'm trying the latest changes in CodeCompletion and the results are very similar: 50% working time and about 60-75% CPU usage (when working).

Even better, Code::Blocks just died with a "Reading from 00000000" error.

Revision 1889.

[rickg22]

<i>Even better, Code::Blocks just died with a "Reading from 00000000" error.</i>

If you told me the exact address of the fault (a line # would be even better), i'd gladly appreciiate it.

[Ceniza]

Sorry Rick but the report isn't good enough to point you somewhere since I always compile with -s -O3 instead of -g.

I'll rebuild it again with -g and try the same test. If I get something I'll tell you.

[Ceniza]

Program received signal SIGSEGV, Segmentation fault.
[Switching to thread 2772.0x4ac]
0x77c17026 in msvcrt!memcpy () from C:\WINDOWS\system32\msvcrt.dll
(gdb) bt
#0  0x77c17026 in msvcrt!memcpy () from C:\WINDOWS\system32\msvcrt.dll
#1  0x10063346 in wxStringBase::ConcatSelf ()
   from C:\WINDOWS\system32\wxmsw26u_gcc_custom.dll
#2  0x65ed1cf8 in wxStringBase::ConcatSelf (this=0x14e0ed8c, nLen=239120052,
    src=0x1bd2dac) at C:/MinGW/include/wx/string.h:281
#3  0x65ed1d96 in wxStringBase::append (this=0x14e0ed8c, str=@0x651fc78)
    at C:/MinGW/include/wx/string.h:400
#4  0x65ed6438 in wxString::append (this=0x14e0ed8c, str=@0x651fc78)
    at C:/MinGW/include/wx/string.h:1157
#5  0x65ed66fc in wxString::operator<< (this=0x14e0ed8c, s=@0x651fc78)
    at C:/MinGW/include/wx/string.h:889
#6  0x65eab056 in ParserThread::DoParse (this=0x14e0ed20)
    at plugins/codecompletion/parser/parserthread.cpp:654
#7  0x65eacc4f in ParserThread::HandleClass (this=0x14e0ed20, isClass=true)
    at plugins/codecompletion/parser/parserthread.cpp:1017
#8  0x65eaa51b in ParserThread::DoParse (this=0x14e0ed20)
    at plugins/codecompletion/parser/parserthread.cpp:518
#9  0x65ea9c00 in ParserThread::Parse (this=0x14e0ed20)
    at plugins/codecompletion/parser/parserthread.cpp:372
#10 0x65ed1695 in ParserThread::Execute (this=0x14e0ed20)
    at plugins/codecompletion/parser/parserthread.h:43
#11 0x61965fa9 in PrivateThread::Entry (this=0x2c051c8)
    at sdk/cbthreadpool.cpp:86
...