Loading big projects on windows is extreamly slow

[BlueHazzard]

Hi,
loading the attached project on my new windows machine takes

Code Select Expand

ProjectManager::LoadProject took: 124.372 seconds.
This loading makes codeblocks completely non repsonsive
in a linux VM (considerable less performant) loading the same project takes only

Code Select Expand

ProjectManager::LoadProject took: 9.684 seconds.

I think the main reason is the wxWidgets function wxFileName::MakeRelativeTo

more information will follow...

attached is the ziped project file in question

[oBFusCATed]

Do you have other timing lines in the log?
This is the output for me on linux (Ryzen 7 1700):

Code Select Expand


cbProject::Open took: 1.428 seconds.
NativeParser::DoFullParsing took: 0.068 seconds.
ProjectManager::SetProject took: 0.073 seconds.
ProjectManager::LoadProject took: 2.434 seconds.


This is my version info:

Code Select Expand


Name             : Code::Blocks
Version          : svn-r12278
SDK Version      : 2.6.0
Scintilla Version: 3.7.5
Author           : The Code::Blocks Team
E-mail           : info@codeblocks.org
Website          : http://www.codeblocks.org

wxWidgets Library (wxGTK port)
Version 3.0.4 (Unicode: wchar_t, debug level: 1),
compiled at May 14 2020 22:46:15

Runtime version of toolkit used is 2.24.
Compile-time GTK+ version is 2.24.32.


Are you using a release build of everything? Do you reproduce this if you use minimal set of plugins?

[oBFusCATed]

If you want to really know what is going on, then V-Tune is one of the the easiest tool to use.
And it seems that it supports mingw: https://ib-krajewski.blogspot.com/2019/07/vtune-amplifier-and-profiling-mingw-on.html

Hint: You all you need is "basic hot spots" type of profile and then look at the "callers and callees" tab. It should work on both Intel and Amd processors.

[BlueHazzard]

Looks cool, but i am not able to download it....
every time i try to download it asks me for user name and password, or for registration... I register but never get the email, i click resend, get the email, click on confirm, try to download, get error message that i am not logged in...
bu***sh**t website...

[stahta01]

Remember to check your junk mail folder online if using GMail it blocks a lot of stuff.

Tim S.

[BlueHazzard]

There seems to be two accounts, one normal intel account and one "intel dev account" in my profile is written "dev account activated" but if i try to login in my dev account i get wrong username/password.... ridiculous complicated...
on the amd page for µProf it was a simple download link, no registration, nothing... Sadly it seems that µProf is not able to use gdb symbols....

Code Select Expand

cbProject::Open took: 83.271 seconds.
NativeParser::DoFullParsing took: 0.590 seconds.
ProjectManager::SetProject took: 0.686 seconds.
ProjectManager::LoadProject took: 92.235 seconds.


[oBFusCATed]

Why are times so different? The first time you posted was 124.372 seconds, this one is 92.

[Miguel Gimenez]

In the Sign In to Get Your Download dialog there is a link in the bottom left corner, Maybe next time. Please take me to my download, it worked for me.

[BlueHazzard]

In the Sign In to Get Your Download dialog there is a link in the bottom left corner, Maybe next time. Please take me to my download, it worked for me.

This did it.... Thank you!

[BlueHazzard]

Code Select Expand

Function / Call Stack CPU Time Module Function (Full) Source File Start Address
GetLongPathNameW 65.7% KERNEL32.DLL GetLongPathNameW [Unknown] 0x1800068c0
  ↖ wxFileName::GetLongPath ← wxFileName::Normalize 65.7% wxmsw314u_gcc_custom.dll wxFileName::GetLongPath(void) const [Unknown] 0x64c5dd10
    wxFileName::MakeRelativeTo 62.2% wxmsw314u_gcc_custom.dll wxFileName::MakeRelativeTo(wxString const&, wxPathFormat) [Unknown] 0x64c64b50
      ↖ cbProject::CalculateCommonTopLevelPath ← cbProject::Open ← cbProject::cbProject ← ProjectManager::LoadProject ← MainFrame::DoOpenProject ← MainFrame::OpenGeneric ← MainFrame::OnDropFiles ← MainFrame::DoOnFileOpen ← MainFrame::OnFileOpen ← wxAppConsoleBase::CallEventHandler ← wxEvtHandler::ProcessEventIfMatchesId ← wxEventHashTable::HandleEvent ← wxEvtHandler::TryHereOnly ← wxEvtHandler::DoTryChain ← wxEvtHandler::ProcessEvent ← wxWindowBase::TryAfter ← wxEvtHandler::SafelyProcessEvent ← wxMenuBase::DoProcessEvent ← wxMenuBase::SendEvent ← wxFrame::MSWWindowProc ← DispatchMessageW ← wxGUIEventLoop::Dispatch ← wxEventLoopManual::ProcessEvents ← wxEventLoopManual::DoRun ← wxEventLoopBase::Run ← wxAppConsoleBase::MainLoop ← CodeBlocksApp::OnRun ← func@0x64c73c90 ← WinMain ← func@0x4dd760 ← func@0x40114b ← func@0x4010ed ← BaseThreadInitThunk ← RtlUserThreadStart 38.7% codeblocks.dll cbProject::CalculateCommonTopLevelPath(void) cbproject.cpp 0x7096cc5c
      ↖ cbProject::AddFile ← ProjectLoader::DoUnits ← ProjectLoader::Open ← cbProject::Open ← cbProject::cbProject ← ProjectManager::LoadProject ← MainFrame::DoOpenProject ← MainFrame::OpenGeneric ← MainFrame::OnDropFiles ← MainFrame::DoOnFileOpen ← MainFrame::OnFileOpen ← wxAppConsoleBase::CallEventHandler ← wxEvtHandler::ProcessEventIfMatchesId ← wxEventHashTable::HandleEvent ← wxEvtHandler::TryHereOnly ← wxEvtHandler::DoTryChain ← wxEvtHandler::ProcessEvent ← wxWindowBase::TryAfter ← wxEvtHandler::SafelyProcessEvent ← wxMenuBase::DoProcessEvent ← wxMenuBase::SendEvent ← wxFrame::MSWWindowProc ← DispatchMessageW ← wxGUIEventLoop::Dispatch ← wxEventLoopManual::ProcessEvents ← wxEventLoopManual::DoRun ← wxEventLoopBase::Run ← wxAppConsoleBase::MainLoop ← CodeBlocksApp::OnRun ← func@0x64c73c90 ← WinMain ← func@0x4dd760 ← func@0x40114b ← func@0x4010ed ← BaseThreadInitThunk ← RtlUserThreadStart 19.7% codeblocks.dll cbProject::AddFile(int, wxString const&, bool, bool, unsigned short) cbproject.cpp 0x7096f898
      ↖ HeaderDirTraverser::OnFile ← wxDir::Traverse 3.8% codecompletion.dll HeaderDirTraverser::OnFile(wxString const&) systemheadersthread.cpp 0x6775ca54
    ↖ wxPathList::Add ← ConfigManager::LocateDataFile 3.5% wxmsw314u_gcc_custom.dll wxPathList::Add(wxString const&) [Unknown] 0x64c521e0
    ↖ NormalizePath ← ParserBase::FindFileInIncludeDirs ← ParserBase::FindFirstFileInIncludeDirs ← ParserBase::GetFullFileName ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← Parser::Parse ← Parser::ParseFile ← ParserThread::HandleIncludes ← ParserThread::DoParse ← ParserThread::Parse ← ParserThread::Execute ... 0.0% codeblocks.dll NormalizePath(wxFileName&, wxString const&) globals.cpp 0x709f339e

Really nice tool i have to say... Installation gives some problems, but it is nice to use...

Why are times so different? The first time you posted was 124.372 seconds, this one is 92.

Don't know, probably had something running in the background... This machine is quite some working horse for me...
I have tested in now a few times, and came to a middle of 63 seconds with an nearly idle PC, with the timing percentage on the top

[oBFusCATed]

Which tab is this info coming from? Seems to be "bottom something".
Why is this API so slow for you?
Do you see matching times if you switch the "CPU Time" column from percentages to actual time (right click in the column)?

[oBFusCATed]

Can someone else report the time on his/her windows machine?

[Miguel Gimenez]

Loading the project with C::B in output31 directory:

Code Select Expand

ProjectManager::LoadProject took: 38.701 seconds.

The devel31 one:

Code Select Expand

ProjectManager::LoadProject took: 40.413 seconds.

When using VTune with the devel31 executable:

Bottom-up

Code Select Expand

Function / Call Stack CPU Time Module Function (Full) Source File Start Address
NtOpenFile 9.435s ntdll.dll NtOpenFile [Unknown] 0x4b2f1a20
NtQueryDirectoryFileEx 7.939s ntdll.dll NtQueryDirectoryFileEx [Unknown] 0x4b2f2b10
NtClose 5.195s ntdll.dll NtClose [Unknown] 0x4b2f17c0
NtQueryAttributesFile 4.033s ntdll.dll NtQueryAttributesFile [Unknown] 0x4b2f1ac0
LoadLibraryExW 3.281s KernelBase.dll LoadLibraryExW [Unknown] 0x1010f9f0
func@0x71a3ea30 2.285s COMCTL32.DLL func@0x71a3ea30 [Unknown] 0x71a3ea30
NtQueryInformationFile 2.113s ntdll.dll NtQueryInformationFile [Unknown] 0x4b2f17e0
NtCreateFile 1.624s ntdll.dll NtCreateFile [Unknown] 0x4b2f1c40
NtYieldExecution 1.600s ntdll.dll NtYieldExecution [Unknown] 0x4b2f1b50
NtUserMsgWaitForMultipleObjectsEx 1.454s win32u.dll NtUserMsgWaitForMultipleObjectsEx [Unknown] 0x10005860
wxFileName::GetPath 0.894s wxmsw314u_gcc_custom.dll wxFileName::GetPath(int, wxPathFormat) const [Unknown] 0x64e82900


Top-down Tree

Code Select Expand


Function Stack CPU Time: Total CPU Time: Self Module Function (Full) Source File Start Address
NtOpenFile 17.0% 9.435s ntdll.dll NtOpenFile [Unknown] 0x4b2f1a20
NtQueryDirectoryFileEx 14.3% 7.939s ntdll.dll NtQueryDirectoryFileEx [Unknown] 0x4b2f2b10
NtClose 9.4% 5.195s ntdll.dll NtClose [Unknown] 0x4b2f17c0
NtQueryAttributesFile 7.3% 4.033s ntdll.dll NtQueryAttributesFile [Unknown] 0x4b2f1ac0
LoadLibraryExW 5.9% 3.281s KernelBase.dll LoadLibraryExW [Unknown] 0x1010f9f0
func@0x71a3ea30 4.1% 2.285s COMCTL32.DLL func@0x71a3ea30 [Unknown] 0x71a3ea30
NtQueryInformationFile 3.8% 2.113s ntdll.dll NtQueryInformationFile [Unknown] 0x4b2f17e0
NtCreateFile 2.9% 1.624s ntdll.dll NtCreateFile [Unknown] 0x4b2f1c40
NtYieldExecution 2.9% 1.600s ntdll.dll NtYieldExecution [Unknown] 0x4b2f1b50
NtUserMsgWaitForMultipleObjectsEx 2.6% 1.454s win32u.dll NtUserMsgWaitForMultipleObjectsEx [Unknown] 0x10005860
wxFileName::GetPath 1.6% 0.894s wxmsw314u_gcc_custom.dll wxFileName::GetPath(int, wxPathFormat) const [Unknown] 0x64e82900


GetLongPathNameW only took 81 ms

C::B head
wxWidgets 3.1.4 32 bits
Windows 10 20H2 64 bits
i7-8700K @ 3.7 GHz
SSD disk

[oBFusCATed]

Top-down is mostly useless view. Callers and callees is a lot better. Ideally generating flame-graphs is the most presentable way, but unfortunately there is no native way to do it with vtune.

@Miguel Gimenez: Are you using debug (-Og/-O0) or release (-O2/-O3) build of wx?

[oBFusCATed]

Also if you want to have meaningful profiles you need to build everything with optimizations (-O2 or higher), symbols (-g), and no stripping (no -s or calls to strip).