Code completion using LSP and clangd

[Pecan]

I'm building this plugin with the latest wx 3.2.0 release.

The source code I use is the latest rev: [r67]

It looks like I see "Requested token Not found" MessageBox when I use the mouse right context menu-> find declaration.

I switch back to an old rev [r66], which I build against wx 3.1.7, and I don't see this issue there.

Does any one notice the same issue?

Thanks.

EDIT:

I just build the revision 66 against wx 3.2.0, and it works OK.
So, my guess is some regression in this plugin's source code in revision 67.

That message box is still in rev 66 also. So some logic changed that is causing clangd to send back an empty response in rev 67.

Can you tell me how to re-create the situation with rev 67?

[ollydbg]

That message box is still in rev 66 also. So some logic changed that is causing clangd to send back an empty response in rev 67.

Can you tell me how to re-create the situation with rev 67?

I'm using C::B for my own big project.
Let me create a minimal project for testing(reproduce this bug)

[ollydbg]

I see one issue:

When I open the client or server log(which locates under the C:\Users\[myusername]\AppData\Local\Temp\)

I got to see something like:

Code Select Expand


09:59:08.833 SystemPath: F:\code\msys2-64\mingw64\bin;F:\code\msys2-64\mingw64;..\usr\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Windows\System32;C:\Windows;


However, I see the string "..\usr\bin" is wrong here.

Because in the C::B Menu->Settings->Compiler->Tool Chain executable->Addtional Paths, I have one string

Code Select Expand


$(TARGET_COMPILER_DIR)../usr/bin


So, it looks like the TARGET_COMPILER_DIR is not replaced by the "F:\code\msys2-64\mingw64".[/code]

[ollydbg]

That message box is still in rev 66 also. So some logic changed that is causing clangd to send back an empty response in rev 67.

Can you tell me how to re-create the situation with rev 67?

I'm using C::B for my own big project.
Let me create a minimal project for testing(reproduce this bug)

Here is my research on the "find declaration failure".

For some unknown reasons, the find declaration gives empty result, see the log below:

Code Select Expand


11:30:17.039 <<< GoToDeclaration:
file:///F:/MyProject/JsonRead.cpp,line[41], char[30]

11:30:17.040 <<< Content-Length: 211



{"id":"textDocument/declaration","jsonrpc":"2.0","method":"textDocument/declaration","params":{"position":{"character":30,"line":41},"textDocument":{"uri":"file:///F:/MyProject/JsonRead.cpp"}}}

11:30:17.155 >>> readJson() len:61:
{"id":"textDocument/declaration","jsonrpc":"2.0","result":[]}



While for another place, it works OK, see the log below:

Code Select Expand



11:30:30.454 <<< GoToDeclaration:
file:///F:/MyProject/ImagePanel.cpp,line[207], char[21]

11:30:30.455 <<< Content-Length: 214



{"id":"textDocument/declaration","jsonrpc":"2.0","method":"textDocument/declaration","params":{"position":{"character":21,"line":207},"textDocument":{"uri":"file:///F:/MyProject/ImagePanel.cpp"}}}

11:30:30.659 >>> readJson() len:199:
{"id":"textDocument/declaration","jsonrpc":"2.0","result":[{"range":{"end":{"character":14,"line":35},"start":{"character":6,"line":35}},"uri":"file:///F:/MyProject/ImagePanel.h"}]}



So, it looks like the problem is in clangd side.

I will do more research on this issue, and try to make a minimal sample code.

Strange that clangd does not show the errors when parsing:

Code Select Expand

LSP diagnostics: JsonRead.cpp|:|----Time: 11:30:08.905---- (0 diagnostics)|

[ollydbg]

I stripped down to a minimal code snippet: (you should save the code to UTF8 format)

Code Select Expand


int abc;


int main()
{
    abc = 3;
}


#include <string>


std::string Utf8ToGbk(const std::string& strUtf8);

std::string Utf8ToGbk(const std::string& strUtf8)
{
    // 上面的函数
    // return unicodeString.ToStdString(); // 默认使用当前操作系统的编码格式，Windows通常为GB2312
    return strUtf8;
}


You can see that if you right click on the "strUtf8", and find declaration gives empty result. The same as the "abc".

But if you remove the line:

Code Select Expand

// 上面的函数

Then, everything works expected.

Can you guys reproduce this bug?
Thanks.

BTW: I have some local patches to enable the the utf8 handling, but I'm not sure they are related.

Code Select Expand


-------------------- clangd_client/src/LSPclient/client.cpp --------------------
index 26ae994..fa1d7ca 100644
@@ -1162,7 +1162,7 @@ bool ProcessLanguageClient::DoValidateUTF8data(std::string& strdata)
             if (not i18n) //if not internationalization show U(<codepoint>)
             {
                 // With internationalization the wxUniChar gets an assert in wxString::Format
-                wxUniChar uniChar(invChar);
+                wxUniChar uniChar((unsigned int)invChar);
                 msg += wxString::Format("position(%d), hex(%02hhX), U(%x), \'%s\'", invloc, (unsigned int)invChar, (int)uniChar.GetValue(), invStr );
             }
             else


and

Code Select Expand


-------------------- clangd_client/src/LSPclient/client.cpp --------------------
index df36437..66bfad8 100644
@@ -2629,6 +2629,7 @@ void ProcessLanguageClient::LSP_DidChange(cbEditor* pEd)
     }

     didChangeEvent.text = edText;
+    didChangeEvent.text = edText.ToUTF8(); //Trying to find bad utf8 problem
     std::vector<TextDocumentContentChangeEvent> tdcce{didChangeEvent};
     DocumentUri docuri = DocumentUri(fileURI.c_str());
     // **debugging**


[Pecan]

I see one issue:

When I open the client or server log(which locates under the C:\Users\[myusername]\AppData\Local\Temp\)

I got to see something like:

Code Select Expand


09:59:08.833 SystemPath: F:\code\msys2-64\mingw64\bin;F:\code\msys2-64\mingw64;..\usr\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Windows\System32;C:\Windows;


However, I see the string "..\usr\bin" is wrong here.

Because in the C::B Menu->Settings->Compiler->Tool Chain executable->Addtional Paths, I have one string

Code Select Expand


$(TARGET_COMPILER_DIR)../usr/bin


So, it looks like the TARGET_COMPILER_DIR is not replaced by the "F:\code\msys2-64\mingw64".[/code]

I don't understand the point here.
That line in the log is just info produced by client.cpp line 372 and line 394. It's the system path as seen by wxGetEnv() and is just for information.

Code Select Expand


            wxString envPath;
            wxGetEnv("PATH", &envPath);
            logLine = "SystemPath: " + envPath;
            writeClientLog(logLine);


The code to find the path to clangd.exe is in Locate_ClangdDir() invoked at client.cpp line 195.
Search for "// Locate folder for Clangd" (without quotes).

[Pecan]

I stripped down to a minimal code snippet: (you should save the code to UTF8 format)

Code Select Expand


int abc;


int main()
{
    abc = 3;
}


#include <string>


std::string Utf8ToGbk(const std::string& strUtf8);

std::string Utf8ToGbk(const std::string& strUtf8)
{
    // 上面的函数
    // return unicodeString.ToStdString(); // 默认使用当前操作系统的编码格式，Windows通常为GB2312
    return strUtf8;
}


You can see that if you right click on the "strUtf8", and find declaration gives empty result. The same as the "abc".

But if you remove the line:

Code Select Expand

// 上面的函数

Then, everything works expected.

Can you guys reproduce this bug?
Thanks.
...snip...

I am unable to re-produce this error.
Find declaration of strUtf8 jumps to the strUtf8 parameter.
Here's the log entry.

Code Select Expand


10:37:36.951 <<< GoToDeclaration:
file:///C:/temp/OllyDbgGoToDeclConsole/main.cpp,line[27], char[14]
10:37:36.951 <<< Content-Length: 207

{"id":"textDocument/declaration","jsonrpc":"2.0","method":"textDocument/declaration","params":{"position":{"character":14,"line":27},"textDocument":{"uri":"file:///C:/temp/OllyDbgGoToDeclConsole/main.cpp"}}}

10:37:37.041 >>> readJson() len:196:
{"id":"textDocument/declaration","jsonrpc":"2.0","result":[{"range":{"end":{"character":48,"line":23},"start":{"character":41,"line":23}},"uri":"file:///C:/temp/OllyDbgGoToDeclConsole/main.cpp"}]}



Attached: a .zip of a .wmv recording the sucessful behavior.

[ollydbg]

I stripped down to a minimal code snippet: (you should save the code to UTF8 format)

Code Select Expand


int abc;


int main()
{
    abc = 3;
}


#include <string>


std::string Utf8ToGbk(const std::string& strUtf8);

std::string Utf8ToGbk(const std::string& strUtf8)
{
    // 上面的函数
    // return unicodeString.ToStdString(); // 默认使用当前操作系统的编码格式，Windows通常为GB2312
    return strUtf8;
}


You can see that if you right click on the "strUtf8", and find declaration gives empty result. The same as the "abc".

But if you remove the line:

Code Select Expand

// 上面的函数

Then, everything works expected.

Can you guys reproduce this bug?
Thanks.
...snip...

I am unable to re-produce this error.
Find declaration of strUtf8 jumps to the strUtf8 parameter.
Here's the log entry.

Code Select Expand


10:37:36.951 <<< GoToDeclaration:
file:///C:/temp/OllyDbgGoToDeclConsole/main.cpp,line[27], char[14]
10:37:36.951 <<< Content-Length: 207

{"id":"textDocument/declaration","jsonrpc":"2.0","method":"textDocument/declaration","params":{"position":{"character":14,"line":27},"textDocument":{"uri":"file:///C:/temp/OllyDbgGoToDeclConsole/main.cpp"}}}

10:37:37.041 >>> readJson() len:196:
{"id":"textDocument/declaration","jsonrpc":"2.0","result":[{"range":{"end":{"character":48,"line":23},"start":{"character":41,"line":23}},"uri":"file:///C:/temp/OllyDbgGoToDeclConsole/main.cpp"}]}



Attached: a .zip of a .wmv recording the sucessful behavior.

Hi, Pecan, thanks for the test and the screen cast video. I will check what's the problem here.

BTW: do you use the wx 3.2 library? Or the wx 3.1.7  for building the codeblocks and the clangd_client.

[Pecan]

@OllyDbg

I still use wx3.1.5 and the compiler used by the nightly.
I always compile, debug and publish with the wx that's used with the nightly.

I've noticed that clangd can behave peculiarly if a request is made to it and a previous error exists. When I correct any previous error, the subsequent clangd requests tend to work.

See if fixing the main functions "return int" fixes the problem. Just a guess...



Thanks for all your testing.

[ollydbg]

@OllyDbg

I still use wx3.1.5 and the compiler used by the nightly.
I always compile, debug and publish with the wx that's used with the nightly.

I've noticed that clangd can behave peculiarly if a request is made to it and a previous error exists. When I correct any previous error, the subsequent clangd requests tend to work.

See if fixing the main functions "return int" fixes the problem. Just a guess...



Thanks for all your testing.

Thanks for the reply and help.

I just added a "return 0;" statement in the main function, but the same issue.

I have rebuild the whole C::B and clangd_client plugin with the wx 3.2.0 today. But still the same issue.

I will try to rebuild the C::B and clangd_client plugin with wx 3.1.7 later today to see whether it is the wx related issue.

[ollydbg]

I will try to rebuild the C::B and clangd_client plugin with wx 3.1.7 later today to see whether it is the wx related issue.

The issue persist with wx 3.1.7. So, it looks like this is not a wx related issue.

Can someone share a C::B + clangd_client plugin, so that I can tested in my PC? Thanks.

[ollydbg]

I have upload a screen cast mp4 file in zip file about this issue.

See here:

go declaration got error Issue 1 asmwarrior/clangd_client_bug_2022

Because this forum does not allow a file larger than 512K, I upload to github issue, clangd_go_decl_bug.zip

[ollydbg]

Hi, Pecan, I did some test, I tred to use different clangd from winlibs, I tried to used to use the online visual studio code.

It looks like clangd does not have this issue. so I guess the issue is inside the clangd_plugin code. clangd_plugin rev66 about one months ago works OK.

I compared rev67 and rev66, especially the log file. By comparing the logs from clangd_plugin rev66 and rev67, I noticed some difference.

It looks like the textDocument/didOpen method is different. In rev67, it just send an empty string.


Here is the log of rev66, it looks good. (Note I cut down the source file path, because it was a bit long)

Code Select Expand


20:25:11.290 <<< Content-Length: 614



{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"languageId":"cpp","text":"\r\n\r\n\r\n\r\nint abc;\r\n\r\n\r\nint main()\r\n{\r\n    abc = 3;\r\n    return 0;\r\n}\r\n\r\n\r\n#include <string>\r\n\r\n\r\nstd::string Utf8ToGbk(const std::string& strUtf8);\r\n\r\nstd::string Utf8ToGbk(const std::string& strUtf8)\r\n{\r\n    // 上面的函数\r\n    // return unicodeString.ToStdString(); 默认使用当前操作系统的编码格式，Windows通常为GB2312\r\n    return strUtf8;\r\n}\r\n\r\n\r\n\r\n","uri":"file:///D:/project/test5-readtext/a.cpp","version":0}}}




In rev67, the log looks wrong, because an empty string is passed.

Code Select Expand


20:27:05.074 <<< Content-Length: 185



{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"languageId":"cpp","text":"","uri":"file:///D:/test5-readtext/a.cpp","version":0}}}



You can see, the "text":"". So, it just send the empty string to the clangd?

[ollydbg]

OK, it looks like the change in rev67 is here:

Code Select Expand


@@ -1664,9 +1683,13 @@ bool ProcessLanguageClient::LSP_DidOpen(cbEditor* pcbEd)
     // save current length of the file
     m_FileLinesHistory[pcbEd] = pCntl->GetLineCount();

-    wxString strText = pCntl->GetText();
-    //-const char* pText = strText.mb_str();        //works //(ph 2022/01/17)
-    const char* pText = strText.ToUTF8();           //ollydbg  220115 did not solve illegal utf8char
+    #if wxCHECK_VERSION(3,1,5) //3.1.5 or higher
+    wxString strText = pCntl->GetText().utf8_string(); //solves most illegal utf8chars
+    #else
+    //const char* pText = strText.mb_str();         //works //(ph 2022/01/17)
+    wxString strText = pCntl->GetText().ToUTF8();  //ollydbg  220115 did not solve illegal utf8chars
+    #endif
+    const char* pText = strText.c_str();


This code change looks wrong (cause my issue) here.

[ollydbg]

OK, it looks like the change in rev67 is here:

Code Select Expand


@@ -1664,9 +1683,13 @@ bool ProcessLanguageClient::LSP_DidOpen(cbEditor* pcbEd)
     // save current length of the file
     m_FileLinesHistory[pcbEd] = pCntl->GetLineCount();

-    wxString strText = pCntl->GetText();
-    //-const char* pText = strText.mb_str();        //works //(ph 2022/01/17)
-    const char* pText = strText.ToUTF8();           //ollydbg  220115 did not solve illegal utf8char
+    #if wxCHECK_VERSION(3,1,5) //3.1.5 or higher
+    wxString strText = pCntl->GetText().utf8_string(); //solves most illegal utf8chars
+    #else
+    //const char* pText = strText.mb_str();         //works //(ph 2022/01/17)
+    wxString strText = pCntl->GetText().ToUTF8();  //ollydbg  220115 did not solve illegal utf8chars
+    #endif
+    const char* pText = strText.c_str();


This code change looks wrong (cause my issue) here.

I just revert this changes in rev67, and rebuild the clangd_client plugin, and my issue is gone!