Multiline Search & Replace

rickg22 · November 08, 2009, 03:03:32 AM

Hi guys! I'm modifying my fresh SVN copy of Code::Blocks to support Multi-line search and replace (I'm needing it to refactor a 400-files ASP project, and we need to save as much time as possible).

Currently I've only been able to modify the replace dialog (the search dialog is not as critical), and only the XRC (but I'm about to start with the C++ code). See the attachment for a screenshot.

What do you think?

[attachment deleted by admin]

rickg22 · November 08, 2009, 05:58:01 AM

I've managed to make the changes in replacedlg.cpp.

In some files, replacing works, but in others, it doesn't. I've realized it's a line-endings issue. The wxTextCtrl interprets a line feed as LF, while the files can vary their endings in LF, CR or CRLF.

To solve this, I need to make the following steps:

If the search string does NOT contains a CR or LF, search normally.

If it does,
1) Get the EOL mode from C::B.
2) Convert the "find" string to this EOL mode.
3) Convert the file to be searched (the in-memory buffer, i mean) to this EOL mode so line endings are consistent. We're going to replace in files, anyway.
4) Convert the "replace" string to this EOL mode.
5) Find and replace the string as we'd do with any other strings.

This should be done in EditorManager::Replace(cbStyledTextCtrl* control, cbFindReplaceData* data).

I'll keep posting my progress.

BTW, regex search hasn't been tested yet. Wish me luck!

blueshake · November 08, 2009, 07:42:14 AM

Seems cool,waiting for your good news.

rickg22 · November 08, 2009, 07:45:30 AM

Progress update.

Sun, Nov 8, 2009.

The Multi-line Search and replace algorithm has been finished and tested successfully, in an unsaved file, with default EOL mode set to CRLF.

The EOL conversion of the search and replace expressions is done inside replacedlg.cpp. Inside EditorManager::Replace and EditorManager::ReplaceInFiles we check the search expression for LFs and CRs. If there's a match, the whole file is converted to the current EOL mode to avoid false positives/negatives. Then the search takes place normally.

Search & Replace in a single file works, for both normal and regex searches.

Extended Regular Expressions is enabled.

Before:

Code Select


<html>
<body>
hola mundo!
</body>
</html>

Search Expression:

Code Select


<html>
<body>
(.*)
</body>
</html>

Replace expression:

Code Select


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Hello world!</title>
</head>
<body>
\1
</body>
</html>

After:

Code Select


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Hello world!</title>
</head>
<body>
hola mundo!
</body>
</html>

There are still some quirks in the UI that need to be worked on.

Multi-line S&R in files has not been tested yet.

Will continue to report tomorrow.
Good night!

mariocup · November 08, 2009, 10:49:17 AM

seems very very cool.

Jenna · November 08, 2009, 11:06:09 AM

Quote from: mariocup on November 08, 2009, 10:49:17 AM
seems very very cool.

Absolutely correct.

Does it also work with mixed line-edings ?

rickg22 · November 08, 2009, 10:58:12 PM

Quote from: jens on November 08, 2009, 11:06:09 AM
Quote from: mariocup on November 08, 2009, 10:49:17 AM
seems very very cool.

Absolutely correct.

Does it also work with mixed line-edings ?

It didn't work with mixed line-endings at first. I needed to convert the line endings of both the search/replace expressions AND of files. What I do is checking if the search/replace expressions contain a LF or CR. In that case, we convert the current file to the configured EOL mode so line endings aren't mixed anymore. We're going to search/replace, so it's not a problem if the file is modified. Of course, this is done INSIDE the undo action, so that if the match isn't found, the file is undo'ed. The undoing is already done with existing code inside EditorManager, i only had to move the begin-undo-action before the line conversion.

Here's the patch for EditorManager:

Code Select


int EditorManager::Replace(cbStyledTextCtrl* control, cbFindReplaceData* data)
{
    if (!control || !data)
        return -1;

    if (control->GetReadOnly())
    {
        cbMessageBox(_("This file is read-only.\nReplacing in a read-only file is not possible."),
                     _("Warning"), wxICON_EXCLAMATION);
        return -1;
    }

    bool AdvRegex=false;
    int replacecount=0;
    int foundcount=0;
    int flags = 0;

    control->BeginUndoAction(); // The undo is set at this point in case we need to convert the EOLs.

    if((data->findText.Find(_T("\n")) != wxNOT_FOUND) || (data->findText.Find(_T("\r")) != wxNOT_FOUND))
    {
        // If it's a multi-line S&R, We need to convert the EOLs, or the find may be inaccurate.
        // This will slow down processing as the whole file needs to be preprocessed, but it's a small price
        // compared to searching and replacing multiple lines by hand.
        static const int default_eol = platform::windows ? wxSCI_EOL_CRLF : wxSCI_EOL_LF;
        int eolMode = Manager::Get()->GetConfigManager(_T("editor"))->ReadInt(_T("/eol/eolmode"), default_eol);
        control->SetEOLMode(eolMode);
        control->ConvertEOLs(eolMode);
    }
    CalculateFindReplaceStartEnd(control, data);
    // ... the rest of the code is kept as it was.

For find in files, it's a similar code.

This check is done only for the find string, since having a multiline replace string isn't a problem. It's the find string that gives us all the problems.

Algorithm aside, my real problem right now is getting the ctrl-tab key to work. I don't want the tabs to change between the single-line and multiple-line tabs, but between the Replace and Replace in files. How do I disable a wxNotebook control from accepting ctrl-tab keys?

Just in case, I'm going to change the wxNotebook for a wxChoicebook. I hope ctrl-tab isn't affected this way. (And I think it looks prettier and less bulky).

rickg22 · November 09, 2009, 01:50:39 AM

Ugh, this sucks!

wxNotebook interferes with my tabs and I find no way to make it ignore keys.
wxChoicebook completely deletes the tabs.

So I've decided to use a pair of radio buttons and emulate a wxNotebook by hiding and showing panels. This way the ctrl-tab isn't messed with, and the radio buttons are what I wanted in the first place.
It may take me a couple of hours to make it work.

Wish me luck!

rickg22 · November 09, 2009, 07:31:25 AM

:x ARGH! It took me four friggin' hours to get the dialog to work. It's incredible that the GUI was much more difficult to work on than the algorithm itself.

Okay, progress report:

The UI is done. Yay!

Replace in files hangs C::B. Not good. I guess I introduced a bug in there, or I didn't think well the replace in files part. It might take me another hour to get it to work.

Wish me luck!

rickg22 · November 09, 2009, 07:56:06 AM

Well well well. What do you know? The original XRC file had the "Project files" and "Open files" scope swapped. No wonder it was hanging. It was searching on ALL the files. I'll keep debugging.

rickg22 · November 09, 2009, 08:40:05 AM

It seems I've found another "feature". The confirmation dialog is only shown when you select "all". I wonder if this is wise, due to the fact that finding the *FIRST* match might take a while when doing multiline S&R.

But taking that aside, I've managed to complete the algorithm succesfully.

There's a minor problem: Converting the line endings of ALL files to the search expression is completely overkill. It's much better to just convert the search and replace expressions to match the files' EOL mode.

This is easy. Unfortunately, files with mixed line endings might get some matches skipped. I'm afraid there's no easy way around this.

rickg22 · November 09, 2009, 08:54:28 AM

FINISHED!

I've just mailed Jens the patch file for the multiline S&R.

Note that the patch only affects Find-and-Replace (CTRL-R), not Find (CTRL-F).

MortenMacFly · November 09, 2009, 09:43:52 AM

Quote from: rickg22 on November 09, 2009, 07:56:06 AM
The original XRC file had the "Project files" and "Open files" scope swapped.

Notice that this was inconsistent. That was modified in a patch lately. So hopefully you left the XRC file untouched and corrected the code (although I was quite sure that I did it already in SVN...?!).

rickg22 · November 09, 2009, 02:20:26 PM

Quote from: MortenMacFly on November 09, 2009, 09:43:52 AM
Quote from: rickg22 on November 09, 2009, 07:56:06 AM
The original XRC file had the "Project files" and "Open files" scope swapped.
Notice that this was inconsistent. That was modified in a patch lately. So hopefully you left the XRC file untouched and corrected the code (although I was quite sure that I did it already in SVN...?!).

At first I thought that I had swapped those lines accidentally while editing the XRC, so I diffed against HEAD. The version in the repository was unfixed. And to fix it... um... no, it was the XRC that I modified. I added like a ton of lines, why not just swap two lines near the end?

Sorry.

Okay, I modified my local copy to swap again the XRC, and now I fixed the editormanager code to reflect the changes.
Hey, guess what. I still have the SVN authentication. Should I commit to head now?

MortenMacFly · November 10, 2009, 06:28:23 AM

Quote from: rickg22 on November 09, 2009, 02:20:26 PM
Should I commit to head now?

Not head, but the scintilla branch would be fine. :-)
(Unless Jens has serious doubts.)

Code::Blocks Forums

News:

Multiline Search & Replace

rickg22

rickg22

blueshake

rickg22

mariocup

Jenna

rickg22

rickg22

rickg22

rickg22

rickg22

rickg22

MortenMacFly

rickg22

MortenMacFly