NO

Author Topic: Handle CLI progress '\r' in an edit control  (Read 2717 times)

Offline tony74

  • Member
  • *
  • Posts: 53
Handle CLI progress '\r' in an edit control
« on: December 02, 2024, 11:08:31 PM »
I'm capturing the output of a CLI to an edit control, but this particular CLI app uses '\r' to show it's progress (I think... I didn't build it).

So on the console, the numbers stay in one place while the progress increments. But, the edit control doesn't handle '\r' like the console, so the progress numbers just run all over the edit window.

In all other respects, everything works fine, it's just when it gets to the last line, where it shows the progress percentage.

Is there any way to handle this that won't impact normal lines in an edit control? Like emulate the way the console handles '\r'?

My test rig:
Code: [Select]
#include <windows.h>
#include <stdio.h>

int main(int argc, char **argv)
{
 for(int i=0; i<=50; i++){

     printf("%03d%%\r ",i);
     Sleep(200);
 }

return 0;
}

My Run/Capture function:
Code: [Select]
#include <windows.h>
#include <windowsx.h>
#include <stdio.h>
#include <stdlib.h>
#include <direct.h>

#define RBYTES 4096
#define PIPERR -1
#define CPRERR -2

void appendtext(char *text);


//args: HWND of edit control,  char* Path of CLI program to run
//////////////////////////////////////////////////////////////////////
int runapp(HWND hread, char *cliapp)
{
    BOOL   ok               = TRUE;
    HANDLE hStdInPipeRead   = NULL;
    HANDLE hStdInPipeWrite  = NULL;
    HANDLE hStdOutPipeRead  = NULL;
    HANDLE hStdOutPipeWrite = NULL;
    DWORD  dwRead           =  0;
    DWORD  dwExitCode       =  0;
    char   lpCmdLine[1024]  = {0};
    LPSTR  lpAppName        = NULL;
    char   buf[RBYTES]      = {0};


    memset(buf,0,sizeof(buf));
    strcpy(lpCmdLine, cliapp);

    // Create two pipes.
    SECURITY_ATTRIBUTES sa = { sizeof(SECURITY_ATTRIBUTES), NULL, TRUE };
    ok = CreatePipe(&hStdInPipeRead,  &hStdInPipeWrite,  &sa, 0);
    if (ok == FALSE) return PIPERR;
    ok = CreatePipe(&hStdOutPipeRead, &hStdOutPipeWrite, &sa, 0);
    if (ok == FALSE) return PIPERR;

    // Create the CLI process.
    STARTUPINFO si = {0};
    si.cb          = sizeof(STARTUPINFO);
    si.dwFlags     = STARTF_USESTDHANDLES;
    si.hStdError   = hStdOutPipeWrite;
    si.hStdOutput  = hStdOutPipeWrite;
    si.hStdInput   = hStdInPipeRead;
    PROCESS_INFORMATION pi = {0};

    LPSECURITY_ATTRIBUTES lpProcessAttributes = NULL;
    LPSECURITY_ATTRIBUTES lpThreadAttribute   = NULL;
    BOOL   bInheritHandles    = TRUE;
    DWORD  dwCreationFlags    = CREATE_NO_WINDOW;
    LPVOID lpEnvironment      = NULL;
    LPSTR  lpCurrentDirectory = NULL;
   

    ok = CreateProcess (
            lpAppName,
            lpCmdLine,
            lpProcessAttributes,
            lpThreadAttribute,
            bInheritHandles,
            dwCreationFlags,
            lpEnvironment,
            lpCurrentDirectory,
            &si,
            &pi );

    if (ok == FALSE) return CPRERR;

    // Close pipes we don't need.
    CloseHandle(hStdOutPipeWrite);
    CloseHandle(hStdInPipeRead);


    // main loop for reading output from the cli app.
    ok     = ReadFile(hStdOutPipeRead, buf, RBYTES, &dwRead, NULL); //initial read
    while (ok == TRUE)                                              //read while TRUE 
    {
        buf[dwRead] = '\0';
        appendtext(buf);     
        ok = ReadFile(hStdOutPipeRead, buf, RBYTES, &dwRead, NULL); //next read
    }


    // Clean up and exit.
    CloseHandle(hStdOutPipeRead);
    CloseHandle(hStdInPipeWrite);
   
    GetExitCodeProcess(pi.hProcess, &dwExitCode);
    return dwExitCode;
}


// append text to edit control
//////////////////////////////////////////////////////////////////////
void appendtext(char *text)
{
    int len = GetWindowTextLength(hedit);
     
    SendMessage(hedit, EM_SETSEL    , (WPARAM)len, (LPARAM)len);
    SendMessage(hedit, EM_REPLACESEL,  TRUE      , (LPARAM)text);
}

// clear edit control window
//////////////////////////////////////////////////////////////////////
void editclear(HWND hedit)
{
    SetWindowText(hedit,0);
}

Offline HellOfMice

  • Member
  • *
  • Posts: 203
  • Never be pleased, always improve
Re: Handle CLI progress '\r' in an edit control
« Reply #1 on: December 03, 2024, 12:30:37 AM »
Hello,

I see 3 solutions:

1-To subclass the EDIT control
2-To use RICHEDIT
3-To use SetDlgItemText

Philippe RIO
--------------------------------
Kenavo

Offline John Z

  • Member
  • *
  • Posts: 920
Re: Handle CLI progress '\r' in an edit control
« Reply #2 on: December 03, 2024, 03:58:38 AM »
Hi Tony74,

As an initial thing you might write the capture to a file so tha you can examine the data and be sure of the format.
It would be easy to write the buffer each time.  The use a hex editor to view the binary.  You will know for sure the line ending.

So you get the line and if it has a \r set that to 0 to remove it, you know or can get the line length
so you know the character location to check easy if one line at a time.  if more than one line (which could be likely given you allocate a 4K buffer, then you'll parse the line into multiple line and process each before sending to the edit control.

When the line has a \r you will you send the text without the \r because it now is end-of-string \0
then after the line is sent to the edit box immediately move the cursor back to the start of the line with
SendMessage (HANDLE of Edit control,EM_SETSEL, 0, -1); // selects all text

then the next entry will overwrite the old value emulating a line return without carriage feed.  If there is text above the progress text you'lll replace the 0 with the character count before the progress text.

This will not impact other uses of the edit box because those won't be using the \r strip.

John Z

Other option is to select and delete the prior text before each subsequent text output to the control
« Last Edit: December 03, 2024, 10:23:12 AM by John Z »

Offline TimoVJL

  • Global Moderator
  • Member
  • *****
  • Posts: 2151
Re: Handle CLI progress '\r' in an edit control
« Reply #3 on: December 03, 2024, 02:05:06 PM »
perhaps a replace function is solution for that.
first get position in edit control and then in loop replace added text to new.
buffer parsing is needed for processing needed text.
« Last Edit: December 04, 2024, 06:30:02 AM by TimoVJL »
May the source be with you

Offline tony74

  • Member
  • *
  • Posts: 53
Re: Handle CLI progress '\r' in an edit control
« Reply #4 on: December 03, 2024, 08:14:00 PM »
@John Z  I think I'll give your suggestion a try. Maybe something like this to look at line endings:
Code: [Select]
// if EOL is '\r', return 1
int isCREOL(char *s)
{
  char *p;
  for( p=s; *p; p++ );
  --p;
  return(*p==0x0d?1:0);
}
And then follow-up with handling the carrot.

Luckily it's just the last line in this case, but it shouldn't matter if subsequent lines follow.
My only concern is reading the pipes fast enough, don't want to miss chars or, worse yet, deadlock because of more code slowing the read down.

Should be ok, thought. Thanks everybody for having a look, I'll post the project when everything comes together (or be back for more expert help if it doesn't...).

( This is a ytdlp front-end, I've used it for about a year but have just put up with letting ytdlp run in it's console, figured it was about time to incorporate it's output into the GUI ).

Thanks.



« Last Edit: December 03, 2024, 08:17:25 PM by tony74 »

Offline tony74

  • Member
  • *
  • Posts: 53
Re: Handle CLI progress '\r' in an edit control
« Reply #5 on: December 04, 2024, 02:12:50 AM »
Well, the good news is it didn't bog-down the pipes. But unless I got it all wrong, it didn't have any effect on the number updatating.

Code: [Select]

  ok = ReadFile(hStdOutPipeRead, buf, RBYTES, &dwRead, NULL);       //initial read
    while (ok == TRUE)                                              //read while TRUE 
    {
        buf[dwRead] = '\0';                                         //terminate current read

        if(isCREOL(buf))                                            //if '\r' EOL
        {
           buf[dwRead-1]='\0';                                      //dump '\r'

           appendtext ( buf );                                      //send txt to edit
           SendMessage( hedit, EM_SETSEL, cr.bytes, -1 );           //cursor to SOL 
                                                                    //Don't update position
           plog(buf, cr.bytes, dwRead, fp);                         //log to ConsoleLog.txt 
        }
        else 
        {   
           appendtext(buf);                                         //normal EOL
           cr.bytes += dwRead;                                      //update text position

           plog(buf, cr.bytes, dwRead, fp);                         //log to ConsoleLog.txt 
        }                                         
   
        ok = ReadFile(hStdOutPipeRead, buf, RBYTES, &dwRead, NULL); //next read
    }


The logs looked a little 'funny', but the edit control put everything in the right places, here's an excerpt:

Code: [Select]
buf:L  cr.bytes:1  dwRead:1

buf:ine one of test normal text
  cr.bytes:30  dwRead:29

buf:L  cr.bytes:31  dwRead:1

buf:ine two of test normal text
  cr.bytes:60  dwRead:29

buf:L  cr.bytes:61  dwRead:1

buf:ine three of test normal text
  cr.bytes:92  dwRead:31

buf:L  cr.bytes:93  dwRead:1

buf:ine four of test normal text
  cr.bytes:123  dwRead:30

buf:L  cr.bytes:124  dwRead:1

buf:ine five of test normal text
  cr.bytes:154  dwRead:30

buf:0  cr.bytes:155  dwRead:1

buf:00%   cr.bytes:155  dwRead:5

buf:0  cr.bytes:156  dwRead:1

buf:01%   cr.bytes:156  dwRead:5

buf:0  cr.bytes:157  dwRead:1

buf:02%   cr.bytes:157  dwRead:5

buf:0  cr.bytes:158  dwRead:1

buf:03%   cr.bytes:158  dwRead:5

buf:0  cr.bytes:159  dwRead:1

buf:04%   cr.bytes:159  dwRead:5

buf:0  cr.bytes:160  dwRead:1

buf:05%   cr.bytes:160  dwRead:5

buf:0  cr.bytes:161  dwRead:1

buf:06%   cr.bytes:161  dwRead:5

buf:0  cr.bytes:162  dwRead:1

buf:07%   cr.bytes:162  dwRead:5

buf:0  cr.bytes:163  dwRead:1

buf:08%   cr.bytes:163  dwRead:5

buf:0  cr.bytes:164  dwRead:1

buf:09%   cr.bytes:164  dwRead:5

buf:0  cr.bytes:165  dwRead:1

buf:10%   cr.bytes:165  dwRead:5

buf:0  cr.bytes:166  dwRead:1

buf:11%   cr.bytes:166  dwRead:5

buf:0  cr.bytes:167  dwRead:1

buf:12%   cr.bytes:167  dwRead:5

You notice it did separate the 'normal' text from the lines ending in '\r', so there's that.

I will say it 'looked good on paper', seemed like it should have worked, but at least in my attempts, it didn't. Could be I'm just not doing it right.
« Last Edit: December 04, 2024, 02:16:10 AM by tony74 »

Offline TimoVJL

  • Global Moderator
  • Member
  • *****
  • Posts: 2151
Re: Handle CLI progress '\r' in an edit control
« Reply #6 on: December 04, 2024, 07:53:56 AM »
This was close to work with concount.exe output
Code: [Select]
DWORD  nPos = 0;
while (ReadFile(hStdOutPipeRead, buf, RBYTES, &dwRead, NULL))
{
buf[dwRead] = '\0';
//OutputDebugString(buf);
//appendtext(hread, buf, nPos);
SendMessage(hread, EM_SETSEL    , (WPARAM)nPos, (LPARAM)-1);
SendMessage(hread, EM_REPLACESEL,  TRUE      , (LPARAM)buf);
//if (dwRead > 1 && buf[dwRead-1] != '\n') nPos--;
if (dwRead > 1 && buf[dwRead-1] == '\r') nPos--;
else nPos += dwRead;
}
Code: [Select]
void appendtext(HWND hedit, char *text, DWORD pos)
{
SendMessage(hedit, EM_SETSEL    , (WPARAM)pos, (LPARAM)-1);
SendMessage(hedit, EM_REPLACESEL,  TRUE      , (LPARAM)text);
}
DbgView output
Code: [Select]
[41228] L
[41228] ine one of test normal text
[41228] L
[41228] ine two of test normal text
[41228] L
[41228] ine three of test normal text
[41228] L
[41228] ine four of test normal text
[41228] L
[41228] ine five of test normal text
[41228] 0
[41228] 00% 
[41228] 0
[41228] 01% 
[41228] 0
[41228] 02% 
[41228] 0
[41228] 03% 
[41228] 0
[41228] 04% 
[41228] 0
[41228] 05% 
[41228] 0
[41228] 06% 
[41228] 0
[41228] 07% 
[41228] 0
[41228] 08% 
[41228] 0
[41228] 09% 
[41228] 0
[41228] 10% 
[41228] 0
[41228] 11% 
[41228] 0
[41228] 12% 
« Last Edit: December 05, 2024, 11:46:25 AM by TimoVJL »
May the source be with you

Offline John Z

  • Member
  • *
  • Posts: 920
Re: Handle CLI progress '\r' in an edit control
« Reply #7 on: December 04, 2024, 11:15:44 AM »
Well, the good news is it didn't bog-down the pipes. But unless I got it all wrong, it didn't have any effect on the number updatating.

You notice it did separate the 'normal' text from the lines ending in '\r', so there's that.

I will say it 'looked good on paper', seemed like it should have worked, but at least in my attempts, it didn't. Could be I'm just not doing it right.

So from the picture that was attached it shows that the percentage line is not being returned to the beginning of the line.  So it seems the 'cr.bytes' variable is being incremented when \r lines are found.  In the line
SendMessage( hedit, EM_SETSEL, cr.bytes, -1 ); 
so in this line try SendMessage( hedit, EM_SETSEL, cr.bytes-k, -1 );
k=0 until the first \r line is found thereafter k=4  this should backup the the next output to the start of that line.

John

Got your code and I'll look too.

Offline John Z

  • Member
  • *
  • Posts: 920
Re: Handle CLI progress '\r' in an edit control
« Reply #8 on: December 04, 2024, 11:53:38 AM »
Hi Tony74

Here is code to try, and resulting screen output.  Not optimized - but works -
For example I used a static variable so the second run after using 'clear' won't be right but
you can easily fix that.

John Z
« Last Edit: December 04, 2024, 11:57:16 AM by John Z »

Offline tony74

  • Member
  • *
  • Posts: 53
Re: Handle CLI progress '\r' in an edit control
« Reply #9 on: December 04, 2024, 09:32:47 PM »
@Timo: Thanks for all your work. I couldn't figure out why the buffer was doing that, it looked like an extra one-byte 'read'. It was all there, just split up.
That split-read is kinda critical.

@John Z: I'll run your code when I log off, looks pretty promising from the screen-grab. I guess you saw the 'error of my ways'?

While you guys were working on that, I reworked the concount.exe a little to more accurately mirror the actual ytdlp output. The reason I'm not using ytdlp itself is that it has to go online, grab a file, then output it's stats as it downloads. That's not conducive to repetitive testing. So I 'fake it' instead.

I threw together a function to grab the updating-substrings (ytdlp updates more than just percentage-complete) and send them to a control, which worked out ok. Just wanted to see if it would work.

The 'fly in the ointment' with any of this is not so much the edit control as it is the split buffer reads, maybe you've fixed that, but I'm still dealing with it (pipe buffer, not the edit control).

Anyway, John, I'll grab your code and get back to it. 

Thanks again Timo and John, I'll post back with  updates.



Offline tony74

  • Member
  • *
  • Posts: 53
Re: Handle CLI progress '\r' in an edit control
« Reply #10 on: December 04, 2024, 09:58:08 PM »
@Timo: I saw what you did there, good catch. Getting back to the ReadFile split-read issue, are the one-byte reads just an artifact of ReadFile or of the way the pipes are working? Would it be ok to combine the two reads in another buffer? Do you think this is consistent behavior or would it change with different input?

If this is consistent behavior, I could combine the reads before sending them to the edit control, or updating-substrings to a static control, etc.

Thanks again

Offline tony74

  • Member
  • *
  • Posts: 53
Re: Handle CLI progress '\r' in an edit control
« Reply #11 on: December 05, 2024, 03:06:01 AM »
@John Z: Dude, that was a master stroke, you nailed it!

I added variable-length line support, because the real-world apps that use this need it (including ytdlp) - Line 106 readapp.c.

I also added updating static views in the GUI - Line 108 readapp.c.

Only thing now is I'm still losing the first byte. I know you fixed this, but when I padded out K more than 1 (via cr.len), it all fell apart. keeping it at srlen(buf)+1 works, so far. Maybe you can have a look, you know what you're doing better than I do for edit controls.

But yeah, it's coming right along.



Offline John Z

  • Member
  • *
  • Posts: 920
Re: Handle CLI progress '\r' in an edit control
« Reply #12 on: December 05, 2024, 11:42:53 AM »
Hi Tony74,

Here is a revision.

Changes:
Static variable is removed, cr structure is global with added parameters, CLEAR now works, Last line looks ok.

Cheers,

John Z

Just one way to do it, not claiming the best  :)

Offline TimoVJL

  • Global Moderator
  • Member
  • *****
  • Posts: 2151
Re: Handle CLI progress '\r' in an edit control
« Reply #13 on: December 05, 2024, 03:14:57 PM »
In Windows 7, strange problems comes occasionally  :(
This was from console3
Code: [Select]
[youtube] Extracting URL: http://www.youtube.com/watch?v=FNGAvwzgKf4
[youtube] FNGAvwzgKf4: Downloading webpage
[youtube] FNGAvwzgKf4: Downloading ios player api JSON
[youtube] FNGAvwzgKf4: Downloading web creator player API JSON
[youtube] FNGAvwzgKf4: Downloading m3u8 information
[info] FNGAvwzgKf4: Downloading 1 format(s): 623+140
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total Fragments: 2388
[download] Destination: C:\yt-dlp\Video\New fornthite chapte 6 Bosses and Mythics!.f623.mp4

download]  006% of ~  1.2GiB at  923dowdownload]  030% of ~  6.0GiB at  923.57KiB/s ETA 13:30 (frag 31/2388)
May the source be with you

Offline John Z

  • Member
  • *
  • Posts: 920
Re: Handle CLI progress '\r' in an edit control
« Reply #14 on: December 05, 2024, 03:23:01 PM »
Hi TimoVJL

In Windows 7, strange problems comes occasionally  :(
This was from console3

I ran on test with console3_b on WIN 7 Pro. Not good.  Weird it ran ok once, but the next time badly.
Attached log file, you can see what is happening - - - EVERY read gets just 1 byte - it could be a problem with
the pipes.

I then ran on WIN 7 Home as well.  Same results.

In both cases the first run is good subsequent runs are impacted as the log shows.  This basically continues until the system is rebooted then it will work again for the first time.  After that nope.

I think this points to an issue with the pipes where the reboot clears out something - so that's probably where to investigate.  Too bad I don't have WIN 10 anymore.....

Tony74 - you might want to attach the code for the concount.exe portion - perhaps the issue originates there.

John Z
« Last Edit: December 05, 2024, 04:35:02 PM by John Z »