NO

Recent Posts

Pages: [1] 2 3 ... 10
1
Work in progress / Re: zlib 1.2.8 source converted for __stdcall
« Last post by WiiLF23 on Yesterday at 10:22:27 PM »
Keep in mind, zlib1 (note the 1 at the end of the filename) is a mainstream effort from the authors to maintain a corrected compilation of the library, and its codebase in the wild. The documentation explains this very throughly, as well as the mistake that was overlooked prior to the mass release of zlib (without the 1) at the original release date. The naming convention is important. What you name the file after doesn’t mean much, as long as it is a corrected build (away from regular zlib.xx).

People need to know this, who are aware of the libs history.

So, I recommend you indicate this as to clarify it is the corrected code.
2
Windows questions / Re: WIN32 ComboBox control
« Last post by WiiLF23 on Yesterday at 10:09:53 PM »
The combobox control has a buggy history, and often developers will attempt to alter the controls appearance but will find that the win32 shell team implemented a group of pixels to draw the glyph, but it is all tied together so you have to either draw it all yourself or leave it alone in respect to its default style.

There are also undocumented API messages, which stump many. Not my favourite control to work with, and often I just use a Edit control and I draw a non-client area region and paint my own dropdown with highlight events (WM_PAINT, WM_MOUSEMOVE) with a Hittest on the rects region, and I go from there in respect to the label items.

I probably won’t ever use a traditional combobox anytime soon lol
3
Expert questions / Re: TikToken
« Last post by WiiLF23 on Yesterday at 09:52:22 PM »
I would love to convert this, just to stick it to Python.

I’m not a fan of it, however given the use of vectors and a range of “modules”, I would just grab the bindings and cave in.

A pure rewrite would utilize AVX/AVX2 or the SSE instructions (with CPU vendor detection of course). So that alone is worth considering if desiring a scratch implementation in C. Pelles has vector support, you will find this in the project settings.

Basically, you would need the API documentation and the rest is up to the C programming to align with the OpenAI API documentation.

It looks like some work outside of the Python C bindings.
4
Expert questions / Re: TikToken
« Last post by HellOfMice on Yesterday at 12:48:26 PM »
For OpenAI a token is a group of three or four characters. The solution I have made is to divide the length of each word by three and add 1 word length is greather than three.


OpenAI tokenizer https://platform.openai.com/tokenizer

5
Expert questions / Re: TikToken
« Last post by HellOfMice on Yesterday at 12:36:27 PM »
Yes. It computes the tokens.
6
Expert questions / Re: TikToken
« Last post by Vortex on May 14, 2024, 10:11:23 PM »
Hello,

Are you referring to this project?

https://github.com/openai/tiktoken
7
Expert questions / Re: TikToken
« Last post by frankie on May 14, 2024, 12:27:12 PM »
Python official docs.
The way around this, or this.
8
Expert questions / TikToken
« Last post by HellOfMice on May 13, 2024, 06:34:21 AM »
Hello,


Is it possible to interface TikToken with Pelle's C. I have searched but did not find anything.
TikToken is written in python.
In the case it is possible, how to do it, please.


Thank you for your help
9
Assembly discussions / Binary resource data with INCBIN
« Last post by Vortex on May 08, 2024, 09:42:44 PM »
Hello,

Here is an example to read binary data from a compiled resource script with INCBIN :

Code: [Select]
.386
.model flat,stdcall
option casemap:none

include Dialog.inc

includelib  \PellesC\lib\Win\kernel32.lib
includelib  \PellesC\lib\Win\user32.lib

DlgProc PROTO :DWORD,:DWORD,:DWORD,:DWORD

.data

Resource:
INCBIN Rsrc.res,80
msg db 'Dialog box with menu',0

title1 db 'Hello!',0

IDM_HELLO   equ 11
IDM_EXIT    equ 12
IDC_HELLO   equ 110
IDC_EXIT    equ 120
IDC_EDIT    equ 3000

.code

start:

    invoke  GetModuleHandle,NULL
    invoke  DialogBoxIndirectParam,eax,ADDR Resource,\
            NULL,ADDR DlgProc,NULL
    invoke  ExitProcess,eax

DlgProc PROC hWnd:HWND,uMsg:UINT,wParam:WPARAM,lParam:LPARAM

    .IF uMsg==WM_INITDIALOG
       
        invoke  LoadIcon,NULL,IDI_APPLICATION
        invoke  SendMessage,hWnd,WM_SETICON,ICON_SMALL,eax
        lea     eax,[Resource+0A0h]
        invoke  LoadMenuIndirect,eax
        invoke  SetMenu,hWnd,eax
       
    .ELSEIF uMsg==WM_CLOSE
   
        invoke SendMessage,hWnd,WM_COMMAND,IDM_EXIT,0

    .ELSEIF uMsg==WM_COMMAND

        mov eax,wParam
       
        .IF lParam==0
       
            .IF ax==IDM_HELLO
           
                invoke  MessageBox,NULL,ADDR msg,ADDR title1,MB_OK
               
            .ELSEIF ax==IDM_EXIT
           
                invoke  EndDialog, hWnd,NULL
            .ENDIF
           
        .ELSE
       
            mov edx,wParam
            shr edx,16
           
            .IF dx==BN_CLICKED
                               
                .IF ax==IDC_HELLO
               
                    invoke  SendMessage,hWnd,WM_COMMAND,IDM_HELLO,0
                   
                .ELSEIF ax==IDC_EXIT
               
                    invoke  EndDialog,hWnd,NULL
                .ENDIF
               
            .ENDIF
           
        .ENDIF
       
    .ELSE
   
        mov eax,FALSE
        ret
       
    .ENDIF
   
        mov eax,TRUE
        ret
       
DlgProc ENDP

END start
10
Bug reports / MOVED: IDE TreeView limited files
« Last post by frankie on May 08, 2024, 09:35:14 AM »
This topic has been moved to General discussions.

https://forum.pellesc.de/index.php?topic=11184.0
Pages: [1] 2 3 ... 10