pointers to callback functions in DLL

[mennowo]

Hello,

currently I am building a DLL (native win32) that reads data from a variable number of other programs, and can display this data in a single window. Because there is no (easy) way of sending pointers to data into a DLL, I use callback funtions.

When a program loads the DLL, it stores information about itself in a structure, which is located in an array in shared memory inside the DLL (using #pragma data_seg). The next program loading the DLL will use the next slot in the array, etc. The window is only created once, on DLL_PROCESS_ATTACH. Among the information stored, are two pointers to functions, that exist inside the application loading the DLL. They can be called from the DLL and will return small bits of data to be used for displaying in the DLL window.

All this works fine when the DLL is loaded by one application. A second instance of the same application also works fine: I have a dialogue where I can choose between applications, and then choose which data I wanna see, and the data is displayed.

However, things go wrong when trying this with two seperate applications loading the same DLL. They load fine, and run fine, right unto the moment when the DLL uses a callback function of the second application. Then I get an access violation. This surprises me, cause the debugger shows that there is a pointer loaded in the pointer-to-function variable, and the function arguments are fine and should not be out of bounds in the host application... I find it hard to know where to look now...

So I am wondering: is there something I am missing, or am I trying something not supposed to be possible? Any ideas welcome!

Menno

[frankie]

To be honest I've never tryied this before, but I can guess that while the DLL is loaded once in phisycal memory its addresses are mapped in the same process space as the application calling it. Using two instances of same application at least share same virtual address so incidentally the access is correct. But when you map DLL in a different application process space the virtual addresses could diverge causing an access violation.
Could you post a minimal project showing it?
For interprocess communication you can use memory files or pipes.
Eventually you want look here

[mennowo]

I think you are right. I did read that page, but thought placing all data inside a shared array of structures could solve the issues. The page does however specifically mention pointers to functions:

It is possible that the DLL itself could get loaded at a different address in the virtual address spaces of each process. It is not safe to have pointers to functions in the DLL or to other shared variables.

I guess the second application to load the DLL, places its pointers in a different address space? I'm actually not sure what all that mean, and what virtual address means :-). I thought that since I used a large array of structures, which is shared, the loaded pointers to functions, from whichever app they are loaded, would be accessible anyhow for the DLL winproc...

Below some code snippets to show the idea.

First is the shared data segment:

Code: [Select]

#pragma data_seg ("shared")
TCHAR szDllName[] = TEXT("testsharedmem");
static HWND hWndMMir;
// the array has to be real, not pointers to structures (not allowed)
static CONTROLLERSTRUCT controllers[MAX_CONTROLLERS] = {0};
#pragma data_seg ()
#pragma comment(linker, "/SECTION:shared,RWS")
Where CONTROLLERSTRUCT is the structure containing the data from the applications that will load the DLL:

Code: [Select]

typedef short (CALLBACK * GIVESTATECB) (int, int, int);
typedef char * (CALLBACK * GIVEELEMSTR) (int, int);
struct controllerstruct{
short id;
char isattached;
char name[16];
GIVEELEMSTR pfnGiveElemString;
GIVESTATECB pfnGiveElemState;
// this structure contains info for objects on the screen that will display info from the apps
CTRACKERSTRUCT ctracker[MAX_TRACKERS];
HBRUSH std_background;
};
typedef struct controllerstruct CONTROLLERSTRUCT;
The DLLmain function:

Code: [Select]

// main functie
int WINAPI DllMain(HINSTANCE hInstance, DWORD fwdReason, PVOID pvReserved)
{
switch(fwdReason)
{
case DLL_PROCESS_ATTACH:
if(init) // do this once
{
hInstMMir = hInstance;
// Threadproc will create a single window and start the message loop
CreateThread(0, (SIZE_T)NULL, ThreadProc, (LPVOID)"TestDLLWin", (DWORD)NULL, (LPDWORD)NULL);
init = 0;
}
break;
case DLL_THREAD_ATTACH:
break;
case DLL_THREAD_DETACH:
break;
case DLL_PROCESS_DETACH:
// method to clean up when an app "disconnects" from the DLL
RemoveController();
break;
}
return TRUE;
}
In the application loading the DLL I have the following:

Code: [Select]

char * CALLBACK GiveElemString(int type, int elem);
short CALLBACK GiveElemState(int type, int elem, int state);
char * CALLBACK GiveCCOLString(int type, int elem)
{
switch(type)
{
case Fase:
return (char *)FC_code[elem];
break;
// etc...
}
}
short CALLBACK GiveElemState(int type, int elem, int state)
{
switch(type)
{
case Fase:
if(elem >= FCMAX) return -1;
switch(state)
{
case fcstat_R: return(R[elem]);
break;
// etc...
}
//etc...
}
}
Upon loading the application, this is called once:

Code: [Select]

hinstDLL = LoadLibrary("multi_mir.dll");
AddControllerFunc=(FnAddCT)GetProcAddress((HMODULE)hinstDLL, "AddController");
UpdateControllerFunc=(FnUpdCT)GetProcAddress((HMODULE)hinstDLL, "UpdateController");
*ctnum = AddControllerFunc((GIVESTATECB)GiveElemState, (GIVEELEMSTR)GiveElemString);
All is fine, right up until a dialogue proc inside the DLL calls the following code to fill a list box with strings, which have to be loaded from one of the "connected" applications:

Code: [Select]

for(i = controllers[ct].typenmin[checktype]; i < controllers[ct].typenmax[checktype]; ++i)
{
// ct is chosen before, it is the appropriate application; every next app to load the dll gets a higher number in the controllers structures array
SendDlgItemMessage(hWndDlg, IDC_CUSTOMDB_LIST2, LB_ADDSTRING, 0,
(LPARAM)controllers[ct].pfnGiveCcolElemString(checktype, i));
}
When two applications have loaded the DLL, this last part works fine for the first to be loaded, but when I choose the second app, and then try to have the listbox filled, the first one crashes due to an access violation.

I can sort of understand that due to virtual addresses things go wrong, but this is a little beyond the depth of my knowledge. Apparently, the current method is not safe... Might there be another way to achieve this?

[jj2007]

Two applications may load the same DLL, and each of them can share memory with its own DLL. However, the two instances of the DLLs use different memory, i.e. you cannot access app2 memory from app1 via the DLL.

Sharing memory was possible with Win 3.1, and is still possible with 16-bit apps via NTVDM.exe, but not for Win32 applications.

[frankie]

Menno,
your application is a little mess. Let start from beginning.
A computer have some memory (4Gb-64-128Gb? ) that is a 'resource' that could be available to programs. It is called 'Phisycal Memory' and is made by memory chips on your board (hardware). Phisycal memory is arranged in a linear array from address 0 to that corresponding to last byte you have on the machine.
Because each program may need whole addressabe memory (addressing capacity of CPU) and because you normally have more than one program in memory they built a system to fool each program in such a way that each one can believe to own all the memory. Moreover each program starts at same address (0x40000).
How can this be achieved? Using a 'Virtual Memory System'. A VMS get a piece of available phisycal mem and assign it to process memory assigning to it a virtual address in the process memory space. The CPU have some special registrers that translate in realtime the virtual address requested by a program to the phisycal address assigned by OS. Of course there is a set of translation values for each process. When a process is scheduled and run by OS the appropriate values are loaded in MMU (Memory Management Unit) of the CPU. The OS will take care to switch these values contextually with Process switching.
For sake of completeness the VMS also swaps memory tranches to and from disk transparently to the application, this way with a small ammount of real phisycal memory your program can believe that you have all memory you want.
Now When you run an application the OS create a process space made of a virtual memory accessible from address 0 to max allowed by OS, set a small ammount of real phisycal memory then load program at start address. The running program can in turn load DLL's, data, allocate dynamic memory etc.
If another program loads the same DLL triggers an optimizing function of OS: the DLL is not reload, but the phisycal address of executable part is 'mapped' in the virtual memory of the new process  . Data is always freshly allocated. The 'mapped' address is the first one available in process space (typically DLL's would load at 0x100000).
Last, but very very important, DLL code runs in a thred of the calling process.
Now you can understand that calling functions from two different processes means to call functions from different memory address.
Running code, for the window, in different process means to access always to different data....
If you want share memory between processes I suggest to use a memory mapped file.
You can create an independent program that create the memory file to collect data, that a library (static or dynamic) of functions that access to your file.

[TimoVJL]

Maybe WM_COPYDATA is easier way to transfer data from/to application, if both application have message loop ?
Look here

[jj2007]

Maybe WM_COPYDATA is easier way to transfer data from/to application, if both application have message loop ?

For this case ("reads data from a variable number of other programs, and can display this data in a single window") WM_COPYDATA is indeed a very good solution. The apps sending the data do not need a message loop, i.e. you can send the WM_COPYDATA message from a console app, too.

[japheth]

Sharing memory was possible with Win 3.1, and is still possible with 16-bit apps via NTVDM.exe, but not for Win32 applications.

I's say this is not quite correct. There is the SHARED attribute for PE sections, which will make the section's memory ...  well, guess!

I haven't tried if the SHARED attribute works for applications, but it definitely works for dlls. IIRC there were a few restrictions for SHARED sections on the Win9x platform - the dlls had to reside in a certain address space above the 2 GB barrier. For OSes based on NT this isn't necessary.

[jj2007]

I haven't tried if the SHARED attribute works for applications, but it definitely works for dlls.

That sounds really interesting. Can you post an example (assembler would be fine, too )? Given that a single WM_COPYDATA message costs around 4,000 cycles, shared memory would definitely be much faster...

[japheth]

Can you post an example (assembler would be fine, too )?

Of course I'm able to provide an example. However, are you a little child that can't do a single step without a helping hand of its mom or dad?  It's rather trivial:

Code: [Select]

    .386
    .model flat, stdcall

    includelib <kernel32.lib>
    includelib <msvcrt.lib>

CStr macro text:vararg
local xxx
.const
xxx db text, 0
.code
exitm <offset xxx>
endm

getchar proto c
printf  proto c :vararg
GetTickCount proto
ExitProcess  proto :dword

SDATA segment flat public shared 'DATA'
nCnt dd 0
SDATA ends

    .code
start:
    invoke GetTickCount
    mov ecx, nCnt
    mov nCnt, eax
    invoke printf, CStr("old cnt=%X, new cnt=%X",10), ecx, eax
    .repeat
        invoke getchar
    .until eax == 'q'
    invoke printf, CStr("cnt=%X",10), nCnt
    invoke ExitProcess, 0

    end start

jwasm -coff test.asm
link test.obj /subsystem:console /libpath:<path where msvcrt and kernel32 libraries can be found>

You'll have to run this application twice and concurrently, just in case you don't know what to do with this sample  . Then you'll notice that in one instance the initial value is != 0.

[TimoVJL]

Thanks  japheth.
Similar example with PellesC too:

Code: [Select]

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

#pragma data_seg(".shared")
DWORD nCounter = 0;
#pragma data_seg()
#pragma comment(linker,"/SECTION:.shared,RWS")

void __cdecl WinMainCRTStartup(void)
{
TCHAR szMsg[100];
wsprintf(szMsg, TEXT("%d"), nCounter++);
MessageBox(0, szMsg, 0, MB_OK);
wsprintf(szMsg, TEXT("%d"), nCounter);
MessageBox(0, szMsg, 0, MB_OK);
ExitProcess(0);
}another example of shared memory.

Code: [Select]

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

void *CreateSharedMem(HANDLE *hMap, DWORD nSize)
{
void *pData;
*hMap = CreateFileMapping(INVALID_HANDLE_VALUE, 0, PAGE_READWRITE, 0, nSize, "MySharedMem");
if (*hMap == NULL)
return NULL;
pData = MapViewOfFile(*hMap, FILE_MAP_WRITE, 0, 0, 0);
if (pData == NULL)
CloseHandle(*hMap);
return pData;
}

void ReleaseSharedMem(HANDLE *hMap, void *pData)
{
if (!pData) return;
if (pData) UnmapViewOfFile(pData);
if (*hMap) {
CloseHandle(*hMap);
*hMap = NULL;
}
}

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpszCmdLine, int nCmdShow)
{
HANDLE hMap;
TCHAR szTxt[100];
int *pData = CreateSharedMem(&hMap, 4096);
if (pData) {
wsprintf(szTxt, "%d", *pData);
MessageBox(0, szTxt, 0, MB_OK);
(*pData)++;
wsprintf(szTxt, "%d", *pData);
MessageBox(0, szTxt, 0, MB_OK);
ReleaseSharedMem(&hMap, pData);
}
return 0;
}

[jj2007]

Of course I'm able to provide an example. However, are you a little child that can't do a single step without a helping hand of its mom or dad? 

Thank you so much, grandpa! It would have taken me hours to google up this exotic syntax

SDATA segment flat public shared 'DATA'
nCnt dd 0
SDATA ends