Developer Support Visual C++ and C#
Troubleshooting PInvoke Related Issues ![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
I am back with some more PInvoke Stuff. Recently I was working on a PInvoke issue which I found interesting.
I have a C++ dll which has a function whose signature is
int TestFunc(IN_STRUCT in_Params, RET_STRUCT * pret_Par).
I wanted to call this function from C#. Function has two arguments. First argument is input structure which will be filled from C# code and passed to C++ code. Second argument is output structure which is filled in C++ code and output to C# code.
Here are the C struct definitions and a function that needs to be marshaled
#include "stdafx.h"
#include <stdio.h>
#include "Objbase.h"
#include <malloc.h>
typedef struct IN_STRUCT
{
BYTE CMD_PType;
BYTE CMD_PTType;
BYTE CMD_OC;
BYTE CMD_Seq;
};
typedef struct RET_STRUCT
{
BYTE RET_OC;
BYTE RET_Seq;
BYTE RET_RetBytes;
char *str;
BYTE RET_PD[10];
};
extern "C" __declspec(dllexport) /
int TestFunc(IN_STRUCT in_Params, RET_STRUCT * pret_Par)
{
int iRet = 0;
pret_Par->RET_OC = in_Params.CMD_OC;
pret_Par->RET_Seq = in_Params.CMD_Seq;
pret_Par->RET_RetBytes = 6;
pret_Par->RET_PD[0] = 0;
pret_Par->RET_PD[1] = 10;
pret_Par->RET_PD[2] = 20;
pret_Par->RET_PD[3] = 30;
pret_Par->RET_PD[4] = 40;
pret_Par->RET_PD[5] = 50;
pret_Par->str = new char(30);
strcpy(pret_Par->str,"This is sample PInvoke app");
return iRet;
}
Managed Structure equivalent to Native Structure:
namespace ConsoleApplication1
{
class Program
{
//This structure will be filled up by C++ Test.dll and returned back
With values to C# code.
[StructLayout(LayoutKind.Sequential)]
public struct RET_STRUCT
{
public byte RET_OC;
public byte RET_Seq;
public byte RET_RetBytes;
[MarshalAs(UnmanagedType.LPStr)]
public String RET_STR;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public byte[] RET_PD;
};
//The values of this structure will be used to fill up IN_STRUCT and
passed to C#
[StructLayout(LayoutKind.Sequential)]
public struct IN_STRUCT
{
public byte CMD_PT;
public byte CMD_PTType;
public byte CMD_OC;
public byte CMD_Seq;
};
//C++ dll containing the func
[DllImport("Test.dll")]
public static extern int TestFunc(IN_STRUCT i, ref RET_STRUCT r);
static void Main(string[] args)
{
IN_STRUCT cmd_params = new IN_STRUCT();
RET_STRUCT ret_Params = new RET_STRUCT();
//Fill up the cmd_params
cmd_params.CMD_OC = 0x02;
cmd_params.CMD_PTType = 0x00;
cmd_params.CMD_Seq = 1;
//Call the C++ function to fill ret_params
int iRet = TestFunc(cmd_params, ref ret_Params);
//Print out the returned values
Console.WriteLine("Returned Values/n");
Console.WriteLine(ret_Params.RET_OC + " " + ret_Params.RET_Seq +
" ");
for (int i = 0; i < ret_Params.RET_RetBytes; i++)
Console.WriteLine("/n" + ret_Params.RET_PD[i]);
Console.WriteLine(ret_Params.RET_STR);
Console.ReadLine();
}
}
}
After executing the code I was expecting a valid output. But I ended up with Access Violation. I used windbg to troubleshoot this issue.
I spawned exe from windbg and tried to see call stack.
0:000> kv
ChildEBP RetAddr Args to Child
002cec30 76fc5883 006515c8 00000001 00000000 ntdll!RtlpLowFragHeapFree+0x31 (FPO: [0,10,4])
002cec44 76b9c56f 000b0000 00000000 049a3a48 ntdll!RtlFreeHeap+0x101 (FPO: [3,0,4])
002cec58 7565dc2c 000b0000 00000000 049a3a50 KERNEL32!HeapFree+0x14 (FPO: [3,0,0])
002cec6c 7565dc53 7573e6f4 049a3a50 002cec88 ole32!CRetailMalloc_Free+0x1c (FPO: [2,0,0])
002cec7c 6c7e8410 049a3a50 002cec9c 6c8084bd ole32!CoTaskMemFree+0x13 (FPO: [1,0,0])
002cec88 6c8084bd 00109d34 00000001 00109d48 mscorwks!FieldMarshaler_StringAnsi::DestroyNativeImpl+0x16 (FPO: [1,0,0])
002cec9c 6c8088e5 00109d30 0065340c 1670b1d2 mscorwks!LayoutDestroyNative+0x3a (FPO: [2,0,0])
002cee8c 6c73539b 002cef58 00000000 1670b182 mscorwks!CleanupWorkList::Cleanup+0x2ea (FPO: [2,116,4])
002ceedc 001cad4c 002cef18 01020000 00109d30 mscorwks!NDirectSlimStubWorker2+0x120 (FPO: [1,12,4])
WARNING: Frame IP not in any known module. Following frames may be wrong.
002cefa4 6c7013a4 00700876 002cefd8 00000000 0x1cad4c
002cefe0 6c6f1b4c 010d2816 00000003 002cf070 mscorwks!PreStubWorker+0x141 (FPO: [1,13,4])
002ceff0 6c7021b1 002cf0c0 00000000 002cf090 mscorwks!CallDescrWorker+0x33
……….
0:000> da 049a3a50
049a3a50 "My Name is Jyoti Patel"
From the call stack it’s clear that InteropMarshaller ( NDirectSlimStubWorker2) is trying to deallocate string using CoTaskMemFree.
There are two solutions to this problem.
1. As deallocation is done using CoTaskMemFree, Allocate the memory using CoTaskMemAlloc.
Changing line of code in C++ from
pret_Par->str = new char(30);
pret_Par->str = (char*)CoTaskMemAlloc(30);
Issue was resolved.
(When a string buffer allocated by native code is marshaled to managed code, CLR Interop marshaller will allocate a managed string object and copy the contents of native buffer to the managed string object. Now in order to prevent a potential memory leak, CLR Interop Marshaller will try to free allocated native memory. It does so by callingCoTaskMemFree. The decision to call CoTaskMemFree is by-design. This can at times lead to crash, if memory was allocated by the called-native function using any API other than CoTaskMemAlloc family of API’s as custom allocators may allocate on different heaps.)
In this case, memory was allocated by malloc, and ends up being freed by CoTaskMemFree and hence we see an AV)
2. If you are not able to change the C++ code and you want to allocate memory using new/malloc other solution is to use Intptr and do custom marshalling by calling corresponding methods from Marshal class to do marshalling.
[StructLayout(LayoutKind.Sequential)]
public struct RET_STRUCT
{
public byte RET_OC;
public byte RET_Seq;
public byte RET_RetBytes;
public IntPtr RET_STR;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
public byte[] RET_PD;
};
……….
………
………
Console.WriteLine(Marshal.PtrToStringAnsi(ret_Params.RET_STR));
Jyoti Patel and Manish Jawa
Comments
alanjmcf said:
Is there a wee typo in the last case? To use Marshal.PtrToStringAnsi shouldn't the RET_STR field then be a IntPtr rather than the original String type?
Jyoti Patel said:
Hi Alanjmcf,
It was a typo. Thanks for correcting.
Jeff said:
Very timely post, which explained a crash I've been tearing my hair out - code that works on XP but crashes on Windows 7.
In my particular case, I had a native function that was returning a 'const char *' which I was trying to marshal (as LPSTR) into a string.
I can't see any way to tell the marshalling mechanism that the result is supposed to be 'const' and thus it should not free it. Do I really need to do the IntPtr/Marshal.PtrToStringAnsi() approach?
Jyoti Patel said:
Hi Jeff,
If I understand you correctly.
If your C++ function is returning const char*
You can either use IntPtr and Marshal.PtrToStringAnsi() or you can use StringBuilder.
extern "C" __declspec(dllexport) const char* ReturnString( )
{
char * ch = (char*)CoTaskMemAlloc(20);
strcpy(ch,"This is sample");
return ch;
}
You can use string builder in C# code, as string are immutable.
[DllImport("Test.dll",CharSet = CharSet.Ansi)]
public static extern StringBuilder ReturnString();
StringBuilder str = ReturnString();
It I am not able to understand your question properly. Please feel free to contact me at jpatel_at_microsoft.com
Jeff said:
No, you've missed the point. Lets assume that my C++ DLL, which already serves other applications and thus cannot be changed, looks like this:
__declspec(dllexport) const char *message(int key)
{
switch(key):
default: return "Bad Key";
case 1: return "Key 1";
case 2: return "Two two";
}
}
Its far more complicated than that, indexing into other data structures which will remain alive for the duration of the application. The point is that the caller is not supposed to free the result, its 'const'. If I make that entry point allocate memory, it will leak when all the other existing callers use it.
It appears to be impossible to marshal this across to C# because there is no 'const' keyword anywhere in the MarshalAs() attribute, presumably because there is no concept of 'deleting' .NET objects, only letting them expire through garbage collection. Its only at the bridge between native and managed that explicit memory management becomes necessary, I understand that.
I have already used the IntPtr/PtrToStringAnsi approach successfully - I was asking whether there was some other aspect of the MarshalAs() attribute mechanism that I was unaware of that would allow me to tell the marshalling mechanism that it didn't need to free the input LPStr
(Note also, I'm doing this in comments, rather than private email because I spent way too long googling to try to find any sort of discussion on this topic - I'd much rather leave a trail that others in the same boat can follow)
Scot said:
'const' means "don't modify it", NOT "don't free it". You aren't looking for a way to specify constant-ness. When the marshaled type is a string, the contract is that the marshaler will free it.
Jeff said:
@Scot:
Um, if freeing it doesn't modify it, I'll eat my hat. You can't pass 'const' pointers into free(), you can't use 'delete' on const pointers, etc. The only mechanism any C/C++ API has to ensure its results are not free'd is to declare the result as 'const'. Its only in .NET where you can't *ever* explicitly delete an object, and thus the distinction is blurred.
Its pointless to worry about the 'can't modify' aspect of it, since by definition marshalling creates a completely different object; of course there would be no way that the original could be modified by manipulation of the .NET object.
Can you point me at any explicit statement made anywhere that explains that the marshalling API contract including the freeing of the input data? I haven't seen any reference to that behaviour anywhere so far.
I'm quite happy to be wrong on this, but this article (specifically in point 1) seems to suggest that the marshalling interface has just "decided to use CoTaskMemFree()" and that it can "lead to a crash". That suggests to me that the marshalling API's *dont* define that input strings will be freed; otherwise there would be no need to clarify here, just refer to the appropriate msdn page.
manishjawa said:
I would like to clarify a few things here. Marshalling interface has not "just decided to use CoTaskMemFree" , it definitely is a conscious decision which would have been made taken into account several things. I do not know the exact details of why the decision was taken as of now.
But I tend to disagree with you on constness and liveliness part of it. Const modifies a type; that is, it restricts the ways in which an object can be used, rather than specifying how the constant is to be allocated/freed. The fact that a constant or for that matter static is allocated on read-only storage by VC++ compiler is a optimization made by VC++ compiler and is not something that is mandated to it by C++ standard .
Also marshalling does not neccesarily mean that a new object will be always created. It is a neccessity in cases where layout of object is totally different but if layout of object is same or simillar , marshaller is free to reuse the same memory location. Unless otherwise stated the fact that marshaller allocates new object is an implementation detail and not a part of contract of marshalling interface per se.
Jeff said:
If it was a conscious decision, can someone please point me to the MSDN documentation that describes the behaviour that was apparently included by design? And can someone point out to me why it behaves differently on on .NET 2.0 vs 3.5. The exact same code works fine on 2.0, no debugger output, etc whereas it crashes almost immediately on 3.5 - at the least I should expect to see a release note somewhere explaining that a memory leak was fixed, at the expense of possible crashes.
Yes, const modifies a type and the type in question is 'char'. 'const char *' is "pointer to constant characters" - you can change the value of the pointer itself, but not of what it points to. You explicitly cannot delete, or free "what it points to". This is irrespective of whether the value are in a read-only segment, or are just a pointer into my seperately maintained symbol table of objects which will remain live for the duration of the application. It makes no difference, you are not allowed to free that object. And the problem here is that there appears to be no way to tell the marshaller that fact.
I accept that marshalling may not create a new object - in fact, this is what I though the original problem was, that the String class was retaining a pointer to *my* character array, but that would again introduce undocumented behaviour, since Marshalling makes no claims about a limited life-span of the marshalled object, and there's no way it could assume the array would remain alive once this function returns. Nevertheless, I have claimed nothing about whether Marshalling creates new objects or not.
What I asked is "where does it say that marshalling will discriminantely free its input"? I same discriminantley because its *not* indiscriminant, it doesn't free everything, only strings. What happen if I tell it that a function returns a pointer to a struct, which contains 3 reals (note, I do this currently, this is not speculation, I need this to work). Is it documented that marshalling will also free that structure? (Note, I am not asking about structures that contain pointers to other fields, or other marshallable structures, just primitive data types)
What I want to know is "where is the secret list of extra rules about what gets freed and what doesn't"?
Jeff said:
Actually, on the 'pointer to struct containing reals', I've double-checked my source code and I'm already using the IntPtr/Marshal.PtrToStructure() approach there so its not a problem for me.
manishjawa said:
http://msdn.microsoft.com/en-us/library/f1cf4kkz.aspx
This article explains rules for memory management by Interop Marshaller in general. Are you observing the difference between .net 2.0 and .net 3.5 or is the difference between XP and Vista ?
Jeff said:
Thank you for that reference, it obviously is documented and it was my Googling skills in error.
From my reading of that page, its never going to be an issue for arbitrary structure pointers because the only way you can marshall them is as IntPtr which won't be automatically free'd.
Its hard to tell whether I'm getting the problem because of the 2.0/3.5 difference or the XP/Vista/7 difference, because technically we are compiled against 2.0, but we don't have that available on Vista or 7 and thus it automagically upgrades itself to using 3.5.
Our app wants to try and run regardless of which version of .NET is available (to as much an extent as that is possible), and we also run as a plugin inside AutoCAD which means that we can't dictate the contents of a .config file to force a specific version of .NET
Once again, thanks for putting up with my annoying questions.