SDK Package: read debuggee memory efficiently

I am writing an extension to inspect some custom data types (.NET) in the Visual Studio debugger. To do so I need to read some larger objects from the debugee. For example I'd like to access a property that returns a large byte[] array.

So this is what I got working so far. (Assume that the property Matrix is the large byte[] array:

IDebugExpression2 expression;
string error;
uint errorCharIndex;

epressionContext.ParseText("test.Matrix", (enum_PARSEFLAGS.PARSE_EXPRESSION), 10, 
                           out expression, out error, out errorCharIndex);
IDebugProperty2 debugProperty = null;
expression.EvaluateSync(enum_EVALFLAGS.EVAL_NOSIDEEFFECTS,1000, null, out 
                        debugProperty);

DEBUG_PROPERTY_INFO[] info = new DEBUG_PROPERTY_INFO[1];
debugProperty.GetPropertyInfo(enum_DEBUGPROP_INFO_FLAGS.DEBUGPROP_INFO_ALL, 0, 1000,
                             null, 0, info);

This works fine, and the fields of info totally make sense. But wenn I try to access to contents of the variable in the why I would do for a native C++ object, i do not get a IMemoryBytes2 object:

IDebugMemoryContext2 memoryContext;
debugProperty.GetMemoryContext(out memoryContext);
IDebugMemoryBytes2 memoryBytes;
debugProperty.GetMemoryBytes(out memoryBytes);

memoryBytes is null. So how can I read the contents of this byte[] array without doing it element wise? Is there a way to copy a .NET object from the debuggee to the debugger? I think there must be if I look at the DebuggerVisualizer mechanism available for .NET. But I need this in a Visual Studio Package.