x64 Porting gotcha #3. Serializing collections

When Microsoft ported MFC to 64 bits they also changed the return type for the GetSize() and GetCount() methods in the collection classes. They changed the return type from the 32 bit DWORD to the 64 bit DWORD_PTR on x64. This has implications if you write your own collection serialization methods. For example if you use a CMap<> to map a thread id to an object you will want to write your own serialization code.

For example (error checking removed for simplification), consider the serialization of this collection.

	CMap<DWORD, DWORD, runningObjectManager *, runningObjectManager *>		threadObjectStatistics;

Saving

	CSingleLock	lock(&threadObjectStatisticsSect, TRUE);
	POSITION	pos;

	ar << threadObjectStatistics.GetCount();

	pos = threadObjectStatistics.GetStartPosition();
	while(pos != NULL)
	{
		runningObjectManager	*rom = NULL;
		DWORD			threadID;
	
		threadObjectStatistics.GetNextAssoc(pos, threadID, rom);
		ar << threadID; rom->save(ar);
	}

Loading

	CSingleLock	lock(&threadObjectStatisticsSect, TRUE);
	DWORD		i, count;

	ar >> count;
	for(i = 0; i < count; i++) { runningObjectManager *rom = new runningObjectManager(); DWORD threadID; ar >> threadID;
		rom->load(ar);
		threadObjectStatistics.SetAt(threadID, rom);
	}

In the above code, the first item saved/loaded is the number of objects in the CMap. After that, the thread id and the complex object associated with the type are saved/loaded for each pair of objects in the CMap. The code above uses a DWORD to load the size. This won’t work for x64 because the count of objects is taken directly from the GetCount() method (or GetSize() for some collection types).

	ar << threadObjectStatistics.GetCount();

x86, the return type is DWORD, and the count is saved as DWORD (32 bit)

x64, the return type is DWORD_PTR, and the count is saved as DWORD_PTR (64 bit)

This is a problem because the loading code is expecting a DWORD.

	DWORD		i, count;

	ar >> count;

Update (23/4/2012): Turns out the same issue affects the STL collections as well. If you are directly serializing the result from the size() method in an STL collection you will be faced with the same problem as I describe for MFC.

Solution 1

One solution is simply to change the type being loaded from a DWORD to a DWORD_PTR.

	DWORD_PTR	i, count;

	ar >> count;

Solution 2

An alternative solution is to always save the size as a DWORD.

	DWORD		count;

	count = (DWORD)threadObjectStatistics.GetCount();
	ar << count;

Conclusion

You may think this is a trivial issue and why write a blog post about it? I agree the actual problem is trivial and the fix for it is also trivial. However, the fact that you have a mismatch in datatypes being serialized because you directly serialized the return value from GetCount() is not so obvious. So much so that this particular issue escaped our attention (and got past a static analyser) until today.

So yes, it’s a trivial problem but it’s a hidden problem and it will cause all sorts of issues during serialization and when you go looking for it you’ll probably look straight past it for a while. Hopefully, I’ve just saved you a few hours of banging your head on a brick wall, or more likely your desk.