Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[cabbott]

I am seeing odd problems when running Vampyre across multiple threads.

On odd occasions it raises an exeption indicating either a Access Violation or Access Denied error reading/writing to memory. Eventually ending up with Out Of Memory errors.

Trying to debug it is a bit like the cat in the box scenario. The problem goes away if I try debugging because the threads have time to avoid memory conflicts.

Are there any areas within the following function calls that I might need to look out for or tweak?
TSingleImage.Create()
DetermineStreamFormat()
LoadFromStream()
ResizeImage()

I am only using the JPEG side of things at present

The Out Of Memory errors seem to stem from a memory leak when an exception occurs.
I'm not convinced all the allocated memory is being freed within Vampyre when an exception occurs. I am calling FreeImage() at the end of my processing regardless of the what happens during the above function calls.

If I run only one thread, or introduce delays everything is fine.

[Galfar]

Could you try loading different file type than JPEG?
I'm not sure if underlying JpegLib is threadsafe.
Does it also happen if you comment out ResizeImage (this
function should be safe)?

[cabbott]

Not using ResizeImage makes no difference, the problem still occurs.

I will do some more testing on another format over the weekend - need to convert 10,000 JPEG's to some other format to do this properly!

The errors indicate the app is out of memory, but only 110MB is allocated, well below the 2GB limit of a 32-bit app. So this suggests it is out of stack space.  I've tried increasing this but the problem still occurs at roughly the same place which would suggest it isn't this.

I've run the app through a leak checker and there are no leaks.

The only other possibility is memory fragmentation, causing the memory manager to eventually be unable to allocate one large chunk of memory. Not sure what can be done about this short of pre-allocating memory to load the images in.

I said that the problem doesn't occur outside of threads, this is not true, it does. It just takes a lot more images to create the problem - which again points to memory fragmentation.

I'll keep trying different methods of using the library and see if there is a better way of doing it. I am currently creating a new thread for each image, and within this thread a new set of calls to Vampyre.
This is obviously not the best way of doing it as this will clearly cause a lot of fragmentation. I think I need to move to 8 static threads with only 8 image objects.

Will let you know how I get on.

[Galfar]

Ok let me know what you'll find out please.

[cabbott]

Ok, looks like it is the Delphi memory manager causing fragmentation problems.
I have switched to FastMM and as if by magic, all the problems have gone away.

I would recommend FastMM to anyone thinking about serious memory usage in Delphi. It even has dialog where you can dynamically see all the memory allocations whilst your app is running.

It's available here http://sourceforge.net/projects/fastmm/.

[Galfar]

Great.

You are using older Delphi then Delphi 2006? FastMM is default memory manager
since this version.

[cabbott]

Yes, Delphi 7.  Because even the latest version only supports 32-bit, I can't really justify upgrading.