Compressing 2gb+ logs

FlashPro Manager software
Post Reply
iketh
Posts: 44
Joined: Sat Apr 12, 2014 7:55 pm

Compressing 2gb+ logs

Post by iketh »

You all need help with that code? I have a zip dll that's extremely reliable and free in closed-source applications and wrapper code I wrote and have been using over 5 years in my own applications... let me know... will create any size zip
User avatar
Hondata
Site Admin
Posts: 10616
Joined: Thu Aug 15, 2002 12:13 pm
Location: Torrance, CA
Contact:

Re: Compressing 2gb+ logs

Post by Hondata »

Options menu -> Settings -> General -> File -> Compress Datalog Files

Raw datalog file 78,090,607 bytes
Zipped datalog file 2,289,729 bytes
Compressed datalog file 1,522,579 bytes

So our compression works better than zip, but the file save time and memory use is higher than without any compression at all.
Hondata
iketh
Posts: 44
Joined: Sat Apr 12, 2014 7:55 pm

Re: Compressing 2gb+ logs

Post by iketh »

no I mean flashpro manager refuses to compress large datalogs whatsoever
User avatar
Hondata
Site Admin
Posts: 10616
Joined: Thu Aug 15, 2002 12:13 pm
Location: Torrance, CA
Contact:

Re: Compressing 2gb+ logs

Post by Hondata »

Probably because it is running out of memory during the compression process. I see what you were getting at now.
Hondata
iketh
Posts: 44
Joined: Sat Apr 12, 2014 7:55 pm

Re: Compressing 2gb+ logs

Post by iketh »

ya im using tablet with 4GB ram, i'm sure that's what it is

can't use my desktop, refuses to connect no matter how many times plug and unplug usb
iketh
Posts: 44
Joined: Sat Apr 12, 2014 7:55 pm

Re: Compressing 2gb+ logs

Post by iketh »

but regardless, can be written to work in any amount of ram
User avatar
Hondata
Site Admin
Posts: 10616
Joined: Thu Aug 15, 2002 12:13 pm
Location: Torrance, CA
Contact:

Re: Compressing 2gb+ logs

Post by Hondata »

It's not really the compression process, but that the log needs to be paged in memory while being compressed. Windows does this up until around 3 gb total memory for a 32 bit process. In theory a 64 bit process has much higher memory limits but in my testing I found that it would run out of memory at about the same 3 gb point. Thus I have not released the 64 bit software as there is no advantage currently. As for the compression problem, I'll have to write my own paging.
Hondata
Post Reply