Page 1 of 1

Compressing 2gb+ logs

Posted: Fri Mar 13, 2015 9:10 am
by iketh
You all need help with that code? I have a zip dll that's extremely reliable and free in closed-source applications and wrapper code I wrote and have been using over 5 years in my own applications... let me know... will create any size zip

Re: Compressing 2gb+ logs

Posted: Fri Mar 13, 2015 9:23 am
by Hondata
Options menu -> Settings -> General -> File -> Compress Datalog Files

Raw datalog file 78,090,607 bytes
Zipped datalog file 2,289,729 bytes
Compressed datalog file 1,522,579 bytes

So our compression works better than zip, but the file save time and memory use is higher than without any compression at all.

Re: Compressing 2gb+ logs

Posted: Fri Mar 13, 2015 9:33 am
by iketh
no I mean flashpro manager refuses to compress large datalogs whatsoever

Re: Compressing 2gb+ logs

Posted: Fri Mar 13, 2015 12:54 pm
by Hondata
Probably because it is running out of memory during the compression process. I see what you were getting at now.

Re: Compressing 2gb+ logs

Posted: Sat Mar 14, 2015 9:08 am
by iketh
ya im using tablet with 4GB ram, i'm sure that's what it is

can't use my desktop, refuses to connect no matter how many times plug and unplug usb

Re: Compressing 2gb+ logs

Posted: Sat Mar 14, 2015 9:09 am
by iketh
but regardless, can be written to work in any amount of ram

Re: Compressing 2gb+ logs

Posted: Tue Mar 17, 2015 4:47 pm
by Hondata
It's not really the compression process, but that the log needs to be paged in memory while being compressed. Windows does this up until around 3 gb total memory for a 32 bit process. In theory a 64 bit process has much higher memory limits but in my testing I found that it would run out of memory at about the same 3 gb point. Thus I have not released the 64 bit software as there is no advantage currently. As for the compression problem, I'll have to write my own paging.