64 bit PrjZipWSDir Add-In

[John Z]

Hi,

Has anyone tried to compile PrjZIPWSDir_WS_a2.ZIP 64 bit project in Pelles V12? 
Preferably on WIN 11, but 10 OK too.

Here is the source by TimoVJL and I have NO doubts it worked before .... in 2019
https://forum.pellesc.de/index.php?topic=7178.msg29242#msg29242

I am in the final stage of upgrading a program to 64 bit per Pelle's admonishment that it 'should' be 64 

My 32 bit source used a two file version of minizip.  Which was extremely easy to implement, but has never been upgraded to 64 bit.
https://github.com/kuba--/zip/tree/master

So I'm trying to use TimoVJL's 64 bit dll project code to get started on a 64 bit zip implementation, not as a dll however.

When compiling the sources to learn how to implement, I get an access error on miniz_tdef.obj

Code: [Select]

Building miniz_tdef.obj.
fatal error: Internal error: 'Access violation' at 0x00007ff7ae3b3e99.
*** Error code: 1 ***
Done.
Looking for any ideas or input on the cause which hopefully is not that it is a WIN 11  problem....

Thanks,

John Z

Interestingly when I compile the two file version of minizip as a 64 bit  I also get the same access error.

Code: [Select]

fatal error: Internal error: 'Access violation' at 0x00007ff7ae3b3e99.
*** Error code: 1 ***
Done.

[frankie]

John I attach an amended version that compiles succesfully.
This is a compiler bug that fails on the code line:

Code: [Select]

d->m_dict[TDEFL_LZ_DICT_SIZE + dst_pos] = c;You'll find the workaround in the attached workspace.
I'm opening a bug report on the issue.

[John Z]

Thanks very much frankie,

My head is sore from banging against the wall trying to figure it out.  Ouch!

John Z

I'm going to need to invest in a beer company... 

Update:
Found similar syntax in the two file version.  Interestingly only a problem in 64 bit compile.

Thanks again for helping me get past this roadblock...

[TimoVJL]

Thanks frankie !

[Vortex]

Hi frankie and Timo,

Thanks for your efforts. Regarding the original code in Github, why a lot of functions all collected in a single .c file? Shouldn't the author provide a modular file set?

[John Z]

Regarding the original code in Github, why a lot of functions all collected in a single .c file? Shouldn't the author provide a modular file set?

Not really realistic imo.  Several issues one of which is mainly related to storage for example.  Hard disk storage is inefficient because the software does not support multiple block sizes.  So for example you end up using 404Kb disk space for only 89.6Kb of files because the 114 files are all small. An intelligent HD system would set aside multiple areas each with different block sizes then the file storage first priority would be the closest larger block space.  So now breaking a project into a per-procedure file format would result in hundreds to thousands of tiny files which on a large hard drive with 16K blocks would really impact the storage.

Another issue would be the system accessing the files to compile,  Open and close 50 files or one file?  Upload and download from GitHub 50 small source or one source file?

So I prefer a file for each major function in the program, that file includes all procedures (mostly) that are required for that function. So I end up with 45 C files rather than 650 ... and not counting header files ...

BUT not everyone agrees - so there is that.... however I've not run across even one project on GitHub or SourceForge that has a file per procedure.... this topic may be as adamantly argued about as are religion and politics   

John Z

Attached jpg is from the help files which are in html for a project by topic and shows wasted HD space....

[Vortex]

Hi John,

Every function \ procedure should be moved to it's separate source file. Contradicting this simple rule means that the linker will copy even the unused modules to the final executable becoming larger. The exception is the inter-dependencies in the same module.

[John Z]

Hi Vortex,

Well I'm going to suggest we agree to disagree.
Different strokes for different folks.
Each to their own the old lady said as she kissed the cow. 

I also believe it is very easy to detect unused procedures.  An add-in for Pelles C could find suspect procedures that should be checked for use. So this reason really can not be said to be a major factor.  Even in Pelles C right clicking can tell if a procedure has calls to it.

I also think it would be an easy implementation within a compiler to handle 'unused' procedures. Pelles already removes 'dead code' so next step could be 'dead procedures' 

In addition the executable becoming slightly larger is certainly not an issue for non-embed applications.  In embedded or memory limited systems I agree every byte is precious.

John Z

[TimoVJL]

For static libraries things like COMDAT are important.
Otherwise have to use different files for object files.
Compiler don't know, if code goes to static library.
Vortex knows about objects files and libraries more than me.

[John Z]

A good point to clarify - nothing I’ve said is meant to be referring to libraries.
I’m just referring to building an executable directly from source code.

John Z

[TimoVJL]

still same problem with linking.

[Vortex]

Hi John,

It's the linker's job to find and extract the correct object module from a static library.

Code: [Select]

I’m just referring to building an executable directly from source code.
That's OK for smaller projects but imagine the case of a giant project. You only need to recompile the modifed modules to save time.

[John Z]

Thanks Vortex,

That is certainly an area where I fall short of any experience.  I think the largest program I’ve ever created was less than 1.5Meg.  So I have a tiny view port thru which I’ve made my comments.  Whilst you, TimoVJL, frankie, and others are obviously more experienced in real world programs.

So I appreciate your inputs and patience in explaining the real world situation. Should I progress to larger developments I’ll be sure to follow your advice.

Regards & Thanks

John Z

[Vortex]

Hi John,

It's OK, no worries. The only tool extracting object modules to merge them with the main object file is Jeremy Gordon's GoAsm :

How to use static code libraries
In GoAsm, you can use the ready-made code and data in static code libraries by just calling the required function in your source script, giving the name of the library containing the function, for example

CALL zlibstat.lib:compress
You can also use equates to shorten the call for example:-

LIB1=c:\prog\libs\zlibstat.lib
CALL LIB1:compress
Examples using INVOKE are:-

INVOKE zlibstat.lib:compress,[pCompHeap],ADDR ComprSize,[pHeap],[DataSize]
INVOKE LIB1:compress,[pCompHeap],ADDR ComprSize,[pHeap],[DataSize]
If your path contains spaces you must put the path and filename in quotes.

What happens when you call a library function
The above coding causes GoAsm to load the associated code and data and merge it with GoAsm's output file (object file) during assembly. You can then send the output file to the linker in the usual way, but the linker is not interested in the source of the code and data (the library file) at all: all the code and data associated with the function has already been loaded by GoAsm. The only additional work which you may need to do at link-time is to ensure that the linker is aware of any additional DLLs needed by the ready-made functions. For example, a ready-made function may call a Windows API in OLEAUT32.dll. In order to avoid a "symbol not found" error you need to add that DLL to GoLink's command line. If you are using another linker you would add the import library for OLEAUT32.dll to the list of import libraries given to the linker.

Not the Microsoft method
GoAsm's method of using static code libraries is quite different from the method used by the Microsoft tools. The MS linker adds code and data at link-time. If you prefer you can still use that method with GoAsm. To do so you need to feed the GoAsm output files to the Microsoft linker and tell the linker to look for the functions in the appropriate static code library.
See using GoAsm with various linkers.

https://www.godevtool.com/GoasmHelp/GoAsm.htm#merge

[John Z]

Hi frankie,

This is a compiler bug that fails on the code line:

Code: [Select]

d->m_dict[TDEFL_LZ_DICT_SIZE + dst_pos] = c;

I've got some more code that compiles without error as 32 bit but shows the same "fatal error: Internal error: 'Access violation' at 0x00007ff7ae3b3e99."  when compiled as 64 bit.  I've changed all of the d->xxx[ calculations] to the workaround you showed by calculating the index value(s) ahead so that they all become simple d->xxx[ value].  So there appears to be something else under 64 bit.

So is there a particular technique you used to identify code causing the access violation, or is it based on your years of experience?

I managed to get it to compile (and work) in 64 bit by a code change which makes no sense to me. yet it works. 
change this line: unsigned int probe_pos = d->m_hash[hash]; // access violation crash
into this line     :               int probe_pos = d->m_hash[hash]; // no crash - working zip code

Give a man a fish he eats for a day, teach him to fish and he eats for the rest of his life ....

John Z