[TC 11.50b6] Internal 7zip: BCJ2 problem.

[MaxX]

My new tests of 11.50b6 x64.

I pack the folder with Ungoogled Chromium browser + it's user data inside. Thins mean that we have inside executables (exe, dll...) and regular data. The first one needs BCJ2, another - does not.

My configs to use
https://i.imgur.com/NstKeoj.png
https://i.imgur.com/gqEBv7t.png
One is TC, other is common for total7zip.wcx and for standalone 7zip installed.
The plugin total7zip.wcx uses the same 7z.dll and exe as standalone is, both are v24.08 x64.

Both are 64M dic, 273 word, unlimited solid block, ultra (9) level, 2 threads + identical extras qs yx=9 yv=1600 (need that for other purposes, this yv does not affect here at all).

Results of package test:

UngoogledChromium_tc7z.7z     155 065 039     13.11.2024 19:33     -a--
UngoogledChromium_t7zw.7z     142 116 065     13.11.2024 19:27     -a--
UngoogledChromium_7zip.7z     142 116 065     13.11.2024 19:36     -a--

Here: tc7z means internal TC packer (used via bit7z), t7zw is done by total7zip.wcx plugin and 7zip is done by standalone 7-zip.

#1 is bigger because does NOT use BCJ2, it has one LZMA2 block (but it should USE it here!)
#2 and #3 are identical, and both have 2 blocks (BCJ2+LZMA2 and LZMA2).

Conclusion.
Need to find and fix the reason why BCJ2 is lost.

[lelik007]

2MaxX
Hm, the only visual difference here is header encryption...

[lelik007]

2MaxX
Delete -myx=9 from TC's Extra Parameters and try again. For some reason It doesn't work as it should, it lowers the level of analysis. A wrong parsing or a wrong value I suppose. It works as -myx0 and vice verse put -myx0 in 7-zip.

[MaxX]

Delete -myx=9

That in not suitable for me. I need this switch defined as 9.
But I agree that it looks like a bug here.

[lelik007]

2MaxX
It acts like -myx=0, yes, this should be fixed.

[MaxX]

UPDATE:
There is NO problem with unckecked mark "use internal 7zip".
It means that ONLY "internal" 7zip is affected.

UPDATE2:
Finally TC packs archives via 7zg.exe ("internal" NOT used) identical to standalone 7zip packed ones.

[lelik007]

2MaxX
I've checked, -myx=8 works, it maybe used until it fixed, it has no difference with -myx=9.

[ghisler(Author)]

I have checked it in the debugger, TC passes both -myx8 and -myx9 in exactly the same way to the 7-Zip dll: As an UINT4 property named "yx", set to numeric value 8 or 9, respectively. I also tried to pass parameter "yx" as a string with value "9", but got "Invalid parameter" back.
I checked in the debugger, tcbit7z calls function SetProperties() in ISetProperties interface with this option for yx:
name: yx
value: variant of type=UI4, value=9

I do indeed get a much larger archive when packing an EXE file with -myx9 than with -myx8, but I have no idea why.

[lelik007]

2ghisler(Author)
I packed with .dll from here,
viewtopic.php?p=463836#p4638362
the same thing what MaxX, UngoogledChromium
TC's external and internal packers match each other and match a command if I pack with 7zG.exe from CMD.
I used -md384m -mmt12 -myx -mx -slp

[ghisler(Author)]

Nice to hear that! Please let me know if you find any other parameters which cause differences between internal and external packer.

[lelik007]

2ghisler(Author)
I added this and checked with additional options the files are identical.
-md384m -mfb273 -mmt12 -myx -mx -slp -myv=2201 -mqs

[lelik007]

2ghisler(Author)
And Christian check this out, please when you finish 7-zip implementation:
viewtopic.php?t=84116
This can be done in future minor versions.

[ghisler(Author)]

TC's internal CRC32 is already table based. I'm not using SSE, though.

[lelik007]

2ghisler(Author)

TC's internal CRC32 is already table based.

Yes, but it's just one table likely, Slicing-by-4 uses 4 tables, Slicing-by-8 uses 8 tables and 16 tables for Slicing-by-16.

This is why they are faster. You can see from here:
https://github.com/stbrumme/crc32/blob/master/Crc32.cpp#L678
to what point in your code the table goes on.

About any SSE set, they're not needed, it's my fault - this is my misunderstanding what Igor Pavlov said.

And browny was right, libdeflate has libdeflate_crc32 function, and it uses Slicing-by-8.
https://github.com/ebiggers/libdeflate/blob/master/libdeflate.h#L325
here yoг can see how many tables the method has:
https://github.com/ebiggers/libdeflate/blob/master/lib/crc32_tables.h

And I primary mean x64, for x32 I think whatever there is should stay for the compatibility reasons.

[ghisler(Author)]

Interesting, I'm already importing libdeflate_crc32, so I will try to use that.