Propose optional CPUARCH target AlderLakeAVX512, as aliase of SapphireRapids

Context
It's been reasonably well documented that on alder lake, if, and only if, the GraceMount Ecore's are disabled, it becomes possible to enable all of the available AVX512 instruction available on the Golden Cove Pcore's the same core's used in SapphireRapids.
For relevant workloads, many of which OpenBLAS has AVX512 accelerated code paths, this can lead to a significant performance uplift.
As of release 0.3.19, even if ecores are disabled and AVX512 is available, the build system will not make use of them automatically.
Current workaround
The user has the option of either passing:
CFLAGS='-O3 -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16'
CXXFLAGS='-O3 -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16'
FFLAGS='-O3 -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16'

or alternatively passing
CFLAGS='-O3 -march=native -mavx512f -mavx512dq -mavx512ifma -mavx512cd -mavx512bw -mavx512vl -mavx512bf16 -mavx512vbmi -mavx512vbmi2 -mavx512vnni -mavx512bitalg -mavx512vpopcntdq -mavx512vp2intersect'
CXXFLAGS='-O3 -march=native -mavx512f -mavx512dq -mavx512ifma -mavx512cd -mavx512bw -mavx512vl -mavx512bf16 -mavx512vbmi -mavx512vbmi2 -mavx512vnni -mavx512bitalg -mavx512vpopcntdq -mavx512vp2intersect'
FFLAGS='-O3 -march=native -mavx512f -mavx512dq -mavx512ifma -mavx512cd -mavx512bw -mavx512vl -mavx512bf16 -mavx512vbmi -mavx512vbmi2 -mavx512vnni -mavx512bitalg -mavx512vpopcntdq -mavx512vp2intersect'

NOTE: It is preferred to use the -march=sapphirerapids option as GCC, Clang, LLVM based ICX and ICC will then preserve the relevant AVX512 cost functions for auto-vectorization vs other instruction pathways, instead of generic costs meant as a catch all across all supported architectures.
NOTE: We must disable the AMX tile and AMX instructions as the additional hardware was not built into Alder Lake.
Proposed solution/Request
I'd like to propose that, if the user supplies an architecture flag AlderLakeAVX512, it be aliased to be enable all features of Sapphire Rapids not explicitly requiring the AMX Tile.
Sources on AVX512 support on alder lake:
Anandtech article
Phoronix Article

OpenBenchmarking Results of disabling 8 ecores to enable AVX512 on 8 pcores
Testing of AVX512 per instruction cost and pipeline on Alderlake
discussion of AVX512 performance support and performance

You should be able to achieve just that by building with TARGET=SAPPHIRERAPIDS already. The autodetection code will default to Haswell as it does not "know" if the Ecores are disabled (and going to stay that way)

TARGET=SAPPHIRERAPIDS

Unless I'm mistaken, this would mean that AMX instructions would also be compiled into source, which are not supported on alder lake and would cause a crash if/when any routines call on them.
Specifically any calls that can be accelerated by int8 or bf-16 are expected to be offloaded to the AMX tile instead of being done in AVX 512 because of the much higher throughput of the AMX matrix instructions vs the AVX vector instructions.

Ah sorry, I replied to fast - should have written "TARGET=SKYLAKEX" (though an OpenBLAS built for SapphireRapids would probably work just as well, provided one does not actually throw any bfloat16 data at it)

"TARGET=SKYLAKEX"

Yeah, Id thought of doing this as a work around, but this would also produce a sub-optimal build unfortunately for 2 reasons:

it would not take advantage of  the additional AVX512 instructiotions added over Skylake
It would use the wrong cost functions (saw your edit, including this for the sake of documenting for others in this thread, but there's a little bit more to it, see below)

2a. This is also the reason we don't want to use march=native or march=alderlake +mavx[list of instructions to include], because the compiler will then select cost functions to balance between both the Golden Cove Pcores && the gracemount ecores, leaving performance on the table due to things like different ALU's, cache size's, having SMT etc.
2b. We'd also end up not using all available AVX instructions, since SKYLAKEX supports significantly less.
SkylakeX:
Intel Skylake Server CPU with 64-bit extensions, MOVBE, MMX,
SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, POPCNT, PKU,
AVX, AVX2, AES, PCLMUL, FSGSBASE, RDRND, FMA, BMI,
BMI2, F16C, RDSEED, ADCX, PREFETCHW, CLFLUSHOPT,
XSAVEC, XSAVES, AVX512F, CLWB, AVX512VL, AVX512BW,
AVX512DQ and AVX512CD instruction set support.

SapphireRapids:
Intel sapphirerapids CPU with 64-bit extensions, MOVBE, MMX,
SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, POPCNT, PKU,
AVX, AVX2, AES, PCLMUL, FSGSBASE, RDRND, FMA, BMI,
BMI2, F16C, RDSEED, ADCX, PREFETCHW, CLFLUSHOPT,
XSAVEC, XSAVES, AVX512F, CLWB, AVX512VL, AVX512BW,
AVX512DQ, AVX512CD, AVX512VNNI, AVX512BF16,
MOVDIRI, MOVDIR64B, AVX512VP2INTERSECT, EN-
QCMD, CLDEMOTE, PTWRITE, WAITPKG, SERIALIZE,
TSXLDTRK, UINTR, AMX-BF16, AMX-TILE, AMX-INT8 and
AVX-VNNI instruction set support.

of which the new instructions relevant for alder lake are:
AVX512VNNI, AVX512BF16,
MOVDIRI, MOVDIR64B, AVX512VP2INTERSECT, EN-
QCMD, CLDEMOTE, PTWRITE, WAITPKG, SERIALIZE,
TSXLDTRK, UINTR, and
AVX-VNNI

Which means we do have BF16 support in AVX, the same way as Sapphire rapids, but don't have int8 at all.

Hmmm. As you appear to have the hardware, can you confirm that switching AVX512 on/off by whatever means gets accurately reflected in the capabilities it reports e.g. through /proc/cpuinfo (or more precisely through the cpuid bits that OpenBLAS already knows to check via the support_avx512() and support_avx512_bf16() helper functions in cpuid_x86.c (build-time) and driver/others/dynamic.c (run-time in DYNAMIC_ARCH builds) ? And is it safe to assume by now that the temporarily orphaned
AVX512 hardware is producing correct results ?

seems to be big-small CPUs where half of cores have no avx-512.

seems to be big-small CPUs where half of cores have no avx-512.

Err, yes, but the point is that AVX512 appears to be completely disabled by default, until one disables the little guys in the BIOS - at which point it transforms into something in between a SkylakeX and a SapphireRapids cpu.
If this is accurately reflected in the cpuid bits it would simplify checking for this particular (non-default) operating mode, and probably warrant a dedicated TARGET.

We would need at least /proc/cpuinfo from both modes? If any of stepping codes change, it is already instrumented, and if CPUID bits change, then too, the only danger is asymmetric CPUID-s on same system?

Hmmm. As you appear to have the hardware, can you confirm that switching AVX512 on/off by whatever means gets accurately reflected in the capabilities it reports e.g. through /proc/cpuinfo (or more precisely through the cpuid bits that OpenBLAS already knows to check via the support_avx512() and support_avx512_bf16() helper functions in cpuid_x86.c (build-time) and driver/others/dynamic.c (run-time in DYNAMIC_ARCH builds) ? And is it safe to assume by now that the temporarily orphaned
AVX512 hardware is producing correct results ?

Yes, I can confirm that performance is correct (tested against known good results on each core over the course of multiple days with no results that deviated) and that, after a reboot and enabling or disabling instructions in bios instructions are reported via cpuid as having or not having the relevant instructions. (Depending on configuration)
The sources in the original post have some screen shots of the CPUID bits being changed for AVX512 enablement, but for convenience here's a system output:
processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 151
model name	: 12th Gen Intel(R) Core(TM) i7-12700K
stepping	: 2
microcode	: 0x15
cpu MHz		: 3600.000
cache size	: 25600 KB
physical id	: 0
siblings	: 16
core id		: 28
cpu cores	: 8
apicid		: 57
initial apicid	: 57
fpu		: yes
fpu_exception	: yes
cpuid level	: 32
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l2 invpcid_single cdp_l2 ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves split_lock_detect avx_vnni avx512_bf16 dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect md_clear serialize pconfig arch_lbr avx512_fp16 flush_l1d arch_capabilities
vmx flags	: vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_mode_based_exec tsc_scaling usr_wait_pause
bugs		: spectre_v1 spectre_v2 spec_store_bypass swapgs
bogomips	: 7219.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

The procedure for enabling AVX512 support varies slightly based on mainboard vendor, but Asus MSI Gigabyte and AsRock have all committed (and delivered) support in bios for enabling AVX512 once the ecores are all disabled. This is exclusive to the z690 chipset as of now.
I'll have to pull the latest source for a clean build environment, but once completed I'll verify that OpenBLASs can properly detect the intructions for both build and runtime (how this then effects build flags and compiler costs would then need to be profiled I think?)

seems to be big-small CPUs where half of cores have no avx-512.

Correct, it's a hybrid architecture x86 CPU with different ISA support between the 2 cores.
Still awaiting confirmation from intel, but it also seems that only the K class chips will have the ability to enable AVX-512.
functionally that means that we have 3 chips available:
8 core i9 12900k with 30MB of L3
8 core i7 12700k with 25MB of L3
6 core i5 12600k with 20MB of L3
The difference in cache is that the L3 is shared and is still enabled even when the ecores are disabled.

between a SkylakeX and a SapphireRapids cpu.

This is correct, though the per core performance would actually partially out pace that of the Sapphire rapids version due to higher available power budget per core (and therefore frequency) and slightly increased L3 per core due to the disabled e-cores still contributing public L3

cpu_info_avx512.txt

very quick and dirty patch (Sapphire Rapids is currently treated like Cooper Lake and I just noticed autodetection does not yet have it as a separate type):
--- cpuid_x86.c 2021-12-19 14:07:28.243783197 +0100
+++ cpuid_x86.cnew      2021-12-20 22:50:44.629040294 +0100
@@ -1495,6 +1495,10 @@
         switch (model) {
         case 7: // Alder Lake desktop
         case 10: // Alder Lake mobile
+         if(support_avx512_bf16())
+            return CPUTYPE_COOPERLAKE;
+          if(support_avx512())
+            return CPUTYPE_SKYLAKEX;
           if(support_avx2())
             return CPUTYPE_HASWELL;
           if(support_avx())

How about the peak throughput of AVX512 instructions comparing to their AVX2 counterparts on Alder Lake? On some archetectures, the dedicated avx512 unit on port 5  is disabled, leaving the theoretical GFLOPS of AVX512 the same as that of AVX2.

Also it is documented that AVX-512 anywhere switches off turbo boost, so 1-core performance also should be compared, going back even to sandybridge (AVX1 only)

How about the peak throughput of AVX512 instructions comparing to their AVX2 counterparts on Alder Lake? On some architectures, the dedicated avx512 unit on port 5 is disabled, leaving the theoretical GFLOPS of AVX512 the same as that of AVX2.

I've gotten as high as 5x throughput as compared to AVX2 on alder lake for code that is very AVX512 optimized such as particle simulations.
In part this is due to how when the ecore's are disabled (a requirement to use avx512) the ring bus is then relative to the p core frequency.
Typically ring frequency = (e core multiplier -3) * base clock (aka bclk)
for the i9 12900k:
Ecore on
Max ring bus frequency of 3.6GHz, which also drives L3 cache for all cores, memory access etc.
Ecore off
Max ring bus frequency of 4.9 GHz, which also drives L3 cache for all cores, memory access etc. (you also get the keep the L3 of the disabled ecores, meaning your per core L3 cache size goes up)
The per instruction throughput per clock with including avx512 has been formally measured and can be found here:
https://github.com/InstLatx64/InstLatx64/blob/master/GenuineIntel/GenuineIntel0090672_AlderLake_BC_AVX512_InstLatX64.txt
As compared to the standard configuration without avx512 and the 8 ecores enabled
https://github.com/InstLatx64/InstLatx64/blob/master/GenuineIntel/GenuineIntel0090672_AlderLake_BC_InstLatX64.txt

Also it is documented that AVX-512 anywhere switches off turbo boost, so 1-core performance also should be compared, going back even to sandybridge (AVX1 only)

This isn't quite right, it's more that it makes turbo boost less effective.
This is dependent on generation, but I think you're referring to AVX offset. Typically a core that is executing AVX code and having to use XMM or ZMM registers generates more heat/draws more power. As a way of balancing power budget what Intel did is apply a net offset to the current turbo boost frequency. This applies for Turbo boost 2.0 and 3.0
If a chip has a base of 3.5, and is within temperature and power limits, and a load is applied, it will boost to say (base multiplier +11)* baseclock =4.6GHz.
TurboBoost 2.0 behavior (before Alder lake)
As it executes, let's say the program then calls AVX2 code using the XMM registers. These will typically offset current clock by -1 on multiplier, or (3.5 +11(turbo boost)-1(avx2 off set))*base clock= 4.5GHz
let's say that same program then requests AVX512 code, these use the ZMM registers. typically AVX512 is either a -2 or -3 multiplier offset. as above,
we end up at say 4.3Ghz.
Now, if we then exceed/approach power/temperature limits, turbo boost will lessen until it stabilizes. In workstations with proper cooling this will be near maximum frequency -1-2, so we end up with a sustained AVX512 frequency of 4.1GHz
Alder Lake is a little more interesting, but suffice to say, in my testing so far, I've noticed that avx2 has no offset from base standard multiplier, and avx512 does not directly apply an offset, rather it informs the internal system frequency governor to be more "careful" with maxing out clocks. This means that, with a proper cooling set up, I can happily run 8 cores of AVX512 code on the i7 12700k at 5GHz sustained for a tested 24+hrs
Note: AVX2 and AVX1 are pretty much the same in terms of actual usage/performance, you could easily refer to avx2 as avx1.1 from how small the differences were.
Note: AVX512 on the Other hand is a slight pain to deal with from a low level/asm point of view. AVX 512 is not a complete instruction set, but rather a familly of instruction sets that only guarantees a single "common" core of instructions, and vendors can choose to implement other parts of the AVX512 family of instructions as they see fit.
The AVX512 Wikipedia page has a quite good chart showing what family of instructions are supported on what hardware
https://en.wikipedia.org/wiki/AVX-512#CPUs_with_AVX-512

Thank you for providing the links. It looks like the Alder Lake processor is still in half-throughput avx512 design (from comparing the throughput data of avx512/avx2 vfmadd231ps and vfmadd231pd), so probably no significant performance gain will be observed in SGEMM/DGEMM of large matrices when enabling avx512...

This isn't quite right, it's more that it makes turbo boost less effective.

Nope, Haswell clocks to base frequency when AVX2 is engaged on any core, same applies to that modern lake with AVX-512. In Desktop/Laptop context that makes previous generation sometimes slightly better.

avx512/avx2 vfmadd231ps and vfmadd231pd)

for those specific instructions you're right, it's not massive on the instruction itself, but we do save the 4 clock fetch latency from L1
2447 AVX512VL            :{EVEX} VFMADD231PS xmm, xmm, xmm      L:   1.26ns=  4.0c  T:   0.16ns=  0.50c
2448 AVX512VL            :{EVEX} VFMADD231PS ymm, ymm, ymm      L:   1.26ns=  4.0c  T:   0.16ns=  0.50c
2449 AVX512F             :VFMADD231PS zmm, zmm, zmm             L:   1.26ns=  4.0c  T:   0.31ns=  1.00c
for 2048 bits
AVX1(XMM 128 bit registers) 16*(4.0c+0.5c)-> 72c or ~22.32ns
AVX2(YMM 256 bit registers) 8*(4.0c+0.5c)-> 36c or ~11.16ns
AVX512(ZMM 512bit registers) 4*(4.0c+1c)-> 20c or ~6.2ns
Granted it won't be quite this extreme because of being able to prefetch, but you do still end up making more efficient use of cache space and memory with AVX512, but the latency penalty is what dominates on alder lake.
Dug through my archive, here's some actual data on performance:
note: compiled using march=native on gcc 10.3 which does not have a target for Alder lake or Sapphire rapids.
The cost functions were therefore not quite right (as discussed/mentioned here https://github.com/xianyi/OpenBLAS/issues/3490#issuecomment-998015564  )
note:GCC 12, Clang14 are showing promising results as of now, and the "optimal" use case is to declare the architecture as march=sapphirerapids -mno-amx-tile

You can try testing 1-thread DGEMM of OpenBLAS with various problem sizes. I suspect that the speedup will be significant for MNK<=1000000(direct matmul kernels for SkylakeX, not yet for Haswell) but neglectible for MNK>=10000000 when avx512 is enabled:)

You can try testing 1-thread DGEMM of OpenBLAS with various problem sizes. I suspect that the speedup will be significant for MNK<=1000000(direct matmul kernels for SkylakeX, not yet for Haswell) but neglectible for MNK>=10000000 when avx512 is enabled:)

sure, sounds fine. Unless I'm mistaken in my interpretation of the above, if so please correct me

GEMM uses assembly or compiler intrinsics, compiler scheduler is bypassed.

There is no CPUID flag to tell if CPU has one or two AVX512 units, in former case there is no convincing performace gain from AVX-512 over AVX2 on same core.

There is no CPUID flag to tell if CPU has one or two AVX512 units, in former case there is no convincing performace gain from AVX-512 over AVX2 on same core.

To the question of notable performance gains, please see attached tests from opening issue: https://openbenchmarking.org/result/2111077-TJ-ALDERAVX567
Or if preferred here's a battery of test from my own current testing environment: personal link to open bench marking
The P cores themselves on the chip have been confirmed by Intel to be the same Golden Cove core's as what is present on the upcoming Sapphire Rapids.
This includes the branch predictor, the amount of ALU's, the AVX units etc.
The Unofficial word from intel for the disabling of AVX-512 on the Alder Lake platform when E-cores are enabled is to to the Gracemount cores not supporting the same extended ISA.
In turn, due to the current generation OS schedulers and the hardware thread director not being up to the task of being able to direct programs to only run certain instruction (such as AVX-512 in our case) on certain cores/threads, the easier solution was taken.
The Solution was to blunt force disable the cores to be able to enable the extended instructions and therefore bypass the thread director, and then fall back to "normal" hardware assisted scheduling.

I plan to merge the AVX512-specific autodetection change I outlined earlier, but I'm not convinced of the need for a specific build TARGET (at least until the small gap in supported instruction sets w.r.t SapphireRapids becomes relevant). (BTW of your battery of tests I assume the NumPy one will be the most relevant - assuming that it is using a NumPy built to use OpenBLAS ?)

https://www.phoronix.com/scan.php?page=article&item=rocket-lake-avx512&num=2 lc0 is OpenBLAS, avx512 advantage does not justify less cores.

The linked test are of the 8 core 11900k from the rocket lake generation with AVX-512 equivalent to the icelake xeons.  That is not the architecture in question.
If the idea is illustration/comparison against the 10th series comet lake 10900k(s)  where we went from 10 cores with AVX-2 only and how that was detrimental in some workloads, I agree with you.
Where this differs is how, especially in the case of the i7, the ecore's can often be a net detriment to performance.
This is due to a few factors, excluding AVX-512.
The largest of these factors is that the ring bus on Alder Lake is driven by a net offset of the lowest clocked active core, but the ring is shared for both the ecores && pcores.
When ecores are enabled, the ring bus, which drives Dram, L3 cache and other shared subsystems is downclocked relative to them.
Typically this means the ring will run at no higher than 3.2 GHz.
When disabled, the ring clocks offset to the lowest clocked P core, with boosting frequencies typically closer to 4.8-4.9 GHz.
It also free's up the shared L3 cache normally given priority to the e-cores, increasing the effective amount of per core cache available.
Beyond the above, there's the issue of modern schedulers and iterative solvers not being built to deal with  vastly different per core performance on the same host.
On a different distributed Host, sure, but when core A, a P core with SMT, can compute certain tasks 40+% faster than the core L, an ecore without SMT, but the solvers scheduler doesn't know to add different weights per queue per core, you end up with a net loss in performance, even if on the i7 you're "dropping" from 12 cores (8p+4e) to 8p.
(Then comes the question of, since we have AVX-512 built into the chip, and there's a net benefit, why not enable it?)
Now, as for the specifics of the target being aliased to Sapphire Rapids, it comes down to SR and Alder Lake both being built on the same Golden Cove core. In the case of SR it's the only core on board, whereas for AL it's used for the P-core.
This means that any core architecture specific optimizations found for SR should also then be portable to AL.
On the GCC, kernel and LLVM mailing lists amongst other we've already seen some optimizations come in that are identical in code paths for AL and SR for all non AVX related maters.
Any code that is targeted for and has specific SR AVX-512 optimizations courtesy of Intel provides a net executions time benefit then when left to use default instruction cost tables or falling back on arch=icelake/canonlake etc.
In short, as for most optimized libraries, I want to have an option available for workstation users that know what they have available to them to squeeze every last drop of performance out of their system prior to sending to cluster
Beyond an auto detection mechanism, the request for an alias for SR in specific is to guarantee the use of the SR optimized  code paths and instruction cost tables compiler side while building. Over time, if/when SR specific optimizations are contributed, those few of us running an AVX-512 AL configuration would see the benefit.
Part of the reason for this is to preview and begin optimizing different suites for the new core architecture of SR, using AL as an analogue to do so while we await its release to market near the end of q2.
I'm more than happy to build and serve as a guinea pig if that's deemed useful

Why the need for an alias, rather than let the user specify TARGET=SAPPHIRERAPIDS (or not specify a TARGET and have the build script figure out on its own that it is appropriate to build for SR) ? Do you expect or know the cost tables to be dramatically different, requiring -march=alderlake rather than sapphirerapids ?

Why the need for an alias, rather than let the user specify TARGET=SAPPHIRERAPIDS (or not specify a TARGET and have the build script figure out on its own that it is appropriate to build for SR) ? Do you expect or know the cost tables to be dramatically different, requiring -march=alderlake rather than sapphirerapids ?

The Issue I expect to see is that if and when OpenBLAS begins to implement AMX specific instructions into it's routines for accelerating things like int-8 and bf-16, as of now those would cause the creation of a binary that could not run on alderlake.
Alder lake and SR will support int-8 and bf-16 via AVX-512 targets, but SR is incorporating what intel is calling a "tile" on board which acts as a dedicated matrix engine via an on die interconnect. It's similar to the way that AMD has been connecting "Chiplets" to each other via their "infinity fabric".
The reason for wanting/needing to specify -march=sapphirerapids over -march=alderlake is that alder lake has no cost tables associated with AVX-512 at all, and will actually disable the instructions usage in GCC 11-12, Clang 11-14 and Intel ICC/ICX.
for now the work around for most of my testing has been to use
CFLAGS='-O3 -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -$ExtraCFlags'
CXXFLAGS='-O3 -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -$ExtraCXXFlags'
FFLAGS='-O3 -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -$ExtraFFlags'

or the equivalent for a given compiler.
Where -$ExtraCFlags for certain builds using GCC 11.2 or GCC12 pre-release on certain memory sensitive applications will be something along the lines of  -fallow-store-data-races -fgcse-las -fgcse-after-reload -fdevirtualize-at-ltrans -fdevirtualize-speculatively -fsched-spec-load-dangerous -fsched-spec-load -fsemantic-interposition -fgraphite-identity -floop-nest-optimize -ftree-loop-im -ftree-loop-ivcanon -fivopts -ftree-vectorize -flto -fwhole-program -fuse-linker-plugin -funroll-loops  -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free -ltcmalloc_minimal -std=c17
to maximize throughput while also making use of the TCmalloc implementation of C malloc instead of that of GLIBC. (different topic but there seems to be some regressions in Glibcs' memory allocation sub routines and how they interact with heavily vectorized workloads across most architecture I've tested against)  (HOARD and JE malloc are also viable, but that's very much OT)

So -march the same as SR, and the only concern is a possible later addition of AMX to the current SR kernels ? In that case I think this can be handled if and when that happens - at that point there'd either be a need to split off the "pre-AMX" versions for Alder Lake, or guarding the additions with an ifdef HAVE_AMX. (Possibly the latter would be preferable, if Intel starts to market lower-end SR spin-offs with disabled AMX tiles (due to manufacturing defects or power budget) at some point)

So -march the same as SR, and the only concern is a possible later addition of AMX to the current SR kernels ? In that case I think this can be handled if and when that happens - at that point there'd either be a need to split off the "pre-AMX" versions for Alder Lake, or guarding the additions with an ifdef HAVE_AMX.

So for now keep as we are and use -SR?
That's fine by me.
Part of this is me trying to get ahead in some projects/libraries for weirdness in hardware that can/will be coming down the line including workstation based chips on the alder lake platform/socket that will not have AMX tiles but will otherwise present themselves as SR.
The designated chipset is W680 but not other information is currently public.

Right - with the (equivalent of the) above patch also applied to the runtime cpu detection, the avx512 Alder Lake will also be treated as SR in DYNAMIC_ARCH builds (as usually included with Linux distributions, NumPy, etc.) And I'll add the HAVE_AMX property (based on cpuid flags) for future use. Sometime this weekend...

Just dropping by. I also have a big-core only Alder Lake machine (i7-12700k with Asrock bios). When comparing gcc flags of native  vs sapphirerapids without amx, I found:
$ gcc -march=native -Q --help=target > gcc_target_native
$ gcc -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -Q --help=target > gcc_target_spr_no_amx
$ diff gcc_target_native gcc_target_spr_no_amx
12c12
<   -mabm                       		[enabled]
---
>   -mabm                       		[disabled]
27c27
<   -march=                     		alderlake
---
>   -march=                     		sapphirerapids
49c49
<   -mavx512vp2intersect        		[enabled]
---
>   -mavx512vp2intersect        		[disabled]
59c59
<   -mcldemote                  		[disabled]
---
>   -mcldemote                  		[enabled]
71c71
<   -menqcmd                    		[disabled]
---
>   -menqcmd                    		[enabled]
92,93c92,93
<   -mhle                       		[disabled]
<   -mhreset                    		[enabled]
---
>   -mhle                       		[enabled]
>   -mhreset                    		[disabled]
143c143
<   -mprefer-vector-width=      		none
---
>   -mprefer-vector-width=      		256
163c163
<   -msgx                       		[disabled]
---
>   -msgx                       		[enabled]
165c165
<   -mshstk                     		[enabled]
---
>   -mshstk                     		[disabled]
191c191
<   -mtsxldtrk                  		[disabled]
---
>   -mtsxldtrk                  		[enabled]
193c193
<   -mtune=                     		alderlake
---
>   -mtune=                     		sapphirerapids
195c195
<   -muintr                     		[disabled]
---
>   -muintr                     		[enabled]
202c202
<   -mwbnoinvd                  		[disabled]
---
>   -mwbnoinvd                  		[enabled]

This suggests that on top of AMX there are more instruction incompatibilities between sapphirerapids and alderlake. It seems to suggest we should use
-march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mabm -mavx512vp2intersect -mno-cldemote -mno-enqcmd -mno-hle -mhreset -mno-sgx -mshstk -mno-uintr -mno-wbnoinvd

However, even with this following corrections, the -mhle inconsistency seems to be unfixable. -mno-hle fails to disable hardware lock elision if -march specifies a platform with HLE support. I'm not sure if it is by design in gcc or a gcc bug, please correct me if anyone know more about the implications of gcc HLE optimization.
$ gcc -march=sapphirerapids -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mabm -mavx512vp2intersect -mno-cldemote -mno-enqcmd -mno-hle -mhreset -mno-sgx -mshstk -mno-uintr -mno-wbnoinvd -Q --help=target > gcc_target_spr_edited
$ diff gcc_target_native gcc_target_spr_edited
27c27
<   -march=                     		alderlake
---
>   -march=                     		sapphirerapids
92c92
<   -mhle                       		[disabled]
---
>   -mhle                       		[enabled]
143c143
<   -mprefer-vector-width=      		none
---
>   -mprefer-vector-width=      		256
193c193
<   -mtune=                     		alderlake
---
>   -mtune=                     		sapphirerapids

The above testing is done on a Gentoo Linux with big-core only i7-12700k and gcc version 12.3.1 20230526 (Gentoo 12.3.1_p20230526 p2) . If anyone is curious, the optimization flag I use for my setup was
-march=native -mtune=sapphirerapids --param=l1-cache-size=48 --param=l2-cache-size=25600

Which would fix the HLE issue.

AVX-512 is disabled completely by later microcode as reflected in public specs.
https://www.intel.com/content/www/us/en/products/sku/134594/intel-core-i712700k-processor-25m-cache-up-to-5-00-ghz/specifications.html

AVX-512 is disabled completely by later microcode as reflected in public specs.

True. I'm still using old microcode and the OpenBLAS AVX-512 patch for Alder Lake still works perfectly (and I hope it will continually so). I'm just replying to OP's compiler flag suggestions, which seems to leave some inconsistencies behind.

HLE needs to be supported in pthreads via alternative code path invisibly to OpenBLAS. It is not something that OpenBLAS could call directly. It could be left enabled for sake of completeness, but will not improve numeric calculation by any means.


Missing some translations

I can’t get it to find several translations in my template (although it finds some others). For example:

Your card will be charged { total } + VAT every month.

Your card will be charged { coupon_total } + VAT for the first month and then { subscription_amount } + VAT every month.

Your card will be charged { coupon_total } + VAT every month for the first { coupon_duration_in_months } months, and then { subscription_amount } + VAT every month.

Your card will be charged { amount } + VAT every month.

None of the above are found. However, I can match them just fine if I paste the HtmlDirective regex on regex101.com. So the regex works.
Any idea why it would skip them?

I managed to solve it by switching the order of attributes: if translate goes after translate-values, it works.
However, it should work in the original order, too. The regex allows any characters (except >) between translate attribute and the closing >. I can’t figure out why it doesn’t.


Terminal context Environment variables

Description of the new feature/enhancement
Numerous times I tried to automate some settings / profiles edits on the Terminal settings themselves, and I usually lack informations about the current tab environment :

What is the current loaded profile / scheme ?
Is the host the preview package ?

I'm sure there are also more info we could export to env vars based on people needs.
This would be veery useful to manage the terminal settings / themes / profiles directly from the command line !
Proposed technical implementation details (optional)
When opening a new tab, set:

$WT_CURRENT_PROFILE (Microsoft PowerShell, CMD, ...)
$WT_CURRENT_SCHEME
$WT_IS_PREVIEW (1 / 0, if the current terminal running package is the preview one)

This looks like it's roughly the same as the discussion that's taken place in #4566, so I'm gonna forward this discussion to that thread. Thanks!
/dup #4566


CMYK JPEGs cannot be decoded

Steps to reproduce

Open a page with CMYK JPEGs
See infinite loading wheel

Expected behavior
The image should successfully load.
Actual behavior
The image doesn't load, and an infinite loading wheel is shown instead.
Crash logs
No response
Mihon version
Mihon Preview r6458
Android version
Android 14
Device
Google Pixel 7
Other details
Minimal local manga entry for reproduction: CMYK JPEG example.zip
Possibly relevant logcat output:
2024-01-27 16:29:53.840 29705-1580  ImageDecoder            app.mihon.debug                      E  Unsupported color conversion request
2024-01-27 16:29:53.840 29705-1580  Subsamplin...eImageView app.mihon.debug                      E  Failed to decode tile
                                                                                java.lang.RuntimeException: Null region bitmap
                                                                                at com.davemorrissey.labs.subscaleview.decoder.ImageDecoder.decodeRegion(Unknown Source:21)
                                                                                at com.davemorrissey.labs.subscaleview.SubsamplingScaleImageView$TileLoadTask.doInBackground(SourceFile:9)
                                                                                at com.davemorrissey.labs.subscaleview.SubsamplingScaleImageView$TileLoadTask.doInBackground(SourceFile:1)
                                                                                at android.os.AsyncTask$3.call(AsyncTask.java:394)
                                                                                at java.util.concurrent.FutureTask.run(FutureTask.java:264)
                                                                                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                                                                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                                                                at java.lang.Thread.run(Thread.java:1012)

Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have gone through the FAQ and troubleshooting guide.
[X] I have updated the app to version 0.16.1.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

Should potentially be fixed by this upstream pr https://github.com/tachiyomiorg/image-decoder/pull/10


max length attribute for spacy splitter for large docs

For large size  documents spacy splitter doesn't work it throws an error as shown in  below screenshot.
Reason its default max_length is 1000000 and there is no option to increase it. So i added it in this PR.

@baskaryan kindly merge this PR.


scrollbar not working on dynamically created divs

Hi! I create my divs using a json dynamically, just wondering the best way to use this, thanks!
I tried:
`
`
document.querySelector(".div_class").setAttribute("ss-container", ""); SimpleScrollbar.initAll();
No errors. The attribute is added. Everything loaded correctly. But the divs display the regular browser scrollbar.
Also, what's the best way to use this on divs dynamically inserted into a page after page creation?
Thanks! Looks great, If I can get it to work on dynamically created divs.

Note github ate my code text. =) But you get the gist, how do I use this for dynamically created divs?

From README's Manual Binding:

Will give it a try. much thx! And report back. thx!
Question...will this init all of the dynamically pre-created divs at once? Or only just the 1 element at a time.
Because:
document.querySelector('.myClass');
Will only get the first element.
Because...
I used .initAll();
Because the Read Me stated it was for using if you have multiple dynamically precreated elements to initialize all at one time.
I used:
document.querySelector(".div_class").setAttribute("ss-container", ""); SimpleScrollbar.initAll();
Because it was suggested for multiple dynamically created divs already added on the page.
So let's say I have 30 precreated dynamically created divs I want scrollbars for, best way to init them all at once? Or will I need to init one element at a time? I can, I guess. Much Thanks! And for a cool scrollbar kit!

Bump...
So let's say I have 30 precreated dynamically created divs on a page I want SimpleScrollbars for, best way to init them all at once? Much Thanks! And for a cool scrollbar kit!


Optimize rendering runs of spaces when there is no visual change

cmatrix is somewhat of a pathological case for our infrastructure: it
prints out a bunch of green and white characters and then updates them a
million times a second.
It also maintains a column of space between every green character. When
it prints this column, it prints it in "default" or "white".  This impacts GDI, Direct2D and
ConPTY. In the example above, there are 120 runs.
The problem is, printing a space doesn't use the foreground color!
This commit introduces an optimization. When we're about to break a text
cluster because its attributes changed, we make sure that it's not just
filled with spaces and differs only in the foreground parameter.
This lets us optimize both the rendering and the PTY output to look
like this:
G*   *   *   *   *   *   *  B*G
G*   *   *   *   *   *   *
G*   *   *  B*G  *   *   *
G*  B*G  *       *   *   *   *
G*       *       *  B*G  *   *
B*G      *       *      B*G  *
G        *       *          B*G
G       B*G      *

Text will be printed at best line-by-line and at worst only when the
visible properties of the screen actually change. In the example
above, there are only 21 runs.
This speeds up cmatrix remarkably.
PR Checklist

[x] Related to #1064
[x] CLA signed
[ ] Tests added/passed (I haven't run these yet.)
[ ] Requires documentation to be updated
[x] Core contributee

I haven't had a chance to try your patch, but I suspect the reverse video attribute might cause problems with this optimisation. Consider the difference between this sequence...
printf "\e[31m     \e[32m     \e[m"

and this...
printf "\e[7;31m     \e[32m     \e[m"

In both case it's just a run of spaces with a foreground change, but the second sequence would actually require a cluster split wouldn't it?
And if that is the case, I assume the DECSCNM mode would cause similar problems.

YES, thank you. That's very helpful.

It won't change the mergeability of this PR, but would you [request changes] because of that?
Thanks!

So, an attribute knows whether it's been reversed. This is fine for reverse video, but not for global DECSCNM. I'd hate to round trip through the thing that knows the colors just to make this determination...

Regarding the DECSCNM mode, remember that that's a global attribute, so you should only need to look it up once at the start of the paint. If you can't access the global settings from the renderer, then I suppose you'd have to pass in a bool somewhere along the line. Otherwise maybe we should be updating that state in the renderer itself every time the mode is changed?

This works for both normal and invert video with both yes and no DECSCNM

I'm clearing my approval in response to @j4james's comment that happened while I was reviewing

hate me

oh heck no :smile: i love it

Okay, this is as comprehensive as I think I can make it -- i've renamed the function to compensate.

My most recent test case:
printf "\e[31m     \e[32m     \e[m\n\e[7;31m     \e[32m     \e[m\n\e[4;31m     \e[32m     \e[m\n\e[21;31m     \e[32m     \e[m\n\e[9;31m     \e[32m     \e[m\n"

FG1, FG2
Inverse video, FG1, FG2
Underline, FG1, FG2
Double underline, FG1, FG2
Strikethrough, FG1, FG2

I then toggled DECSCNM on and off with that in the buffer.

I'm not sure how you could write a test for that, which is why I'm okay with this not being tested. We don't really have a RendererTest with a mock engine or anything like that (unfortunately)

@msftbot make sure @miniksa signs off on this

Ugh, I explicitly code-formatted this.

Oh, I see. Unformatted code merged into master. Ugh.

/azp run

cannot clear olf dtatus


Document what --memory-swap=0 means

File: engine/admin/resource_constraints.md, CC @mstanleyjones
The latest docs https://docs.docker.com/engine/admin/resource_constraints/#/memory-swap-details do not describe the effect of using --memory-swap=0. The default is -1 which means unlimited, and using a positive integer (i.e. >1) is defined.
Does 0 mean no swap will be allowed, or do I have to set --memory and --memory-swap to the same value to accomplish that?
Filing here as recommended in https://github.com/docker/docker/issues/30716.

I'll take this one up!

Points to @joeuo


android version

my dear friends,
do you know about an android version of such fantastic server?
i really want one !

I expect there are many http servers on Android. For example, this one: https://gist.github.com/komamitsu/1893396


new:usr: revised paper 1, paper 2

I dont understand why report.pdf is generating merge conflicts.

you have not rebased. I can not merge as our system does not allow this. you need to remo e the co nflict yourself


Queue is not fully cleared on stop command

There is always one packet remaining in the buffer.

Let it be known that I committed first.


Do not translate arguments in add_subparsers

add_subparsers is typically called before Gooey initialization, when translations are not loaded.

Anyways, "Please fill out the issue template"

This is a pull request, not an issue. There was no issue template when submitting the pull request.

Whoops! Right you are. When I looked at it this morning, I mistook it as a single line issue report, which is a thing that ranks high on my open source annoyances list. Sorry about that!
Will look at this in a bit.

Merged here: https://github.com/chriskiehl/Gooey/commit/4ef1496a45f3f2e7cf46dcb68a0bc57abfbdb159


Calling go after pop using GoRouter with a StatefulShellRoute breaks navigation

Steps to reproduce

Use provided sample code
Launch App
Tap on "Page B"
Tap on "Details…" button
Tab on "Home" button
Tab on "Page B"
Tab on "Details…" button

Expected results
I would expect the details page to appear again after step 7.
Actual results
No navigation occurs. The app stays on "Page B".
Code sample
Code sample
import 'dart:async';

import 'package:flutter/material.dart';
import 'package:go_router/go_router.dart';

void main() {
  runApp(const MyApp());
}

final GoRouter routerConfig = GoRouter(
  initialLocation: '/a',
  debugLogDiagnostics: true,
  routes: [
    StatefulShellRoute.indexedStack(
      builder: (_, __, StatefulNavigationShell navigationShell) =>
          HomePage(navigationShell: navigationShell),
      branches: [
        StatefulShellBranch(
          routes: [
            GoRoute(
              path: '/a',
              builder: (context, state) => const PageA(),
            )
          ],
        ),
        StatefulShellBranch(
          routes: [
            GoRoute(
              path: '/b',
              builder: (context, state) => const PageB(),
              routes: [
                GoRoute(
                  path: 'detail',
                  builder: (context, state) => const DetailPage(),
                )
              ],
            ),
          ],
        ),
      ],
    )
  ],
);

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp.router(
      title: 'Test',
      routerConfig: routerConfig,
    );
  }
}

class HomePage extends StatelessWidget {
  final StatefulNavigationShell navigationShell;

  const HomePage({
    super.key,
    required this.navigationShell,
  });

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: navigationShell,
      bottomNavigationBar: BottomNavigationBar(
        items: const [
          BottomNavigationBarItem(icon: Icon(Icons.home), label: 'Page A'),
          BottomNavigationBarItem(icon: Icon(Icons.person), label: 'Page B'),
        ],
        onTap: (value) {
          navigationShell.goBranch(value);
        },
        currentIndex: navigationShell.currentIndex,
      ),
    );
  }
}

class PageA extends StatelessWidget {
  const PageA({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Page A'),
      ),
      body: const Center(
        child: Text('…'),
      ),
    );
  }
}

class PageB extends StatelessWidget {
  const PageB({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Page B'),
      ),
      body: Center(
        child: Column(
          mainAxisSize: MainAxisSize.min,
          children: [
            const Text('…'),
            ElevatedButton(
              child: const Text('Details…'),
              onPressed: () {
                GoRouter.of(context).go('/b/detail');
              },
            )
          ],
        ),
      ),
    );
  }
}

class DetailPage extends StatelessWidget {
  const DetailPage({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Detail'),
      ),
      body: Center(
        child: Column(
          mainAxisSize: MainAxisSize.min,
          children: [
            const Text('…'),
            ElevatedButton(
              child: const Text('Home'),
              onPressed: () {
                final router = GoRouter.of(context);
                router.pop();
                router.go('/a');
              },
            )
          ],
        ),
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[GoRouter] Full paths for routes:
           └─ (ShellRoute)
             ├─/a (PageA)
             └─/b (PageB)
               └─/b/detail (DetailPage)
[GoRouter] setting initial location /a
2
[GoRouter] Using MaterialApp configuration
D/EGL_emulation(18322): app_time_stats: avg=252.32ms min=11.35ms max=8030.31ms count=34
D/EGL_emulation(18322): app_time_stats: avg=1745.91ms min=1745.91ms max=1745.91ms count=1
[GoRouter] going to /b
[GoRouter] Using MaterialApp configuration
D/EGL_emulation(18322): app_time_stats: avg=28.74ms min=7.09ms max=403.04ms count=35
[GoRouter] going to /b/detail
W/WindowOnBackDispatcher(18322): OnBackInvokedCallback is not enabled for the application.
W/WindowOnBackDispatcher(18322): Set 'android:enableOnBackInvokedCallback="true"' in the application manifest.
D/EGL_emulation(18322): app_time_stats: avg=50.57ms min=5.31ms max=850.70ms count=25
[GoRouter] popping /b/detail
[GoRouter] going to /a
D/EGL_emulation(18322): app_time_stats: avg=58.27ms min=7.41ms max=848.80ms count=20
[GoRouter] restoring /b/detail
D/EGL_emulation(18322): app_time_stats: avg=31.18ms min=12.00ms max=582.96ms count=39
[GoRouter] going to /b/detail

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.24.0, on macOS 15.1 24B83 darwin-arm64, locale en-US)
    • Flutter version 3.24.0 on channel stable at /Users/dirkmika/projects/SDKs/Flutter/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 80c2e84975 (3 months ago), 2024-07-30 23:06:49 +0700
    • Engine revision b8800d88be
    • Dart version 3.5.0
    • DevTools version 2.37.2

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/dirkmika/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16A242d
    • CocoaPods version 1.15.2

[✗] Chrome - develop for the web (Cannot find Chrome executable at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[✓] Android Studio (version 2023.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)

[✓] VS Code (version 1.95.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.98.0

[✓] Connected device (5 available)
    • sdk gphone64 arm64 (mobile)     • emulator-5554                        • android-arm64 • Android 14 (API 34) (emulator)
    • Mein 13 Pro (mobile)            • 00008110-000A34300C62801E            • ios           • iOS 18.1 22B83
    • iPhone 15 Pro (mobile)          • ED1976BA-F0CC-4A90-BE74-25A9E49D110D • ios           • com.apple.CoreSimulator.SimRuntime.iOS-17-2 (simulator)
    • macOS (desktop)                 • macos                                • darwin-arm64  • macOS 15.1 24B83 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad                • darwin        • macOS 15.1 24B83 darwin-arm64

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

@just-doit
Can you check below issues to see if they resemble your case or not ?
https://github.com/flutter/flutter/issues/145401
https://github.com/flutter/flutter/issues/153934

To me, this looks like different problems.

Thanks for the update. Seeing the same behavior on latest versions.
stable : 3.24.4
master : 3.27.0-1.0.pre.412


osm2pgsql: 1.4.2 -> 1.5.0

Motivation for this change
Update to latest upstream release 1.5.0
Change log: https://github.com/openstreetmap/osm2pgsql/releases/tag/1.5.0
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 127701 run on x86_64-linux 1

1 package built:

osm2pgsql

Builds, runs, macos 10.15


Fix executeModel with POST method

Description
Issue: #524
executeModel was not using opts. Attributes method and body were not being passed on fetch call.
Type of change

Fix

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

7 of 7 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 71.705%

Totals

Change from base Build<PHONE_NUMBER>:
0.007%

Covered Lines:
1708

Relevant Lines:
2226

💛  - Coveralls


missing texture when importing with blender importer

Godot version
4.0 beta
System information
Windows 10
Issue description
if I have a texture assigned to mesh in blender 3.1 and I use Godot's blender importer the texture disappears. The information about roughness, glossiness ... of texture remains but image texture is gone.
Steps to reproduce
In blender assign texture to a mesh. Ui unwrap the mesh by using u then click reset.  assign the image texture. go to Godot 4 new project, set the blender importer folder. then drag the blend file in to the editor. the image texture is missing.
Minimal reproduction project
https://drive.google.com/file/d/1kqZnvaXdAIdPnD66zBS-1_RQt8p-uodw/view?usp=sharing

@Nikuto13 Please upload a minimal reproduction project to make this easier to troubleshoot.

Not a Godot bug, it's a bad file. The reason the texture is missing is because the Blend file does not include the texture. The Blend file did store a preview for it, so I can still see it in the inspector, but the complete texture is missing.

You need to click "Pack Resources" in Blender to ensure the texture is stored inside the Blend file:

If you are still having issues, you need to attach a new minimal reproduction project that displays correctly in Blender.


Discrepancy in database's run_sql() method

Please describe the feature you'd like to see
Currently, the handler param in the run_sql() method is only used in data bricks all other databases are not using it, which leads to different response types since the handler is processed within run_sql() for data bricks and not for other databases. Also, the signature of run_sql() in data bricks deviates from the base. We need to standardize and when we do, we can remove the below check as well.
Describe the solution you'd like
Ideally, we should have the same signatures for all the run_sql() database implementations. This will prevent adding explicit conditions for data bricks in other parts of the code that should ideally be database agnostic.
example: https://github.com/astronomer/astro-sdk/blob/2f8c8d0ccbff3cc13af78685cbefa39f473991c3/python-sdk/src/astro/sql/operators/raw_sql.py#L47
Acceptance Criteria

[ ] Where ever we have used run_sql(), we shouldn't be having any database-specific checks.

This PR is still a work in progress. This requires a return statement to return the cursor object so that its consistent across all the databases. @rajaths010494 mentioned he would pick this up after other tickets are done for 1.5.1.

@rajaths010494 has set up the meeting today. He will provide more details on this ticket.

run_sql we would be using get_wrapped_handler to get results for other databases and make necessary other changes for tests


I tried running vs code with "code --disable-extensions", seems nothing helps.
 It looks like you may be using an old version of VS Code, the latest stable release is 1.77.1. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

Upgrade to the 1.77.1 or downgrade the issue still exist

Can you check if the issue happens with our insiders build https://code.visualstudio.com/insiders that comes with a newer version of the runtime, thanks!

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Sort label isn't used

Currently Sort label isn't used. Either should be used or be removed.

I think the real label is defined in the form or other point of input or output of data.
There will be enough aliases / names of fields, which we use when sorting.

Let's remove it then.


add wikipedia biography dataset

My first PR containing the Wikipedia biographies dataset. I have followed all the steps in the guide. It passes all the tests.

Does anyone know why am I getting this "Some checks were not successful" message? For the code_quality one, I have successfully run the flake8 command.

Ok, I need to update the README.md, but don't know if that will fix the errors

Hi @ACR0S , thanks for adding the dataset!
It looks like black is throwing the code quality error: you need to run make style with the latest version of black (black --version should return 20.8b1)
We also added a requirement to specify encodings when using the python open function (line 163 in the current version of your script)
Finally, you will need to add the tags and field descriptions to the README as described here https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card
Let us know if you have any further questions!

Also, please leave the full template of the readme  with the [More Information Needed] paragraphs: you don't have to fill them out now but it will make it easier for us to go back to later :)

Thank you for your help, @yjernite! I have updated everything (finally run the make style, added the tags, the ecoding to the open function and put back the empty fields in the README). Hope it works now! :)

LGTM!

merging since the CI is fixed on master


[18.03] Backport Signal-Desktop

Motivation for this change
Reason: Signal-Desktop displayed the following message: "This version of Signal Desktop has expired. Please upgrade to the latest version to continue messaging." (see #48436).
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: signal-desktop
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: signal-desktop
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: signal-desktop
Partial log (click to expand)
copying path '/nix/store/109rigjz7m9xkqrxy2igfv5ndc76ik7a-cairo-1.14.10' from 'https://nix-cache.s3.amazonaws.com'...
copying path '/nix/store/mngx0as54sc57rwwgwvyxpdzc8h66hlg-libxkbcommon-0.7.2' from 'https://nix-cache.s3.amazonaws.com'...
copying path '/nix/store/s96v1m6zjphfq3lpkfk7lfg2qcm15fbr-pango-1.40.14' from 'https://nix-cache.s3.amazonaws.com'...
copying path '/nix/store/qhymj46c3vb5sd5kcc182cavdab26vg5-gtk+3-3.22.28' from 'https://nix-cache.s3.amazonaws.com'...
copying path '/nix/store/xpcvlwv1rhi8j3vj8dxhvx1mjnh00irx-librsvg-2.42.2' from 'https://nix-cache.s3.amazonaws.com'...
copying path '/nix/store/iv86gayqsx3j6751yiksj3jcwij2prlb-hook' from 'https://nix-cache.s3.amazonaws.com'...
building '/nix/store/kjxvnam9y9wlim608lz811jra9qv2gp7-signal-desktop-1.16.2.drv'...
unpacking sources
installing
/nix/store/j1v8032aq2v5pgr0haxybjbrlpxv4sxd-signal-desktop-1.16.2


See, which mail addresses didn't resolve token yet

As administrator
I want to see which email address didn't resolve the token yet
so that I'm able to contact the person and ask him to vote to accelerate, optimize voting time.
AC:
GIVEN I received the email with the admin link
WHEN I click on the link and open the admin page
THEN I can see a list of email addresses that haven't resolved their tokens yet.

That completely conflicts the core design decision to have absolutely no connection between token and voter identity. This is simply not possible without compromising anonymity.


Support spaces in filenames

Thanks for the tool. When I used it, it had some issues with spaces in input filenames.
By the way, what is the license for scuesplit?

Good catch.
I added a license, see ec0914d


Update wavebox to 3.1.7

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

Thank you for the contribution. It was merged directly as 3f6d65bdac20eed6e9d161903c23eabba7d47577. Your contribution is still credited to you.


[Snyk] Security upgrade check-dependencies from 1.1.1 to 2.0.0

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to fix one or more vulnerable packages in the `npm` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

package.json

Vulnerabilities that will be fixed
With an upgrade:

Severity
Priority Score (*)
Issue
Breaking Change
Exploit Maturity

661/1000  Why? Recently disclosed, Has a fix available, CVSS 7.5
Uncontrolled resource consumption SNYK-JS-BRACES-6838727
Yes
No Known Exploit

661/1000  Why? Recently disclosed, Has a fix available, CVSS 7.5
Inefficient Regular Expression Complexity SNYK-JS-MICROMATCH-6838728
Yes
No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Commit messages

Package name: check-dependencies
    The new version differs by 55 commits.

03c8847 Tag 2.0.0
65d9ef5 Set Node.js requirement in package.json engines to >=18.3
4917ab0 Simplify the spawn logic
fc04cc8 Drop support for the callback interface
28257dd Tweak ESLint settings
dc16e8a Drop the bluebird devDependency
412337a Drop fs-extra & graceful-fs devDependencies
091279a Drop the findup-sync dependency
10ac9c5 Drop lodash.camelcase & minimist dependencies
35dce52 Update dependencies
2929ba7 Update tested Node.js versions
1f514eb Don't invoke `npm prune`, `npm install` is already enough
65107d2 Drop support for Bower & the `checkCustomPackageNames` option
f28ba1b Avoid `git://` URLs, they're no longer supported
8fdc6ad Limit allowed `packageManager` values
9809f13 Bump word-wrap from 1.2.3 to 1.2.4 (#58)
e522471 Bump semver from 7.3.8 to 7.5.2 (#57)
031416d Bump yaml from 2.2.1 to 2.2.2 (#56)
dff3ab9 Build: Fix running tests on Windows
062005f Build: Update the GitHub Actions config
ecf138f Build: Update dependencies
15c13b3 Build: Remove AppVeyor
89b9df0 Build: Update .gitattributes to files always use LF line endings
db8c172 Build: Tweak GitHub Actions workflow code style

See the full diff

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Uncontrolled resource consumption

Checkmarx One – Scan Summary & Details – 3bcd0612-7ca7-4f4a-aa19-fc1e6d8892eb
New Issues

Severity
Issue
Source File / Package
Checkmarx Insight

CVE-2022-25883
Npm-semver-7.3.5
Vulnerable Package

CVE-2023-26136
Npm-tough-cookie-2.5.0
Vulnerable Package

CVE-2023-37466
Npm-vm2-3.9.11
Vulnerable Package

CVE-2023-37903
Npm-vm2-3.9.11
Vulnerable Package

CVE-2023-42282
Npm-ip-1.1.9
Vulnerable Package

CVE-2023-46233
Npm-crypto-js-3.3.0
Vulnerable Package

CVE-2024-29180
Npm-webpack-dev-middleware-5.3.0
Vulnerable Package

CVE-2024-4067
Npm-micromatch-4.0.5
Vulnerable Package

CVE-2024-4067
Npm-micromatch-3.1.10
Vulnerable Package

CVE-2024-4068
Npm-braces-3.0.2
Vulnerable Package

CVE-2024-4068
Npm-braces-2.3.2
Vulnerable Package

Insecure_Storage_of_Sensitive_Data
/routes/likeProductReviews.ts: 17
Attack Vector

Insecure_Storage_of_Sensitive_Data
/routes/order.ts: 26
Attack Vector

Insecure_Storage_of_Sensitive_Data
/routes/order.ts: 138
Attack Vector

Insecure_Storage_of_Sensitive_Data
/routes/order.ts: 138
Attack Vector

Insecure_Storage_of_Sensitive_Data
/models/address.ts: 24
Attack Vector

Insecure_Storage_of_Sensitive_Data
/models/address.ts: 10
Attack Vector

JWT_No_Signature_Verification
/routes/verify.ts: 101
Attack Vector

JWT_No_Signature_Verification
/routes/verify.ts: 101
Attack Vector

JWT_No_Signature_Verification
/lib/insecurity.js: 84
Attack Vector

Absolute_Path_Traversal
/routes/vulnCodeFixes.ts: 72
Attack Vector

Absolute_Path_Traversal
/routes/vulnCodeFixes.ts: 72
Attack Vector

CVE-2023-44270
Npm-postcss-7.0.39
Vulnerable Package

CVE-2023-44270
Npm-postcss-8.4.5
Vulnerable Package

CVE-2024-21501
Npm-sanitize-html-1.4.2
Vulnerable Package

CVE-2024-28863
Npm-tar-4.4.19
Vulnerable Package

Unpinned Actions Full Length Commit SHA
/zap_scan.yml: 17
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/update-challenges-www.yml: 25
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/codeql-analysis.yml: 23
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/codeql-analysis.yml: 30
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/ci.yml: 338
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/codeql-analysis.yml: 28
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/release.yml: 72
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/update-news-www.yml: 20
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/lint-fixer.yml: 24
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/rebase.yml: 17
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/release.yml: 50
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Unpinned Actions Full Length Commit SHA
/release.yml: 109
Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps...

Angular_Usage_of_Unsafe_DOM_Sanitizer
/frontend/src/app/score-board/score-board.component.ts: 182
Attack Vector

Angular_Usage_of_Unsafe_DOM_Sanitizer
/data/static/codefixes/restfulXssChallenge_4.ts: 59
Attack Vector

Angular_Usage_of_Unsafe_DOM_Sanitizer
/data/static/codefixes/restfulXssChallenge_3.ts: 45
Attack Vector

Fixed Issues

Severity
Issue
Source File / Package

CVE-2022-38900
Npm-decode-uri-component-0.2.2

Passwords And Secrets - Generic Password
/users.yml: 176

Passwords And Secrets - Generic Password
/users.yml: 236

Passwords And Secrets - Generic Password
/users.yml: 115

Passwords And Secrets - Generic Password
/users.yml: 140

Passwords And Secrets - Generic Password
/users.yml: 149

Passwords And Secrets - Generic Password
/users.yml: 63

Passwords And Secrets - Generic Password
/users.yml: 248

Passwords And Secrets - Generic Password
/users.yml: 203

Passwords And Secrets - Generic Password
/users.yml: 166

Passwords And Secrets - Generic Password
/users.yml: 242

Passwords And Secrets - Generic Password
/users.yml: 88

Passwords And Secrets - Generic Password
/users.yml: 107

Passwords And Secrets - Generic Password
/users.yml: 212

Passwords And Secrets - Generic Password
/users.yml: 158

Passwords And Secrets - Generic Password
/users.yml: 123

Passwords And Secrets - Generic Password
/users.yml: 31

Passwords And Secrets - Generic Password
/users.yml: 3

Passwords And Secrets - Generic Secret
/users.yml: 150

Passwords And Secrets - Google OAuth
/default.yml: 57

CVE-2023-26115
Npm-word-wrap-1.2.3

Cx24228ad1-81fd
Npm-sanitize-html-1.4.2

Cx65603961-769c
Npm-debug-2.6.9

Host Namespace is Shared
/docker-compose.test.yml: 3

Host Namespace is Shared
/docker-compose.test.yml: 7

Networks Not Set
/docker-compose.test.yml: 7

Networks Not Set
/docker-compose.test.yml: 3

CVE-2023-2251
Npm-yaml-1.10.2


JSONified Rev2_step2 configs won't run thru GUI

I used the config_file_converter to transform Markram model CSVs to JSONs and tried to run them thru the GUI.
SyntaxError: EOL while scanning string literal
2019-10-03 12:33:45,688 django.server ERROR    "POST /simulate HTTP/1.1" 500 100542
2019-10-03 12:34:53,757 django.request ERROR    Internal Server Error: /simulate
Traceback (most recent call last):
File "/home/henhok/anaconda3/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/henhok/anaconda3/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/henhok/anaconda3/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, callback_args, **callback_kwargs)
File "/home/henhok/anaconda3/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(args, **kwargs)
File "/home/henhok/PycharmProjects/CxSystem2/CxGUI/CxFront/editor/views.py", line 54, in simulate
received_data = eval(list(request._get_post().keys())[0].replace('false','0').replace('true','1'))
File "", line 1
{"anatomy":{"params":{"runtime":"5second","device":"Python","sys_mode":"local","scale":"1","grid_radius":"210um","min_distance":"1um","output_path_and_filename":"/home/henhok/sim_results/step2_viagui.gz","connections_saving_path_and_filename":"#","connections_loading_path_and_filename":"#","number_of_process":16,"default_clock":"0.1ms","trials_per_config":1,"do_init_vms":1,"load_positions_only":0,"do_benchmark":0,"save_generated_video_input_flag":0,"multidimension_array_run":1,"profiling":0},"IN":[{"idx":0,"type":"VPM","number_of_neurons"

I can't run the CxSystem with markram config directly from CxSystem. I'm using Markram_config_file.csv and both Physiology_Parameter and Physiological_Parameters_for_calcium but I'm bombarded with errors and I'm not sure whether I should fix them or they are from wrong config.
Which configs should I use?

This is probably related to issue #24 , i.e. connections_loading_path_and_filename can't be commented out in the GUI and CxSystem crashes (even if you don't use the connectivity data)

Fixed in 8c4c19e635700093d025d68bb1e6b15698a3a56f


[FEATURE REQ] Remove internal setters for result classes like in MessageCountDetails

Library name
Azure.ResourceManager.ServiceBus 1.0.0
Please describe the feature.
I'm using this library in my code to get the message count details from my service bus.
I wrote a custom wrapper (called ServiceBusManagement) to extract this value per my needs which returns MessageCountDetails, let's call it GetMessageCountDetailsAsync.
I have a class (let's call it BusinessLogicClass) that uses this wrapper and I want to unit test the BL class.
For that I need to mock ServiceBusManagement.GetMessageCountDetailsAsync but the issue is that the setters are internals and this prevents me from mocking the return value.
var countDetails = new MessageCountDetails();

Gives:
Error	CS1729	'MessageCountDetails' does not contain a constructor that takes 0 arguments

https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/servicebus/Microsoft.Azure.ServiceBus/src/Management/MessageCountDetails.cs
I can work around by having a proxy class as suggested here https://stackoverflow.com/questions/34992582/c-sharp-mock-a-class-with-an-internal-property-setter but please consider removing the internal setters. Unless it's absolutely critical that only the lib will initiate them (which doesn't look like the case here) then it just makes working with lib less comfortable.
I assume you took a similar approach in other areas as well.

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

We'll provide a model factory method to generate model classes: https://github.com/Azure/autorest.csharp/issues/2112
CC: @ArthurMa1978 @ArcturusZhang

We'll provide a model factory method to generate model classes: Azure/autorest.csharp#2112 CC: @ArthurMa1978 @ArcturusZhang

That would be great

Close it as this is tracked by feature https://github.com/Azure/autorest.csharp/issues/2112


[WIP] SmoothQuant using tensor subclassing

Still WIP
The implementation of SmoothQuant with tensor subclassing (AffineQuantizedTensor) is similar to that of AWQ with the following differences:

SmoothQuant supports both static and dynamic quantization of activation while AWQ only uses dynamic
Matmul is computed in int8 instead of floating point (at least at op level)
The smoothing factor is calculated differently from the equalization scales of AWQ

Hi @jerryzh168  I added a new tensor subclass LinearActivationScaleQuantizedTensor to support x -> x/scale -> quantize x for torch.compile.
If I use LinearActivationQuantizedTensor, the x/sacle is done outside the class (by input_quant_func) and there is a dynamo error about scale during torch.compile. I guess it's because the scale tensors are not on the graph in this case. Putting the scale in the weight tensor solves the problem. And WeightTensorWithLinearActivationScaleMetadata does not quantize activation.
Do you have any concern adding this new class? Thanks.

Hi @jerryzh168 I added a new tensor subclass LinearActivationScaleQuantizedTensor to support x -> x/scale -> quantize x for torch.compile.
If I use LinearActivationQuantizedTensor, the x/sacle is done outside the class (by input_quant_func) and there is a dynamo error about scale during torch.compile. I guess it's because the scale tensors are not on the graph in this case. Putting the scale in the weight tensor solves the problem. And WeightTensorWithLinearActivationScaleMetadata does not quantize activation.
Do you have any concern adding this new class? Thanks.

I think in this case we should be composing WeightTensorWithLinearActivationScaleMetadata LinearActivationQuantizedTensor together, i.e.
weight = to_affine_quantized(float_weight, ...)
# this will quantize input
weight = to_linear_activation_quantized_tensor(weight)
# this will do x / scale
weight = to_weight_tensor_with_linear_activation_scale_metadata(weight)

in dispatch time, we first unwrap the outer most tensor subclass, which will be WeightTensorWithLinearActivationScaleMetadata, so we'll apply scale to activation, then LinearActivationQuantizedTensor, which will quantize activation, and then AffineQuantizedTensor
would this work?

I think in this case we should be composing WeightTensorWithLinearActivationScaleMetadata LinearActivationQuantizedTensor together, i.e.
weight = to_affine_quantized(float_weight, ...)
# this will quantize input
# use https://github.com/pytorch/ao/blob/c87cc9b7286a46e9dfc076fa2417eb9b64ccc807/torchao/quantization/weight_tensor_linear_activation_quantization.py#L13 for static quantization
weight = to_linear_activation_quantized_tensor(weight)  # dynamic quant
# this will do x / scale
weight = to_weight_tensor_with_linear_activation_scale_metadata(weight)

in dispatch time, we first unwrap the outer most tensor subclass, which will be WeightTensorWithLinearActivationScaleMetadata, so we'll apply scale to activation, then LinearActivationQuantizedTensor, which will quantize activation, and then AffineQuantizedTensor
would this work?
the naming for different tensor subclasses is a bit confusing right now I think, we should cleanup a bit later

It works. Thanks

Hi @jerryzh168 It's weird that if I add these lines https://github.com/pytorch/ao/blob/f595ed41b99685cc16fc480ca2218965bb812bed/torchao/kernel/intmm.py#L142C1-L146C1 to avoid overflow of float16, there will be a failure in test_spinquant.py, which does not use float16 at all. And there is another failure with CUDA nightly, whose log is cannot be loaded. And these failures cannot be reproduced in my local environment or on an AWS instance.
So, I have to remove these line and also remove tests for float16. If users try to run in fp16, they will get overflow as well. Do you have any suggestions? Thanks.

Hi @jerryzh168 It's weird that if I add these lines f595ed4/torchao/kernel/intmm.py#L142C1-L146C1 to avoid overflow of float16, there will be a failure in test_spinquant.py, but the test does not use float16 at all.

what is the test failure? is it possible to do the dtype conversion before calling int_scaled_matmul

And there is another failure with CUDA nightly, but its log cannot be loaded. And these failures cannot be reproduced in my local environment or on an AWS instance. So, I have to remove these line and also remove tests for float16. If users try to run in fp16, they will get overflow as well. Do you have any suggestions? Thanks.

I just saw the error, it is talking about some triton related things, I'll rerun first just to confirm

what is the test failure? is it possible to do the dtype conversion before calling int_scaled_matmul, e.g. before y_dot_scaled = int_scaled_matmul(tmp, w_vals_int8_t, x_scales.reshape(-1, 1)) in affine_quantized_tensor.py?

The error is results not all close. One element exceed the tolerance by a small amount. As for dtype conversion, I didn't make such changes in affine_quantized_tensor.py.

  E1018 18:45:40.023529 436 site-packages/torch/_inductor/runtime/triton_heuristics.py:475]     assert lhs.shape[1].value >= 32, "small blocks not supported!"
  E1018 18:45:40.023529 436 site-packages/torch/_inductor/runtime/triton_heuristics.py:475] AssertionError: small blocks not supported!

Thanks for the info. Maybe it's about shapes in the test case? I will modify and try again.

@jerryzh168 I tried to move the dtype conversion to affine_quantized_tensor.py and it worked. Now all checks pass. Thanks.

Hi @jerryzh168 I have updated this PR. Please take a look again. Thanks

BTW, I found torchao's observer behaves differently from pytorch's observer when running on cuda. torchao's observer has its self.min_val and self.max_val on the same device as input tensor but pytorch's observer always has them on cpu. Is that something that needs a fix? Thanks.

BTW, I found torchao's observer behaves differently from pytorch's observer when running on cuda. torchao's observer has its self.min_val and self.max_val on the same device as input tensor but pytorch's observer always has them on cpu. Is that something that needs a fix? Thanks.

I see, I feel min_val/max_val being in the same device as input makes more sense? or are you saying we should add an option here?

BTW, I found torchao's observer behaves differently from pytorch's observer when running on cuda. torchao's observer has its self.min_val and self.max_val on the same device as input tensor but pytorch's observer always has them on cpu. Is that something that needs a fix? Thanks.

I see, I feel min_val/max_val being in the same device as input makes more sense? or are you saying we should add an option here?

Oh, I thought you might want them to have the same behavior. It's alright if that is not an issue.


Shorten to assert_snapshot?

Feel free to shut this down, but when used frequently, assert_snapshot_matches or assert_display_snapshot_matches is a lot of characters (latter is 31, or ~40% of a full line).
Any interest in shortening to assert_snapshot / assert_display_snapshot / etc? There's no other viable comparison than a match / eq, so it could be OK to make that implicit.
(And we'd keep the existing as aliases)

Yeah, I think I would be in favor of this at this point. Those are quite long indeed.


Why no updates since many many months :( please add beta version update

Why no updates since many many months :( please add beta version update???
now windows 11 has new file manager but i love this please update or provide a alpha or beta test version please give us preview

Hello, you can join Files insiders and get access to the beta version. Ask on Discord.


IntelliJ Freezes when Indexing

Hi,
first of all I love this Plugin and your work.
But every plugin version after 1.3.12 freezes my IDE when indexing.
When I downgrade the plugin manually it works without issues.
Below are details about the issue.
IDE Version:

IntelliJ 2024.2 (Clean new install with just this plugin)
IntelliJ 2024.1.x (Old Install)

Plugin Settings:

Game:

Victoria 3 (Current game version 1.7.5)
Unsubscribed all Workshop mods
Removed all mods from \Paradox Interactive\Victoria 3\mod

Mod Projects Tested:

Complex mod with hundreds of files
Empty mod (Example)

descriptor.mod
.metadata/metadata.json

Behavior:

First Load after Plugin install:

Project Opens as normal in IntelliJ
Everything works
Plugin starts indexing
Still working
Indexing reaches its end (according to IDE progressbar in footer)
IDE freezes

Nothing is clickable
IDE has high CPU usage (>10% instead of ~1%)
No progress after hours (Tried leaving it running for 3 hours while doing something else)
IDE UI does not update and turns background grey after dragging it
Only way to close it is via Task Manager

Starting the IDE after killing it before:

IDE freezes instantly because it wants to finish indexing

Same behaviour as above

Other Projects (e. g. Java)

Works as normal

Regards,
Chris

I've been having a similar issues with HOI4. Either it hangs on two specific files or just freezes midway through loading a project
other projects seem to load perfectly fine

Your IDE may be reinstalled, but your IDE's cache and indices may not be invalidated, try:

Invalidate cache and restart
Produce the freeze log (when IDE freezes wait for a minute first kill second and restart 3rd, then should get the freeze log)

Ok I had previously tried invalidating cache (forgot to mention in my inital post), but I tried it again and behaviour keeps to be the same.
Also in my test again I noticed it always seems to get stuck at the same file:

threadDump-20240812-041001.txt

Nothing found from above freeze logs, could thread dump log be provided when IDE is freezing?
(use jstack -l <pid> command, pid of IDEA can be found in Profier tab)

Perhaps I should debug vic3 game directory by myself

@DragonKnightOfBreeze If you look at the logs you can see that _ParadoxScriptLexer is running. Also from my screenshot you can see that it is parsing log_settings_live.json when the UI last updated. Have you tried running that file (I attached the file, from the game folder) through your lexer by hand or putting it in another game folder to see what happens? Maybe your Paradox File Lexer runs into an infinite Loop parsing a JSON file.
Should it even try to parse a JSON file using your Paradox Script Lexer? Maybe that is the issue?

How about the memory have you given to IDEA? Normally, it should be 4G+ or even 8G+.
If IDEA's memory is not enough, various strange problems may be produced.

I have not configured any memory limit and my computer has 32gb ram

Then it may be necessary to configure

It was 4gb. I set it to 8gb.
Same behaviour.

Nothing found from above freeze logs, could thread dump log be provided when IDE is freezing? (use jstack -l <pid> command, pid of IDEA can be found in Profier tab)

Could you please provide a thread dump log by jstack when IDE is freezing?
You can follow this screenshot:

Hi,
attached is the log:
jstack.log

Simply typing @@@ in a script file will cause this issue (ide freeze, infinite loop in lexer)

Should be fixed in 1.3.18

Thank you!


@playlyfe/gql doesn't work with version of "graphql" > "0.11.7"

If the package.json use the latest version of graphql.js package, you have the following error:
[Info  - 10:42:06] NODE_PATH value is: xxxxxx
[Error - 10:42:08] Server initialization failed:
Request initialize failed with message: Cannot find module 'graphql/validation/rules/ArgumentsOfCorrectType'
Using "graphql": "0.11.7" fix the problem...

It should work with versions below 0.13. We are working on getting better support for all graphql versions soon.

I've tried @playlyfe/v3 which is supposed to work with<EMAIL_ADDRESS>but unfortunately there is this hard check in the plugin that effectively forbids it.
https://github.com/kumarharsh/graphql-for-vscode/blob/e52efb01732d46f6444bdcc577188e40cb6da5b8/src/server/server.ts#L68
Is there something I can help with on this? It's kinda surprising that with GraphQL being used more and more these super useful tools have lost interesting in maintaining them.

@kumarharsh I would definitely like to help to get this moving. The extension helped me a lot before, but our backend guys decided to use type Query extends Interface1 & Interface2 ... it doesn't work anymore because older graphql module does not understand these.
I guess the first step would be to figure out to use that compatible branch and have it loaded in the VSCode, shouldn't be that hard hopefully.

Ok, guys, I want to thank for all the hard work and the joy this extension has brought in the past 👏 It's time to move on with the awesome Apollo GraphQL that has emerged. It works out of the box without any complicated configuration and extra packages. I really recommend you to try it out.

@FredyC Sadly it doesn't support referencing a local schema consisting out of several files which I really need.

@levrik Then file the suggestion. Apollo is surely a more active community and there are good chances for that to happen.


add tmp and results

Hello,
setup.py has been fixed for issue #29
Regards.

OS: MacOS
OS Version: High Sierra
Python Version: 2.7.01

Hello,
Thanks for your contribution, PR Merged.
Best regards.


CLI: cicero archive not implemented

The Cicero CLI lists cicero archive as a command, but it is not implemented.

Addressed in PR #252


Failed to construct 'WebSocket': The URL's scheme must be either 'ws' or 'wss'. 'vscode-webview' is not allowed

VS Code Version: 1.66.0-insider
OS Version: MacOS 12.2.1

I try to develop VSCode webview with webpack + HMR. After configure the right CSP, it seems webpack-dev-server can't make a WebSocket Object.
Steps to Reproduce:

clone project https://github.com/tjx666/scripting-listener
install dependencies by npm install
startup webpack dev server for webview npm run start-web
open debugger panel, click Run Extension
go to Extension Development Host workspace
open webview panel by call command Open Scripting Listener Log Viewer
open devtools, will show following screenshot error message

I had configured the CSP to allow ws://localhost:3000, check details here:
 private setupHtmlForWebview() {
        const webview = this.panel.webview;
        const nonce = getNonce();
        const isProd = false;
        const localPort = 3000;
        const localServerUrl = `localhost:${localPort}`;
        const scriptRelativePath = 'webview.js';
        const scriptUri = isProd
            ? webview
                  .asWebviewUri(Uri.joinPath(this.extensionUri, 'build', scriptRelativePath))
                  .toString()
            : `http://${localServerUrl}/${scriptRelativePath}`;

        // prettier-ignore
        const csp = [
            `default-src 'none'`,
            `img-src ${webview.cspSource} 'self' 'unsafe-inline'`,
            `script-src ${isProd ? `'nonce-${nonce}'` : `http://${localServerUrl} http://<IP_ADDRESS>:${localPort}`} 'unsafe-eval'`,
            `style-src ${webview.cspSource} 'self' 'unsafe-inline'`,
            `font-src ${webview.cspSource}`,
            `connect-src ${isProd ? '' : `ws://${localServerUrl} ws://<IP_ADDRESS>:${localPort} http://${localServerUrl} http://<IP_ADDRESS>:${localPort}`}`
        ];
        const html = `<!DOCTYPE html>

        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <meta http-equiv="Content-Security-Policy" content="${csp.join('; ')}">
        <title>Scripting Listener</title>

`;
        this.panel.webview.html = html;
    }

Can you please try putting together a self contained example that clearly demonstrates how the web socket is constructed? I suspect the url for it is getting passed through asWebviewUri (either directly or being resolved relative to a script that was). As the error message says, this is not supported

I have fixed it.
ref:

https://github.com/webpack/webpack-dev-server/issues/4338#event-6250580841
https://github.com/tjx666/scripting-listener/commit/34ace884434794db21d4772c6712d3bfd3725de5


DialogsDemo Update

[X] VBox: same width buttons, 5px spacing
[X] Text in each dialog for demo
[X] Custom result converter, UsernamePassword
[X] showAndWait()

I'd say keep the PR open for now, I'll let you know when the remaining TODO is done. Cheers

@JonathanGiles all done, ready for review

Thanks. Very nice. I pushed some minor tweaks. If you have bandwidth, a demo with Spinner would be appreciated (again, HelloSpinner from OpenJFX can help here).


how to custom same Tag in jsp's page

i have defined in decorator
<sitemesh:write property="JavaScripts"/>
and i have defined a  page ,as follows
<JavaScripts>
    a......
</JavaScripts>

<JavaScripts>
    b......
</JavaScripts>

the result is
a....
but now, i need result is
    a......    and     b......
Is there any way to fix my problem？

Example:
https://github.com/sitemesh/sitemesh3/issues/99


Error running package tests

Suddenly I'm getting this error when running package tests:
W20180821-10:45:24.181(2)? (STDERR) The version of @babel/runtime installed in your node_modules directory
W20180821-10:45:24.198(2)? (STDERR) (7.0.0-rc.1) contains a breaking change which was introduced by
W20180821-10:45:24.199(2)? (STDERR) https://github.com/babel/babel/pull/8266. Please either downgrade by
W20180821-10:45:24.199(2)? (STDERR) running the following command:
W20180821-10:45:24.199(2)? (STDERR)
W20180821-10:45:24.199(2)? (STDERR)   meteor npm install --save-exact<EMAIL_ADDRESS>W20180821-10:45:24.199(2)? (STDERR)
W20180821-10:45:24.199(2)? (STDERR) or update to the latest beta version of Meteor 1.7.1, as explained in
W20180821-10:45:24.200(2)? (STDERR) this pull request: https://github.com/meteor/meteor/pull/9942.
W20180821-10:45:24.200(2)? (STDERR)
W20180821-10:45:24.200(2)? (STDERR) /tmp/meteor-test-run1sp8nu1.yyrk/.meteor/local/build/programs/server/boot.js:475
W20180821-10:45:24.200(2)? (STDERR) }).run();
W20180821-10:45:24.200(2)? (STDERR)    ^
W20180821-10:45:24.200(2)? (STDERR)
W20180821-10:45:24.201(2)? (STDERR) Error: Cannot find module './defineProperty'
W20180821-10:45:24.201(2)? (STDERR)     at makeMissingError (packages/modules-runtime.js:231:12)
W20180821-10:45:24.201(2)? (STDERR)     at require (packages/modules-runtime.js:241:19)
W20180821-10:45:24.201(2)? (STDERR)     at objectSpread.js (packages/modules.js:504:22)
W20180821-10:45:24.201(2)? (STDERR)     at fileEvaluate (packages/modules-runtime.js:343:9)
W20180821-10:45:24.201(2)? (STDERR)     at require (packages/modules-runtime.js:238:16)
W20180821-10:45:24.201(2)? (STDERR)     at logging.js (/tmp/meteor-test-run1sp8nu1.yyrk/.meteor/local/build/programs/server/packages/logging.js:26:45)
W20180821-10:45:24.202(2)? (STDERR)     at fileEvaluate (packages/modules-runtime.js:343:9)
W20180821-10:45:24.202(2)? (STDERR)     at require (packages/modules-runtime.js:238:16)
W20180821-10:45:24.202(2)? (STDERR)     at /tmp/meteor-test-run1sp8nu1.yyrk/.meteor/local/build/programs/server/packages/logging.js:511:15
W20180821-10:45:24.202(2)? (STDERR)     at /tmp/meteor-test-run1sp8nu1.yyrk/.meteor/local/build/programs/server/packages/logging.js:518:3

I've tried doing what is there, but it does not help.
Does anybody know anything about this?
Thanks in advance
Meteor 1.6.1
<EMAIL_ADDRESS>(but it happens in every version)

What is the version of @babel/runtime you're asking for in your packages.json file - and which version do you actually have installed?

I'm closing this due to lack of feedback. It also seems like it's a meteor-related error, unrelated to this package.

@SimonSimCity sorry for the delay, i didn't get the notification :sweat_smile:
Just to clarify, we updated Meteor and it helped. It was probably a combination between babel and meteor, but the weird part is that it started happening out of sudden (probably some weird package-lock.json issues)