Uncaught Error: Link Shader failed - Starting with Beta 11 - on Raspberry Pi 4

Version

Phaser Version: Phaser v3.60.0-beta.13 (WebGL | Web Audio)

Operating system: Ubuntu 22.04 Linux on Raspberry Pi 4

Browser: Chromium

Description

Immediately upon starting Phaser I see this error:
Phaser v3.60.0-beta.12 (WebGL | Web Audio)  https://phaser.io

phaser.js:168613 Uncaught Error: Link Shader failed:

     at WebGLRenderer.createProgram (phaser.js:168613:19)
     at new WebGLShader (phaser.js:169592:38)
     at MultiPipeline.setShadersFromConfig (phaser.js:164876:34)
     at MultiPipeline.boot (phaser.js:164688:14)
     at MultiPipeline.boot (phaser.js:171833:38)
     at PipelineManager.add (phaser.js:162819:22)
     at phaser.js:162682:30
     at Map.each (phaser.js:190491:17)
     at PipelineManager.boot (phaser.js:162680:22)
     at WebGLRenderer.boot (phaser.js:167515:25)

Clicking the link in the console in my browser for "phser.js:168613" brings me to this section of code:
if (!gl.getProgramParameter(program, gl.LINK_STATUS))
{
    throw new Error('Link ' + failed + gl.getProgramInfoLog(program));
}

Note that everything works fine on this same system using 3.60.0-beta.10
The error happens in Beta 11, Beta 12 and Beta 13.
Example Test Code

This happens with any code that loads phaser, but I copied the above error from a copy of the "Hello World" example that I made here:
Phaser 12: https://codepen.io/EkpyroticFrood/pen/NWMJzBa
Additional Information

I am definitely using WebGL and expecting WebGL to work on this platform, as it always has.
I know that the Raspberry Pi 4 isn't probably your highest priority, but as it worked fine up through Beta 10, I'm hoping this is a simple fix to get it working again before 3.60 release.
Firefox
I tried Firefox and the results are the same (works with Beta 10, does not work with Beta 11, 12 or 13), but I got one more line in the error output that might be useful:
Uncaught Error: Link Shader failed:
error: sampler arrays indexed with non-constant expressions is forbidden in GLSL ES 100

Update
Just adding a note that it also behaves the same in Beta 13, which isn't a surprise as I'm sure nobody has had time to look int it yet. This is mostly for my own benefit to track my testing.
If you have any testing you would like me to do let me know. I've done some digging, but I'm not smart enough to see what changes between Beta 10 and 11 relate to this issue. I'm definitely willing to try some suggestions and provide more testing output.

The solution is to install the latest version of Raspbian instead of Ubuntu on the Pi 4.
I wiped my SD card and installed the latest version of Raspbian on it, booted up my Pi 4, and Beta 13 works!
I cannot find anything different in the chrome://gpu info between the two, but clearly something is different with either the video driver itself, Chromium, or something else that helps Phaser run WebGL.
I'm going to go ahead and close this. While I'd like to make it work and/or know why it doesn't work on Ubuntu on the Pi 4, I certainly don't expect you to support an OS on a Single Board Computer that even the hardware maker doesn't directly support. If anything, this is an Ubuntu bug, not a Phaser bug.

i have same issue, i don't have this issue on system node, when i switch to nvm node i encounter exactly same error

@devops724
What OS do you have installed on the pi?
cat /etc/lsb-release
cat /etc/os-release
What browser are you using and what is the version?
Also, you say the node version affects it, but Phaser runs in the browser. Is this a build issue or a runtime issue?

i found this happen on chrome 114 only, in firefox i don't have issue
also after update to chrome 115 issue gone


Copy failure result in Input/output error and Read-only file system

Environment
Windows build number: Microsoft Windows [version 10.0.19041.685]
Your Distribution version: Ubuntu 20.04
Whether the issue is on WSL 2 and/or WSL 1: WSL 2

Steps to reproduce

Try to copy a big file from host filesystem to WSL filesystem (mine is 2.3G) with cp /mnt/c/User/test/bigfile /another/location
Command fails with cp: error writing '/another/location/bigfile': Input/output error
Attempt to create a file in /home or any other place result in Read-only file system (touch: cannot touch 'test': Read-only file system)

After a wsl --shutdown, file system is writable again, and performing the aforementionned steps create the problem again.
dmesg output:
[    0.000000] Linux version 4.19.128-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Tue Jun 23 12:58:10 UTC 2020
[    0.000000] Command line: initrd=\initrd.img panic=-1 pty.legacy_count=0 nr_cpus=5
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[5]:  832, xstate_sizes[5]:   64
[    0.000000] x86/fpu: xstate_offset[6]:  896, xstate_sizes[6]:  512
[    0.000000] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024
[    0.000000] x86/fpu: Enabled xstate features 0xe7, context size is 2432 bytes, using 'compacted' format.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000e0fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000001fffff] ACPI data
[    0.000000] BIOS-e820: [mem 0x0000000000200000-0x00000000f7ffffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000107ffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] Hypervisor detected: Microsoft Hyper-V
[    0.000000] Hyper-V: features 0xae7f, hints 0x20e24, misc 0x20bed7b2
[    0.000000] Hyper-V Host Build:19041-10.0-0-0.685
[    0.000000] Hyper-V: LAPIC Timer Frequency: 0x1e8480
[    0.000000] tsc: Marking TSC unstable due to running on Hyper-V
[    0.000000] Hyper-V: Using hypercall for remote TLB flush
[    0.000000] tsc: Detected 1497.602 MHz processor
[    0.000028] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000029] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000032] last_pfn = 0x108000 max_arch_pfn = 0x400000000
[    0.000058] MTRR default type: uncachable
[    0.000058] MTRR fixed ranges disabled:
[    0.000059]   00000-FFFFF uncachable
[    0.000060] MTRR variable ranges disabled:
[    0.000060]   0 disabled
[    0.000061]   1 disabled
[    0.000061]   2 disabled
[    0.000061]   3 disabled
[    0.000061]   4 disabled
[    0.000062]   5 disabled
[    0.000062]   6 disabled
[    0.000063]   7 disabled
[    0.000064] Disabled
[    0.000066] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[    0.000074] CPU MTRRs all blank - virtualized system.
[    0.000076] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC
[    0.000078] last_pfn = 0xf8000 max_arch_pfn = 0x400000000
[    0.000119] Using GB pages for direct mapping
[    0.000124] BRK [0x02e00000, 0x02e00fff] PGTABLE
[    0.000126] BRK [0x02e01000, 0x02e01fff] PGTABLE
[    0.000127] BRK [0x02e02000, 0x02e02fff] PGTABLE
[    0.000160] BRK [0x02e03000, 0x02e03fff] PGTABLE
[    0.000209] BRK [0x02e04000, 0x02e04fff] PGTABLE
[    0.000286] RAMDISK: [mem 0x02e35000-0x02e44fff]
[    0.000291] ACPI: Early table checksum verification disabled
[    0.000332] ACPI: RSDP 0x00000000000E0000 000024 (v02 VRTUAL)
[    0.000339] ACPI: XSDT 0x0000000000100000 000044 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[    0.000347] ACPI: FACP 0x0000000000101000 000114 (v06 VRTUAL MICROSFT 00000001 MSFT 00000001)
[    0.000361] ACPI: DSDT 0x00000000001011B8 01E184 (v02 MSFTVM DSDT01   00000001 MSFT 05000000)
[    0.000369] ACPI: FACS 0x0000000000101114 000040
[    0.000374] ACPI: OEM0 0x0000000000101154 000064 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
[    0.000379] ACPI: SRAT 0x000000000011F33C 000260 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
[    0.000385] ACPI: APIC 0x000000000011F59C 000070 (v04 VRTUAL MICROSFT 00000001 MSFT 00000001)
[    0.000397] ACPI: Local APIC address 0xfee00000
[    0.001232] Zone ranges:
[    0.001233]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.001234]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.001234]   Normal   [mem 0x0000000100000000-0x0000000107ffffff]
[    0.001235] Movable zone start for each node
[    0.001235] Early memory node ranges
[    0.001236]   node   0: [mem 0x0000000000001000-0x000000000009ffff]
[    0.001236]   node   0: [mem 0x0000000000200000-0x00000000f7ffffff]
[    0.001237]   node   0: [mem 0x0000000100000000-0x0000000107ffffff]
[    0.001417] Zeroed struct page in unavailable ranges: 353 pages
[    0.001418] Initmem setup node 0 [mem 0x0000000000001000-0x0000000107ffffff]
[    0.001419] On node 0 totalpages: 1048223
[    0.001420]   DMA zone: 59 pages used for memmap
[    0.001420]   DMA zone: 22 pages reserved
[    0.001421]   DMA zone: 3743 pages, LIFO batch:0
[    0.001475]   DMA32 zone: 16320 pages used for memmap
[    0.001476]   DMA32 zone: 1011712 pages, LIFO batch:63
[    0.023470]   Normal zone: 512 pages used for memmap
[    0.023472]   Normal zone: 32768 pages, LIFO batch:7
[    0.024146] ACPI: Local APIC address 0xfee00000
[    0.024153] ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
[    0.024385] IOAPIC[0]: apic_id 5, version 17, address 0xfec00000, GSI 0-23
[    0.024388] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.024390] ACPI: IRQ9 used by override.
[    0.024392] Using ACPI (MADT) for SMP configuration information
[    0.024393] smpboot: Allowing 5 CPUs, 0 hotplug CPUs
[    0.024400] [mem 0xf8000000-0xffffffff] available for PCI devices
[    0.024401] Booting paravirtualized kernel on bare hardware
[    0.024402] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns:<PHONE_NUMBER>2750000 ns
[    0.117289] random: get_random_bytes called from start_kernel+0x93/0x4be with crng_init=0
[    0.117296] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:5 nr_node_ids:1
[    0.118377] percpu: Embedded 42 pages/cpu s133400 r8192 d30440 u1048576
[    0.118398] pcpu-alloc: s133400 r8192 d30440 u1048576 alloc=1*2097152
[    0.118398] pcpu-alloc: [0] 0 1 [0] 2 3 [0] 4 -
[    0.118408] Built 1 zonelists, mobility grouping on.  Total pages: 1031310
[    0.118410] Kernel command line: initrd=\initrd.img panic=-1 pty.legacy_count=0 nr_cpus=5
[    0.118876] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.119209] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.135043] Memory: 4023428K/4192892K available (14360K kernel code, 1575K rwdata, 2856K rodata, 1512K init, 2756K bss, 169464K reserved, 0K cma-reserved)
[    0.135312] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=5, Nodes=1
[    0.135316] ftrace: allocating 41883 entries in 164 pages
[    0.147500] rcu: Hierarchical RCU implementation.
[    0.147501] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=5.
[    0.147502] 	All grace periods are expedited (rcu_expedited).
[    0.147503] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=5
[    0.149823] Using NULL legacy PIC
[    0.149825] NR_IRQS: 16640, nr_irqs: 464, preallocated irqs: 0
[    0.150244] Console: colour dummy device 80x25
[    0.150246] console [tty0] enabled
[    0.150250] ACPI: Core revision 20180810
[    0.150335] Failed to register legacy timer interrupt
[    0.150336] APIC: Switch to symmetric I/O mode setup
[    0.150350] Hyper-V: Using IPI hypercalls
[    0.150353] clocksource: hyperv_clocksource_tsc_page: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns:<PHONE_NUMBER>20 ns
[    0.150397] Calibrating delay loop (skipped), value calculated using timer frequency.. 2995.20 BogoMIPS (lpj=14976020)
[    0.150414] pid_max: default: 32768 minimum: 301
[    0.150428] Security Framework initialized
[    0.150439] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.150443] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.150745] x86/cpu: Activated the Intel User Mode Instruction Prevention (UMIP) CPU feature
[    0.150753] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.150754] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.150755] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.150756] Spectre V2 : Mitigation: Enhanced IBRS
[    0.150756] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.150773] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.150774] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.151047] Freeing SMP alternatives memory: 44K
[    0.151831] smpboot: CPU0: Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz (family: 0x6, model: 0x7e, stepping: 0x5)
[    0.151869] Performance Events: unsupported p6 CPU model 126 no PMU driver, software events only.
[    0.151885] rcu: Hierarchical SRCU implementation.
[    0.151998] random: crng done (trusting CPU's manufacturer)
[    0.152275] smp: Bringing up secondary CPUs ...
[    0.152307] x86: Booting SMP configuration:
[    0.152308] .... node  #0, CPUs:      #1
[    0.010189] x86/cpu: Activated the Intel User Mode Instruction Prevention (UMIP) CPU feature
[    0.152605]  #2
[    0.010189] x86/cpu: Activated the Intel User Mode Instruction Prevention (UMIP) CPU feature
[    0.152605]  #3
[    0.010189] x86/cpu: Activated the Intel User Mode Instruction Prevention (UMIP) CPU feature
[    0.152605]  #4
[    0.010189] x86/cpu: Activated the Intel User Mode Instruction Prevention (UMIP) CPU feature
[    0.152605] smp: Brought up 1 node, 5 CPUs
[    0.152605] smpboot: Max logical packages: 1
[    0.152605] smpboot: Total of 5 processors activated (14976.02 BogoMIPS)
[    0.152605] devtmpfs: initialized
[    0.152605] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns:<PHONE_NUMBER>2750000 ns
[    0.152605] futex hash table entries: 2048 (order: 5, 131072 bytes)
[    0.152605] xor: automatically using best checksumming function   avx
[    0.152605] NET: Registered protocol family 16
[    0.152605] ACPI: bus type PCI registered
[    0.152605] PCI: Fatal: No config space access function found
[    0.152605] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.152605] raid6: Forced to use recovery algorithm avx2x2
[    0.152605] raid6: Forced gen() algo avx2x4
[    0.152605] ACPI: Added _OSI(Module Device)
[    0.152605] ACPI: Added _OSI(Processor Device)
[    0.152605] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.152605] ACPI: Added _OSI(Processor Aggregator Device)
[    0.152605] ACPI: Added _OSI(Linux-Dell-Video)
[    0.152605] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.164361] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.165021] ACPI: Interpreter enabled
[    0.165021] ACPI: (supports S0 S5)
[    0.165021] ACPI: Using IOAPIC for interrupt routing
[    0.165021] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.165021] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.165045] SCSI subsystem initialized
[    0.165045] hv_vmbus: Vmbus version:5.0
[    0.165045] PCI: Using ACPI for IRQ routing
[    0.165045] PCI: System does not support PCI
[    0.165045] clocksource: Switched to clocksource hyperv_clocksource_tsc_page
[    0.165045] hv_vmbus: Unknown GUID: c376c1c3-d276-48d2-90a9-c04748072c60
[    0.171236] VFS: Disk quotas dquot_6.6.0
[    0.171246] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.171281] FS-Cache: Loaded
[    0.171305] pnp: PnP ACPI init
[    0.171443] pnp 00:00: Plug and Play ACPI device, IDs PNP0b00 (active)
[    0.171458] pnp: PnP ACPI: found 1 devices
[    0.177307] NET: Registered protocol family 2
[    0.177407] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes)
[    0.177411] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[    0.177431] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[    0.177460] TCP: Hash tables configured (established 32768 bind 32768)
[    0.177726] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[    0.177732] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[    0.177754] NET: Registered protocol family 1
[    0.178324] RPC: Registered named UNIX socket transport module.
[    0.178324] RPC: Registered udp transport module.
[    0.178325] RPC: Registered tcp transport module.
[    0.178325] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.178327] PCI: CLS 0 bytes, default 64
[    0.178667] Trying to unpack rootfs image as initramfs...
[    0.178936] Freeing initrd memory: 64K
[    0.178939] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.178940] software IO TLB: mapped [mem 0xf4000000-0xf8000000] (64MB)
[    0.179019] kvm: no hardware support
[    0.179020] has_svm: not amd
[    0.179021] kvm: no hardware support
[    0.179463] Initialise system trusted keyrings
[    0.179770] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    0.180424] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.180731] NFS: Registering the id_resolver key type
[    0.180734] Key type id_resolver registered
[    0.180734] Key type id_legacy registered
[    0.180736] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[    0.181439] Key type cifs.idmap registered
[    0.181590] fuse init (API version 7.27)
[    0.181921] SGI XFS with ACLs, security attributes, realtime, scrub, no debug enabled
[    0.182677] 9p: Installing v9fs 9p2000 file system support
[    0.182683] FS-Cache: Netfs '9p' registered for caching
[    0.182711] FS-Cache: Netfs 'ceph' registered for caching
[    0.182713] ceph: loaded (mds proto 32)
[    0.188126] NET: Registered protocol family 38
[    0.188129] Key type asymmetric registered
[    0.188130] Asymmetric key parser 'x509' registered
[    0.188142] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
[    0.188144] io scheduler noop registered (default)
[    0.188392] hv_vmbus: registering driver hv_pci
[    0.188522] ACPI: AC Adapter [AC1] (on-line)
[    0.188741] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.189047] Non-volatile memory driver v1.3
[    0.189173] battery: ACPI: Battery Slot [BAT1] (battery present)
[    0.191021] brd: module loaded
[    0.191770] loop: module loaded
[    0.191872] hv_vmbus: registering driver hv_storvsc
[    0.191911] Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
[    0.192241] tun: Universal TUN/TAP device driver, 1.6
[    0.192594] PPP generic driver version 2.4.2
[    0.192663] PPP BSD Compression module registered
[    0.192664] PPP Deflate Compression module registered
[    0.192667] PPP MPPE Compression module registered
[    0.192667] NET: Registered protocol family 24
[    0.192671] hv_vmbus: registering driver hv_netvsc
[    0.195158] scsi host0: storvsc_host_t
[    0.202609] VFIO - User Level meta-driver version: 0.3
[    0.202830] hv_vmbus: registering driver hyperv_keyboard
[    0.203167] rtc_cmos 00:00: RTC can wake from S4
[    0.204737] rtc_cmos 00:00: registered as rtc0
[    0.204754] rtc_cmos 00:00: alarms up to one month, 114 bytes nvram
[    0.204921] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised<EMAIL_ADDRESS>[    0.205255] hv_utils: Registering HyperV Utility Driver
[    0.205256] hv_vmbus: registering driver hv_util
[    0.205295] hv_vmbus: registering driver hv_balloon
[    0.205307] hv_utils: cannot register PTP clock: 0
[    0.205327] dxgk:err: dxg_drv_init  Version: 1
[    0.205335] hv_vmbus: registering driver dxgkrnl
[    0.205380] drop_monitor: Initializing network drop monitor service
[    0.205860] hv_utils: TimeSync IC version 4.0
[    0.205979] hv_balloon: Using Dynamic Memory protocol version 2.0
[    0.206428] hv_balloon: cold memory discard enabled
[    0.209658] Mirror/redirect action on
[    0.210204] IPVS: Registered protocols (TCP, UDP)
[    0.210216] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
[    0.211092] IPVS: ipvs loaded.
[    0.211094] IPVS: [rr] scheduler registered.
[    0.211094] IPVS: [wrr] scheduler registered.
[    0.211095] IPVS: [sh] scheduler registered.
[    0.212400] ipt_CLUSTERIP: ClusterIP Version 0.8 loaded successfully
[    0.212687] Initializing XFRM netlink socket
[    0.212802] NET: Registered protocol family 10
[    0.213883] Segment Routing with IPv6
[    0.215602] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.215689] NET: Registered protocol family 17
[    0.215702] Bridge firewalling registered
[    0.215708] 8021q: 802.1Q VLAN Support v1.8
[    0.215723] sctp: Hash tables configured (bind 256/256)
[    0.215767] 9pnet: Installing 9P2000 support
[    0.215777] Key type dns_resolver registered
[    0.215783] Key type ceph registered
[    0.216018] libceph: loaded (mon/osd proto 15/24)
[    0.216019] hv_vmbus: registering driver hv_sock
[    0.216098] NET: Registered protocol family 40
[    0.216363] registered taskstats version 1
[    0.216369] Loading compiled-in X.509 certificates
[    0.216567] Btrfs loaded, crc32c=crc32c-generic
[    0.217047] rtc_cmos 00:00: setting system clock to 2021-01-05 13:36:30 UTC<PHONE_NUMBER>)
[    0.217062] Unstable clock detected, switching default tracing clock to "global"
               If you want to keep using the local clock, then add:
                 "trace_clock=local"
               on the kernel command line
[    0.220133] Freeing unused kernel image memory: 1512K
[    0.280824] Write protecting the kernel read-only data: 20480k
[    0.282026] Freeing unused kernel image memory: 1984K
[    0.282388] Freeing unused kernel image memory: 1240K
[    0.282390] Run /init as init process
[    0.457958] scsi 0:0:0:0: Direct-Access     Msft     Virtual Disk     1.0  PQ: 0 ANSI: 5
[    0.458498] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    0.458831] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    0.459313] sd 0:0:0:0: [sda] 536870912 512-byte logical blocks: (275 GB/256 GiB)
[    0.459314] sd 0:0:0:0: [sda] 4096-byte physical blocks
[    0.459405] sd 0:0:0:0: [sda] Write Protect is off
[    0.459407] sd 0:0:0:0: [sda] Mode Sense: 0f 00 00 00
[    0.460049] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    0.462793] sd 0:0:0:0: [sda] Attached SCSI disk
[    0.469846] scsi 0:0:0:1: Direct-Access     Msft     Virtual Disk     1.0  PQ: 0 ANSI: 5
[    0.470193] sd 0:0:0:1: Attached scsi generic sg1 type 0
[    0.471263] sd 0:0:0:1: [sdb] 536870912 512-byte logical blocks: (275 GB/256 GiB)
[    0.471265] sd 0:0:0:1: [sdb] 4096-byte physical blocks
[    0.471421] sd 0:0:0:1: [sdb] Write Protect is off
[    0.471423] sd 0:0:0:1: [sdb] Mode Sense: 0f 00 00 00
[    0.471621] sd 0:0:0:1: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    0.474671] sd 0:0:0:1: [sdb] Attached SCSI disk
[    0.479975] EXT4-fs (sda): mounted filesystem with ordered data mode. Opts: discard,errors=remount-ro,data=ordered
[    0.502733] EXT4-fs warning (device sdb): ext4_clear_journal_err:5056: Filesystem error recorded from previous mount: IO failure
[    0.502734] EXT4-fs warning (device sdb): ext4_clear_journal_err:5057: Marking fs in need of filesystem check.
[    0.504352] Adding 1048576k swap on /swap/file.  Priority:-2 extents:1 across:1048576k
[    0.506686] EXT4-fs (sdb): warning: mounting unchecked fs, running e2fsck is recommended
[    0.509323] EXT4-fs (sdb): recovery complete
[    0.510437] EXT4-fs (sdb): mounted filesystem with ordered data mode. Opts: discard,errors=remount-ro,data=ordered
[    1.270622] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   49.069001] hv_balloon: Max. dynamic memory size: 4096 MB
[   61.459944] WSL2: Performing memory compaction.
[   72.147086] EXT4-fs error (device sdb): ext4_validate_block_bitmap:384: comm kworker/u10:1: bg 109: bad block bitmap checksum
[   72.198226] Aborting journal on device sdb-8.
[   72.200092] EXT4-fs (sdb): Remounting filesystem read-only
[  122.513198] WSL2: Performing memory compaction.
[  183.555787] WSL2: Performing memory compaction.
[  244.566958] WSL2: Performing memory compaction.
[  305.270494] EXT4-fs (sdb): error count since last fsck: 190
[  305.270498] EXT4-fs (sdb): initial error at time<PHONE_NUMBER>: ext4_validate_block_bitmap:384
[  305.270501] EXT4-fs (sdb): last error at time<PHONE_NUMBER>: ext4_validate_block_bitmap:384: inode 45321
[  305.579120] WSL2: Performing memory compaction.

WSL logs: https://aka.ms/AAar55x
Expected behavior
Copy either work or fail without making the whole fs read only.
Actual behavior
Failed operation result in read only file system

That wasn't enough to resolve the issue (same problem happen after executing the commands), however, e2fsck prompted to use fsck to resolve bad blocks, which I did, and it seems to have resolved the issue.

That wasn't enough to resolve the issue (same problem happen after executing the commands), however, e2fsck prompted to use fsck to resolve bad blocks, which I did, and it seems to have resolved the issue.

This seems to be consistent with the problem I encountered. When encountering this problem, even if it is restarted, the problem will still occur and cannot be recovered. Do you have the above solution later and how to avoid this problem from happening?
BTW：My Linux kernel version is also 4.19.275


(Do not merge!) Remove getReducer and replaceReducer from public API

As discussed in #350, we want to get rid of them to simplify Store API, and express alternatives to them in userland.

Let's take it slow.
#668 is the first step. We'll revisit whether we really want to do this after we get some feedback on it.


File to output capture to Blob storage using Blob SAS URL "Failed to validate blob url"  and "Failed to output network traffic"

Describe the bug
Maybe doing things wrong here, but the capture isn't uploaded to the SAS URL. Seems  to be complaining. Followed the documentation. Seems to be a little bit limited in description.
Messages are "Failed to validate blob url"  and "Failed to output network traffic"
Related to net/url: invalid control character in URL"
Tried may things below the anonymised logs.
ts=2024-03-24T11:28:37.948Z level=error caller=outputlocation/blob.go:55 msg="Failed to validate blob url" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%2FTfiMWf16D53VxyzPHs%3D\\n\": net/url: invalid control character in URL"
ts=2024-03-24T11:28:37.948Z level=error caller=captureworkload/main.go:57 msg="Failed to output network traffic" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="location \"BlobUpload\" output error: parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%!F(MISSING)TfiMgf16D53VxyzPHs%!D(MISSING)\\n\": net/url: invalid control character in URL\n"

To Reproduce
Steps to reproduce the behavior:

Created a default storage container
Created a new private blob container called captures
Generated a SAS on that container  with required write,read,list permissions (also tried full)
Copied the Blob SAS URL value to my local txt file
Create secret.

kubectl create secret generic capture-blob-storage --from-file=blob-upload-url=./blob-upload-url.txt

Create first capture

# Getting the first available node
if [[ -z $1 ]]; then
  target=`kubectl get nodes -o 'jsonpath={.items[0].metadata.name}'`
else
  target=$1
fi

cat <<EOF | kubectl create -f -
apiVersion: retina.sh/v1alpha1
kind: Capture
metadata:
  name: my-first-capture
spec:
  captureConfiguration:
    captureOption:
      duration: 30s
    captureTarget:
      nodeSelector:
        matchLabels:
          kubernetes.io/hostname: ${target}
  outputConfiguration:
    hostPath: "/tmp/retina"
    blobUpload: capture-blob-storage
EOF

Show the logs of the pod that executed the job.

ts=2024-03-24T11:28:37.948Z level=error caller=outputlocation/blob.go:55 msg="Failed to validate blob url" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%2FTfiMWf16D53VxyzPHs%3D\\n\": net/url: invalid control character in URL"
ts=2024-03-24T11:28:37.948Z level=error caller=captureworkload/main.go:57 msg="Failed to output network traffic" goversion=go1.21.8 os=linux arch=amd64 numcores=2 hostname=aks-agentpool-35551448-vmss000000 podname=my-first-capture-99wqd-wxpx5 error="location \"BlobUpload\" output error: parse \"https://1234cap.blob.core.windows.net/captures?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%!F(MISSING)TfiMgf16D53VxyzPHs%!D(MISSING)\\n\": net/url: invalid control character in URL\n"

Expected behavior
Upload and store my capture file.
Screenshots
If applicable, add screenshots to help explain your problem.
Platform (please complete the following information):

OS: AKSUbuntu-2204gen2containerd-202403.13.0
Kubernetes Version:  1.29.0
Host: AKS (Default dev pattern)
Retina Version: Latest main build 24/3)

Additional context
Add any other context about the problem here.

See differences which can matter. I have created a container and expects that blobs are added there?
Code
blob_test.go
https://retina.blob.core.windows.net/**container/blob**?sp=r&st=2023-02-17T19:13:30Z&se=2023-02-18T03:13:30Z&spr=https&sv=2021-06-08&sr=c&sig=NtSxlRK5Vs4kVs1dIOfr%2FMdLKBVTA4t3uJ0gqLZ9exk%3D
blob-upload-url ( I have set)
https://1234cap.blob.core.windows.net/**captures**?sp=racwdli&st=2024-03-24T11:25:58Z&se=2024-03-24T19:25:58Z&spr=https&sv=2022-11-02&sr=c&sig=0vksBBdje4XlXxxjOJdztOZN%2FTfiMWf16D53VxyzPHs%3D

@mainred can you help triage this ?

@mainred @rbtr
Also tried to use the blob container using the CLI ( 0.0.11 without succes.). Btw also seems that I have to add '--name flag', but this isn't part of the documentation example.
./kubectl-retina capture create  --name example --node-selectors "kubernetes.io/os=linux"  –debug --blob-upload https://test.blob.core.windows.net/captures?sp=rw&st=2024-05-28T19:46:42Z&se=2024-05-29T03:46:42Z&skoid=5df443ed-258d-4282-960c-xxxxxxxxxxxxxktid=84f1e4ea-8554-43e1-8709-f0b8589ea118&skt=2024-05-28T19:46:42Z&ske=2024-05-29T03:46:42Z&sks=b&skv=2022-11-02&spr=https&sv=2022-11-02&sr=c&sig=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx%3D

ts=2024-05-28T20:02:25.274Z level=info caller=outputlocation/blob.go:46 msg="Upload capture file to blob." location=BlobUpload
ts=2024-05-28T20:02:25.880Z level=error caller=outputlocation/blob.go:81 msg="Failed to upload file to storage account" location=BlobUpload error="PUT https://test.blob.core.windows.net/captures/example-aks-agentpool-32992052-vmss000000-20240528080120UTC.tar.gz\n--------------------------------------------------------------------------------\nRESPONSE 401: 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nERROR CODE: NoAuthenticationInformation\n--------------------------------------------------------------------------------\nNoAuthenticationInformationServer failed to authenticate the request. Please refer to the information in the www-authenticate header.\nRequestId:7d325446-a01e-000b-2339-b16b90000000\nTime:2024-05-28T20:02:25.8755372Z\n--------------------------------------------------------------------------------\n"
ts=2024-05-28T20:02:25.883Z level=error caller=captureworkload/main.go:68 msg="Failed to output network traffic" error="location "BlobUpload" output error: PUT https://test.blob.core.windows.net/captures/example-aks-agentpool-32992052-vmss000000-20240528080120UTC.tar.gz\n--------------------------------------------------------------------------------\nRESPONSE 401: 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nERROR CODE: NoAuthenticationInformation\n--------------------------------------------------------------------------------\nNoAuthenticationInformationServer failed to authenticate the request. Please refer to the information in the www-authenticate header.\nRequestId:7d325446-a01e-000b-2339-b16b90000000\nTime:2024-05-28T20:02:25.8755372Z\n--------------------------------------------------------------------------------\n\n"

Opened #410 for the --name flag docs.
@vakalapa I think we need to reassign this for investigation

Thank you @avwsolutions for reporting this issue. Let me take a look.

The issue is caused by the blob URL validation failure.

@rbtr I can take https://github.com/microsoft/retina/issues/410

@avwsolutions sorry for my late response and fix. The PR linked to this issue should fix the issue.

Thanks @mainred 🙂

I will test coming days, so i can use this in my blog article.


Certain keywords return no results

Certain keywords return no results even though the same search on webuy.com does return results. The issue was tested on the GCP version of the app and with both the strict option on and off. Example keywords that return no results:
sonic
terminator
dell
raspberry
windows

Hi @yoshielectron. Thanks for the bug report.
I've just had a quick look at this.  I thought it might be that the items you were searching for weren't available online, but that isn't the issue. That is Terminator in stock online returns over 100 hits.
The query I'm building to the API includes the following parameters:
clickAnalytics=true
facetFilters=[["availability:In Stock Online"]]
facets=["*"]
filters=boxVisibilityOnWeb=1 AND boxSaleAllowed=1
highlightPostTag=__/ais-highlight__
highlightPreTag=__ais-highlight__
hitsPerPage=17
maxValuesPerFacet=1000
page=0
tagFilters=

Perhaps the boxSaleAllowed parameter is the problem? I'll have to dig into this a bit.


[FLINK-27384] solve the problem that the latest data cannot be read under the creat…

What is the purpose of the change
This pull request solve the problem that the latest data cannot be read under the createtime configuration in the hive dimension table.
Brief change log
solve the problem that the latest data cannot be read under the createtime configuration in the hive dimension table
Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
no
This change added tests and can be verified as follows:
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no

@leonardBang hi, could you help me review & merge？Thanks.  jira link : https://issues.apache.org/jira/browse/FLINK-27384

@flinkbot run azure

@leonardBang hi, could you help me review & merge？Thanks.

@empcl Could you also backport the fix to release-1.4 and release-1.15 branch ?


Support for Rotating Images, Text and Sprites

simple2d library could be more widely used if it had support for rotation. It would be easy to add rotation support because both SDL and OpenGL ES/Open GL supports rotation. I would prefer the hardware accelerated OpenGL ES rotation (I'm using simple2d on a Raspberry pi).

Also, it would be very helpful if you could give me some code that can perform a rotation given degrees/radians. Your code is very simple to understand. I get that you are using a struct with surface as the pointer to SDl Surface. And texture id for drawing the texture in OpenGL(es). However, I don't know OpenGL ES(I only know OpenGL) so I would be very happy if you could throw some code to me that I may include in my program so that I don't need to wait and update simple2d when that feature is added. Thanks in Advance

Agreed, would be a great feature to add. We've talked about it over at Ruby 2D, which uses this lib. To implement, I'd probably add a rotate property to the structs, figure out the math (something like this), and do these point translations before drawing here for each GL renderer. Now, just have to find the free time to do all the work 😉. In the meantime, I suppose you could write your own S2D_*_DrawTexture functions with the rotation math.

Added in #88 🌀


Add details about deprecating feature flags enable-custom-task and custom-task-version

Changes
Prior to this PR, the deprecation doc only contains the deprecation about the v1alpha1.Run, didn't mention related feature flag deprecation. This commit adds details about the feature flag deprecation
/kind documentation
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[x] Has Docs included if any changes are user facing
[ ] Has Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including
functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[x] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
[x] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
Deprecation Notice: the feature flag `custom-task-version` will be removed in release v0.47.0.

/assign @vdemeester
cc @abayer @jerop

Sorry, wrong button 🙀 😅

/retest

/test check-pr-has-kind-label

/test pull-tekton-pipeline-go-coverage-df


The lost transactions issue.

Users report that transactions are notoriously being lost when someone issues a large number of transactions at once. This happens regularly when block producers pay their delegators for delegating their stake to them. Some of those transactions are being lost and never make it to the blockchain.

We have found out that the most likely reason for this is that the mempool enforces nonce consecutivity. In particular, a transaction is being accepted into the mempool only if one of the following conditions is met:

the transaction is applicable at the current sender's nonce
the transaction is applicable at a nonce higher than the current sender's nonce AND transactions with ALL the intermediate nonces are already present in the mempool.

When a user issues a large number of transactions at a time, due to inherent networking non-determinism, they can and often do arrive at a particular node out-of-order. The ruling described above then makes the node discard the out-of-order transactions. Not only that, all the later transactions will slso be discarded due to a nonce gap.

For the moment I've got a solution to allow for nonce gaps in the mempool passig all the unit tests. I don't yet have the patch for dropping strategy, while I have to put this task on hold because of the incomming work directly related to the hard fork. My changes so far have been comitted to a branch.


Can Runnables be organized in ad hoc subdirectories?

Hi!
I'm wondering if Runnables have to stay in the project root or if there's a way to organize them into ad hoc subdirectories.
Right now, we need to keep Runnables in the root like
myRunnableDirectory/
Directive.yaml

It'd be great to be able to put them in subdirectories for organization purposes like
Resources/Accounts/myRunnableDirectory
Resources/Accounts/myOtherRunnableDirectory
Resources/Files/getAll
Resources/Files/delete
Directive.yaml

Thanks!

@kee-oth yes, currently they need to be in the project root, however we are changing that. We are working on updates to the CLI which will allow for arbitrary folders.


Update landscape.yml with Lemur

Submission to add Lemur to the category: Observability and Analysis, subcategory: Monitoring
Pre-submission checklist:
Please check each of these after submitting your pull request:

[X] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[X] Is your project closed source or, if it is open source, does your project have at least 300 GitHub stars?
[X] Have you picked the single best (existing) category for your project?
[X] Does it follow the other guidelines from the new entries section?
[X] Have you included a URL for your SVG or added it to hosted_logos and referenced it there? In a separate pull request
[X] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[X] Does your project/product name match the text on the logo?
[X] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[X] ~5 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

Build failed because of:
https://github.com/turbonomic/lemur/wiki does not look like a github repo
https://github.com/turbonomic/lemur/wiki does not look like a github repo
No cached entry, and Lemur has issues with logo: lemur_turbonomic.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/lemur_turbonomic.svg'

Deploy preview for landscape failed.
Built with commit c651b3ccd5d220d3f946dfb7d14725b504b95a99
https://app.netlify.com/sites/landscape/deploys/5e38d6bc6ac43f00072ed281

Build failed because of:
https://github.com/turbonomic/lemur/wiki does not look like a github repo
https://github.com/turbonomic/lemur/wiki does not look like a github repo
No cached entry, and Lemur has issues with logo: lemur_turbonomic.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/lemur_turbonomic.svg'

Deploy preview for landscape failed.
Built with commit 1635cdf359c6472970feeeb5add6ad4a2f3cc053
https://app.netlify.com/sites/landscape/deploys/5e38d737434d10000cf411a7

To fix the build, you would change https://github.com/turbonomic/lemur/wiki to https://github.com/turbonomic/lemur. However, new open source entries need to have 300 stars https://github.com/cncf/landscape#new-entries. Please open a new pull request when you get there.


Unable to build engine UE 5.4.4 on Ubuntu

Output of the ue4-docker info command:
> ue4-docker build 5.4.4
[ue4-docker build] COMMAND-LINE INVOCATION:
[ue4-docker build] ['ue4-docker', '5.4.4']

[ue4-docker build] UNREAL ENGINE VERSION SETTINGS:
[ue4-docker build] Custom build:  No
[ue4-docker build] Release:       5.4.4
[ue4-docker build] Repository:    https://github.com/EpicGames/UnrealEngine.git
[ue4-docker build] Branch/tag:    5.4.4-release

[ue4-docker build] ADVANCED CONFIGURATION OPTIONS:
[ue4-docker build] credential_mode: "secrets"
[ue4-docker build] excluded_components: {"ddc": false, "debug": false, "templates": false}
[ue4-docker build] gitdependencies_args: "--exclude=Android --exclude=Mac --exclude=Win32 --exclude=Win64"

[ue4-docker build] LINUX CONTAINER SETTINGS
[ue4-docker build] Base OS image: nvidia/opengl:1.0-glvnd-devel-ubuntu22.04

[ue4-docker build] GENERAL SETTINGS
[ue4-docker build] Build targets: build-prerequisites minimal source
[ue4-docker build] Changelist override: (None specified)
[ue4-docker build] Not excluding any Engine components.

[ue4-docker build] Warning: attempting to build Linux container images under Windows (e.g. via WSL).
[ue4-docker build] The ue4-docker maintainers do not provide support for building and running Linux
[ue4-docker build] containers under Windows, and this configuration is not tested to verify that it
[ue4-docker build] functions correctly. Users are solely responsible for troubleshooting any issues
[ue4-docker build] they encounter when attempting to build Linux container images under Windows.
[ue4-docker build] Not building the ue4-source image, no Git credentials required.

[ue4-docker build] Image "adamrehn/ue4-build-prerequisites:opengl-ubuntu22.04" exists and rebuild not requested, skipping build.

[ue4-docker build] Image "adamrehn/ue4-source:5.4.4-opengl-ubuntu22.04" exists and rebuild not requested, skipping build.

[ue4-docker build] Building image "adamrehn/ue4-minimal:5.4.4-opengl-ubuntu22.04"...
[+] Building 2309.9s (22/46)                                                                             docker:default
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 10.48kB                                                                            0.0s
 => WARN: InvalidDefaultArgInFrom: Default value for ARG ${NAMESPACE}/ue4-source:${TAG}-${PREREQS_TAG} results in  0.0s
 => WARN: InvalidDefaultArgInFrom: Default value for ARG ${NAMESPACE}/ue4-build-prerequisites:${PREREQS_TAG} resu  0.0s
 => [internal] load metadata for docker.io/adamrehn/ue4-build-prerequisites:opengl-ubuntu22.04                     0.1s
 => [internal] load metadata for docker.io/adamrehn/ue4-source:5.4.4-opengl-ubuntu22.04                            0.1s
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 53B                                                                                0.0s
 => [builder  1/22] FROM docker.io/adamrehn/ue4-source:5.4.4-opengl-ubuntu22.04@sha256:95eda9af46580854012239cfcc  1.1s
 => => resolve docker.io/adamrehn/ue4-source:5.4.4-opengl-ubuntu22.04@sha256:95eda9af46580854012239cfcc9f73b12618  1.1s
 => CACHED [stage-1  1/16] FROM docker.io/adamrehn/ue4-build-prerequisites:opengl-ubuntu22.04@sha256:1007050bff94  0.8s
 => => resolve docker.io/adamrehn/ue4-build-prerequisites:opengl-ubuntu22.04@sha256:1007050bff9468c3442ba641de43c  0.8s
 => [internal] load build context                                                                                0.0s
 => => transferring context: 10.76kB                                                                               0.0s
 => [auth] adamrehn/ue4-source:pull token for registry-1.docker.io                                                 0.0s
 => [auth] adamrehn/ue4-build-prerequisites:pull token for registry-1.docker.io                                    0.0s
 => [auth] adamrehn/ue4-source:pull token for registry-1.docker.io                                                 0.0s
 => CACHED [builder  2/22] COPY set-changelist.py /tmp/set-changelist.py                                           0.0s
 => CACHED [builder  3/22] RUN python3 /tmp/set-changelist.py /home/ue4/UnrealEngine/Engine/Build/Build.version $  0.0s
 => CACHED [builder  4/22] RUN rm -rf /home/ue4/UnrealEngine/.git && echo '' && echo 'RUN directive complete. Doc  0.0s
 => CACHED [builder  5/22] COPY enable-opengl.py /tmp/enable-opengl.py                                             0.0s
 => CACHED [builder  6/22] RUN python3 /tmp/enable-opengl.py /home/ue4/UnrealEngine/Engine/Config/BaseEngine.ini   0.0s
 => CACHED [builder  7/22] COPY patch-filters-xml.py /tmp/patch-filters-xml.py                                     0.0s
 => CACHED [builder  8/22] RUN python3 /tmp/patch-filters-xml.py /home/ue4/UnrealEngine/Engine/Build/InstalledEng  0.0s
 => CACHED [builder  9/22] COPY patch-build-graph.py /tmp/patch-build-graph.py                                     0.0s
 => CACHED [builder 10/22] RUN python3 /tmp/patch-build-graph.py /home/ue4/UnrealEngine/Engine/Build/InstalledEng  0.0s
 => CACHED [builder 11/22] RUN ./Engine/Build/BatchFiles/Linux/Build.sh ShaderCompileWorker Linux Development -Sk  0.0s
 => CACHED [builder 12/22] WORKDIR /home/ue4/UnrealEngine                                                          0.0s
 => ERROR [builder 13/22] RUN ./Engine/Build/BatchFiles/RunUAT.sh BuildGraph  -target="Make Installed Build Li  2308.4s
------
 > [builder 13/22] RUN ./Engine/Build/BatchFiles/RunUAT.sh BuildGraph   -target="Make Installed Build Linux"    -script=Engine/Build/InstalledEngineBuild.xml   -set:HostPlatformOnly=true      -set:WithDDC=true        &&     rm -R -f /home/ue4/UnrealEngine/LocalBuilds/InstalledDDC && echo '' && echo 'RUN directive complete. Docker will now commit the filesystem layer to disk.' && echo 'Note that for large filesystem layers this can take quite some time.' && echo 'Performing filesystem layer commit...' && echo '':
-0.020
-0.020 Running AutomationTool...
-0.020
-0.011 Setting up bundled DotNet SDK
0.005 Rebuilding: AutomationTool assembly not found
0.006 Setting up bundled DotNet SDK
1.258 Microsoft (R) Build Engine version 17.2.0+41abc5629 for .NET
1.258 Copyright (C) Microsoft Corporation. All rights reserved.
1.258
5.966
5.966 Build succeeded.
5.966     0 Warning(s)
5.966     0 Error(s)
5.966
5.966 Time Elapsed 00:00:04.65
5.978 Start UAT Interactively: dotnet AutomationTool.dll BuildGraph -target=Make Installed Build Linux -script=Engine/Build/InstalledEngineBuild.xml -set:HostPlatformOnly=true -set:WithDDC=true
5.979 /home/ue4/UnrealEngine/Engine/Build/BatchFiles/Linux/../../../Binaries/ThirdParty/DotNet/6.0.302/linux/dotnet
6.073 Starting AutomationTool...
6.091 Parsing command line: BuildGraph -target="Make Installed Build Linux" -script=Engine/Build/InstalledEngineBuild.xml -set:HostPlatformOnly=true -set:WithDDC=true
6.118 Initializing script modules...
7.530 Building 32 projects (see Log 'Engine/Programs/AutomationTool/Saved/Logs/Log.txt' for more details)
7.534  Restore...
11.22  Build...
23.04 Build projects time: 15.51 s
23.08 Total script module initialization time: 16.96 s.
23.08 Using /home/ue4/UnrealEngine/Engine/Binaries/ThirdParty/DotNet/6.0.302/linux/dotnet
23.20 Executing commands...
23.31 ****** [1/10] Update Version Files
23.31
23.32 Saving file list to /home/ue4/UnrealEngine/Engine/Saved/BuildGraph/Update Version Files/Tag-Update Version Files.xml
23.32
23.32 ****** [2/10] Compile UnrealEditor Linux
23.32
23.33 Reading local file list from /home/ue4/UnrealEngine/Engine/Saved/BuildGraph/Update Version Files/Tag-Update Version Files.xml
23.35 Running: /home/ue4/UnrealEngine/Engine/Binaries/ThirdParty/DotNet/6.0.302/linux/dotnet "/home/ue4/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.dll" -Target="UnrealEditor Linux DebugGame -Manifest=/home/ue4/UnrealEngine/Engine/Intermediate/Build/Manifest-1-UnrealEditor-Linux-DebugGame.xml -precompile -allmodules" -Target="UnrealEditor Linux Development -Manifest=/home/ue4/UnrealEngine/Engine/Intermediate/Build/Manifest-2-UnrealEditor-Linux-Development.xml -precompile -allmodules" -log="/home/ue4/Library/Logs/Unreal Engine/LocalBuildLogs/UBA-UnrealEditor-Linux-DebugGame.txt"
23.53 Log file: /home/ue4/Library/Logs/Unreal Engine/LocalBuildLogs/UBA-UnrealEditor-Linux-DebugGame.txt
23.74 Creating makefile for UnrealEditor (no existing makefile)
25.64 Parsing headers for UnrealEditor
25.64   Running Internal UnrealHeaderTool UnrealEditor /home/ue4/UnrealEngine/Engine/Intermediate/Build/Linux/UnrealEditor/DebugGame/UnrealEditor.uhtmanifest -WarningsAsErrors
28.05 Total of 22643 written
28.05 Reflection code generated for UnrealEditor in 2.4122012 seconds
28.83 ------- Build details --------
28.83 Using toolchain located at '/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu'.
28.83 Using clang (/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu/bin/clang++) version 'clang version 16.0.6 (github.com/llvm/llvm-project 7cbf1a2591520c2491aa35339f227775f4d3adf6)
28.83 Target: x86_64-unknown-linux-gnu
28.83 Thread model: posix
28.83 InstalledDir: /home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu/bin' (string), 16 (major), 0 (minor), 6 (patch)
28.83 Using bundled libc++ standard C++ library.
28.83 Using lld linker
28.84 Using llvm-ar (/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu/bin/llvm-ar) version 'LLVM (http://llvm.org/):
28.84   LLVM version 16.0.6
28.84   Optimized build. (string)'
28.84 Using fast way to relink  circularly dependent libraries (no FixDeps).
28.84 Targeted minimum CPU architecture: None
28.84 ------------------------------
36.83 Writing dependency list to /home/ue4/UnrealEngine/Engine/Intermediate/DependencyLists/UnrealEditor/DebugGame/Linuxx64/DependencyList-AllModules.txt
36.86 Writing manifest to /home/ue4/UnrealEngine/Engine/Intermediate/Build/Manifest-1-UnrealEditor-Linux-DebugGame.xml
37.12 Creating makefile for UnrealEditor (no existing makefile)
37.67 Parsing headers for UnrealEditor
37.67   Running Internal UnrealHeaderTool UnrealEditor /home/ue4/UnrealEngine/Engine/Intermediate/Build/Linux/UnrealEditor/Development/UnrealEditor.uhtmanifest -WarningsAsErrors
38.91 Total of 0 written
38.91 Reflection code generated for UnrealEditor in 1.2438159 seconds
39.16 ------- Build details --------
39.16 Using toolchain located at '/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu'.
39.16 Using clang (/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu/bin/clang++) version 'clang version 16.0.6 (github.com/llvm/llvm-project 7cbf1a2591520c2491aa35339f227775f4d3adf6)
39.16 Target: x86_64-unknown-linux-gnu
39.16 Thread model: posix
39.16 InstalledDir: /home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu/bin' (string), 16 (major), 0 (minor), 6 (patch)
39.16 Using bundled libc++ standard C++ library.
39.16 Using lld linker
39.16 Using llvm-ar (/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v22_clang-16.0.6-centos7/x86_64-unknown-linux-gnu/bin/llvm-ar) version 'LLVM (http://llvm.org/):
39.16   LLVM version 16.0.6
39.16   Optimized build. (string)'
39.16 Using fast way to relink  circularly dependent libraries (no FixDeps).
39.16 Targeted minimum CPU architecture: None
39.16 ------------------------------
40.66 Writing dependency list to /home/ue4/UnrealEngine/Engine/Intermediate/DependencyLists/UnrealEditor/Development/Linuxx64/DependencyList-AllModules.txt
40.68 Writing manifest to /home/ue4/UnrealEngine/Engine/Intermediate/Build/Manifest-2-UnrealEditor-Linux-Development.xml41.24 Building UnrealEditor...
41.26 Determining max actions to execute in parallel (16 physical cores, 32 logical cores)
41.26   Executing up to 16 processes, one per physical core
41.26   Requested 1.5 GB memory per action, 18.89 GB available: limiting max parallel actions to 12
41.26 Using Parallel executor to run 4345 action(s)
41.27 ------ Building 4345 action(s) started ------
41.37 ** For UnrealEditor-Linux-DebugGame + UnrealEditor-Linux-Development **
41.47 [1/4345] Compile Module.BuildSettings.cpp
41.47 [2/4345] Link (lld) libUnrealEditor-BuildSettings.so
43.02 [3/4345] Compile MiMalloc.c
43.02 [4/4345] Generate Header [x86-64] AnimEncoding_ConstantKeyLerp.ispc
43.02 /home/wojtek/fnmain/Engine/Source/ThirdParty/Intel/ISPC/ispc-1.21.0/src/main.cpp(404): FATAL ERROR: Unhandled signal sent to process; terminating.
43.02 ***
43.02 *** Please file a bug report at https://github.com/ispc/ispc/issues
43.02 *** (Including as much information as you can about how to reproduce this error).
43.02 *** You have apparently encountered a bug in the compiler that we'd like to fix!
43.02 ***
43.84 [5/4345] Compile Module.TraceLog.cpp
43.84 [6/4345] Generate Header [x86-64] AnimationRuntime.ispc
43.84 [7/4345] Compile PCH.Core.h

...

2298.1 [2736/4345] Compile Module.DMXPixelMappingEditor.cpp
2298.1 [2737/4345] Compile Module.TimedDataMonitorEditor.cpp
2298.1 ** For UnrealEditor-Linux-Development **
2299.6 [2738/4345] Compile Module.Launch.cpp
2299.6 ** For UnrealEditor-Linux-DebugGame + UnrealEditor-Linux-Development **
2305.1 [2739/4345] Compile Module.LevelSnapshotsEditor.cpp
2305.1 [2740/4345] Compile Module.CameraCalibrationEditor.2.cpp
2305.1 Total time in Parallel executor: 2150.38 seconds
2305.1 Total execution time: 2167.31 seconds
2305.2 Took 2,281.86s to run dotnet, ExitCode=6
2305.3 UnrealBuildTool failed. See log for more details. (/home/ue4/Library/Logs/Unreal Engine/LocalBuildLogs/UBA-UnrealEditor-Linux-DebugGame.txt)
2305.3 AutomationTool executed for 0h 36m 24s
2305.3 AutomationTool exiting with ExitCode=6 (6)
2305.3 RunUAT ERROR: AutomationTool was unable to run successfully. Exited with code: 6
------

 2 warnings found (use docker --debug to expand):
 - InvalidDefaultArgInFrom: Default value for ARG ${NAMESPACE}/ue4-source:${TAG}-${PREREQS_TAG} results in empty or invalid base image name (line 4)
 - InvalidDefaultArgInFrom: Default value for ARG ${NAMESPACE}/ue4-build-prerequisites:${PREREQS_TAG} results in empty or invalid base image name (line 67)
Dockerfile:32
--------------------
  31 |     WORKDIR /home/ue4/UnrealEngine
  32 | >>> RUN ./Engine/Build/BatchFiles/RunUAT.sh BuildGraph \
  33 | >>>      -target="Make Installed Build Linux" \
  34 | >>>      -script=Engine/Build/InstalledEngineBuild.xml \
  35 | >>>      -set:HostPlatformOnly=true \
  36 | >>>      -set:WithDDC=true \
  37 | >>>       && \
  38 | >>>      rm -R -f /home/ue4/UnrealEngine/LocalBuilds/InstalledDDC && echo '' && echo 'RUN directive complete. Docker will now commit the filesystem layer to disk.' && echo 'Note that for large filesystem layers this can take quite some time.' && echo 'Performing filesystem layer commit...' && echo ''
  39 |
--------------------
ERROR: failed to solve: process "/bin/sh -c ./Engine/Build/BatchFiles/RunUAT.sh BuildGraph \t-target=\"Make Installed Build Linux\" \t-script=Engine/Build/InstalledEngineBuild.xml \t-set:HostPlatformOnly=true \t-set:WithDDC=true \t && \trm -R -f /home/ue4/UnrealEngine/LocalBuilds/InstalledDDC && echo '' && echo 'RUN directive complete. Docker will now commit the filesystem layer to disk.' && echo 'Note that for large filesystem layers this can take quite some time.' && echo 'Performing filesystem layer commit...' && echo ''" did not complete successfully: exit code: 6
[ue4-docker build] Error: failed to build image "adamrehn/ue4-minimal:5.4.4-opengl-ubuntu22.04".

Additional details:

Are you accessing the network through a proxy server? No

[ue4-docker build] Warning: attempting to build Linux container images under Windows (e.g. via WSL).
[ue4-docker build] The ue4-docker maintainers do not provide support for building and running Linux
[ue4-docker build] containers under Windows, and this configuration is not tested to verify that it
[ue4-docker build] functions correctly. Users are solely responsible for troubleshooting any issues
[ue4-docker build] they encounter when attempting to build Linux container images under Windows.


Dropdown always visible when unselecting?

I've found that it's possible to keep the dropdown closed on unselecting, but how to always keep it open when unselecting?
Opposite than this:
https://github.com/select2/select2/issues/3209#issuecomment-149663474

https://select2.github.io/options.html#can-i-prevent-the-dropdown-from-closing-when-a-result-is-select

This appears to be more of a usage question than a bug report or common feature request.  As we continue to keep the Select2 issue tracker geared towards these two areas, bug reports and common feature requests, we are redirecting usage questions to other communities such as the mailing list, IRC, or Stack Overflow.

@kevin-brown The closeOnSelect: false does not work. Thats why I reported an issue. So, the problem is that every time a tag in a multiple list is unselected, the doropdown opens and closes in every other step. Click 1, remove tag, open. Click 2, remove another tag, closed. And so on.

To anyone who faced the same problem, here's workaround:
$el.on('select2:unselect', () => {
    $el.on('select2:closing', (e) => {
        e.preventDefault();
        $el.off('select2:closing');
    });
});


Copy all old pages table entries

Corrects the assumption that only the first entry is the active page table is used.
closes #462

Does this panic you saw in #462 still happen with these changes?

Yeah it still panics when trying to allocate boot info. I will try and do some more digging when I have a chance. It's panicking here.
.

Hmm, so the problem is that up until now we've assumed that all existing memory is only in the first pml4 entry and so when we map in more memory, we assume that all other memory is unused:
https://github.com/rust-osdev/bootloader/blob/ea3f61a57ccd57b7ed37eeed1e53ee6cb2cfdd81/common/src/level_4_entries.rs#L42
Obviously, with the changes introduced in this PR, this is no longer the case as, for example, the frame buffer would also be in the address range mapped by the second pml4 entry. I'm looking into a better way to fix this.

Hmm. An simple fix would be to check what entry the framebuffer is in and mark it as used. I can try this patch out but any other devices that are mapped would also cause a similar issue.

Wait my bad there seems to already be a check for the framebuffer address.

So I got it working. All I added was cloning the second entry in the level 4 and marking it used. I don't know how to go about writing a general solution but I would love to help.

Hmm. An simple fix would be to check what entry the framebuffer is in and mark it as used. I can try this patch out but any other devices that are mapped would also cause a similar issue.
Wait my bad there seems to already be a check for the framebuffer address.

I assume you mean this check, right? That check operates on the address where the frame buffer will be mapped into the kernel which is likely not the same address where it's mapped in the bootloader.

So I got it working. All I added was cloning the second entry in the level 4 and marking it used. I don't know how to go about writing a general solution but I would love to help.

We could mark all entries covered by the frame buffer as used and copy them over when creating the new PML4, but I think the cleaner solution is to just never modify the page tables used by the bootloader. Still, I opened a PR based on your idea as well in #466. The big upside is that it's much simpler than my original implementation in #465.

Superseeded by #466


Have --sleep-interval honor the --skip-download flag

Checklist

[x] I'm reporting a feature request
[x] I've verified that I'm running youtube-dl version 2020.07.28
[x] I've searched the bugtracker for similar feature requests including closed ones

Description

Earlier I opened an issue ( #26287 ) mentioning that the --skip-download flag seems to negate the sleep-interval flag. I noted that I considered it unexpected behavior, since flags like write-info-json do indeed download something.
The manual section for --sleep-interval describes it as "Number of seconds to sleep before each download when used alone [...]". I consider non-video files like the result of --write-info-json to be a 'download'; but it seems like not everyone agrees with this.
The main use for the sleep flag is to prevent 429 errors when doing bulk/continuous network requests, yet a command like:
youtube-dl --sleep-interval 10 --write-info-json --skip-download https://www.youtube.com/user/TheLinuxFoundation
Will eventually cause one, because no sleeping happens.
For that reason, I propose that --sleep-interval be honored for any network activity that downloads files, so as to improve the rate-limiting/429 prevention purpose it currently serves.

This is already requested stop posting duplicates.

It's not. You closed the last one as invalid, stating that it's expected behavior and not a bug. I'm suggesting that the expected behavior be changed to better reflect the reason the flag is actually used. I've noted that in the description.
If it's not a bug or a feature, please tell me how to properly file it.

Also of note: I'd be happy to implement this myself.


createIdentifier for did:ion fails with "digest should be a Buffer"

I commented in #1066, but couldn't reopen issue.   #1074 fixes part of this. The createIdentifier now fails with "digest should be a Buffer" message, which is coming from the multihashes.encode statement in computeCommitmentFromJwk.
Originally posted by @rkreutzer in https://github.com/uport-project/veramo/issues/1066#issuecomment-1363366353

@rkreutzer I've seen this error message in relation to a dependency resolution issue. This is partially the reason we're switching from yarn to pnpm.
Do you happen to have a minimal project that reproduces this issue?


Reference to continual_animations in ThreeDScene

There is a reference to continual_animations in the move_camera method of the ThreeDScene class.  Line 71 in three_d_scene.py has
is_camera_rotating = self.ambient_camera_rotation in self.continual_animations

I haven't delved into the changes to updating mobjects enough to fix this issue yet so I'm hoping someone with more experience can figure out how to fix this.

Fixed in PR#481


🐞Path is outside the allowed directory when using Docker container

Describe the bug
See also: https://github.com/openvinotoolkit/anomalib/issues/1688
Hello,
I have some trouble getting Anomalib to work while using a Docker container, since it uses validate_path(). By default, while using the MVTec dataset, Anomalib checks for the path to be in Path.home(), which is /home/user, while Docker usually works in /workspaces.
Furthermore, I have another configuration where my data is not stored in my home folder. I expect that this will break as well.
Dataset
MVTec
Model
PatchCore
Steps to reproduce the behavior
Install anomalib by cloning the repo
Build the container in VSCode
Launch the command anomalib train --model Patchcore --data anomalib.data.MVTec
OS information
N/A
Expected behavior
Checks that the path is in a list of user-defined approved paths?
Screenshots
No response
Pip/GitHub
GitHub
What version/branch did you use?
No response
Configuration YAML
class_path: anomalib.data.MVTec
init_args:
  root: ./datasets/MVTec
  category: bottle
  image_size: [256, 256]
  center_crop: null
  normalization: imagenet
  train_batch_size: 32
  eval_batch_size: 32
  num_workers: 8
  task: SEGMENTATION
  transform_config_train: null
  transform_config_eval: null
  test_split_mode: FROM_DIR
  test_split_ratio: 0.2
  val_split_mode: SAME_AS_TEST
  val_split_ratio: 0.5
  seed: null

Logs
Unable to paste the logs

Code of Conduct

[X] I agree to follow this project's Code of Conduct

We had to introduce validate_path as part of the security measures we had to follow. However, it looks like this causes some issues to users. Maybe we could flex this a bit

I pinned the problem to the _resolve_path_and_convert_to_list where the validate_and_resolve_path is called with the path set to the directory containing normal/abnormal images, root set to the root directory of the data and base_dir not specified (defaulting to None). The validate_and_resolve function then throws this exception because it checks whether base_dir (None) is part of root + path (/path/to/your/images). I managed to work around it by also setting the base_dir directory to root, although I am not sure this is the best solution.

Hello @samet-akcay, @phcarval . I would like to take up this issue since I am good with docker and python. Please let me know the further details . Thank you .

I'm also having the same problem when working in Docker container. Is it ok to relax the security measures and introduce only a Warning instead of an Error?

Is this still an issue? We merged https://github.com/openvinotoolkit/anomalib/pull/1797, which should fix this error. Let me know you still encounter this one..

I'm closing this issue, but feel free to re-open if you still experience problems.


figure out why sometimes Bench build takes 10x longer

wow, that's kind of nuts.  no wonder you hate computers.

Fixed in #113


DBZ-7300 Refactor Snapshotter interface to be an SPI through SnapshotterProvider.

Closes: https://issues.redhat.com/browse/DBZ-7300

Just a draft PR for discussing on the proposal.
The current 'Snapshotter' interface define different things:

When to create snapshot/streaming
How to do snapshot

The proposal aims to split the behaviors into different interface resulting in:

Snapshotter:  define when the snapshot should be done or not and if continue with streaming or not.
SnapshotterProvider: SPI to provide custom Snapshotter implementations. The snapshotter will be choosed using the snapshot.mode configuration property. The custom option will be removed.
SnapshotQuery: define how data are queried from the database. Custom implementation will be provided specifying the implementing class through the snapshot.query.custom.class and loaded through the class loader. As done before for the Snapshotter interface.
SnapshotLock: define the table lock mode used during schema snapshot. Custom implementation will be provided specifying the implementing class through the snapshot.locking.custom.class and loaded through the class loader.

One the proposal is accepted this PR will provide the implementation for PostgreSQL.

@mfvitale Hi, I really like the approach. I've two major questions

Do we need separate interfaces for querying and locking? IMHO that's something that can be on a single one
What is the advantage of having SnapshotterProvider and don't use service loader directly?

Do we need separate interfaces for querying and locking? IMHO that's something that can be on a single one
What if a user want just customize the query and not the locking? Having a single interface the user is forced to implement also the lock part.
What is the advantage of having SnapshotterProvider and don't use service loader directly?
In that way we can avoid to have the init method in the interface to pass all the useful dependencies, instead they are passed in  the create(Configuration config); and then let the implementation have the constructor.

In that way we can avoid to have the init method in the interface to pass all the useful dependencies, instead they are passed in the create(Configuration config); and then let the implementation have the constructor.

I would also suggest considering how you can explore using the Service marker contract in this context.  In this regard, you may even be able to avoid the Configuration entirely and instead rely on the ServiceRegistry to inject the BeanRegistry which could pass the connector's pre-created configuration object.  This would allow us to centralize all the configuration handling in this connector config object for parsing, etc.

In that way we can avoid to have the init method in the interface to pass all the useful dependencies, instead they are passed in the create(Configuration config); and then let the implementation have the constructor.

I would also suggest considering how you can explore using the Service marker contract in this context. In this regard, you may even be able to avoid the Configuration entirely and instead rely on the ServiceRegistry to inject the BeanRegistry which could pass the connector's pre-created configuration object. This would allow us to centralize all the configuration handling in this connector config object for parsing, etc.

@Naros, good point. What is the difference between the ServiceRegistry and the BeanRegistry?

@Naros, good point. What is the difference between the ServiceRegistry and the BeanRegistry?

So, ServiceRegistry implements a service locator and dependency injection-style solution. You ask the registry for a given service by class type; whatever service is registered by that type will be returned. If no one has asked for that service but it has an initiator (registration) present, the service will be created on the fly before being returned. This helps simplify things like start/stop handlers as services are constructed in the order requested and torn down in the reverse order on connector shutdown. Additionally, a service can annotate a method with @InjectService to have other services dependency injected. I hope that over time, we can move legacy code to services and that new features can be designed as services.
The BeanRegistry itself is one service, and it's designed to simply provide a key/value store of beans by some name. This way, rather than having a constructor with a dozen arguments, we can either (a) pass the registry or (b) utilize the service locator pattern to inject the registry into the class directly to begin to clean up some of these very large ctor argument lists.

Do we need separate interfaces for querying and locking? IMHO that's something that can be on a single one

What if a user want just customize the query and not the locking? With a single interface the user is forced to implement also the lock part.

Not sure how much it's likely, but I can also imagine users want to use various combinations of locking mode and snapshot query. With separate interfaces you can easily create whatever combination you like.

Wondering if SnapshotQuery is not a a duplicate of snapshot.select.statement.overrides.
This is the way the snapshotter query is currently used:
https://github.com/debezium/debezium/blob/c1b7e68319ee56c176ee87cd2c3a3509f708f23c/debezium-core/src/main/java/io/debezium/relational/RelationalSnapshotChangeEventSource.java#L650C5-L668C6
It think `snapshot.select.statement.overrides' is more flexible and easy to use. @jpechane WDYT?

I reviewed the implementation to use the Debezium Service and ServiceProvider. All the interfaces are load via Java SPI and then registered in our DefaultServiceRegistry.
Currently the provided implementations are the one available for PostgreSQL and with the subsequents PRs  the goal is to put all the possible implementation inside the debezium-core module. To provide custom implementation, for example for Snapshotter interface,  users can implement the interface and the set the snapshot.mode property to custom and the provide its name through snapshot.mode.custom.name. Same approach is used for SnapshotQuery and SnapshotLock.


Proposal for Solium Integration

Hi!
We recently released Solium v1 in production.
The Linter contains checks for both style and security practices in Solidity and comes with an automatic code fixer.
I'd love to send in a PR that puts a Solium linting section under the Analysis tab of Remix.
Since some of the security rules exist both in solium as well as remix's static analyser, we will simply disable those rules in Solium so users don't get the same error twice.
In my opinion, the biggest benefits of having Solium in Remix are:

Automatic code fixing for lazy devs!
Remix will not only have security, but also style checking (with no extra development overhead)
Solium can fill the gap for security checks that don't yet exist in remix.

If this seems interesting to you, let me know:)

Maybe solium could have it's own tab or also a section in a tab, but would sit there inside an iframe.
So iframes can be locked down pretty strongly and the parent does not have access to it's content and vice versa. They can then communicate in a very controlled way through the .postMessage API.
It protects both applications from each other.
Do you think this is something that might be able to do the job well?
So the idea is to expose all kinds of API's between remix and "plugins" (which would then more work as little "microservices?" :-) ...which all run in the same browser window communicating via .postMessage
Any thoughts?

@serapath was that directed to me or the maintainers? (If me, then) I think the idea of isolation you're suggesting is a really good one, but I'm not sure if it will play well with the --fix option of solium (this option automatically modifies code to fix a lot of lint errors, so requires write access to the contracts).
Can comment on anything related to the technicalities of remix since I haven't explored developing on it yet (I was planning to do it once this proposal gets approved).

@serapath I think that's a good idea although I m not really sure this iframe solution will met all the requirement.
@duaraghav8 is Solium works in the browser?

@yann300 Solium comes as a JS library, so yes it will

Yes it is a good idea. we plan to make possible to add plugins in Remix. We just started working on it. So yes we'll get back to you asap.

what is the best way to PM you?

@yann300<EMAIL_ADDRESS>or duaraghav8 if you're on Gitter

but I'm not sure if it will play well with the --fix option of solium (this option automatically modifies code to fix a lot of lint errors, so requires write access to the contracts).

Sorry it took some time to come back to this.
Yes, this should not be a problem.
The iframe api can be used to send code and retrieve it back from the plugin after it has been modified.

Maybe just make a wish how or what you would like to receive from the remix-ide to do your work and what kind of API you would expose for that when your module is embedded in an iframe inside a tab in the remix-ide
also what data would you give back to the remix-api after the job is done or what kind of additional methods would you wish or need remix-ide to have to ease your plugins tasks? :-)

@serapath I think the integration has already started, but not sure what is the status @duaraghav8
( could you join remix-dex channel in gitter ? )

The development has started and this issue can be closed since @yann300 has clarified everything. I've joined the channel, the plugin will be delayed because we have a few bug fixes in Solium on higher priority.


[Linux] check for WiFi interface name when obtaining its MAC address

Fixes #23788 [Platform] Properly obtain primary WiFi MAC address on Linux
Currently we're only checking for address not associated with the loopback
interface, which will be problematic for devices with multiple network interfaces.
So updated it to explicitly check for WiFi interface based on the interface name
specified in CHIPDeviceConfig.

Please run './scripts/helpers/restyle-diff.sh' to solve the restyled issue

Remote branch is messed up, will create a new PR on a new branch.


Translate wrongly for JQuery TypeScript

TypeScript Version:  2.0.3 / nightly (2.1.0-dev.201xxxxx)
Code
// A *self-contained* demonstration of the problem follows...
import jq from "jquery";
jq('#pnlTest').text(apple.eat());

Expected behavior:
const jquery_1 = require("jquery");
jquery_1('#pnlTest').text(apple.eat());
Actual behavior:
const jquery_1 = require("jquery");
jquery_1.default('#pnlTest').text(apple.eat());

These issues should be raised at DefinitelyTyped/DefinitelyTyped as they are third party typings not directly managed by this repository.

@kitsonk That doesn't effect the TypeScript emit, the behavior here has to do with allowSyntheticDefaultImports and misunderstandings related ES6 code consuming commonjs when an interop aware environment does a mapping like exports.default = module.exports.
At any rate the emitted code is correct, and if it TypeChecks, then allowSyntheticDefaultImports is true because jquery.d.ts uses
declare var $: JQueryStatic;
export = $;

@avatar21 the translation is correct. Because the syntax
import $ from 'jquery';

is sugar for the precisely equivalent
import { $ as default } from 'jquery';

which is precisely and correctly transpiled to
const $_1 = require("jquery");
$_1.default('#pnlTest').text(apple.eat());

when the module setting is commonjs, which it is by default.

It is still confusing for new comers to TypeScript, especially they have experience with Babel.
I also encounter these problems when using new library of which the document is written in ES6 (because they also support commonjs).
If you are using commonjs, I still think import $ = require('jquery') is the least confusing one. Though it has been deprecated.

Default imports import the default. That's what it means.


Uat elevation mixins update gh

@RachelRVasquez @fuhton does this PR fix the broken layout on the Layout.html page?
https://www.evernote.com/l/ABfdJI9U0fJLHIJpxzOW048gN-wgTnT4XUo
Or should I open a new issue?

@pmmck This resolved this issue.


Relative paths to $HOME

I synced my home directory from another computer that has a different username and /home path. This causes bin to think my executables should be in a different directory between each computer.
Is it possible to download and track downloads in a relative path to my $HOME folder instead of full path?
If it's not possible can you let me know where bin stores the information about the downloads so I can ignore it between my computer syncs? I see it's trying to open .bin/config.json but I don't actually have that file on my computer so I'm not sure where the config is being stored.

Reading through #89 I was able to find the config file is actually ~/.config/bin/config.json and can see the full paths in that file.
I tried changing the $HOME path to ~/ but and update still didn't work
bin update
   • ~/.local/bin/bin v0.11.0 -> v0.12.0 (https://github.com/marcosnils/bin/releases/tag/v0.12.0)

Do you want to continue? [Y/n] y
   • Getting v0.12.0 release for marcosnils/bin
   • Starting download of https://github.com/marcosnils/bin/releases/download/v0.12.0/bin_0.12.0_Linux_x86_64
14.36 MiB / 14.36 MiB [-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 243.28 MiB p/s 0s
   ⨯ command failed            error=Error installing binary open ~/.local/bin/bin: no such file or directory

I synced my home directory from another computer that has a different username and /home path. This causes bin to think my executables should be in a different directory between each computer.

that's a good observation... I'm not sure if we can do something like ~/ since that's shell expantion but setting something $HOME/bin is definitely possible. Would that be enough for your case Justin?

Just pushed a branch here with the env variable feature. I've did some preliminary tests but I'd love if you can help me validate that your use-case is still solved :pray:
https://github.com/marcosnils/bin/pull/new/feat/env_path

Cloned that branch and built bin and removed/installed a binary. I don't see anything different at the CLI and the path in ~/.config/bin/config.json still shows the full home path.
I also tried to update the default_path setting in the config to $HOME/.local/bin and reinstalled an app but it still shows the full path in the config. Is there something else I'm missing that I need to do?

Hey Justin,
This PR doesn't help with bin first install. Since the default_path is initially calculated given the writable paths existing in your user's  $PATH variable.
What this PR does is allowing you to use variables like $HOME in your default_path config variable and bin will resolve them.
In your case, if you're moving to a new machine, you need to make sure that the config file uses the $HOME variable and then after installing bin with this path applied it should resolve the correct path.
Does it make sense?

I also tried to update the default_path setting in the config to $HOME/.local/bin and reinstalled an app but it still shows the full path in the config

Good catch, I missed making this change. I only implemented resolving the variables from the default_path, but it doesn't cascade to binaries path. I'll update this today.

Good catch, I missed making this change. I only implemented resolving the variables from the default_path, but it doesn't cascade to binaries path. I'll update this today.

@rothgar finally found the time to update the PR. Would love some feedback if you find the time :pray:

I built from PR 3 in a fresh machine and see an error on first run.
./bin help
   ⨯ Error loading config file open /home/jgarr/.bin/config.json: no such file or directory

I created the ~/.bin directory and touched ~/.bin/config.json. I ran bin again and was prompted for the default folder but nothing was written to the file. I kept getting prompted for the default folder each time until I installed a binary.

weird... I'll check this. Thx for reporting

Once I installed a binary the config.json file was written to but still had hard coded home paths.

Yes, if you replace your default_path with "$HOME/.local/bin"  and install binaries after that, it should preserve the path correctly.

./bin help
⨯ Error loading config file open /home/jgarr/.bin/config.json: no such file or directory

This is strange, shouldn't have done that. Checking

Just ran bin in a fresh box and got this:
marcos:Projects/bin (feat/package_path) (⎈ |N/A)$ ./bin help

Pick a default download dir:

 [1] /home/marcos/bin
 [2] /home/marcos/.rbenv/shims
 [3] /home/marcos/.cargo/bin
 [4] /home/marcos/.rbenv/shims
 [5] /home/marcos/.krew/bin
 [6] /home/marcos/bin
 [7] /home/marcos/.nvm/versions/node/v16.13.2/bin
 [8] /home/marcos/bin
 [9] /home/marcos/Projects/gopath/bin
 [10] /home/marcos/.fzf/bin
 Select an option:

If you have the time, can you try bin --debug in a fresh install?

ok, I deleted ~/.bin and re-ran the commands
./bin --debug
   • debug logs enabled
   ⨯ Error loading config file open /home/jgarr/.bin/config.json: no such file or directory

❯ mkdir -p ~/.bin

❯ ./bin --debug
   • debug logs enabled
   • User PATH is [/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/jgarr/bin:/home/jgarr/.local/bin:/home/jgarr/.krew/bin:/home/jgarr/.toolbox/bin:/home/jgarr/go/bin://home/jgarr/.cargo/bin:/home/jgarr/.krew/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/jgarr/.rvm/bin]
   • Checking path /home/linuxbrew/.linuxbrew/bin
   • /home/linuxbrew/.linuxbrew/bin seems to be a dir and writable, adding option.
   • Checking path /home/linuxbrew/.linuxbrew/sbin
   • /home/linuxbrew/.linuxbrew/sbin seems to be a dir and writable, adding option.
   • Checking path /home/jgarr/bin
   • Error [no such file or directory] checking path
   • Checking path /home/jgarr/.local/bin
   • /home/jgarr/.local/bin seems to be a dir and writable, adding option.
   • Checking path /home/jgarr/.krew/bin
   • Error [no such file or directory] checking path
   • Checking path /home/jgarr/.toolbox/bin
   • Error [no such file or directory] checking path
   • Checking path /home/jgarr/go/bin
   • Error [no such file or directory] checking path
   • Checking path //home/jgarr/.cargo/bin
   • Error [no such file or directory] checking path
   • Checking path /home/jgarr/.krew/bin
   • Error [no such file or directory] checking path
   • Checking path /usr/local/bin
   • Error [permission denied] checking path
   • Checking path /usr/local/sbin
   • Error [permission denied] checking path
   • Checking path /usr/bin
   • Error [permission denied] checking path
   • Checking path /usr/sbin
   • Error [permission denied] checking path
   • Checking path /bin
   • Error [permission denied] checking path
   • Checking path /sbin
   • Error [permission denied] checking path
   • Checking path /home/jgarr/.rvm/bin
   • Error [no such file or directory] checking path

Pick a default download dir:

 [1] /home/linuxbrew/.linuxbrew/bin
 [2] /home/linuxbrew/.linuxbrew/sbin
 [3] /home/jgarr/.local/bin
 Select an option: 3
   • Download path set to /home/jgarr/.local/bin

 Path   Version         URL     Status

❯ echo '{"default_path": "$HOME/.local/bin"}' > ~/.bin/config.json

❯ ./bin install --debug github.com/kubernetes-sigs/kind
   • debug logs enabled
   • Download path set to $HOME/.local/bin
   • Getting latest release for kubernetes-sigs/kind
   • Checking binary form https://github.com/kubernetes-sigs/kind/releases/download/v0.11.1/kind-linux-amd64
   ⨯ command failed            error=Error installing binary open $HOME/.local/bin: no such file or directory

I verified that ~/.local/bin directory exists

hey @rothgar ! just pushed another PR with a feature to allow the user to set a custom PATH when initializing bin the first time. However, I couldn't reproduce your error which seems to be correlated to a fix that I've recently pushed here:  https://github.com/marcosnils/bin/commit/496a2ba72a073cf42e09e3b6ffb359dd00043e9c
marcos:Projects/bin (feat/env_path) (⎈ |N/A)$ go build && ./bin --debug install github.com/kubernetes-sigs/kind
   • debug logs enabled
   • Config directory is: /home/marcos/.config/bin
   • User PATH is [/home/marcos/bin:/home/marcos/miniconda3/bin:/home/marcos/.rbenv/plugins/ruby-build/bin:/home/marcos/.rbenv/shims:/home/marcos/.rbenv/bin:/home/marcos/.cargo/bin:/home/marcos/miniconda3/bin:/home/marcos/.rbenv/plugins/ruby-build/bin:/home/marcos/.rbenv/shims:/home/marcos/.rbenv/bin:/home/marcos/.krew/bin:/home/marcos/bin:/home/marcos/.nvm/versions/node/v16.13.2/bin:/home/marcos/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin:/home/marcos/.yarn/bin:/home/marcos/Projects/gopath/bin:/usr/share/bcc/tools/:/home/marcos/.luarocks/bin:/home/marcos/.fzf/bin]
   • Checking path /home/marcos/bin
   • /home/marcos/bin seems to be a dir and writable, adding option.
   • Checking path /home/marcos/miniconda3/bin
   • Error [Error setting download path [stat /home/marcos/miniconda3/bin: no such file or directory]] checking path
   • Checking path /home/marcos/.rbenv/plugins/ruby-build/bin
   • Error [Error setting download path [stat /home/marcos/.rbenv/plugins/ruby-build/bin: no such file or directory]] checking path
   • Checking path /home/marcos/.rbenv/shims
   • /home/marcos/.rbenv/shims seems to be a dir and writable, adding option.
   • Checking path /home/marcos/.rbenv/bin
   • Error [Error setting download path [stat /home/marcos/.rbenv/bin: no such file or directory]] checking path
   • Checking path /home/marcos/.cargo/bin
   • /home/marcos/.cargo/bin seems to be a dir and writable, adding option.
   • Checking path /home/marcos/miniconda3/bin
   • Error [Error setting download path [stat /home/marcos/miniconda3/bin: no such file or directory]] checking path
   • Checking path /home/marcos/.rbenv/plugins/ruby-build/bin
   • Error [Error setting download path [stat /home/marcos/.rbenv/plugins/ruby-build/bin: no such file or directory]] checking path
   • Checking path /home/marcos/.rbenv/shims
   • /home/marcos/.rbenv/shims seems to be a dir and writable, adding option.
   • Checking path /home/marcos/.rbenv/bin
   • Error [Error setting download path [stat /home/marcos/.rbenv/bin: no such file or directory]] checking path
   • Checking path /home/marcos/.krew/bin
   • /home/marcos/.krew/bin seems to be a dir and writable, adding option.
   • Checking path /home/marcos/bin
   • /home/marcos/bin seems to be a dir and writable, adding option.
   • Checking path /home/marcos/.nvm/versions/node/v16.13.2/bin
   • /home/marcos/.nvm/versions/node/v16.13.2/bin seems to be a dir and writable, adding option.
   • Checking path /home/marcos/bin
   • /home/marcos/bin seems to be a dir and writable, adding option.

   • Checking path /usr/share/bcc/tools/
   • Error [Error setting download path [stat /usr/share/bcc/tools/: no such file or directory]] checking path
   • Checking path /home/marcos/.luarocks/bin
   • Error [Error setting download path [stat /home/marcos/.luarocks/bin: no such file or directory]] checking path
   • Checking path /home/marcos/.fzf/bin
   • /home/marcos/.fzf/bin seems to be a dir and writable, adding option.

Pick a default download dir:

 [1] /home/marcos/bin
 [2] /home/marcos/.rbenv/shims
 [3] /home/marcos/.cargo/bin
 [4] /home/marcos/.krew/bin
 [5] /home/marcos/.nvm/versions/node/v16.13.2/bin
 [6] /home/marcos/Projects/gopath/bin
 [7] /home/marcos/.fzf/bin
 Select an option or type a custom value: $HOME/bin
   • Download path set to $HOME/bin
   • Getting latest release for kubernetes-sigs/kind
   • Removing kind-windows-amd64 (URL https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673593) with score 11 lower than 21
   • Removing kind-linux-ppc64le (URL https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673582) with score 16 lower than 21
   • Removing kind-linux-arm64 (URL https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673571) with score 16 lower than 21
   • Keeping kind-linux-amd64 (URL https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673562) with highest score 21
   • Removing kind-darwin-arm64 (URL https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673552) with score 6 lower than 21
   • Removing kind-darwin-amd64 (URL https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673531) with score 11 lower than 21
   • Checking binary from https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673562
   • Starting download of https://api.github.com/repos/kubernetes-sigs/kind/releases/assets/37673562
6.50 MiB / 6.50 MiB [-------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1.99 MiB p/s 4s
   • Copying for<EMAIL_ADDRESS>into /home/marcos/bin/kind

Mind doing one last update and trying again? Make sure your env_path is fully updated and doesn't have any missing commits :pray:

Oh, I was building off pr/3 and not the env_path branch. I just rebuilt from the correct branch and it looks like it's working for me. 👍


[vcpkg] Add aria2 as vcpkg required tool (step 1)

Related: https://github.com/microsoft/vcpkg/pull/20585.

@BillyONeal Ready to merge?


Kmeans notebook cleanup

If you haven't already, please read through our contributing guidelines before opening your PR
What is the purpose of this PR?
Closes #876.
How did you implement your changes
Added more clear descriptions of what is actually being shown by the generated heat maps. Also fixed a few pandas warning messages that were clogging up the notebook outputs.
The visualizations all needed to be updated as well to reflect the example dataset with a cropped fov1.
Remaining issues
If anyone has specific additions they want added into the kmeans notebook, let me know!

[ ] Possibly add more visualizations specifically for if someone used neighbor frequencies as their input.
[ ] The kmeans notebook always regenerates the neighborhood matrix. It might be helpful to be able to simply read in a previously generated matrix, like how we have in the mixing score notebook.

I think it looks great! Just some minor suggestions for wording, but open to other suggestions.
Not sure if it falls in the scope of this pull request, but I think we had discussed before replacing the example data in this notebook (which are some random points from the UCSF dataset I think) with the example dataset that is used in the other notebooks. Do we want to do that here?


[Feature Request] Install Content While Closed

Please make it so content continue's to install while closed. If its necessary and possible, maybe disable the true sleep mode and just turn the screens and audio off when closed.

AFAIK this is already working while installing CIAs from SD card, but not working while installing from the network (sockfile and http)