Check performance on big databases (clickhouse datasets)

a) yandex metrica dataset (~8gb compressed)
data: https://clickhouse.com/docs/en/getting-started/example-datasets/metrica/
schema(hits): https://clickhouse.com/docs/en/getting-started/tutorial/
queries: https://clickhouse.com/benchmark/dbms/
b) github dataset (~75gb compressed)
https://ghe.clickhouse.tech
https://ghe.clickhouse.tech/#download-the-dataset

I tried to load github dataset (~75 compressed Gb).
It takes 4.5 hours to load 1.2 billion records (total size is 3.1 billion records) after it 1Tb partition space is exhausted.
Memory usage of pageserver is  constant (1Gb), compute instance of postgres consumes 100% CPU, pageserver  17%, decompressor 27% and 44% - ch2pg converter.
Size of repository is almost the same as size of generated WAL, which makes me think that size of layered repo is almost the same as size of vanilla postgres database.
Such large amplification factor (more than 25 times) is really strange, but looks like it is problem of vanilla postgres, not zenith. I will try to load data in buffered repo with enabled compression. Just to check if "mission is possible".

I have implemented compression for image layers in layered repo (https://github.com/zenithdb/zenith/pull/1002) and was able to load this data set in ~7 hours:
COPY<PHONE_NUMBER>
Time: 25759837,648 ms (07:09:19,838)

403Gb
133188  files

Unfortunately first "select count(*)" query didn't completed because of disk space exhaustion.
It's not quite clear to me why it happen. First of all pageserver generates 45Gb of log.
Most of info! messages in layered_repoistory.rs should be replaced with trace!.
But looks like it is not the only problem causing space exhaustion.
Size of layered repo increased from 403Gb to 596Gb. Obviously autovacuum and "select count()" are trying to set hint bits on the pages. And it should cause almost rewriting of the while database.
This is why I expected that execution of this "select count()" query will take almost the same time (~7hours)  as initial copy. May be faster, because of parallel plan. But it not clear to me why GC doesn't remove old layers.
And after disk space exhaustion and crash of postgres, I was not able to restart compute node because of another problem: GC horizon now doesn't take in account disk_consistent_lsn. So GC removes versions which are not yet flushed to the disk. As a result basebackup is failed because requested version is already garbage collected.
Minor issue is presence of 2021-12-16T07:07:23.630855Z  WARN loading timeline{timeline=ed2f594420b74a350b1300f4bda91015 tenant=fad672e3e3ab90ea305854ff01345c7a}: unrecognized filename in timeline dir: ephemeral-153863
warnings in the pageserver.log. list_files() should be aware about presence of ephemeral files.

Loading layer map takes 50 minutes and use 10Gb of RAM:(
It seems to be absolutely not acceptable. May be it is partly caused by my support of compression of image layers (it requires mapping for each chapter=page in book)

One more unclear thing with this incident: prev_record_lsn in metadata file is not set. I do not understand now how it is possible.

One more unclear thing with this incident: prev_record_lsn in metadata file is not set. I do not understand now how it is possible.

That's normal, see this comment in checkpoint_internal:
            // We can only save a valid 'prev_record_lsn' value on disk if we
            // flushed *all* in-memory changes to disk. We only track
            // 'prev_record_lsn' in memory for the latest processed record, so we
            // don't remember what the correct value that corresponds to some old
            // LSN is. But if we flush everything, then the value corresponding
            // current 'last_record_lsn' is correct and we can store it on disk.
            let ondisk_prev_record_lsn = if disk_consistent_lsn == last_record_lsn {
                Some(prev_record_lsn)
            } else {
                None
            };

So we can only set prev_record_lsn in the metadata file, if no more WAL records have been processed after the on_disk_consistent_lsn. That's usually not the case, except when the pageserver is being shut down.
That's OK. It works, because if there are any WAL records after the disk_consistent_lsn, then when the pageserver restarts, it will fetch those records from the safekeeper and process them again, and that will set the in-memory prev_record_lsn value again.

More info from "big data":
Vacuum analyze is active during 8 hours and proceeded  just half of the table.
CPU usage:
pageserver: 20% (with periodic 80% peaks)
wal-redo 15%
vacuum: 2%

More info from "big data": Vacuum analyze is active during 8 hours and proceeded just half of the table. CPU usage: pageserver: 20% (with periodic 80% peaks) wal-redo 15% vacuum: 2%

Hmm. I'm guessing that the vacuum process spends most of its time waiting on fetching the pages from the page server. Let's do some quick back of the envelope calculation:
number of pages processed = (400 GB / 2) / 8 kB = 26214400
time spent = 8h = 28800000 ms
time per page = (28800000 ms) / 26214400 ~= 1 ms / page

Now, that was 200 GB compressed, and I'm not sure what the compression ratio was, but if we guess that it was 2, then the time per page would be 0.5 ms. 0.5 ms latency is pretty good I think.
However, since it's a sequential scan, we should be able to do much better than that by prefetching. The pageserver clearly had a lot of CPU capacity left. Sooner or later we need to implement prefetching...

I am running queries now...
The speed is not so bad: disk read rate is about 250..350Mb/sec. Looks like we are mostly limited by disk speed.
I have two main concerns:

Size of pageserver is increased to 16Gb. May be it is caused by large chapters hash map in book caused by compression. I will check.
Time of loading layer map is about one hour. It is completely unacceptable. May be it is also caused by compression and reading large hash table. But I do not think. Just reading 150k files can not be fast.

2. Time of loading layer map is about one hour. It is completely unacceptable. May be it is also caused by compression and reading large hash table. But I do not think. Just reading 150k files can not be fast.

150k files in itself is not that much. I wrote a little test program and ran it on my laptop:
//! Test program that creates, reads, and then deletes 200k files in
//! current directory
use std::fs;
use std::fs::File;
use std::io::prelude::*;

fn main() -> std::io::Result<()> {
    let num_files = 200000;

    // create 200k files
    for i in 1..=num_files {
        let mut file = File::create(format!("foo-{}.txt", i))?;
        file.write_all(b"Hello, world!")?;
    }
    println!("created {} files", num_files);

    // read them all
    for i in 1..=num_files {
        // read file, check the contents
        let mut file = File::open(format!("foo-{}.txt", i))?;
        let mut contents = String::new();
        file.read_to_string(&mut contents)?;
        assert_eq!(contents, "Hello, world!");
    }
    println!("read {} files", num_files);

    // delete them again
    for i in 1..num_files {
        fs::remove_file(format!("foo-{}.txt", i))?;
    }
    println!("deleted {} files", num_files);

    Ok(())
}

This runs in about 6s:
/data/lotsoffiles$ time ./lots_of_files
created 200000 files
read 200000 files
deleted 200000 files

real	0m6.161s
user	0m1.117s
sys	0m5.016s

btw, same script on macos:
> time ./ftest
created 200000 files
read 200000 files
deleted 200000 files
./ftest  3.10s user 67.26s system 66% cpu 1:46.35 total

I think it is so fast because files are just created and cached in memory.
[ec2-user@ip-172-31-18-140 7e05a1d9d4b6d6bf7a312f46d93479c8]$ time find . -name rel\* -exec ls -l {} \; | wc -l

133540

real	4m17.966s
user	2m33.787s
sys	1m49.583s

But what is actually slow:
[ec2-user@ip-172-31-18-140 7e05a1d9d4b6d6bf7a312f46d93479c8]$ time find . -name rel\* -exec /data/zenith/target/release/dump_layerfile {} \; > layer.map
real    90m10.131s
user    43m56.031s
sys     45m4.689s

github-queries.zip
Execution of 62 (out of 81) queries is completed in about 27 hours (comparing with few seconds in case of clickhouse).
But it is not a problem of Zenith - I expect that Vanilla will show similar result.It is just result of columnar storage with efficient compression and efficient executor.

Normal shutdown of page server takes 5 minutes: as far as I see in debugger - this time is spent in saving in-memory layers. I wonder what we are storing in open layers if last 48 hours I run only read-only queries?
Certainly, Postgres performs some updates itself (setting hints, autovacuum,...) But vacuum analyze was completed long ago and all hints also should by sent because table was traversed multiple times...
 LayeredTimeline::init_current_logical_size  enforce this load. I wonder if we really need to maintain incremental logical relation size and do it in this way...

With commented LayeredTimeline::init_current_logical_size  pageserver launches immediately (< second).

One more notice: execution of the first query after restart cause loading of layer map to get relation size. And it is done in single thread and takes significant amount of time (~50 minutes). So unlike sequential scan, which is using parallel plan and takes ~30 minutes, time of first query execution is 50 + 30 minutes.

Time of loading uncompressed data: 07:48:43 (about 45 minutes larger than with enabled compression, but this is different VM with larger SSD and less CPU cores).
Size of repository: 1.3Tb (v.s. 0.4Tb with compression). Size of generated WAL: 2.3Tb
Node restart time (without my fix for get_relish_size) - 43 seconds.
It certainly much smaller than 50 minutes in case of compressed database, but still long enough (it means that node will be inaccessible about one minute).
So optimization of laze loading layers is still needed.
@hlinnaka what do you think about  https://github.com/zenithdb/zenith/pull/1035 ?
Time of sequential scan (with parallel plan) - 28 minutes. It is about 3 times slows than with compressed database. But once again: different VM with less cores, so it may be not related with compression.

Some more results. At new EC2 storage VM select speed is more than 3 times slow (>30minutes vs 9 minutes).
It is caused by less number of CPU cores. It can be considered as good news: looks like pageserver performance (at least for get_page_at_lsn) is well scaling and depends on number of cores. Nothing surprising here, because all pages should be already reconstructed, so wal_redo postgres should not be a bottleneck.
But there is one strange thing I observed: I size of database is  1.3Tb, but execution of  selects produce 5.3Tb of WAL which cause disk space exhaustion. All this 5.3Tb consists of FPI_FOR_HINT wal records. I wonder why it is necessary to set hint bits more than once? Or may be be we are loosing some FPI updates... In any case I am going to investigate it, because there is something definitely wrong in the fact that 75Gb of compressed original text data cause sending through network 8Tb of WAL.

Looks like it is really HEAP_XMIN_COMMITTED which is set for different records in the page at different moments of time, which cause writing this page multiple times.
I think we should really try to remove wal_log_hints=on from Zenith compute node configuration.

Time of loading database for vanilla Postgres is almost the same as for Zenith: 8 hours. So Zenith is even slightly faster:)
Size is the same: 1.3Tb.

Time of sequential scan of vanilla Postgres is 10 minutes. It is 3 times faster than of Zenith at the same VM, but slower than previous VM with compressed databases... Which is 3 times smaller.

Some more information about execution of seqscan at zenith (select count(*) from github_events):
CPU usage:
Tasks: 204 total,   7 running, 105 sleeping,   0 stopped,   0 zombie
%Cpu(s): 34,6 us, 29,3 sy,  0,0 ni, 25,0 id,  0,5 wa,  0,0 hi, 10,6 si,  0,0 st
KiB Mem : 97724032 total,  4414864 free,  6157892 used, 87151280 buff/cache
20812 ec2-user  20   0 8599192   5,5g   4304 S 470,4  5,9   2568:04 pageserver
 9750 ec2-user  20   0 1134664  14952  13508 R  50,8  0,0   0:57.89 postgres
 9737 ec2-user  20   0 1135620  18224  16164 R  50,5  0,0   0:57.91 postgres
 9751 ec2-user  20   0 1134664  15296  13852 R  50,5  0,0   0:57.44 postgres
 9754 ec2-user  20   0 1134664  14928  13488 S  50,5  0,0   0:57.83 postgres
 9748 ec2-user  20   0 1134732  15608  14144 S  50,2  0,0   0:57.66 postgres
 9749 ec2-user  20   0 1134664  15452  14004 S  50,2  0,0   0:57.36 postgres
 9752 ec2-user  20   0 1134664  14756  13312 S  50,2  0,0   0:57.54 postgres
 9753 ec2-user  20   0 1134732  15628  14188 S  49,8  0,0   0:57.50 postgres

Profile of page server:
  10,88%  serving Page Se  [kernel.kallsyms]     [k] __lock_text_start
   9,58%  serving Page Se  [kernel.kallsyms]     [k] finish_task_switch
   4,32%  serving Page Se  [kernel.kallsyms]     [k] copy_user_enhanced_fast_str
   3,66%  serving Page Se  libc-2.26.so          [.] __memmove_avx_unaligned_erm
   2,10%  serving Page Se  libpthread-2.26.so    [.] __pthread_rwlock_rdlock
   1,86%  serving Page Se  libpthread-2.26.so    [.] __libc_recv
   1,74%  serving Page Se  libc-2.26.so          [.] _int_malloc
   1,25%  serving Page Se  [kernel.kallsyms]     [k] syscall_enter_from_user_mod
   1,21%  serving Page Se  libpthread-2.26.so    [.] __libc_send
   1,15%  serving Page Se  libc-2.26.so          [.] __memset_avx2_erms
   1,12%  serving Page Se  libpthread-2.26.so    [.] __GI___libc_pread64
   1,05%  serving Page Se  [kernel.kallsyms]     [k] rmqueue_pcplist.constprop.0
   0,95%  serving Page Se  libc-2.26.so          [.] malloc
   0,93%  serving Page Se  libc-2.26.so          [.] _int_free

iotop:
At the beginning of query (~10 minutes):
Total DISK READ :    1129.52 M/s | Total DISK WRITE :       0.00 B/s
Actual DISK READ:    1129.52 M/s | Actual DISK WRITE:       0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 9755 be/4 ec2-user  141.45 M/s    0.00 B/s  0.00 %  1.40 % pageserve~g Page Se]
 9757 be/4 ec2-user  140.11 M/s    0.00 B/s  0.00 %  1.38 % pageserve~g Page Se]
 9756 be/4 ec2-user  141.08 M/s    0.00 B/s  0.00 %  1.36 % pageserve~g Page Se]
 9760 be/4 ec2-user  142.31 M/s    0.00 B/s  0.00 %  1.36 % pageserve~g Page Se]
 9758 be/4 ec2-user  140.96 M/s    0.00 B/s  0.00 %  1.32 % pageserve~g Page Se]
 9761 be/4 ec2-user  142.18 M/s    0.00 B/s  0.00 %  1.28 % pageserve~g Page Se]
 9759 be/4 ec2-user  141.09 M/s    0.00 B/s  0.00 %  1.27 % pageserve~g Page Se]
 9738 be/4 ec2-user  140.35 M/s    0.00 B/s  0.00 %  1.23 % pageserve~g Page Se]

At the end of query:
Total DISK READ :     269.64 M/s | Total DISK WRITE :     457.99 M/s
Actual DISK READ:     269.64 M/s | Actual DISK WRITE:     300.82 M/s
  TID  PRIO  USER     DISK READ DISK WRITE>  SWAPIN      IO    COMMAND
 9731 be/4 ec2-user   15.76 K/s  203.78 M/s  0.00 %  0.00 % pageserve~ceiver th]
 9754 be/4 ec2-user    0.00 B/s   63.04 M/s  0.00 % 15.54 % postgres:~r PID 9737
 9749 be/4 ec2-user    0.00 B/s   63.04 M/s  0.00 %  5.57 % postgres:~r PID 9737
 9752 be/4 ec2-user    0.00 B/s   31.52 M/s  0.00 %  0.00 % postgres:~r PID 9737
 9737 be/4 ec2-user    0.00 B/s   31.52 M/s  0.00 %  6.60 % postgres:~0) EXPLAIN
 9751 be/4 ec2-user    0.00 B/s   31.52 M/s  0.00 %  0.00 % postgres:~r PID 9737
 9750 be/4 ec2-user    0.00 B/s   15.76 M/s  0.00 %  0.00 % postgres:~r PID 9737
 9753 be/4 ec2-user    0.00 B/s   15.76 M/s  0.00 %  6.16 % postgres:~r PID 9737
 9748 be/4 ec2-user    0.00 B/s    2.05 M/s  0.00 %  0.00 % postgres:~r PID 9737

Explain analyze:

Finalize Aggregate  (cost=151709859.71..151709859.72 rows=1 width=8) (actual time=2219692.556..2219803.221 rows=1 loops=1)
Buffers: shared read=149835910 dirtied=30356258 written=29476205
->  Gather  (cost=151709858.88..151709859.69 rows=8 width=8) (actual time=2219692.439..2219803.214 rows=8 loops=1)
Workers Planned: 8
Workers Launched: 7
Buffers: shared read=149835910 dirtied=30356258 written=29476205
->  Partial Aggregate  (cost=151708858.88..151708858.88 rows=1 width=8) (actual time=2219687.099..2219687.100 rows=1 loops=8)
Buffers: shared read=149835910 dirtied=30356258 written=29476205
->  Parallel Seq Scan on github_events  (cost=0.00..151334269.10 rows=149835910 width=0) (actual time=1.457..2188547.639 rows=389974750 loops=8)
Buffers: shared read=149835910 dirtied=30356258 written=29476205
Planning Time: 0.053 ms
Execution Time: 2219803.254 ms
(12 rows)
Time: 2219807,793 ms (36:59,808)

So, as you can see if postgres is just reading data, then we doing it with disk speed (~1Gb/sec) and so 1.3Tb database should be traversed in about 10 minutes. But then postgres starts writing hit bits and... it significantly reduce read speed up to 200Mb/sec and total query time is much larger (~30 minutes).

What is strange to me: this query traverse the whole relation.
But hint bits are not set for all tuples. Just some fraction of pages is update. If I repeat this query once again, then I execution time and number of dirtied pages is almost the same.

Normal shutdown of pageserver may take signficant amount of time (~hour in my case), because of enforced checkpoint on close.
#0  0x00007f7a560d5f13 in pread64 () from /lib64/libpthread.so.0
#1  0x000055ce0eb3b2f6 in std::sys::unix::fd::FileDesc::read_at () at library/std/src/sys/unix/fd.rs:108
#2  std::sys::unix::fs::File::read_at () at library/std/src/sys/unix/fs.rs:858
#3  <std::fs::File as std::os::unix::fs::FileExt>::read_at () at library/std/src/os/unix/fs.rs:219
#4  0x000055ce0e4b30ed in <pageserver::virtual_file::VirtualFile as std::os::unix::fs::FileExt>::read_at::{{closure}} (
    file=0x7f7a54ad9010) at pageserver/src/virtual_file.rs:360
#5  pageserver::virtual_file::VirtualFile::with_file (self=<optimized out>, func=...) at pageserver/src/virtual_file.rs:245
#6  <pageserver::virtual_file::VirtualFile as std::os::unix::fs::FileExt>::read_at (self=<optimized out>,
    buf=&mut [u8](size=8192) = {...}, offset=2097152) at pageserver/src/virtual_file.rs:360
#7  0x000055ce0e403ea8 in pageserver::layered_repository::ephemeral_file::EphemeralFile::fill_buffer (
    buf=&mut [u8](size=8192) = {...}, self=<optimized out>, blkno=<optimized out>)
    at pageserver/src/layered_repository/ephemeral_file.rs:80
#8  <pageserver::layered_repository::ephemeral_file::EphemeralFile as std::os::unix::fs::FileExt>::read_at (
    self=<optimized out>, dstbuf=<error reading variable: access outside bounds of object referenced via synthetic pointer>,
    offset=<optimized out>) at pageserver/src/layered_repository/ephemeral_file.rs:126
#9  0x000055ce0e43a5b2 in <pageserver::layered_repository::page_versions::PageVersionReader as std::io::Read>::read (
    self=0x7ffdb8996c88, buf=&mut [u8](size=8192) = {...}) at pageserver/src/layered_repository/page_versions.rs:130
#10 std::io::Read::read_buf::{{closure}} (b=&mut [u8](size=8192) = {...})
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/io/mod.rs:818
#11 std::io::default_read_buf (read=..., buf=<optimized out>)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/io/mod.rs:474
#12 std::io::Read::read_buf (self=0x7ffdb8996c88, buf=<optimized out>)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/io/mod.rs:818
---Type <return> to continue, or q <return> to quit---
#13 std::io::copy::stack_buffer_copy (reader=<optimized out>, writer=0x7ffdb8996938)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/io/copy.rs:135
#14 0x000055ce0e4ec7be in <W as std::io::copy::BufferedCopySpec>::copy_to (reader=0x7ffdb8996c88, writer=0x7ffdb8996938)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/io/copy.rs:75
#15 std::io::copy::generic_copy (reader=0x7ffdb8996c88, writer=0x7ffdb8996938)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/io/copy.rs:64
#16 <std::sys::unix::kernel_copy::Copier<R,W> as std::sys::unix::kernel_copy::SpecCopy>::copy (self=...)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/sys/unix/kernel_copy.rs:144
#17 std::sys::unix::kernel_copy::copy_spec (read=0x7ffdb8996c88, write=0x7ffdb8996938)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/sys/unix/kernel_copy.rs:75
#18 std::io::copy::copy (reader=0x7ffdb8996c88, writer=0x7ffdb8996938)
    at /rustc/23f69235ad2eb9b44ac1a55eeaa3f9b484d9de4a/library/std/src/io/copy.rs:50
#19 pageserver::layered_repository::blob::BlobWriter<W>::write_blob_from_reader (self=0x7ffdb8996938, r=0x7ffdb8996c88)
    at pageserver/src/layered_repository/blob.rs:33
#20 pageserver::layered_repository::delta_layer::DeltaLayer::create (conf=<optimized out>, timelineid=..., tenantid=...,
    seg=..., start_lsn=..., end_lsn=..., dropped=<optimized out>, page_versions=0x7f79a65b2338, cutoff=..., relsizes=...)
    at pageserver/src/layered_repository/delta_layer.rs:458
#21 0x000055ce0e510614 in pageserver::layered_repository::inmemory_layer::InMemoryLayer::write_to_disk (self=0x7f79a65b2300,
    timeline=0x55ce0fcf6210) at pageserver/src/layered_repository/inmemory_layer.rs:633
#22 0x000055ce0e412172 in pageserver::layered_repository::LayeredTimeline::evict_layer (self=0x55ce0fcf6210, layer_id=...)
    at pageserver/src/layered_repository.rs:1595
#23 pageserver::layered_repository::LayeredTimeline::checkpoint_internal (self=0x55ce0fcf6210, checkpoint_distance=0)
    at pageserver/src/layered_repository.rs:1490
#24 0x000055ce0e4075a0 in <pageserver::layered_repository::LayeredTimeline as pageserver::repository::Timeline>::checkpoint::{{closure}} () at pageserver/src/layered_repository.rs:981
#25 prometheus::histogram::Histogram::observe_closure_duration (self=0x7ffdb8997a50, f=...)
    at /home/ec2-user/.cargo/registry/src/github.com-1ecc6299db9ec823/prometheus-0.12.0/src/histogram.rs:715
#26 <pageserver::layered_repository::LayeredTimeline as pageserver::repository::Timeline>::checkpoint (self=0x55ce0fcf6210,
    cconf=<error reading variable: access outside bounds of object referenced via synthetic pointer>)
    at pageserver/src/layered_repository.rs:979
#27 pageserver::layered_repository::shutdown_timeline (tenant_id=..., timeline_id=..., timeline=<optimized out>)
    at pageserver/src/layered_repository.rs:364
#28 0x000055ce0e406783 in <pageserver::layered_repository::LayeredRepository as pageserver::repository::Repository>::shutdown (
    self=0x55ce0fcf6a90) at pageserver/src/layered_repository.rs:286
#29 0x000055ce0e5123af in pageserver::tenant_mgr::shutdown_all_tenants () at pageserver/src/tenant_mgr.rs:146
#30 0x000055ce0e35f227 in pageserver::start_pageserver::{{closure}} (signal=<optimized out>)
    at pageserver/src/bin/pageserver.rs:623
#31 zenith_utils::signals::ShutdownSignals::handle (self=..., handler=...) at /data/zenith/zenith_utils/src/signals.rs:54
#32 0x000055ce0e350079 in pageserver::start_pageserver (conf=0x55ce0fc46a30) at pageserver/src/bin/pageserver.rs:607
#33 0x000055ce0e34deac in pageserver::main () at pageserver/src/bin/pageserver.rs:511

Total DISK READ :     200.00 M/s | Total DISK WRITE :     268.92 M/s
Actual DISK READ:     200.00 M/s | Actual DISK WRITE:     259.36 M/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
20812 be/4 ec2-user  200.00 M/s  268.92 M/s  0.00 % 57.52 % pageserve~-daemonize

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
20812 ec2-user  20   0 9072948   7,4g   6584 D  42,3  7,9   2780:10 pageserver

It seems to be something unexpected, because checkpoint interval is 10 seconds and checkpoint distance is 256Mb.
Memory usage of pageserver is ~10Gb, so huge memory consumption is not caused by compression (large chapter=page map in books).

With wal_log_hints=off query time is reduced to 20 minutes and read speed ~1Gb/sec.
Vanilla read speed is 2Gb and so query time is twice smaller:
 Finalize Aggregate  (cost=154693498.43..154693498.44 rows=1 width=8) (actual time=598915.133..598918.536 rows=1 loops=
1)
   Buffers: shared hit=7027 read=149828883
   ->  Gather  (cost=154693497.60..154693498.41 rows=8 width=8) (actual time=598915.023..598918.527 rows=8 loops=1)
         Workers Planned: 8
         Workers Launched: 7
         Buffers: shared hit=7027 read=149828883
         ->  Partial Aggregate  (cost=154692497.60..154692497.61 rows=1 width=8) (actual time=598910.669..598910.670 ro
ws=1 loops=8)
               Buffers: shared hit=7027 read=149828883
               ->  Parallel Seq Scan on github_events  (cost=0.00..153721180.08 rows=388527008 width=0) (actual time=3.
493..577347.045 rows=389974750 loops=8)
                     Buffers: shared hit=7027 read=149828883
 Planning:
   Buffers: shared hit=22
 Planning Time: 15.066 ms
 Execution Time: 598918.819 ms
(14 rows)

Time: 598934,552 ms (09:58,935)

With 11 parallel workers time of query execution by Zenith is reduced to 12 minutes and read speed to 1.6Gb.
So, looks like 8 workers is not enough to utilize all disk IO speed at pageserver. But the more tenants/parallel worlers will have, the less difference with Vanilla Postgres will be.
In other words: get_page_at_lsn roundtrip adds extra delays comparing to Vanilla Postgres. Prefetch can help here. But if there are multiple active clients and so multiple working threads in pages server, then is extra delay is not critical.

 Finalize Aggregate  (cost=151199055.77..151199055.78 rows=1 width=8) (actual time=749494.704..749495.529 rows=1 loops=
1)
   Buffers: shared read=149835910 dirtied=18417765 written=18417382
   ->  Gather  (cost=151199054.64..151199055.75 rows=11 width=8) (actual time=749494.670..749495.524 rows=12 loops=1)
         Workers Planned: 11
         Workers Launched: 11
         Buffers: shared read=149835910 dirtied=18417765 written=18417382
         ->  Partial Aggregate  (cost=151198054.64..151198054.65 rows=1 width=8) (actual time=749484.789..749484.790 ro
ws=1 loops=12)
               Buffers: shared read=149835910 dirtied=18417765 written=18417382
               ->  Parallel Seq Scan on github_events  (cost=0.00..150925625.71 rows=108971571 width=0) (actual time=1.
176..725032.365 rows=259983167 loops=12)
                     Buffers: shared read=149835910 dirtied=18417765 written=18417382
 Planning:
   Buffers: shared hit=146 read=12
 Planning Time: 85.741 ms
 Execution Time: 749495.584 ms
(14 rows)

Time: 749583,549 ms (12:29,584)

I tried 100Gb layers.Load time is almost the same (7h30m vs. 7:48), sequential scan execution time is almost the same (20 minutes). So at least for this scenario: bulk load + seqscan, segment size has almost no impact on performance.

this is complete, isn't it?
@knizhnik could summarise the conclusion and make it easily discoverable? (e.g. add the summary or a link to it into the issue body or commid as an markdown text into the repo)
@kelvich , WDYT is appropriate? how can we preserve these test results?

how can we preserve these test results?

IMO we should just automate them -- let's create an issue to automate them in our perf-tests (and implement that feature when safekeepers will be able to delete wal)

Summary:
Zenith is able to load large data sets (~1Tb) and run queries on such database. Database size and population time are comparable with Vanilla. Query execution time is about 3 times slower than Vanilla.
The following problems were detected:

Long startup time because of loading layer map (50 minutes for compressed database, 4 minutes for uncompressed)
Memory consumption caused by storing layer map in memory (~16Gb  for compressed database, ~10Gb for uncom,pressed database)
Long shutdown (~1 hour) because of reconstructions pages by checkpointer
Ephemeral files are not removed on restart

and 3) were fixed (prohibiting image layer creation on shutdown).
For 1) there is uncommitted PR   #1035 (IMHO should be committed before lunch)

I have implemented compressed storage (PR #1002). It significantly (~3 times) reduce storage size and queries execution time. But each page is stored as separate chapter in bookfile. Not  sure if it the most optimal format which cause problems with loading layer map and high memory usage. Partly it is solved by #1035. But I am not sure that should dd compression support right now.
Other results:

Using large segment sizes (100Gb) doesn't signficantly affect performance and storage size (in this scenario).
To full utilize read capacity of pageserver, there should be multiple parallel workers (or parallel queries). So it makes sense to increase default setting for max_parallel_workers and max_parallel_workers_per_gather for Zenith.
wal_log-hints should be switched off, otherwise a lot of extra WAL wirh HINT FPI is generated (5.3Tb for 1.3Tb data set). As far as Postgres log only first page hint update aftre checkpoint, it cause multiple modifications of the same page if it is evicted from shared_buffers.


How to make a custom field: id?

As i understand - now my "id" generated from mongoose "_id" field, but how to make it custom?

You need to create another field. id is used by Relay and it encodes the resource type along the mongoose _id too.

@tothandras ok, as then i can create unique field or rewrite _id during create new document?

@tothandras i try used "unique: true" in mongo schema and it did not work...(

@Dugnist for what do you need a custom id?

does this work or not?
new Schema({
    identifier: { type: String, index: { unique: true } }
})

@tothandras i need create a full object that include field "id" on the client-side and then push them on server...

@Dugnist hmmm.. why not just use the ids you get back when inserting objects with a mutation? so you can have the ids on the client side too for new objects

@tothandras yes - index: { unique: true } is working. Thank you. And you really don't know maybe how can circumvent restriction with "id" field? The policy of my project require creating unique id on the client...(

@tothandras unfortunately i cannot do any request with my custom field "ID" without default field "id"...(
This mutation not working... I know why and it sadly...(
mutation{
updateDocument(input:{
name: "Name of Document",
ID : "123",
clientMutationId: "1"
}) {
changedDocument {
ID
}
}
}

@Dugnist To be honest I've never heard unique id and client side in the same sentence. :) As the user has full control over the sent object, you eventually need to validate it server side.
What you can try is to use a plural query with the ID and then you can get the actual id to do the mutation.. it's something


should caveats have randomly generated head names?

could it happen that a block provides facts that match what a caveat's query must produce? Not sure it is possible with the way queries are implemented, but maybe this should be verified

tested in 8a31984. Maybe it should be a test case

test case added in 04a789e


doc: sunset, make maint status clear

This PR will record the maintenance status, strip out funding/sponsoring things, so that we can archive the repository for now. I hope I didn't miss anything.

Sorry to see this change, but I very much appreciate taking the time to clarify the status of this project.
For others who need IPFS in rust, we're working on iroh, and would be happy to chat.
To the rust-IPFS maintainers, I'd love to take some time to chat about did & did not work on this project, ideally so we can all grow & learn.
Thanks so much for your efforts!

@b5 we do have one interested in picking up maintainership, so I guess we are a bit in a holding pattern.
I can be reached from the commiter email address for any discussions with the disclaimer that I haven't yet peeked at iroh.

Thats unfortunate. Been doing passive maintenance on my own branch of rust-ipfs though do feel there could be more work done.


Hyperlinks

Needs minimum 4 hyperlinks (I'm not sure if they can just be a part of the navigation bar)

I think the requirement is additional 4, not counting the ones in the navigation bar


Feat: As a user I should be able to click on the answers and get feedback

Our quiz app is going to have 4 answers each. In out data we already define the correct answer.
For this task make the answers clickable.
When when the user clicks on the answer, show them feedback by changing the style of this answer.
If the selected answer is correct make the styles green, otherwise red.

Done as part of #6


Style sheet is not loaded on client page transition on production

Hi, I found another mysterious behavior which occurs only on production mode.
Reprudction
With vue-router example,
npm i cross-env
modify start scripts on package.json as follows vite build && vite build --ssr && cross-env NODE_ENV=production node ./server
npm run start
Expected
Home is green and About is red
Observed
Home and About is black. css for these components are not loaded.

Note that this does not occur in dev mode.
I suspect vite's bug, but vite's playground demo works correctly on production environment...
The problem occurs vps's client-routing too.

I couldn't reproduce https://github.com/brillout/vite-plugin-ssr_vue-router.

I can reproduce it with your repo.
npm install
npm run prod

color of h1Home and h1 About are both black which is green and red with npm run start.

I confirmed this only occours when you build app on windows. On linux, this does not happen. Maybe this is caused by esbuild.

Just had a look as well. It does seem like a Vite bug.

Is this part of code generation is made by vite?

Yes.
I expect this to be an edge case since no Windows user has reported this yet.
If it's a blocker for you, I'd recommend to create a Vite PR; they usually quickly merge bug fix PRs.


prefer const constructor

By playing new linter rule of dart-lang/linter#342 on flutter package

Thanks!
cc @Hixie

Awesome

Thanks @a14n ! Please add your name to AUTHORS file :)


[KSF-98] Move application selection widget from bottom to top

Wrong destination branch.
Re-made PR as https://github.com/kaaproject/sandbox-frame/pull/64


paper-responsive-change event name cut off in documentation

Moved from https://github.com/PolymerElements/paper-drawer-panel/issues/64
In the documentation page for paper-drawer-panel, the name of the paper-responsive-change event title is cut off, and appears to be paper-responsive-.
I believe this is because the description of the event does not flow onto multiple lines.  Note the top of the "h" is peaking out from the next line.

Looks fixed:


add new dim group types and add default week_of_year and month_of_year to mappings

This PR adds:

day_of_year
quarter_of_year
week_of_month
aliases week_index to week_of_year

Additionally, it tests all of these and adds week_of_year and month_of_year to the default timeframes both in seeding and in mapping contexts

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (22d7c8b) 81.68% compared to head (8c944f0) 81.69%.
Report is 1 commits behind head on master.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master     #144   +/-   ##
=======================================
  Coverage   81.68%   81.69%
=======================================
  Files          46       46
  Lines        5384     5386    +2
  Branches     1329     1329
=======================================
+ Hits         4398     4400    +2
  Misses        753      753
  Partials      233      233

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Adds map to API 422 returns for invalid latitude / longitude requests and shows the described bounding box on the map.

This PR adds a simple Leaflet map to the API's 422 HTTP code returns when an invalid latitude / longitude has been requested. The map has the bounding box that is described in the returned page to more quickly highlight where our data boundaries are for our point based queries.
An example of this error can be found at: http://localhost:5000/temperature/point/80.0628/-146.162
Note: The color of the bounding box may need to be adjusted to be more visible, but let me know if you think that is the case.
Closes #420

@Joshdpaul @kyleredilla Thanks for your reviews! I changed the color to a dark orange that better highlights the boundaries and I think looks good. Have one more look at that.
As for the internal server error issues, that appears to be a separate issue that I'm going to make a Github issue for now. So we shouldn't hold this PR back because of that issue, as it was not introduced with any of this code.

Yeah the orange looks a lot better! 👍🏻


Register a normalizer for literal

Closes #5897.

[x] Tests added / passed
[x] Passes black dask / flake8 dask

I wasn't sure if the return value should include the class of the literal object, so that tokenize(literal(x)) and tokenize(x) would be different. I'm assuming that if both are used then they're intended to mean the same thing, so I haven't.

Thank you for finding and resolving this issue @bmerry !
My guess is that it's slightly safer to treat them differently.  I can't actually think of a situation where this would matter, but it seems safer to include some slight difference, maybe even just return "literal", normalize_token(lit())

Ok, done as you suggested. I had to change the test so it now just tests that the tokenization is consistent, so it's no longer a regression test for #5897. Let me know if you think it's worth adding a test on normalize_token.

Thanks @bmerry !


Specify dayfirst=False in parse() call

Fixes https://github.com/jlafon/PynamoDB/issues/109

Will close this PR and cherry-pick the change into a release branch per @jpinner-lyft

Cherry-picked as https://github.com/lyft/PynamoDB/commit/10d5802a36f3d3209ca0f50e563ff760687f236f for https://github.com/lyft/PynamoDB/releases/tag/<IP_ADDRESS>


Fix make install command

Shall we list the make install issues here?

@nirandaperera what are the issues? you mean installation path, etc?

Now the command installs the include files and libraries to a given location. We need to add versioning to the artifacts


Add missing import on RadioGroup/Radio example

#31
Noticed this when implementing some radios in a form, thought it might mitigate some questions. Thanks for the great library 👍  Saves me a ton of time.

Nice catch!


Maintenance of ventilation value

When you switch from Hot/Cold to Off,  it maintains the ventilation value (high, low, medium). When you switch from off to hot/cold, ventilation returns to Auto.
I believe (personal opinion), it would be better to maintain the previous ventilation value.

Hi, I confirm that the current operating mode is as you described.
The fan speed is reset when you change the HVAC mode (auto/cold/hot/fan/dry mode).
This way of working is used to try to make the integration compatible with all (or most) existing air conditioners.
This is because I have noticed, from my experience and from other users' reports, that some air conditioner models do not allow switching to a specific mode unless the fan is first set to "auto".
I am pretty sure that this only applies to the dry mode, but I cannot be 100% sure.
Also, this is not always true, because some air conditioners need to set the fan to minimum when switching to dry mode (and this is why the integration implements some compatibility flags to be activated to solve these problems).
This explains the reason for the choice to always reset the fan speed to "auto" (except in cases where the compatibility option is enabled).
I recently implemented the ability to configure the default fan speed that is used for all modes except dry mode.
And you can do this by activating the "Default fan speed when setting HVAC mode" flag which activates an entity, where you can choose the default fan speed, which will be set when you change the HVAC mode.
I also want to point out that if the air conditioner is turned on without changing the HVAC mode, the fan is not reset.
Actually, the Home Assistant interface does not allow you to turn on the air conditioner without necessarily having to set a HVAC mode; but if you try to call the "climate.turn_on" action to turn on your air conditioner, you will notice that the air conditioner turns on using the previous mode and fan speed.
Example:
action: climate.turn_on
target:
  entity_id: climate.name_of_your_air_conditioner

After explaining the reasons for the choice and the possible workarounds that can be used, I might also find a way to keep the previous fan speed only in the cold and heat modes (which I think are the most used) and make this option configurable.
It won't be a high priority task but I'll try to look into it anyway.
Thanks for the suggestion.

You need to go to the Home Assistant settings, select the Tuya Smart IR AC integration and from the list of air conditioners you have configured click on the configure button.
The air conditioner configuration page will open (the same one you configured when you first inserted the air conditioner), going down you will find the "Default fan speed when setting HVAC mode" flag; activate it and confirm the changes.
Now among the entities exposed by your air conditioner you should also see a combobox from which you can select the default fan speed.

Forgive me, but I don't see it

Very strange... can you try clearing your browser cache and/or opening the home assistant page in an incognito tab to see if that changes anything?

Perdonami se rompo le balle.
Scrivo solo per contribuire nel migliorare il tuo fantastico lavoro.
Non pretendo soluzioni rapide nemmeno soluzioni in generale.
Già così mi hai fatto un ottimo servizio.
Il giorno ven 11 ott 2024 alle ore 09:21 EnzoD86 @.***>
ha scritto:

Very strange... can you try clearing your browser cache and/or opening the
home assistant page in an incognito tab to see if that changes anything?
—
Reply to this email directly, view it on GitHub
https://github.com/EnzoD86/tuya-smart-ir-ac/issues/27#issuecomment-2406710762,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR3HCNGEXINCIWY5OQHYOS3Z254BNAVCNFSM6AAAAABPXBQOGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBWG4YTANZWGI
.
You are receiving this because you authored the thread.Message ID:
@.***>

Non ti preoccupare, infatti sto cercando di capire come mai hai questo comportamento anomalo.
Se puoi/vuoi raggiungerci nel gruppo telegram dove posso darti una migliore assistenza e switchiamo in italiano:
https://t.me/+iHgzgJfY41BjMzU0

Fatto. Grazie
Il giorno ven 11 ott 2024 alle ore 09:42 EnzoD86 @.***>
ha scritto:

Non ti preoccupare, infatti sto cercando di capire come mai hai questo
comportamento anomalo.
Se puoi/vuoi raggiungerci nel gruppo telegram dove posso darti una
migliore assistenza e switchiamo in italiano:
https://t.me/+iHgzgJfY41BjMzU0
—
Reply to this email directly, view it on GitHub
https://github.com/EnzoD86/tuya-smart-ir-ac/issues/27#issuecomment-2406742401,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AR3HCNENYP4FHRZYPDIE22LZ256O7AVCNFSM6AAAAABPXBQOGSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBWG42DENBQGE
.
You are receiving this because you authored the thread.Message ID:
@.***>

Solved. My update error.

Hi, after analyzing the code that manages the fan speed setting, I confirm that the integration already works like this.
The fan speed is maintained when the "Default fan speed when setting HVAC mode" option is disabled.


Clarify Services Resiliency : "All Azure management services"

Could you please clarify the assertion "All Azure management services are architected to be resilient from region-level failures" within the "Service Resiliency" section ?
Which Azure services are you talking about ? Is it the control plane or is it some Azure Services ?
Thanks

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cca70845-100f-2cfd-aba3-34b7fa4a5264
Version Independent ID: 475b26e5-9e23-f769-3bd5-cbc4a678dbf0
Content: What are Azure Availability Zones?
Content Source: articles/availability-zones/az-overview.md
Service: azure
GitHub Login: @cynthn
Microsoft Alias: cynthn

Or maybe the Azure "Management and Governance" services as listed here: https://docs.microsoft.com/en-us/azure/?product=management-and-governance ?

@olileger Thank you for your query, i am assigning this issue to our document author @cynthn to further help on this.

I'm just going through some older items and the section referenced in this issue is no longer in the doc. I'm not sure if this was fixed, then removed or if something else happened. If there is still some confusion, please reopen this issue or create a new one.
#please-close


[question] is it possible to port this to eslint?

Hi @mgechev,
since it is possible to write the very same* syntax in js with babel-plugin-angular2-annotations i wonder if it is possible to modify codelyzer to also support eslint.
I have to admit i haven't looked into how to create custom eslint, tslint rules. If they have similar APIs or if it is possible to re-create each rule with eslint at all.

method parameter decorators should be soon supported as well

Hey @tsm91, in theory it is possible - we can turn the JS and TS ASTs to a common format and write our walkers/visitors for their shape. On the other hand, this way we'll loose metadata we get from the type annotations which exists in the TS AST.
Although it is possible, I'd prefer to keep this feature once we have full feature set for TypeScript development.

Okay, thanks for the info

Although it is possible, I'd prefer to keep this feature for once we have full feature set for TypeScript development.

fair enough


[PE-D][Tester A] User guide search command example description

Under the user guide, there was an example provided "search e Summer Fashion returns events with tags named Summer or Fashion". However, if there is an event tagged with both "Summer" and "Fashion", it will be reflected in the command output as well.
The user guide will need to include something like "...returns events with tags named Summer or Fashion or Both", to match the feature's actual output.

Labels: severity.VeryLow type.DocumentationBug
original: M-kew/ped#1

4 lines above this in the UG there is a line that says

Events with tags matching at least one keyword will be returned (i.e. OR search).

Which should clarify any doubts the user has about how the search is done.
On top of that, the English word "or" is not exclusive by default.
This will be closed as it is a non-issue.


Issue # 2  ring buffer data structure

a simple ring buffer for issue #2.

When creating the pull request use a more descriptive name--what was issue #2?  In this case, you should use a title like "Ring buffer data structure".  Then you should have a description for the request--that is where you should reference the Issue you are solving.  So in this case you could use something like "Adds a ring buffer data structure to solve issue #2".
I will add a few other comments/suggestions throughout the code.  You should be able to make the changes and then update the pull request to use your new code.

@huyang1988 @pradeep-k @wenhuizhang
Other than the small changes listed above, there are two larger changes that should be made:

You should add a separate file ring_test.c which has a main function and does a few simple unit tests on your data structure.  This will help verify that the code works correctly.
The ring.h file should only define the function prototypes and your actual code should be in a file named ring.c.

Also, since you will now have several C files that all work together, you should create a Makefile in that directory which can correctly build the library and your tests.
I haven't read through the logic of your code carefully,  but it looks like you are on the right track and the functions you added to the spec are good!

Thank you so much for the detailed explanation, it is really helpful!!!
We will set then name as "Ring buffer data structure" for the next version
:)
Wenhui
On Fri, Jan 16, 2015 at 8:30 AM, Tim Wood<EMAIL_ADDRESS>wrote:

When creating the pull request use a more descriptive name--what was issue
#2 https://github.com/gwAdvNet2015/adv-net-samples/issues/2? In this
case, you should use a title like "Ring buffer data structure". Then you
should have a description for the request--that is where you should
reference the Issue you are solving. So in this case you could use
something like "Adds a ring buffer data structure to solve issue #2
https://github.com/gwAdvNet2015/adv-net-samples/issues/2".
I will add a few other comments/suggestions throughout the code. You
should be able to make the changes and then update the pull request to use
your new code.
--
Reply to this email directly or view it on GitHub
https://github.com/gwAdvNet2015/adv-net-samples/pull/26#issuecomment-70253338
.

--
V/R,
Wenhui Zhang
Email<EMAIL_ADDRESS>Telephone: 1-(703) 424 3193

You've fixed most the problems, but now you have combined your fix for this issue with something else--the code you add for this pull request should only be related to the ring buffer.
You need to figure out how to:

revert your commit https://github.com/wenhuizhang/adv-net-samples/commit/84085f1e2db78fa3a5f80768096a169f8e91698f which was to the style guide
temporarily revert your commit to the socket file about handling IP addresses https://github.com/wenhuizhang/adv-net-samples/commit/5c0e05809e5102b947ab6b9d59401f10d289a404 since it isn't related to this.  What you probably want to do is have a separate branch for each of the Issues you work on, that way the commits won't get mixed up and it will be easier to push them.  See if you can figure this out!

I also have a few changes I will make inline with the code.

If you can't figure out how to undo the bad commits it is OK.  Focus on making the fixes to ring* and when I merge your code with the main repository I can have it ignore the changes you made to the other files.
Going forward I suggest you do all your work in branches (one per issue), and try to leave your "master" branch as a clean copy of this repo!


cavalier, endogamic

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[c,e]

/trunk merge


Incorrect NOXR Formula Code in Hourly NOx Rate CEMS

The NOXR Formula Code column displays the NOx Mass Formula Code in both the Hourly NOx Rate CEMS View and the download.
Background Information:

The Emission (EM) Views are defined using the camdaux DATASET, DATATABLE and DATACOLUMN tables.
The DATASET table GROUP_CD column that indicates an EM View dataset when the value equals "EMVIEW".  Each "EMVIEW" represents a single EM view.
The DATATABLE table is the DATASET table's child and only contains one row for each "EMVIEW" dataset.
The DATACOLUMN table is the DATATABLE table's child and contains a row for each column in each data table.
The DATACOLUMN table contains an ALIAS column that indicates the column from which to pull data in the source data.

Emissions data-- Hourly NOx Rate CEMS View and the download need to be corrected to show the correct code. It should be showing the NOXR Formula Code.

ECMPS Dev Application
https://ecmps-dev.app.cloud.gov/emissions

Per Dwayne, the steps to fix this and verify it are:

Check the current JSON produced by the Emission View endpoint.
a.  https://api.epa.gov/easey/dev/emissions-mgmt/swagger/#/
b.  Click the Authorize button and enter a valid API Key (PXPWlQGB3wKXotkWN1PbSwbSoM7CoWW0ZMPWYtfc ) and click the Authorize button.
c.  Navigate to the Emissions Views section and open the “GET: /emissions-mgmt/emissions/views/{viewCode}” endpoint.
d.  Click the Try It Out button.
e.  Entering the following for my example.
i.  viewCode: “NOXRATECEMS”
ii. monitorPlanId: “MDC-9FF127451AE0415ABF9BBF6A0B036A1A”
1.  For ORIS 10 (Green County), Locations CS0EBN, 1, 2
iii.    stackPipeIds: “CS0EBN”
iv. reportingPeriod: “2022 Q1”
v.  attachFile: true
1.  Will produce the JSON for the emission view.
f.  Click the Execute button.
g.  Click the Donwload File button to get the response JSON.
i.  You many need a program to reformat the JSON to make it readable.
Check the current Emissions page.
a.  https://ecmps-dev.app.cloud.gov/emissions
b.  Do not log in.
c.  Expand “Greene County” (ORIS 10) and click the Open link for Configuration “1, 2, CS0EBN”.
d.  Click on the “Greene County 1, 2, CS0EBN” tab that appears, which should open selection information.
e.  Update the following:
i.  Reporting Period(s): “2022 Q1”
ii. Locations: “CS0EBN”
iii.    View Template: “Hourly NOx Rate CEMS View”
f.  Click “Apply Filter(s)”, which should display a grid.
g.  Check the “NOXR Formula Code” and “NOX Formula Code” columns.
i.  They should initially and erroneously have the same value (F-24A) for each row.
The Change: On the Dev database, update ALIAS to “noxrFormulaCode” for the camdaux.DATACOLUMN row where the parent DATASET.DATASET_CD equals “NOXRATECEMS” and the DATACOLUMN.NAME equals “nox_rate_formula_cd”.
Check the JSON again using the steps under step 1.
a.  The JSON should now contain a new “noxrEquationCode” elements with the value “F-6”.
Check the Emissions page using the steps under step 2.
a.  The “NOXR Formula Code” column should now have the value “F-6”.

I made the database change on dev; here are screenshots of the JSON and emissions pages showing the changes.

Confirmed the Hourly NOx Rate CEMS View contains the correct formula code for the NOXR Formula Code column. Check global view and download and workspace view and download.

Verified in test for the following two configurations (matching NOXR formula codes in ECMPS 2.0 test and FACT):

ORIS 298, Location LM1 for the reporting period 2023 Q1

ORIS 470, Location 1 for the reporting period 2022 Q1


crash

crash-2018-04-21_13.07.59-server.txt
fml-junk-earlystartup.log
latest.log
forge-1.12.2-<IP_ADDRESS>11-universal.jar
spongeforge-1.12.2-2611-7.1.0-BETA-3002.jar
projectcore-7.1.0-0.2.2.jar https://trentech.org/jenkins/job/ProjectCore/36/
projectportals-7.1.0-0.14.2.jar https://trentech.org/jenkins/job/ProjectPortals/47/
Edit: https://i.imgur.com/ioDm3Ft.png

https://gist.github.com/ziceptor/9cb6928d2086377fefb881fe3e118fe6
https://gist.github.com/ziceptor/4a6e84c39887532d9aa3490919f615f3
projectportals-7.1.0-0.15.0.jar https://trentech.org/jenkins/job/ProjectPortals/48/


Add initial maverick configuration ui

[ ] interface for maverick commands, start, stop, etc...
[ ] interface to view service status
[ ] interface to view and edit config files

Basic interface to view and edit configuration files is complete. Working on a UI for commands and system processes now.

https://github.com/xtermjs/xterm.js
Use terminal for maverick command feedback

ooh xterm.js is a nice idea!

added the -web component of xterm.js https://github.com/goodrobots/maverick-web/commit/dc27bb11fe104abbde747e6f53bb9de513c84d5c
just hacked it in, but we can link this to -api for two outcomes:

Show realtime terminal output from the process runner (e.g. maverick configure)
https://github.com/goodrobots/maverick-api/blob/7ef29f4283e618320710c825c3b21513ab62bb96/maverick_api/modules/base/util/process_runner.py

Allow the user to interact with the terminal from the webui using the process runner

https://github.com/wavesoft/local-echo/blob/master/lib/LocalEchoController.js

awesome!  would be great to have a mav terminal built in

The terminal support was working (ish) prior to the API refactor in -web. It would be nice to get that back as it saves a user from needing to SSH in for basic stuff.

'Commands' re-instated.  If it's going to be a full terminal (which is great), should probably be renamed to Console/Terminal.
Console works-ish, restricted to a particular directory and the output is a bit hoky.  If we can get it working better will be fantastic to have.

Yeah, the plan is to have the full terminal working. -api is okay at this stage, the remaining work is in the -web component.


⚠ Bofrost.de has degraded performance

In f29f58c, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1254 ms

Resolved: Bofrost.de performance has improved in fd4d1ef.


Minor improvements.

I switched to "local" for the time-based program, but it would be trivial to change it to "UTC" if you prefer that.


Outline Is Very Slow

Create a file with 25k times the following line:

function bla() {}

Unfold "Outline"
Observe that it freezes the UI

This is already a problem in TextBuffer.ts - the outline makes typing/deleting noticeable slow.

Observations:

This runs pretty quickly (300ms) for me in Code OSS.
It takes 4s when having 32k lines in Code Insiders.
It runs pretty quickly (300ms) in a fresh new Code Insiders.

Big culprit:

Call tree:

Profile-20210929T170504.zip
Given these findings, I have to forward this to you @jrieken. Let me know if you find that the performance bottleneck is in fact in the widget.

@sandy081 for each element I check the config-service to know if its filtered or not, these filters are defined per language, e.g for Java interfaces might be filtered and for TypeScript its enum-members etc. I see that each call takes ~3ms and that sums up with many, many element.
This is the call in question: https://github.com/microsoft/vscode/blob/f523f65f3985eb6493cae5d219de35f1d7ba6f7a/src/vs/workbench/contrib/codeEditor/browser/outline/documentSymbolsOutline.ts#L102
Unsure if I can workaround but I believe one service-property of the configuration service should be speed 🙊

Unsure if I can workaround but I believe one service-property of the configuration service should be speed 🙊

Cannot you just cache it and use configuration on change as invalidation?

Thanks @sandy081

Great job, thanks @sandy081!


subfinder brute force is taking a LONG time

subfinder brute force is taking a LONG time
maybe skip bruteforce

changed in https://github.com/x1mdev/ReconPi/commit/6ed61598baeffa93789f44a88fc23bb3e7e90521


[DOC] : creation of PULL-REQUEST-TEMPLATE.

Description:
There is no pull request template in this project . its essential to have it.
Goals:

Define template structure and content.
Integrate template into repository workflow.

Proposed Solution:
1.Create .github/PULL_REQUEST_TEMPLATE.md file.
2.Include sections for:

Description of changes
Related issues/PRs
screenshots.
Additional context/notes.

steps for implementation :
1.Design template structure.
2.Draft content.
To Reproduce
Steps to reproduce the behavior:

Go to project repo.
Click on pull requests.
click new pull request button on the right top corner of the page.
see description of pull template, it will be empty as there is no pull request template.

Implementing a pull request template improves the process by providing a standardized format for contributors to describe changes, reference related issues, and provide testing instructions, enhancing clarity.

Sure, you can work on this. Assigning.

Feel free to reopen if you still want to work on this.


Add support for Spec.Storage.Pvc.Path component

This allows users to specify the path within a pvc which should be exported by samba. Includes an integration test which tests this new feature.

Setting to draft for now since the path update test fails.

/retest centos-ci/sink-clustered/mini-k8s-1.23

As discussed over voice the planner refactor PR has been merged. You'll need to manually rebase and resolve the conflicts that arise from the changed planner code.

/retest centos-ci/sink-clustered/mini-k8s-1.23

/test centos-ci/sink-clustered/mini-k8s-1.23

/retest centos-ci/sink-clustered/mini-k8s-1.23


Autogenerated id column should be positive-int

I want to mark a Doctrine entity $id field that is autogenerated by database engine as @var positive-int. However PHPStan complains.
class Entity
{
    /**
     * @var positive-int|null
     */
    #[ORM\Column(type: 'integer', options: ['unsigned' => true])]
    #[ORM\Id]
    #[ORM\GeneratedValue(strategy: 'IDENTITY')]
    private ?int $id;
}

PHPStan output:
Property App\Rma\Domain\Rma\Rma::$id type mapping mismatch: database can contain int but property expects int<1, max>|null.

['unsigned' => true] does not work for all drivers (e.g. in postgres)

If support would be added, it needs to be driver-aware.

@javaDeveloperKid database will still allow you to write your own value in the field instead of using the default generated value, which is then only constrained by the type of the field (so you can write -2 in an id field if the DB field is a signed integer type)


Add a tool to request info when a bug is moved to Core::Performance

Resolves #1887
Checklist

[ ] Type annotations added to new functions
[ ] Docs added to functions touched in main classes
[x] Dry-run produced the expected results
[ ] The to-be-announced tag added if this is worth announcing

Dry-run
The following bugs were moved into the Core::Performance component; a comment to request more information was added:

BugSummary

1819303

ark.intel.com makes Firefox consume all available memory

1260981

[Meta] Identify the cause of performance problems in using google docs

Comment example - bug 1819303
This bug was moved into the Performance component.
:kamil.paral, could you make sure the following information is on this bug?

For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/ , upload it and share the link here.
For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

Thank you.

What if the information is already present in the bug?

What if the information is already present in the bug?

In that scenario the needinfo will be cleared with/without a confirmation that the information is already in the bug.
We could add a sentence at the end to request a response: "If the requested information is already in the bug, please confirm that."
@julienw wdyt?

Yeah, this sounds good to me. We can adjust later if we see this case happening too much.

Could we add some basic checks to prevent some obvious cases? E.g. if an attachment with "memory" in its name is in the bug, we don't ask for a memory report; if there is a "share.firefox.dev" in any of the comments, we don't ask for a performance profile.

This would work for me too :-)

Could we add some basic checks to prevent some obvious cases? E.g. if an attachment with "memory" in its name is in the bug, we don't ask for a memory report; if there is a "share.firefox.dev" in any of the comments, we don't ask for a performance profile.

Nice idea! However, doing simple checks could have high false negatives. I suggest monitoring the cases after deploying this and adjusting later if this introduces much noise.
I filed an issue to follow up on this idea (i.e., #1926).
@marco-c wdyt?

I would add some simple checks now, the ones that are very unlikely to have false negatives and are easy to implement.

"share.firefox.dev" in the comments is definitely a link to a profile, so it is very unlikely to cause false negatives.
if you go to about:memory and click on "Measure and save...", Firefox will save a memory-report.json.gz file on your disk, so  "memory-report.json.gz" in the attachments is also very unlikely to cause false negatives.

Dry-run
2023-03-10 23:13:31,200 - INFO - Run tool moved_to_performance.py
2023-03-10 23:13:32,903 - INFO - Tool moved_to_performance.py has finished.

The following bugs were moved into the Core::Performance component; a comment to request more information was added:

BugSummary

1821555

Both FB Marketplace & Twitter lock up for 30-45 seconds every time I try to scroll.

1821424

Memory leakage 110.0.01 Win11 22H2

1820078

I try to access https://mysafaricost.com on firefox browser and it redirects always on some pages and stops and displays a broken site at the end while it opens on another browser.

1795009

Open multiple tabs becomes slow

Comment example - bug 1795009
This bug was moved into the Performance component.
:zero.arst, could you make sure the following information is on this bug?

For slowness or high CPU usage, capture a profile with http://profiler.firefox.com/, upload it and share the link here.
For memory usage issues, capture a memory dump from about:memory and attach it to this bug.
Troubleshooting information: Go to about:support, click "Copy raw data to clipboard", paste it into a file, save it, and attach the file here.

If the requested information is already in the bug, please confirm that.
Thank you.

@julienw in the context of doing some checks to avoid asking for information that is already present in the bug, do you think we should consider only content that was submitted be the reporter?

I never used a checklist on Bugzilla, does it actually work? Can a user mark something in a checklist as done? Do they need specific permissions to do that if the comment is from somebody else (the bot in this case)?

@julienw in the context of doing some checks to avoid asking for information that is already present in the bug, do you think we should consider only content that was submitted be the reporter?

Yes, this sounds reasonable.
I wonder if we should also have a time limit. For example if the information is older than say 6 months, we could assume this is outdated and still ask about it.

I wonder if we should also have a time limit. For example if the information is older than say 6 months, we could assume this is outdated and still ask about it.

Done in 2a7cae1fedb6b20c046711f564b5df0a82f47bd6

I never used a checklist on Bugzilla, does it actually work? Can a user mark something in a checklist as done? Do they need specific permissions to do that if the comment is from somebody else (the bot in this case)?

Unfonratintly, Bugzilla does not support markdown checklists.

Maybe when the data is present but not recent enough, we should adjust the comment to ask to refresh the info (otherwise people will ignore it and say that it is already present)

Maybe when the data is present but not recent enough, we should adjust the comment to ask to refresh the info (otherwise people will ignore it and say that it is already present)

Do you think 13820a756713e2715b82a0e5ecac7f4fb7a80d81 would be enough? If not, I can implement what you suggested.


Error: Cannot find module 'ansi'

Extension Version: 0.1.11
VS Code Version: 1.5.3
OS Version: Ubuntu 16.04
Node.js Version (result of command node -v): 4.2.6
NPM Version (result of command npm -v): 3.5.2

Steps to Reproduce:

Select pick colour in Sass file
No previews and crash when picker chosen.

Log:
(Click menu of VS Code "Help > Toggle Developer Tools", and click "Console" tab)
[Extension Host]
threadService.ts:217[Extension Host] Starting new check on: /home/will/local/history-project/_content/history.md [markdown]
threadService.ts:217[Extension Host] Start child process...
threadService.ts:217[Extension Host] Try to get npm via usage info.
threadService.ts:217[Extension Host] npm directory path: /usr/share/npm
threadService.ts:217[Extension Host] npm path: /usr/share/npm/lib/npm.js
threadService.ts:217[Extension Host] Base directory path: /home/will/.vscode/extensions/anseki.vscode-color-0.1.11
threadService.ts:217[Extension Host] Retry (Error: Cannot find module 'electron-prebuilt')
threadService.ts:217[Extension Host] Try to load: /usr/share/npm/lib/utils/spawn.js
threadService.ts:217[Extension Host] Error: Cannot find module 'ansi'
    at Function.Module._resolveFilename (module.js:339:15)
    at Function.Module._load (module.js:290:25)
    at Function.I.n._load (/usr/share/code/resources/app/out/vs/workbench/node/extensionHostProcess.js:12:25921)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at Object.<anonymous> (/usr/share/npm/lib/ls.js:15:13)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)(anonymous function) @ threadService.ts:217
messageService.ts:119[processBridge]: Error: Cannot find module 'ansi'

Hi @wjdp, thank you for the report.
It seems that loading ansi module the NPM requires failed.
Does your NPM work fine?
Please check that the module was installed into /usr/share/npm/node_modules/ansi.

It's not there. NPM seems to be working fine. I've tried to manually install which has not helped:
[~]$ sudo npm install -g ansi
/usr/local/lib
`--<EMAIL_ADDRESS>

The npmlog module that is a part of NPM depends on the ansi.
Then, the ansi should exist in /usr/share/npm/node_modules/npmlog/node_modules/ansi.
If that is not there, try:
cd /usr/share/npm/node_modules/npmlog
sudo npm install

BTW, does the issue happen in only SASS files?

No, any time I try to call any commands this project has added to the command palette.
[/usr/share/npm/node_modules/npmlog]$ sudo npm install
[sudo] password for will:
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
<EMAIL_ADDRESS>/usr/share/npm/node_modules/npmlog
+--<EMAIL_ADDRESS>| `--<EMAIL_ADDRESS>|   +--<EMAIL_ADDRESS>|   +--<EMAIL_ADDRESS>|   `--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>| `--<EMAIL_ADDRESS>`--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | | `--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  |   `--<EMAIL_ADDRESS>  |     +--<EMAIL_ADDRESS>  |     +--<EMAIL_ADDRESS>  |     +--<EMAIL_ADDRESS>  |     +--<EMAIL_ADDRESS>  |     +--<EMAIL_ADDRESS>  |     +--<EMAIL_ADDRESS>  |     | `--<EMAIL_ADDRESS>  |     `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   |   `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | | +--<EMAIL_ADDRESS>  |   | | +--<EMAIL_ADDRESS>  |   | | `--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | | `--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | | +--<EMAIL_ADDRESS>  |   | | +--<EMAIL_ADDRESS>  |   | | | `--<EMAIL_ADDRESS>  |   | | +--<EMAIL_ADDRESS>  |   | | `--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   |   `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | | +--<EMAIL_ADDRESS>  |   | | +--<EMAIL_ADDRESS>  |   | | `--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   | `--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   | `--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   | `--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   +--<EMAIL_ADDRESS>  |   |   `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | |   `--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | |   +--<EMAIL_ADDRESS>  | |   `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>(git+https://github.com/bcoe/foreground-child.git#ad7ba1d4a84e8b199cbfb5d30e01e586390978cc)
  | | `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | | +--<EMAIL_ADDRESS>  | | | | +--<EMAIL_ADDRESS>  | | | | +--<EMAIL_ADDRESS>  | | | | +--<EMAIL_ADDRESS>  | | | | +--<EMAIL_ADDRESS>  | | | | `--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | |   `--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | | +--<EMAIL_ADDRESS>  | | | | `--<EMAIL_ADDRESS>  | | | +--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | |   +--<EMAIL_ADDRESS>  | | |   +--<EMAIL_ADDRESS>  | | |   +--<EMAIL_ADDRESS>  | | |   `--<EMAIL_ADDRESS>  | | |     +--<EMAIL_ADDRESS>  | | |     +--<EMAIL_ADDRESS>  | | |     | +--<EMAIL_ADDRESS>  | | |     | | +--<EMAIL_ADDRESS>  | | |     | | | +--<EMAIL_ADDRESS>  | | |     | | | | `--<EMAIL_ADDRESS>  | | |     | | | +--<EMAIL_ADDRESS>  | | |     | | | `--<EMAIL_ADDRESS>  | | |     | | `--<EMAIL_ADDRESS>  | | |     | +--<EMAIL_ADDRESS>  | | |     | `--<EMAIL_ADDRESS>  | | |     `--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | +--<EMAIL_ADDRESS>  | | | `--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>(git+https://github.com/bcoe/spawn-wrap.git#aa0c61536567be2bacb91cba5a786c9044ae65ea)
  | | `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   |   `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   | +--<EMAIL_ADDRESS>  |   | | `--<EMAIL_ADDRESS>  |   | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | | `--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  |   +--<EMAIL_ADDRESS>  |   `--<EMAIL_ADDRESS>  |     `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  `--<EMAIL_ADDRESS>
[/usr/share/npm/node_modules/npmlog]$ ls node_modules/ansi
examples  History.md  lib  package.json  README.md

However after restarting vscode I'm still getting:
[processBridge]: Error: Cannot find module 'ansi'

Ummm...
I'm still looking for clues.
This issue is reported first time. I tried installing Node.js v4.2.6 and NPM v3.5.2 in Ubuntu, then the extension works fine. And also other environments.

I tried renaming the ansi to ansi_, then I got "Error: Cannot find module 'ansi'". Therefore the loading seems to be opposed by something in your computer.
By log messages, the "Error: Cannot find module 'ansi'" happened at /usr/share/npm/lib/ls.js:15:13. But ansicolors module is loaded in that line, not ansi, and the ansicolors doesn't load ansi. The NPM might be broken? Or another version of NPM was installed?
The message "Starting new check on..." in the log is not a message by Color Picker extension, I think that it is a message by another extension. So, any error message was shown by other extensions? I mean, the loading might be opposed by another extension.

Done all this so far on my laptop. Just switched to desktop (Also Ubuntu 16.04) which does not have a vscode install on it. Installed latest and then colour picker plugin before anything else. Ctrl+Shift+P "Pick Colour", exact same result.
threadService.ts:217[Extension Host] Start child process...
threadService.ts:217[Extension Host] Try to get npm via usage info.
threadService.ts:217[Extension Host] npm directory path: /usr/share/npm
threadService.ts:217[Extension Host] npm path: /usr/share/npm/lib/npm.js
threadService.ts:217[Extension Host] Base directory path: /home/will/.vscode/extensions/anseki.vscode-color-0.1.11
threadService.ts:217[Extension Host] Retry (Error: Cannot find module 'electron-prebuilt')
threadService.ts:217[Extension Host] Try to load: /usr/share/npm/lib/utils/spawn.js
threadService.ts:217[Extension Host] Error: Cannot find module 'ansi'
    at Function.Module._resolveFilename (module.js:339:15)
    at Function.Module._load (module.js:290:25)
    at Function.I.n._load (/usr/share/code/resources/app/out/vs/workbench/node/extensionHostProcess.js:12:25921)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at Object.<anonymous> (/usr/share/npm/lib/ls.js:15:13)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)(anonymous function) @ threadService.ts:217
messageService.ts:119[processBridge]: Error: Cannot find module 'ansi'

Could be my NPM installs, will have a look later.

That log also said that the error occurred in /usr/share/npm/lib/ls.js:15.
Could you show me the line #15 of /usr/share/npm/lib/ls.js?
Since Node.js v4.2.6 includes NPM v2.14.12, you might have updated NPM to v3.5.2. The old version might still exist in your computer, and it might have made the trouble.
Or, if you use a npmrc file, it might have made the trouble.

same error in Windows 10 (1607):
Extension Version: 0.1.11
VS Code Version: 1.5.3
OS Version: Windows 10 (1607)
Node.js Version (result of command node -v): 6.7.0
NPM Version (result of command npm -v): 3.10.3
Log:
workbench.main.js:118 [Extension Host] Error: ENOENT: no such file or directory, open 'C:\Users\test\.vscode\extensions\anseki.vscode-color-0.1.11\node_modules\electron-prebuilt\path.txt'
    at Error (native)
    at Object.fs.openSync (fs.js:584:18)
    at Object.module.(anonymous function) [as openSync] (ELECTRON_ASAR.js:158:20)
    at Object.fs.readFileSync (fs.js:431:33)
    at Object.fs.readFileSync (ELECTRON_ASAR.js:426:29)
    at Object.<anonymous> (C:\Users\Leung\.vscode\extensions\anseki.vscode-color-0.1.11\node_modules\electron-prebuilt\index.js:4:42)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)(anonymous function) @ workbench.main.js:118emitTwo @ events.js:100emit @ events.js:185handleMessage @ internal/child_process.js:718channel.onread @ internal/child_process.js:444
workbench.main.js:93 [processBridge]: Error: ENOENT: no such file or directory, open 'C:\Users\test\.vscode\extensions\anseki.vscode-color-0.1.11\node_modules\electron-prebuilt\path.txt'

Hi @epoweripione,
That seems be not same error.
This issue is "Error: Cannot find module 'ansi'", your case is "Error: ENOENT: no such file or directory, open...".
It seems that this is a duplicate of #7. Please comment again to that issue.

No reply came, and I close this issue.

Hello, know post is closed but still. Had the same problem.
Iv deleted nodejs and npm, and installed them not from standart ubuntu reppos but with this ones: https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
After doing:
sudo apt-get remove nodejs npm
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs
Color picker works fine after reinstalling it.

And btw sry for my english.

Hi @SonyaHon, thank you for the report.
Yes, this issue occurs when the NPM doesn't work.
The Color Picker requires the NPM.

@anseki, as i understood it requires npm version 3.10.10, and it will not work with 3.5.2

I made sure that it works with NPM v2.*. Of course it works with v3.5.2 also.
Your NPM v3.5.2 might have been broken. Some NPM commands seem to be working even if the NPM is broken.
As everyone knows, since NPM is updated frequently, multiple versions often conflict.

Had this problem too.
I think it was caused because I had a nodejs (I think its legacy) and the node commands on the terminal, which were using different versions. I fixed it by removing nodejs (sudo apt-get remove nodejs) and using nvm to install the latest LTS version of nodejs (v6.10.3) which is using the 3.10.10 version of npm.
I'm using Ubuntu 16.04 by the way :)

Hi @mizusu, thank you for the report.
Your solution may help people 😄

The same problem with npm 3.5.2
Color picker works fine after: npm install -g npm (as root)

Hi @facsa79, thank you for the report.
It seems that your NPM was broken.

Had this problem too.
I think it was caused because I had a nodejs (I think its legacy) and the node commands on the terminal, which were using different versions. I fixed it by removing nodejs (sudo apt-get remove nodejs) and using nvm to install the latest LTS version of nodejs (v6.10.3) which is using the 3.10.10 version of npm.
I'm using Ubuntu 16.04 by the way :)

Thanks for this, it really helped. All i needed do is remove the nodejs and installing the latest version as of the time am doing this >>>
Using Ubuntu
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs
you can also check out this link for more info https://github.com/nodesource/distributions/blob/master/README.md

Hi @wise1993, thank you for your report. :smile:

I solved this issue by uninstalling react-scripts and installing them again :)

Hi @vnkt66, thank you for your report. :smile:
I don't know what effect the react-scripts has in your environment, but this issue occurs when the NPM doesn't work.

im solving this with installing node with nvm. im using ubuntu 18.04

Hi @aacassandra, thank you for your report. :smile:
Your NPM was fixed maybe.


Async methods should take cancellation token

For example the middleware is async and should take cancellation tokens.

We should probably make sure this is coordinated or someone's going to be in a merge conflict nightmare! 😆
Side note: this might be a good time to introduce a strongly typed delegate for what is currently just Func<ITurnContext, Task> because now that's going to have to become Func<ITurnContext, CancellationToken, Task> allllll over the place.


Plugin ID blocks

What type of PR is this?
/kind design
/kind documentation
Any specific area of the project related to this PR?
/area registry
/area build
/area documentation

What this PR does / why we need it:
This PR aims to introduce precise guidelines for plugin ID usage. In particular, it mandates specific rules for reserved ID blocks. Also, it handles the 0 value exceptional cases, which libs may use in the future for particular purposes.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jasondellaluce, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [jasondellaluce,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


feat(avatar): seeeded random colors

feat(avatar): seeeded random colors

:hammer_and_wrench: Type Of Change

[ ] Fix
[x] Feature
[ ] Refactoring
[ ] Documentation

:book: Description
My quick and fun project for the holidays 🎄. Added seeded randomization to the avatar. Just pass in a seed as a prop and it will always generate the same randomized colors for that seed.
:bulb: Context
Allows using the same randomly generated colors for a specific user.
:pencil: Checklist

[x] I have reviewed my changes
[x] I have added all relevant documentation

:link: Sources
https://www.npmjs.com/package/seedrandom

BTW not sure why the preview for this PR isn't working

BTW not sure why the preview for this PR isn't working

Yeah super weird.. it is now that it rebuilt.


budgiePlugins.budgie-media-player-applet: init at unstable-2023-10-14

Description of changes
Sets PYTHONPATH in the wrapper for Python plugins and adds the budgie-media-player-applet plugin.
I am not sure if the way I'm exposing (the passthru.requiredPythonModules bit) and merging (the lib.forEach plugins ... bit) the Python dependencies is idiomatic enough, the rest of the code is taken from all the other libpeas-based apps but as far as I have seen no Python-based plugin is actually packaged for them yet.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

budgie.budgie-desktop-with-plugins
budgiePlugins.budgie-media-player-applet

Hmmm, I don't really involve in python ecosystem, it is unclear to me what might happen on any python stuff that runs on the desktop when setting PYTHONPATH in a desktop package (I feel like I might try patchPythonScript in the plugin packages if possible, but I really haven't checked how budgie python plugins work here) :thinking:

I was able to reduce it to just pygobject3 in PYTHONPATH, if I do so in the applet itself it fails to import the library.

I now saw how libpeas are done here. I wonder if we should patch libpeas here, previously we do these:

https://github.com/NixOS/nixpkgs/blob/master/pkgs/desktops/gnome/misc/nautilus-python/fix-paths.patch
https://github.com/NixOS/nixpkgs/blob/master/pkgs/desktops/cinnamon/nemo-extensions/nemo-python/python-path.patch

From what I saw in nautilus-python, it probably should be run before pygobject_init, but I did not test yet
https://github.com/GNOME/libpeas/blob/1.36/loaders/python/peas-plugin-loader-python.c#L252C3-L252C17

I now saw how libpeas are used here. I wonder if we should patch libpeas here, previously we do this:
master/pkgs/desktops/gnome/misc/nautilus-python/fix-paths.patch
From what I saw in nautilus-python, it probably should be run before pygobject_init, but I did not test yet
<EMAIL_ADDRESS>
https://github.com/NixOS/nixpkgs/compare/master...FedericoSchonborn:nixpkgs:budgie-media-player-with-libpeas-fix-paths-patch
It works, I didn't test anything else apart from Pluma and Eye of MATE plugins though.

@bobby285271 @jtojnar
Do I make a PR for the libpeas change? Should I base it on/wait for #247766?

Yeah the change should probably be applied to libpeas2 too (which is being packaged in https://github.com/NixOS/nixpkgs/pull/247766), still haven't got around to try the branch yet :upside_down_face:

master...FedericoSchonborn:nixpkgs:budgie-media-player-with-libpeas-fix-paths-patch
It works, I didn't test anything else apart from Pluma and Eye of MATE plugins though.

That looks great at a glance. We definitely do not want to set PYTHONPATH in a wrapper in anything that can run other programs (which is basically anything).
The downside is that pygobject will be always pulled in, even when there are no Python plug-ins, like in GNOME Builder ≥ 45. But it is probably fine, when we are already pulling in libpython.
And it does not cover any other Python dependencies that the plug-ins might need. For that we will need to inject the directories to site directly in the plug-ins, similarly to how we do it in GNOME Shell extensions.
And we will probably need to do something similar with lgi to make it work with lua.

Almost forgot about this :grin:
@bobby285271


Only test for consistency of pauli extension when not falling back to unitary

This saves about 40 seconds off pytest cirq.
Not sure I like how I solved this.

Automerge cancelled: A status check is failing.


[SPARK-23524] Big local shuffle blocks should not be checked for corruption.

What changes were proposed in this pull request?
In current code, all local blocks will be checked for corruption no matter it's big or not.  The reasons are as below:
Size in FetchResult for local block is set to be 0 (https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala#L327)
SPARK-4105 meant to only check the small blocks(size<maxBytesInFlight/3), but for reason 1, below check will be invalid. https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala#L420
We can fix this and avoid the OOM.
How was this patch tested?
UT added

Jenkins, test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1099/
Test PASSed.

Test build #87714 has started for PR 20685 at commit 535916c.

Test build #87714 has finished for PR 20685 at commit 535916c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87714/
Test FAILed.

Test build #87741 has started for PR 20685 at commit 110c851.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1123/
Test PASSed.

Jenkins, retest this please

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1132/
Test PASSed.

Merged build finished. Test PASSed.

Test build #87755 has started for PR 20685 at commit 110c851.

Test build #87755 has finished for PR 20685 at commit 110c851.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87755/
Test FAILed.

Jenkins, retest this please.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1142/
Test PASSed.

Test build #87770 has started for PR 20685 at commit 110c851.

Test build #87770 has finished for PR 20685 at commit 110c851.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87770/
Test FAILed.

Merged build finished. Test FAILed.

The failed test is not related， I can pass in my local

Jenkins, retest this please.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1171/
Test PASSed.

Test build #87807 has started for PR 20685 at commit 110c851.

Test build #87807 has finished for PR 20685 at commit 110c851.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87807/
Test PASSed.

cc @cloud-fan @jiangxb1987
Could you please help take a look.

We should update the doc of SuccessFetchResult#size, currently it's saying estimated size of the block, used to calculate bytesInFlight.

@squito @cloud-fan
Thanks you so much for reviewing. I refined accordingly.  Please take another look when you have time.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1316/
Test PASSed.

Test build #88003 has started for PR 20685 at commit 1cd8653.

Test build #88003 has finished for PR 20685 at commit 1cd8653.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88003/
Test PASSed.

From what I've seen in https://github.com/apache/spark/pull/15923 , this corruption check is mostly to detect network failures, so seems like we don't need this check for local blocks at all.
cc @davies  @zsxwing

it'll also help with disk corruption ... from the stack traces in SPARK-4105 you can't really tell what the source of the problem is.  it'll be pretty hard to determine what the source of corruption is if we start seeing it again.  anyway, I don't feel that strongly about it either way.

sounds reasonable. The purpose of this corruption check is to fail fast to retry the stage(re-shuffle), so disk corruption should also be counted.

LGTM

@cloud-fan @squito
Thanks a lot !

Test build #88033 has finished for PR 20685 at commit 4e4f075.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88033/
Test PASSed.

I think #20179 probably already fixed the data corruption issue.

yea very likely, but I'm not 100% sure. How about we merge this one first to fix the mistake for local shuffle blocks, and then think about whether or not we should remove this corruption check?

lgtm

thanks, merging to master/2.3!


[CI] Reusable asan

Signed-off-by: Esteban Martinena<EMAIL_ADDRESS>

We will probably need to change the name for asan job in the PR requirements.


Feature: bytes bytes

Some more utilities to pack and unpack from and to bytes. Pretty basic stuff, surprised it wasn't implemented yet.

Ignoring red tests, they are the same ones failing on master.


Syntax highlighting, commenting, and line numbers not working

Hey there. It seems that JSBin is able to run my code and spit out the correct errors but syntax highlighting, commenting shortcut, and line numbers are not working today even though I have these features turned on in my settings.

Any ideas why this might be happening?
Thank in advance! :)

You've switched to textarea mode. Open the big top menu (click on the jsbin icon in the menu) then uncheck textarea editor mode

oh wow thank you!


How to use controller support in devilutionX

Hi,
I saw that you added instructions on how to compile this using Visual Studio. I wonder if the controller support could be extended to devilutionX as well.

I'm working on fixing up Devilution-NX https://github.com/erfg12/devilution-nx

Neat!, let me know if you need testing.

It's now in devilution-X thanks to everyone converting my DirectX code to OpenGL.


Allow per-buffer disabling of whitespace checking

I'd like to disable airline whitespace checking on a per-buffer basis.  I use the QuickFix Reflector plugin which makes QuickFix buffers writable, causing them to be checked for trailing whitespace.  There doesn't appear to be a way to disable this checking on a per-buffer basis.  I'm currently using the below one-line patch combined with logic in my .vimrc to set the variable b:airline_whitespace_disabled, but anything that would give manual control on a per-buffer basis would suit.
Thanks,
Michael Henry
Patch:
diff --git a/bundle/airline/autoload/airline/extensions/whitespace.vim b/bundle/airline/autoload/airline/extensions/whitespace.vim
index 7e13d47..5f7dae2 100644
--- a/bundle/airline/autoload/airline/extensions/whitespace.vim
+++ b/bundle/airline/autoload/airline/extensions/whitespace.vim
@@ -51,6 +51,7 @@ endfunction

 function! airline#extensions#whitespace#check()
   if &readonly || !&modifiable || !s:enabled || line('$') > s:max_lines
+          \ || exists('b:airline_whitespace_disabled')
     return ''
   endif

Sounds good. Would you like to create a PR for this (please don't forget to update the documentation). Thanks.

Thanks; I've submitted a pull request.


Different table creation strategies for Iceberg

Athena engine v3 supports CTAS for Iceberg.
Since engine v2 does not support that, what about adding a table_strategy in the table materialization with the following alternatives : tmp_table and ctas.

ctas is by default for all non iceberg tables and the only possibility.
tmp_table must be set for engine v2 for Iceberg tables.
ctas or tmp_table can be chosen for engine v3 for Iceberg tables.

What do you think ?

Nice idea, I like it, so by default our ctas

https://github.com/dbt-athena/dbt-athena/pull/74#issuecomment-1337015815
Meaning everything will be created using CTAS.


Page Navigation is Missing

Cannot navigate between the login screen and the home screen

Fixed #92 Commit: dea5bcf


Language filter is not accurate

See image, but it looks like CSS/npm/etc issues are tagged 'Python' incorrectly. Perhaps the project they are in has some python in it? But this isn't helpful if I am interested in hacking on some python issues.

@snakescott one of the problems on how Govcode works is that we pull the language info from what Github thinks the main language for the repo is and we use it for the issue filter.
That is not very granular, Can you think of any other way that we can make it more so?