Implement partially mapped bitmaps: hybrid Java heap + off heap

This is an issue raised by @kishoreg and related to how Pinot uses Roaring bitmaps.
Currently, we support ImmutableRoaringBitmap... MutableRoaringBitmap and RoaringBitmap. The ImmutableRoaringBitmap class stores the containers off-heap (possibly on disk). Other classes (MutableRoaringBitmap and RoaringBitmap) store the data on Java's heap.
However, there are instances where we would like just one container to be stored in memory and to be mutable, whereas the rest of the containers should be off-heap and immutable.
I think that we are lucky in that this can be implemented in a sane manner, without having to go crazy and risk lots of bugs. However, I have not yet come up with a design that I like, so I want to open this up for discussion. Another issue is that I am not 100% clear on what would best serve Pinot's needs.
A MutableRoaringBitmap is implemented as a derived class of ImmutableRoaringBitmap.
The key implementation concept is the attribute  PointableRoaringArray highLowContainer in the class ImmutableRoaringBitmap. We currently have two types of PointableRoaringArray. The base ImmutableRoaringBitmap class uses an ImmutableRoaringArray. Unsurprisingly, the MutableRoaringBitmap uses a MutableRoaringArray.
If one looks at the MutableRoaringBitmap class itself, it is rather thin. Maybe 1500 lines, but a lot of it may not be needed in a hybrid model, except maybe for an add method. That is, we inherit many useful methods from ImmutableRoaringBitmap.
I think that one possibility would be to implement something like a HybridRoaringBitmap that would be an instance of the base base ImmutableRoaringBitmap. It would have an ImmutableRoaringArray but also a MappeableContainer.
It might behave much like a MutableRoaringBitmap, except that any attempt at modifying a mapped container would generate an exception. I am not sure how you make sure that there is only ever just one mutable container.
I will stop here and open up the discussion.
See also https://github.com/RoaringBitmap/RoaringBitmap/issues/193 which might ever offer a viable alternative.

Thanks for starting creating this issue your initial analysis of the existing code is encouraging.
It would have an ImmutableRoaringArray but also a MappeableContainer.  Did you mean ImmutableRoaringArray and ArrayContainer?
The HybridBitmap would contain ImmutableRoaringArray pointing to a bunch of mappeableContainer and an ArrayContainer that is mutated until a set(index) method is called where index can no longer be set in the ArrayContainer. At this point, based on the cardinality of the ArrayContainer, we create the MappeableContainer implementation and add it to  ImmutableRoaringArray. At this point, we clear the ArrayContainer or create a new container whatever makes sense (time spent by JVM to Garbage collect the ArrayContainer vs zeroing all the bits).
This should ensure there is only one mutable container right?. We will have the assertion in add(index), add(start, end) add(index[]) methods to ensure that index/indices are always monotonically increasing.
Now the tricky thing when we convert the ArrayContainer to MappeableContainer, we need to provide a hook to the caller so this MappeableContainer is created off-heap (It can be DirectMemory or on Disk and MMap'd). The hook can be in form of a callback or we can take in a ByteBufferProvider that returns a ByteBuffer given a capacity. Its up to the caller to come up with the ByteBufferProvider and it can be made optional. We can have a default implementation that simply creates one in memory.
Does that make sense?

c.c. @blasd

@kishoreg
(Note: I'd like to have @blasd's feedback on this issue as well, as I think there is overlapping interests and engineering.)

Did you mean ImmutableRoaringArray and ArrayContainer?

I think I meant what I wrote. We have MappeableArrayContainer which is an instance of MappeableContainer. However, we might not want to assume that, in general, the new container in flight would be a MappeableArrayContainer.

The HybridBitmap would contain ImmutableRoaringArray pointing to a bunch of mappeableContainer and an ArrayContainer. The ArrayContainer is in memory and is mutated we fill up the container. Once the container is full, we create a MappeableContainer and add it to ImmutableRoaringArray. After this, we can clear the ArrayContainer or create a new container whatever makes sense (time spent by JVM to Garbage collect the ArrayContainer vs zeroing all the bits).

I'd be tempted to open up the design space a bit. It may be worth it to consider the scenario where ArrayContainer/MappeableArrayContainer is not the best way to store the container in flight. At least, it might be preferable not to close the door early on.

This should ensure there is only one mutable container right?. We will have the assertion in add(index), add(start, end) add(index[]) methods to ensure that index/indices are always monotonically increasing.

Right.

Now the tricky part is - Providing a hook to Pinot or the caller of the API when the ArrayContainer is converted into a MappeableContainer. The hook can be in form of a callback or we can take in a ByteBufferProvider that returns a ByteBuffer given a capacity. It's up to the caller to come up with the ByteBufferProvider and it can be made optional. We can have a default implementation that simply creates one in memory. Does that make sense?

Yes. That sounds like a good plan.
Again, I'd like for @blasd  to get involved (and maybe others) so that we can come up  with the best design possible. It would be best if this design could be used by different systems, not just Pinot...

In https://github.com/RoaringBitmap/RoaringBitmap/pull/192, I suggested introducing/modifying multiple interfaces to facilitate decorator implementation working over RoaringBitmap and ImmutableRoaringBitmap without requiring duplication.
It led to propose the introduction of interfaces over  [Immutable]RoaringBitmap RoaringArray and further Container. I feel like these enhancements are directly directly with the ability of mixing various RoaringArray in a HybridRoaringBitmap. However, I am no quite sure to grasp yet the actual usecase underlying this issue (especially regarding the need for a hook manipulating raw ByteBuffer).
Could you provide some additional insights about what would look like the hook for Pinot? (e.g. is it a hook to enable Pinot deciding when to push the Container off-heap? or do you suggest a callback when such an event happens? ).
What would be a typical workload for HybridRoaringBitmap? (e.g. a lot of .add with growing longs, in which case you  expect Containers to be pushed off-heap when they can not be modified anymore, or rather the ability to .add in a Random location in which case you expect relevant Container to be switch to Heap)? I ask this as I am not sure why we should restrict only one container to live on-heap.
@lemire
Is there any reason why RoaringArray and MutableRoaringArray do not share most of their implementation?

+1 on modifying the interfaces such that most of the code can be shared across implementations.
Our main motivation is to support multi-tenancy in Pinot. We are able to do this on our historical nodes because none of the bitmap data structures(immutable roaring bitmap) reside in memory (they are mmaped and loaded on demand). We are able to scale to 1000+ tables on a single node.
Unfortunately, we cannot do this in real-time (where we ingest data in real-time from Kafka) nodes as the bitmap. This forces us to use MutableRoaringBitmap (v/s Immutable in the historical nodes). MutableRoaringBitmap needs to be kept in memory until we flush the segment to disk. Based on the available memory on the box, we can only keep up to X bitmaps in memory at any time.

The amount of memory available on the node dictates the number of tables. This results in poor utilization of CPU and storage.
If we end up using most of the memory for bitmap, then Kernel has to page in and page out the actual data during query processing which impacts latency.
MutableRoaringBitmap adds more load on the GC.

Ideally, we could have capitalized on the fact that we always append new records. Having a hybridRoaringBitmap where the latest container is mutable but it gets converted to immutable and flushed to disk, will allow us to scale to a large number of tables on a node.
Hope this helps.
Having said that, I am open to the idea of allowing any container to be updated but this is much more complicated. The size of the container already on disk can change after mutation which means we need to implement copy-on-write semantics. @lemire mentioned that Go implementation of bitmap had this concept. This is easier to implement if the containers are in memory since Go/JVM manage the memory and Garbage collect previous containers. But the on Disk implementation of copy-on-write model will be complex, inefficient and probably not worth the effort.

@blasd
Part of the code redundancy was motivated by performance. A MutableRoaringBitmap is much slower than a RoaringBitmap... There is a pretty severe penalty for going through a ByteBuffer... And you certainly do not want to use ByteBuffer objects if you can avoid them. ByteBuffers in Java are not nice. I could elaborate if you want.  Even just holding an array in a ByteBuffer is not nice... as a ByteBuffer tends to use more overhead memory than we'd like, all things considered.
Then there is branching (in its many forms): we try to avoid megamorphic calls if we can.
So while I would not be surprised if you can get the same functionality and the same performance with half the code... one should certainly not discard performance concerns when considering code improvements.

@kishoreg  Have you given any serious thought on how the containers get rewritten? This could drive part of the code design as well.
If you have one file per bitmap, it is not too bad,  you can just append the new container when the times come.  But is this a viable model for you?
We should not dismiss the copy-on-write model so quickly.
I imagine something like this.  You do not care where these pointers reside: the containers do not  have to be consecutive.

We have immutable containers. Should you ever copy them, then a mutable copy is made (so we are in the copy-on-write model). Within the code, all we have is a pointer (maybe within a ByteBuffer).
We have mutable containers. They are in java memory and work just as usual.

Then you have some "write out" function. It does not touch the immutable containers. However, it takes the mutable containers and then call a user-provided serialization function that writes the mutable container out.
We could then add an "add" method that is such that whenever a new mutable container is created, and one mutable container container already exists, the "write out" function gets called automatically.
I am probably not explaining myself very well... but I think we could get the copy-on-write approach to work well.

I think you can generalise this as a LRU cache consisting of Containers with an eviction policy that converts a cached Container to a MappableContainer. I think @kishoreg would like to be able to register a cache eviction listener ("Now the tricky part is - Providing a hook to Pinot or the caller of the API when the ArrayContainer is converted into a MappeableContainer. "). My interpretation is that he is describing a cache with only one container; his key requirement, given in-order writes, is that the Container currently being written to is fast to write to. If you generalise this as a cache consisting of hot Containers you get a win for fast membership queries too: common queries don't hit ByteBuffers. There's no need to bring MappableContainer and Container under the same interface in order to implement this.

@richardstartin You might be onto something.
We could think of a design where you have a RAM budget and you maintain containers according to your budget, and serialize containers when you are about to run out.
@kishoreg Your thoughts?

Thinking about this as an LRU cache is interesting and will be amazing if we can get one solution that covers a variety of scenarios without perf issues. Couple of questions -

what kind of cache are we looking at, read-only or write-through/around/back.
what would the interaction between cache eviction listener and the framework look like? Sample interfaces would help me get a better idea.
the latest container may not be the hottest container, its possible there are a lot of reads and no writes to the bitmap. In this case, how can we ensure that latest container is never evicted.

Regarding @lemire comment
Have you given any serious thought on how the containers get rewritten? This could drive part of the code design as well.
If you have one file per bitmap, it is not too bad, you can just append the new container when the times come. But is this a viable model for you?
Having one file per bitmap might result in running out of file handles.  I am thinking of pre-allocating a chunk of buffer for each bitmap and create a new one when that fills up. There will be another data structure that will maintain the list of start offsets for each unique value. Initially, we might do this offset management in memory. But of them will be modeled as append-only data structures. This is at a high level and I am sure we will run into some interesting scenarios when we start implementing it.

@kishoreg
LRU doesn’t specify R or W. A cache configured to evict the least recently written container solves your use case, and makes the freshest real-time data quick to query. The code would look similar to an eviction policy to remove the least recently read container.
I imagine eviction notification being similar to other cache implementations, a BiConsumer<Short, Container> or similar should suffice.
I envisage that you may only modify data through the cache.
I also expect that pre allocated BitmapContainers will be the fastest to write to (prior to reuse, Arrays.fill(bitmap, 0) will vectorise on a modern JVM, writes are constant time without binary search.)
The key point is this data structure can sit on top of what already exists, without reworking existing code and risking a maze of virtual calls.

@richardstatin Can you elaborate on how we might implement this cache?

@lemire prototypes speak louder than words and I think it should be relatively simple to start. I’ll get back to you shortly.

@lemire I wrote a very rough implementation of a cyclic bitmap which saves down to an underlying MutableRoaringBitmap when the buffer overflows. It assumes that values are written in order and that key change is the trigger for cache management. I wrote contains to delegate to the underlying bitmap when the requested key is not found. You can write simple implementations of rank, select, flip and so on in the same pattern. I haven't really thought about how to e.g. xor a cache with a cache yet. Should it produce a cache? What if the cached keys don't intersect? A buffered bitmap? A bitmap? Not really sure. The code is dog rough but should be enough to outline the idea.

@kishoreg  Can you give us your thoughts on this prototype?
https://github.com/richardstartin/RoaringBitmap/blob/e09a4a60335ae524b26fb57d7a10df474490d559/roaringbitmap/src/main/java/org/roaringbitmap/buffer/WAL.java

Thanks @richardstartin for the quick poc. Most of them look good. But there is no hook for the caller to generate the new container.
private void advanceAndRecycle() {         int evictionIndex = (head + 1) % containerCount;         int evictedKey = keys[evictionIndex];         if (evictedKey != -1) {             // TODO - figure out what kind of container is best for the data being evicted here             int card = containerCardinality(evictionIndex);             MappeableContainer container = card > WORD_COUNT                     ? new MappeableBitmapContainer(LongBuffer.wrap(bitmap, evictionIndex, WORD_COUNT), card)                     : new MappeableArrayContainer(createArray(evictionIndex, card), card);             // eviction listener goes here             for (EvictionListener listener : evictionListeners) {                 listener.onEviction((short)evictedKey, container);             }             mutableRoaringBitmap.appendContainer((short)evictedKey, container);             recycleBitmap(evictionIndex);         }         head = evictionIndex;     }
The creation of new MappeableContainer should be done outside the library. A simple solution is to take MappeableContainerProvider that takes in bitmap, card and returns MappeableContainer.
I can provide my own implementation where the buffer is mmap'ed.
           mutableRoaringBitmap.appendContainer((short)evictedKey, container);
Is it ok to append a mappeableContainer to mutableRoaringBitmap?

Ok so you don’t want an eviction listener (which was my interpretation, but a term I introduced to the conversation) at all. It’s more that you want to control the container creation. So EvictionListener in that prototype would be more like a Function<WALContainer, MappableContainer> in this example, and you’d only have one of them, with a default implementation similar to the code above the eviction listeners loop.

That works.  Do we expect users to provide the container count or Is it just part of poc.

It defaults to 1 if you don't specify it.

@kishoreg I implemented this specifically for RoaringBitmap in this pull request. There's no hook, but the performance improvement is very good compared to repeatedly adding data, and allows for run optimisation container by container. So whatever you come up with, it's likely it will be worthwhile doing the work performance-wise.

@richardstartin
Yes, the PR is interesting because it is simple, lends itself to high performance... there are possible limitations... like the fact that it is hard to combine queries and updates... but that's maybe ok. So I think we are maybe converging toward a solution that is much better than what I imagined if the Pinot problem is a good fit.
@kishoreg : I suggest you have a look at the PR. The idea is to have a "Writer" object. The "Writer" can use a small amount of memory (fixed). This is much easier to optimize. If something like this would do what you want to do, we could optimize the hell out of it.

That PR doesn't set out to solve Pinot's problem, it provides a mechanism to speed up unsorted writes by ~20x, but followed on from this discussion. I don't think the cache idea I proposed could work, or that it has been particularly helpful to developing a design here (I retract the proposal).
If the approach taken in #199 is relevant to solving this problem feel free to "optimize the hell out of it."

@richardstartin
I am aware of why you wrote it... and to be clear, I think your implementation is quite fast in many cases... in fact, you have hard numbers... but I mean that we can optimize it for the Pinot context... if we know more about their needs.
In any case, it is one possible route that @kishoreg  might want to review.


x/tools/gopls: CL 237517 breaks tempModfile at startup

What version of Go are you using (go version)?

$ go version
go version devel +d286e61b67 Mon Jun 15 23:29:23 2020 +0000 linux/amd64
$ go list -m golang.org/x/tools
golang.org/x/tools v0.0.0-20200618023723-47c907e25813
$ go list -m golang.org/x/tools/gopls
golang.org/x/tools/gopls v0.0.0-20200618023723-47c907e25813

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOENV="/home/myitcv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/myitcv/gostuff/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/myitcv/gos"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/myitcv/gostuff/src/github.com/myitcv/govim/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build549147748=/tmp/go-build -gno-record-gcc-switches"

What did you do?
govim has a test to ensure that the tempModfile setting does not modify a go.mod file on startup:
https://github.com/govim/govim/blob/e2a1802131f331832d667736aa27f023636f0c3a/cmd/govim/testdata/scenario_tempmodfile/modfile_changes.txt
CL 237517 appears to have broken this test:
--- FAIL: TestScripts (1.19s)
    main_test.go:77: using gopls at "/tmp/gobin-gopls-installdir036629073/gopls"
    --- FAIL: TestScripts/scenario_tempmodfile (0.03s)
        --- FAIL: TestScripts/scenario_tempmodfile/modfile_changes (5.38s)
            testscript.go:348:
                # A simple test that verifies we get a diagnostic for imports
                # that are missing from our go.mod file (0.000s)
                # Open main.go and save to ensure we are actually doing something via gopls (0.168s)
                # Sleep because we don't have an event we are waiting for (5.001s)
                # Verify that there have been no changes to go.mod (0.005s)
                > cmp go.mod go.mod.golden
                [diff -go.mod +go.mod.golden]
                 module mod.com

                -go 1.15

                FAIL: testdata/scenario_tempmodfile/modfile_changes.txt:16: go.mod and go.mod.golden differ

i.e. with that CL applied, the resulting go.mod file after opening and saving main.go has been modified with go 1.15 added (in my local tests, that's the version of Go I am using)
Caught in the daily govim vs gopls@master tests:
https://github.com/govim/govim/actions/runs/139442828
What did you expect to see?
The test continuing to pass.
What did you see instead?
As above.

cc @stamblerre @findleyr
FYI @leitzler

Hmm, looks like this test is not -short, which is why we didn't catch it in our govim integration.
I can switch back to running all tests, not just -short, though I may need to extend our timeout to 20m, which is quite long.  Is there a good way to skip just the installer tests?

The install scripts will only run if GOVIM_RUN_INSTALL_TESTSCRIPTS=true
https://github.com/govim/govim/blob/e2a1802131f331832d667736aa27f023636f0c3a/cmd/govim/main_test.go#L258
So you should be fine by default.

The install scripts will only run if GOVIM_RUN_INSTALL_TESTSCRIPTS=true

Ah right, I forgot that. Thanks.


[xDNN] Update xDNN version 1.2.1.

let's expose add more functions to support computing for A: BF16, B: BF16, C:BF16, and then merge it.

let's add more functions to support computing for A: BF16, B: BF16, C:BF16, and then merge it.

@pujiang2018  Add transposed weight BMM.


Add support to configure multiple GraphTermServiceProvider instances

In order to flexibly support user-defined segmentation of code system content across one or more graph databases and for consistency with RemoteTermServiceProvider we would like to allow for multiple configuration elements in fhir-server-config.json as follows:
            "graphTermServiceProviders": [{
                "__comment": "Configuration for Berkeley DB Java Edition + Lucene",
                "enabled": false,
                "configuration": {
                    "storage.backend": "berkeleyje",
                    "storage.directory": "data/graph",
                    "index.search.backend": "lucene",
                    "index.search.hostname": "data/searchindex",
                    "storage.read-only": true,
                    "query.batch": true,
                    "query.batch-property-prefetch": true,
                    "query.fast-property": true
                }
            },{
                "__comment": "Configuration for Apache Cassandra + ElasticSearch",
                "enabled": true,
                "timeLimit": 10000,
                "configuration": {
                    "storage.backend": "cql",
                    "storage.hostname": "<IP_ADDRESS>",
                    "index.search.backend": "elasticsearch",
                    "index.search.hostname": "<IP_ADDRESS>:9200",
                    "storage.read-only": true,
                    "query.batch": true,
                    "query.batch-property-prefetch": true,
                    "query.fast-property": true
                }
            }],

When I started my server with the old config, it failed with a NPE.  We need to handle the absence of config parameters just as well as their presence.

I opened a PR to address this config issue and that has been merged.


Tags for 0.5.4 release

Just noticed that the tags needed an update with the new release. Do we need to make a change anywhere else in the code or docs?

d'oh :)


fixed broken link

This change is

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/c5b3aad3ca76319064468a074947c2fa40454fa5/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/d71e1bc93fb717f67c4aa59c505da2ca52ae91c9/

Thanks, @nstewart!


Deprecate App::with_defaults()

Closes #638

r? @kbknapp

Is @homu dead?

@homu r+

:pushpin: Commit 2608540 has been approved by kbknapp

:hourglass: Testing commit 2608540 with merge c9230c1...

Doesn't look like it?

:sunny: Test successful - status

It seemed so for a while


Juliainstaller Crash Report

Attempted to install Julia using "curl -fsSL https://install.julialang.org | sh" and got the following message:
"Well, this is embarrassing.
Juliainstaller had a problem and crashed. To help us diagnose the problem you can send us a crash report.
We have generated a report file at "/var/folders/hz/jsqbp65n72bgd53c3prn1q8h0000gp/T/report-10855ac7-c6a2-4623-bc1c-5cee2ba660a4.toml". Submit an issue or email with the subject of "Juliainstaller Crash Report" and include the report as an attachment.

Homepage: https://github.com/JuliaLang/juliaup

We take privacy seriously, and do not perform any automated error collection. In order to improve the software, we rely on people to submit reports.
Thank you kindly!"
The report file contains the following information:
"name" = "Juliainstaller"
"operating_system" = "Mac OS 14.4.0 [64-bit]"
"crate_version" = "1.14.7"
"explanation" = """
Panic occurred in file 'src/operations.rs' at line 1021
"""
"cause" = """
called Result::unwrap() on an Err value: Failed to open file /Users/Reese/.zshrc.
Caused by:
Permission denied (os error 13)"""
"method" = "Panic"
"backtrace" = """
0: 0x105923175 - core::result::unwrap_failed::h951d84d71b0bada2
1: 0x105740aa2 - juliaup::command_config_modifypath::run_command_config_modifypath::h100e4c863f62efca
2: 0x1055ef798 - juliainstaller::main::hb4c72525a8c2646f
3: 0x1055d88a6 - std::sys_common::backtrace::__rust_begin_short_backtrace::h4c8c12f28586b664
4: 0x1055f6843 - _main
5: 0x7ff81a456366 - """
I updated my Mac software from 14.1.1 to 14.4.0 and had the same issue both times.

+1
"name" = "Juliainstaller"
"operating_system" = "Mac OS 14.3.0 [64-bit]"
"crate_version" = "1.14.7"
"explanation" = """
Panic occurred in file 'src/operations.rs' at line 1021
"""
"cause" = """
called Result::unwrap() on an Err value: Failed to open file /Users/~/.bash_profile.
Caused by:
Permission denied (os error 13)"""
"method" = "Panic"

+1
"name" = "Juliainstaller"
"operating_system" = "Mac OS 14.5.0 [64-bit]"
"crate_version" = "1.14.9"
"explanation" = """
Panic occurred in file 'src/operations.rs' at line 1021
"""
"cause" = """
called Result::unwrap() on an Err value: Failed to open file /Users/vyas/.bash_profile.
Caused by:
Permission denied (os error 13)"""
"method" = "Panic"
"backtrace" = """
0: 0x1007dbd94 - core::result::unwrap_failed::h8e3b933261dd7fec
1: 0x10069f990 - juliaup::command_config_modifypath::run_command_config_modifypath::hca63d35b576be3c9
2: 0x1005af58c - juliainstaller::main::hb0bee2b2a4d9e158
3: 0x10059b268 - std::sys_common::backtrace::__rust_begin_short_backtrace::h2b1ac18042ef3658
4: 0x1005b3c4c - _main"""


No issues found but...

...just wanted to thank you, @vtimofeev. This works really nicely and smoothly.

Thanks!


USART: Remove mode check in enter_mode

This patch addresses issue https://github.com/atsams-rs/atsamx7x-rust/issues/27 wherein USART configuration fails if no other mode was selected prior to that. This is done by permitting the mode to be set multiple times, even if it is the same mode.
Initially, the USART is in the default USART mode but has not actually been configured. Therefore the enter_mode routine will exit early and skip configuration entirely unless care is taken and a dummy reconfiguration step is done before switching to USART.
The consequence of not performing this check is that the bus configuration will reset and possibly glitch momentarily if the same mode is setup twice, but that should not be done anyhow and has not been encouraged as far as I can tell.

For future references, preferring a rebase-unto over merge improves history readability.


Paste into phone field adds "ext." when there is none.

When pasting a phone number "561 228 1612" into the input and allow extension is true, the plugin automatically formats the number which is good but adds an empty extension, so the phone number ends up like this: "(561) 228-1612 ext."
How do you prevent that from happening.

Thanks for reporting this. I think it would be fixed by passing true as the third option to the _handleInputKey call on line 489 of src/js/intlTelInput.js, but this needs testing, and if it works, it needs commenting and building and checking the tests still pass (and ideally adding a new test) and then doing a pull request, which unfortunately I don't have the time to do right now, but I'm very open to other people contributing this work!

Fixed in v7.0.0


Rethink SQLite, are there better alternatives?

Is SQLite the best choice for tinyvector? Would a NoSQL database make more sense for embeddings databases? Discussion needed.

duckdb? or too close to chroma? easy to embed and use for SQL people like me.
love this project idea btw. practical/simple is the way.

I looked into RocksDB for the Rust version, prolly not the best for adding attribute querying tho


Fails to compile on aarch64 with rust 1.82.0 : root Cargo.toml is missing field version

On a Mac:
cargo build
error: failed to parse manifest at `/Users/davidr/workspaces/tinywasm/Cargo.toml`

Caused by:
  missing field `version`
  in `package`

On an arm32v7 Raspberry Pi, it builds fine.

Hm, seems weird that cargo is behaving differently for both targets. This CI run on 1.82 also didn't error on macos with an m1. The version field shouldn't be required as the root package is never published (https://doc.rust-lang.org/cargo/reference/manifest.html#the-version-field, since 1.75), can you double check your rust version? Might be a cargo bug.

$ rustc --version
rustc 1.82.0 (f6e511eec 2024-10-15)
$ cargo --version
cargo 1.82.0 (8f40fc59f 2024-08-21)
$ machine
arm64e

Nope.
$ cargo build --workspace
error: failed to parse manifest at `/Users/davidr/workspaces/tinywasm/Cargo.toml`

Caused by:
  missing field `version`
  in `package`

In my personal projects I use cargo set-version <n.n.n> which sets all the workspace member versions, and the root, in a single command.

Hm, odd. I don't have my mac setup with me right now so I can check in a couple of days. Until then I guess just adding a "0.0.0" version to the root [package] should work, seems like a small cargo bug.
I'm using cargo-workspaces to publish/version since it can independently version specific workspace members.

No rush. Just wanted it added to the backlog, for visibility.


Capabilities error handling should be checked

error handling should be checked
Originally posted by @mlipok in https://github.com/Danp2/au3WebDriver/pull/263#discussion_r830701133

Can this be continued in the existing #105?

Yes


Save Build to URL

Do we need a hasher? https://stackoverflow.com/questions/18279141/javascript-string-encryption-and-decryption


Add options --warmup and --n-iteration to RunONNXModel.py

Added two options to RunONNXModel.py:

--warmup to specify the number of warmup inferences. Default 0 means no warmup.
--n-iteration to specify the number of repeated interences. Default 1.

For example:
ONNX_MLIR_HOME=./Debug python ../utils/RunONNXModel.py --model matmul.onnx --warmup=2 --n-iteration=10
Temporary directory has been created at /tmp/tmp8o1fomgu
Compiling the model ...
  took  0.24071840941905975  seconds.

Loading the compiled model ...
  took  0.0001874864101409912  seconds.

Generating random inputs ...
  - 1st input's shape (3, 4), element type float32. Value ranges [-0.1, 0.1]
  - 2nd input's shape (4, 3), element type float32. Value ranges [-0.1, 0.1]
  done.

Running inference ...
  1st warmup: 0.0001206621527671814 seconds
  2nd warmup: 1.1652708053588867e-05 seconds
  1st iteration: 9.611248970031738e-06 seconds
  2nd iteration: 8.64267349243164e-06 seconds
  3rd iteration: 8.07642936706543e-06 seconds
  4th iteration: 9.253621101379395e-06 seconds
  5th iteration: 7.830560207366943e-06 seconds
  6th iteration: 7.860362529754639e-06 seconds
  7th iteration: 7.659196853637695e-06 seconds
  8th iteration: 7.577240467071533e-06 seconds
  9th iteration: 7.584691047668457e-06 seconds
  10th iteration: 7.547438144683838e-06 seconds
  Statistics (excluding warmup): min 7.547438144683838e-06, max 9.611248970031738e-06, mean 8.16434621810913e-06, stddev 7.097068136673706e-07

LGTM, I would add min, avg, stddev stats.

Good suggestion! I added them. The output looks like this:
ONNX_MLIR_HOME=./Debug python ../utils/RunONNXModel.py --model matmul.onnx --warmup=2 --n-iteration=10
Temporary directory has been created at /tmp/tmp8o1fomgu
Compiling the model ...
  took  0.24071840941905975  seconds.

Loading the compiled model ...
  took  0.0001874864101409912  seconds.

Generating random inputs ...
  - 1st input's shape (3, 4), element type float32. Value ranges [-0.1, 0.1]
  - 2nd input's shape (4, 3), element type float32. Value ranges [-0.1, 0.1]
  done.

Running inference ...
  1st warmup: 0.0001206621527671814 seconds
  2nd warmup: 1.1652708053588867e-05 seconds
  1st iteration: 9.611248970031738e-06 seconds
  2nd iteration: 8.64267349243164e-06 seconds
  3rd iteration: 8.07642936706543e-06 seconds
  4th iteration: 9.253621101379395e-06 seconds
  5th iteration: 7.830560207366943e-06 seconds
  6th iteration: 7.860362529754639e-06 seconds
  7th iteration: 7.659196853637695e-06 seconds
  8th iteration: 7.577240467071533e-06 seconds
  9th iteration: 7.584691047668457e-06 seconds
  10th iteration: 7.547438144683838e-06 seconds
  Statistics (excluding warmup): min 7.547438144683838e-06, max 9.611248970031738e-06, mean 8.16434621810913e-06, stddev 7.097068136673706e-07

Jenkins Linux amd64 Build #11953 [push] Add options --warmup and... started at 23:13

Jenkins Linux ppc64le Build #10960 [push] Add options --warmup and... started at 00:18

Jenkins Linux s390x Build #11966 [push] Add options --warmup and... started at 00:13

Jenkins Linux amd64 Build #11953 [push] Add options --warmup and... failed after 1 hr 31 min

Jenkins Linux s390x Build #11966 [push] Add options --warmup and... passed after 1 hr 37 min


[BEAM-8144] Update hub setup in run_rc_validation.sh

Update hub version to latest stable version 2.12.0 (required by hub api)
Add local git repo setup block, using token to access Git API (e.g. git push and hub)
Remove USER_REMOTE_URL from script.config, instead use GITHUB_TOKEN which is more flexible. GITHUB_TOKEN should be created ahead from Github account.
Use hub api to auto-comment on PR for triggering Python ReleaseCandidate Jenkins job.
Fix repo typo in verify_release_build.sh

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.


Update readme to include vs theme

Hi,
Sorry, just realised I forgot to update the readme to include the new VS theme in the last pull request!

Thanks!


bug in Day / Month / Year time picker

How to reproduce:

select a month with 30 or 31 days and select the 30th or the 31st day
scroll to a different month which has less days than the one from the step above
the day should reset to the current month's maximum but instead remains at 30 or 31

Actually this is not fully resolved. See screenshot below:

I will file a PR for it soon.

Actually this is not fully resolved. See screenshot below:

I will file a PR for it soon.


Rename the existing "light wallet" and "thin wallet" gRPC services to avoid a name collision;

From #587:

Note: there's an unfortunate naming collision here, because we're currently sometimes using "wallet service" to refer to RPCs provided by full nodes with no user data to a wallet, rather than an RPC provided by a wallet to something that wants to use the chain. As a fix, I'd suggest that we align on using "client protocol" for RPCs provided by full nodes and "wallet protocol" for RPCs provided by the wallet service.

#600 renames these away from "wallet", but introduces a new collision with "light client".


fix: return-values of Commit::footers() and Commit::breaking_description() use lifetime of message…

…instead of the one of self of the parsed structure, which allows
these return values to stay alive even after Commit perished.

New release out with this fix.

Awesome, thanks a lot!
I noticed that now that smart-release can make releases easy, the most difficult thing to do consistently is to produce decent changelogs. There I envision a non-destructive tool that can merge information from the commit history into the respective changelog, none-destructively, allowing the user to write their own even based on the initial version created by the tool. A possible workflow would be to run cargo changelog once to update the changelogs related to an impending release, edit them manually until they are nice, keeping the 'unreleased' title, followed by cargo smart-release to update this title to the actual version number that it then can choose itself if not specified.
Further down the road, this would enable dumping the changelog of the current release into the release tag object, and even run gh releases create to have that picked up by github to create nice release notifications.
With all that in place, making high-quality releases of complex workspaces should finally be achieved.

I feel like your post needs to end with maniacal laughter :)
Personally, I like my release to be made in my CI in response to my tag.  This keeps the hosting system agnostic to the language release process.  I use release-notes.py in a CI job to create my release.
btw if you want to use Conventional to help with releases, try out committed to encourage use of it.
As I said before, I go back and forth on release notes.  I'm not anti-automated, something is better than nothing.  Adoption of Conventional hasn't been great and I dislike dealing with merge-conflicts in CHANGELOGs.  Also, once you get to workspaces, it becomes unclear which crate the message applies to when dealing with multi-crate changes.  One idea is to have a Github Action to auto-submit a changelog fragment which you can manually review and cleanup before release.

I feel like your post needs to end with maniacal laughter :)

😆

Personally, I like my release to be made in my CI in response to my tag. This keeps the hosting system agnostic to the language release process. I use release-notes.py in a CI job to create my release.

Thanks for sharing - I think ultimately that's also what I will do at least for gitoxide. CI would probably pull the changes from the tag object and write it directly into the release. Apparently GitHub does that automatically (?) or can be told to do that, so it shouldn't be too hard either way. Then again, if gh is available, it seems so much easier to trigger this locally. Let's see. (Yes, 180° turn of opinion in one paragraph, pure madness).

btw if you want to use Conventional to help with releases, try out committed to encourage use of it.

I saw it, but also know I will be the worst offender. Having tried 'conventional' before I know I only want to use it for changelog generation, while most of the commits will just document those tiny steps during development.

As I said before, I go back and forth on release notes. I'm not anti-automated, something is better than nothing.

I couldn't agree more, manual editing is required for awesome changelogs. However, I think there should be a way to generate a frame for the user to fill in and auto-update titles during release. This should be the best of both worlds then.

Adoption of Conventional hasn't been great and I dislike dealing with merge-conflicts in CHANGELOGs.

My plan is to only deal with changelogs before release, pulling the relevant messages from the commit history.

Also, once you get to workspaces, it becomes unclear which crate the message applies to when dealing with multi-crate changes. One idea is to have a Github Action to auto-submit a changelog fragment which you can manually review and cleanup before release.

Absolutely - sometimes a crate used by others changes, and there is a commit spanning multiple crates. This potentially makes the commit appear in multiple changelogs, and it's the user to to decide what to do with it. I plan to have a mechanism to identify all entries in a changelog and add hidden (but parsable) remarks to allow hiding them. That way, next time the changelog is generated the hidden ones won't re-appear. The same mechanism can be used to change entries, i.e. improve their wording or add details.
Thanks for git-conventional btw., I think it's very well done and saved me a lot of time. On a whim I felt it would be nice to have it parse bytes just because the unspecified (but assumed to be UTF-8) git commit encoding, so it's certainly fair to use str instead. Who knows, maybe the possible legacy encodings would make markers like : impossible to find anyway and it's better to try converting it to UTF-8 when encountered.


Bundle generation error

Last four lines are returned to me as an error. If it's proper behavior, then it should be information, otherwise it is some kind of bug:
in "/Users/sheerun/Sites/foobar/opost/src/" using the "yml" format.
Do you confirm generation [yes]?
Bundle generation
Generating the bundle code: OK
Checking that the bundle is autoloaded: OK
Confirm automatic update of your Kernel [yes]?
Enabling the bundle inside the Kernel: OK
Confirm automatic update of the Routing [yes]?
Importing the bundle routing resource: FAILED
The command was not able to configure your everything automatically.
You must do the following changes manually.
Bundle FooBarOpostBundle is already imported.```

I had a fix for this issue. I have done field testing but haven't got anyone to review the change. If anyone has time, please let me hear your comment.
The change is available at https://github.com/shiroyuki/SensioGeneratorBundle/commit/55640b1983dc18d4c2b406258796bbf9e07ecbf2.

@shiroyuki You never sent a pull request for it, so it is logical you did not got review. We are not checking manually all forks to see changes in them. If you don't asked to pull it into the main repo, it will probably never happen.

@stof I just got my colleague to help me review. However, due to that I fixed this in multiple commits, I decided to send out the patch instead.

a pull request done in several commits is not an issue. And it makes things far easier than posting a patch in a comment

@stof then, you would be happy that i just made a pull request a while ago. Sorry, i'm very new to the process here.


Chat - After marking a message as unread, sending a new message causes the chat to scroll up

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.73-1
Reproducible in staging?: Y
Reproducible in production?: N
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4555724&group_by=cases:section_id&group_order=asc&group_id=296775
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Issue reported by: Applause - Internal Team
Action Performed:

Open a chat
Mark a message as Unread
Send a message

Expected Result:
After marking a message as unread, sending a new message should not cause the chat to scroll up it should maintain the current position
Actual Result:
After marking a message as unread, sending a new message causes the chat to scroll up
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/f4f92185-2d27-46b8-bbaf-9fac19c34047
View all open jobs on GitHub

@tylerkaraszewski  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

We think that this bug might be related to #vip-vsp

sounds like this is not related to the API so removing the web deploy blocker label, this should also hep us figure out if it was your PR (or another app PR) or not.

I confirm it's caused by my PR. Will open a follow-up PR today with the fix.

Revert PR merged, requesting CP 🙏

I don't see a CP request in #deployer @Beamanator ?

This has been fixed here: https://github.com/Expensify/App/pull/42167#issuecomment-2111203635

Fixed, so closing 👍  This issue has been linked in https://github.com/Expensify/App/issues/35011, the original implementation PR, so we know to fix this bug in the next PR 👍


Fix AudioNode.disconnect error when disconnecting from signals/params

WebAudio's AudioNode.disconnect will throw an error if specifying an input number when disconnecting from an AudioParam (at least in chrome and firefox for linux). This pull request checks if we're disconnecting from an AudioParam before calling WebAudio's native disconnect method, and calls it without specifying an input if we are.

Coverage decreased (-0.01%) to 98.766% when pulling c3c096f07d1122a74c5c83fd71a2e3d7133bbf61 on jbpuryear:dev into 147341688c46d41c79896e37496302f0dd2a8d7a on Tonejs:dev.

Coverage decreased (-0.01%) to 98.766% when pulling c3c096f07d1122a74c5c83fd71a2e3d7133bbf61 on jbpuryear:dev into 147341688c46d41c79896e37496302f0dd2a8d7a on Tonejs:dev.

thanks!


update Tendermint genesis

closes #916

[x] Referenced an issue explaining the need for the change
[x] Updated all relevant documentation in docs
[x] Updated all code comments where relevant
[x] Wrote tests
[ ] Added entry in .changelog/

Codecov Report

Merging #917 (531fadf) into master (21ef18f) will increase coverage by 0.0%.
The diff coverage is 83.3%.

:exclamation: Current head 531fadf differs from pull request most recent head 7e1e42e. Consider uploading reports for the commit 7e1e42e to get more accurate results

@@          Coverage Diff           @@
##           master    #917   +/-   ##
======================================
  Coverage    70.6%   70.7%
======================================
  Files         203     203
  Lines       16217   16206   -11
======================================
+ Hits        11465   11473    +8
+ Misses       4752    4733   -19

Impacted Files
Coverage Δ

tendermint/src/block/size.rs
12.5% <ø> (ø)

tendermint/src/genesis.rs
100.0% <ø> (ø)

tendermint/src/validator.rs
69.8% <72.7%> (-0.2%)
:arrow_down:

light-client/src/operations/voting_power.rs
95.2% <100.0%> (-0.1%)
:arrow_down:

rpc/tests/kvstore_fixtures.rs
95.4% <100.0%> (ø)

rpc/tests/parse_response.rs
98.0% <100.0%> (ø)

testgen/src/validator.rs
84.5% <100.0%> (+0.1%)
:arrow_up:

tendermint/src/block/commit_sig.rs
77.2% <0.0%> (-2.3%)
:arrow_down:

tendermint/src/timeout.rs
83.0% <0.0%> (-1.9%)
:arrow_down:

tendermint/src/abci/path.rs
0.0% <0.0%> (ø)

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 21ef18f...7e1e42e. Read the comment docs.


Dynamically added picker causes resize JS error

Dynamically added and then hidden select pickers cause a JS error when you resize the browser.  To reproduce:
https://plnkr.co/edit/PnylbTpbC6AZhZIh0OXi?p=preview

Click 'Add Picker' button to dynamically create a new bootstrap-select picker.
Select an option.
Click 'Disable Picker' button to hide the selectpicker (destroy causes the same behavior).
Resize the browser.  You'll get a JS error about the resizing (lots of errors actually):

"bootstrap-select.js:820 Uncaught TypeError: Cannot read property '0' of undefined
    at scroll (bootstrap-select.js:820)
    at bootstrap-select.js:918
    at dispatch (jquery.min.js:3)
    at q.handle (jquery.min.js:3)"

I believe it's only when the select picker is dynamically added and then removed.
Happens in: IE 11, Firefox 52, Chrome 66 and 68, Safari 10. Both Windows and Mac OS.
Environment: Bootstrap 4.1.x, bootstrap-select 1.13.2
This may or may not relate to Issue #2034.

I've come across a similar issue, and I believe it's part of a larger problem.
There are two issues here:

The big one - a lot of the event functions (in my case getSelectPosition) either don't use the target in the event, or always expect the element to exist and be visible, which isn't always the case. When it's not the case, it immediately errors instead of cancelling.
The reason the issue still occurs on destroy, is because the destroy function doesn't actually remove any event hooks. This one is a big big big memory leak in my opinion, and a problem in general because every time .selectpicker() is run on an element, even after a 'destroy', the event hooks for scroll and resize are stacked. And stacked. And stacked. It's really, really bad.

Released in v1.13.4!


PackageLicense: Removes PackageLicenseUrl and Adds PackageLicenseFile since PackageLicenseUrl is deprecated

Description
This originated from an 3582 issue where the license was using an Url and not showing up in Nuget.org in the details of the package. PackageLicenseUrl was deprecated and it was recommended to use either PackageLicenseFile or PackageLicenseExpression for immutability reasons. I am providing link to the original issue and discussion.
There was an option to leave LICENSE without extension or rename with .TXT extension. I opted to use the original without the .TXT as I do not see either a PRO or a CON to one over the other, but I have no qualms with either option.
In regard to not using PackageLicenseExpression. I am not really seeing a PRO or a CON on the surface as to using this or not for this particular repo.
Type of change
.NET project file to include PackageLicenseFile and exclude PackageLicenseUrl.
Closing issues
To automatically close an issue: closes #IssueNumber

LGTM

LGTM. Can you run dotnet pack on the project and inspect the package with Nuget Package Explorer to verify the license is inside?

Yes, I already did this and verified. Using all options in the description.

@msftbot wait for @kirankumarkolli to approve


Call Extension for contact

Description
I am using React-Native to handle the call via Twilio.
All works as expected. However, if we click the call from the contact on iPhone, then it doesn't do anything.
I tested a similar app called "TextNow", and it can make the call directly from the contact list of iPhone.
Do you have any suggestions on how we can resolve it?
Steps to Reproduce

Receive the call
The call logs show on the contact
Click the contact

Code
React Native.
I've used React Native twilio programmable voice.
Expected Behavior
It should open the app and make the call
Actual Behavior
It doesn't open anything.
Reproduces How Often
100%
Versions
All relevant version information for the issue.
Voice iOS SDK
React Native.
Xcode
8.3.3
iOS Version
Ios 10
iOS Device
Iphone 6, iPhone 8 plus, iphone X

Hi @v-grant
Thanks for reaching out. Although the quickstart does not demonstrate this function, but I believe if the contact is configured and linked with your iOS app that uses the Voice SDK, you should receive a [CXProviderDelegate provider:performStartCallAction:] callback and you should be able to continue from there and eventually make outbound calls via the SDK API.

Hi, @bobiechen-twilio Thanks for your reply.
I can see the call logs on my iPhone once I received the incoming call. However, if I am going to make the outbound call from the call logs, I can't see my App on the caller's list.
How can I register this Twilio-based app on the caller list on iPhone?

`- (void)provider:(CXProvider *)provider performStartCallAction:(CXStartCallAction *)action {
NSLog(@"provider:performStartCallAction");
self.audioDevice.enabled = NO;
self.audioDevice.block();

[self.callKitProvider reportOutgoingCallWithUUID:action.callUUID startedConnectingAtDate:[NSDate date]];
__weak typeof(self) weakSelf = self;
[self performVoiceCallWithUUID:action.callUUID client:nil completion:^(BOOL success) {
__strong typeof(self) strongSelf = weakSelf;
if (success) {
[strongSelf.callKitProvider reportOutgoingCallWithUUID:action.callUUID connectedAtDate:[NSDate date]];
[action fulfill];
} else {
[action fail];
}
}];
}`
This is the performStartCallAction handler from 'react-native-twilio-programmable-voice'

Hi @v-grant
While I am still trying adding an IntentExtension to the quickstart, perhaps this tutorial could give you the similar idea how to initiate a call from system contacts or call history: https://medium.com/textnowengineering/ios-hands-free-calling-with-siri-and-intents-e05879c47e71.

Okay I think I got this figured out a bit further and am able to initiate a call from the recents (call history):

You need to specify the supported handle types

  CXProviderConfiguration *configuration = ...
  configuration.supportedHandleTypes = [NSSet setWithArray:@[@(CXHandleTypeGeneric), @(CXHandleTypePhoneNumber)]];

Import the Intents framework and implement the delegate method

// AppDelegate.m

@import Intents;

- (BOOL)application:(UIApplication *)application
continueUserActivity:(NSUserActivity *)userActivity
restorationHandler:(void(^)(NSArray<id<UIUserActivityRestoring>> *restorableObjects))restorationHandler {
    INStartAudioCallIntent *callIntent = (INStartAudioCallIntent *)userActivity.interaction.intent;
    NSString *handle = callIntent.contacts[0].personHandle.value;

    // Use the handle to start a call

    return YES;
}

Check out the docs of INStartAudioCallIntent for more information about the contacts object.

Hi @bobiechen-twilio , Thank you very much for your help.
Let me try it on today.


switch to NUnit for unit testing

Hi @kai-zeng, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Kai Zeng). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

LGTM.

LGTM

LGTM


connection duration.

Hi.
Please add a timer for the connection duration.

Hello, use the new version (1.0.8) and check example for how to use the timer.


Compiler: noise reduction for terminating quantum Fourier transforms

What is the expected behavior?
Just submitted a PR to Aqua for the "semiclassical" Fourier transform option, but as its use depends on the FT directly preceding measurement, it could be complimented by having this checked in the compilation pass. I.e. if a circuit has a standard QFT directly preceding a measurement, the compiler could automatically change that QFT into the semiclassical.
This reduces noise effects drastically by replacing quantum two-qubit control gates with a measurement and classical control. See the noise reduction demo here:
SemiclassFTdemo.zip

Not sure if Im fully following. Are you suggesting a transpiler pass?

Ah sorry, yes a transpiler pass. I will be generalizing the circuit in Aqua to inverse and approximate QFTs as well, and I figured adding a transpiler pass in Terra that returns an optimized circuit with the final QFT replaced by the reduced noise version may make sense. If so, should I simply follow the pattern and submit it in a PR with the rest of them in https://github.com/Qiskit/qiskit-terra/tree/master/qiskit/transpiler/passes?

I think this makes more sense in Aqua right now. It's not so easy for the compiler to automatically detect that a (sub-)circuit is a quantum fourier transform. Ultimately when we have better library support for these kinds of things it could be done. But for now I think it's easier for the circuit constructor to request this optimization. Also it's not clear that this optimization is always worth it. It depends on the latency and error of measurement+feed-forward. Current devices don't support feed-forward at all.

That makes a lot of sense, thanks for letting me know!


Roles and Permissions

Is your feature request related to a problem? Please describe.
At this point, there are no defined roles/permissions
Describe the solution you'd like
Create a way to define/check user's role and permissions

Requirements

[ ] Check for privilege
[ ] Check if registered


loot-tracker: make clue scroll drops stackable (#11416)

closes #11416
Fixes the bug where clue scroll variations (excluding beginner & master) were not being stacked in look tracker.

You don't need to include the issue number in your commits. It can get spammy for the issue author everytime you push.


Add cache for get_model_defs to save disk I/O

Use a cache for model definitions to reduce disk I/O when the same model definition is loaded multiple times.

This change results in a ~300-fold increase in speed when loading the same model 1000 times repeatedly.


Attempting to edit a table column redirects to another table

When trying to edit a table column, Superset redirects to another table's definition, followed by the table column listing. This happens across tables, regardless of the database type.
Expected results
Being able to edit the table column.
Actual results
It's redirected and I can't edit.
Screenshots

How to reproduce the bug

Go to a table definition, then columns
Click on the edit button next to a column

Environment
Present
Checklist
Make sure these boxes are checked before submitting your issue - thank you!

[ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[ ] I have reproduced the issue with at least the latest released version of superset.
[ ] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
Add any other context about the problem here.

Thank you Jeremy for reporting, this issue will be prioritized. @benceorlai

This is weird indeed. @jeremy-ecoenergy as a work around you could use the datasource editor in explore to make changes to your table.

This is happening in the "legacy table editor" which we're in the process of deprecating. It's related to an old bug in FAB where is stores the browser history in a session variable. The solution here is to deprecate the legacy table editor.

Explore -> edit datasource works fine for editing columns, but doesn't allow one to add new columns (which is why I'm trying to do). In this case it's a virtual table derived from an SQL query.

Hi @jeremy-ecoenergy
There was a pr that fixed this in Superset 11 days ago. That improvement was made available in the cloud version a few days later so it should be working as expected. Could you confirm that it is now working?

It's not clear how this PR fixes the issue? The columns from the SQL query can't be synced.
I did find a workaround by using an incognito window and the legacy data source editor, but it was still buggy.

I might have misunderstood your explanation, sorry. You are correct, for virtual datasets, the column sync is not working at the moment. There is another PR (PR-10645) pending for that, we are working on it as we speak.
Additionally, this functional area is getting a lot of changes these weeks, the full extent of the work is shared publicly in the Superset roadmap here --> https://github.com/apache-superset/superset-roadmap/projects/1#card-45405983


Puts back protected users features

Generally, the ROC-AUC metric goes up 1-4 points.  All models benefit significantly.


resourceid entity

including resourceid KeyvaultMassSecretRetrieval
Required items, please complete
Change(s):

including field resourceid in KeyvaultMassSecretRetrieval for properly entity mapping

Reason for Change(s):

entity missing

Version Updated:

Required only for Detections/Analytic Rule templates
See guidance below

Testing Completed:

yes

Checked that the validations are passing and have addressed any issues that are present:

See guidance below

Guidance <- remove section before submitting

Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:
Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR.  Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process.  Providing associated Issues that this resolves also easily connects the reason.

Change(s):

Updated syntax for XYZ.yaml

Reason for Change(s):

New schema used for XYZ.yaml
Resolves ISSUE #1234

Version updated:

Yes
Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly.  If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally.
https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:

Guidance for Detection checks
General contribution guidance
PR validation troubleshooting

Hello @cmaneiro, you have added the detection to repo. Please create a solution folder with solution name under Solutions folder.
Please refer this Solution. Solutions/Microsoft Exchange Security - Exchange Online
Also add input file and solution metadata which is used to repackage the solution.
If this is a standalone content move the file to standalone detection folder.
Thanks.

Hello @cmaneiro, you have added the detection to repo. Please create a solution folder with solution name under Solutions folder. Please refer this Solution. Solutions/Microsoft Exchange Security - Exchange Online Also add input file and solution metadata which is used to repackage the solution.
If this is a standalone content move the file to standalone detection folder. Thanks.

hi @v-prasadboke,
This is not a new detection, is a modification to an existent one, is for Azure KeyVault

Hello @cmaneiro, Its a Analytic rule which you have added to the repo. Also this is not a standalone Rule now it has been moved to Solution Azure Key Vault.
You can move you file to this specific folder Solutions/Azure Key Vault/Analytic Rules

Hello @cmaneiro, We are waiting for your reply.

Thank for the commit @cmaneiro, I'll check it and let you know by tomorrow EOD.


Update README.md

Please describe the changes this PR makes and why it should be merged:
Status and versioning classification:

Codecov Report

Merging #106 (aea97e1) into main (0383d41) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #106   +/-   ##
=======================================
  Coverage   98.58%   98.58%
=======================================
  Files          41       41
  Lines        1832     1832
  Branches      330      330
=======================================
  Hits         1806     1806
  Misses         26       26

Flag
Coverage Δ

tagscript
99.58% <ø> (ø)

tagscript-plugin-discord
96.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@all-contributors please add @ShoXcy  for docs


Running tests fails on empty test suite

Was just re-running build that ran successful yesterday. Now after upgrading to navcontainerhelper version <IP_ADDRESS> it fails.
The test is supposed to run a test suite called "UNIT-TESTS". This test suite is currently empty (no tests have been added to this extension yet - I'm just setting up the "template"). With version <IP_ADDRESS> this was not a problem.
I would actually prefer it to fail if the test suite does not exist or is empty. I would just prefer an error message not quite as cryptic as the current:
_2019-11-10T20:31:25.9948927Z Importing Objects from C:\ProgramData\NavContainerHelper\Extensions\SFO-BaseApp-bld\PsTestTool-4\PSTestToolPage.fob (container path)
2019-11-10T20:31:43.7263665Z Objects successfully imported
2019-11-10T20:31:50.0139947Z Creating user manager\containeradministrator as user
2019-11-10T20:32:17.2194986Z Title: PS Test Tool Page Edit - Test Tool
2019-11-10T20:32:17.2212039Z Title: {0001FD69-0000-0000-0108-0000836BD2D2}
2019-11-10T20:32:17.2612301Z [
2019-11-10T20:32:17.2618672Z     {
2019-11-10T20:32:17.2624331Z         "type":  "ClientGroupControl",
2019-11-10T20:32:17.2630488Z         "mappingHint":  "TOOLBAR",
2019-11-10T20:32:17.2636834Z         "name":  "CommandBar",
2019-11-10T20:32:17.2641889Z         "children":  [
2019-11-10T20:32:17.2648265Z                          {
2019-11-10T20:32:17.2653652Z                              "type":  "ClientGroupControl",
2019-11-10T20:32:17.2659177Z                              "mappingHint":  "",
2019-11-10T20:32:17.2664620Z                              "name":  "Control_Home",
2019-11-10T20:32:17.2670160Z                              "children":  [
2019-11-10T20:32:17.2677173Z                                               {
2019-11-10T20:32:17.2680303Z                                                   "type":  "ClientGroupControl",
2019-11-10T20:32:17.2685820Z                                                   "mappingHint":  "",
2019-11-10T20:32:17.2690999Z                                                   "name":  "Control_Manage",
2019-11-10T20:32:17.2698208Z                                                   "children":  [
2019-11-10T20:32:17.2703943Z                                                                    {
2019-11-10T20:32:17.2709402Z                                                                        "caption":  "Delete",
2019-11-10T20:32:17.2714418Z                                                                        "type":  "ClientActionControl",
2019-11-10T20:32:17.2720256Z                                                                        "name":  "Control_Delete",
2019-11-10T20:32:17.2725424Z                                                                        "identifier":  "{0FCE965D-436A-418c-B11C-5D6EC3B94F3A}"
2019-11-10T20:32:17.2731060Z                                                                    }
2019-11-10T20:32:17.2736084Z                                                                ],
2019-11-10T20:32:17.2744640Z                                                   "caption":  "Manage",
2019-11-10T20:32:17.2752500Z                                                   "identifier":  "{37F878F7-1EB4-4B40-A6D5-55E87F26029B}"
2019-11-10T20:32:17.2761232Z                                               },
2019-11-10T20:32:17.2770378Z                                               {
2019-11-10T20:32:17.2776132Z                                                   "type":  "ClientGroupControl",
2019-11-10T20:32:17.2782307Z                                                   "mappingHint":  "",
2019-11-10T20:32:17.2794889Z                                                   "name":  "Process",
2019-11-10T20:32:17.2801647Z                                                   "children":  [
2019-11-10T20:32:17.2806544Z                                                                    {
2019-11-10T20:32:17.2810883Z                                                                        "caption":  "Run \u0026Selected",
2019-11-10T20:32:17.2819787Z                                                                        "type":  "ClientActionControl",
2019-11-10T20:32:17.2827370Z                                                                        "name":  "RunSelected",
2019-11-10T20:32:17.2836194Z                                                                        "identifier":  "{0001FD69-0000-000A-0008-0000836BD2D2}"
2019-11-10T20:32:17.2844452Z                                                                    }
2019-11-10T20:32:17.2849375Z                                                                ],
2019-11-10T20:32:17.2856329Z                                                   "caption":  "Process",
2019-11-10T20:32:17.2861743Z                                                   "identifier":  "{47F878F7-1EB4-4B40-A6D5-55E87F26029B}"
2019-11-10T20:32:17.2868278Z                                               },
2019-11-10T20:32:17.2873202Z                                               {
2019-11-10T20:32:17.2879699Z                                                   "type":  "ClientGroupControl",
2019-11-10T20:32:17.2884158Z                                                   "mappingHint":  "",
2019-11-10T20:32:17.2890698Z                                                   "name":  "Page",
2019-11-10T20:32:17.2895049Z                                                   "children":  [
2019-11-10T20:32:17.2937375Z                                                                    {
2019-11-10T20:32:17.2938062Z                                                                        "caption":  "Refresh",
2019-11-10T20:32:17.2938435Z                                                                        "type":  "ClientActionControl",
2019-11-10T20:32:17.2938761Z                                                                        "name":  "Control_Refresh",
2019-11-10T20:32:17.2939073Z                                                                        "identifier":  "{57b9dd2f-e826-4d69-a433-34f779a5552d}"
2019-11-10T20:32:17.2939391Z                                                                    }
2019-11-10T20:32:17.2939743Z                                                                ],
2019-11-10T20:32:17.2940141Z                                                   "caption":  "Page",
2019-11-10T20:32:17.2940458Z                                                   "identifier":  "{CDBA910D-E283-4A38-AD03-C9D42763C3CE}"
2019-11-10T20:32:17.2943245Z                                               }
2019-11-10T20:32:17.2949629Z                                           ],
2019-11-10T20:32:17.2953954Z                              "caption":  "Home",
2019-11-10T20:32:17.2961330Z                              "identifier":  "{17F878F7-1EB4-4B40-A6D5-55E87F26029B}"
2019-11-10T20:32:17.2968023Z                          },
2019-11-10T20:32:17.2976032Z                          {
2019-11-10T20:32:17.2980047Z                              "type":  "ClientGroupControl",
2019-11-10T20:32:17.2989493Z                              "mappingHint":  "",
2019-11-10T20:32:17.2992812Z                              "name":  "Control_Actions",
2019-11-10T20:32:17.2998547Z                              "children":  [
2019-11-10T20:32:17.3004606Z                                               {
2019-11-10T20:32:17.3009937Z                                                   "caption":  "Run \u0026Selected",
2019-11-10T20:32:17.3015404Z                                                   "type":  "ClientActionControl",
2019-11-10T20:32:17.3021772Z                                                   "name":  "RunSelected",
2019-11-10T20:32:17.3027036Z                                                   "identifier":  "{0001FD69-0000-000A-0008-0000836BD2D2}"
2019-11-10T20:32:17.3033668Z                                               }
2019-11-10T20:32:17.3039515Z                                           ],
2019-11-10T20:32:17.3046921Z                              "caption":  "\u0026Actions",
2019-11-10T20:32:17.3053105Z                              "identifier":  "{67ff59f0-c388-4aab-8026-104dc2a9f53d}"
2019-11-10T20:32:17.3058624Z                          },
2019-11-10T20:32:17.3065158Z                          {
2019-11-10T20:32:17.3070952Z                              "type":  "ClientGroupControl",
2019-11-10T20:32:17.3077578Z                              "mappingHint":  "",
2019-11-10T20:32:17.3083155Z                              "name":  "ViewsGroup",
2019-11-10T20:32:17.3088658Z                              "children":  [
2019-11-10T20:32:17.3092422Z
2019-11-10T20:32:17.3099001Z                                           ],
2019-11-10T20:32:17.3104685Z                              "caption":  "Views",
2019-11-10T20:32:17.3110152Z                              "identifier":  ""
2019-11-10T20:32:17.3115340Z                          },
2019-11-10T20:32:17.3120948Z                          {
2019-11-10T20:32:17.3126276Z                              "type":  "ClientGroupControl",
2019-11-10T20:32:17.3131720Z                              "mappingHint":  "CommandBarHelpGroup",
2019-11-10T20:32:17.3137317Z                              "name":  "HelpGroup",
2019-11-10T20:32:17.3142627Z                              "children":  [
2019-11-10T20:32:17.3145332Z
2019-11-10T20:32:17.3151104Z                                           ],
2019-11-10T20:32:17.3156166Z                              "caption":  "",
2019-11-10T20:32:17.3161536Z                              "identifier":  ""
2019-11-10T20:32:17.3166883Z                          },
2019-11-10T20:32:17.3171580Z                          {
2019-11-10T20:32:17.3177094Z                              "type":  "ClientGroupControl",
2019-11-10T20:32:17.3182288Z                              "mappingHint":  "CommandBarLayoutGroup",
2019-11-10T20:32:17.3187784Z                              "name":  "LayoutGroup",
2019-11-10T20:32:17.3192880Z                              "children":  [
2019-11-10T20:32:17.3195555Z
2019-11-10T20:32:17.3201072Z                                           ],
2019-11-10T20:32:17.3206118Z                              "caption":  "",
2019-11-10T20:32:17.3211509Z                              "identifier":  ""
2019-11-10T20:32:17.3216685Z                          },
2019-11-10T20:32:17.3221913Z                          {
2019-11-10T20:32:17.3227051Z                              "type":  "ClientStaticStringControl",
2019-11-10T20:32:17.3232922Z                              "name":  "SystemIndicator",
2019-11-10T20:32:17.3238321Z                              "value":  "",
2019-11-10T20:32:17.3243432Z                              "identifier":  ""
2019-11-10T20:32:17.3248833Z                          },
2019-11-10T20:32:17.3253895Z                          {
2019-11-10T20:32:17.3259286Z                              "type":  "ClientGroupControl",
2019-11-10T20:32:17.3264263Z                              "mappingHint":  "",
2019-11-10T20:32:17.3269471Z                              "name":  "Control_Microsoft_Dynamics_NAV",
2019-11-10T20:32:17.3274419Z                              "children":  [
2019-11-10T20:32:17.3279590Z                                               {
2019-11-10T20:32:17.3284983Z                                                   "type":  "ClientGroupControl",
2019-11-10T20:32:17.3289811Z                                                   "mappingHint":  "",
2019-11-10T20:32:17.3294769Z                                                   "name":  "Page",
2019-11-10T20:32:17.3299953Z                                                   "children":  [
2019-11-10T20:32:17.3302476Z
2019-11-10T20:32:17.3307544Z                                                                ],
2019-11-10T20:32:17.3312751Z                                                   "caption":  "Page",
2019-11-10T20:32:17.3317838Z                                                   "identifier":  "{CDAB913B-E283-4A38-AD03-C9D42763C1CA}"
2019-11-10T20:32:17.3323547Z                                               },
2019-11-10T20:32:17.3328691Z                                               {
2019-11-10T20:32:17.3333946Z                                                   "type":  "ClientGroupControl",
2019-11-10T20:32:17.3339393Z                                                   "mappingHint":  "",
2019-11-10T20:32:17.3344662Z                                                   "name":  "Control_Send_To",
2019-11-10T20:32:17.3350001Z                                                   "children":  [
2019-11-10T20:32:17.3352666Z
2019-11-10T20:32:17.3357973Z                                                                ],
2019-11-10T20:32:17.3362934Z                                                   "caption":  "Print \u0026 Send",
2019-11-10T20:32:17.3368391Z                                                   "identifier":  "{CDBB910B-E283-4A38-AD03-C9D42763C1CE}"
2019-11-10T20:32:17.3373688Z                                               }
2019-11-10T20:32:17.3378856Z                                           ],
2019-11-10T20:32:17.3515629Z                              "caption":  "Dynamics 365 Business Central",
2019-11-10T20:32:17.3519481Z                              "identifier":  "{d7c9f73f-9844-47ff-b4f4-88a77eb4e103}"
2019-11-10T20:32:17.3520401Z                          }
2019-11-10T20:32:17.3520746Z                      ],
2019-11-10T20:32:17.3520961Z         "caption":  "",
2019-11-10T20:32:17.3521222Z         "identifier":  "{FB8CCCF3-54AC-4d77-9AD8-136CEC25583A}"
2019-11-10T20:32:17.3521553Z     },
2019-11-10T20:32:17.3521805Z     {
2019-11-10T20:32:17.3522063Z         "identifier":  "{886D4A84-3808-4545-92CF-374593FBC32F}",
2019-11-10T20:32:17.3522177Z         "name":  "FilterControl",
2019-11-10T20:32:17.3522479Z         "type":  "ClientFilterLogicalControl"
2019-11-10T20:32:17.3522576Z     },
2019-11-10T20:32:17.3522880Z     {
2019-11-10T20:32:17.3522979Z         "type":  "ClientStringControl",
2019-11-10T20:32:17.3523354Z         "name":  "CurrentSuiteName",
2019-11-10T20:32:17.3523568Z         "value":  "UNIT-TESTS",
2019-11-10T20:32:17.3523784Z         "identifier":  "{0001FD69-0000-001E-0008-0000836BD2D2}"
2019-11-10T20:32:17.3523995Z     },
2019-11-10T20:32:17.3524189Z     {
2019-11-10T20:32:17.3524391Z         "Control1":  [
2019-11-10T20:32:17.3524683Z
2019-11-10T20:32:17.3524793Z                      ],
2019-11-10T20:32:17.3525024Z         "type":  "ClientRepeaterControl",
2019-11-10T20:32:17.3525266Z         "name":  "Control1",
2019-11-10T20:32:17.3525479Z         "identifier":  "{0001FD69-0000-0001-0008-0000836BD2D2}"
2019-11-10T20:32:17.3525691Z     },
2019-11-10T20:32:17.3525889Z     {
2019-11-10T20:32:17.3526100Z         "type":  "ClientGroupControl",
2019-11-10T20:32:17.3526274Z         "mappingHint":  "ACTIONBAR",
2019-11-10T20:32:17.3526595Z         "name":  "ActionBar",
2019-11-10T20:32:17.3526748Z         "children":  [
2019-11-10T20:32:17.3526839Z                          {
2019-11-10T20:32:17.3526984Z                              "caption":  "OK",
2019-11-10T20:32:17.3527067Z                              "type":  "ClientActionControl",
2019-11-10T20:32:17.3527175Z                              "name":  "{B594146A-9CEF-4767-AE76-E9721CD6FB50}",
2019-11-10T20:32:17.3527688Z                              "identifier":  "{B594146A-9CEF-4767-AE76-E9721CD6FB50}"
2019-11-10T20:32:17.3529140Z                          }
2019-11-10T20:32:17.3534204Z                      ],
2019-11-10T20:32:17.3539878Z         "caption":  "",
2019-11-10T20:32:17.3545281Z         "identifier":  "{EC1DA4B5-D7A8-4091-A7B3-2DB6107B6C56}"
2019-11-10T20:32:17.3550858Z     }
2019-11-10T20:32:17.3556041Z ]
2019-11-10T20:32:20.4065439Z Validation errors occured. Error is: {
2019-11-10T20:32:20.4065763Z     "Severity":  0,
2019-11-10T20:32:20.4089114Z     "Description":  "The CAL Test Suite does not exist. Identification fields and values: Name=\u0027UNIT-TESTS\u0027",
2019-11-10T20:32:20.4092273Z     "DescriptionShort":  null,
2019-11-10T20:32:20.4095307Z     "IsLocal":  false,
2019-11-10T20:32:20.4096517Z     "OriginatingControlId":  null
2019-11-10T20:32:20.4096897Z }
2019-11-10T20:32:20.4097141Z at Run-Tests, C:\ProgramData\NavContainerHelper\Extensions\SFO-BaseApp-bld\PsTestTool-4\PsTestFunctions.ps1: line 326
2019-11-10T20:32:20.4097415Z at , : line 49
2019-11-10T20:32:20.4097667Z at Run-Tests, C:\ProgramData\NavContainerHelper\Extensions\SFO-BaseApp-bld\PsTestTool-4\PsTestFunctions.ps1: line 326
2019-11-10T20:32:20.4097919Z at , : line 49
2019-11-10T20:32:20.4099975Z ##[error]Validation errors occured. Error is: {
"Severity":  0,
"Description":  "The CAL Test Suite does not exist. Identification fields and values: Name=\u0027UNIT-TESTS\u0027",
"DescriptionShort":  null,
"IsLocal":  false,

"OriginatingControlId":  null

}
At C:\Program Files\WindowsPowerShell\Modules\navcontainerhelper\<IP_ADDRESS>\AppHandling\Run-TestsInNavContainer.ps1:374 c
har:17

            throw $_.Exception.Message

            ~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : OperationStopped: (Validation erro...olId":  null

}:String) [], RuntimeException
+ FullyQualifiedErrorId : Validation errors occured. Error is: {

"Severity":  0,

    "Description":  "The CAL Test Suite does not exist. Identification fields and values: Name=\u0027UNIT-TESTS\u0

027",
"DescriptionShort":  null,

"IsLocal":  false,

"OriginatingControlId":  null

}
2019-11-10T20:32:20.4101185Z ##[error]PowerShell exited with code '1'.
2019-11-10T20:32:20.4144745Z ##[section]Finishing: Run Tests in Container_

The difference between .17 and .18 is that .18 fails if there is a validation error on the test tool page.
Reading through the message above, that is what happened.
Not failing causes infinite loops or could cause tests to report prior results again.
Could you share the command used? + what container version you are using?
and you say that the test app is empty?

Yes, the test suite I'm trying to run is empty! Which was fine Saturday... I know I properbly should test if the container returns any tests, before trying to run the tests.
The image is onprem:14.0.29537.32323-dk.
The command we run:
Write-Host "Running test suite tests."
if (-not $TestSuites) {
$TestSuites = $SandboxType.TestSuitesExecuted
}
Foreach ($TestSuite in $TestSuites) {
$Success = Run-TestsInNavContainer                     -AzureDevOps Error
-containerName $SandboxType.containerName                     -XUnitResultFileName $TestResultsFile
-testSuite $TestSuite                     -verbose
-detailed                     -credential $credential
-returnTrueIfAllPassed `
-debugMode
if ($Success) {
Write-Host "Test suite $TestSuite ran succesfully. Check the test results file $TestResultsFile."
} else {
Write-Error "Test suite $TestSuite did not run succesfully. Check the test results file $TestResultsFile."
}
}

Thinking about it, this should fail.
If you did not have debugmode on, you would have seen this error:
##[error]Validation errors occured. Error is: {
"Severity": 0,
"Description": "The CAL Test Suite does not exist. Identification fields and values: Name=\u0027UNIT-TESTS\u0027",
Which shows the problem - there is no test suite.
Trying to execute tests and finding no test suite probably shouldn't return all is ok, you might end up in situations where your test app doesn't publish and everything is ok.
Remember, that it is perfectly ok to specify a specific version of containerhelper in your pipeline and grab the newest weekly or monthly in a controlled environment.
I do my best to not break anything (new functionality is almost always hidden behind parameters), but in situations like this, where something returns OK where it should return error - I shouldn't do that.

I agree it shoud fail and I've updated out scripts, so that it will not call the run tests. :)


[BUG] Z-axis disagreement for stacks taller than they are X or Y

Describe the bug
Using the GUI, trying to run a model on a z-stack that has more frames than it has pixels (e.g. 266 frames of 200x200px). The model behaves fine on its own, and the GUI is otherwise treating the z-stack just fine, but the drawing the masks results in ERROR: index 200 is out of bounds for axis 0 with size 200 from somewhere inside the if branch at cellpose/gui/io.py:505. I expect this has to do with the decision on cellpose/gui/io.py:169 that chooses the smallest dimension to be Z, but I was unable to figure out how to carry the gui's choice of Z into this layer of the code properly.
To Reproduce
Steps to reproduce the behavior:

Load a z-stack with more frames than edge pixels
Click on 'Run Cyto3'
See error in console

Run log
...
2024-09-25 14:57:23,102 [INFO] 44 cells found with model in 8.088 sec
GUI_INFO: 44 masks found
GUI_INFO: plane 0 outlines processed
GUI_INFO: plane 50 outlines processed
GUI_INFO: plane 100 outlines processed
GUI_INFO: plane 150 outlines processed
GUI_INFO: plane 200 outlines processed
ERROR: index 200 is out of bounds for axis 0 with size 200

thanks for noticing this, it's now fixed


[Question] How to properly register new metals

I'm playing a modpack with loads of metals (alumium, nickel, tin, osmium,...) and I would like an easy way to register them.

hi, molten metals are hard-coded and they can't be registered by modpack developers

:(
maybe a new feature suggestion?

I'm planning on supporting metals from other mods in the future


Add ObservedTimestamp to entry

Resolves #367
This is a breaking change because entry.Timestamp is no longer initialized by default.

Just to confirm: any existing parsing operation will continue to write to Timestamp. If there is no parsing than the Timestamp will remain empty. ObservedTimestamp will be always initialized to now and is not affected by parsers. Is this correct?

That's exactly correct.


UI Improvement

Issue Description:
The "UI Improvement" issue aims to enhance the user interface (UI) of our project to make it more intuitive, visually appealing, and user-friendly. This could involve various aspects such as layout adjustments, color schemes, typography, and overall design enhancements.
Requirements:

Analyze the current UI and identify areas for improvement.
Implement design changes that enhance the user experience and align with the project's goals and aesthetics.
Ensure responsiveness across different devices and screen sizes. min 320px - max 1440px

If you have any further questions or need more information, feel free to ask. We appreciate your interest in contributing to the UI improvement, and your contributions will be invaluable in making our project better.

Hi @ajaynegi45,
I would like to contribute towards this issue

Hi @ajaynegi45, I would like to contribute towards this issue

Hi @jaisheel15,
Thank you for expressing your interest in working on this issue. I'm delighted to inform you that I have assigned this issue to you. Your willingness to contribute to our project is much appreciated.
Design Resources:

Follow this Figma Design

Feel free to start working, and if you have any questions or need assistance during the process, please don't hesitate to reach out.

Acc to the figma designs you provided should I make a login page

Acc to the figma designs you provided should I have to make a login page

you can

Acc to the figma designs you provided should I have to make a login page

you can

I have to only make the ui , right.

Acc to the figma designs you provided should I have to make a login page

you can

I have to only make the ui , right.

Yes


Target labels in probe options

Copied from https://github.com/google/cloudprober/issues/652, originally filed by @xenofree.
Hi,
Is it possible or will it be possible to configure probe option through file_target like this ?
resource {
  name: "10.x.x.x"
  labels {
     key: "hostname"
     value: "xxxxxxxx"
  }
  labels {
     key: "query_type"
     value: A
  }
  labels {
     key: "resolved_domain"
     value: "xxx.xxxxxx.com"
  }
}

probe {
  name: "dns"
  type: DNS
  targets {
    file_targets {
      file_path: "/data/cloudprober/config.d/dns/targets.yml"
      re_eval_sec: 60
    }
  }
  additional_label {
      key: "hostname"
      value: "@target.label.hostname@"
  }
  additional_label {
      key: "resolved_domain"
      value: "@target.label.resolved_domain@"
  }
  dns_probe {
    query_type: @target.label.query_type@
    resolved_domain: "@target.label.resolved_domain@"
  }
  interval_msec: 60000  # 60s
  timeout_msec: 10000   # 10s
}

My comment:
It's going to be hard to implement a generic label replacement mechanism.
The way it works in HTTP probe is:
If "relative_url" or "host" is not defined in the probe config, and target has these labels, it automatically uses these labels.
We can do something similar for DNS probe.

If "relative_url" or "host" is not defined in the probe config

Correction - it looks like fqdn is the label which the HTTP probe inspects:
https://github.com/cloudprober/cloudprober/blob/75b20eb82be1ec143f230804889582dd1087a126/probes/http/request.go#L60-L69
e.g
    resource {
      name: "google_homepage"
      labels {
        key: "fqdn"
        value: "www.google.com"
      }
    }

Probing configuration through target labels is now documented here:
https://cloudprober.org/docs/how-to/targets/#probe-configuration-through-target-fields


Unable to resolve @Test or any other anotation

TestNG Version
7.0.0-Beta4
Expected behavior
Upon adding @Test, it shouldn't result into error
Actual behavior
Test cannot be resolved to a Type.
Also unable to find org.testng.annotation in testng package
Is the issue reproductible on runner?
Yes

[ ] Shell
[x] Maven
[ ] Gradle
[ ] Ant
[ ] Eclipse
[ ] IntelliJ
[ ] NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

Please make use of TestNG 7.0.0-beta5. beta4 was a bad build.
since beta5 is not yet available on Maven central, you may have to add a repository section that looks like below to your pom.xml (or) to your ~/.m2/settings.xml
                <repository>
                    <id>jcenter</id>
                    <name>jcenter</name>
                    <url>http://jcenter.bintray.com</url>
                </repository>


Can't filter on array of arrays

Hello guys,
I have a JSON like this :
{ 	"_source": { 		"Asset": [ 			{ 				"Ref": [ 					{ 						"string": [ 							"1358420" 						], 						"@field": "REF" 					} 				], 				"ImageType": [ 					[ 						{ 							"string": "Image", 							"@code": "ImageType-DAM_MTYP_IMAGE", 							"@field": "IMAGE_TYPE" 						} 					], 					[ 						{ 							"string": "Packshot", 							"@code": "ImageType-DAM_MTYP_3014", 							"@field": "IMAGE_TYPE" 						} 					] 				] 			},	 			{ 				"Ref": [ 					{ 						"string": [ 							"896_1" 						], 						"@field": "REF" 					} 				], 				"ImageType": [ 					[ 						{ 							"string": "Product Photo", 							"@code": "ImageType-DAM_MTYP_3011", 							"@field": "IMAGE_TYPE" 						} 					], 					[ 						{ 							"string": "Packshot", 							"@code": "ImageType-DAM_MTYP_3014", 							"@field": "IMAGE_TYPE" 						} 					] 				] 			} 		] 	} }

I have to filter the assets on Ref and ImageType code.
I'm using this filter for the Ref :
Filter refFilter = Filter.filter(where("Ref[0].string[0]").is("1358420"));
parse(json).read("_source.Asset[?]", refFilter);
This is working.
Now for the ImageType code :
Filter imageTypeFilter = Filter.filter(where("ImageType[0][0].@code").is("ImageType-DAM_MTYP_IMAGE"));
This is working.
However,
Filter imageTypeFilter = Filter.filter(where("ImageType[0][1].@code").is("ImageType-DAM_MTYP_3014"));
This doesn't work.
Can you tell me why please ? Is it a problem in my code or is there a bug ?
Thank you in advance for your answer. I'm struggling with this for hours...
Regards

Could you please modify the JSON above such that it's valid? Right now it's invalid, e.g. because of the trailing commas at the end.

Done. Sorry I tried to simplify it at the maximum. The original one is more than 11000 lines !

I simplified it a bit more. This is similar to the object in your "Asset" selected by the first filter.
[
   {
      "foo" : [
         [
            {
               "prop" : "a"
            }
         ],
         [
            {
               "prop" : "b",
              "i want": "this"
            }
         ]
      ]
   },
   {
      "foo" : [
         [
            {
               "prop" : "c"
            }
         ],
         [
            {
               "prop" : "d",
               "not": "this"
            }
         ]
      ]
   }
]

This works:
$..[?(@.foo[0][0]['prop']=='a')]

This doesn't:
$..[?(@.foo[0][1]['prop']=='b')]

Yes, this is exactly my problem. Does this mean there is a bug ?

Guess it’s a bug. I already encountered a similar problem in #472

I assume it won't be fixed in the next release ?

@sbrooymans in case you're not following the issues in this repo: There hasn't been a release recently but in #500 someone said there was a release in Q1 this year. But since there's no pull request on this specific issue so far, I doubt it's gonna be fixed in the near future.

Regardless of the JsonPath implementation and limitations, in these cases the appropriate indexes are [1][0] and not [0][1] for what you are searching for.


What am I doing wrong?

I installed the package but I keep getting this error message.  I cannot initiate the program.
C:\WINDOWS\system32>$ pip install pydfs-lineup-optimizer
'$' is not recognized as an internal or external command,
operable program or batch file.

we use pyhton3 so usev
pip3 install pydfs-lineup-optimizer

Ok I am now getting this error for running draftkings golf lineups.
1 optimizer = get_optimizer(Site.DRAFTKINGS, Sport.GOLF)
----> 2 optimizer.load_players_from_csv("DKSalaries.csv")
3 for lineup in optimizer.optimize(n=10):
4     print(lineup)
5     print(lineup.players)  # list of players
~\anaconda3\lib\site-packages\pydfs_lineup_optimizer\lineup_optimizer.py in load_players_from_csv(self, filename)
136         if not csv_importer:
137             csv_importer = self._settings.csv_importer
--> 138         self._players = csv_importer(filename).import_players()
139
140     def load_lineups_from_csv(self, filename: str) -> List[Lineup]:
~\anaconda3\lib\site-packages\pydfs_lineup_optimizer\sites\draftkings\classic\importer.py in import_players(self)
55
56     def import_players(self):
---> 57         with open(self.filename, 'r') as csv_file:
58             start_line = 0  # Find line with 'TeamAbbrev', that's where players data starts
59             while True:
FileNotFoundError: [Errno 2] No such file or directory: 'DKSalaries.csv'

you didnt download your draftkings player list?

you have to point this to your file
----> 2 optimizer.load_players_from_csv("DKSalaries.csv")

Got it to work thanks.  How do i do CPT mode?