RegexSet - enhancement

I tried using RegexSet and I noticed that for my specific usage the performance was significantly slower than merging all my regex into one as in regex1|regex2|...|regexN.
I do know that when you combine it into one regex you lose the ability to get all matches without iterating it.
From my experience, in many applications that rely heavily on MANY regexp, for example a log processing, the match rate is usually very low (most messages are uninteresting info vs. error, as an example) thus this trade off is worth it.
Although it's not really a trade off, since when you find a matching line, you can run the existing RegexSet on it and get all the required info.
There is another caveat which can be resolved easily, named groups cannot be duplicated.
I workaround this before by taking any user input regex and doing two changes to it:

Put the entire regex in a (?P<GROUP_NUM>regexp).
Rename each group inside the user regexp, for example if regexp number 5 is (?P<Name>.*), it will be rewritten as (?P<GROUP_5>(?P<GROUP_5_NAME>.*))`.

I suggest this as an extra mode to enhance the existing RegexSet, I'll be happy to work on implementing it if the maintainers think it's worthwhile.

Sorry, but I don't understand what you're proposing. Can you make it more concrete in terms of what APIs you would add?

API wise it basically one optional argument in the constructor. For example:
RegexSet::new(&[
    r"[a-z]+@[a-z]+\.(com|org|net)",
    r"[a-z]+\.(com|org|net)",
    OR_MATCH_MODE
]).unwrap();

The main point is changing the matching algorithm to use the one merged regex instead of the existing flow.
(obviously only under this mode)

A very basic draft of this, you can run with run_regexp or run_regexp_array, I tested on a large file, seeing significant speed differences (depending on how many on the type of regex).
use regex::Regex;
use regex::RegexSet;
use std::fs::File;
use std::io::{self, BufRead};
use std::path::Path;
use std::time::Instant;

fn read_lines<P>(filename: P) -> io::Result<io::Lines<io::BufReader<File>>>
where
    P: AsRef<Path>,
{
    let file = File::open(filename)?;
    Ok(io::BufReader::new(file).lines())
}

fn read_eme(filename: &str) -> Vec<String> {
    let mut regexp_array = Vec::new();
    if let Ok(lines) = read_lines(filename) {
        for line in lines {
            if let Ok(ip) = line {
                let _re = Regex::new(&ip).unwrap();
                regexp_array.push(ip.clone());
            }
        }
    }
    regexp_array
}

fn read_eme2(filename: &str) -> String {
    let mut count = 0;
    let mut regexp = String::new();
    if let Ok(lines) = read_lines(filename) {
        for line in lines {
            if let Ok(ip) = line {
                if count > 0 {
                    regexp.push('|');
                }
                regexp.push_str(&format!("(?P<GROUP_{}>{})", count, &ip));
                count += 1;
            }
        }
    }
    regexp
}

fn run_regexp() {
    let regexp_str = read_eme2("./patterns.eme");
    println!("regexp_str: '{:?}'", regexp_str);
    let regex = Regex::new(&regexp_str).unwrap();

    let mut count = 0;
    let mut t0 = Instant::now();

    if let Ok(lines) = read_lines("./test_100.log") {
        for line in lines {
            if count % 10000 == 0 {
                let t1 = Instant::now();
                println!("Line {:?} , {:?}, {:?}", &line, count, t1 - t0);
                t0 = t1;
            }
            if let Ok(ip) = line {
                for cap in regex.captures_iter(&ip) {
                    println!("cap: '{:?}', line: '{:?}'", cap, ip);
                }
            }
            count += 1;
        }
    }
}

fn run_regexp_array() {
    let regexp_array = read_eme("./patterns.eme");
    let regexp_set = RegexSet::new(&regexp_array).unwrap();
    let mut count = 0;
    let mut t0 = Instant::now();

    if let Ok(lines) = read_lines("./test.log") {
        for line in lines {
            if count % 10000 == 0 {
                let t1 = Instant::now();
                println!("Line {:?} , {:?}, {:?}", &line, count, t1 - t0);
                t0 = t1;
            }
            if let Ok(ip) = line {
                let matches: Vec<_> = regexp_set.matches(&ip).into_iter().collect();
                for m in matches {
                    println!("m: '{:?}', pattern: '{:?}', line: '{:?}'", m, regexp_array[m], ip);
                }
            }
            count += 1;
        }
    }
}

fn main() {
    run_regexp();
    //run_regexp_array();
}

I don't think API changes are the right path here. If there's a performance difference then we should fix that. I believe there are tickets for that.

I agree about the API changes.
I'll work on a PR with something more solid for you to provide feedback on (due note I'm not a Rust expert so I expect much criticism on my code).
Also found some related issue https://github.com/rust-lang/regex/issues/259

OK, but it would probably be better to discuss what specifically you'll be doing in the PR if possible. Even if it's just a sketch. I may not merge it based purely on its design, for example.

I think having to preprocess the capture group names on the caller side is probably the right path here. I don't think complicating the regex crate API is the right path here.


Automatic TLDs update for Zend\Validator\Hostname

Hi,
I improve the update script provided in #7303 :

in bin/ directory
update $validTlds definition in file library/Zend/Validator/Hostname.php
several errors detection

usage :  php ./bin/update_hostname_validator.php
Is the script name ok ?
Patrick

My only reservation is that his only does new TLDs, there is no mechanism if a gTLD is removed from the list. Might it be better to generate the entire list and simply replace $validTLDs? Of course if you want to spend the time, you could also look into the Zend\Code\Generator classes and see of that might help.

A ha! I found what I was looking for. The Root Zone Database is also available as a daily-generated text file, though only with A names - https://data.iana.org/TLD/tlds-alpha-by-domain.txt
Which would be best to pull from? The Validator class does have its home-grown decodePunycode() function that could be used if idn_to_utf8() was not available. It would help remove the dependance on checking for specific CSS to parse TLDs from an HTML file.

Hello,

My only reservation is that this script only inserts new TLDs, there is no mechanism if a gTLD is
removed from the list. Might it be better to generate the entire list and simply replace $validTLDs? Of
course if you want to spend the time, you could also look into the Zend\Code\Generator classes and
see of that might help.

I don't make a diff.
I put the new list in place.
Zend\Code has so less document. Can't figure out how to use it.
Usage Zend\Code\Generator : don't know how to parse an existing class, and if the generated class will respect zf2 coding style... that's why I made a "replace" algorithme.
Patrick

A ha! I found what I was looking for. The Root Zone Database is also available as a daily-generated
text file, though only with A names - https://data.iana.org/TLD/tlds-alpha-by-domain.txt

Very nice !

Which would be best to pull from? The Validator class does have its home-grown decodePunycode()
function that could be used if idn_to_utf8() was not available. It would help remove the dependance
on checking for specific CSS to parse TLDs from an HTML file.

I will push a new PR using this method.
Patrick

@moderndeveloperllc
I update the PR, with usage of https://data.iana.org/TLD/tlds-alpha-by-domain.txt.
The good news is that we don't take into account :

'test' like .テスト
'country-code not assigned' TLD like .bl.

BTW, I don't understand why .oracle is present in 'text-version' of the database, but not in the 'html-version' :-/
Besides, I use Zend\Validator\Hostname::decodePunycode() is idn_to_utf8() is not present. However, I have to use reflexion to make this method accessible :-/
Patrick

Sorry about the code generation misunderstanding. It was too late last night when I was trying to mentally parse your code! I remember when I was thinking about rewriting the validator last year I too noticed that the text list had more than the HTML list. The .oracle domain definitely exists, though I'm not sure anyone can actually buy one. It looks to be a vanity gTLD that's owned by Oracle.
I imagine that @weierophinney will want you to write a unit test for this.

@moderndeveloperllc
I don't see what to unittest, as this is a script you run before a release, and you see modifications made by commit Hostname validator. BTW, where do I put unit test ?
Otherwise, I have break a testcase :
1) ZendTest\Validator\HostnameTest::testAdditionalUTF8TLDs
The input appears to be a DNS hostname but cannot match TLD against known list
The input does not appear to be a valid URI hostname
The input does not appear to be a valid local network name - test123.δοκιμή
Failed asserting that false matches expected true.
/home/travis/build/zendframework/zf2/tests/ZendTest/Validator/HostnameTest.php:504

For me, test123.δοκιμή is not a valid Hostname, as δοκιμή is not listed on https://data.iana.org/TLD/tlds-alpha-by-domain.txt.
However, δοκιμή is listed on http://www.iana.org/domains/root/db as type = "test" and Sponsoring organisation = "Internet Assigned Numbers Authority".
What do you think ?
Is a 'test' TLD valid ??
Patrick

I would say that is not a valid TLD. The root domain database page for that TLD indicates, "This domain is not present in the root zone at this time."

FYI, The last Tracis failed build is not due to my PR
Legend: ?-unknown, I-invalid file syntax, file ignored, .-no changes, F-fixed, E-error
   1) ServiceManager.php (empty_return)
      ---------- begin diff ----------
      --- Original
      +++ New
      @@ @@

      -        return null;
      +        return;
           }


fix issue 18937 - [REG 2.080.0] std.experimental.allocator: compiling…

… make needs an unreasonable amount of memory for structs that contain static arrays

Adding this PR to the test case from issue 18937:
for dmd in dmd2.079 dmd2.080 dmd2.git; do for v in -version=small -version=large -version=vlarg; do /usr/bin/time -f "$dmd $v: time: %E, memory: %M KiB" $dmd -c -unittest $v test.d; done; done:
dmd2.079 -version=small: time: 0:00.11, memory: 40288 KiB
dmd2.079 -version=large: time: 0:00.11, memory: 40584 KiB
dmd2.079 -version=vlarg: time: 0:00.11, memory: 40768 KiB
dmd2.080 -version=small: time: 0:00.45, memory: 133164 KiB
dmd2.080 -version=large: time: 0:04.62, memory: 1342576 KiB
Error: out of memory
Command exited with non-zero status 1
dmd2.080 -version=vlarg: time: 1:30.94, memory: 3713420 KiB
dmd2.git -version=small: time: 0:00.17, memory: 45272 KiB
dmd2.git -version=large: time: 0:00.15, memory: 50016 KiB
dmd2.git -version=vlarg: time: 0:00.25, memory: 69864 KiB

So, still a bit worse than pre-regression, but much more tolerable.

Couldn't we put the check for static arrays into a CTFE'd function? That is, never unpack static arrays on the template level; rather make the condition (pseudocode) enum isAllZeroBits = isAllZeroBits!(value[0]) && allElementsEqual(value);?

That didn't occur to me. Seems to work, but it's a tiny bit slower than the template stuff.
I tested this:
        enum isAllZeroBits = ()
        {
            static if (value.length == 0) return true;
            else static if (.isAllZeroBits!(typeof(value[0]), value[0]))
            {
                foreach (e; value)
                {
                    if (e !is value[0]) return false;
                }
                return true;
            }
            else return false;
        }();

And got these stats:
dmd2.git -version=small: time: 0:00.13, memory: 45472 KiB
dmd2.git -version=large: time: 0:00.16, memory: 50832 KiB
dmd2.git -version=vlarg: time: 0:00.28, memory: 73108 KiB

The difference really isn't large, though. I wouldn't mind changing it to this, if you prefer it over the template version. Maybe this version will benefit from NewCTFE?

Found a small problem: Comparing static arrays with == is no good, because an element's opEquals might give a result that has nothing to with its bits (e.g. always true or always false). And is gives a deprecation warning saying it compares by reference, not by values. It actually seems to do the right thing during CTFE, but the deprecation is probably going to become an error at some point, so I don't want to add code that triggers it.
So I've added a bitwiseIdentical function which steps into static arrays to do is on the elements. It would be nice if there was a simpler way to do this, but I can't see one.
With this, the template version lost the slight advantage it had in the T.init case. So I've switched to the CTFE foreach version. It performs better on values that are not T.init. Also added tests for funky opEquals and nested static arrays.
Current stats for the test case of issue 18937:
dmd2.git -version=small: time: 0:00.14, memory: 45472 KiB
dmd2.git -version=large: time: 0:00.17, memory: 51692 KiB
dmd2.git -version=vlarg: time: 0:00.32, memory: 75480 KiB

Ping? This is a regression fix, and I'd like to move forward here.


Remove Paket

This is a spin off of https://github.com/fslaborg/flips/issues/138#issuecomment-752134887.
Replace Paket with NuGet package dependencies specified in the project files.
I am willing to make this change.

Go for it!

I don't always recommend removing Paket.  Material Design in XAML is an example of a project that uses Paket and where I have been able to contribute without any problems from Paket.
Let's try to keep Paket.
On main, executing sh build.sh successfully compiles the code (and unexpectedly to me also runs the tests, which all pass).  Then in Visual Studio, I have these compile errors.
Severity	Code	Description	Project	File	Line	Suppression State
Error	FS0078	Unable to find the file 'C:\Code\ThirdParty\flips\Flips\bin\x64\Debug\netstandard2.0\Flips.dll' in any of C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\3.1.0\ref\netcoreapp3.1 C:\Code\ThirdParty\flips\Flips.Examples c:\program files (x86)\microsoft visual studio\2019\enterprise\common7\ide\commonextensions\microsoft\fsharp\	Flips.Examples	C:\Code\ThirdParty\flips\Flips.Examples\FSC	1	Active

Severity	Code	Description	Project	File	Line	Suppression State
Error	FS0078	Unable to find the file 'C:\Code\ThirdParty\flips\Flips\bin\x64\Debug\netstandard2.0\Flips.dll' in any of C:\Program Files\dotnet\packs\Microsoft.NETCore.App.Ref\3.1.0\ref\netcoreapp3.1 C:\Code\ThirdParty\flips\Flips.Tests c:\program files (x86)\microsoft visual studio\2019\enterprise\common7\ide\commonextensions\microsoft\fsharp\	Flips.Tests	C:\Code\ThirdParty\flips\Flips.Tests\FSC	1	Active

Why is that?  How can I avoid this?
A temporary workaround is to unload Flips.Examples and Flips.Tests.  I would prefer not to do that though.

What I really needed was consistent builds, and I think I have that now that PR #143 is complete.  I don't think Paket is bothering me at the moment.


[QPG] lighting-app: Persist level and color

Use DeferredAttributePersistenceProvider to save CurrentX, CurrentY and CurrentLevel attributes. Restore them on startup.

Can the Darwin failure be waived?
[Default] Error(src/credentials/CHIPCert.cpp:642: CHIP Error 0x0000005C: Wrong node ID): Generating operational certificate failed xctest(28117,0x104635600) malloc: nano zone abandoned due to inability to preallocate reserved vm space.
Possibly already causing issues the last four days: https://csamembers.slack.com/archives/G014G30SVV0/p1705622046224799


Pixman software renderer

After renderer v6, we'll be able to start working on a pixman software renderer.  The returned wlr_buffer can be exported to a DMA-BUF via drmPrimeHandleToFD.

DRM backend: import the DMA-BUF via drmPrimeFDToHandle (or GBM?) and scan it out
Other backends: ???

Need to be careful wrt. GEM handle ref-counting.

Where should wlr_egl go? Sould it belong to the implementation of the wlr_renderer?
Also how would the renderer choosing strategy work? egl then pixman, always egl unless an env var is specified?

Where should wlr_egl go? Sould it belong to the implementation of the wlr_renderer?
Also how would the renderer choosing strategy work? egl then pixman, always egl unless an env var is specified?

Where should wlr_egl go? Sould it belong to the implementation of the wlr_renderer?

Probably in wlr_gles2_renderer yes.

Also how would the renderer choosing strategy work? egl then pixman, always egl unless an env var is specified?

By default, if there's a render node, try GLES2. Otherwise try pixman. We probably want an env var to force a specific renderer.

Where should wlr_egl go? Sould it belong to the implementation of the wlr_renderer?

Probably in wlr_gles2_renderer yes.

Also how would the renderer choosing strategy work? egl then pixman, always egl unless an env var is specified?

By default, if there's a render node, try GLES2. Otherwise try pixman. We probably want an env var to force a specific renderer.

More thoughts:

As discussed on IRC, the DRM dumb buffer allocator will get GEM handles and somehow needs to share them with the DRM backend. We need to be careful wrt. ref'counting issues with GEM handles.

One solution is to open the primary DRM node multiple times (via the session) and then the allocator can do some drmPrimeHandleToFD calls to implement wlr_buffer_impl.get_dmabuf.
Another solution would be to have a wlr_buffer_impl.get_gem_handle function, but this would be fragile as it would completely break if both parties don't have the same file description.

The DRM dumb buffer allocator should probably check for DRM_CAP_DUMB_BUFFER.
For the Wayland/X11/headless backends, DRM dumb buffers are not appropriate. We'll probably need a shm allocator (which uses shm_open), use wl_shm on the Wayland backend, use MIT-SHM's AttachFd on X11, and a new wlr_buffer_impl.get_shm_fd function to share the FDs between the allocator and the backends.
We need a way to access the buffers from the Pixman renderer.

We could special-case the DRM dumb buffer allocator in the Pixman renderer, but it would prevent the shm allocator and third-party allocators to work.
We could add a wlr_buffer_impl.map function to create a memory mapping for the buffer.

More thoughts:

As discussed on IRC, the DRM dumb buffer allocator will get GEM handles and somehow needs to share them with the DRM backend. We need to be careful wrt. ref'counting issues with GEM handles.

One solution is to open the primary DRM node multiple times (via the session) and then the allocator can do some drmPrimeHandleToFD calls to implement wlr_buffer_impl.get_dmabuf.
Another solution would be to have a wlr_buffer_impl.get_gem_handle function, but this would be fragile as it would completely break if both parties don't have the same file description.

The DRM dumb buffer allocator should probably check for DRM_CAP_DUMB_BUFFER.
For the Wayland/X11/headless backends, DRM dumb buffers are not appropriate. We'll probably need a shm allocator (which uses shm_open), use wl_shm on the Wayland backend, use MIT-SHM's AttachFd on X11, and a new wlr_buffer_impl.get_shm_fd function to share the FDs between the allocator and the backends.
We need a way to access the buffers from the Pixman renderer.

We could special-case the DRM dumb buffer allocator in the Pixman renderer, but it would prevent the shm allocator and third-party allocators to work.
We could add a wlr_buffer_impl.map function to create a memory mapping for the buffer.

We need a way to access the buffers from the Pixman renderer.

To expand on this a little bit more: if we want something that works with both the DRM dumb allocator and the shm allocator, we have a few solutions:

Add a wlr_buffer_impl.get_data_ptr function that returns a pointer to an mmap'ed region. This would be pretty close to what @bl4ckb0ne has implemented in the PR: mmap happens in the allocator, there's only a single mmap'ed region, and the renderer just creates a pixman surface with the data pointer.
Add wlr_buffer_impl.begin_access and wlr_buffer_impl.end_access. This would be similar to (1), however it would let the allocator know when the mmap'ed region is being accessed by the renderer. In case the buffer comes from an untrusted client, this is necessary to make sure SIGBUS doesn't take the whole compositor down. See https://wayland.freedesktop.org/docs/html/apc.html#Server-structwl__shm__buffer_1a809cb5d6b33338c62bbca6daa4138667. We may want to render to a client buffer for the screencopy protocol.
Add wlr_buffer_impl.map and wlr_buffer_impl.unmap functions, and let the renderer create the memory mappings. This would allow us to have potentially more efficient mappings, e.g. a write-only mapping for rendering and a read-only mapping for screencopy.

I like the option 1 for now, I think it's the easiest/quickest one. Do you want a separate patch for it or can I add it to the drm dumb allocator one and patch the gbm allocator after?

GBM doesn't need to implement it.

Done in #2700. Though I'm getting segfaults when I rebase #2661 to use it, still investigating.


assert_failure is not working well with piped commands

Due diligence

[ ] I am familiar with the "Reporting issues" section of CONTRIBUTING.md
[V] I have searched the existing issues and haven't found one that matches what I'm reporting or requesting now.
[ ] If this is a bug report, I have searched the Bash changelog for information that may explain or provide clues to the behavior I'm observing and reference it in the body of the report.

Framework, Bash, and operating system version information
#!/usr/bin/env bats

load ../../testing/enviroment

@test "vanilla bats" {
   run $( ls | grep nonexist )
   [ "$status" -eq 0 ]
}

@test "using assert with file that foes not exists (should fail -> test should be successful" {
   run $( ls | grep nonexist )
   assert_failure
}

@test "using assert with file that foes exists (should not fail -> test fail" {
   run $( ls | grep t2.bats )
   assert_failure
}

Output:
 $ bats t2.bats
 ✓ vanilla bats
 ✗ using assert with file that foes not exists (should fail -> test should be successful
   (in test file t2.bats, line 14)
     `assert_failure' failed
   expected failure, but command succeeded
   STATUS: 0
   OUTPUT:

 ✓ using assert with file that foes exists (should not fail -> test fail

3 tests, 1 failure

Description
I copied the assertion scripts for Bats into my own project to use with Bats for testing command line.
It seems that there is a problem with pipe commands. The result is opposite to what I expected (see code example).

Short answer/quick fix: Wrap your pipeline in a function or script, not in $().
This isn't a failure of the assertion; Bash evaluates the $() expression and passes the result to run, rather than passing the pipeline to run to execute.
In the $(ls | nonexistent) case, the expression evaluates to the empty string. run effectively has no arguments, resulting in a successful status. The equivalent effect can be observed on the command line:
$ $()
$ echo $?
0

In the $(ls | t2.bats) case, the expression evaluates to t2.bats, resulting in the command run t2.bats. Since t2.bats isn't executable, the run t2.bats command fails, and assert_failure passes.

Hi thank you for you reply. I had a feeling that it is something related to how bash works, rather with your function.
However, the suggestion of wrapping with a function does not work. Maybe I did it wrong? How does the function know when to return 0/1 status?

load ../../testing/enviroment

f() {
   ls | grep nonexist
}

@test "vanilla bats" {
   run $( ls | grep nonexist )
   [ "$status" -eq 0 ]
}

@test "vanilla bats should fail (file exists)" {
   run $( ls | grep t2.bats )
   [ "$status" -eq 0 ]
}

@test "using assert with file that does not exists (should fail -> test should be successful)" {
   run $( ls | grep nonexist )
   assert_failure
}

@test "using assert with file that does exists (should not fail -> test fail)" {
   run $( ls | grep t2.bats )
   assert_failure
}

@test "using assert on a function  with file that does not exists (should fail -> test should be successful)" {
   run f
   assert_failure
}

@test "using assert on a function with file that does exists (should not fail -> test fail)" {
   run f
   assert_failure
}

Ouput:
10:18 $ bats t2.bats
 ✓ vanilla bats
 ✗ vanilla bats should fail (file exists)
   (in test file t2.bats, line 18)
     `[ "$status" -eq 0 ]' failed
 ✗ using assert with file that does not exists (should fail -> test should be successful)
   (in test file t2.bats, line 24)
     `assert_failure' failed
   expected failure, but command succeeded
   STATUS: 0
   OUTPUT:

 ✓ using assert with file that does exists (should not fail -> test fail)
 ✓ using assert on a function  with file that does not exists (should fail -> test should be successful)
 ✓ using assert on a function with file that does exists (should not fail -> test fail)

6 tests, 2 failures

The last two test cases are calling the same function with the same arguments. Instead, try:
load ../../testing/enviroment

f() {
   ls | grep "$1"
}

@test "using assert on a function, nonexistent file (should fail -> test should be successful)" {
   run f 'nonexistent'
   assert_failure
}

@test "using assert on a function, existing file (should not fail -> test fail)" {
   run f 't2.bats'
   assert_failure
}

You should see:
$ bats t2.bats
 ✓ using assert on a function, nonexistent file (should fail -> test should be successful)
 ✗ using assert on a function, existing file (should not fail -> test fail)
   (in test file t2.bats, line 14)
     `assert_failure' failed
   expected failure, but command succeeded
   STATUS: 0
   OUTPUT:
   t2.bats

2 tests, 1 failure

Oh, that was very careless of me. Thank you for the help :smile:
Is there a reason why the assert library can not be part of bats officially?

Just haven't gotten around to it yet is all. Even if it doesn't go into bats-core/bats-core, I'll probably post it as a separate repo somewhere. Nothing in that directory depends on anything in the rest of go-script-bash.


Update mytoyota in manifest file

Hello, I think we should change the mytoyota version also in manifest, because Homassistant is not aware of the version specified in the poetry toml file

My head is not working to day. 🤦 I often forget all the places where i have to update version information and such. I know that poetry can do a lot of this for me, so I probably should look into that. 😄
I will close this, though i appreciate the effort. I will push a fix in a couple of minutes.

New update pushed. Sorry for the inconvenience


error in running train_gnns.py file

Hi, I tried to execute this command: python train_gnns.py datasets=ba_2motifs models=gcn and I got the following error:
Traceback (most recent call last):
  File "train_gnns.py", line 262, in main
    trainer.train(train_params=train_params, optimizer_params=optimizer_params)
  File "train_gnns.py", line 146, in train
    loss = self._train_batch(batch, batch.y)
  File "train_gnns.py", line 47, in _train_batch
    logits = self.model(data=data)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/workspace/notebooks/external_src/GStarX/gnnNets.py", line 359, in forward
    emb = self.get_emb(*args, **kwargs)
  File "/workspace/notebooks/external_src/GStarX/gnnNets.py", line 348, in get_emb
    x = self.convs[i](x, edge_index)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/workspace/notebooks/external_src/GStarX/gnnNets.py", line 258, in forward
    x = torch.matmul(x, self.weight)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1178, in __getattr__
    type(self).__name__, name))
AttributeError: 'GCNConv' object has no attribute 'weight'

What can i do to fix it. Thanks in advance.

This looks like a PyG error. Can you check your PyG version please? The train_gnn.py script is adapted from the version of DIG that builds up on PyG 1.7.2. Recently, PyG updated to 2.0, which has changed a lot from PyG 1.X

Recently I am a bit busy with other stuff. I will find a time to make the model to be compatible with PyG 2.0

yes,using PyG 1.7.2 solves the issue. I assumed it is mentioned in requirements.txt. I would very much appreciate if you could provide the code working with PyG 2.0. Thanks a lot for your help.


Integration tests

Updated as per review comments.

Can you remove the lines in test_equivalence.py that are no longer needed after moving the integration test?  Imports for subprocess, os, and sys can be removed.  Also the line that defines example_dir can be removed.  Thanks.

Hi @hemantdwave, were you able to resolve the testing on your end via an alternative solution so that this and #6 can be closed?


Weighted Pool Exit Concern - Refactoring function and tests;

Refactoring Exit concern functions;
Refactoring Exit concern integration tests;
Adding unit tests file;

This PR seems to change joins too which seems to be what #327 covers? Rather than having the same changes in separate PRs its easier to either keep them separate, e.g. no join stuff in this one, or combine all changes to single PR. Note I've already reviews #327 so if you do combine PRs try to address those comments too. Thanks.

This PR seems to change joins too which seems to be what #327 covers? Rather than having the same changes in separate PRs its easier to either keep them separate, e.g. no join stuff in this one, or combine all changes to single PR. Note I've already reviews #327 so if you do combine PRs try to address those comments too. Thanks.

Ok! I will remove the other PR and put the Join PR comments and fixes in this branch


Can it be distributed

I want a distributed k/v db

tiedot is designed to offer excellent throughput for single machine operation.
As of now, distributed operation modes such as HA, sharding, are not supported.

Feel free to come back if you have more questions.


$ref from JSON is not recognized and replaced by any

In my yaml I have references to json models in the local files, like this
 schemas:       details:         $ref: 'models/details.json'
When I generate TS, I am gettin the property type as "any". No errors are shown. Just a missing type in a result.
When I describe details as Yaml in this file, then everything works as expected, and generator creates an interface. So it's something about the external JSON models that is not working. However the same setup works with another TS generator, so models themselves are correct.

Is any way to work with separate schemas with relative ref?
I have sane issue with any-types


The method Raven_Client::getDefaultPrefixes() works incorrectly on Windows

The incorrect code: https://github.com/getsentry/sentry-php/blob/1.6.1/lib/Raven/Client.php#L168-L172
private function getDefaultPrefixes()
{
    $value = get_include_path();
    return explode(':', $value);
}

Im actually not entirely sure how we should go about making this work.

@dcramer i think on windows the include paths use ; as separator.

Just verified. Uses ; for Windows and : everywhere else: http://php.net/manual/en/ini.core.php#ini.include-path

https://github.com/getsentry/sentry-php/pull/397


[Mod Update] More random drops from nodes

Mod(s) Affected

squareone_ores
squareone_exp
squareone_items/nodes.lua
xdecor/src.lua

Checklist of planned development

[X] randomize the dropped lump count of common ores like copper and silver
[x] add drop items to basic nodes like dirt, sand, snow
[x] #12 converting to mod update since i've made a few changes
[x] while mining, if the main inventory is full, place the mined item into bag if it exists
[x] remove code from items/nodes.lua, functions have been isolated into own mod
[x] if the inventory is full, attempt to place items into the enderchest if it is in players' inventory or bag, otherwise place items in the bag if equipped
[x] increase enderchest item capacity

Other brainstorm ideas

[x] for higher exp gives more items dropped? maybe another time

**Are there potential issues with this update?

[ ] Possibly if there are too many random items dropped from a single node, maybe lag?

Deployment Status

[ ] Not Ready, still planning or stalled
[ ] Not Ready, under development
[ ] Ready, not uploaded
[ ] Ready and uploaded, pending server reboot
[x] Is live on server

After Deployment
Any new bugs being reported?

[ ] No, Close ticket
[x] Yes, Describe and open new ticket.

drops are too frequent -- lowered drop amounts will be applied next reboot

Anything else to add?

enderchest inventory is no longer created for every player who joins, waste of space. the inventory is now only created when the player crafts an enderchest.
for players who have an enderchest, craft an enderchest to increase the inventory size.

Add Screenshots if needed.

so far no issues, item drops are less frequent after update, probably will reduce frequency again next update.  closing ticket


How do I access the logger?

Hi, I couldn't find any information in the documentation.
How can I access the logger, e.g. to add a handler? I would like to write the output to a file.
I have a bit experience with flask so I tried to access the flask logger
#!/usr/bin/env python3

import connexion
import logging
from logging.handlers import WatchedFileHandler

from swagger_server import encoder
from swagger_server.database import init_db, db_session
from swagger_server.tools import create_path

def main():
    app.run(port=8080)

logging.basicConfig(level=logging.INFO)
init_db()

app = connexion.App(__name__, specification_dir='./swagger/')
app.app.json_encoder = encoder.JSONEncoder
app.add_api('swagger.yaml', arguments={'title': 'P7 Renderservice'})

application = app.app

logPath = './data/logs/service.log'
create_path(logPath)
handler = WatchedFileHandler(logPath)
handler.setLevel(logging.DEBUG)
application.logger.addHandler(handler)

@application.teardown_appcontext
def shutdown_session(exception=None):
    db_session.remove()

if __name__ == '__main__':
    main()

-> https://github.com/padok/REST-Event-API-example/blob/master/swagger_server/main.py
but my file remains empty
I want to save or modify what the app already print to stdout; for example
<IP_ADDRESS> - - [02/May/2018 03:02:46] "GET /v1/event HTTP/1.1" 200 -
<IP_ADDRESS> - - [02/May/2018 03:02:46] "GET /v1/event HTTP/1.1" 200 -
<IP_ADDRESS> - - [02/May/2018 03:02:46] "GET /v1/event HTTP/1.1" 200 -
<IP_ADDRESS> - - [02/May/2018 03:02:47] "GET /v1/event HTTP/1.1" 200 -

Thanks for your help!

you did more or less correct.
Please use application factory: http://flask.pocoo.org/docs/0.12/patterns/appfactories/
I use a custom extension for logging, but the essential is:
'''
handler = logging.StreamHandler()
handler.setLevel(config['LOGGING_LEVEL'])
handler.setFormatter(logging.Formatter(config['LOGGING_FORMAT']))
app.logger.addHandler(handler)
app.logger.setLevel(app.config['LOGGING_LEVEL'])
'''
after you use the the logger with the correct level.
Instead of StreamHandler it could be any other handler. Or it could be even proxy.

Do you have an example @roman-telepathy-ai ?

logging example is above. a note here: if accessing connexion app, then use app.app.
regarding - App factories just have a look at docs or search for "flask tutorial" (miguel will be on top).

I have figured this out. I'll explain now. I didn't use the app factory pattern though but hopefully my code is clear enough.
I have a special validator class. The code is as follows:

import datetime
from connexion.decorators.validation import RequestBodyValidator
from jsonschema import ValidationError
from connexion import problem
from flask import Flask

app = Flask(__name__)
BAD_REQUEST = 'Bad Request'

class RequestBodyValidatorVerboseErrorLogging(RequestBodyValidator):

    def validate_schema(self, data, url):
        if self.is_null_value_valid and is_null(data):
            return None

        try:
            self.validator.validate(data)
        except ValidationError as exception:
            app.logger.error("{timestamp} {url} VALIDATION_ERROR: {error} REQUEST_BODY: {request_body}".format(timestamp=datetime.datetime.utcnow().isoformat(),url=url, error=exception.message, request_body=data))
            return problem(400, BAD_REQUEST, str(exception.message))

        return None

I want to log the error messages of this class to a rotating file. Earlier, I was using the common logger namespace convention but my suspicion was that the Flask application was taking precedence. I changed the logging to use app.logger.error instead of logger.error. Notice my import above.
My main API file is as follows:
#!/usr/bin/env python3

import connexion
import logging
import sys
from event_api_validators import RequestBodyValidatorVerboseErrorLogging
from logging.handlers import RotatingFileHandler

SWAGGER_PATH = '/event_api/'
LOG_PATH = '/error_logs/event_api_error_log.log'
RESOLVED_SWAGGER_FILE = 'api_index_resolved.json'

log_handler = RotatingFileHandler(LOG_PATH, mode='a', maxBytes=100*1024*1024,
                                 backupCount=10, encoding=None, delay=False)
log_handler.setLevel(logging.ERROR)

validator_map = {
    'body': RequestBodyValidatorVerboseErrorLogging
}

event_api = connexion.App(__name__, validator_map=validator_map)
event_api.app.logger.addHandler(log_handler)
event_api.app.logger.setLevel(logging.ERROR)
event_api.add_api(RESOLVED_SWAGGER_FILE, swagger_path=SWAGGER_PATH)

if __name__ == '__main__':
    event_api.run(port=80)

This works with the RotatingFileHandler. Additionally, I still write to stream with the default log handler as well.

exactly
event_api.app.logger.addHandler(log_handler)
event_api.app.logger.setLevel(logging.ERROR)

in your validator class, use
from flask import current_app
current_app.logger.info("flask logger: foo")
...
instead of `app = Flask(__name__)`

Thanks! Much cleaner.


Requesting new build to sync with 1.42.0

Can a new release be made now that 1.42.0 has gone out the door? I need it for 094917781ba66902f587554dd0e1c1a11ec10d7a and 99e583a9977341f6805b916817e80303f6ea640e as I continue my experiments with the semantic highlighting support.
Thanks!

Published 6.1.1 client and server.

Thanks, @dbaeumer!


impl a function to parse chunk sizes

All HTTP/1.1 clients and servers must understand the chunked encoding
and therefore must parse chunk sizes. The parse_chunk_size function
uses relies on the internals of httparse so it can't be implemented as
easily in an external crate. It is based on hypers implementation but
unlike hyper it operates on &[u8] and returns partial if the chunk size
is incomplete. For this reason it is more suitable for async HTTP code.
This is a breaking change because it adds a new variant to the Error
enum.

Excellent, I've been meaning to do something like this.
I wonder if it makes sense to return httparse::Error from this function... The only error that could possibly happen is ChunkSize. Perhaps just make that pub struct InvalidChunkSize;? Would also mean it wouldn't be a breaking change...

It now uses pub struct InvalidChunkSize; so it is no longer a breaking change but the return type of parse_chunk_size is a lot less nice.


https letsencrypt failing

trying to add https to redmine, which by default uses 80 and 443 in its own container, http works fine, https is failing
redmine config labels
rap.le_email<EMAIL_ADDRESS>rap.le_host: redmine.cloud.com
io.rancher.container.hostname_override: container_name
io.rancher.container.pull_image: always
rap.host: redmine.cloud.com
rap.proto: https
results in failure, log file below
4/18/2018 9:06:00 AMcrond[52]: wakeup dt=60
4/18/2018 9:06:00 AMcrond[52]: file root:
4/18/2018 9:06:00 AMcrond[52]:  line /app/letsencrypt.sh
4/18/2018 9:07:00 AMcrond[52]: wakeup dt=60
4/18/2018 9:07:00 AMcrond[52]: file root:
4/18/2018 9:07:00 AMcrond[52]:  line /app/letsencrypt.sh
4/18/2018 9:07:27 AMranchergen.1 | level=warning msg="Unexpected format of service port: 80/tcp"
4/18/2018 9:07:27 AMranchergen.1 | level=info msg="All templates processed. Waiting for changes in Metadata..."
4/18/2018 9:07:37 AMranchergen.1 | level=warning msg="Unexpected format of service port: 80/tcp"
4/18/2018 9:07:37 AMranchergen.1 | level=info msg="Destination file %s has been updated/app/letsencrypt.conf"
4/18/2018 9:07:37 AMranchergen.1 | level=info msg="Executing notify command '/app/letsencrypt.sh'"
4/18/2018 9:08:00 AMcrond[52]: wakeup dt=60
4/18/2018 9:08:00 AMcrond[52]: file root:
4/18/2018 9:08:00 AMcrond[52]:  line /app/letsencrypt.sh
4/18/2018 9:08:43 AMranchergen.1 | level=info msg="[/app/letsencrypt.sh]: "Sleep 30s before Using Acme server https://acme-v01.api.letsencrypt.org/directory""
4/18/2018 9:08:43 AMranchergen.1 | loud.com certificates... (redmine.cloud.com)""
4/18/2018 9:08:43 AMranchergen.1 | euxc0nS1x34XAjkJvCqz6xhC15CdhZEc: \"\r""
4/18/2018 9:08:43 AMranchergen.1 | title>""
4/18/2018 9:08:43 AMranchergen.1 | ]: "level=info msg=\"Processing all templates once.\"""
4/18/2018 9:08:43 AMranchergen.1 | ormed :: Error creating new authz :: DNS name does not have enough labels""
4/18/2018 9:08:43 AMranchergen.1 | mplates processed. Exiting.\"""
4/18/2018 9:09:00 AMcrond[52]: wakeup dt=60
4/18/2018 9:09:00 AMcrond[52]: file root:
4/18/2018 9:09:00 AMcrond[52]:  line /app/letsencrypt.sh
4/18/2018 9:10:00 AMcrond[52]: wakeup dt=60
4/18/2018 9:10:00 AMcrond[52]: file root:
4/18/2018 9:10:00 AMcrond[52]:  line /app/letsencrypt.sh
4/18/2018 9:11:00 AMcrond[52]: wakeup dt=60

nevermind... its resolved


docs: fix the name of a response code detail

"original_ip_detection_failed" does not exist, use the correct value "rejecting_because_detection_failed" instead.
Fixes #25147
Signed-off-by: Michael Kaufmann<EMAIL_ADDRESS>Risk Level: Low
Docs Changes: replaced "original_ip_detection_failed" with "rejecting_because_detection_failed"

/docs

Rendered page: https://storage.googleapis.com/envoy-pr/014e482/docs/configuration/http/http_conn_man/response_code_details.html


case.then() automatically adds quotes around a supplied value

case.then() automatically adds quotes around a supplied value.
This isn't desirable when, for example, I want to use an aggregator in the then clause:
caseStatement.when('"plan"."id" IS NOT NULL');
caseStatement.then('JSON_AGG("plan".*)');

will output:
CASE WHEN ("plan"."id" IS NOT NULL) THEN 'JSON_AGG("plan".*)'

I dunno if there's an option to pass that will prevent this from happening?

You can add DefaultQueryBuilderOptions to the case builder to archive what you want.  Try to check which options you can apply here in you specific case I guess this solution: squel.case({ replaceSingleQuotes: true }) will work.
Optionally singleQuoteReplacement which let you specify the character to replace with.
Similar to #330


Clarify installation procedure

The procedure for installing zplug is not clear or does not work properly from the documentation in README.md. Using "the best way" curl command, I get a successful installation and the ~/.zplug directory is created, but the zplug binary is not found in my path. This is not described in the docs.
Next after installation is the "Usage" section which shows you how to add zplug commands to your .zshrc file, but these will of course fail since the zplug command is not working at this point.
The next thing that seems to work is that source ~/.zplug/init.zsh is mentioned, this actually sets up the zplug command to work, but am I supposed to put this line into my .zshrc so that init.zsh is run every time I source my .zshrc?
Please clarify how zplug is supposed to be installed and used in chronological order so its easier to follow and understand the steps.
I'm doing this on Ubuntu 18.04 and had no errors thrown by the installation script.

So you just run the install one-liner and correctly setup the environment and it'll work. This isn't a plug and play solution. Scour GitHub to see how other's have setup their .zshrc file and other relevant config files. You'd be amazed at the amount of dotfiles people post on here and it's really a treasure trove of info.
So to answer your question,

install with curl one-liner
read through all info in the wiki to build a basic .zshrc file
run zplug install to get the plugins you reference in .zshrc
reload your config ie. source  ~/.zshrc and you should be at a working prompt (albeit not visually appealing, that's the fun part)


Update OpenSearch versions in main and 1.x branches

Please update the version consumed in main and 1.x branch according to OpenSearch main and 1.x respectively.

I think this old issue is not relevant any more. Closing.


updated the source.json and added a test script

NOTE To run the test.py will need the jsonschema package installed
I saw a comment from you on a different issue I think related to having geometries in the schema.  I've added the geojson.json for reference.  Let me know if there are other attributes that need to be added to the source.json schema.  None of the exising state json have anything beyond just the 6 properties currently listed in the schema, so not sure of the structure you are looking for.

The comment about geojson was for the schema for results, not sources

I think there is also going to need to be an option file field, because some zipfiles will contain multiple items.

If there is a required package, it should go in a requirements file https://pip.readthedocs.org/en/1.1/requirements.html

Opps, did not see this PR before I pushed mine. @slusset, if you reference the issue your commits or PRs are fixing, it'll show up in that issue's discussion. Just like this: #16

I think this PR is superseded now. @JesseCrocker close if you agree?


Consider skipped checks successful

Right now, if the pr bot encounters skipped checks, it thinks they've failed and it doesn't action (see https://github.com/apache/beam/pull/21921#issuecomment-1158904893). This fixes that issue.
Part of #21417

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Can one of the admins verify this patch?

R: @lostluck
Should be a quick one liner

Thank you!


How do I customize the style of the button?

I've tried to follow the instructions in the CONTRIBUTE.md file but yarn start resulted in a syntax error eventually. Also I believe I'm able to toggle the styles of the .utterences and the corresponding iframe object. However, when I try to override the styles of more specific elements, e.g. the button, it just didn't work.
Is there any way to specify these styles using local css files? I believe this is really useful as different sites adopt different themes themselves. Full customization sounds reasonable to me.

Utterances does not support custom CSS. More info: https://github.com/utterance/utterances/issues/197#issuecomment-536203416.
Contributing a theme is currently the only way to change the styles.


Regenerate library

Regenerates the Ruby library to include changes from recent updates to
the OpenAPI spec including PRs:
https://github.com/mxenabled/openapi/pull/40
https://github.com/mxenabled/openapi/pull/41

:shipit:


Examples for k8s are wrong

Various example yaml does not reflect the right way to setup on k8s.
One such example is the args for this file https://github.com/GoogleCloudPlatform/endpoints-samples/blob/master/k8s/esp_echo_http.yaml
It is the same thing on the google documentation on cloud endpoints, arguments passed should be -p -a -s -v etc.
Link to the google documentation that does not comply with the code in ESP: https://cloud.google.com/endpoints/docs/openapi/get-started-kubernetes

Hello,
To the best of my knowledge, the arguments described in the example and in the documentation are in agreement with the code in ESP: see the argument declarations here https://github.com/cloudendpoints/endpoints-tools/blob/master/start_esp/start_esp.py#L396-L565
Are you saying that the command-lines in the examples and in the documentation are not working?
Thanks!

Derp... Looks like I was constantly using dashes instead of underscore. Sorry about that.


pulsar: 1.104.0 -> 1.105.0

Release note: https://github.com/pulsar-edit/pulsar/releases/tag/v1.105.0
Description of changes
This Pulsar release was made from a Windows computer, the SHA256SUMS.txt file was encoded in UTF-16LE with BOM. Most of the work was enhancing the update script to handle this case.
Also implemented the recommendation from SuperSandro2000: https://github.com/NixOS/nixpkgs/pull/226616#discussion_r1170658908
Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/2261


aws_ecs_cluster: Warning: Argument is deprecated

Terraform validate ...

Warning: Argument is deprecated

  with aws_ecs_cluster.test,
  on aws_ecs_cluster__test.tf line 4, in resource "aws_ecs_cluster" "test":
   4:   capacity_providers = []

Use the aws_ecs_cluster_capacity_providers resource instead
Success! The configuration is valid, but there were some validation warnings as shown above.

change committed that should fix this one.

i deleted the ecs cluster and can not verify.
will assume it works
thank y ou


There is no support for collapsing an already expanded textview.

I would also love to see a function for collapsing!


{lib-}mediainfo{-gui}: 0.7.79 -> 0.7.80

From the changelog:
Version 0.7.80, 2015-11-30

+ Matroska: support of MKVMerge statistics tags (duration frame count,
  stream size, bit rate) per track, thanks to ndjamena
+ FLAC: Channel positions, thanks to ndjamena
+ FLAC: difference between detected bit depth and stored bit depth
+ MPEG-TS: if DTVCC transport stream is present and no DTVCC service
  descriptor, scan also in the middle of the file in order to detect
  more caption services
+ Subtitle frame rate computing if frame count and duration are
  available (hidden by default)
+ Subtitles in Matroska: count of elements
+ Matroska, MXF and MP4/MOV: detection of truncated files
+ DTS: difference between ES Matrix and ES Discrete
+ DTS: display ES Matrix or ES Discrete even if HRA or MA is present
+ DTS: difference between DTS-HRA with 96k option and pure DTS-96/24
+ DTS: detection of DTS:X
+ Samples per frame info
+ AC-3: detection of Atmos inside TrueHD
+ Video frame rate: showing precision of 1/1.001 frame rates (e.g.
  "23.976 (24000/1001) fps" and "23.976 (23976/1000) fps")
+ MPEG-4/MOV: showing the complete list of compatible brands in the
  CodecID field
+ MPEG-4/MOV: Alternate groups
+ MPEG-4/MOV: "Disabled" tag
+ MPEG-4/MOV: "Forced" tag
+ MPEG-4/MOV: showing links between tracks (chapters for, subtitles for,
  fallback for)
+ MXF: handling of more acquisition metadata items
+ MXF: Package name
+ AVC: Store method of interlaced content (Interleaved Fields or
  Separated Fields)
+ EBUCore: acquisition metadata (Proof of concept, for feedback only)
x Matroska: frame rate detection algorithm revisited, less wrong numbers
  are expected
x SDP/Teletext: some pages were sometimes (when present in 2 different
  SDP lines) displayed several times
x MPEG-4/MOV: some hint tracks were not displayed
+ Hongkongese language added
+ Option "Full parsing"

nox farted.

Restarted the build.


[MINT-3006] Refactor Backend to Support Model Plan Phases

MINT-3006
Description

In order to support frontend and email template needs we have anticipatedStatuses to include a phase enumeration value for easy state handling. Also, in order to more easily infer attribute intent we have updated the name from anticipatedStatuses to suggestedPhase. A null phase result indicates that there is no currently suggested phase.
How to test this change

Open Postman and run any of the endpoints to validate a phase and corresponding suggested statuses
Check unit tests

PR Author Checklist

[x] I have provided a detailed description of the changes in this PR.
[x] I have provided clear instructions on how to test the changes in this PR.
[x] I have updated tests or written new tests as appropriate in this PR.
[x] Updated the Postman Collection if necessary.

PR Reviewer Guidelines

It's best to pull the branch locally and test it, rather than just looking at the code online!
When approving a PR, provide a reason why you're approving it

e.g. "Approving because I tested it locally and all functionality works as expected"
e.g. "Approving because the change is simple and matches the Figma design"

Don't be afraid to leave comments or ask questions, especially if you don't understand why something was done! (This is often a great time to suggest code comments or documentation updates)
Check that all code is adequately covered by tests - if it isn't feel free to suggest the addition of tests.

Could you update the postman?  Still using anticipatedStatuses

Could you update the postman? Still using anticipatedStatuses

Good catch. Postman is updated but I forgot to re-export after resolving a merge conflict. It will be in shortly.


Second invocation of gb build all fails with permission denied error on windows

After running a successful gb build -v all on a hello world app, I tried to run the same command again immediately:
$ gb build -v all
DEBUG project root "c:\\Users\\jason_000\\gbtest"
DEBUG matchPackages: [c:\Users\jason_000\gbtest\src] all
DEBUG args: [github.com/constabulary/gb github.com/constabulary/gb/cmd github.com/constabulary/gb/cmd/gb github.com/constabulary/gb/cmd/gb-env mysqltest]
DEBUG loadPackage: github.com/constabulary/gb false (c:\Users\jason_000\gbtest\src\github.com\constabulary\gb)
DEBUG loadPackage: github.com/constabulary/gb/cmd false (c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd)
DEBUG loadPackage: github.com/constabulary/gb/cmd/gb true (c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb)
DEBUG loadPackage: github.com/constabulary/gb/cmd/gb-env true (c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb-env)
DEBUG loadPackage: mysqltest true (c:\Users\jason_000\gbtest\src\mysqltest)
DEBUG buildPackage: github.com/constabulary/gb
DEBUG buildPackage: github.com/constabulary/gb/cmd
DEBUG buildPackage: github.com/constabulary/gb/cmd/gb
DEBUG {main github.com/constabulary/gb/cmd/gb c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb} is a main package, not installing
DEBUG buildPackage: github.com/constabulary/gb/cmd/gb-env
DEBUG {main github.com/constabulary/gb/cmd/gb-env c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb-env} is a main package, not installing
DEBUG buildPackage: mysqltest
DEBUG {main mysqltest c:\Users\jason_000\gbtest\src\mysqltest} is a main package, not installing
INFO github.com/constabulary/gb/cmd/gb
DEBUG gc:gc github.com/constabulary/gb/cmd/gb c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\github.com\constabulary\gb\cmd\gb\main.a [build.go main.go plugin.go test.go]
DEBUG cd c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb; [C:\Go\pkg\tool\windows_amd64\6g -p github.com/constabulary/gb/cmd/gb -pack -o C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\github.com\constabulary\gb\cmd\gb\main.a -I C:\Users\JASON_~1\AppData\Local\Temp\gb238271103 -I c:\Users\jason_000\gbtest\pkg\windows\amd64 -complete build.go main.go plugin.go test.go]
INFO github.com/constabulary/gb/cmd/gb-env
DEBUG gc:gc github.com/constabulary/gb/cmd/gb-env c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb-env C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\github.com\constabulary\gb\cmd\gb-env\main.a [main.go]
DEBUG cd c:\Users\jason_000\gbtest\src\github.com\constabulary\gb\cmd\gb-env; [C:\Go\pkg\tool\windows_amd64\6g -p github.com/constabulary/gb/cmd/gb-env -pack -o C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\github.com\constabulary\gb\cmd\gb-env\main.a -I C:\Users\JASON_~1\AppData\Local\Temp\gb238271103 -I c:\Users\jason_000\gbtest\pkg\windows\amd64 -complete main.go]
INFO mysqltest
DEBUG gc:gc mysqltest c:\Users\jason_000\gbtest\src\mysqltest C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\mysqltest\main.a [main.go]
DEBUG cd c:\Users\jason_000\gbtest\src\mysqltest; [C:\Go\pkg\tool\windows_amd64\6g -p mysqltest -pack -o C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\mysqltest\main.a -I C:\Users\JASON_~1\AppData\Local\Temp\gb238271103 -I c:\Users\jason_000\gbtest\pkg\windows\amd64 -complete main.go]
DEBUG cd .; [C:\Go\pkg\tool\windows_amd64\6l -o c:\Users\jason_000\gbtest\bin\gb-env -L C:\Users\JASON_~1\AppData\Local\Temp\gb238271103 -L c:\Users\jason_000\gbtest\pkg\windows\amd64 C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\github.com\constabulary\gb\cmd\gb-env\main.a]
DEBUG cd .; [C:\Go\pkg\tool\windows_amd64\6l -o c:\Users\jason_000\gbtest\bin\mysqltest -L C:\Users\JASON_~1\AppData\Local\Temp\gb238271103 -L c:\Users\jason_000\gbtest\pkg\windows\amd64 C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\mysqltest\main.a]
DEBUG cd .; [C:\Go\pkg\tool\windows_amd64\6l -o c:\Users\jason_000\gbtest\bin\gb -L C:\Users\JASON_~1\AppData\Local\Temp\gb238271103 -L c:\Users\jason_000\gbtest\pkg\windows\amd64 C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\github.com\constabulary\gb\cmd\gb\main.a]
DEBUG build duration: 79.0402ms map[compile:111.0999ms link:16.003ms]
FATAL command "build" failed: [C:\Go\pkg\tool\windows_amd64\6l -o c:\Users\jason_000\gbtest\bin\gb -L C:\Users\JASON_~1\AppData\Local\Temp\gb238271103 -L c:\Users\jason_000\gbtest\pkg\windows\amd64 C:\Users\JASON_~1\AppData\Local\Temp\gb238271103\github.com\constabulary\gb\cmd\gb\main.a]: exit status 1
cannot create c:\Users\jason_000\gbtest\bin\gb: Permission denied

It looks like this is the result of gb build trying to rebuild the gb.exe file which is busy (because it is being executed).  Shouldn't the gb build command be smart enough to know that it doesn't need to rebuild gb.exe?
Or should the documentation instruct users to remove the gb source package after the tool has been built?

Just rebuilding my application package prevents the error, as you'd expect:
$ gb build -v mysqltest
DEBUG project root "c:\\Users\\jason_000\\gbtest"
DEBUG args: [mysqltest]
DEBUG loadPackage: mysqltest true (c:\Users\jason_000\gbtest\src\mysqltest)
DEBUG buildPackage: mysqltest
DEBUG {main mysqltest c:\Users\jason_000\gbtest\src\mysqltest} is a main package, not installing
INFO mysqltest
DEBUG gc:gc mysqltest c:\Users\jason_000\gbtest\src\mysqltest C:\Users\JASON_~1\AppData\Local\Temp\gb233400775\mysqltest\main.a [main.go]
DEBUG cd c:\Users\jason_000\gbtest\src\mysqltest; [C:\Go\pkg\tool\windows_amd64\6g -p mysqltest -pack -o C:\Users\JASON_~1\AppData\Local\Temp\g
b233400775\mysqltest\main.a -I C:\Users\JASON_~1\AppData\Local\Temp\gb233400775 -I c:\Users\jason_000\gbtest\pkg\windows\amd64 -complete main.g
o]
DEBUG cd .; [C:\Go\pkg\tool\windows_amd64\6l -o c:\Users\jason_000\gbtest\bin\mysqltest -L C:\Users\JASON_~1\AppData\Local\Temp\gb233400775 -L
c:\Users\jason_000\gbtest\pkg\windows\amd64 C:\Users\JASON_~1\AppData\Local\Temp\gb233400775\mysqltest\main.a]
DEBUG removing work directory: C:\Users\JASON_~1\AppData\Local\Temp\gb233400775
DEBUG build duration: 157.1156ms map[compile:14.0041ms link:119.0232ms]

At the moment, packages are cached, but commands are always rebuilt.
This does cause a chicken and egg problem using gb to rebuild gb, but it's partially window's fault for not letting us overwrite files in use.
I'll tackle the former in #37

Just curious - but why default to always rebuilding commands? Is there something about their dependency tree that makes them prone to false negatives (not rebuilding when it should have been)?

So you always got something from issuing go build. This position may need some nuancing.

On 11 May 2015, at 10:46, Jason Buberel<EMAIL_ADDRESS>wrote:
Just curious - but why default to always rebuilding commands? Is there something about their dependency tree that makes them prone to false negatives (not rebuilding when it should have been)?
—
Reply to this email directly or view it on GitHub.


Apply new color variables to webapps

The old variables should be replaced with the new ones.

Important!!, also replace all hardcoded values that don't use the variables themselves!

Webapps:

[ ] compose
[ ] admin
[ ] wf
[ ] reporter
[ ] privacy
[ ] discovery
[ ] one

Tested and confirming the changes. There is no need for Cl entry here. Moving the card to Done.


Release mergeback version 6.8.0

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@855ab6d). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1197   +/-   ##
=======================================
  Coverage        ?   65.81%
=======================================
  Files           ?       14
  Lines           ?      351
  Branches        ?        0
=======================================
  Hits            ?      231
  Misses          ?      120
  Partials        ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 855ab6d...45fb3c6. Read the comment docs.


Forking migrations notworking

Found a bug? Please fill out the sections below.
Issue Summary
django.db.migrations.exceptions.NodeNotFoundError: Migration communication.0001_initial dependencies reference nonexistent parent node ('customer', '0005_auto_20181115_1953')
Steps to Reproduce
setup django project
then add oscar in it
have Forking an app "customer"
after done with development
code push to git
on deploying to server
running migrations getting error

django.db.migrations.exceptions.NodeNotFoundError: Migration communication.0001_initial dependencies reference nonexistent parent node ('customer', '0005_auto_20181115_1953')

Technical details

Python version: Run python --version.
Python 3.8.3

Django version: Look in your requirements.txt, or run pip show django | grep Version.
Version: 3.0.7

Oscar version: Look in your requirements.txt, or run pip show django-oscar | grep Version.
Version: 2.1

Migrating customised apps (models unchanged)
If you have customised an app, but have not touched the models nor migrations, you’re best off copying the migrations from Oscar. This approach has the advantage of pulling in any data migrations. Find the updated(!) Oscar in your virtualenv or clone the Oscar repo at the correct version tag. Then find the migrations, copy them across, and migrate as usual. You will have to adapt paths, but something akin to this will work:
$ cdsitepackages oscar/apps/shipping/migrations
$ copy *.py <your_project>/myshop/shipping/migrations/


Our MCs have the wrong organization label

We didn't add the organization as one of the values passed when installing the cluster-gcp app, so the default value test was used.

I updated the $mc-user-values configmap for gohan, gunther and frieza. This should be fixed now.


[BULK] - DocuTune - Rebranding of Azure Active Directory to Microsoft Entra terminology (part 4)

Applying Azure AD rebranding changes as announced on July 11, 2023 and outlined in https://aka.ms/AzureADNewName. Changes are part of the Microsoft-wide rebranding effort.
CorrelationId: 2520d58e-25ff-469e-af14-2834d2a1505e
PointOfContact: @CelesteDG
#docutune

#docutune-draft


Replace glide Package manager with go mods

Current situation
We currently use glide for package management and it has several problems associated with it such as with glide we have to make sure we are in a go path which is sometime very frustrating to do.
Ideal future situation
as recent versions fo go now supports go modules we should go with them as they allow the flexibility to place our code even outside a GOPATH. Also other projects like nebraska already use go mods
Implementation options
We can start by removing glide file and run go mod init   github.com/org/proj_name.
go mod will fetch all the required dependency by itself.

https://github.com/flatcar-linux/updateservicectl/pull/2 was merged. Thanks!


Fix state change of dead members in LB healthmonitor

Members in LB health monitor failed to update their
state when dead. Fixed member active/inactive
state transition.
Closes-rally-bug: DE619
Partial-Bug: 1259965

Unit tests pass :+1:

Looks like bits and pieces come from https://review.openstack.org/#/c/61673/2
Not sure if we should include a reference to that in the message http://wikicentral.cisco.com/display/PROJECT/Commit+Message+Guidelines

I'd agree with Abe. I'd like to see the commit message updated to reflect the upstream review. It saves a lot of time later if we look into why something was merged.


Not able to move untitled elements in dashboard?

It looks like the moveable feature of elements are tied to the title of each element so that if you remove them, the elements are no longer moveable? Not sure if I'm missing something here...

That's correct. One can only grab onto the header to move elements. That's by design, as elements may have interactive components themselves.

Is there a way to target the title div? Just so I can add custom styling

There's no id or class on the div, so that's not something you can use. I suppose it might be possible to target the first div child of the div with class eg-elc (the class on each element).


"Open in External App" opens verified links in Kiwi Browser instead of a default browser (with the "Better Open With" application))

Describe the bug
"Open in External App" opens links in Kiwi Browser instead of the default browser I use.
I use the Android App "Better Open With" which replaces the default browser so I can select apps I want to open with, which was a problem introduced with Android 11
https://play.google.com/store/apps/details?id=com.aboutmycode.betteropenwith&hl=en_US
To Reproduce
Steps to reproduce the behavior:
In Kiwi Browser, or a PWA on Kiwi Browser, open a YouTube or Reddit Link by holding and selecting "Open with External Browser", which just opens the link in Kiwi Browser.
I use the "Better Open With" app set as default browser. I have YouTube with "open supporting links off". If I enable it, and turn on the Kiwi Browser setting in Accessibility where it opens external apps by default, it will open the specific app but this does not allow me a choice with my chosen app.
Expected behavior
When I select "Open with External Application", it should open the "Better Open With" menu allowing me to choose between multiple browsers or third party apps to use to open the links. To be honest, I'm not sure what the original behaviour is meant to be since I have always used "Better Open With" but even if I disable that application, it still just opens Kiwi browser for the link.
Smartphone (please complete the following information):

Device: [Samsung S20 FE]
Android Version : [Android 13, One UI 5.1 ]
Kiwi Browser Version [116.0.5845.27]

Additional context
Using a PWA, is there way to bypass the internal browser to use an external browser? I find my work around is to open the internal browser, then select the browser default I use.

May be a duplicate of https://github.com/kiwibrowser/src.next/issues/920


Benchmark and compare fpsim v1.0 and v2.0

New functionality seems to have slowed down simulations. Quantify by how much, detect time sinks and identify where it can be optimised.

@pausz @mzimmermann-IDM @robynstuart
I tested running example_sim.py (kenya as location, 500 agents) in both v1 and v2 and below were my results:

v1: 4.57s
v2:
RandomChoice: 4.24s
SimpleChoice: 39.5s

Clearly the additional logic in the SimpleChoice contraceptive module is significantly increasing the runtime. I did some profiling in Python and found the following results. The first image is sorted by Own Time (time within the function itself) and the second is sorted by Cumulative Time (time in the function and functions it calls). The amount of time in 'docformat' is surprising (it looks like it's a function within scipy?) especially because doing the same profiling with RandomChoice only shows docformat taking up a cumulative .4% whereas here it is 38%...I'm assuming it has something to do with generating the distributions? Besides this potential bottleneck, the next best area of focus I see would be the set_dur_method function, which cumulatively takes up nearly 70% of the runtime.

Thanks @emilydriano - I've just looked into this too and set_dur_method is definitely the problem.
Couple of options that I can see:

if drawing from scipy distributions is really so slow, i guess we could cache the values, although i would like to understand why this isn't necessary for any other starsim model
there are 5 different events that trigger a call to set_dur_method: stopping contraception, starting contraception, switching methods, choosing a method 1m postpartum, choosing a method 6m postpartum. So the lines that sample from distributions get called up to n_timesteps * n_age_bins * 5 * n_methods times. For a 10 year sim with 5 age bins, that's 27000 calls...

Looking at the total number of calls (n_timesteps * n_age_bins * n_events * n_methods), I think I can remove the n_age_bins part. It would help a lot to remove the n_methods part too. I am wondering how much predictive power is gained by using separate distributions for each method. We currently have to loop over each method and set the durations based on the distribution for that method, but if we only had to do this once (i.e., if all methods used the same distribution), we'd speed things up a lot. The individual methods could still have different parameters. Would be keen for thoughts @mzimmermann-IDM @MObrien-IDM

@robynstuart or @emilydriano can you point me to where the set_dur_method code is? For some reason it's not coming up when I search... I want to make sure I know what it's doing before talking about this!

@emilydriano i've made some changes that have sped things up quite a bit: https://github.com/fpsim/fpsim/pull/359

@mzimmermann-IDM the set_dur_method code is here: https://github.com/fpsim/fpsim/blob/methodtime-jun22/fpsim/methods.py#L221

In fpsim.utils.sample:
# drawing samples like this
samples = sps.gamma.rvs(a=par1, scale=par2, size=size)
# instead of
samples = sps.fisk(c=par1, scale=par2).rvs(size=size)

should improve performance. Apparently the latter form has more overhead formatting docstrings.
Though based on profiling stats it does not seem to have huge impact. Ran example_sim.py with SimpleChoice and 10_000 agents.
samples = sps.gamma.rvs(a=par1, scale=par2, size=size)

samples = sps.fisk(c=par1, scale=par2).rvs(size=size)

@robynstuart @emilydriano
I've attempted some additional refactoring in set_dur_method in #377 that reduced runtime another 20% -- hope I didn't break stuff.
This draft PR doesn't have to be merged, just illustrates that the increased performance comes from removing the if/elif/else from inside the for loop in set_dur_method. I'm not fond of the many static methods that we now have in SimpleChoice :hands.
example_sim.py with SimpleChoice and 500 agents, Average runtime (over 10 runs): 3.623 s
example_sim.py with SimpleChoice and 10_000 agents, Average runtime (over 10 runs): 8.8 s


Move RAG image to ansible repository

Description
Move RAG image, which was stored under Tami's quay.io account (ttakamiy) to an repository under Ansible (quay.io/ansible/aap-rag-content).
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library
[ ] Bump-up library or tool used for development (does not change the final image)
[ ] CI configuration change
[X] Konflux configuration change

Related Tickets & Documents

Related Issue # AAP-34876
Closes #

Checklist before requesting a review

[ ] I have performed a self-review of my code.
[ ] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

It was created by an error.  Closing without merging.


(temporarily) replace compose-file deep links to v3 docs

fixes https://github.com/docker/docker.github.io/issues/9996
The documentation curently doesn't contain reference docs for the unified (version-less) compose-file syntax, so for now replacing all links to point to the v3 compose-file reference, which is still present.

ping @StefanScherer @aiordache PTAL

ping @StefanScherer @aiordache PTAL


ArrayIndexOutOfBoundsException from encode(long... numbers)

Problem:
If the list of longs passed to the method public String encode(long... numbers) contains a negative value, it may result in a java.lang.ArrayIndexOutOfBoundsException being thrown.  Here is the offending code snippet (see added inline comments.
  private String _encode(long... numbers){
    int numberHashInt = 0;
    for(int i = 0; i < numbers.length; i++){
      numberHashInt += (numbers[i] % (i+100));  // If numbers contains a negative long value then numberHashInt could be negative
    }
    String alphabet = this.alphabet;
    char ret = alphabet.toCharArray()[numberHashInt % alphabet.length()]; // if numberHashInt is negative, an ArrayIndexOutOfBoundsException will be thrown

Possible Solutions:

In the method public String encode(long... numbers) replace the following line

if (number ><PHONE_NUMBER>740992L) {
with
if (number ><PHONE_NUMBER>740992L || number < 0) {

In the method private String _encode(long... numbers) use the absolute value of the long values in numbers when computing the hash:

int numberHashInt = 0;
for(int i = 0; i < numbers.length; i++){
  numberHashInt += (Math.abs(numbers[i]) % (i+100));
}

Fixed.
When encoding negative numbers, the result should be ""(empty string), same behaviour as the javascript implementation.
https://github.com/ivanakimov/hashids.js/blob/master/lib/hashids.js#L111-L115

Just to make it clear, the fix has not been released. It has been fixed in HEAD.


Fix fix unit test fail when try to save minidump file

Proposed changes
Problem:
In fe unit test, [StmtExecutorTest] testStmtWithUserInfo,testKillOtherFail,testKillOther failed
Reason:
when execute failed, minidump would work automaticly and try to dump messages of optimizer.But
it needs some information in context, and these informations initialize only if we open enable_minidump switch
Solved:
Add switch of enable minidump also when execute failed
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall


[Bug]: Ollama models missing from selection list if name is capitalized

What happened?

# Definition of custom endpoints
endpoints:
  custom:
      - name: "Ollama" # "Ollama" -> no models. "ollama" -> models
        apiKey: "ollama"
        baseURL: "http://host.docker.internal:11434/v1/chat/completions"
        models:
          default: [
            "llama3:latest"
          ]
          fetch: true
        titleConvo: true
        titleModel: "current_model"
        summarize: false
        summaryModel: "current_model"
        forcePrompt: false
        modelDisplayLabel: "ollama"
        capabilities: ["retrieval", "actions", "tools", "image_vision", "code_interpreter"]

When I updated from v0.75.2 -> v0.75, my Ollama model list stopped loading. I saw that my models were loading from the /api/models endpoint correctly, but they were under the key "ollama".
The docs say that this should be case insensitive:

    # with `ollama` (case-insensitive), as it does in this example.

The example uses the titlecase "Ollama" as the example custom_endpoint name:
https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints/ollama
  custom:
    - name: "Ollama"
      apiKey: "ollama"
      # use 'host.docker.internal' instead of localhost if running LibreChat in a docker container
      baseURL: "http://localhost:11434/v1/chat/completions"
      models:
        default: [
          "llama2",
          "mistral",
          "codellama",
          "dolphin-mixtral",
          "mistral-openorca"
          ]
        # fetching list of models is supported but the `name` field must start
        # with `ollama` (case-insensitive), as it does in this example.
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      summarize: false
      summaryModel: "current_model"
      forcePrompt: false
      modelDisplayLabel: "Ollama"

Steps to Reproduce

Setup Ollama using a custom_endpoint and download a model
Set the name to "Ollama"
-> No models returned
Set the name to "ollama"
-> All models returned

What browsers are you seeing the problem on?
No response
Relevant log output
No response
Screenshots
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Closed by https://github.com/danny-avila/LibreChat/pull/4657

Problem solved, thank you


add reporting infrastructure on server RPCs + time-series metrics

Motivation
The goal of this PR is to allow applications to generate comprehensive time-series metrics for server-side RPCs. gRPC servers written in Go can already easily add such functionality through Go modules like go-grpc-prometheus. This PR provides a comparable solution for Tonic servers.
Why is this PR necessary? The existing mechanisms for hooking into the request stack did not seem to support the time-series metrics use case well enough because:

Interceptors do receive the status code from calling the underlying service (nor any notification for that matter that an RPC had ended). Thus, an interceptor cannot be used to generate metrics relating to completion of RPCs.

The generated server stubs implement Service<http::Request, Response = <http::Response>>. Any "metrics service" wrapping those servers would have no access to some of the data it would need without additional hacky parsing. For example, the method name would need to be parsed again from the request path (while the service name could come from NamedService). And the status code would have to be parsed from the grpc-status header stored in the http::Response.  Both of those solutions seem very hacky.

Solution
This PR defines a tonic::reporter::Reporter struct which wraps a function/closure that is called with the service name, method name, and the RPC type of each inbound RPC call. The function returns an instance of the ReportingCallback trait which receives all further events related to that specific RPC, specifically, stream message received, stream message sent, and RPC completion.
The generated server stubs gain support for:

with_reporter and with_interceptor_reporter methods which add a reporter to a server.
a NamedMethod trait impl to supply the parsed service name and method name to tonic::server::Grpc without any need for double parsing of the request path.

The new tonic-metrics crate contains a reporting callback that use the metrics crate to generate the following metrics:

Counters:

grpc_server_started_total with labels grpc_type, grpc_service, and grpc_method
grpc_server_handled_total with labels grpc_type, grpc_service, grpc_method, and grpc_code
grpc_server_msg_received_total with labels grpc_type, grpc_service, and grpc_method
grpc_server_msg_sent_total with labels grpc_type, grpc_service, and grpc_method

Histogram:

grpc_server_handling_seconds with labels grpc_type, grpc_service, and grpc_method

Open Questions:

This change is somewhat behind the transport feature. Maybe it should be behind a separate reporting feature instead? And probably needs me to recheck it is all behind the transport feature.

Future

Client Support: The changes made to tonic::server::Grpc could also be made in a similar manner to tonic::client::Grpc. This would serve the same purpose of generating time-series metrics and tracing spans.

Alternatives

This use case could also probably be solved by reorganizing Tonic and generated stubs to better support middleware that want to use Tonic types in their signature like tonic:Status for errors instead of http::Response. That sort of solution seemed more complicated to me though than providing the reporter abstraction in this PR. Moreover, such a reorganization  would probably require a significant amount of design work by someone more familiar with Tonic than me.

For example, the method name would need to be parsed again from the request path (while the service name could come from NamedService). And the status code would have to be parsed from the grpc-status header stored in the http::Response. Both of those solutions seem very hacky.

To me a generic solution based on tower doesn't sound hacky at all. In fact that is the exact solution we will be exploring in tower-http https://github.com/tower-rs/tower-http/issues/57. I have already built a similar middleware at work and it's not bad.

For example, the method name would need to be parsed again from the request path (while the service name could come from NamedService). And the status code would have to be parsed from the grpc-status header stored in the http::Response. Both of those solutions seem very hacky.

To me a generic solution based on tower doesn't sound hacky at all. In fact that is the exact solution we will be exploring in tower-http tower-rs/tower-http#57. I have already built a similar middleware at work and it's not bad.

Is that middleware at work just for HTTP or does it include gRPC metrics? Generic middlewares with Tonic currently seem limited to just mapping http::Request to hyper::Response from which a few issues arise in my view:

The gRPC status code is stored in the grpc-status header of the http::Response and not in the HTTP status. It seems duplicative to me for Tonic to encode tonic::Status into that header, only for a generic middleware to have to decode it again (or at least the integer code).

This PR is able to instrument the number of message sent/received over streaming RPCs. It seemed much easier to just hook into tonic::Streaming to count those messages. Any ideas on how a generic Hyper middleware could do that? (I'm not that familiar with how http::Request is used by streaming RPCs.)

The service name and method name are already parsed by the stubs. The generic Hyper middleware would need to parse the path and extract those separately, but would not be able from the path alone to know the type of RPC. (This is the least of the problems but is at least straightforward to do.)

I would also prefer a solution that leverages more of the tower::Service abstraction. However, as just mentioned, Tonic doesn't seem to have any "intermediate" way to insert middleware "closer" in the stack to the RPC call and use types that are more gRPC-specific than the generic HTTP types.
Using metrics as an example, I would like to be able to write a middleware that could intercept the tonic::Status for an RPC before it is encoded into a http::Response and also map over the streams used for streaming APIs before they are encoded. That seemed like it would require more than just me to think it over. Hence:

Moreover, such a reorganization would probably require a significant amount of design work by someone more familiar with Tonic than me.

Are there any planned refactors that would help in the regard?

Is that middleware at work just for HTTP or does it include gRPC metrics?

gRPC is still HTTP. So yes it does include gRPC. Not quite at the level that something thats able to hook into the guts of tonic could do but still enough for measuring latency, traffic, errors, and throughput. That is all I've ever needed in terms of "framework level" metrics. Most other things goes in application code in my experience.
For measuring streams you should be able to write a Body wrapper that adds metrics to poll_data or poll_trailers. You'll only receive opaque Bytes chunks but thats probably fine but that should be fine for counting the number of messages. I do something similar for the Trace middleware in tower-http thats currently being developed.
I'm assuming that since each message in a gRPC stream is provided to Tonic as one struct it'll comes through Body::poll_data as one encoded Bytes chunk but I could very well be wrong. If its not the case it might require new APIs in tonic to support metrics for streams.

Using metrics as an example, I would like to be able to write a middleware that could intercept the tonic::Status for an RPC before it is encoded into a http::Response

Yeah that wouldn't be possible as the tower Service would sit outside of the tonic Channel. However I'm not sure what the use case for that is. What metrics does that enable?

Are there any planned refactors that would help move toward a Service-based solution?

I don't think there are plans to change how tonic hooks into Tower.

gRPC is still HTTP. So yes it does include gRPC. Not quite at the level that something thats able to hook into the guts of tonic could do but still enough for measuring latency, traffic, errors, and throughput. That is all I've ever needed in terms of "framework level" metrics. Most other things goes in application code in my experience.

gRPC at some level is still HTTP but they differ in error semantics. The following test passes:
 #[test]
    fn to_http_test() {
        let status = Status::internal("foo");
        let http = status.to_http();
        assert_eq!(http.status(), hyper::StatusCode::OK);
    }

I.e., gRPC status codes are encoded as 200 OK at the HTTP level. Thus inspecting HTTP status alone is insufficient to monitor error conditions with gRPC services.

True but you can look at the "grpc-status" response header instead.

Closing as this approach seems to not be favored (and I solved locally in a different way in any event).

I really liked the generic metrics support in tonic. Can this be reconsidered to avoid each one implementing their own?

@rohitjoshi please file a separate issue if you have any feature requests :)


Login module

Hi @Kelvinskell,
I would be happy to contribute for the login module. Using the Flask-Login, the web app can handle users and store sessions. If you are planning to use a database - could you please let me know just in case I plan on creating a branch and implementing the login module so that it doesn't affect your tech stack.
Thanks,
Anshul M

Hi @Ansh743,
I'd appreciate your offer of contribution.
You're most welcome to contribute for the login service.
I plan to use the MYSQL database for storing user credentials - user ID (The primary key - which should be auto incremented.), username, password and email address.
You might also add other user parameters as you deem fit.
I'd prefer we use GitFlow Workfkow, but if you're not comfortable with that, you could suggest another Workfkow (Forking Workfkow maybe).
I anticipate your contributions.


remote explorer terminal can't input git password

today I update vscode to last version, and use remote explorer connet remote linux server, when I write "git pull" in my project, I did not see password input windows like usual, so it make me can't git pull my project in terminal. thank
you to debug

I think latest update just broke a tonne of things when tunnelling into remote linux device. I can't call a bunch of commands I normally could before today.

Try reverting to 1.62.0

Try reverting to 1.62.0

1.62.0 can solve my problem that passwprd not place to input

Likely related to https://github.com/microsoft/vscode/issues/136791


Delay KV bulk delete

Overview
Fixes #783
Previously, we would run a full KV namespace sync BEFORE uploading the worker script to the edge. This caused some unacceptable downtime for updated assets, as the Worker would continue  handling requests to paths with outdated KV keys. This PR does a few things in service of re-ordering these operations, but the bottom line is that now on Wrangler publish, when the project is a Workers Site, we:

validate the wrangler.toml
run wrangler build
query the site's KV namespace for existing keys and diff them against the local site bucket to build the list of files to upload, the list of keys to delete, and the asset manifest to be uploaded with the Worker script.
execute the bulk file upload
upload the worker with its new asset manifest
execute the bulk file delete

This has the added benefit of NOT deleting old values in the case where the Worker upload fails, since the first KV operation is additive.
Things this is not
This PR introduces a little more verbosity into preview/upload and publish, however there is considerable overlap between those operations already, but I'd like to address those in a separate PR in order to expedite this fix.
I'm also interested in optimizing the preview command's behavior (refrain from deleting until a ctrl-C or something like that) but not here.
💅
There's a few house-keeping changes here, though more remain which I'll include as inline comments:

we now pass a set of KeyValuePairs to kv::bucket::upload, rather than a bucket.
some house-keeping in the publish command, mostly renaming and some refactors
remove unused/unsupported bucket attribute from KvNamespace struct

update: i'm not  necessarily gunning to get this into 1.8.1; i'm happy to wait on 1.9.0. one of the refactors here removes the bucket attribute from kv-namespaces, which from what i could tell was never supported, but i want to do a little extra testing to confirm that there isn't any support here so i don't break unsuspecting crafty users.

update: i'm not necessarily gunning to get this into 1.8.1; i'm happy to wait on 1.9.0. one of the refactors here removes the bucket attribute from kv-namespaces, which from what i could tell was never supported, but i want to do a little extra testing to confirm that there isn't any support here so i don't break unsuspecting crafty users.

Is there an ETA for 1.9.0 or for this to be released (even as an alpha/RC)?

Is there an ETA for 1.9.0 or for this to be released (even as an alpha/RC)?

We'll be putting this out on its own as a point release (1.8.2)


5.2.0 dunfell updates

These commits fix the existing boards (colibri-imx6ull and verdin-imx8mm) to build and work with the current Toradex 5.2.0 BSP releases. I have tested the tdx-reference-minimal-image on both boards.  On the verdin-imx8mm I tested both U-Boot autopatching and EFI Grub.

@kacf can you take a look at this? For some reason, I cannot assign reviewers to PRs in this repository


Update sbt-tpolecat to 0.1.19

Updates io.github.davidgregory084:sbt-tpolecat from 0.1.16 to 0.1.19.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.github.davidgregory084", artifactId = "sbt-tpolecat" } ]

labels: sbt-plugin-update, semver-patch

Superseded by #270.


Weird behaviour of parameters_alfven_wave.toml with constant initial conditions in REPL

In GitLab by @erik-f on Jun 10, 2020, 22:59
When I execute
Trixi.run("examples/parameters_alfven_wave.toml", n_steps_max=1, initial_conditions="initial_conditions_constant")

on b006e89f in the REPL I get very high L2 and Linf errors for B2 and B3:
Variable:       rho              rho_v1           rho_v2           rho_v3           rho_e            B1               B2               B3               psi
L2 error:       2.09587921e-16   2.00505223e-16   4.67253468e-16   1.07374002e-16   4.12845635e-15   4.50940304e-16   1.08194682e+00   5.00000000e-01   1.76565492e-16
Linf error:     3.33066907e-16   7.49400542e-16   1.77635684e-15   1.66533454e-16   1.42108547e-14   8.88178420e-16   1.08194682e+00   5.00000000e-01   7.41458237e-16

I get similar results when running more than one step.
However I added a test case to test_examples.jl in the usual format (locally) that computes completely different (around 1e-16) results than a just started REPL.

I'll close this since we haven't been able to reproduce the weird results on other machines (including CI on Windows, macOS, and Linux).


Adding customerAdminGroupId to OSA cluster swagger spec

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[x] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-js

Can one of the admins verify this patch?

Automation for azure-sdk-for-python

Automation for azure-sdk-for-ruby

This should have WaitForARMFeedback label but I can't add it

Automation for azure-sdk-for-go

Automation for azure-sdk-for-java

Automation for azure-sdk-for-net
A PR has been created for you:
https://github.com/azure/azure-sdk-for-net/pull/5611
.NET SDK Commits:
https://github.com/adxsdknet/azure-sdk-for-net/pull/5611/commits/7709dd7afd31491038a8b01ab500f78f24bf8d36

@praries880 or @majastrz  can this get merged?

@praries880 or @majastrz can this get merged?

@praries880 can merge it in. The ARM side doesn't have permissions to merge this.


Symbolic link under windows

Is there any reason to not implement Simbolic links under windows? I tryed the following quick and dirt mod and it seems to work.
#ifdef _WIN32
#include <windows.h>
#include <winbase.h>
#endif
static int make_link(lua_State *L)
{
        const char *oldpath = luaL_checkstring(L, 1);
        const char *newpath = luaL_checkstring(L, 2);
#ifndef _WIN32
        return pushresult(L,
                (lua_toboolean(L,3) ? symlink : link)(oldpath, newpath), NULL);
#else
  int err = CreateSymbolicLink(newpath, oldpath, 0);
  if (!err) return pusherror(L, "make_link is not supported on Windows");
  else pushresult(L, lua_toboolean(L,3), NULL);
#endif
}

Note: I compiled with mingw and _WIN32_WINNT=0x0600 .

No reason other than nobody did it yet. :)
A proper implementation needs the following:

version checks to make sure that the APIs are available (that is, that all Windows versions that luafilesystem currently supports will continue to build without breaking)
implementations that make sense with the lfs APIs (for the lfs.link function for example, if Windows supports symbolic links only and not hard links, the function should only create a link when the third argument is true)
check that the existing tests run and exercise the new code (and add some if necessary)
following the coding style of the rest of the code (variable names, indentation, etc.)

As a side note, there is a library called LibNTLink that implements link(), symlink(), unlink(), lstat(), and readlink() on Windows. Maybe that library would be of use.
I am currently in the process of editing LFS to make it use LibNTLink for this.

@BrisingrAerowing that's interesting, but according to the link it requires GCC and win32api MinGW packages. LFS also runs with MSVC.

I'll take a look and see if I can get it to compile with MSVC. Looking at the code it doesn't use any GCC-Specific things, so it shouldn't take too much to get it working.
It does have a few compilation bugs though (missing method parameters mostly), but those are easily fixed.